Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Lighting Documentation

Welcome to the Lighting documentation.

Lighting is a high-performance, lightweight web framework written in Zig, featuring idiomatic Go-aligned ergonomics, structured concurrency via std.Io.Group, deterministic memory overhead, and zero external dependencies.


Documentation Roadmap

The documentation is organized into five specialized sections:

  • Getting Started: Conceptual foundations, installation instructions via build.zig.zon, and a minimal quick-start tutorial.
  • User Guide: Comprehensive, task-oriented guides for building web applications — covering server lifecycle, routing syntax, request inspection, response generation, state injection, middleware pipelines, static files, and WebSockets.
  • Internals: In-depth technical analysis for systems engineers and contributors — exploring structured concurrency dispatch, zero-copy protocol parsing, decision-tree routing algorithms, arena allocators, and TigerStyle defensive invariants.
  • Cookbook: Real-world, end-to-end applications including REST APIs, SQLite with embedded SPAs, WebSocket chat servers, and command-line file servers.
  • Project & Evolution: Architecture design specifications, development workflow, and protocol milestones.

Overview & Philosophy

Lighting is a modern web framework for Zig. It bridges two worlds: the proven, ergonomic developer experience of Go’s net/http and the low-level mechanical sympathy, zero-allocation control, and deterministic memory safety of Zig.


Core Tenets

1. Idiomatic Web Ergonomics

Zig is a low-level systems language, but web APIs do not have to be convoluted. Lighting adapts the battle-tested conventions popularized by Go’s net/http:

  • *ResponseWriter: Intuitive response streaming with built-in 4KB stack buffer optimization.
  • *Request: Clean URL inspection, path parameter lookup, header parsing, and body streaming.
  • Fat-Pointer Handler: Universal handler interface with zero-overhead state injection (Handler.bind).
  • ServeMux: Pattern-based request router with Go 1.22+ wildcard and method-prefix support.

2. Structured Concurrency via std.Io.Group

Rather than relying on thread pools with task queues or heavy OS thread-per-connection architectures, Lighting embraces native structured concurrency:

  • Each accepted connection is dispatched using std.Io.Group.concurrent.
  • Connections run as lightweight fibers / coroutines without OS thread starvation.
  • Connection lifetimes are strictly bounded by their parent Server.serve scope, guaranteeing clean cancellation and resource reclamation on shutdown.

3. Deterministic Memory & Zero Unbounded Allocation

  • Zero Heap Fast Path: Small HTTP responses (<= 4KB) are buffered directly on the stack and flushed to the socket without invoking GPA.
  • Per-Request Arena: All allocations generated during request parsing, path parameter extraction, and route dispatch live within a single per-request arena allocator. At the end of the request cycle, the arena is reset in O(1) time.
  • Bounded Connection Buffers: Incoming socket buffers are fixed-size (8KB–16KB) to prevent unbounded memory amplification attacks.

4. TigerStyle Robustness

Lighting adheres to the TigerStyle engineering ethos:

  • Strict assertion of preconditions and invariants at every layer.
  • Explicit limits on header count, URI length, chunk framing, and body draining.
  • Strict RFC 9110 / RFC 9112 protocol compliance.
  • Zero external third-party dependencies.

Comparison with Go net/http

Lighting achieves feature and ergonomic parity with Go’s standard library across both server and client domains:

  • Server-Side: Pattern routing (ServeMux with Go 1.22+ wildcard semantics), *Request and *ResponseWriter primitives, Flusher (SSE/LLM streaming), Hijacker (WebSocket), typed Cookie, MultipartForm, FileServer, and ReverseProxy.
  • Client-Side: High-level convenience methods (get, post, postForm, head, do), multi-hop redirect following (max_redirects), session persistence via CookieJar, Keep-Alive connection pooling (ConnPool), and streaming response decoding (Content-Length, Chunked, EOF).
  • Architectural Boundaries: Lighting focuses strictly on high-performance HTTP/1.1; TLS termination is delegated to reverse proxies (such as Nginx or Caddy), and @panic inside a handler is fatal to the process rather than caught per-fiber.

For a line-by-line capability matrix and API comparison, see the Comparison with Go.

Comparison with Go net/http

Lighting is designed with deliberate architectural sympathy toward Go’s standard net/http package. To maintain sharp architectural focus and zero-overhead performance, Lighting specializes strictly as a high-performance HTTP server engine. The complementary HTTP client implementation resides in the stdgo project (net/http).

This document provides a comprehensive feature parity and API mapping comparison between Go net/http and Lighting.


1. Feature Parity Overview

1.1 Server-Side Capabilities

CapabilityGo net/httpLightingArchitectural Notes
Go 1.22+ Routinghttp.ServeMux✅ Full ParityPattern routing with method constraints, {id}, {filepath...}, /{$} exact match, and automatic 405 Method Not Allowed.
Structured Concurrencygo c.serve()✅ Full ParityNative std.Io.Group.concurrent user-space Fiber dispatching per connection with zero thread starvation.
Dual-Mode ResponseWriterhttp.ResponseWriter✅ Full ParityAutomatic stack-buffered Content-Length (<4KB) and streaming chunked transfer coding (>=4KB).
Flusherw.(http.Flusher).Flush()✅ w.flush()Immediate socket flush without closing connection (essential for Server-Sent Events and LLM token streaming).
Hijackerw.(http.Hijacker).Hijack()✅ w.hijack()Raw TCP socket handoff for protocol upgrades.
WebSocketgolang.org/x/net/websocket✅ lighting.extra.wsFull RFC 6455 handshake, frame parsing, XOR unmasking, Ping/Pong, and echo.
Static File Serverhttp.FileServer(http.Dir)✅ lighting.extra.fs.FileServerRFC 7233 Range requests (audio/video seek), ETag/If-Modified-Since (304), MIME deduction, and directory listing.
Query Parametersreq.URL.Query()✅ req.url.query()Multi-value parsed query map (QueryParams), single-key zero-copy lookup.
Cookiesreq.Cookie(), http.SetCookie()✅ lighting.CookieRFC 6265 typed cookie parser, iterator, and Set-Cookie builder (w.setCookie).
Multipart Formreq.ParseMultipartForm()✅ lighting.MultipartFormStreaming RFC 7578 multipart/form-data parser with configurable memory limit and disk spilling.
Reverse Proxyhttputil.ReverseProxy✅ lighting.extra.proxy.ReverseProxyForwarding proxy with RFC 9110 hop-by-hop header stripping and X-Forwarded-* header injection.
Route Timeoutshttp.TimeoutHandler✅ lighting.timeoutHandler()Route-level timeout middleware canceling handler execution and returning 504 Gateway Timeout.

1.2 Scope and Client Boundary

Lighting intentionally confines its core scope to server-side HTTP serving. For client-side HTTP operations (such as http.Get, http.Post, Keep-Alive pooling, redirect following, and CookieJar), use the companion stdgo (https://github.com/jiacai2050/stdgo) repository (net/http package), which mirrors Go’s standard client API identically in Zig.


2. Server API Mapping Reference

The table below maps common Go server idioms directly to their Lighting counterparts:

TaskGo IdiomLighting Equivalent
Create Routermux := http.NewServeMux()var mux = http.ServeMux.init(allocator);
Register Routemux.HandleFunc("GET /users/{id}", fn)try mux.handleFunc("GET /users/{id}", handler);
Extract Path Paramid := req.PathValue("id")const id = req.pathValue("id");
Write Statusw.WriteHeader(http.StatusOK)w.setStatus(.ok);
Write Headerw.Header().Set("Content-Type", "text/plain")try w.header().set("Content-Type", "text/plain");
Write Responsew.Write([]byte("hello"))try w.writeAll("hello");
Stream Flushw.(http.Flusher).Flush()try w.flush();
Protocol Hijackconn, _, _ := w.(http.Hijacker).Hijack()const stream = try w.hijack();
Listen & Servehttp.ListenAndServe(":8080", mux)var server = http.Server.init(allocator, io, .{ .handler = mux.handler() });
try server.listenAndServe("0.0.0.0", 8080);
Non-blocking Closesrv.Close()server.close();
Graceful Shutdownsrv.Shutdown(ctx)server.shutdown();

3. Architectural Differences & Current Boundaries

While Lighting strives for developer ergonomics on par with Go, systems differences between Go’s runtime and Zig lead to intentional architectural distinctions:

  1. Memory Allocation:
    • Go: Relies on a garbage collector (GC) for request/response lifetimes and escaped buffers.
    • Lighting: Uses connection-scoped and per-request ArenaAllocator lifecycles with strict bounds. Responses <= 4KB use stack memory without GPA invocations, achieving deterministic latency.
  2. TLS / HTTPS:
    • Go: Includes an in-tree TLS 1.3 implementation (crypto/tls) in the standard library.
    • Lighting: Focuses strictly on high-performance plaintext HTTP/1.1; SSL termination is typically delegated to fronting reverse proxies (e.g., Nginx, Caddy, or AWS ALB).
  3. HTTP/2:
    • Go: Bundles HTTP/2 framing with automatic ALPN negotiation.
    • Lighting: Specializes in ultra-low-latency HTTP/1.1 microservices. HTTP/2 is planned for future phases.
  4. Panic Isolation:
    • Go: Recovers from handler panics per-goroutine via defer recover(), returning an HTTP 500 without crashing the server process.
    • Lighting: All Zig error returns (anyerror) are caught and gracefully converted into HTTP 500 responses. However, language-level @panic calls will terminate the process because Zig does not provide a universal cross-platform stack unwind/catch mechanism.

4. Request and Response Models: Go vs. Lighting

The table below contrasts how Go net/http and Lighting model HTTP requests and responses:

DimensionServer RequestServer ResponseGo Client Request (stdgo)Go Client Response (stdgo)
Go net/http*http.Requesthttp.ResponseWriter (interface)*http.Request (monolithic struct)*http.Response (struct)
Lighting / stdgolighting.Requestlighting.ResponseWriter (struct)stdgo.net.http.Requeststdgo.net.http.Response
Data FlowInbound (read/parse)Outbound (serialize/write)Outbound (construct/send)Inbound (read/parse)
Target URLRelative path and query (req.url.path, req.url.query)N/AAbsolute URI with scheme and hostN/A
Memory & LifecycleZero-copy slices borrowed from socket buffer; per-request Arena lifetimeDual-mode stack buffer (<4KB) or chunked stream; connection lifetimeCaller-managed payload and outbound headersHeap headers; streaming socket body reader (resp.body.read)
Type Design & SafetyZero-copy parsed view. Handlers read paths, query params, and body directly from socket/arena buffers with no allocations.Active writer/sink supporting flush, cookies, and connection hijacking.Builder model. Clear separation prevents runtime field collisions (such as Go’s RequestURI conflict on clients).Parsed status and streaming reader. Connection returned to Keep-Alive pool upon EOF.

Installation & Setup

Lighting requires Zig 0.17 (or master build compatible with the 0.17 API release).


Adding to build.zig.zon

To declare Lighting as a dependency in your Zig project, add it to your build.zig.zon:

.{
    .name = .my_web_app,
    .fingerprint = 0x8a9e1029c374b5fa, // Run `zig build` to generate/update
    .version = "0.1.0",
    .dependencies = .{
        .lighting = .{
            .url = "git+https://github.com/jiacai2050/lighting.git#main",
            .hash = "...", // Generated by `zig build` or `zig fetch`
        },
    },
    .paths = .{
        "build.zig",
        "build.zig.zon",
        "src",
    },
}

You can automatically resolve and pin the hash by running:

zig fetch --save git+https://github.com/jiacai2050/lighting.git

Integrating with build.zig

In your root build.zig, expose the lighting module to your executable target:

const std = @import("std");

pub fn build(b: *std.Build) void {
    const target = b.standardTargetOptions(.{});
    const optimize = b.standardOptimizeOption(.{});

    // Obtain the lighting dependency
    const lighting_dep = b.dependency("lighting", .{
        .target = target,
        .optimize = optimize,
    });
    const lighting_mod = lighting_dep.module("lighting");

    // Add executable
    const exe = b.addExecutable(.{
        .name = "my_web_app",
        .root_module = b.createModule(.{
            .root_source_file = b.path("src/main.zig"),
            .target = target,
            .optimize = optimize,
            .imports = &.{
                .{ .name = "lighting", .module = lighting_mod },
            },
        }),
    });

    b.installArtifact(exe);
}

Quick Start

In this guide, you will build a complete, runnable web application using Lighting in under 5 minutes.


1. Minimal Server Code

Create src/main.zig with the following content:

const std = @import("std");
const http = @import("lighting");

pub fn main(init: std.process.Init) !void {
    const allocator = init.gpa;
    const io = init.io;

    // 1. Initialize the router
    var mux = http.ServeMux.init(allocator);
    defer mux.deinit();

    // 2. Register a basic ping endpoint
    try mux.handleFunc("GET /ping", struct {
        fn handle(w: *http.ResponseWriter, _: *http.Request) anyerror!void {
            try w.header().set("Content-Type", "text/plain; charset=utf-8");
            try w.writeAll("pong\n");
        }
    }.handle);

    // 3. Register a route with path parameters
    try mux.handleFunc("GET /hello/{name}", struct {
        fn handle(w: *http.ResponseWriter, req: *http.Request) anyerror!void {
            const name = req.pathValue("name") orelse "stranger";
            try w.header().set("Content-Type", "text/plain; charset=utf-8");

            var buf: [128]u8 = undefined;
            const msg = try std.fmt.bufPrint(&buf, "Hello, {s}!\n", .{name});
            try w.writeAll(msg);
        }
    }.handle);

    // 4. Configure and run the server
    var server = http.Server.init(allocator, io, .{
        .handler = mux.handler(),
        .read_timeout_ns = 5 * std.time.ns_per_s,
        .write_timeout_ns = 5 * std.time.ns_per_s,
    });

    std.debug.print("🚀 Lighting web server running at http://127.0.0.1:8080\n", .{});
    try server.listenAndServe("127.0.0.1", 8080);
}

2. Running and Testing

Build and launch your application:

zig build run

In another terminal, test the endpoints using curl:

# Test /ping
$ curl -i http://127.0.0.1:8080/ping
HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Content-Length: 5
Connection: keep-alive

pong

# Test path parameter
$ curl -i http://127.0.0.1:8080/hello/alice
HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Content-Length: 15
Connection: keep-alive

Hello, alice!

What Just Happened?

  1. ServeMux.init(allocator): Creates a decision-tree router with zero-allocation route lookups.
  2. handleFunc("GET /route", handlerFn): Binds a stateless function directly to an HTTP method and path pattern.
  3. req.pathValue("name"): Reads extracted path segments in O(1) time without any string copying or heap allocations.
  4. w.writeAll(...): Streams response data into a 4KB stack buffer, flushing directly to the client socket when full or on completion.
  5. server.listenAndServe(host, port): Dispatches accepted connections into concurrent std.Io.Group tasks managed by the runtime.

Server & Lifecycle

Server is the core network orchestrator in Lighting. It manages socket binding, connection accept loops, structured concurrency tasks, socket deadlines, and graceful shutdown.


1. Initializing the Server

Server.init takes an allocator, the Zig std.Io runtime handle, and ServerOptions:

const http = @import("lighting");

var server = http.Server.init(allocator, io, .{
    .handler = root_handler,
    .header_bytes_max = 16 * 1024,                  // Max allowed request header size (default: 8KB)
    .read_timeout_ns = 10 * std.time.ns_per_s,      // Whole-request body read timeout
    .read_header_timeout_ns = 2 * std.time.ns_per_s,// Header parsing timeout
    .write_timeout_ns = 10 * std.time.ns_per_s,     // Response flush timeout
    .idle_timeout_ns = 60 * std.time.ns_per_s,      // Keep-alive idle connection timeout
});

ServerOptions Reference

OptionTypeDefaultDescription
handlerHandler(Required)The root HTTP handler or router dispatching incoming requests.
header_bytes_maxusize8 * 1024 (8KB)Maximum memory allocated for reading and parsing request headers before returning 431 Request Header Fields Too Large.
read_header_timeout_ns?u64nullMaximum duration allowed to read the HTTP request line and headers.
read_timeout_ns?u64nullMaximum duration allowed to read the full request (including body).
write_timeout_ns?u64nullMaximum duration allowed to write the response back to the client.
idle_timeout_ns?u64nullMaximum duration an idle keep-alive connection waits for a new request before closing.

2. Starting the Server

Lighting offers two ways to bind and start listening:

Using Host and Port

try server.listenAndServe("0.0.0.0", 8080);

Using a Pre-Bound std.Io.net.Server

If you need custom socket options or systemd socket activation:

try server.serve(my_listener);

3. Concurrency Model: Structured Concurrency

Unlike traditional multi-threaded servers that pre-allocate a fixed worker thread pool with unbounded queues, Lighting uses native std.Io.Group.concurrent:

  1. The main fiber runs the non-blocking accept loop.
  2. For each incoming connection, group.concurrent(io, serveConnWorker, .{ self, stream }) is called.
  3. Each connection executes independently within a lightweight fiber coroutine.
  4. When socket I/O blocks (e.g. waiting for client packets), the fiber suspends in the OS kernel epoll/kqueue, yielding CPU immediately with zero spinloop overhead.

4. Graceful Shutdown and Lifecycle

Server manages its resources entirely within the execution scope of serve(). No manual deinit() is required.

Lighting provides two methods to terminate the server (matching Go’s net/http):

  • server.close(): Non-blocking. Signals the server to stop accepting new connections and returns immediately (ideal for signal handlers or inside HTTP route handlers).
  • server.shutdown(): Blocking. Triggers termination and waits until all in-flight requests are completed and serve() has completely returned.
var server = http.Server.init(allocator, io, .{
    .handler = mux.handler(),
});

// Non-blocking exit (e.g. from an HTTP admin endpoint):
// server.close();

// Blocking graceful shutdown (e.g. from main thread awaiting drain):
// server.shutdown();

// serve() runs until close() or shutdown() is invoked:
try server.serve(listener);

Shutdown Workflow:

  1. When close() or shutdown() is called, an internal atomic signal wakes up the accept loop and marks the server stopped.
  2. In-flight requests are allowed to complete while incoming requests are stopped.
  3. All background connection workers in self.group are cancelled and awaited.
  4. serve() returns cleanly with all listeners closed and workers joined, unblocking any waiting shutdown() call.

Routing & URL Patterns

Lighting features a high-performance, radix-tree-based request router called ServeMux, heavily inspired by Go 1.22’s routing enhancements.


1. Initializing ServeMux

var mux = http.ServeMux.init(allocator);
defer mux.deinit();

To pass the router to your server or middleware chain, use mux.handler():

var server = http.Server.init(allocator, io, .{
    .handler = mux.handler(),
});

2. Route Registration Syntax

Route patterns take the form:

[METHOD ][HOST]/[PATH]

Examples:

  • "GET /": Matches GET requests to / and any sub-path (prefix match).
  • "GET /{$}": Matches GET requests strictly to / (exact match).
  • "POST /api/v1/users": Matches POST requests to /api/v1/users.
  • "/healthz": Matches any HTTP method (GET, POST, PUT, etc.) to /healthz.
  • "api.example.com/items": Matches requests with Host: api.example.com specifically.

3. Path Parameters

Path segments enclosed in curly braces {...} capture dynamic URL segments:

Single-Segment Wildcards ({name})

Matches exactly one non-empty path segment between slashes /:

try mux.handleFunc("GET /users/{id}", struct {
    fn handle(w: *http.ResponseWriter, req: *http.Request) anyerror!void {
        const id = req.pathValue("id") orelse return error.MissingParam;
        try w.writer().print("User ID: {s}\n", .{id});
    }
}.handle);
  • GET /users/42 -> id = "42"
  • GET /users/42/profile -> Does not match this rule (falls through).

Multi-Segment Catch-All Wildcards ({path...})

Captures the remainder of the URL path including slashes:

try mux.handleFunc("GET /static/{filepath...}", struct {
    fn handle(w: *http.ResponseWriter, req: *http.Request) anyerror!void {
        const filepath = req.pathValue("filepath") orelse "";
        try w.writer().print("Requested file: {s}\n", .{filepath});
    }
}.handle);
  • GET /static/css/theme.css -> filepath = "css/theme.css"
  • GET /static/images/logo.png -> filepath = "images/logo.png"

4. Exact Root Matching (/{$})

By default in standard URL routing, a trailing slash / acts as a subtree prefix. If you want a handler that only executes for the root URL / without catching unregistered routes, use the Go 1.22+ /{$} syntax:

try mux.handleFunc("GET /{$}", struct {
    fn handle(w: *http.ResponseWriter, _: *http.Request) anyerror!void {
        try w.writeAll("Welcome to the homepage!\n");
    }
}.handle);

5. Specificity and Precedence Rules

When multiple patterns could match a given request, ServeMux selects the most specific route using deterministic precedence rules:

  1. Host-specific matches take precedence over wildcard host matches.
  2. Method-specific matches take precedence over any-method matches.
  3. Path segment precedence: Literal Segment > Single-Segment Wildcard {id} > Catch-All Wildcard {path...}

For example, given:

  • GET /users/admin (Literal)
  • GET /users/{id} (Wildcard)

A request to GET /users/admin will reliably dispatch to the literal handler, whereas GET /users/123 dispatches to the wildcard handler.


6. Automatic 405 Method Not Allowed

If a route matches a registered path pattern but fails to match the HTTP method, ServeMux automatically:

  1. Traverses the routing tree to collect all registered methods for that path.
  2. Sets the standard Allow header (e.g., Allow: GET, HEAD).
  3. Responds with 405 Method Not Allowed.

Request Handling

The *http.Request struct represents an incoming HTTP/1.1 request. It provides zero-copy access to the HTTP method, parsed URI, path parameters, headers, and streaming body reader.


1. Request Structure

pub const Request = struct {
    method: []const u8,                 // e.g. "GET", "POST"
    uri: []const u8,                    // Raw request URI, e.g. "/items?page=2#top"
    proto: []const u8,                  // e.g. "HTTP/1.1"
    headers: HeaderMap,                 // Parsed HTTP headers
    url: Url,                           // Parsed path, query string, etc.
    arena: std.mem.Allocator,           // Per-request arena allocator
    body_reader: BodyReader,            // Streaming body reader (supports chunked & fixed length)
    path_values: std.StringHashMapUnmanaged([]const u8),
    remote_addr: ?std.Io.net.IpAddress, // Client IP address
};

2. Inspecting the URL & Query Parameters

The req.url struct provides pre-parsed URL components:

// Given GET /search?q=zig&page=1
const path = req.url.path;         // "/search"
const raw_query = req.url.raw_query; // "q=zig&page=1"

// Look up a specific query parameter:
if (req.url.query.get("q")) |search_term| {
    std.debug.print("Search query: {s}\n", .{search_term});
}

3. Extracting Path Parameters

When matching wildcard routes (e.g. /users/{id}), extract dynamic parameters using req.pathValue:

const user_id = req.pathValue("id") orelse "default";

Path parameter slices point directly into the connection’s read buffer or request arena. No string copies are performed.


4. Reading Headers

Request headers can be inspected using req.headers.get:

if (req.headers.get("Authorization")) |token| {
    // Validate Bearer token
}

if (req.headers.get("Accept-Encoding")) |encodings| {
    // Check for gzip or br support
}

Header lookups are case-insensitive per RFC 9110.


5. Reading the Request Body

Lighting provides a streaming BodyReader that handles both Content-Length and Transfer-Encoding: chunked transparently:

Reading the Entire Body into Memory

Use the per-request arena to read the full body safely with an upper bound:

// Read up to 1MB of body data
const max_body_size = 1024 * 1024;
const body_bytes = try req.body_reader.readAllAlloc(req.arena, max_body_size);

std.debug.print("Received body ({d} bytes): {s}\n", .{ body_bytes.len, body_bytes });

Streaming the Body Incrementally

For large file uploads or memory-constrained environments, stream chunk by chunk:

var chunk_buf: [4096]u8 = undefined;
while (true) {
    const bytes_read = try req.body_reader.read(&chunk_buf);
    if (bytes_read == 0) break; // EOF reached

    // Process chunk_buf[0..bytes_read]
}

6. The Per-Request Arena

Every *http.Request includes req.arena: std.mem.Allocator.

  • Any memory allocated with req.arena is automatically freed when the request completes.
  • You do not need to call defer arena.free(ptr).
  • Perfect for parsing JSON payloads, assembling strings, or storing temporary context.

Response & Streaming

The *http.ResponseWriter struct is used to construct and send the HTTP response to the client. It implements a dual-mode buffering design: small responses stay strictly on the stack, while larger bodies transition into chunked streaming automatically.


1. Setting Status Codes & Headers

Headers can be manipulated via w.header():

try w.header().set("Content-Type", "application/json; charset=utf-8");
try w.header().set("X-Server", "Lighting/0.1.0");
try w.header().set("Cache-Control", "no-cache");

Explicit Status Code

By default, the status code is 200 OK. To return a different status code:

w.writeHeader(.not_found); // 404 Not Found
// or:
w.writeHeader(.created);   // 201 Created
// or use the setStatus alias:
w.setStatus(.no_content);  // 204 No Content

Note

w.writeHeader(...) must be called before writing body bytes. Once bytes are written, the response headers are locked and automatically transmitted to the client.


2. Writing Body Content

Lighting provides multiple convenient methods to write body data:

Direct Byte Slices

try w.writeAll("Hello from Lighting!\n");

Formatted Output (w.writer())

The w.writer() method returns a standard Zig std.io.Writer supporting print:

try w.writer().print("Counter value: {d}, User: {s}\n", .{ 42, "admin" });

3. The 4KB Stack Buffer Optimization

For optimal latency and throughput, ResponseWriter embeds a 4KB internal stack buffer:

+--------------------------------------------------------------+
|             ResponseWriter (allocated on stack)              |
|                                                              |
|   +------------------------------------------------------+   |
|   | 4KB Stack Buffer (Accumulates headers & small body)  |   |
|   +------------------------------------------------------+   |
+--------------------------------------------------------------+
  • Responses <= 4KB: When handle returns, Lighting calculates the exact Content-Length, synthesizes the HTTP response header, and sends both the headers and body in a single vectored writeSocketAll syscall. Zero heap allocation occurs.
  • Responses > 4KB: As soon as the 4KB buffer overflows, Lighting automatically switches to Transfer-Encoding: chunked and flushes buffers sequentially across the network.

4. Streaming Large Data

For large payloads or file downloads, stream directly into w.writeAll:

try w.header().set("Content-Type", "application/octet-stream");

var offset: usize = 0;
while (offset < total_size) {
    const chunk = getNextChunk(offset);
    try w.writeAll(chunk);
    offset += chunk.len;
}

5. Early Redirects & Common Errors

You can also send standard error responses or redirects quickly:

// 301 Moved Permanently
try w.header().set("Location", "/new-path");
w.writeHeader(.moved_permanently);

// 400 Bad Request
w.writeHeader(.bad_request);
try w.writeAll("Invalid JSON body\n");

6. Manual Flushing & Connection Hijacking

Immediate Flushing (w.flush())

By default, responses are buffered in memory and flushed automatically when your handler finishes. If you are serving real-time events (e.g., Server-Sent Events / SSE) or progressive chunks, invoke w.flush():

try w.writeAll("data: event 1\n\n");
try w.flush(); // Sends headers (if not sent) and flushes the 4KB buffer immediately

If Content-Length was not specified, the first call to flush() automatically transitions the HTTP/1.1 connection to Transfer-Encoding: chunked.

Connection Hijacking (w.hijack())

For custom binary protocols, raw TCP tunnels, or WebSocket upgrades, w.hijack() gives you complete ownership of the underlying client std.Io.net.Stream:

const stream = try w.hijack();
// Sockets hijacked this way are unregistered from server shutdown tracking.
// You are now solely responsible for reading, writing, and closing the stream:
defer stream.close(w.io);

Warning

Any trailing request bytes already prefetched into the server’s connection buffer are not forwarded. Ensure your handshake requires the client to wait for the server’s upgrade response before transmitting protocol frames.

Handlers & State Injection

Handlers are the core building blocks for processing HTTP requests in Lighting. Lighting uses a fat-pointer interface (Handler) that mirrors Go’s http.Handler, enabling clean composition, zero-cost state injection, and seamless middleware integration.


1. The Handler Interface

A Handler is represented by a fat pointer containing an opaque context pointer and a virtual function table (VTable):

pub const Handler = struct {
    ptr: *anyopaque,
    vtable: *const VTable,

    pub const VTable = struct {
        serveHTTP: *const fn (ctx: *anyopaque, w: *ResponseWriter, req: *Request) anyerror!void,
    };

    pub fn serveHTTP(self: Handler, w: *ResponseWriter, req: *Request) anyerror!void {
        return self.vtable.serveHTTP(self.ptr, w, req);
    }
};

2. Stateless Handlers (handleFunc)

For endpoints that do not require state (e.g., database connections or external caches), register a stateless function using mux.handleFunc:

try mux.handleFunc("GET /ping", struct {
    fn handle(w: *http.ResponseWriter, _: *http.Request) anyerror!void {
        try w.writeAll("pong\n");
    }
}.handle);

Under the hood, mux.handleFunc wraps the function using Handler.fromFn, avoiding pointer dereferences.


3. Stateful Handlers & Dependency Injection (Handler.bind)

In real-world applications, endpoints frequently need access to shared services such as database pools, Redis clients, configuration objects, or user sessions.

Lighting provides Handler.bind, allowing you to bind any struct instance and its member method directly into a Handler:

const AppState = struct {
    db_pool: *DatabasePool,
    app_version: []const u8,

    pub fn getUser(self: *AppState, w: *http.ResponseWriter, req: *http.Request) anyerror!void {
        const id = req.pathValue("id") orelse return error.MissingId;

        // Access database via self.db_pool
        const user = try self.db_pool.queryUser(id);

        try w.header().set("Content-Type", "application/json");
        try w.writer().print("{{\"id\": \"{s}\", \"name\": \"{s}\", \"v\": \"{s}\"}}\n", .{
            user.id, user.name, self.app_version,
        });
    }
};

// In main.zig:
var state = AppState{
    .db_pool = &db,
    .app_version = "1.0.0",
};

// Bind the instance and method:
try mux.handle("GET /users/{id}", http.Handler.bind(&state, AppState.getUser));

Why Handler.bind Excels:

  1. Zero Heap Allocation: Wraps the stack or heap instance directly using compile-time @call.
  2. Type-Safe: The method signature fn (*Self, *ResponseWriter, *Request) anyerror!void is validated at compile time.
  3. No Global Variables: Keeps your state completely isolated and thread-safe.

4. Struct-Level Handlers (Handler.from)

If a struct implements a serveHTTP method directly:

const MetricsHandler = struct {
    counter: std.atomic.Value(u64) = .init(0),

    pub fn serveHTTP(self: *MetricsHandler, w: *http.ResponseWriter, _: *http.Request) anyerror!void {
        const count = self.counter.fetchAdd(1, .monotonic) + 1;
        try w.writer().print("Total requests: {d}\n", .{count});
    }
};

var metrics = MetricsHandler{};
try mux.handle("GET /metrics", http.Handler.from(&metrics));

Middleware

Middleware functions intercept and wrap HTTP request processing. They are ideal for cross-cutting concerns like logging, timing, CORS headers, rate limiting, and panic/error recovery.


1. The Middleware Signature

A middleware in Lighting is a function or struct that wraps an inner Handler and returns an augmented Handler:

pub const MiddlewareFn = *const fn (next: Handler) Handler;

2. Built-in LoggingHandler

Lighting comes with an out-of-the-box LoggingHandler that records the HTTP method, request path, status code, and latency in microseconds or milliseconds:

const http = @import("lighting");

var mux = http.ServeMux.init(allocator);
// ... register routes ...

// Wrap mux with the logging middleware:
var logging = http.LoggingHandler{
    .next = mux.handler(),
};

var server = http.Server.init(allocator, io, .{
    .handler = logging.handler(),
});

Sample output:

info: GET /ping 200 (42µs)
info: GET /users/123 200 (1.45ms)
info: GET /missing 404 (28µs)

3. Writing Custom Middleware

You can write custom middleware either using a struct or with closures.

Struct-based middleware provides the cleanest memory safety and state retention in Zig:

pub const HeaderMiddleware = struct {
    next: http.Handler,
    server_tag: []const u8,

    pub fn serveHTTP(self: *HeaderMiddleware, w: *http.ResponseWriter, req: *http.Request) anyerror!void {
        // Pre-processing: inject response header
        try w.header().set("X-Framework", self.server_tag);

        // Call the downstream handler
        return self.next.serveHTTP(w, req);
    }

    pub fn handler(self: *HeaderMiddleware) http.Handler {
        return http.Handler.from(self);
    }
};

Usage:

var headers_mw = HeaderMiddleware{
    .next = mux.handler(),
    .server_tag = "Lighting 0.1.0",
};

var logging_mw = http.LoggingHandler{
    .next = headers_mw.handler(),
};

var server = http.Server.init(allocator, io, .{
    .handler = logging_mw.handler(),
});

4. Middleware Chaining Pattern

To combine multiple middleware cleanly without nested initializations, define a chain helper:

pub fn chain(inner: http.Handler, middlewares: anytype) http.Handler {
    var current = inner;
    inline for (middlewares) |mw| {
        current = mw(current);
    }
    return current;
}

Or nest struct wrappers:

Client Request
      │
      ▼
[Logging Middleware] (Starts clock)
      │
      ▼
[Security Headers Middleware] (Injects headers)
      │
      ▼
[Router (ServeMux)] (Matches route pattern)
      │
      ▼
[Endpoint Handler] (Processes business logic)

Static Files

Lighting provides a production-grade static file server module in lighting.extra.fs. It supports RFC 9110 HTTP Range requests (for video/audio seeking), ETag caching, directory traversal defense, and responsive dark-mode directory browsing.


1. Quick Setup

const std = @import("std");
const http = @import("lighting");

var fs = http.extra.fs.FileServer.init(allocator, .{
    .root_dir = "./public",         // Filesystem directory to serve
    .strip_prefix = "/static",      // Strip "/static" from the request path
    .auto_index = true,             // Enable HTML directory listing if no index.html
    .index_file = "index.html",     // Default file to serve when targeting a directory
    .show_hidden = false,           // Disallow access to dot-files (.git, .env, etc.)
});

// Mount on the router:
try mux.handle("GET /static/{path...}", fs.handler());

2. Configuration Options (FileServerOptions)

OptionTypeDefaultDescription
root_dir[]const u8(Required)The root directory on the local filesystem. Supports both relative and absolute paths.
strip_prefix?[]const u8nullURL path prefix to strip before resolving filesystem paths (e.g., "/static" or "/assets").
auto_indexbooltrueIf true, renders a responsive directory listing when no index.html exists in the folder.
index_file[]const u8"index.html"Default file served when a directory is requested.
show_hiddenboolfalseWhen false, requests targeting files or directories starting with . return 404 Not Found.
chunk_bytes_maxusize64 * 1024 (64KB)Buffer size used for streaming file contents across the network.

3. RFC 9110 Features

HTTP 206 Partial Content (Range Seeking)

Supports video seeking, large file resumption, and audio streaming:

  • Range: bytes=0-1023: Returns the first 1024 bytes with 206 Partial Content and Content-Range: bytes 0-1023/500000.
  • Range: bytes=1000-: Returns all bytes from byte offset 1000 to end-of-file.
  • Range: bytes=-500: Returns the final 500 bytes of the file.
  • If the range is completely outside file bounds, responds with 416 Range Not Satisfiable.

ETag & HTTP 304 Caching

  • Generates deterministic ETags based on file size and last-modification timestamp.
  • Honors If-None-Match request headers. If the file has not changed, responds immediately with 304 Not Modified and zero body payload, saving bandwidth.

Traversal Defense & Security

  • Strips redundant slashes, dot-segments (., ..), and percent-encoded traversal attempts (%2e%2e%2f).
  • Any path attempting to escape outside root_dir returns 404 Not Found.
  • Symlinks escaping the root boundary are safely rejected.

4. Built-in Directory Browser

When auto_index = true and a directory without an index.html is requested:

  • Displays file names, sizes in human-readable units (B, KB, MB, GB), and last-modified dates.
  • Automatic dark/light theme switching based on user system preferences (@media (prefers-color-scheme: dark)).
  • Folders are sorted before files, with convenient breadcrumb navigation and up-directory links.

WebSocket

Lighting provides first-class support for real-time bidirectional communication via the lighting.extra.ws module. It handles RFC 6455 upgrade handshakes, frame decoding, masking, and message streaming.


1. Upgrading an HTTP Connection

To upgrade an HTTP connection to a WebSocket connection, invoke ws.upgrade from inside any route handler:

const http = @import("lighting");
const ws = http.extra.ws;

try mux.handleFunc("GET /ws", struct {
    fn handle(w: *http.ResponseWriter, req: *http.Request) anyerror!void {
        // Perform RFC 6455 upgrade handshake
        var conn = ws.upgrade(w, req, .{}) catch |err| {
            std.log.warn("WebSocket handshake failed: {}", .{err});
            return;
        };
        defer conn.close();

        std.log.info("Client connected via WebSocket!", .{});

        // Loop reading frames
        while (try conn.readMessage(req.arena)) |msg| {
            switch (msg.type) {
                .text => {
                    // Echo message back with prefix
                    var reply_buf: [512]u8 = undefined;
                    const reply = try std.fmt.bufPrint(&reply_buf, "Echo: {s}", .{msg.payload});
                    try conn.writeText(reply);
                },
                .binary => {
                    // Handle raw binary frame
                    try conn.writeBinary(msg.payload);
                },
                .close => {
                    std.log.info("Client requested clean close", .{});
                    break;
                },
            }
        }
    }
}.handle);

2. Reading Messages

conn.readMessage(arena) parses incoming frames and handles RFC 6455 details:

  • Unmasking: Incoming client frames are masked; the connection unmasks payloads in place.
  • Control Frames: Automatically responds to Ping frames with Pong frames.
  • Fragmentation: Reassembles fragmented frames across packets.
  • Memory Lifecycle: Returns payload slices allocated within req.arena for zero-leak memory safety.

3. Sending Messages

  • Text Messages: try conn.writeText("hello world") (sends a UTF-8 encoded text frame).
  • Binary Messages: try conn.writeBinary(byte_slice) (sends a raw binary frame).
  • Close: conn.close() sends a close control frame and cleanly closes the underlying socket.

API Reference

The interactive HTML API reference for Lighting is automatically generated from Zig doc comments via zig build docs.


Local Generation

You can build and preview the API documentation locally:

# Build the HTML API documentation
zig build docs

# Preview with a local web server
python3 -m http.server -d zig-out/docs 8080

System Architecture & I/O Flow

Lighting is engineered from first principles as an embedded, zero-dependency HTTP/1.1 engine. This chapter details the lifecycle of a TCP connection and the end-to-end data flow through the system.


High-Level Request Pipeline

flowchart TD
    classDef endpoint stroke:#2b7fff,stroke-width:2px;
    classDef process stroke:#059669,stroke-width:2px;
    classDef decision stroke:#d97706,stroke-width:2px;

    Client(["HTTP Client"]) -->|"TCP Connection"| Listener["Server.serve / listenAndServe Accept Loop"]
    Listener -->|"self.group.concurrent"| Fiber["std.Io.Group Fiber / Task"]
    Fiber -->|"serveConn"| ConnBuf["Connection Read Buffer (8KB ~ 16KB)"]
    
    subgraph Connection_Loop ["Keep-Alive Loop (serveConn)"]
        direction TB
        ConnBuf --> Precheck{"Does buffer already contain<br/>CRLF CRLF boundary? (Pipelining)"}
        Precheck -->|"No"| SocketRead["io_util.readSocketWithTimeout (Absolute Deadline)"]
        Precheck -->|"Yes"| Parser["proto.parseRequestHead (Zero-Copy Parsing)"]
        SocketRead --> Parser
        
        Parser --> Arena["Initialize Per-Request ArenaAllocator"]
        Arena --> Req["Construct Request (BodyReader + Headers)"]
        Arena --> Res["Construct ResponseWriter (4KB Dual-Mode Stack Buffer)"]
        
        Req --> Router["ServeMux (Go 1.22+ Decision Tree Router)"]
        Router -->|"Extract Path Param {id}"| Handler["User Handler (serveHTTP)"]
        
        Handler -->|"Write Response"| Res
        Res -->|"finish"| OutVecs["Stack Vecs Output (Zero-Heap Serialization)"]
        OutVecs -->|"writeSocketAllTimeout"| Client
        
        Res --> DrainCheck{"Unread Body Drain"}
        DrainCheck -->|"> 256KB Threshold"| ConnClose["Close Connection (Anti-DoS)"]
        DrainCheck -->|"<= 256KB Threshold"| Reuse["Shift Leftover Bytes to Head, Reuse Connection"]
    end
    style Connection_Loop stroke:#6b7280,stroke-width:2px,stroke-dasharray: 5 5;

    class Client,ConnClose,Reuse endpoint;
    class Listener,Fiber,ConnBuf,SocketRead,Parser,Arena,Req,Res,Router,Handler,OutVecs process;
    class Precheck,DrainCheck decision;

Lifecycle Stages

1. Connection Accept Loop

  • The listening socket is polled asynchronously within the main fiber.
  • Upon receiving a client handshake, a new TCP stream is accepted.
  • The stream is passed to group.concurrent(io, serveConnWorker, .{ self, stream }).

2. Keep-Alive Loop (serveConn)

  • An 8KB–16KB buffer (conn_buf) is allocated for the connection.
  • Pipelining Check: Before executing a socket read syscall, the server checks if leftover bytes from a previous pipelined request already contain the HTTP header boundary \r\n\r\n. If so, socket I/O is bypassed.
  • Zero-Copy Parsing: parseRequestHead validates HTTP framing and slices headers in place without allocating heap memory.

3. Per-Request Scoping

  • A dedicated std.heap.ArenaAllocator is initialized for the request.
  • Temporary strings, route path-value maps, and body reader buffers reside inside this arena.
  • The user handler runs to completion.

4. Zero-Heap Response Serialization

  • If the response is <= 4KB, it is formatted entirely inside the stack buffer of ResponseWriter.
  • Status line, response headers, and body are dispatched in a single vectored write (writev).

5. Residual Body Draining & Recycling

  • If the client uploaded an unread body <= 256KB, the server automatically drains residual bytes to keep the socket synchronized.
  • Any unread pipelined bytes are shifted to the front of conn_buf using std.mem.copyForwards.
  • The arena is reset, and the connection waits for the next request.

Structured Concurrency

Lighting adopts native structured concurrency via std.Io.Group, aligning directly with Go’s go c.serve() model while avoiding the pitfalls of unbounded thread pools.


1. Structured Concurrency vs Thread Pools

Traditional web servers in C or Rust often use a fixed-size worker thread pool fronted by a mutex-protected task queue:

  • Queue Saturation: When slow clients or keep-alive connections hold worker threads, queue backpressure blocks the entire server.
  • Complex Teardown: Coordinating thread shutdown, draining task queues, and freeing connection states requires distributed ref-counting or lock-free queues.

In Lighting, every accepted connection is scheduled onto the runtime fiber scheduler:

// Inside Server accept loop:
try self.group.concurrent(self.io, serveConnWorker, .{ self, stream });

Advantages of Structured Concurrency:

  1. Hierarchical Lifecycles: All running connection tasks belong to self.group. When termination is signaled or serve() exits, calling self.group.cancel(self.io) immediately notifies all connection workers.
  2. Guaranteed Joining: Awaiting the group guarantees that every connection has completed its shutdown cleanup before serve() returns.
  3. No Thread Starvation: Fibers yield cooperatively to the Zig runtime whenever they wait on network socket reads or writes.

2. Absolute Deadline Timeouts (io_util.zig)

Relative socket timeouts (e.g. setsockopt(SO_RCVTIMEO)) are susceptible to Slowloris attacks: a malicious client sending 1 byte every 4.9 seconds can hold a connection open indefinitely against a 5-second relative timeout.

Lighting implements absolute deadline timeouts:

pub fn readSocketWithDeadline(
    io: std.Io,
    stream: std.Io.net.Stream,
    buf: []u8,
    deadline: ?std.Io.Clock.Awake.Instant,
) !usize {
    // Calculates the remaining time relative to an immutable instant in the future
}
  • Header Deadline: Sockets have a strict deadline (e.g., 2 seconds from TCP handshake) to complete the entire HTTP header frame.
  • Keep-Alive Idle Deadline: When a connection becomes idle, it sleeps with an idle deadline. If no new request arrives within the window, the kernel closes the connection cleanly.

Zero-Copy Protocol Parser

The HTTP/1.1 protocol parser in src/proto/parser.zig is designed for high throughput, zero heap allocation, and strict RFC 9112 conformance.


1. Zero-Copy In-Place Slicing

Traditional parsers copy strings into newly allocated buffers for method names, header keys, and values.

Lighting takes a different approach:

  • The connection read buffer (conn_buf) receives raw socket data.
  • As parseRequestHead validates HTTP tokens, it produces slices []const u8 pointing directly into conn_buf.
  • Zero heap memory is requested from the OS or GPA during request line and header parsing.

2. Request Smuggling Defenses (RFC 9112 Section 6.3)

HTTP request smuggling occurs when upstream proxies and downstream servers disagree on request body boundaries. Lighting enforces strict RFC 9112 defenses:

  1. Both Transfer-Encoding and Content-Length Present:
    • RFC 9112 §6.3 mandates that if a request contains both headers, Transfer-Encoding takes precedence.
    • To prevent desynchronization attacks against naive proxies, Lighting strips the Content-Length header entirely when chunked is detected.
  2. Invalid Transfer-Encoding:
    • If Transfer-Encoding contains an unsupported coding (e.g. gzip as transfer coding), Lighting immediately rejects the request with 501 Not Implemented.
  3. Invalid Content-Length:
    • If Content-Length contains non-numeric characters, multiple values, or exceeds max integer bounds, Lighting immediately returns 400 Bad Request.

3. Pipelining & CRLF Boundary Detection

HTTP/1.1 pipelining allows clients to send multiple requests in a single TCP stream without waiting for responses.

  • After a request finishes, leftover bytes in conn_buf are scanned for the \r\n\r\n boundary.
  • If a full header frame is already present in the buffer, Lighting parses it immediately without incurring an unnecessary recv syscall.
  • If a partial frame is present, remaining bytes are shifted forward to the buffer head via std.mem.copyForwards before the next socket read.

Routing Tree Algorithm

The router in src/mux.zig implements a decision tree inspired by Go 1.22’s router enhancements, supporting prefix trees, single-segment wildcards, multi-segment catch-alls, and method dispatch.


1. Decision Tree Traversal (RoutingNode)

flowchart TD
    classDef branch stroke:#2b7fff,stroke-width:2px;
    classDef literal stroke:#059669,stroke-width:2px;
    classDef wildcard stroke:#d97706,stroke-width:2px;
    classDef multi stroke:#dc2626,stroke-width:2px;

    R["Root Routing Node"] -->|"1. Match Host"| H["Host Branch<br/>('example.com' or wildcard)"]
    H -->|"2. Match Method"| M["Method Branch<br/>('GET', 'POST' or any)"]
    
    subgraph SG_Path ["3. Segment Match & Backtracking (Precedence Order)"]
        direction TB
        M --> P1["Seg 1: 'users' (Literal)"]
        P1 --> P2["Seg 2: 'special'<br/>(Exact Literal: Highest Priority)"]
        P1 --> P3["Wildcard: '{id}'<br/>(Single-Segment: Medium Priority)"]
        P1 --> P4["Multi: '{filepath...}'<br/>(Catch-All Multi-Segment: Low Priority)"]
    end
    style SG_Path stroke:#6b7280,stroke-width:2px,stroke-dasharray: 5 5;

    class R,H,M branch;
    class P1,P2 literal;
    class P3 wildcard;
    class P4 multi;

2. Segment Matching Order

Routing nodes prioritize child matches using deterministic specificity:

  1. Exact Literal Segment:
    • Evaluates literal strings directly (e.g., "users").
    • O(1) string equality check against incoming segment.
  2. Single-Segment Wildcard ({name}):
    • Matches any single segment between two slashes.
    • Extracted key-value pair is stored into req.path_values.
  3. Multi-Segment Catch-All ({path...}):
    • Captures all remaining path segments including slashes.
    • Lowest priority in a branch; acts as fallback.

3. Backtracking Resolution

If an exact literal match branch fails downstream (for instance, /users/admin/settings when only /users/{id} is registered), the routing engine backtracks cleanly to wildcard and multi-segment candidate nodes instead of prematurely returning 404 Not Found.


4. Automatic 405 Method Collection

When a request matches a registered URL path but the requested HTTP method differs:

  1. ServeMux walks all sibling method branches for the resolved path.
  2. It aggregates all valid methods (e.g., GET, HEAD, POST).
  3. It sets the standard Allow header and returns 405 Method Not Allowed.

Memory Model & Buffer Recycling

Lighting is designed to avoid heap fragmentation and provide deterministic memory usage under sustained network loads.


1. The 4KB Dual-Mode ResponseWriter

Every ResponseWriter allocates a 4KB fixed buffer directly on the thread stack:

+--------------------------------------------------------------+
|            ResponseWriter (4KB Stack Fast-Path)              |
|                                                              |
|   [ Status Line ] + [ Headers ] + [ Body Data <= 4KB ]       |
+--------------------------------------------------------------+

Fast Path (<= 4KB)

  1. Headers and body data are accumulated in memory.
  2. When the handler exits, Lighting computes Content-Length: <len> automatically.
  3. Slices are packaged into a stack-allocated IO vector array ([3]std.posix.iovec).
  4. A single vectored write transmits the entire HTTP frame to the network socket.
  5. No heap allocations are performed.

Overflow Path (> 4KB)

  1. Once 4KB is exceeded, Lighting flushes headers with Transfer-Encoding: chunked.
  2. Subsequent writes stream in chunks directly to the socket.

2. Per-Request Arena Allocator

Instead of using the general-purpose allocator (GPA) for request processing, Lighting provisions a std.heap.ArenaAllocator:

var arena_state = std.heap.ArenaAllocator.init(self.allocator);
defer arena_state.deinit();

var req = Request{
    .arena = arena_state.allocator(),
    // ...
};
  • All temporary objects (URL query maps, parsed headers, route parameters, JSON DOMs) use req.arena.
  • Upon request completion, arena_state.deinit() (or reset()) reclaims all memory in O(1) time.
  • This eliminates pointer tracking, prevents use-after-free bugs, and drastically reduces memory fragmentation.

3. Residual Body Draining

When a client sends a request with a large body that the handler ignores (e.g., an unauthorized POST /upload returning 401 Unauthorized without reading the file), unread bytes remain in the TCP socket.

If the server simply attempts to read the next HTTP request, it will mistake the leftover upload data for HTTP verbs, leading to parse failures and desync attacks.

Lighting implements bounded automatic draining:

  • Drain Threshold: Up to 256KB of leftover body is drained automatically.
  • DoS Protection: If leftover body exceeds 256KB, Lighting closes the connection immediately rather than wasting CPU cycles downloading discarded bytes.

TigerStyle & Defensive Limits

Lighting is developed following TigerStyle principles — an engineering discipline distilled from mission-critical systems like TigerBeetle. It prioritizes explicit limits, defensive invariants, and strict RFC protocol compliance over premature flexibility.


1. Explicit Defensive Bounds

Unbounded inputs are a primary cause of out-of-memory (OOM) crashes and DoS vulnerabilities. Lighting enforces explicit architectural limits across all subsystems:

LimitValueRFC / Rationale
Max Header Size8KB ~ 16KBProtects against memory exhaustion from oversized headers (431 Request Header Fields Too Large).
Max Headers Count128Bounds header parsing stack size and hash table allocations.
Max Body Drain Limit256KBPrevents malicious slow drains on rejected keep-alive requests.
Max URL URI Length8KBRFC 9112 URI limit (414 URI Too Long).
Connection BuffersBounded 8KB ~ 16KBEach connection uses deterministic memory. 10,000 idle connections consume only ~80MB RAM.

2. Invariants and Preconditions

Every function in Lighting uses assertions to verify invariants at boundaries:

  • Buffer Invariants: debug.assert(read_index <= write_index); ensures buffer pointers cannot desynchronize.
  • Socket Ownership: Streams are owned by exactly one fiber task at any point in time.
  • HTTP Status Code Bounds: Status codes outside 100..599 trigger compile-time or runtime assertion failures.

3. RFC Compliance Matrix

  • RFC 9110 (HTTP Semantics):
    • Strict Host header presence check on HTTP/1.1 requests (400 Bad Request if missing).
    • Automatic Allow header formatting on 405 Method Not Allowed.
    • Proper handling of If-None-Match for HTTP 304 Not Modified caching.
    • Standard Range unit verification (bytes) and 416 Range Not Satisfiable for invalid ranges.
  • RFC 9112 (HTTP/1.1 Messaging):
    • Chunked transfer coding parsing with boundary verification.
    • Stripping conflicting Content-Length when Transfer-Encoding: chunked is present to neutralize request smuggling.
  • RFC 6455 (WebSocket Protocol):
    • Strict SHA-1 / Base64 Sec-WebSocket-Accept generation.
    • Proper masking and opcode validation for client frames.

Minimal REST Service

This recipe demonstrates building a clean, zero-dependency REST microservice with routing, dynamic path variables, and request logging.

Source code: examples/hello/main.zig


Code Walkthrough

const std = @import("std");
const http = @import("lighting");

pub fn main(init: std.process.Init) !void {
    const allocator = init.gpa;
    const io = init.io;
    const port: u16 = if (init.environ_map.get("PORT")) |p|
        std.fmt.parseInt(u16, p, 10) catch 7000
    else
        7000;

    var mux = http.ServeMux.init(allocator);
    defer mux.deinit();

    // 1. Root greeting (Exact match)
    try mux.handleFunc("GET /{$}", struct {
        fn handle(w: *http.ResponseWriter, _: *http.Request) anyerror!void {
            try w.header().set("Content-Type", "text/plain; charset=utf-8");
            try w.writeAll("⚡️ Welcome to Lighting web framework!\n");
        }
    }.handle);

    // 2. Dynamic path parameter: /hello/{name}
    try mux.handleFunc("GET /hello/{name}", struct {
        fn handle(w: *http.ResponseWriter, req: *http.Request) anyerror!void {
            const name = req.pathValue("name") orelse "friend";
            try w.header().set("Content-Type", "text/plain; charset=utf-8");
            try w.writer().print("Hello, {s}!\n", .{name});
        }
    }.handle);

    // 3. Wrap with logging middleware
    var logging = http.LoggingHandler{ .next = mux.handler() };

    var server = http.Server.init(allocator, io, .{
        .handler = logging.handler(),
    });

    std.log.info("Hello server listening on http://127.0.0.1:{d}...", .{port});
    try server.listenAndServe("127.0.0.1", port);
}

Running the Example

make run-hello

Test with curl:

$ curl -i http://127.0.0.1:7000/hello/zig
HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Content-Length: 11
Connection: keep-alive

Hello, zig!

SQLite & Embedded SPA

This recipe demonstrates building a full-stack CRUD web application with an embedded dark-mode Single Page Application (SPA), REST API endpoints, and a local SQLite database using state injection (Handler.bind).

Source code: examples/todo/


Key Highlights

1. State Injection with Handler.bind

Instead of relying on global database pointers, the App struct holds the database handle and binds methods cleanly to the router:

pub const App = struct {
    allocator: std.mem.Allocator,
    db: *SqliteDB,

    pub fn handleList(self: *App, w: *http.ResponseWriter, req: *http.Request) anyerror!void {
        const todos = try self.db.list(req.arena);
        try w.header().set("Content-Type", "application/json");
        try w.writer().print("{f}\n", .{std.json.fmt(todos, .{})});
    }

    pub fn handleCreate(self: *App, w: *http.ResponseWriter, req: *http.Request) anyerror!void {
        const body = try req.readBodyAlloc(64 * 1024);
        const parsed = try std.json.parseFromSlice(
            struct { title: []const u8 },
            req.arena,
            body,
            .{ .ignore_unknown_fields = true },
        );
        defer parsed.deinit();

        const todo = try self.db.create(req.arena, parsed.value.title);
        w.writeHeader(.created);
        try w.header().set("Content-Type", "application/json");
        try w.writer().print("{f}\n", .{std.json.fmt(todo, .{})});
    }
};

// Route binding in main:
try mux.handle("GET /api/todos", http.Handler.bind(&app, App.handleList));
try mux.handle("POST /api/todos", http.Handler.bind(&app, App.handleCreate));
try mux.handle("POST /api/todos/{id}/toggle", http.Handler.bind(&app, App.handleToggle));
try mux.handle("DELETE /api/todos/{id}", http.Handler.bind(&app, App.handleDelete));

2. Single-Binary Distribution with @embedFile

The frontend HTML/JS/CSS assets are embedded directly into the compiled executable at compile time:

pub fn handleIndex(_: *App, w: *http.ResponseWriter, _: *http.Request) anyerror!void {
    const html = @embedFile("index.html");
    try w.header().set("Content-Type", "text/html; charset=utf-8");
    try w.writeAll(html);
}

The resulting binary has zero filesystem asset dependencies and can be deployed anywhere as a single standalone executable.


Running the Example

make run-todo

Open http://127.0.0.1:7000 in your web browser to interact with the SPA.

Real-Time WebSocket App

This recipe demonstrates setting up real-time bidirectional WebSocket communication with an interactive client interface.

Source code: examples/ws/


Server Implementation

const std = @import("std");
const http = @import("lighting");
const ws = http.extra.ws;

const index_html = @embedFile("index.html");

pub fn main(init: std.process.Init) !void {
    const allocator = init.gpa;
    const io = init.io;

    var mux = http.ServeMux.init(allocator);
    defer mux.deinit();

    // 1. Serve embedded interactive WebSocket HTML client
    try mux.handleFunc("GET /{$}", struct {
        fn handle(w: *http.ResponseWriter, _: *http.Request) anyerror!void {
            try w.header().set("Content-Type", "text/html; charset=utf-8");
            try w.writeAll(index_html);
        }
    }.handle);

    // 2. WebSocket upgrade and message echo loop
    try mux.handleFunc("GET /ws", struct {
        fn handle(w: *http.ResponseWriter, req: *http.Request) anyerror!void {
            var conn = ws.upgrade(w, req, .{}) catch |err| {
                std.log.warn("WebSocket upgrade error: {}", .{err});
                return;
            };
            defer conn.close();

            while (try conn.readMessage(req.arena)) |msg| {
                switch (msg.type) {
                    .text => {
                        var reply_buf: [512]u8 = undefined;
                        const reply = std.fmt.bufPrint(&reply_buf, "Echo: {s}", .{msg.payload}) catch msg.payload;
                        try conn.writeText(reply);
                    },
                    .binary => {
                        try conn.writeBinary(msg.payload);
                    },
                    .close => break,
                }
            }
        }
    }.handle);

    var server = http.Server.init(allocator, io, .{
        .handler = mux.handler(),
    });

    try server.listenAndServe("127.0.0.1", 7000);
}

Running the Example

make run-ws

Visit http://127.0.0.1:7000 to send text and binary messages and see the real-time responses.

Static File Server CLI

This recipe demonstrates using lighting.extra.fs to build a full-featured static file server CLI with RFC 9110 Range seeking, ETag caching, and an adaptive dark-mode directory browser.

Source code: examples/fs/


Server Implementation

const std = @import("std");
const http = @import("lighting");

pub fn main(init: std.process.Init) !void {
    const allocator = init.gpa;
    const io = init.io;

    var args = try std.process.argsAlloc(allocator);
    defer std.process.argsFree(allocator, args);

    // Default to current directory if not specified
    const target_dir = if (args.len > 1) args[1] else ".";

    var fs = http.extra.fs.FileServer.init(allocator, .{
        .root_dir = target_dir,
        .auto_index = true,
        .index_file = "index.html",
        .show_hidden = false,
    });

    var mux = http.ServeMux.init(allocator);
    defer mux.deinit();

    try mux.handle("GET /{path...}", fs.handler());
    try mux.handle("HEAD /{path...}", fs.handler());

    var logging = http.LoggingHandler{ .next = mux.handler() };

    var server = http.Server.init(allocator, io, .{
        .handler = logging.handler(),
    });

    std.log.info("Serving directory '{s}' at http://127.0.0.1:7000...", .{target_dir});
    try server.listenAndServe("127.0.0.1", 7000);
}

Running the Example

Run in the project directory:

make run-fs

Or pass a custom directory:

cd examples/fs && zig build run -- /path/to/my/docs

Features included:

  • Video/Audio Streaming: Supports HTML5 <video> scrubbing via HTTP 206 Partial Content.
  • Bandwidth Optimization: HTTP 304 Not Modified caching via strong ETags.
  • Security: Traversal attacks like /../../etc/passwd are sanitized and safely rejected.

Lighting Web Framework Design & Implementation Specification

Version: 1.0.0
Language Version: Zig 0.17.0
Design Philosophy: Completely custom-built protocol stack (strictly avoiding std.http.Server), 1:1 parity with Go net/http API, first-principles driven, and TigerStyle industrial robustness.


1. Project Background and Design Goals

1.1 Why Avoid Zig’s Standard Library std.http.Server

  • Performance and Allocation Bottlenecks: The standard library server introduces internal buffer copies and inefficient stream destructuring, which severely limits throughput in persistent Keep-Alive connections and high-concurrency environments.
  • Missing Go-Style Engineering Abstractions: Lacks the familiar ResponseWriter, Handler fat-pointer interfaces, and modern pattern routing (Go 1.22+) that developers expect.
  • Vulnerable Protocol Edge Cases: Lacks Slowloris absolute deadline terminations, lacks HTTP request smuggling defenses (RFC 9112), and risks deadlocking on pipelined connection batches.

1.2 Core Architectural Principles

  1. API Parity with Go net/http: Provides identical developer experience with w.writeAll(), w.header().set(), w.writeHeader(), r.pathValue(), and ServeMux.
  2. Zero-Copy First: The request line, URL, and header names/values are directly sliced from the underlying socket read buffer without redundant allocations or copies.
  3. Deterministic Memory Lifecycles: Each HTTP request is backed by an independent ArenaAllocator, performing an O(1) bulk deallocation immediately after request completion to prevent memory fragmentation and leaks.
  4. Kernel-Grade Synchronization with Zero Busy Waiting: Leverages standard futex synchronization primitives (std.Io.Mutex and std.Io.Condition), ensuring 0.0% CPU usage when idling.

2. Overall System Architecture

2.1 Module Structure

lighting/
├── build.zig                 # Build script and test targets
├── build.zig.zon             # Package metadata
├── bench/
│   └── main.zig              # Benchmark server and endpoint test harness
├── docs/
│   └── design.md             # This design document
├── tests/
│   └── e2e/                  # End-to-end integration test suite
│       ├── helpers.zig       # TestServer and I/O test fixtures
│       ├── root.zig          # E2E test suite aggregator
│       ├── proto_test.zig    # Core HTTP semantics
│       ├── routing_test.zig  # Pattern routing and matching
│       ├── rfc_security_test.zig # RFC 9112 and security compliance
│       ├── concurrency_test.zig # Server lifecycle and workers
│       ├── streaming_test.zig   # Flush and Hijack primitives
│       ├── cookie_multipart_test.zig # Form and Cookie parsing
│       ├── extra_test.zig    # WebSocket, FileServer, and Proxy
│       └── client_test.zig   # HTTP client integration tests
└── src/
    ├── root.zig              # Public package exports (Server, Handler, Mux, etc.)
    ├── server.zig            # Connection listening, Keep-Alive scheduling, timeout enforcement
    ├── handler.zig           # Handler fat-pointer interface and closures
    ├── mux.zig               # Go 1.22+ decision-tree pattern routing engine
    ├── request.zig           # Request struct, URL parsing, and pathValue extraction
    ├── response_writer.zig   # Dual-mode buffering ResponseWriter (Auto Content-Length / Chunked)
    ├── header.zig            # Case-insensitive HeaderMap (order-preserving, multi-value support)
    ├── thread_pool.zig       # Futex-synchronized worker pool (O(1) cursor queue)
    ├── cookie.zig            # Cookie parsing and Set-Cookie serialization
    ├── multipart.zig         # Multipart form data streaming parser
    ├── timeout.zig           # Route-level timeout middleware
    ├── client/               # HTTP client implementation (Client, Request, Response, Transport)
    ├── extra/                # High-level ecosystem extensions (ws, fs, proxy)
    └── proto/
        ├── method.zig        # HTTP Method enum and zero-copy string parsing
        ├── status.zig        # HTTP Status Code and status phrase mappings
        ├── parser.zig        # Zero-copy HTTP request header parser
        ├── chunk_reader.zig  # Streamed Body reader (Chunked decoding & trailer absorption)
        └── io_util.zig       # Vectored socket I/O with absolute deadline timeouts

2.2 Request Processing Data Flow

flowchart TD
    classDef endpoint stroke:#2b7fff,stroke-width:2px;
    classDef process stroke:#059669,stroke-width:2px;
    classDef decision stroke:#d97706,stroke-width:2px;

    Client(["HTTP Client"]) -->|"TCP Connection"| Listener["Server.serve / listenAndServe Accept Loop"]
    Listener -->|"self.group.concurrent"| Fiber["std.Io.Group Fiber / Task"]
    Fiber -->|"serveConn"| ConnBuf["Connection Read Buffer (8KB ~ 16KB)"]
    
    subgraph Connection_Loop ["Keep-Alive Loop (serveConn)"]
        direction TB
        ConnBuf --> Precheck{"Does buffer already contain<br/>CRLF CRLF boundary? (Pipelining)"}
        Precheck -->|"No"| SocketRead["io_util.readSocketWithTimeout (Absolute Deadline)"]
        Precheck -->|"Yes"| Parser["proto.parseRequestHead (Zero-Copy Parsing)"]
        SocketRead --> Parser
        
        Parser --> Arena["Initialize Per-Request ArenaAllocator"]
        Arena --> Req["Construct Request (BodyReader + Headers)"]
        Arena --> Res["Construct ResponseWriter (4KB Dual-Mode Stack Buffer)"]
        
        Req --> Router["ServeMux (Go 1.22+ Decision Tree Router)"]
        Router -->|"Extract Path Param {id}"| Handler["User Handler (serveHTTP)"]
        
        Handler -->|"Write Response"| Res
        Res -->|"finish"| OutVecs["Stack Vecs Output (Zero-Heap Serialization)"]
        OutVecs -->|"writeSocketAllTimeout"| Client
        
        Res --> DrainCheck{"Unread Body Drain"}
        DrainCheck -->|"> 256KB Threshold"| ConnClose["Close Connection (Anti-DoS)"]
        DrainCheck -->|"<= 256KB Threshold"| Reuse["Shift Leftover Bytes to Head, Reuse Connection"]
    end
    style Connection_Loop stroke:#6b7280,stroke-width:2px,stroke-dasharray: 5 5;

    class Client,ConnClose,Reuse endpoint;
    class Listener,Fiber,ConnBuf,SocketRead,Parser,Arena,Req,Res,Router,Handler,OutVecs process;
    class Precheck,DrainCheck decision;

3. Core Protocol Engine and Zero-Copy Parsing

3.1 Zero-Copy Request Header Parser (src/proto/parser.zig)

  • Direct Memory Slicing: Extracted target, header.key, and header.val slices point directly into the connection’s read buffer conn_buf with zero heap allocations during parsing.
  • RFC 9112 Section 6.3 Request Smuggling Defense: If multiple Content-Length headers are received with conflicting values, the parser rejects the request immediately with error.InvalidContentLength, causing the server to respond with 400 Bad Request.
  • RFC 9112 Section 6.1 Transfer-Encoding Precedence: If both Transfer-Encoding: chunked and Content-Length are present, content_length is explicitly forced to null, ensuring chunked framing governs the message boundaries.
  • RFC 9112 Section 7.1 Mandatory Host Header: HTTP/1.1 requires every request message to include a Host header. Requests lacking a Host header are rejected with 400 Bad Request.

3.2 Streamed Body Reader (src/proto/chunk_reader.zig)

  • Unified Stream Interface (BodyReader): Provides a consistent read(dest: []u8) !usize method whether the body is governed by a fixed Content-Length or dynamic Transfer-Encoding: chunked.
  • Chunk Extensions Support: RFC 9112 permits parameters on chunk size lines (e.g., 1a;name=val\r\n). The decoder strips ; extensions safely before parsing hexadecimal integers.
  • Buffer Overflow Prevention & Trailer Absorption: Caps chunk size lines at 64 bytes to prevent memory exhaustion; properly absorbs all optional trailer headers following the terminal 0\r\n chunk until an empty line is reached.
  • Fast-Fail on Oversized Bodies: In readAllAlloc, if remaining_content_length > max_size, it fails fast immediately with error.StreamTooLong without performing redundant socket reads.

3.3 Dual-Mode ResponseWriter (src/response_writer.zig)

  • Dual-Mode Buffering:
    • Small Response Mode (< 4KB): Body data accumulates in an internal 4KB stack buffer. When the handler completes, finish() automatically computes and sets Content-Length: <len>, writing the status line, headers, and body in a single writev system call.
    • Streaming / Large Response Mode (>= 4KB or early flushes): If data exceeds 4KB without an explicit Content-Length, the writer automatically transitions to Transfer-Encoding: chunked.
  • Zero-Allocation Stack Header Buffering: flushHeaders() utilizes a stack-allocated vector array stack_vecs: [64][]const u8. For 99.99% of requests, header serialization occurs entirely on the stack with zero heap allocation.
  • RFC 9110 Section 8.6 Status Code Enforcement: Automatically prohibits emitting Content-Length headers or body payloads for 204 No Content and 304 Not Modified responses.
  • Explicit Chunk Streaming (flush()): Allows handlers to emit partial response chunks immediately without waiting for buffer saturation. If headers have not been sent and no explicit Content-Length was declared, flush() transitions framing to Transfer-Encoding: chunked.
  • Connection Takeover (hijack()): Enables protocol upgrades (e.g., WebSocket). Flushes any pre-buffered response data before handing over the raw std.Io.net.Stream. If Content-Length was not declared, the flushed prefix is chunk-framed without a terminal chunk (0\r\n\r\n). Trailing request bytes already read into the connection buffer are not forwarded, so protocols must follow request-response handshake discipline. On hijack the server unregisters the connection from shutdown tracking immediately, relinquishing stream lifecycle management to the caller.

4. Modern Pattern Routing Engine (Go 1.22+ Standard)

Modeled after Go’s standard library implementation in net/http/routing_tree.go, the router utilizes a 3-tier decision tree:

4.1 Pattern Syntax and Supported Types

  • HTTP Method Matching: "GET /ping", "POST /users".
  • Path Parameters: "GET /users/{id}", extracted in zero-copy fashion using req.pathValue("id").
  • Catch-All Wildcards: "GET /files/{filepath...}", matching all remaining path segments (e.g., docs/2026/report.pdf).
  • Exact Root Matching: "/{$}" matches only the exact root path "/", preventing greedy capture of other registered routes.

4.2 Decision Tree Traversal (RoutingNode)

flowchart TD
    classDef branch stroke:#2b7fff,stroke-width:2px;
    classDef literal stroke:#059669,stroke-width:2px;
    classDef wildcard stroke:#d97706,stroke-width:2px;
    classDef multi stroke:#dc2626,stroke-width:2px;

    R["Root Routing Node"] -->|"1. Match Host"| H["Host Branch<br/>('example.com' or wildcard)"]
    H -->|"2. Match Method"| M["Method Branch<br/>('GET', 'POST' or any)"]
    
    subgraph SG_Path ["3. Segment Match & Backtracking (Precedence Order)"]
        direction TB
        M --> P1["Seg 1: 'users' (Literal)"]
        P1 --> P2["Seg 2: 'special'<br/>(Exact Literal: Highest Priority)"]
        P1 --> P3["Wildcard: '{id}'<br/>(Single-Segment: Medium Priority)"]
        P1 --> P4["Multi: '{filepath...}'<br/>(Catch-All Multi-Segment: Low Priority)"]
    end
    style SG_Path stroke:#6b7280,stroke-width:2px,stroke-dasharray: 5 5;

    class R,H,M branch;
    class P1,P2 literal;
    class P3 wildcard;
    class P4 multi;

4.3 Precedence Rules and Fallbacks

  • Specificity Precedence: Literal exact match > Single-segment wildcard {id} > Multi-segment wildcard {path...}.
  • Automatic Backtracking: If a single-segment wildcard branch fails to match downstream, the engine backtracks cleanly and attempts multi-segment wildcard fallbacks.
  • Automatic 405 Method Not Allowed: When a path is registered but the request method does not match, the tree collects all allowed methods for that path and responds with 405 Method Not Allowed with an Allow header (e.g., Allow: POST or Allow: GET, HEAD).

5. Concurrency, Memory, and Lifecycle Management

5.1 Go-Aligned Structured Concurrency (std.Io.Group.concurrent)

  • Go net/http Parity (go c.serve()): Instead of a fixed OS thread pool with a bounded task queue, Server adopts Zig 0.17’s native structured concurrency via std.Io.Group. Each accepted connection is immediately spawned with group.concurrent(io, serveConnWorker, .{ self, stream }).
  • Zero OS Thread Starvation & Fiber Execution: Under std.Io.Evented (Linux io_uring, macOS Dispatch, BSD kqueue), connections run as lightweight user-space Fibers (green threads) with non-blocking I/O. Idle Keep-Alive connections do not consume OS threads or CPU.
  • Deterministic Structured Lifecycle: The lifetime of all connection tasks is anchored to std.Io.Group. When shutting down, group.cancel(io) cleanly terminates all active tasks and reclaims Fiber stacks with zero memory leaks.

5.2 Keep-Alive and Pipelined Requests

  • Pipelining Deadlock Elimination: Before calling the blocking socket read, serveConn inspects whether complete headers (\r\n\r\n) already reside in conn_buf from pipelined batches, avoiding false read timeouts.
  • Bounded Body Drain (256 KB): Paralleling Go’s maxPostHandlerReadBytes, if a handler completes early, the server drains up to 256 KB of remaining body bytes. If unread content exceeds 256 KB, the server actively terminates the TCP connection to prevent resource exhaustion attacks. If the body is exactly 256 KB, it drains to EOF and cleanly retains Keep-Alive reuse.
  • Direct Idle Deadline Wait (Zero Slicing Overhead): Because idle connections in the Fiber/Concurrent model do not block OS worker threads, idle Keep-Alive waits are passed directly to io_util.readSocketWithTimeout with the absolute idle_deadline. Sockets sleep directly in the OS event loop without CPU-wasting periodic time slices.

5.3 Comprehensive Timeout & Protocol Protections

  • Slowloris Mitigation (read_header_timeout_ns): Computes a fixed monotonic deadline timestamp (Clock.Timestamp) using std.Io.Timeout.toDeadline(io). Even if an attacker trickles bytes at 1 byte/sec, reaching the absolute deadline triggers 408 Request Timeout and severs the socket.
  • Idle Connection Reclamation (idle_timeout_ns): Persistent Keep-Alive connections waiting between requests are subject to idle timeouts, closing silently without leaking file descriptors.
  • Read / Write Deadlines (read_timeout_ns / write_timeout_ns): Seamlessly enforced within BodyReader and ResponseWriter socket loops.
  • Expect: 100-continue Support (RFC 9110 §10.1.1): When an HTTP/1.1 client sends Expect: 100-continue, the server immediately transmits HTTP/1.1 100 Continue\r\n\r\n before the handler blocks reading the body, preventing two-way deadlocks. Unrecognized expectations receive 417 Expectation Failed.
  • RFC 9110 §6.6.1 Date Header Enforcement: ResponseWriter automatically generates and formats a standard IMF-fixdate Date header (e.g. Date: Sun, 06 Nov 1994 08:49:37 GMT) on all responses unless explicitly overridden by the handler.
  • Direct Socket Handle I/O (src/proto/io_util.zig): Bypasses std.Io.net.Stream.read/write destructuring layers to perform zero-overhead vectored operations with absolute deadline support directly via std.Io.Operation.

6. Performance Benchmarks and Verification

6.1 Automated Integration Tests

Run zig build test --summary all:

  • 83/83 tests pass.
  • Test coverage includes: Zero-copy request line/header parsing, chunked decoding and trailer absorption, dual-mode ResponseWriter, Keep-Alive connection reuse, HTTP pipelining batch execution, request smuggling defenses, 204/304 status code compliance, Go 1.22 decision tree routing, 405 Method Not Allowed, 408 Slowloris termination, and 256 KB body drain caps.

6.2 Concurrency Benchmark Results (Apple Silicon, 8 Worker Threads)

Benchmarked using ApacheBench (ab -c 20 -n 2000):

Test ScenarioTotal Req / ConcurrencyThroughput (QPS)Mean LatencyFailures
Static Endpoint (/ping)2000 / 2028,172 req/sec0.035 ms0
Pattern Parameter Endpoint (/users/42)2000 / 2029,478 req/sec0.034 ms0
  • CPU Utilization Profile: Idles at strictly 0.0% CPU; under maximum load, threads wake up instantaneously and distribute work evenly across cores; returns to 0.0% immediately upon test completion.

Benchmarks

The benchmark suite evaluates Lighting against standard Go (net/http) and Rust (actix-web) web servers with aligned timeout policies and hardware-aware core pinning.

For the full test suite, automated execution scripts, and comparative results:

Lighting Development & Debugging Guide

This guide compiles common workflows, debugging tips, and verification techniques for working on the Lighting web framework.


1. Verifying HTTP Keep-Alive

Validating persistent connection reuse (Keep-Alive) is critical when developing network services. The simplest and most reliable way to test this is with curl.

Specify multiple URLs in a single curl invocation with verbose mode (-v):

curl -v http://127.0.0.1:7000/ http://127.0.0.1:7000/

When Keep-Alive is operating correctly, two key markers appear in the verbose output:

  1. At the end of the first request:

    < HTTP/1.1 200 OK
    < Connection: keep-alive
    < Content-Length: 46
    <
    * Connection #0 to host 127.0.0.1 left intact   <-- Connection remains open
    
  2. At the beginning of the second request:

    * Re-using existing connection with host 127.0.0.1 <-- Reuses TCP socket, no new handshake
    > GET / HTTP/1.1
    > Host: 127.0.0.1:7000
    

1.2 Testing Sequential Endpoints with --next

To test different paths or endpoints over the exact same persistent TCP connection:

curl -v http://127.0.0.1:7000/hello/alice --next http://127.0.0.1:7000/hello/bob

Both HTTP requests are transmitted sequentially across the same socket, with Re-using existing connection displayed before the second request.


1.3 Negative Verification (Connection: close)

To confirm proper behavior when the client requests disconnection via Connection: close:

curl -v -H "Connection: close" http://127.0.0.1:7000/ http://127.0.0.1:7000/

Both server and client acknowledge the termination:

  • < Connection: close
  • * Closing connection 0
  • The subsequent request triggers a fresh TCP three-way handshake (* Connected to ...).

Lighting Feature Parity & Roadmap

This document provides a gap analysis comparing Lighting against Go’s standard net/http library and outlines the roadmap for future development.


1. Current State vs. Go net/http

Lighting provides a solid foundation for HTTP/1.1 services:

  • Go 1.22+ Pattern Routing: Method constraints, zero-copy path parameters ({id}), wildcards ({filepath...}), exact match (/{$}), and automatic 405 Method Not Allowed.
  • Structured Concurrency: Native std.Io.Group.concurrent task dispatching (paralleling Go’s go c.serve()) with user-space Fiber execution.
  • Zero-Copy Protocol Parser: In-place header and path extraction directly from the socket buffer.
  • Dual-Mode ResponseWriter: Automatic Content-Length buffering (< 4KB) and streaming chunked transfer encoding for larger bodies.
  • Production Hardening: Slowloris defense via monotonic header deadlines, Keep-Alive connection reuse, RFC 9112 request smuggling prevention, and connection-scoped Arena reuse with high-water mark protection (request_arena_retain_limit).

2. Feature Parity Matrix

The following table summarizes the status of Lighting’s phased features:

CapabilityGo net/httpLighting CurrentImpact / Notes
http.Flusherw.(http.Flusher).Flush()✅ ResponseWriter.flush()Immediate output in Server-Sent Events (SSE) and LLM token streaming.
http.Hijackerw.(http.Hijacker).Hijack()✅ ResponseWriter.hijack()Raw TCP socket handoff for protocol upgrades.
WebSocketgolang.org/x/net/websocket✅ lighting.extra.wsFull RFC 6455 handshake, frame parsing, XOR unmasking, Ping/Pong, and echo.
Static File Serverhttp.FileServer(http.Dir)✅ lighting.extra.fs.FileServerRFC 7233 Range requests (audio/video seek), ETag / If-Modified-Since (304), MIME detection, and responsive directory browser.
Query Parametersreq.URL.Query() multi-value map✅ req.url.query() / queryParam()Multi-value parsed query map (QueryParams), single-key zero-copy lookup, and key existence checks.
Cookiesreq.Cookie(), http.SetCookie()✅ lighting.Cookie, req.cookie(), w.setCookie()RFC 6265 typed cookie parser, iterator, and Set-Cookie builder supporting SameSite, HttpOnly, Secure, Expires, and Max-Age.
Multipart Formreq.ParseMultipartForm()✅ lighting.MultipartForm, req.multipartForm()Streaming parser for RFC 7578 multipart/form-data with disk spilling for file uploads exceeding memory thresholds.
Reverse Proxyhttputil.ReverseProxy✅ lighting.extra.proxy.ReverseProxyReverse proxy forwarding requests, stripping RFC 9110 hop-by-hop headers, injecting X-Forwarded-*, and streaming bodies.
Route Timeoutshttp.TimeoutHandler✅ lighting.timeoutHandler()Route-level timeout middleware canceling handler execution and returning 504 Gateway Timeout on overrun.
HTTP Clienthttp.Client, Transport pool✅ lighting.ClientHigh-level HTTP client with get, post, postForm, do, CookieJar, ConnPool, and redirect following.
Panic Recoverydefer recover() per connection⚠️ Errors caught, @panic fatalHandler panics crash the task/process instead of returning HTTP 500; anyerror gracefully handled as 500.
TLS / HTTPSListenAndServeTLS❌ Not implementedRequires an external reverse proxy (e.g. Nginx, Caddy) for SSL termination.
HTTP/2 SupportBuilt-in via ALPN negotiation❌ HTTP/1.1 onlyNo multiplexing over a single TCP connection.

3. Phased Roadmap

Phase 1: Real-Time & Streaming Primitives (Completed)

  • ResponseWriter.flush(): Allow handlers to immediately push buffered chunks to the TCP socket without closing the connection. Enables Server-Sent Events (SSE) and generative AI chat streaming.
  • ResponseWriter.hijack(): Allow handlers to take ownership of the underlying std.Io.net.Stream. Enables WebSocket servers and protocol upgrade handshakes.
  • Handler Fault Isolation: Add graceful error reporting and stack recovery if handler code triggers unexpected failures.
  • WebSocket Support (lighting.extra.ws): RFC 6455 compliance with frame parsing, control frames, and interactive echo clients.

Phase 2: Enhanced Request Parsing & Utilities (Completed)

  • Cookie Support: Add a zero-copy Cookie parser and Set-Cookie builder supporting RFC 6265 attributes (HttpOnly, Secure, SameSite, Max-Age).
  • Comprehensive URL Query Parameter Decoder: Extend URL.queryParam into full multi-value iterator and helper to decode percent-encoded query parameters.
  • Multipart Form Data Parser: Streamed parser for multipart/form-data with configurable memory and file size limits for file uploads.

Phase 3: Ecosystem Handlers & Middleware (Completed)

  • Static File Server (lighting.extra.fs.FileServer): Serve directory trees with automatic Content-Type deduction, RFC 7233 Range requests, and If-Modified-Since / ETag 304 caching.
  • Reverse Proxy Handler (lighting.extra.proxy.ReverseProxy): Built-in reverse proxy with Director callback to rewrite request headers, strip hop-by-hop headers, and stream responses back.
  • Route-Level Timeout Middleware (lighting.timeoutHandler): Middleware to cancel or abort specific handler executions exceeding an allotted duration.

Phase 4: Protocol Expansion

  • Native TLS / HTTPS Support: Integrate with Zig TLS implementations or OS-native TLS facilities to serve HTTPS directly on port 443.
  • HTTP/2 Protocol Support: Support HTTP/2 framing, header compression (HPACK), and frame-level stream multiplexing.