# The HTTP Request Journey in Go: From Network to Handler

Tracing the complete journey of an HTTP request through every layer of the Go stack

Canonical HTML: https://stage.vvivid.dev/posts/http_in_golang/
Published: 2025-08-08


For the HTTP/1.x server path shown here, Go accepts a connection, reads and parses the request, routes it through `ServeMux`, and calls your handler. Understanding those steps makes connection lifetimes, middleware, timeouts, and response-writing bugs easier to debug. HTTP/2 uses a different connection and stream model.


As a backend guy, I've been tasked with building various APIs for different projects, and most of them I built using Go. For the majority of the time, what most people do is write a handler or middleware to process and respond to requests. Today, we're going to do something different. We'll go down a rabbit hole together into the abstractions that Go has taken away from us. We'll see all the interesting things that Go does before handing us the request and response objects in our handler. After this deep dive, we'll understand how to parse requests, how routing matching works, middleware execution, and finally the architecture that Go uses to handle each request.

## Start with a handler registered through http.HandleFunc {#the-simple-beginning-httphandlefunc}

Let's start with something we all know:

```go
http.HandleFunc("/hello", func(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hello, World!")
})
http.ListenAndServe(":8080", nil)
```

Simple, right? I used to think this was just "magic" that somehow worked. But there's actually a whole orchestrated series of processes and steps that make this happen, and we don't seem to appreciate it.

## HTTP/1.x request flow from socket to handler {#the-complete-journey-what-really-happens}

Here's a simplified version of Go's actual `net/http` source code when you call `ListenAndServe`:

```go
// From Go's net/http package (simplified)
func ListenAndServe(addr string, handler Handler) error {
    server := &Server{Addr: addr, Handler: handler}
    return server.ListenAndServe()
}

func (srv *Server) ListenAndServe() error {
    // Step 1: Create TCP listener
    ln, err := net.Listen("tcp", srv.Addr)
    if err != nil {
        return err
    }
    return srv.Serve(ln)
}

func (srv *Server) Serve(l net.Listener) error {
    // Step 2: Accept connections loop
    for {
        rw, err := l.Accept()
        if err != nil {
            // handle temporary errors with backoff
            continue
        }
        
        // Step 3: Handle connection in goroutine
        c := srv.newConn(rw)
        go c.serve()
    }
}

func (c *conn) serve() {
    // Step 4: Connection state management
    c.setState(StateNew)
    
    for {
        // Step 5: Read and parse HTTP request
        req, err := c.readRequest()
        if err != nil {
            break
        }
        
        // Step 6: Create response writer
        w := &response{conn: c, req: req}
        
        // Step 7: Find handler (route matching)
        handler := c.server.Handler
        if handler == nil {
            handler = DefaultServeMux
        }
        
        // Step 8: Execute handler
        handler.ServeHTTP(w, req)
        
        // handle keep-alive, connection cleanup, etc.
    }
}

func (c *conn) readRequest() (*Request, error) {
    // parse request line: "GET /path HTTP/1.1"
    // parse headers: "Content-Type: application/json"
    // create Request object with method, URL, headers, body
    // return parsed request
}
```

Now let's break down each step to see what Go actually does at each stage.

## Step 1: TCP Socket Creation

```go
// From server.ListenAndServe()
func (srv *Server) ListenAndServe() error {
    ln, err := net.Listen("tcp", srv.Addr)
    if err != nil {
        return err
    }
    
    // set socket options like SO_REUSEADDR
    // configure TCP keep-alive settings
    // handle IPv4/IPv6 dual stack if needed
    
    return srv.Serve(ln)
}
```

Go automatically sets `SO_REUSEADDR` to handle the "address already in use" error. When your server closes, connections go into TIME_WAIT state, but this setting tells the OS to reuse the address immediately.

## Step 2: Accept Loop - Waiting for Connections

```go
// From server.Serve()
func (srv *Server) Serve(l net.Listener) error {
    for {
        rw, err := l.Accept()
        if err != nil {
            // check if error is temporary
            // implement exponential backoff for temporary errors
            // log permanent errors and continue
            continue
        }
        
        c := srv.newConn(rw)
        c.setState(StateNew)
        go c.serve() // Each connection gets its own goroutine
    }
}
```

Key points:
- `Accept()` blocks until a connection arrives
- Each connection gets its own goroutine (2KB initial stack)
- Go handles temporary network errors with automatic retry
- 10,000 connections ≈ 20MB total memory

## Step 3: Connection Handling

```go
// From conn.serve()
func (c *conn) serve() {
    c.setState(StateNew)
    
    for {
        // read request with timeout
        // handle connection state transitions
        // manage keep-alive behavior
        
        w, err := c.readRequest()
        if err != nil {
            // handle read errors, bad requests
            // close connection if needed
            break
        }
        
        c.setState(StateActive)
        // process request...
        c.setState(StateIdle) // or StateClosed
    }
}
```

**Connection States:**
- **StateNew**: Just accepted, not processing yet
- **StateActive**: Currently handling a request  
- **StateIdle**: Waiting for next request (keep-alive)
- **StateHijacked**: Connection taken over (WebSockets)
- **StateClosed**: Connection finished

Go manages these states automatically for keep-alive optimization.

## Step 4: Reading the Request

```go
// From conn.readRequest()
func (c *conn) readRequest() (*Request, error) {
    c.r.setReadLimit(c.server.maxHeaderBytes() + 4096)
    
    // read request line: "GET /path HTTP/1.1"
    line, err := readLine(c.bufr)
    if err != nil {
        return nil, err
    }
    method, requestURI, proto := parseRequestLine(line)
    
    // read headers
    mimeHeader, err := c.readMIMEHeader()
    if err != nil {
        return nil, err
    }
    
    // create Request object
    req := &Request{
        Method: method,
        URL:    parseRequestURI(requestURI),
        Proto:  proto,
        Header: Header(mimeHeader),
        Body:   c.r, // streaming body reader
        // set other fields...
    }
    
    return req, nil
}
```

Key point: Go streams the request body through `req.Body` instead of loading everything into memory. Headers are parsed first, then body is available as an `io.Reader`.

## Step 5: HTTP Parsing Details

### Header Parsing
```go
// From textproto.ReadMIMEHeader()
func ReadMIMEHeader(r *Reader) (MIMEHeader, error) {
    m := make(MIMEHeader)
    
    for {
        line, err := r.readLineSlice()
        if len(line) == 0 {
            break // empty line = end of headers
        }
        
        // find colon separator
        i := bytes.IndexByte(line, ':')
        if i < 0 {
            return nil, errors.New("malformed header")
        }
        
        key := canonicalMIMEHeaderKey(line[:i])
        value := string(bytes.TrimSpace(line[i+1:]))
        m[key] = append(m[key], value)
    }
    return m, nil
}
```

Go normalizes header names with `canonicalMIMEHeaderKey()`. "content-type" becomes "Content-Type".

### Body Handling  
```go
// The request body in your handler
func handler(w http.ResponseWriter, r *http.Request) {
    // r.Body is a streaming reader, not loaded into memory
    body, err := io.ReadAll(r.Body)
    defer r.Body.Close()
    
    // only reads what you actually request
}
```

## Step 6: Route Matching (ServeMux)

```go
// From ServeMux.match()
func (mux *ServeMux) match(path string) (h Handler, pattern string) {
    // exact match first
    v, ok := mux.m[path]
    if ok {
        return v.h, v.pattern
    }
    
    // then try longest prefix match
    for _, e := range mux.es {
        if strings.HasPrefix(path, e.pattern) {
            return e.h, e.pattern
        }
    }
    
    // fallback to root handler
    return NotFoundHandler(), ""
}

func (mux *ServeMux) Handler(r *Request) (h Handler, pattern string) {
    // handle host-specific patterns
    // normalize path (remove .. and . components)
    // check for trailing slash redirects
    
    return mux.match(r.URL.Path)
}
```

If you register "/users/" and someone hits "/users", Go automatically redirects with a 301.

Custom mux avoids global state issues:
```go
mux := http.NewServeMux()
mux.HandleFunc("/hello", helloHandler)
```

## Step 7: Request Processing

```go
// From conn.serve() - before calling handler
func (c *conn) serve() {
    for {
        req, err := c.readRequest()
        if err != nil {
            // handle various error types
            if c.r.hitReadLimit() {
                c.writeError(431) // Request Header Fields Too Large
            }
            // check for client disconnect, timeout, etc.
            break
        }
        
        // handle Expect: 100-continue
        if req.expectsContinue() {
            if req.Body == nil || c.writeHeaders() {
                c.write([]byte("HTTP/1.1 100 Continue\r\n\r\n"))
            }
        }
        
        w := &response{
            conn: c,
            req:  req,
            // setup response writer...
        }
        
        // find and execute handler
        handler := c.server.Handler
        if handler == nil {
            handler = DefaultServeMux
        }
        handler.ServeHTTP(w, req)
        
        // start reading next request in background (pipelining)
        // handle connection keep-alive
    }
}
```

## Step 8: Handler Execution - Finally, Your Code!

```go
// Your handler gets called via ServeHTTP
func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {
    f(w, r)
}

// What your handler receives
func myHandler(w http.ResponseWriter, r *http.Request) {
    // r.Method = "GET", "POST", etc.
    // r.URL.Path = "/api/users"  
    // r.Header = parsed HTTP headers
    // r.Body = streaming request body
    
    // w.Header() = response headers (must set before writing body)
    // w.Write() = write response body
    // w.WriteHeader() = set status code
    
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(200)
    w.Write([]byte(`{"message": "hello"}`))
    
    // Go handles response buffering, connection cleanup, etc.
}
```

By the time your handler runs, Go has:
1. Accepted the TCP connection
2. Parsed the HTTP request completely
3. Matched the route pattern
4. Created Request and ResponseWriter objects
5. Set up panic recovery

After all that setup, your handler finally runs. Notice there's no `go` keyword here. It runs in the same goroutine as the connection. Why? Because HTTP/1.1 says responses must come back in order. If a browser sends Request A then Request B, it needs Response A then Response B, not the other way around.

This is why different clients can be served in parallel (different goroutines), but one client's requests are handled sequentially. Makes sense when you think about it!

### Response Writing

The `ResponseWriter` is buffered, but headers are special:

```go
func handler(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    // Headers must be set before writing body!
    w.WriteHeader(200)
    w.Write([]byte(`{"message": "hello"}`))
}
```

Once you call `Write()` or `WriteHeader()`, you can't modify headers anymore. This is a common gotcha when headers aren't being set.

## The Response Journey Back

The response follows the reverse path, but there are some interesting optimizations:

### Automatic Headers

Go automatically sets several headers for you:
- `Date`: Current timestamp
- `Content-Length`: If it can determine the size
- `Connection`: Keep-alive behavior

### Chunked Transfer Encoding

If Go doesn't know the content length, it automatically uses chunked encoding:

```go
func handler(w http.ResponseWriter, r *http.Request) {
    for i := 0; i < 10; i++ {
        fmt.Fprintf(w, "Chunk %d\n", i)
        w.(http.Flusher).Flush() // Send immediately
        time.Sleep(time.Second)
    }
}
```

This is perfect for streaming responses or server-sent events. This is perfect for real-time dashboards. Instead of polling, the server can just keep sending updates.

### The Keep-Alive Decision

After each request, Go decides: keep this connection open or close it? If the client said "Connection: close" or the server's shutting down, it closes. Otherwise, it goes to StateIdle and waits for the next request.

There's also some clever timeout management happening. Go uses `Peek(4)` to look for the start of the next request without consuming the bytes. The timeout only kicks in AFTER the client starts sending something, not while they're thinking. It's like a patient waiter who only starts the timer after you've opened the menu.

## Error Handling and Edge Cases

Go handles a lot of edge cases that I never thought about:

### Timeout Handling

```go
server := &http.Server{
    Addr:           ":8080",
    Handler:        myHandler,
    ReadTimeout:    10 * time.Second,
    WriteTimeout:   10 * time.Second,
    IdleTimeout:    120 * time.Second,
}
```

These timeouts prevent connections from hanging forever, especially useful when clients disappear mid-request.

### Panic Recovery

If your handler panics, Go automatically recovers and sends a 500 response:

```go
func panicHandler(w http.ResponseWriter, r *http.Request) {
    panic("Something went wrong!")
    // Go handles this gracefully
}
```

## Performance Insights I Wish I Knew Earlier

### Connection Pooling

Go automatically pools connections for outgoing requests:

```go
client := &http.Client{
    Transport: &http.Transport{
        MaxIdleConns:        100,
        MaxIdleConnsPerHost: 10,
    },
}
```

Understanding this helped me optimize my microservices communication significantly.

### Keep-Alive Behavior

HTTP keep-alive is enabled by default, which means connections are reused:

```go
// One TCP connection can handle multiple requests
GET /users HTTP/1.1
Host: api.example.com
Connection: keep-alive

HTTP/1.1 200 OK
Content-Length: 1234
Connection: keep-alive

GET /posts HTTP/1.1  // Same connection!
Host: api.example.com
```

### Memory Usage

Each goroutine starts with a 2KB stack, but it can grow:

```go
// This won't explode your memory (usually)
for i := 0; i < 10000; i++ {
    go handleConnection(conn)
}
```

But be careful with long-running goroutines that accumulate large stacks.

## Debugging Tricks I've Learned

### Request Dumping

```go
func debugHandler(w http.ResponseWriter, r *http.Request) {
    dump, _ := httputil.DumpRequest(r, true)
    log.Printf("Request: %s", dump)
}
```

This is useful for debugging weird client behavior.

### Connection State Monitoring

```go
server := &http.Server{
    Addr: ":8080",
    ConnState: func(conn net.Conn, state http.ConnState) {
        log.Printf("Connection %v: %v", conn.RemoteAddr(), state)
    },
}
```

Super useful for understanding connection patterns in production.

### pprof Integration

```go
import _ "net/http/pprof"

go func() {
    log.Println(http.ListenAndServe("localhost:6060", nil))
}()
```

This gives you runtime insights into goroutines, memory usage, and CPU profiles.

## Common Gotchas That Bit Me

### 1. Not Closing Request Bodies

```go
resp, err := http.Get("https://api.example.com")
if err != nil {
    return err
}
defer resp.Body.Close() // Don't forget this!
```

Forgetting this will leak connections and eventually exhaust your connection pool.

### 2. Header Case Sensitivity

```go
// These are all the same to Go
r.Header.Get("Content-Type")
r.Header.Get("content-type")
r.Header.Get("CONTENT-TYPE")
```

But when setting headers, use the canonical format: `w.Header().Set("Content-Type", "...")`

### 3. Query Parameter Parsing

```go
// URL: /search?q=hello%20world&category=tech
query := r.URL.Query()
q := query.Get("q") // "hello world" (automatically decoded)
```

Go automatically URL-decodes query parameters, which is usually what you want.

## What This All Means for Real Applications

Understanding this flow has helped me:

1. **Debug production issues** faster by knowing where to look
2. **Optimize performance** by understanding the bottlenecks
3. **Make better architectural decisions** about timeouts and connection limits
4. **Write more robust error handling** by understanding what can go wrong

The key insight for me was realizing that Go's HTTP package isn't just a simple wrapper. It's a sophisticated system designed for high-performance, concurrent web services.

## What to inspect when a Go HTTP request fails {#wrapping-up}

When I started this deep dive, I thought I was just going to figure out why my connections were hanging. Instead, I ended up with a much deeper appreciation for Go's HTTP implementation.

The beautiful thing about Go is that it makes the simple case (like my original hello world) trivial, while still giving you access to all this power when you need it. You can write `http.HandleFunc("/", handler)` and have a working server, but when you need to optimize for production, all these knobs and insights are available.

If you're working with HTTP in Go, I highly recommend diving into the `net/http` source code sometime. It's surprisingly readable and you'll learn a ton about both HTTP and Go's concurrency patterns.

---

## Related protocol and server articles {#more-deep-dives-you-might-like}

**[Building a JSON-RPC Server in Go](/posts/jsonrpc_example/)** - See how Go handles a different protocol pattern. We build a complete JSON-RPC implementation using similar Go internals and reflection patterns.

---

Got questions about any of this? Hit me up at [oudavid@vvivid.dev](mailto:oudavid@vvivid.dev). I love talking about this stuff!