The landscape, as it actually is​

gorilla/websocket is the long-standing default, and it's what wshub is built on top of. It gives you one thing well: a Conn with ReadMessage/WriteMessage, ping/pong, close-frame handling, and permessage-deflate support — a correct implementation of RFC 6455, nothing more. Everything past "I have a frame" — registries, rooms, fanout, backpressure — is yours to write.

coder/websocket (the maintained continuation of nhooyr.io/websocket) is the modern take on the same layer: zero dependencies, first-class context.Context support, a smaller surface area than gorilla's. It's a genuinely nicer API to hold a single connection with. It makes the same choice gorilla does, though — transport only. No rooms, no broadcast primitive, nothing about what happens when you have ten thousand of these open at once.

gobwas/ws goes a level lower than either: raw frame headers, your own buffer reuse, your own masking. It exists for people who want to hand-roll the protocol for performance reasons gorilla and coder's abstractions get in the way of. Correct and fast, and further from "batteries included" than any library on this list.

melody is the closest thing that already exists to what wshub is going for — a thin convenience layer over gorilla that adds broadcast-to-all, broadcast-to-filtered, per-session key/value storage, and automatic ping/pong timeout handling. If what you need is "send this to everyone, and don't leak dead connections," melody solves it in a dozen lines and you should probably use it. What it doesn't have: rooms as a first-class concept, a backpressure policy when a slow client's buffer fills, or any story for running more than one node.

centrifuge (and its companion server, Centrifugo) sits at the other end entirely. It's a real-time messaging platform — channels, presence, history/recovery on reconnect, JWT-based auth — either embedded as a library or run as its own clustered server that your services talk to over a client SDK. It's the right answer if you're building the realtime equivalent of a message bus and want most of the plumbing decided for you. It's also a bigger adoption decision than a single WebSocket endpoint usually calls for: a new protocol surface between your services and your clients, and in the server form, a whole additional system to run and operate.

None of this is a knock on any of them. They're each solving the problem they set out to solve. The gap is what's left standing between "a raw transport library" and "a hosted real-time platform" — an embeddable hub layer: something you go get into your own service, not a server you stand up next to it, that gives you rooms, backpressure, graceful shutdown, and multi-node scaling as library code you control.

What that layer actually has to do​

This isn't abstract. Across the rest of this series, the specific problems that show up the moment a gorilla/websocket demo meets real traffic — and that neither a transport library nor a full platform server quite hands you in the shape a single Go service wants — were:

• A registry and rooms. Someone has to track which *Client values are live and which room each one belongs to, safely under concurrent connect/disconnect. Article 1 walks through why a plain map behind a mutex isn't enough once broadcast and unregister can run at the same time.
• Backpressure with an actual policy, not a crash. One slow client shouldn't be able to block a broadcast to everyone else, and an unbounded queue for that client is just a slower way to run out of memory. wshub's answer — bounded per-client channels plus a configurable drop policy (DropOldest or DropNewest) with a notification hook — is the subject of article 3.
• A graceful shutdown that actually waits. "Close every connection" is one line. Draining in-flight sends, handling a client that's open but silent, and doing it within a deadline instead of hanging a rolling deploy needs a small state machine — covered in article 5.
• More than one node, correctly. The moment you run two replicas behind a load balancer, "broadcast to everyone" has to cross a process boundary without double-delivering to clients on the node that originated the message. Article 4 covers the adapter pattern wshub uses for this (Redis and NATS, pluggable, with node-ID dedup built in) — the part that a single-process library like melody has no story for at all, and that Centrifugo solves by giving you a clustered server with its own Redis-backed engine, rather than an adapter you plug into a service you already run.
• Honest numbers for the question everyone asks first. Article 2 measures what 10,000 open connections and a sustained broadcast actually cost, in real memory and real latency, instead of leaving you to extrapolate from a microbenchmark.

None of these are hard problems individually. They're the kind of thing a competent team writes correctly on the third try, after shipping the first two versions to production and finding out what they missed. wshub's pitch is narrow: do that work once, benchmark it, and keep it zero-allocation on the hot path (SendToClient is 105 ns/op with zero allocations at 100K registered clients — reproducible with go test -bench=. -benchmem ./...), so the team doesn't have to re-derive it.

Is it "the last one you need"?​

Honestly — depends on what you're building. If you want a real-time platform with presence, message history, and reconnect recovery as a managed concern, and you're fine adopting a server and a client SDK to get it, centrifuge/Centrifugo is a more complete answer than wshub is trying to be. If all you need is "broadcast this to everyone on one node," melody is lighter weight and you don't need wshub's configuration surface at all.

Where wshub earns the "last one" framing is the specific, common middle case: a single Go service that owns its own WebSocket endpoint, needs rooms and backpressure and graceful shutdown to be correct from day one, and will eventually need more than one node — without wanting to stand up a separate real-time server to get there. That's the gap none of the libraries above fill as a library, and it's the one wshub was built to close.

go get github.com/KARTIKrocks/wshub

The room-based chat example is the fastest way to see rooms and broadcasting working end to end; the full capability list is in the README, and the rest of this series goes deep on each piece above.

This is article three in a series on production-grade WebSocket servers in Go — article one listed the eight problems production traffic surfaces, article two went deep on connection-scaling arithmetic. This one is about the mechanism that's easy to get approximately right and surprisingly easy to get exactly wrong: dropping messages under load without dropping correctness along with them. The code below is from wshub's actual send path, not a simplified sketch — the races it guards against are the reason the "obvious" implementation isn't the one that shipped.

The 80% version, and why it's not enough​

The standard advice for a slow-consumer problem is: don't write to the socket directly, write to a bounded channel, and make the write non-blocking.

select {
case client.send <- data:
    // enqueued
default:
    // buffer full — drop it
}

This is correct as far as it goes, and it's most of what you need. A writePump goroutine drains client.send and owns the actual socket writes, so a full buffer never blocks the caller — broadcast, targeted send, whatever code path got you here returns immediately either way. This is why SendToClient costs ~105 ns with zero allocations at 100K clients in wshub's benchmarks: the send path never waits on I/O, only on a channel operation that's guaranteed not to block.

But two things are still unhandled, and both of them only show up under concurrency — which is exactly why they're easy to ship without noticing.

Race 1: the client can close between your check and your send​

A Client has a closed flag, set when the connection tears down. The obvious guard is to check it before sending:

if client.closed {
    return false // don't bother sending
}
client.send <- data

This looks safe and isn't. Nothing prevents the client from disconnecting — and its writePump from closing client.send — in the window between the check and the send. That's a plain time-of-check-to-time-of-use race, and when it loses, the result isn't a dropped message: it's a panic. Sending on a closed channel is one of the handful of things Go will not let you recover from gracefully by checking state first, because there's no atomic "check and send" primitive for channels the way there is for, say, sync/atomic.CompareAndSwap.

wshub's actual send path still does the closed check — as a fast-path optimization, to skip the panic/recover machinery for the common case where a client is already known to be gone — but it doesn't trust that check. Every send is wrapped in a recover:

defer func() {
    if r := recover(); r != nil {
        if isChanSendPanic(r) {
            ok = false // client.send was closed — the client is disconnecting
            return
        }
        panic(r) // re-panic for anything else
    }
}()

The isChanSendPanic check matters more than it looks. Go doesn't give you a typed error for "send on closed channel" — it's a runtime panic carrying a plain string, and that string is the only signal available:

func isChanSendPanic(r any) bool {
    // ...
    return strings.Contains(msg, "send on closed channel")
}

The re-panic on anything else is the important part of this design, not an afterthought. A recover that swallows every panic turns real bugs — a nil pointer dereference two calls deep in a message handler, an index out of range in application code — into silent, undebuggable message drops. Matching the specific panic string and re-panicking on everything else means this recover catches exactly the one race it exists to catch, and gets out of the way of every other failure.

Race 2: evicting under DropOldest isn't one operation either​

DropNewest (discard the message that didn't fit) is race-free almost by definition — there's nothing to coordinate, you just don't enqueue. DropOldest (evict the head of the queue to make room for the new message, because only the latest state matters) is a different story: it's two channel operations — a receive to evict, a send to enqueue — and if two goroutines hit this path for the same client at once, both can decide the buffer is full, both start evicting, and you can end up dropping two messages to make room for one.

wshub serializes this specific sequence with a per-client mutex, and only for this path — the fast, common non-blocking send above never touches it:

func (h *Hub) trySendDropOldest(client *Client, item sendItem) bool {
    client.sendMu.Lock()
    defer client.sendMu.Unlock()

    // Re-check: the buffer may have drained while we waited for the lock —
    // writePump could have pulled a message off it in the meantime.
    select {
    case client.send <- item:
        return true
    default:
    }

    // Evict the oldest message, then enqueue the new one. Retry twice: a
    // concurrent fast-path sender (one that hasn't hit this lock at all)
    // can still slide into the slot we just freed before our own enqueue
    // runs, so the first evict+enqueue attempt can lose the race and needs
    // a second try rather than giving up and dropping the new message too.
    for range 2 {
        select {
        case dropped := <-client.send:
            h.notifySendDropped(client, dropped.data)
        default:
        }
        select {
        case client.send <- item:
            return true
        default:
        }
    }
    return false
}

The mutex only has to serialize this path against itself — the non-blocking fast path everywhere else still runs lock-free, so DropNewest (the default) pays none of this cost, and DropOldest pays it only on the buffer-full path, not on every send. That's the actual tradeoff DropOldest buys you: strictly-correct "keep only the newest N" semantics, at the cost of a mutex acquisition exactly when a client is already struggling to keep up — which is the one time you can least afford a slow send path, and precisely why the retry logic exists instead of a simpler "evict once, give up if that races."

Choosing a policy is a semantics question, not a performance one​

Given the two policies work at different cost, it's tempting to treat DropNewest as the "cheap" default and DropOldest as an opt-in for when you need it. That's true, but it's not the deciding factor — the deciding factor is what a dropped message means for your protocol:

• DropNewest is correct for append-only streams — chat messages, log lines, discrete events — where every message is independently meaningful and losing the newest of several queued ones is recoverable; the client still gets a consistent, if incomplete, history.
• DropOldest is correct for state-replacement streams — a live cursor position, a stock price, a presence indicator — where only the current value matters and delivering a stale one is actively wrong. Here, dropping the new message under DropNewest would mean showing the client outdated state for longer, which is worse than showing it fewer updates.

Get this backwards — DropOldest on a chat feed, say — and you get subtly corrupted output (a client that sees "waves goodbye" without ever having seen "says hello") instead of a clean gap, which is a harder bug to notice in code review and a harder one to explain in an incident writeup.

A drop is not allowed to be silent​

Every drop, on either policy, runs through one function:

func (h *Hub) notifySendDropped(client *Client, data []byte) {
    h.logger.Warn("Client send buffer full, message dropped", "clientID", client.ID)
    h.metrics.IncrementMessagesDropped()

    if h.hooks.OnSendDropped != nil {
        h.hooks.OnSendDropped(client, data)
    }
}

Three separate outlets, and each earns its place:

• The metric (wshub_messages_dropped_total in the Prometheus subpackage) is what you graph and alert on. A drop rate that's zero for weeks and then climbs is a real signal — a client of yours changed behavior, or you're underprovisioned for current traffic — and it's invisible without this.
• The hook (OnSendDropped) is the extension point for a policy decision the library can't make for you: disconnect a client after N consecutive drops, downgrade them to a lower-fidelity message format, or page someone if drops start hitting a specific high-value client. This is where client.CloseWithCode earns its keep, called from inside the hook.
• The log line is for incident diagnosis — which client, roughly when.

Worth knowing before you rely on it in production: that log line is not rate-limited. It's a distinct code path from wshub's handshake-failure logging, which is explicitly rate-limited — because a failed handshake is something an unauthenticated client can trigger for free — while a message drop only happens for a client the hub has already accepted and is actively serving. That's a reasonable place to draw the line, but it also means a genuinely pathological slow client under sustained broadcast load can produce one Warn line per dropped message. If you're running at a scale where that's a real concern, the fix isn't in the log line — it's sampling or de-duplicating in your own OnSendDropped hook, which sees every drop before you decide whether it's worth another log entry.

What this buys you​

None of this changes the big picture from article one: bound the buffer, make the send non-blocking, pick a drop policy that matches your protocol's semantics, and make every drop observable. What changes with the code above is confidence that the mechanism is actually correct under concurrency, not just correct in the case your tests happened to exercise — a time-of-check-to-time-of-use race and a lost-update race are exactly the kind of bugs that pass every test you write single-threaded and show up three weeks into production, at 2 a.m., under real concurrent load.

go get github.com/KARTIKrocks/wshub

The full DropPolicy and hook reference is in the docs, and the send-path benchmarks referenced above are reproducible with go test -bench=. -benchmem ./... — methodology in the README.

This is article five in a series on production-grade WebSocket servers in Go — see article one for the full list of problems this series covers, and articles two, three, and four for connection scaling, backpressure, and multi-node delivery. This one walks through wshub's actual Drain/ Shutdown implementation and the specific races each piece of it defends against.

A state machine, not a bool​

A hub is in exactly one of three states — StateRunning, StateDraining, StateStopped — stored as a single atomic int32 rather than guarded by a mutex. That choice matters on the read side: Health() and Ready() are meant to answer instantly under real concurrent load (a Kubernetes probe hitting /readyz every few seconds shouldn't contend with connection-registration locks), so every state read is a lock-free atomic load, not a mutex acquisition.

The transition into draining uses a compare-and-swap, and the reason is concurrency, not style:

if !h.state.CompareAndSwap(int32(StateRunning), int32(StateDraining)) {
    current := h.State()
    if current == StateStopped {
        return nil // already shut down
    }
    // current == StateDraining: fall through to wait on the same signal.
} else {
    // We won the CAS — we are the drain initiator.
    // ...start the idle reaper, etc.
}

Drain is safe to call more than once, including concurrently, because exactly one caller wins the CAS and becomes "the initiator" — the one that logs the start, checks for the zero-clients fast path, and spawns the idle reaper. Every other caller, including one that arrives after draining has already started, just falls through to waiting on the same completion signal. Without the CAS, two goroutines calling Drain at the same time — plausible if both an HTTP preStop hook and a signal handler try to initiate shutdown — would both start their own idle reaper, wastefully and redundantly walking the same client list on separate tickers.

One completion signal, closed exactly once​

That completion signal is a channel, drainDone, and it has two independent writers: Drain itself closes it when every client has disconnected, and Shutdown closes it to force-unblock any Drain call still waiting — for instance if the drain timeout is about to expire and something needs to force the issue immediately. Closing a channel that's already closed panics, so both call sites go through the same sync.Once:

h.drainOnce.Do(func() { close(h.drainDone) })

This is a small detail with an easy failure mode if you write it yourself without a guard: exactly one of "drain finished naturally" or "shutdown forced it closed" will happen first in any given run, and which one wins is a race by construction — the whole point is that either path can trigger completion. Coordinating that through sync.Once rather than a manual "only close if not already closed" check with a boolean flag is what makes it correct under the race instead of merely correct in whichever order you happened to test.

The idle reaper: what "graceful" actually requires​

Here's the mechanism that turns "wait for clients to disconnect" from a hope into a bound. During drain, a background goroutine periodically checks every client's outbound send buffer:

for _, client := range snap.slice {
    if len(client.send) == 0 {
        if _, tracked := firstIdleAt[client]; !tracked {
            firstIdleAt[client] = now
        } else if now.Sub(firstIdleAt[client]) >= h.drainTimeout {
            _ = client.CloseWithCode(websocket.CloseGoingAway, "server draining")
            delete(firstIdleAt, client)
        }
    } else {
        delete(firstIdleAt, client) // has pending work — not idle
    }
}

Precision matters here: this measures the send buffer being continuously empty — "the server has had nothing left to tell this client for the full drain timeout" — not literally whether the client is responsive. That's a deliberate, narrower signal than "is this client still alive," and it's the right one for this job: during a drain you're not trying to detect a broken client, you're trying to bound how long you wait for ordinary, well-behaved connections that simply have no current reason to disconnect on their own. A chat client sitting on an open tab is not going to close its own socket just because your server would like to restart — proactively closing it with CloseGoingAway (1001), a real WebSocket close code, is what lets it distinguish "the server said goodbye" from a dropped connection and reconnect cleanly.

The check interval is derived from the timeout itself, floored so short timeouts (tests, aggressive rollout policies) stay responsive:

interval := max(h.drainTimeout/4, 100*time.Millisecond)

With the default 30-second drain timeout that's a 7.5-second tick, and the worst case costs two of them, not one: a connection that goes idle right after a tick can wait up to a full interval before firstIdleAt even gets recorded, and the close itself only fires on a later tick too — so the real bound on top of drainTimeout is up to 2 × interval (~15s here), not just one. That's a concrete, quantifiable bound you can reason about when setting your own preStop grace period, not an unspecified "eventually."

Shutdown: the part that actually forces connections closed​

Drain alone never force-closes an active connection — only idle ones past the timeout. Whatever's left when the context passed to Drain expires is Shutdown's job, and it doesn't loop over clients calling Close() individually. It cancels the hub's shared context:

h.state.Store(int32(StateStopped))
h.drainOnce.Do(func() { close(h.drainDone) }) // unblock any pending Drain
h.cancel()

That single cancel() is what tears down every remaining connection, and it works the same way article four's Redis adapter closes its subscription: a small watcher goroutine per client does nothing but wait on ctx.Done() and then call closeConn(), which is what actually unblocks the readPump's blocking ReadMessage() call. It's the same shape — a goroutine whose only job is to close something from outside the blocking loop that's holding it — showing up twice in the same codebase because it's the correct answer to the same underlying problem: you can't make a blocking call return early from inside itself, so something else has to close the resource it's blocked on.

One ordering detail in Shutdown is there specifically because of that same adapter mechanism:

// Close the adapter before waiting on goroutines — the subscriber
// goroutine may block on a channel read that only unblocks on close.
if h.adapter != nil {
    if err := h.adapter.Close(); err != nil {
        h.logger.Error("adapter close failed", "error", err)
    }
}

If Shutdown waited on its own WaitGroup first and closed the adapter after, it could deadlock against exactly the pattern described in article four: the adapter's receive goroutine is parked on sub.Channel(), which only unblocks when something calls sub.Close() — and that call lives inside the adapter's own Close(), not inside the hub. Closing the adapter first guarantees that call happens before anything waits for it to finish.

Everything after that is bookkeeping in the right order — the worker pool (if WithParallelBroadcast was configured) drains its remaining queued work before its own workers exit, then Shutdown waits on the hub's WaitGroup bounded by the passed context, returning nil on clean exit or the context's error if the deadline hits first — the same "select on completion vs. context" shape as Drain.

Putting it back together​

The two-phase call from article one now has a fuller explanation behind it:

ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

hub.Drain(ctx)    // stop new connections; idle ones get CloseGoingAway
                  // after drainTimeout; active ones finish on their own
hub.Shutdown(ctx) // cancel the shared context — force-closes whatever's left,
                  // closing the adapter first so its own goroutines can exit

Skipping Drain and calling Shutdown alone is a valid, different choice — every connection gets force-closed immediately via context cancellation, with no CloseGoingAway grace period. That's the right call for a genuine emergency stop; it's the wrong one for a routine rolling deploy, which is why the two are separate methods instead of one with a flag.

go get github.com/KARTIKrocks/wshub

Full reference for Drain, Shutdown, and WithDrainTimeout is in the Hub docs. That closes out this five-part series — from the eight problems a production WebSocket server has to solve, through connection scaling, backpressure, multi-node delivery, and now shutdown. The README has the full API surface and benchmark methodology for anything not covered along the way.

This is article two in a series on what "production-grade" means for a Go WebSocket server — the first one listed eight problems production traffic surfaces. This one goes deep on the first number everyone asks about, using wshub's own published load-test numbers as the concrete case.

Why the C10K framing is stale​

The original C10K problem (Dan Kegel, 1999) was about servers that spent one OS thread per connection. Ten thousand threads meant ten thousand ~1–8 MB stacks and a kernel scheduler context-switching between them — real resource pressure, independent of what the handler actually did.

Go doesn't have that problem by construction. A goroutine's stack starts at 2 KB and grows on demand; the runtime's own scheduler multiplexes goroutines onto a small, GOMAXPROCS-bound pool of OS threads. Critically, a goroutine blocked on conn.Read() or conn.Write() doesn't block an OS thread at all — the runtime's netpoller (backed by epoll on Linux, kqueue on BSD/macOS) parks it and reuses the thread for other goroutines until the socket is actually readable or writable. Ten thousand goroutines blocked on network I/O cost ten thousand parked stacks, not ten thousand blocked threads. This is what makes goroutine-per-connection — rather than an event-loop-with-callbacks architecture — a reasonable default in Go, when the same model was a liability in C or early Java.

So "can Go handle 10K connections" was already answered by the runtime. The real questions are narrower:

1. How much memory does each connection actually cost, end to end?
2. What OS-level limits sit between your process and 10,000 open sockets?
3. At what point does something other than connection count become the bottleneck?

The actual memory budget​

wshub's connection handler is exactly two long-lived goroutines per client — a readPump blocked in conn.ReadMessage(), and a writePump draining a buffered chan []byte and owning all writes to the socket (gorilla/websocket requires a single writer; concurrent writes from two goroutines is a data race the library doesn't protect against). The HTTP handler goroutine that performed the upgrade returns immediately after spawning those two — it does not stick around, so it's not part of the steady-state cost.

Per connection, that's:

• Two goroutine stacks (2 KB initial, growing only if a call frame needs it — in practice these pumps stay shallow).
• Two gorilla/websocket I/O buffers, 1 KB each by default (ReadBufferSize/WriteBufferSize in wshub's Config).
• A buffered send channel, SendChannelSize slots deep (default 256) — note this holds []byte references, not preallocated byte arrays, so it costs roughly 256 × 24 bytes of slice headers when full, not 256 KB.
• Bookkeeping: a map entry in the hub's client registry, connection metadata, and whatever callbacks are registered.

Measuring this in aggregate rather than adding up estimates is more honest, so here's what wshub's end-to-end load test actually measures (real gorilla/websocket dialer connections against a real httptest.Server, memory sampled via runtime.MemStats and divided by connection count):

Two things worth noticing. First, per-connection memory is flat across an order of magnitude of scale (~25–27 KB) — there's no hidden O(n) cost lurking in the client registry or the broadcast path that shows up only at higher connection counts. Second, handshake throughput actually improves with scale (15.7K → 36.9K conn/s), which is a load-test-harness artifact, not a server property: connecting more clients amortizes fixed startup costs (goroutine scheduling ramp-up, initial GC assist) over more connections. At 10,000 clients that's still only ~260 ms wall-clock to establish every connection.

Multiply 25.9 KB by however many connections you're planning for and you have a real memory floor: 100,000 idle connections is roughly 2.6 GB just for connection-holding, before your application logic allocates anything. That's the number to put in a capacity plan, not a guess.

The OS limits, in the order you'll hit them​

None of these are Go-specific, and all of them are checkable before you find out about them from an outage.

1. File descriptor limit (ulimit -n). Every WebSocket connection is one socket, and every socket is one file descriptor. Most Linux distributions default the per-process soft limit to 1024 — far below 10,000. Check it with ulimit -n, and raise it wherever the process actually starts: LimitNOFILE=65536 in a systemd unit, --ulimit nofile=65536:65536 for Docker, or ulimit -n 65536 before exec in a shell-launched process. Budget headroom above your target connection count for log files, outbound connections to Redis/NATS if you're using an adapter, and health-check sockets.

2. Listen backlog (net.core.somaxconn). This one only bites during a connection burst, not at steady state. The kernel queues completed TCP handshakes for accept() in a backlog whose size is capped by net.core.somaxconn (historically 128 on older kernels, higher on modern defaults); Go's net.Listen requests a backlog but the kernel silently caps it at this value. If 5,000 clients try to connect within the same handshake-timeout window — a mobile app coming back online after a network blip, a game client reconnect storm — and your backlog is smaller than the burst, the kernel starts dropping or refusing the excess before your accept loop ever sees them. This is exactly why wshub's own load-test client caps itself to 200 concurrent in-flight handshakes with a semaphore: not because the server can't take more, but because an unbounded connect burst measures the test harness's backlog behavior instead of the server's. In production, size somaxconn to your worst realistic reconnect-storm burst, not your steady-state connection count.

3. Ephemeral ports — but check which side actually needs them. This is where a lot of write-ups get it backwards. A listening server doesn't consume an ephemeral port per accepted connection — every inbound connection shares the same server-side (IP, port) pair; what makes each connection unique is the client's IP and port. The ephemeral port range (net.ipv4.ip_local_port_range on Linux, roughly 28,000–60,000 ports by default) only matters for the side initiating outbound connections: a load-generator opening 10,000 client sockets from one machine, or your server's own outbound connections to a Redis/NATS adapter. If you're only accepting inbound WebSocket connections, this limit is irrelevant to your connection ceiling — don't spend time tuning it based on advice written for a different topology.

4. GOMAXPROCS and the scheduler, not connection count. wshub's load test header prints 12 cores on the machine these numbers came from — this matters for dispatch work, not for holding connections open. A goroutine parked on a socket read costs no CPU; a goroutine actively processing a message does. The next section is where this actually shows up.

Where 10,000 stops being "just a number"​

Holding 10,000 idle connections open is close to free. Doing something with all of them concurrently is a different problem, and it's where wshub's own benchmarks show a real cliff — not in connection count, but in sustained fanout:

Throughput keeps climbing, but p50 latency jumps almost three orders of magnitude between 5,000 and 10,000 clients under sustained broadcast load (100 msg/s from a single broadcaster, 10-second window). The documented explanation is scheduler pressure: every broadcast at this scale is pushing into thousands of buffered channels, and reading off each of those channels and writing to each socket is real, concurrently-scheduled CPU and syscall work — competing for the same GOMAXPROCS-bound thread pool that every other goroutine in the process shares, including the read side of the load-test harness itself. The hub's own dispatch loop stays allocation-free at this scale (the in-process broadcast micro-benchmarks confirm that part in isolation); the cost is downstream of dispatch, in the fan of individual socket writes.

The practical implication: "10,000 connections" and "10,000 connections all receiving a broadcast every 10ms" are different capacity questions, and the second one is the one that actually constrains a single node. wshub exposes three knobs for this, in the order I'd reach for them:

• SendChannelSize — a deeper per-client buffer absorbs short bursts without invoking the drop policy, at the cost of more memory per connection and, if actually filled, more end-to-end latency for whatever's sitting in the queue.
• WithCoalesceWrites(true) — batches queued text messages into a single WebSocket frame (newline-separated) before they hit the socket, trading a small amount of client-side parsing for fewer syscalls per connection under high message rates.
• Horizontal scaling — past a real per-node ceiling, add nodes behind a Redis or NATS adapter rather than tuning a single process further. That's its own set of correctness problems (node-ID deduplication, not blocking local delivery on the bus) — enough for the next article in this series.

One knob that's not on this list: WithParallelBroadcast. It exists for backward compatibility, but wshub's own load tests found it consistently slower than the default serial broadcast in practice — the per-call lock/defer overhead of parallel dispatch doesn't pay for itself against real sockets, even though it looks like it should help on paper. Measure before reaching for parallelism; the bottleneck here isn't the dispatch loop.

The checklist​

Getting a Go process to comfortably hold 10,000 WebSocket connections is not a performance-engineering project. It's five checks:

1. ulimit -n raised above your target connection count plus headroom, at the layer that actually launches the process (systemd/Docker/shell).
2. net.core.somaxconn sized to your worst realistic reconnect burst, not your steady-state count.
3. Memory budget = connection count × ~26 KB (adjust for your own SendChannelSize/buffer configuration) — a real number for capacity planning, not a guess.
4. Don't tune ephemeral ports on the server unless it's also making 10,000 outbound connections — that limit belongs to whoever dials, not whoever listens.
5. Benchmark fanout, not just connect — a server holding 10,000 idle connections and one broadcasting to all of them every 10ms are different capacity questions, and the second is where you'll actually find your ceiling.

Every default referenced here — buffer sizes, channel depth, coalescing, drop policy — is a wshub.Config or Hub option, not something you have to reimplement to test for yourself:

go get github.com/KARTIKrocks/wshub

Reproduce the numbers in this post directly:

make loadtest LOADTEST_ARGS="-scenario connect -clients 10000"
make loadtest LOADTEST_ARGS="-scenario fanout -clients 10000"

Full configuration reference is in the docs, and the complete benchmark methodology is in the README.

It isn't. The hard part starts at the traffic your demo never saw: a client that stops reading, a deploy that needs to happen without dropping connections, a second replica that has no idea the first one exists. None of that is in the WebSocket RFC, and none of it is optional once real users are on the other end of the socket.

This post is the list of problems I hit, in the order production traffic surfaces them, and what each one actually requires — not "add a mutex," but the specific mechanism and the tradeoff it forces. I ended up packaging the result as an open-source library, wshub, but the problems here apply whether you use it or write your own.

1. The connection lifecycle is two goroutines, not one​

A net.Conn is full-duplex, but gorilla/websocket's *Conn is not safe for concurrent writes from multiple goroutines — only one writer at a time, ever. That single constraint decides the shape of every connection you manage:

• One readPump goroutine blocks on conn.ReadMessage() and feeds inbound frames to your handler.
• One writePump goroutine owns the connection exclusively for writes, draining a per-client buffered channel (chan []byte) and calling conn.WriteMessage itself.

Every other part of the system — your HTTP handlers, your business logic, another goroutine reacting to a Redis message — writes to that channel and never touches the socket directly. That single rule (only the writePump writes) is what makes the rest of this list tractable. Skip it and you'll eventually get a data race the race detector won't catch in dev, because it only reproduces under concurrent load.

The cost: three long-lived goroutines per connection (readPump, writePump, plus whatever handles the HTTP upgrade), each with its own stack. At 10,000 connections that's the dominant driver of memory per connection — roughly 25–27 KB in practice, most of it goroutine stacks and channel buffers, not the socket itself.

2. Concurrent access to the client map​

The hub needs a map[string]*Client for O(1) lookup and a way to iterate it for broadcast — and both happen from different goroutines constantly. sync.RWMutex is the obvious answer, but the naive version has a trap:

// Don't do this — the lock is held for the entire broadcast
func (h *Hub) Broadcast(data []byte) {
    h.mu.RLock()
    defer h.mu.RUnlock()
    for _, c := range h.clients {
        c.send <- data // blocks here if a client's channel is full
    }
}

If one client's send channel is full, this blocks the whole broadcast — and since it's holding an RLock, it blocks every other goroutine trying to register or deregister a client, too. One slow client now stalls the entire hub. The fix is to take a snapshot of the client slice under the lock, then release the lock before touching any channel:

func (h *Hub) Broadcast(data []byte) {
    h.mu.RLock()
    snapshot := h.clientSnapshot() // copy the slice, still under RLock
    h.mu.RUnlock()

    for _, c := range snapshot {
        c.trySend(data) // non-blocking; see backpressure below
    }
}

This is the difference between a broadcast that's O(1) allocations and lock contention that scales with client count, versus one that's actually lock-free for the read path. Measured with mock clients: broadcasting to 100,000 clients takes ~22.6 ms with zero allocations; to 1,000,000 clients, ~269 ms, still zero allocations. Targeted sends (SendToClient) are O(1) regardless of hub size — ~105 ns whether there are 100K or 1M clients on the hub, because they go straight through the map, never through the snapshot.

3. Broadcast is not one primitive — it's four​

"Send this to everyone" is the easy case. Production traffic needs:

• Broadcast — everyone.
• Broadcast to a room — a scoped subset (a chat channel, a game instance, a document's collaborators).
• Broadcast except — everyone but the sender, so you don't echo a user's own message back to them.
• Send to user — not a connection, a user, who may have three tabs open and three separate *Clients. This needs its own index (map[userID][]*Client) alongside the connection map, because a user reconnecting on a fourth device shouldn't require touching every existing connection's metadata.

Each of these has to preserve the same non-blocking guarantee as plain broadcast. BroadcastExcept in particular is tempting to implement as "broadcast, then skip in the loop" — which is exactly what you want, as long as the exclusion check doesn't force an allocation per call. Done right, it costs one allocation per call (for the exclusion set), not per client.

4. Backpressure: the problem nobody's demo has​

Here's the scenario every WebSocket server eventually hits: one client is on a flaky mobile connection, or its own event loop is blocked, or it just stopped reading. The server keeps trying to write. What happens?

If you write directly to the socket, that goroutine now blocks on a platform TCP buffer that never drains, and depending on how you built the send path, this can propagate all the way back to the broadcast loop — the "one slow client stalls everyone" bug again, this time in the OS, not your mutex.

The mitigation is a bounded per-client channel with an explicit drop policy when it's full:

func (c *Client) trySend(data []byte) {
    select {
    case c.send <- data:
    default:
        // buffer full — apply policy instead of blocking
        c.applyDropPolicy(data)
    }
}

There are two reasonable policies, and the right one depends on what the message means:

• Drop newest (default) — discard the message that just arrived. Correct for append-only streams like chat, where losing the latest of several queued messages is recoverable.
• Drop oldest — evict the head of the queue to make room. Correct for state-replacement streams — a live cursor position, a stock ticker, a presence indicator — where only the latest value matters and an old one is actively wrong to deliver.

Either way, silently dropping data is a bug waiting to be discovered in production unless it's observable. A drop hook — OnSendDropped(client, data) — that increments a metric or logs the client ID is not optional; it's the difference between "we tuned the buffer size after seeing drop-rate alerts" and "we found out during a user complaint three weeks later."

5. Graceful shutdown is two phases, not one​

os.Signal handling for SIGTERM is the easy half. The question that matters is: what happens to the 8,000 connections currently open when the pod gets killed for a routine deploy?

Closing them all immediately is a correctness bug disguised as a shutdown routine — every one of those clients gets a hard disconnect and has to reconnect, all at once, which is its own thundering-herd problem against whatever server picks them up next.

The pattern that actually works is two distinct phases:

ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

hub.Drain(ctx)    // phase 1: stop accepting new connections (HTTP 503),
                  // let existing ones finish naturally
hub.Shutdown(ctx) // phase 2: force-close anything still open

During drain, new upgrade attempts get rejected, but existing connections keep working normally — they finish their in-flight request/response cycles and disconnect on their own terms. Connections that go idle during this window (no traffic for the drain timeout, default 30s) get proactively closed with a proper CloseGoingAway (1001) frame rather than left hanging until the hard deadline. Only what's left after that gets force-closed in phase two. This is what makes a rolling deploy invisible to end users instead of a visible blip every release.

6. Rate limiting protects the hub from its own clients​

Connection-level backpressure protects against slow readers. It does nothing against a client — malicious or just buggy — that opens connections faster than you can process them, or floods messages at a rate no downstream consumer can absorb. That needs limits enforced before the expensive part of the pipeline runs:

• Max total connections — a hard ceiling on hub size, so a runaway client can't exhaust file descriptors for everyone else.
• Max connections per user — bounds the blast radius of one compromised or misbehaving account, and catches the "client library retries without backoff" bug before it becomes an incident.
• Max rooms per client — the room-membership equivalent; without it, one client subscribing to every room turns every room broadcast into a full broadcast.

The critical detail is when the user-ID limit gets checked. If a client can exist unauthenticated and get a user ID assigned afterward, there's a window where MaxConnectionsPerUser simply doesn't apply to it. Assigning the user ID atomically as part of the upgrade — not as a second step after the connection is already registered — closes that window.

7. You can't operate what you can't see​

Every mechanism above — backpressure drops, drain progress, rate-limit rejections — is invisible unless something is counting it. A production WebSocket service needs, at minimum:

• A metrics interface your monitoring stack can plug into (connection count, messages sent/dropped, handler latency), kept as an interface rather than a hard Prometheus dependency, so libraries that don't want Prometheus in their dependency tree aren't forced to have it.
• /healthz and /readyz as distinct signals. Liveness answers "is the process alive"; readiness answers "should the load balancer route new connections here" — and during a drain, those two answers correctly diverge: alive, but not ready.
• Lifecycle hooks (OnConnect, OnDisconnect, OnSendDropped, BeforeDisconnect) as the seam where you attach logging, auditing, or cleanup, without reaching into the hub's internals to do it.

None of this is glamorous, and all of it is what turns a 3 a.m. page into a five-minute diagnosis instead of a two-hour one.

8. One process is not enough — and pub/sub isn't optional​

Eventually one machine's connection count or CPU is the ceiling, and you need a second node. This is where a WebSocket server's statefulness bites: unlike a stateless HTTP API, SendToUser("alice", ...) only works if this process holds Alice's connection. If your load balancer routed her to node B and the message originated on node A, a naive multi-node deployment silently drops it.

The fix is a message bus between nodes — Redis pub/sub or NATS are the common choices — behind a small adapter interface:

type Adapter interface {
    Publish(ctx context.Context, msg AdapterMessage) error
    Subscribe(ctx context.Context, handler func(AdapterMessage)) error
    Close() error
}

Every broadcast and targeted send publishes to the bus in addition to delivering locally, and every node subscribes and re-delivers to whichever of its own local connections match the target. Two details make this correct rather than merely functional:

• Node-ID deduplication. Without tagging each published message with its originating node, a broadcast fans out once locally and once again when the node receives its own message back off the bus.
• Local delivery must never block on the adapter. If Redis is slow or down, clients on a healthy node should keep working. The adapter publish path degrades independently from the local dispatch path — an adapter outage should never become a local outage.

If you also need cluster-wide counts (GlobalClientCount, GlobalRoomCount) rather than per-node numbers, that's a separate periodic presence-gossip mechanism, not something adapters give you for free — and it's worth keeping optional, since not every deployment needs it and gossip has its own cost.

What this adds up to​

None of these eight problems is individually hard. What makes a WebSocket server "production-grade" is that all eight have to be solved together, and several of them interact — backpressure policy changes what a slow client during drain looks like; rate limiting has to compose with the upgrade path's atomicity; the adapter layer has to respect the same non-blocking guarantee as local broadcast. Solve them one at a time in isolation and you'll find the seams later, under load, at the worst time.

I built these ideas into an open-source Go package called wshub — rooms, backpressure with configurable drop policies, two-phase drain/shutdown, connection and rate limits, a pluggable metrics interface with an official Prometheus subpackage, and Redis/NATS adapters for horizontal scaling, all benchmarked and MIT-licensed. It's built on top of gorilla/websocket, not a replacement for it — the protocol handling was never the hard part.

go get github.com/KARTIKrocks/wshub

If you're currently maintaining a hand-rolled Hub struct with a map[*Client]bool and a broadcast chan []byte, most of what's above is probably already on your TODO list. The documentation has the full API, and the README's benchmark section has the methodology behind every number quoted here.

This is article four in a series on production-grade WebSocket servers in Go — see article one for the full list of problems, and article two and article three for connection-scaling and backpressure. This one is about the specific correctness rules a message bus has to satisfy before "just add Redis" is actually a fix rather than a new set of bugs — using wshub's own Redis adapter as the worked example.

The interface has to be this small​

Scaling out means every broadcast, every targeted send, needs a second path alongside local delivery: publish it somewhere every other node is listening. The interface for that "somewhere" is deliberately three methods:

type Adapter interface {
    Publish(ctx context.Context, msg AdapterMessage) error
    Subscribe(ctx context.Context, handler func(AdapterMessage)) error
    Close() error
}

Small enough that Redis Pub/Sub, NATS, or a hand-rolled bus are all one afternoon's implementation — the interface makes no assumption about delivery guarantees, ordering, or persistence beyond "fire this at every other node." That minimalism is deliberate: the correctness rules below live in how the hub uses this interface, not in the interface itself, so any transport that satisfies these three methods inherits them for free.

Rule 1: local delivery never waits on the bus​

Every broadcast method does two things, in this order:

func (h *Hub) Broadcast(data []byte) {
    h.broadcast(sendItem{msgType: websocket.TextMessage, data: data})
    h.publishToAdapter(AdapterMessage{
        Type:    AdapterBroadcast,
        MsgType: websocket.TextMessage,
        Data:    data,
    })
}

Local delivery happens first, synchronously, and unconditionally. Publishing to the adapter happens second, and a failure there is logged and counted — never propagated back to break local delivery:

func (h *Hub) publishToAdapter(msg AdapterMessage) {
    if h.adapter == nil {
        return
    }
    msg.NodeID = h.nodeID
    if err := h.adapter.Publish(h.ctx, msg); err != nil {
        h.logger.Error("adapter publish failed", "error", err, "type", msg.Type)
        h.metrics.IncrementErrors("adapter_publish")
    }
}

This ordering is the whole point. If Redis is down, degraded, or just slow, every node's local clients keep working normally — they simply stop receiving updates that originated on other nodes, which is a partial, recoverable degradation instead of every WebSocket connection in the fleet going down because the message bus had a bad five minutes. A message bus that can take down local delivery when it fails isn't actually decoupled from it.

Rule 2: every node has to ignore its own echo​

Here's the bug you'd write without thinking about it twice: Node A broadcasts. Node A delivers locally and publishes to Redis. Every node subscribed to that Redis channel — including Node A itself — receives the message back. Without a check, Node A now delivers the same broadcast to its own clients twice.

The fix is a single equality check on receive:

func (h *Hub) handleAdapterMessage(msg AdapterMessage) {
    // Ignore messages originating from this node.
    if msg.NodeID == h.nodeID {
        return
    }
    // ... dispatch locally, and never re-publish.
}

msg.NodeID is stamped by the publishing node in publishToAdapter above, not read back from the transport — Redis Pub/Sub doesn't tell you who published a message, so the dedup key has to travel inside the payload. The second half of that comment — "never re-publish" — matters just as much as the dedup check: handleAdapterMessage calls the same local-delivery functions the direct API does (broadcast, sendToUserLocal, and so on), never publishToAdapter. Skip that distinction and a two-node cluster turns into an infinite relay: A publishes, B receives and re-publishes, A receives B's re-publish and re-publishes again, forever.

Rule 3: closing the subscription correctly is harder than opening it​

The Redis adapter's Subscribe spawns two goroutines, not one, and the reason is specific to how go-redis's PubSub type manages its own lifetime:

// Closing the PubSub is what closes the channel the receive loop ranges
// over, so it has to happen from outside that loop — a `defer sub.Close()`
// on the receive goroutine can only run once the loop has already exited,
// which never happens. go-redis does not tie the PubSub's lifetime to the
// context passed to Subscribe, so this watcher is also what makes context
// cancellation stop delivery.
go func() {
    defer a.wg.Done()
    <-ctx.Done()
    _ = sub.Close()
}()

go func() {
    defer a.wg.Done()
    ch := sub.Channel()
    for msg := range ch {
        // ...deserialize and dispatch
    }
}()

The receive loop's exit condition is "the channel closes," and the channel only closes when something calls sub.Close() — so a naive single-goroutine implementation that tries defer sub.Close() around the for range loop deadlocks forever, because the deferred call can't run until the loop it's supposed to unblock has already exited. A second, dedicated goroutine whose only job is watching the context and closing the subscription from outside the loop is what actually makes cancellation work.

There's a second, quieter race in the same function, in how the two goroutines get counted:

a.mu.Lock()
// ...
a.wg.Add(2)
a.mu.Unlock()

// ... (goroutines started here, after the lock is released)

wg.Add(2) happens before the lock is released and before either goroutine actually starts — not after. Go's own sync.WaitGroup documentation calls this out directly: calls to Add that increment the counter from zero must happen before a Wait that could observe it. If Add happened after starting the goroutines instead, Close could call wg.Wait() in the gap between the goroutines being scheduled and the Add call actually running, see a zero counter, and return immediately — reporting the adapter closed while both goroutines are still mid-startup. Reserving both slots synchronously, before releasing the lock, closes that window entirely.

What "scaling" actually looks like end to end​

Put together, two wshub processes sharing one Redis instance behave like this: a client connects to whichever node its load balancer routes it to; WithAdapter(redisAdapter) and a stable WithNodeID are the only configuration difference from a single-node setup; every broadcast and targeted send reaches every connected client on both nodes, with each message delivered exactly once per client no matter which node it landed on. wshub ships this as a runnable example, not just a diagram — examples/multinode starts two nodes against one Redis container and exposes /stats on each, printing both local clients (hub.ClientCount()) and global clients (hub.GlobalClientCount()) side by side, so the split is visible instead of asserted.

That GlobalClientCount number comes from a separate mechanism — presence gossip, opt-in via WithPresence(interval) — not from the adapter itself. Every tick, each node publishes its own local client and room counts as an AdapterPresence message; every other node caches the latest report per node ID and evicts it if the reporting node goes quiet for three missed ticks (presenceTTL defaults to 3 × interval — WithPresence(5 * time.Second) means a node that stops reporting for 15 seconds silently drops out of everyone else's global count). This is intentionally decoupled from message delivery: you can run multi-node broadcast without presence at all, and add cluster-wide counts later without touching how messages move.

Try it​

go get github.com/KARTIKrocks/wshub/adapter/redis

rdb := goredis.NewClient(&goredis.Options{Addr: "localhost:6379"})
adapter := redis.New(rdb)

hub := wshub.NewHub(
    wshub.WithAdapter(adapter),
    wshub.WithNodeID("node-a"),
    wshub.WithPresence(5*time.Second),
)
go hub.Run()

Run the two-node demo directly:

cd examples/multinode && go run .

The Adapters guide covers the full interface and the NATS adapter for anyone not already running Redis; the Presence guide covers the gossip mechanism in more depth. Next in this series: what it takes to take one of these nodes out of the cluster without dropping a single in-flight connection.