Go structured logging: data races and leaking context fields

TL;DR: Deriving a per-request logger from a shared base is the standard Go pattern, and in several widely used logging libraries it hands two goroutines the same underlying map. Sometimes that's a concurrent map writes panic. More often it's quieter: one request's trace ID written onto another request's log line, with nothing malformed to notice. PainHunt's DevTools data carries 78 signals in this cluster, mostly filed as GitHub issues. The wedge isn't another logger — it's making the failure visible before it ships.

The evidence

PainHunt's DevTools category holds 4,013 high-scoring signals (10+/15). Within it, 78 form a concurrency-and-logging cluster, average intensity 7.5/10, average score 11.7, sourced overwhelmingly from GitHub (52), then Medium (6), Discourse (4), HackerNews (4) and Dev.to (3).

That source mix matters. GitHub dominance means these arrive as reproductions and stack traces filed against a project, not as opinions in a thread. What they describe:

• Data races causing Go runtime panics when multiple goroutines use Entry.WithContext concurrently — surfacing as fatal error: concurrent map writes.
• Field leakage between goroutines — context-specific fields like trace IDs, request IDs and user IDs leak from one goroutine's context into another's log entries.
• A shared map reference in the derived entry causes mutations on a child entry to reach back into the parent.

The three are one bug wearing different clothes. The derived entry doesn't own its fields; it borrows them.

Why now

Three things converged.

The per-request logger became the default shape. Distributed tracing made it normal to attach a trace ID, span ID, request ID and user ID to a logger at the edge of a handler and pass it down. That means the derive-a-logger call now happens on every request, on every goroutine, in the hot path — where five years ago it happened once at startup.

Go's concurrency model makes the sharing invisible at the call site. logger.WithContext(ctx) reads like it returns a new thing. Whether it copies the field map or aliases it is an implementation detail of the library, and nothing in the type system distinguishes the two.

And the mature libraries can't simply fix it. Copy-on-derive means an allocation on every log call, in libraries whose entire pitch is allocation counts. That's a real trade-off, not an oversight — which is why the issues stay open.

The wedge

The gap is detection, not another logger.

• A go vet-style check for entries crossing goroutine boundaries. Flag a logger entry derived outside a goroutine and used inside it, or captured by a closure passed to go. This is the specific taint path behind all three symptoms, and it's narrow enough to keep false positives survivable.
• A race-detector harness aimed at logging specifically. -race already finds these, but only if a test happens to exercise concurrent logging on a shared base. A generated test that derives N entries from one base across N goroutines turns "we might have this" into a red build.
• A leak assertion for tests. Assert that a log line carries exactly the fields its own context set — nothing more. Field leakage produces well-formed output, so it survives every existing check; this is the only kind of assertion that catches it.
• A strict-mode wrapper rather than a replacement. Copy-on-derive, opt in per service, so a team can pay the allocation where correctness matters (audit logs, anything with a user ID) and skip it where it doesn't. Easier to adopt than migrating a logger, and it doesn't ask anyone to abandon a library they trust.

Risks and honest caveats

• The obvious buyer may not pay. These issues are filed by maintainers and engineers against free, mature, well-loved libraries. The person who needs this most has the smallest budget. A paying customer is more likely a platform team that has already been burned by a wrong trace ID during an incident.
• The right fix might belong upstream. A well-argued patch to an existing logger helps far more people than a new tool, and correspondingly makes a much weaker business. Worth being honest about that before building.
• Static analysis lives or dies on false positives. Loose enough to catch closure captures will fire on safe code; tight enough to stay quiet will miss the interesting cases. That trade-off is the product, and having the idea doesn't solve it.
• 78 signals is a live pattern, not a market size. It says the bug is real and being filed. It says nothing about how many teams would install something to prevent it.
• -race already exists. Anyone disciplined about running it under concurrent tests is largely covered. The wedge only pays off if you're honest that you're selling the harness, not the detection.

How to validate this further

Read the DevTools signals in the Pain Point Browser and check how many of the GitHub reports ended in a merged fix versus a still-open issue — that ratio tells you whether this is a tooling gap or a patience gap. For an adjacent security-tooling wedge with the same maintainer-as-reporter dynamic, see JWT kid header path traversal and self-hosted CVE exposure response. Then pressure-test the buyer question — the real risk here — with the Idea Validator and how to validate a startup idea.