How do you optimize Go performance: profiling, GC, allocations?

Long Answer

Great Go performance comes from a tight loop: measure → understand → change → re-measure. I treat CPU time and allocation rate as first-class metrics, then tune garbage collection only after reducing memory churn.

1) Measure first: CPU, memory, and latency
Start with production-realistic loads. Embed net/http/pprof or capture offline profiles with:

• CPU profile: pprof or go tool pprof on binaries/HTTP endpoints to find top stacks (inclusive/exclusive time).
  
• Heap/alloc profiles: identify allocation hotspots and object types; check bytes allocated per operation.
  
• Block/Mutex profiles: reveal goroutine contention and lock hotspots.
  
• runtime/trace: visualize scheduling, network/GC assists, and long stop-the-world (STW) pauses.
  Always pair with benchmarks (testing.B, -bench, -benchtime) and pprof diff to validate wins.
  

2) Fix the big rocks: algorithmic and I/O
Before micro-tuning:

• Replace N² algorithms, unnecessary JSON round-trips, or chatty I/O with batching/streaming (bufio.Reader/Writer, io.Copy).
  
• Cache derived data that is expensive but stable.
  
• Use pipelining and concurrency where it reduces critical path latency (measure context switches vs gain).
  

3) Reduce allocations (lower GC pressure)
Most Go slowdowns trace to allocation churn:

• Escape analysis: run go build -gcflags=all=-m=2 to see “escapes to heap”. Prefer value types; avoid taking addresses of locals; keep small structs inlined.
  
• Preallocate: make([]T, 0, n) and make(map[K]V, hint) to avoid growth copies and rehashing.
  
• Reuse buffers: bytes.Buffer, strings.Builder (for strings), and sync.Pool for short-lived, same-size objects. Pool carefully (amortize, but do not hoard).
  
• Avoid conversions: minimize []byte↔string copies; use strings.Builder or keep data as []byte through the pipeline when possible.
  
• Streaming parsers: use json.Decoder/Encoder or a faster codec with reuse; avoid json.Marshal into temporary []byte if you can stream to io.Writer.
  
• Zero-copy slices: slice instead of copy when safe; be explicit about lifetimes to avoid keeping giant backing arrays alive.
  

4) Tune garbage collection only after reducing churn
Go’s concurrent mark-sweep GC scales well when the live heap is small and allocation rate is sane.

• Targets: watch GODEBUG=gctrace=1 (or metrics) for heap size, assist ratios, and pause times (P50/P99).
  
• GOGC / debug.SetGCPercent: raises/lowers heap growth target (e.g., GOGC=100 default; higher reduces GC frequency but increases memory).
  
• GOMEMLIMIT / debug.SetMemoryLimit: cap total memory to keep processes inside container limits; GC will pace to the limit.
  
• Shave roots: reduce pointer-rich structures (use arrays over pointer-linked lists; compact representations like []uint32 for IDs).
  
• Object sizing: fewer, larger objects can reduce marking overhead, but do not inflate live set unnecessarily.
  

5) Concurrency and contention

• Replace coarse locks with sharding ([256]sync.Mutex or sync.Map for read-heavy, low-write cases).
  
• Prefer channel pipelines where they simplify flow; avoid over-buffering that inflates memory.
  
• Profile block/mutex waits; a faster critical section often beats exotic lock designs.
  

6) HTTP, DB, and serialization hot paths

• HTTP: enable http.Server timeouts; reuse connections (Transport pooling); compress conditionally; avoid per-request allocations in middleware.
  
• DB: batch queries, reuse prepared statements, tune pool sizes; scan into preallocated structs.
  
• Serialization: consider faster codecs (e.g., msgpack) when JSON dominates CPU; keep allocs low with reusable decoders/encoders.
  

7) Validate with disciplined benchmarking

• Microbenchmarks around hotspot funcs; table-driven benchmarks with realistic sizes.
  
• -benchmem to track allocs/op; regressions fail CI.
  
• Use pprof diff (before/after) and trace to ensure no hidden tail latencies.
  

8) Production hardening

• Export runtime metrics: GC cycles, pause totals, heap/live objects, goroutines, sched latency.
  
• Canary and A/B deploy optimizations; track P50/P95/P99 latency, throughput, and RSS.
  
• Keep flags configurable: GOGC, GOMEMLIMIT, pool sizes; document safe ranges.
  
  

The rule of thumb: cut allocations first, then let GC do less work, and only then tweak GC knobs. Always verify changes with profiles and workloads that mirror production.

‍

Table

‍

Common Mistakes

• “Tuning GC first” without cutting allocation rate.
  
• Ignoring escape analysis; passing pointers around needlessly.
  
• Growing slices/maps implicitly (no cap/hint) causing reallocation churn.
  
• Overusing sync.Pool (pooling tiny/rare objects) or retaining large pooled buffers too long.
  
• Excessive []byte↔string conversions and JSON buffering.
  
• Blind concurrency: goroutine spam or coarse locks that add contention.
  
• Trusting microbenchmarks that do not match production sizes or data shapes.
  
• Shipping changes without pprof/trace validation and P99 latency tracking.
  
  

Sample Answers

Junior:
“I start with pprof to see where CPU time goes and use -benchmem to reduce allocations. I preallocate slices, avoid sync APIs, and switch to json.Decoder to stream. If pauses are high after that, I adjust GOGC slightly and re-measure.”

Mid-level:
“I run CPU/heap and block profiles under realistic load, then fix escapes (-m=2), reuse buffers with bytes.Buffer and sync.Pool, and add capacities to slices/maps. I cap memory with GOMEMLIMIT in containers and tune GOGC to balance RSS and pause time. CI runs pprof diffs and benchmarks.”

Senior:
“My method is measure-first: pprof + trace to separate CPU, alloc, and contention. I cut allocation rate (value semantics, prealloc, streaming), redesign hot paths, then set GOGC and GOMEMLIMIT based on observed pause/heap curves. I shard locks, batch I/O, and export GC/runtime metrics; optimizations ship behind canaries and fail CI on alloc/latency regressions.”

‍

Evaluation Criteria

Strong answers show:

• Profile-driven workflow (CPU/heap/block/mutex, trace).
  
• Concrete allocation reduction tactics (escape fixes, prealloc, buffer reuse, fewer conversions).
  
• Correct GC tuning order (optimize churn first, then GOGC/GOMEMLIMIT).
  
• Awareness of I/O and JSON streaming and connection pooling.
  
• Validation via benchmarks, diffs, and production metrics (P95/P99, RSS).
  Red flags: guessing, GC-only tweaks, premature sync.Pool everywhere, or ignoring contention and I/O.
  
  

Preparation Tips

• Practice net/http/pprof and go tool pprof (top, list, web, peek).
  
• Use -gcflags=all=-m=2 to study escapes; refactor to keep values on stack.
  
• Write testing.B benchmarks with -benchmem and realistic sizes.
  
• Compare buffered vs unbuffered I/O; try json.Decoder vs Marshal.
  
• Experiment with GOGC and GOMEMLIMIT; watch gctrace output.
  
• Capture a runtime/trace and read scheduler/GC timelines.
  
• Build a checklist: prealloc, reuse, avoid copies, shard locks, stream I/O.
  
  

Real-world Context

A telemetry API spent 40% CPU in JSON + allocated 50KB/op. Switching to streaming json.Decoder, reusing buffers, and eliminating []byte↔string copies cut allocations 6× and CPU 30%. A payment service hit OOM in containers: after lowering allocation churn and setting GOMEMLIMIT, GC stabilized and RSS stayed within limits, improving P99 latency by 35%. Another service stalled on mutexes; block profiles revealed a global map—sharding the map and preallocating capacity doubled throughput.

‍

Key Takeaways

• Measure first with pprof and trace; optimize evidence, not hunches.
  
• Reduce allocations (fix escapes, prealloc, buffer reuse) to lower GC work.
  
• Tune GOGC/GOMEMLIMIT only after cutting churn; verify pauses/RSS.
  
• Stream I/O and JSON, batch, and pool judiciously.
  
• Validate with benchmarks + pprof diff and watch P95/P99 + memory in prod.
  
  

Practice Exercise

Scenario:
Your Go API ingests large JSON payloads and spikes CPU/RSS under load in Kubernetes.

Tasks:

1. Add net/http/pprof and capture CPU, heap, and block profiles at peak.
   
2. Run go build -gcflags=all=-m=2; refactor hot functions to avoid escapes and keep values on the stack.
   
3. Replace buffered Marshal calls with streaming json.Decoder/Encoder; introduce bufio for network/file I/O.
   
4. Preallocate slices/maps using known sizes; rewrite loops to avoid intermediate slices.
   
5. Introduce buffer reuse (bytes.Buffer, sync.Pool) for per-request scratch space; verify no long-lived references.
   
6. Set an initial GOMEMLIMIT that matches pod limits; adjust GOGC ±25% based on gctrace pause/heap behavior.
   
7. Create testing.B benchmarks with production-sized payloads; run -benchmem. Compare pprof diffs before/after.
   
8. Canary deploy; monitor P95/P99 latency, allocations/op, GC pause totals, and RSS.
   
   

Deliverable:
A short report with profiles, code diffs, benchmark tables, and production graphs demonstrating reduced allocations, stable GC, and improved latency.