How do you design and test REST or gRPC APIs in Go?

Long Answer

A production-grade Go REST or gRPC API blends clear contracts, safe concurrency, predictable middleware, rigorous validation, and dependable testing. The goal is correctness under load, graceful failure, and maintainable code that is easy to reason about.

1) Contracts and structure

Begin with a contract-first approach: OpenAPI for REST or protobuf for gRPC. Generate clients and server stubs. Organize code into layers: transport (HTTP or gRPC), service (business rules), and storage (repository interfaces). Keep handlers thin and move logic to services so tests stay focused and fast.

2) Input validation and error semantics

Validate at the boundary. For REST, bind JSON to typed DTOs and validate with tag-based validators; for gRPC, validate request messages in interceptors or service methods. Enforce strict types, required fields, ranges, and formats. Return precise errors: HTTP uses structured problem responses with stable error codes; gRPC uses status codes with typed details. Reject unknown fields to prevent silent drift. Normalize time to UTC and validate pagination, sorting, and filters.

3) Middleware and interceptors

Standardize cross-cutting concerns with middleware (REST) or interceptors (gRPC). Common layers include request and correlation identifiers, authentication and authorization, input size limits, rate limiting, timeouts and deadlines, structured logging, metrics, and tracing. Apply idempotency keys to create/update endpoints susceptible to retries. Ensure every handler reads ctx for deadlines and cancellations.

4) Concurrency handling and resource safety

Use Go’s strengths safely. For parallel operations, prefer small, composable workers governed by contexts, semaphores, and rate limiters. Guard shared state with channels or immutable snapshots; avoid unprotected global variables. For long-running work, delegate to background workers that honor shutdown signals and backpressure. Use errgroup to fan out with aggregate error handling and early cancellation. Bound all pools to protect memory and downstream services.

5) Timeouts, retries, and idempotency

Configure server timeouts (read, write, idle). For clients, use circuit breakers and capped exponential backoff with jitter. Make write paths idempotent where possible to tolerate retries. Store deduplication keys when necessary. Always pass deadlines to repositories and external calls.

6) Observability

Instrument every request. Emit structured logs with correlation identifiers, method names, latencies, and status codes. Export metrics (request counts, durations, error rates, queue depths) and traces across boundaries (HTTP, gRPC, database). Add health, readiness, and liveness endpoints that verify dependencies. Include build information for rapid rollback correlation.

7) Data access and transactions

Abstract storage with interfaces that are easy to mock in unit tests and easy to swap in integration tests. Use transactions to maintain invariants and apply optimistic concurrency (for example ETags or version fields) to avoid lost updates. Keep the service layer free from SQL, and return domain errors rather than database-specific ones.

8) Testing strategy

Adopt a layered approach:

• Unit tests target pure services with fake repositories. Validate edge cases, idempotency, and error mapping.
  
• Transport tests exercise handlers or RPC methods with in-memory servers. For REST, test encoding, headers, and status codes. For gRPC, use an in-process server and client.
  
• Integration tests boot real dependencies (for example PostgreSQL, Redis, Kafka) via containers, apply migrations, and run end-to-end scenarios. Include contract tests that compare responses to the OpenAPI schema or protobuf expectations.
  
• Nonfunctional tests validate performance and concurrency behavior: run parallel requests, assert no data races (with the race detector), and measure tail latencies.
  

9) Security and privacy

Terminated TLS, strict CORS where applicable, authn and authz middleware, input size limits, and request body caps protect the surface. Avoid logging secrets or PII; redact sensitive fields centrally. Validate JWTs or mTLS identities early and propagate claims in context.

10) Operations and lifecycle

Implement graceful shutdown: stop accepting new requests, cancel contexts, drain worker pools, flush logs, and close connections. Keep configuration external and validated at startup. Provide feature flags for migrations and cutovers. Document runbooks and budgets (SLOs for latency and error rates).

This blueprint yields a Go API that is deterministic, observable, and resilient. By validating inputs, isolating logic, governing concurrency, and testing thoroughly—from units to containers—you preserve correctness and performance as the system scales.

‍

Table

‍

Common Mistakes

• Binding JSON directly into domain structs and skipping validation.
  
• Mixing business logic in handlers, making testing hard.
  
• Ignoring context deadlines and cancellations; goroutines leak on client aborts.
  
• Unbounded worker pools and shared mutable maps without synchronization.
  
• Returning vague errors; mapping all failures to the same status code.
  
• Logging request bodies with secrets or PII.
  
• Skipping integration tests with real databases and only relying on mocks.
  
• No graceful shutdown; connections drop mid-flight.
  
• Missing idempotency on retried writes; duplicates appear under load.
  
• Failing to run the race detector and benchmarks for hot paths.
  
  

Sample Answers

Junior:
“I define an OpenAPI contract, create handlers that validate input with struct tags, and move logic into services. Middleware adds logging and timeouts. I write unit tests for services and handler tests with an in-memory server.”

Mid:
“I design REST or gRPC with contract-first schemas, validation at the edge, and interceptors for auth, rate limits, and tracing. Concurrency uses errgroup with context and bounded worker pools. Integration tests run with containers for the database and message broker and verify status codes and schemas.”

Senior:
“I operate a layered architecture: generated clients, strict validation, uniform middleware, and idempotent writes. Concurrency is bounded with semaphores, and every call carries deadlines. Observability includes structured logs, histograms, and traces. CI runs unit, transport, and containerized integration tests, plus race detection and benchmarks. Rollouts use health checks and graceful shutdown.”

‍

Evaluation Criteria

Look for a coherent Go API approach that:

• Uses contract-first design (OpenAPI or protobuf) and thin handlers.
  
• Enforces input validation and precise error mapping.
  
• Applies middleware for auth, limits, logging, metrics, tracing, and timeouts.
  
• Handles concurrency with context, errgroup, and bounded pools.
  
• Implements idempotency, retries, and graceful shutdown.
  
• Provides observability (logs, metrics, traces) and health endpoints.
  
• Includes layered testing: unit, transport, and containerized integration with schemas.
  Red flags: shared globals, no deadlines, no validation, only mocks, vague errors, or missing shutdown.
  
  

Preparation Tips

• Build a small service twice: REST with OpenAPI and gRPC with protobuf.
  
• Practice validation with strict JSON decoding and tag rules; reject unknown fields.
  
• Add middleware for request identifiers, timeouts, auth, rate limit, logging, metrics, and tracing.
  
• Implement a worker pool with a semaphore and errgroup, passing context for cancellation.
  
• Write unit tests for services, transport tests with in-memory servers, and integration tests using containers.
  
• Run the race detector, add benchmarks for hot endpoints, and profile allocations.
  
• Implement graceful shutdown and verify no goroutine leaks.
  
• Document error codes and add golden responses for contract stability.
  
  

Real-world Context

A payments team built a Go gRPC API with protobuf contracts and strict request validation. Middleware added mTLS auth, rate limits, and tracing. Hot paths used errgroup with bounded concurrency and deadlines, eliminating goroutine leaks during spikes. Integration tests spun PostgreSQL and Redis in containers and verified idempotent charge creation under retries. A structured error map simplified client handling. When traffic doubled, histograms and traces pinpointed a slow database index; after a targeted migration, p95 latency dropped by forty percent without changing code paths. Clear contracts, validation, and integration testing kept releases predictable.

‍

Key Takeaways

• Start contract-first; keep handlers thin and services testable.
  
• Validate strictly at boundaries; map errors precisely.
  
• Standardize cross-cutting concerns with middleware or interceptors.
  
• Use context, errgroup, and bounded pools for safe concurrency.
  
• Enforce timeouts, retries, and idempotency.
  
• Instrument logs, metrics, and traces; provide health checks.
  
• Run unit, transport, and containerized integration tests; use the race detector.
  
• Shut down gracefully and avoid shared mutable state.
  
  

Practice Exercise

Scenario:
You will build a Go REST and gRPC service for creating and listing orders. Requirements include strict validation, secure middleware, bounded concurrency for batch listing, and thorough integration testing.

Tasks:

1. Contracts: Define OpenAPI for REST and protobuf for gRPC with consistent fields. Generate server stubs and clients.
   
2. Validation: Create DTOs with required tags, ranges, and format checks. Reject unknown JSON fields and return typed errors or status details.
   
3. Middleware: Add request identifiers, auth (for example JWT), rate limiting, timeouts, structured logging, metrics, and tracing. Ensure handlers respect context deadlines.
   
4. Concurrency: Implement a listing service that fans out repository calls using errgroup with a semaphore cap. Make writes idempotent with request keys.
   
5. Data layer: Implement repository interfaces and transactional upserts. Add optimistic concurrency with version fields.
   
6. Testing:
   
   • Unit tests for services, including edge cases and idempotency.
     
   • Transport tests: in-memory HTTP server for REST and in-process gRPC server for RPC flows.
     
   • Integration tests: spin PostgreSQL in containers, run migrations, and verify end-to-end behaviors, including retries and timeouts.
     
   • Run the race detector and a benchmark for list endpoints.
     
     
7. Operations: Add health, readiness, and liveness endpoints, plus graceful shutdown that drains in-flight work.

‍

Deliverable:
A repository with contracts, layered code, middleware, bounded concurrency, and a complete test suite that demonstrates a robust Go REST or gRPC API ready for production.