How to architect dApps balancing on-chain logic and scale?

Long Answer

Designing a dApp that balances on-chain security with off-chain scalability starts by defining a trust contract: what must be cryptographically enforced on-chain vs what can be computed, cached, or orchestrated off-chain and still be correct (or safely recoverable). Then you align data flow, execution, and UX around that contract.

1) Contract scope: keep the core invariant on-chain
Put asset custody, settlement, permission checks, invariant enforcement, and upgrade governance on-chain. Contracts expose minimal, explicit state transitions. Favor pull-based patterns (users claim funds) to avoid stuck pushes, and design for idempotency (replays safe). Emit rich events for downstream indexing (entity keys, deltas, version).

2) Execution tier: scale with rollups and proofs
If L1 gas/latency is high, move execution to L2 rollups (optimistic or zk). Optimistic gives cheap throughput with challenge windows; zk offers faster finality via proofs. For compute-heavy steps (matching, scoring), use off-chain compute + on-chain verification when possible (e.g., submit a zk proof that a rule was satisfied), or record commitments on-chain and settle later.

3) Read path: event-sourced index + query layer
Blockchains are write-optimized but read-hostile. Build an event-sourced indexer that subscribes to logs across finality depth and materializes read models (searchable, denormalized views) in a DB or search engine. Tools: The Graph/viem-powered indexers/custom workers. Apply reorg safety: track canonical head, roll back orphaned blocks, and re-project state. Cache results with TTL + ETag keyed by block number; expose cursor-based pagination rooted in block height to make results reproducible.

4) Write path: queues, relayers, and AA
Clients prepare transactions locally and sign. Use Account Abstraction (ERC-4337) or meta-transactions so users can pay fees flexibly or sponsors can subsidize. Put a relayer behind a rate-limited API; maintain an outbox queue with idempotency keys (nonce, op hash). Retry with exponential backoff; fence duplicates by nonce. On confirmation, mark operations confirmed → finalized after N blocks to withstand reorgs.

5) Off-chain features: keep them verifiable
Move feeds, recommendations, aggregation, analytics, search off-chain for speed, but anchor outputs:

• Determinism: derive from on-chain events + immutable data snapshots.
  
• Attestation: sign responses or store Merkle roots of datasets on-chain periodically.
  
• Prove-or-recover: if an off-chain service misbehaves, users can recompute from the chain or ignore untrusted hints.
  

6) Oracles and external data
For price feeds and real-world inputs, rely on battle-tested oracle networks. Use medianization and staleness guards; contracts should reject outdated or wildly deviating values. Record which feed/version was used in state to aid audits.

7) Data availability and integrity
Prefer rollups with strong data availability (DA). If using off-chain DA, anchor commitments on-chain so anyone can reconstruct state. For multi-chain, treat each chain as a failure domain; minimize trust in bridges, or use light-client/zk-based messaging.

8) UX: resilient states and progressive disclosure
Expose pending → included → confirmed → finalized states. Stream progress (mempool detection, inclusion block, confirmations). Allow speed-ups/cancels where protocol permits. Fall back to “read-only, delayed” mode during congestion; never show “confirmed” before finality depth. Surface gas estimates with slippage buffers and preflight validations (allowance, balance).

9) Reliability engineering

• RPC redundancy: pool multiple providers; implement quorum/fastest-wins reads with response validation.
  
• Health & backpressure: rate limit writes per user/IP; shed load gracefully.
  
• Reorg handlers: central module to revert projections and notify clients.
  
• Idempotent pipelines: every consumer processes block events exactly-once-effectively via checkpoints + dedupe keys.
  
• Observability: label every request with (chainId, block, txHash); track lag to head, reorg rate, index catch-up time, mempool→inclusion latency, and error taxonomy.
  

10) Security and governance
Use role-separated keys (ops vs upgrade), time-locks, and on-chain pause mechanisms. Sign all off-chain API responses; pin build artifacts and ABI hashes. For governance, prefer transparent upgrade proxies with community-visible proposals, or immutable cores with modular extensions.

Putting it together
A practical architecture: Smart contracts (L2) enforce assets and rules. A listener consumes logs from redundant RPCs, projects to read stores (Postgres/Elastic), and powers APIs/GraphQL. A relayer accepts signed ops (4337/userOps), ensures nonce ordering, and submits to the bundler/sequencer. A jobs tier performs off-chain analytics and posts attestations/commitments on-chain periodically. The client uses optimistic UI but only treats results as final after confirmations, coping with reorgs transparently.

‍

Table

‍

Common Mistakes

• Overloading contracts with heavy logic to “keep it pure,” causing gas bloat and upgrade pain.
  
• Treating RPC responses as truth: no quorum, no validation, single-provider dependence.
  
• Showing “confirmed” after first inclusion; no finality buffer → user sees rollbacks on reorgs.
  
• No idempotency: duplicate submissions on retries create nonce chaos.
  
• Indexers that don’t handle reorgs; projections drift from reality.
  
• Oracles without staleness/medianization; contracts accept stale prices.
  
• Ignoring DA; can’t reconstruct state if off-chain stores corrupt.
  
• Poor AA/meta-tx hygiene: sponsor abuse, missing rate limits.
  
• Lack of observability; failures are invisible (no block lag metrics, no reorg counters).
  
• Upgrades hidden or uncontrolled; no timelocks or community visibility.
  
  

Sample Answers (Junior / Mid / Senior)

Junior:
“I’d keep funds and permission checks on-chain, but build an indexer for fast reads. I’d show pending → confirmed after a few blocks to avoid reorg issues. Heavy features like search run off-chain.”

Mid:
“I’d deploy core contracts on an L2 rollup, emit rich events, and project them into a Postgres/Elastic read model. A relayer with idempotency keys handles meta-tx/AA. We use multiple RPCs and treat data as finalized after N blocks. Oracles feed prices with freshness checks.”

Senior:
“I define a trust contract: custody, invariants, and governance on-chain; scalable reads via event-sourced indexers; compute off-chain with attestations or zk proofs. The write path goes through a rate-limited relayer (4337) with nonce fencing and retries. Reliability comes from RPC quorum, reorg-aware projections, and finality states in the UI. Security uses timelocked upgrades and signed API responses. This balances safety, latency, and cost with measurable SLOs.”

‍

Evaluation Criteria

• Clear boundary between on-chain must-haves and off-chain scalable parts.
  
• Use of L2/rollups and, where relevant, zk proofs to cut cost while preserving trust.
  
• Robust read model via event-sourced indexers with reorg handling and finality awareness.
  
• Solid write path: meta-tx/AA, relayer queues, idempotency, retries, nonce control.
  
• Correct reorg/finality UX (pending/included/confirmed/finalized).
  
• Secure oracle design with freshness/medianization.
  
• RPC redundancy and validation; DA considerations.
  
• Observability: lag-to-head, inclusion latency, reorg rate, error taxonomy.
  
• Upgrade/governance discipline (timelocks, audits).
  Answers that keep “everything on-chain” or fully trust off-chain without proofs score poorly.
  
  

Preparation Tips

• Build a toy dApp: ERC-20 + simple marketplace on an L2.
  
• Implement an indexer that replays events, supports rollback on reorg, and materializes listings.
  
• Add a relayer: accept signed orders/userOps, enforce idempotency, backoff retries, and AA paymasters.
  
• Integrate an oracle with staleness + deviation checks; record which round you used.
  
• In the UI, model states: pending → included → confirmed (k blocks) → finalized; include speed-up/cancel.
  
• Simulate failures: drop an RPC, trigger a local reorg, delay oracle updates. Verify resiliency.
  
• Add metrics: block lag, inclusion time, reorg count, index catch-up, relayer success rate.
  
• Prepare a 60–90s pitch explaining the trust boundary, why reads are off-chain but verifiable, and how finality prevents user confusion.
  
  

Real-world Context

A DeFi app moved order matching off-chain, keeping settlement on-chain with zk-verified constraints; gas fell 60% while maintaining safety. An NFT marketplace rebuilt reads with an event-sourced indexer and block-keyed caches; search latency dropped to <100 ms and reorg incidents no longer corrupted listings. A wallet added 4337 AA with a relayer; success rate rose because users could sponsor fees, while idempotency eliminated duplicate submissions. During a provider outage, RPC quorum + finality-aware UI kept status accurate. The pattern: on-chain for trust, off-chain for speed, bridged by proofs, events, and disciplined reliability engineering.

‍

Key Takeaways

• Keep custody/invariants on-chain; move heavy reads/compute off-chain.
  
• Use rollups/zk for scale; indexers for fast, reorg-safe reads.
  
• Write path = relayer + AA + idempotency + retries.
  
• UI shows pending→confirmed→finalized; never over-promise.
  
• Oracles, RPC quorum, DA, and governance are non-negotiable.
  
  

Practice Exercise

Scenario: You’re building a decentralized order book. Users submit orders frequently; fills settle on-chain. Product wants fast search/quotes (<150 ms), cheap fees, and accurate status even during reorgs or provider hiccups.

Tasks:

1. On-chain: Write minimal contracts: post/cancel orders, settle fills, enforce nonces/permissions, emit rich events. Deploy on an L2.
   
2. Indexer: Build an event-sourced projector that tracks open orders, best bids/asks, and user balances. Support rollback on reorg (parent hash mismatch) and expose block-keyed queries with cursors.
   
3. Relayer & AA: Accept signed orders/userOps; enforce idempotency (orderId+nonce), rate limit per address, and retry with backoff. Submit to bundler/sequencer; record tx hash and inclusion block.
   
4. Oracles: Consume price feeds with staleness/deviation checks and record feed round used per fill.
   
5. UI/UX: Show pending → included → confirmed (k blocks) → finalized; allow cancel/replace; display quotes from the indexer (not RPC scans).
   
6. Reliability: Use 3 RPCs with quorum reads; track lag-to-head, inclusion latency, reorg rate, and index catch-up time.
   
7. Validation: Chaos test by pausing one RPC and introducing a short reorg on a testnet; verify orders roll back and re-confirm correctly.
   
8. SLOs: Quotes <150 ms p95, inclusion→confirmed <30s p95, reorg recovery <5s.
   
   

Deliverable: A 2-minute walkthrough (diagram + metrics) explaining the trust boundary, off-chain index, relayer flow, reorg handling, and how UX stays honest under stress.