ReviewOS

stacks/bun-router

perf: every value-returning handler drains its own response through a stream reader

#892
Closed chris opened this 8 days ago ·
8 days ago

A route that returns a plain value - the documented idiom, and what most handlers are - costs roughly twice what it needs to, and the reason is one missing header.

wrapResponse() serializes the value itself, so it holds the finished body string and knows its exact byte length. It then throws that away and builds new Response(body, { headers: { 'Content-Type': ... } }) with no Content-Length.

applyResponseCompression() reads Content-Length to decide whether a body is below the compression threshold. With no length it cannot tell a small answer from the start of a large stream, so it falls through to compressUnknownLengthResponse() → peekBody(), which opens a reader, buffers up to a kilobyte, and hands back a re-emitting stream. For seventeen bytes of JSON.

Measured

bun:jsc.profile() over a 20s load on GET /bench/json returning { hello: 'world' }, self time:

 39.0%  onRSDefaultControllerStartFulfilled
 19.9%  end
  8.2%  getReader
  ...
  0.9%  ReadableStream

Over two thirds of the router's JavaScript CPU is the stream machinery for a body that was already a complete string in memory.

At a fixed 10,000 req/s, microseconds of server CPU per request (Bun 1.4.1, oha, three repeats, median):

static JSONone path param
before36.936.0
after20.322.5

A 45% and 37% reduction, and it applies to every handler that returns an object, a string, a number or a boolean.

Fix

wrapResponse() states the length of every body it serializes - Buffer.byteLength for text and JSON, byteLength for an ArrayBuffer/Uint8Array. Nothing else changes: a Response a handler built is still returned untouched, and a ReadableStream still has no length until it ends.

createCompressedResponse() already deletes Content-Length when it pipes, so a body that does get compressed is unaffected.

Sign in to comment on this issue.