Turn a latency headline into a testable budget

“Sub-40ms orchestration” sounds like a result. Without an implementation, workload, clock boundary, and recorded samples, it is only an ambition. The useful move is to turn that headline into a budget that can later be tested.

The AXON signal path is an animated explanation of an agent-interface concept. It is not an orchestrator or a profiler. This exercise uses its former timing headline to show what would be needed before publishing such a number as evidence.

Define the start and the finish

A hand-off might mean “the sender decided to delegate until the receiver accepted the work.” It might instead mean “the external action completed.” Those are different measurements. The first can exclude model generation and action execution if it says so; it cannot quietly change its definition when the longer result is inconvenient.

Write a contract before collecting samples. For example: start when the sender has a fully prepared request, stop when it receives an acknowledgement that the receiver has accepted the required context. Include validation, transport, queueing, and any mandatory durable acknowledgement that blocks acceptance.

Allocate the hypothetical budget

The following values are planning inputs, not observations: 4 ms for validation, 6 ms for routing, 12 ms for transport and context, and 8 ms for receiver acceptance. They total 30 ms, leaving 10 ms of a 40 ms target unallocated.

Unallocated time is room in the plan, not a statistical guarantee about the tail. If a required synchronous cost exceeds the target, revise the design or the target. Renaming the cost “background work” does not make its dependency disappear.

Follow the critical path

For sequential stages, individual durations add to the total. Independent stages that run concurrently can contribute approximately their maximum duration rather than their sum, but only if the required result really waits for that combination and no hidden serialization remains.

Also count retries, queue wait, connection setup, and payload size. A warm existing connection with a tiny payload answers a narrower question than the first request after a cold start. Report both when both matter.

Do not add each stage’s p95 and call it the end-to-end p95. Quantiles do not generally compose that way: stages have different distributions and correlations. Measure the end-to-end duration for each request, then compute the percentile of those totals.

Move work only when its failure policy is clear

A rich trace view or analytics aggregation may happen after acceptance. A durable record that is required before executing a consequential action may not. Separate “not blocking this response” from “allowed to vanish.”

An asynchronous queue introduces backpressure, capacity limits, retries, and delivery semantics. Decide what the sender does when the queue cannot accept a required event. Options include rejecting the action, reducing scope, or pausing for recovery. Silently discarding the record changes the product’s guarantees.

Caching authorization can save a hop, but it creates an expiry and revocation problem. A faster answer based on stale permission is not a successful optimization. Include the policy version and scope in the design of the cache.

Choose clocks and a reproducible workload

Use a monotonic clock for elapsed duration within one process. performance.now() is useful in the browser for that purpose. Do not subtract arbitrary timestamps from two machines and assume their clocks are synchronized. Measuring sender-side request-to-acknowledgement duration avoids that particular mistake, while still requiring a precise acknowledgement definition.

A useful report includes:

• The implementation version, topology, region, runtime, and resource limits.
• Payload sizes, concurrency, arrival pattern, duration, and warm-up treatment.
• Sample count, success rate, errors, timeouts, retries, and the percentile method.
• Warm and cold results separately when startup changes the path.
• Raw or suitably redacted samples and the script that derives the summary.

Failures must not disappear from the story. A service that finishes only its easiest requests quickly can have attractive successful-request percentiles and a poor user experience. Report the timeout policy and unsuccessful proportion beside latency.

Use the right label on the interface

A useful mockup says “Target: 40 ms” near the number and explains its boundary. A future benchmark should instead link to a dated method and result. Neither a scrolling trace animation nor a browser smoke test can establish an orchestration latency figure when no orchestrator is running.

The same discipline applies to a portfolio: publish the experiment you actually performed, the source you actually inspected, or the design you actually propose. A clear budget is useful before a service exists, provided it remains a budget.