07. Are Parallel Tool Calls Actually Faster?

The read-only task required the current path and a first-level directory listing. Kimi requested two Bash calls in parallel. Codex combined both reads in one Shell command.

The answer was correct in both cases, but the runtime handled three different strategies:

The theoretical latency

For independent durations t1 and t2, ignoring overhead:

serial ≈ t1 + t2
parallel ≈ max(t1, t2)

Model latency, authorization, process startup, serialization, and rate limits add fixed cost. Parallelizing two 8 ms local reads may produce an end-to-end difference smaller than ordinary model-request variance.

Only independent actions can overlap

The following require ordering:

• Create a directory, then write into it.
• Obtain credentials, then call an API.
• Read configuration, then choose a target.
• Modify the same resource twice.

Dependencies belong in runtime or plan structure rather than in the model’s informal memory.

Partial failure is the primary cost

Representing the batch as one failure repeats completed actions. That becomes dangerous for messages, orders, and other external side effects.

Concurrency increases resource pressure

Ten calls proposed in one model response do not authorize ten simultaneous connections. The executor applies global and per-tool concurrency limits, per-domain rate limits, total runtime and output budgets, cancellation signals, and timeouts.

The model cannot override these limits through arguments.

Representing dependencies

A minimum schedule represents calls as nodes and “must wait for” relations as directed edges. A node becomes eligible for authorization only after all predecessors complete. The runtime checks model-proposed dependencies for cycles, shared write resources, and budget conflicts.

Two reads of different files can overlap. Two writes to different paths may still share a build cache or external quota. Comparing tool names is not enough.

Cancellation has a scope

When one call fails, the event says whether cancellation applies to one item, its batch, or the task. Completed calls retain results. Running calls enter cancel_requested. Calls not yet started can become skipped.

The trace distinguishes “timed out itself” from “cancelled because a sibling failed.” Replay needs that distinction to decide which item can be retried.

A meaningful comparison

Use three tool pairs:

1. Two 10 ms local reads.
2. Two independent 500 ms network simulations.
3. One success plus one deterministic timeout.

Run serial, parallel, and merged strategies 30 times. Report P50 and P95 latency, where P50 is the median and P95 bounds 95% of samples. Also record calls, partial-failure recoveries, and duplicate side effects.

Parallelism is not an Agent maturity level. It applies only when actions are demonstrably independent, separately authorized, and correctly joined. End-to-end measurement determines whether it is faster.

Critical path determines the gain

The critical path is the dependency path with greatest total duration.

serial time = Σ duration_i
parallel lower bound = longest dependency path under resource limits

Multiple calls in one response prove protocol co-occurrence, not concurrency. Evidence requires start and finish times from one clock with overlapping intervals.

Read-only calls can still observe incompatible versions of changing state. Snapshots, transactions, or resource versions establish whether their results can be combined.

Concurrency limits belong to resource pools: CPU quotas, API rate tokens, browser instances, and database connections. Test independent, chain, diamond, and slow-branch graphs under serial, unlimited, and limited scheduling. Report queue delay, peak concurrency, rate limits, cancellations, completeness, and P95 tail latency.