A workflow can become cheaper without finishing sooner, or faster without becoming cheaper. Input processing, output generation, tool execution, page loading and human waits contribute differently. Measure those components separately before deciding which optimization helped.
Define the clock you are measuring
Time to first output measures responsiveness. Model completion time measures when generation finishes. End-to-end task duration includes tools, browser work, retries and any required confirmation. A chart should name the clock it uses.
For a hypothetical 30-second task, imagine 5 seconds of inference, 20 seconds of page loading and 5 seconds of other work. Halving inference saves 2.5 seconds, not half the entire task. This is an illustration, not a measured Frugent result.
Find the critical path
Parallel work helps only where tasks are independent and the resources can handle the concurrency. Two browser actions on the same state may need to remain sequential. A faster model call cannot remove a required wait for a website to finish an operation.
OpenAI’s latency guidance includes reducing generated output, reducing requests and considering parallel work. Apply those ideas to the parts of your own workflow that dominate elapsed time, rather than expecting the same result across every task.
Do not infer spend from duration
A slow page load may use no additional model tokens. A fast sequence of many short model calls can still incur substantial charges. Record usage and time as different measurements and join them with the task identifier.
When evaluating action reuse, separate planning avoided from browser execution that still happens. A cached procedure may reduce reasoning work while the website’s network latency remains unchanged. That is a useful saving even if a user notices only a small speed improvement.
Report distributions and failures
Compare median and slower-tail task durations across equivalent workloads. Averages can hide occasional long stalls, and excluding failures can make a fragile workflow appear fast. Label whether human wait time is included.
Choose an optimization using both quality and the user’s deadline. For an interactive task, predictable completion may matter more than the lowest possible inference bill. For offline work, a longer completion window may be acceptable if quality and all-in cost improve.
Capture spans that do not double-count time
Record task start and finish, each model request’s start and finish, tool spans, browser waits and confirmation waits. If requests overlap, adding their durations does not equal elapsed task time. Keep the wall-clock total separate from summed component work. State whether a provider’s inference duration is measured directly or inferred from a request span that also includes network overhead. Compare the same timing definition across baseline and optimized runs.