Picking strategy—the method an agent uses to choose which action, tool, or information source to use next—can separate strong performers from the rest when tasks involve multiple steps and uncertain outcomes. Research on language-model agents suggests that performance depends not only on the model’s capabilities but also on how effectively it selects and sequences tools: in the 2023 ToolLLM study, a tool-use model trained on 16,000 real-world APIs achieved a ToolBench score of 49.4% with its best inference strategy, illustrating that strategy and evaluation choices can materially affect results. High performers tend to match their picking strategy to the task: they plan when dependencies matter, retrieve evidence when facts are uncertain, use tools when computation or live information is needed, and stop when further action is unlikely to help.
Picking strategy defines agent decision quality
An agent is a system that observes a task or environment and selects actions intended to achieve a goal. A picking strategy is its decision policy for choosing among available next steps, such as answering immediately, asking a clarifying question, searching for evidence, calling an API, delegating work, or checking a draft. It is therefore broader than tool selection: it also includes deciding when to act, how much effort to spend, and when to stop.
There is no single universally superior strategy. The appropriate choice depends on task complexity, uncertainty, the cost of errors, tool reliability, and available time. A simple factual request may call for a direct answer; a time-sensitive or high-stakes request may require retrieval and verification. The practical distinction between top performers and weaker agents is often whether they make that match deliberately rather than following one fixed pattern.
What “high performance” means
High performance means reliably completing the user’s actual task—not merely producing fluent text. Useful measures include task success, factual accuracy, tool-call correctness, latency, and cost. In benchmark settings, these metrics can diverge: an agent may improve its success rate by using more tools while also increasing response time or expense. Evaluation should therefore report both outcome quality and resource use, and should test whether the strategy generalizes beyond familiar tasks.
Why strategy matters
A picking decision determines what evidence or capability enters the next step of an agent’s reasoning. A mistaken choice can compound: an irrelevant search introduces noise, an unnecessary tool call wastes time, and a premature answer can leave a critical uncertainty unresolved. Conversely, a well-chosen action can reduce uncertainty and prevent downstream errors. The 2023 ToolLLM work is a useful reminder that tool access alone does not guarantee effective tool use; agents need methods for selecting and executing appropriate calls.
Picking strategy shifts separate high-performing agents from the rest
In agent design, a picking strategy is the rule or process that maps the current task state to a next action. The state may include the user’s request, information already gathered, confidence in the current answer, available tools, and remaining budget. This definition aligns with the broader idea of a policy in reinforcement learning: a policy selects actions based on an agent’s state, while a practical language-model agent may implement that policy through instructions, planning, learned behavior, or a combination of these.
Picking strategies can be grouped by their primary decision mechanism. The categories overlap, but they clarify what an agent is doing and when it may be useful:
- Direct selection: choose an answer or action immediately when the task is clear and low-risk.
- Plan-based selection: break a complex request into dependent steps before acting.
- Retrieval-based selection: seek relevant documents or current evidence before answering.
- Tool-based selection: call a calculator, database, API, code interpreter, or other capability to perform work more reliably.
- Adaptive or confidence-based selection: choose the next action based on uncertainty, expected benefit, cost, and risk.
- Verification and stopping: check whether the result meets requirements, then stop or take another corrective action.
Task-adaptive selection
Task-adaptive selection means choosing a strategy according to the structure and stakes of the request rather than applying the same workflow to every prompt. A request to rephrase a sentence may need no external tools. A request for current market prices may require browsing or a live data source. A request to compare several policy documents may need retrieval, evidence tracking, and a structured synthesis.
The relevant signal is not simply task length. A short question can be high-risk or time-sensitive, while a long creative assignment may be completed without outside information. Agents should consider ambiguity, recency, consequence of error, and whether the required information is available internally. This approach reduces both under-processing—answering without needed evidence—and over-processing, such as searching the web for a stable definition.
Planning and decomposition
Planning selection is the choice to divide a goal into smaller steps and determine their order. It is valuable when later actions depend on earlier findings, such as gathering sources before comparing claims or inspecting data before writing a report. Decomposition also makes progress easier to verify: the agent can check each subtask instead of treating a complex request as one undifferentiated generation.
The ReAct paper by Yao and colleagues describes a pattern in which language models interleave reasoning and actions, allowing observations from tools to inform subsequent decisions. This is different from producing a complete plan once and following it regardless of new evidence. For uncertain tasks, an effective agent often uses a provisional plan, acts, observes the result, and updates its next step. Planning is most useful when it remains responsive to evidence rather than becoming a rigid script.
Retrieval and tool selection
Retrieval selection means seeking external or stored information when the answer depends on evidence the agent cannot safely infer. Tool selection extends this idea to actions such as calculations, database queries, code execution, and API calls. The central question is whether a tool is likely to improve the answer enough to justify its cost, delay, and possible failure modes.
ToolLLM, introduced by Qin and colleagues in 2023, presented ToolBench, a benchmark built from instructions involving 16,000 real-world APIs, alongside ToolLLaMA and an automatic evaluator called ToolEval. The researchers reported a 49.4% score for their strongest ToolLLaMA inference setup on ToolBench. That figure is a benchmark result—not a general measure of agent quality—but it demonstrates why tool-use strategies need to be tested against concrete tasks rather than assumed to work because tools are available.
In practice, tool choice should account for the source’s authority and freshness. A calculator is preferable to mental arithmetic for error-sensitive calculations; an official database is usually preferable to an unverified summary for regulated or statistical facts. Agents should also check whether a tool returned usable information. A successful call is not the same as a correct result.
Confidence, verification, and stopping
Confidence-based selection uses uncertainty to decide whether the agent should answer, gather more evidence, or request clarification. Confidence should not be treated as a guarantee of correctness: language models can express certainty without reliable calibration. Better systems combine confidence signals with observable checks, such as source agreement, tool output validation, or explicit completion criteria.
Verification is a picking strategy because it selects a review action after an initial answer or result. It may involve checking calculations, confirming that a citation supports the associated claim, or testing whether generated code runs. Stopping is equally important. An agent should stop when the task’s requirements are met and the expected value of another action is lower than its cost. Without a stopping rule, repeated searches or revisions can add latency without improving quality.
How high-performing agents choose the next action
Strong picking behavior can be understood as a repeated decision cycle: identify the task, estimate uncertainty and risk, select an action, inspect the result, and decide whether to continue. This connects planning, retrieval, tool use, and verification into one process. The key is not maximizing the number of actions; it is choosing actions that are likely to improve task success.
Match effort to uncertainty and error cost
An agent should spend more effort when uncertainty or the cost of being wrong is high. For low-risk tasks with clear instructions, direct completion may be efficient. For medical, legal, financial, or safety-related requests, the agent should be more cautious, distinguish general information from professional advice, and rely on authoritative evidence where appropriate. The strategy should also account for user preferences: a request for a quick overview does not call for the same depth as an audited research brief.
This can be expressed as a practical decision rule: use an additional action when its expected improvement in accuracy or completeness outweighs its cost and risk. The agent need not calculate a literal numerical value in every interaction, but the principle helps prevent both impulsive answers and wasteful tool use.
Preserve evidence through the workflow
High-performing research agents keep track of which source supports which claim, distinguish direct evidence from inference, and check whether sources are current enough for the question. This practice reduces the chance that a plausible but unsupported statement survives into the final answer. It also makes revisions more targeted: if a claim lacks support, the agent can retrieve evidence for that claim rather than restarting the entire task.
A practical evidence workflow is to identify claims that require support, retrieve sources suited to those claims, compare conflicting evidence, and verify that the final wording does not overstate what the sources establish. For example, an agent reporting a benchmark result should identify the benchmark and setup, not present the score as a universal measure of intelligence.
Adapt to feedback and failed actions
A robust agent treats a failed search, malformed API response, or contradictory source as evidence that its current approach may need to change. It can reformulate a query, choose another tool, ask the user a clarifying question, or state that the available evidence is insufficient. Repeating the same unsuccessful action without modification is a weak picking strategy because it ignores the information contained in failure.
Human feedback is another valuable signal. If a user corrects an assumption or narrows the desired format, the agent should update its plan rather than continue optimizing for the original interpretation. This makes the workflow more reliable in open-ended tasks where requirements become clear through interaction.
Real-world implications and evaluation
Picking strategy matters wherever agents operate across tools, information sources, or multi-step tasks. In customer support, an agent must decide whether to answer from approved guidance or consult an account system. In data analysis, it must choose whether to inspect, clean, calculate, or visualize. In research, it must distinguish when a claim needs fresh evidence from when a stable explanation is sufficient. The same underlying principle applies: select the next action that best advances the user’s goal while controlling risk.
Evaluate success, cost, and reliability together
A meaningful evaluation should measure more than final-answer accuracy. It should include task completion, unsupported-claim rates, tool-call success, latency, and cost. It should also test failure recovery and performance on unfamiliar tasks. The ToolBench work illustrates one way to evaluate API use at scale, but benchmark scores remain sensitive to task design, evaluator assumptions, and the tools available at test time.
Teams can compare strategies through controlled tests: hold the model and task set constant, vary the picking policy, and measure both quality and resource use. Results should be broken down by task type and risk level. A strategy that wins on short, self-contained prompts may perform poorly on current-information requests or long workflows.
A practical decision checklist
Before acting, an agent—or a team designing one—can use a short checklist to improve consistency:
- Clarify the requested outcome, format, and constraints.
- Identify what is known, what is uncertain, and what may have changed over time.
- Choose the least costly action likely to meet the required quality bar.
- Use authoritative tools or sources when the task requires external evidence or exact computation.
- Inspect tool outputs and preserve the connection between evidence and claims.
- Verify high-impact results and stop once requirements are satisfied.
Conclusion: deliberate picking improves agent performance
Picking strategy is the agent’s method for choosing its next action, and it shapes accuracy, efficiency, and reliability. The most useful categories—direct, plan-based, retrieval-based, tool-based, confidence-driven, and verification-oriented selection—are not competing recipes so much as options to apply according to the task. Research such as ToolLLM’s ToolBench results shows that tool use is a measurable capability, while work such as ReAct emphasizes the value of acting, observing, and adjusting.
For teams building or deploying agents, the next step is to evaluate picking policies against representative tasks, measuring success alongside latency, cost, and error rates. For users, the practical lesson is to favor agents that explain uncertainty, use evidence when needed, and verify consequential results. Better performance comes not from doing more at every turn, but from choosing the right next move.
Sources: Qin et al., ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs, 2023, https://arxiv.org/abs/2307.16789; Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models, 2022, https://arxiv.org/abs/2210.03629; Sutton and Barto, Reinforcement Learning: An Introduction, second edition, MIT Press, 2018, http://incompleteideas.net/book/the-book-2nd.html.
