Tools Optimization

When building complex AI agents, it’s common to provide the model with a massive list of available functions (e.g., search_database, send_email, calculate_math, create_ticket). However, sending 50 tool definitions in a single request when the user just said “Hello” causes two major problems:

Token Waste: Defining tools consumes a massive amount of input tokens.
Hallucination Risk: The more tools a model sees, the more likely it is to get confused and try to call the wrong tool.

LLM Router’s Tools Optimization engine solves this by actively scoring and filtering your tools array before sending it to the upstream model.

How Tool Optimization Works

When a request containing a tools array arrives, our internal Gateway AI acts as a Tool Selector Agent.

It analyzes the user’s prompt against the descriptions of all provided tools.
It assigns a Relevance Score (0.0 to 1.0) to each tool.
It detects Dependencies (e.g., if create_element requires get_context, both are scored highly).
If a tool’s score falls below your configured threshold (acceptScore), it is stripped from the request entirely.

Configuration

You configure this behavior inside the gateway.toolOptimization object.

TypeScript

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llmrouter.app/v1",
  apiKey: process.env.LLM_ROUTER_API_KEY,
});

async function main() {
  const response = await client.chat.completions.create({
    model: "claude-3-5-sonnet",
    messages: [{ role: "user", content: "What is 256 multiplied by 14?" }],

    // Imagine 20 different tools defined here
    tools: [
      {
        type: "function",
        function: { name: "calculator", description: "Performs math" },
      },
      {
        type: "function",
        function: { name: "send_email", description: "Sends an email" },
      },
      {
        type: "function",
        function: { name: "search_db", description: "Searches users" },
      },
    ],

    // @ts-expect-error - Custom LLM Router extension
    gateway: {
      toolOptimization: {
        enabled: true, // Master switch
        acceptScore: 0.5, // Only keep tools scoring 0.5 or higher
        alwaysInclude: ["search_db"], // These tools bypass the filter and are ALWAYS sent
      },
    },
  });

  console.log(response.choices[0].message.content);
}
main();

What happens in this example?

The user asks a math question.
The Gateway scores calculator at 1.0. It scores send_email at 0.0.
It sees search_db in your alwaysInclude array.
Result: The router strips send_email and sends ONLY calculator and search_db to Claude. You save tokens, and Claude doesn’t get distracted.

Configuration Properties

The `toolOptimization` Object

Property	Type	Default	Description
`enabled`	`boolean`	`true`	Toggles the probabilistic tool filtering engine on or off.
`acceptScore`	`number`	`0.5`	The minimum relevance score (0.0 to 1.0) required for a tool to be included in the final request to the LLM.
`alwaysInclude`	`string[]`	`[]`	An array of exact function names (e.g., `["get_weather"]`). These tools will always be sent to the LLM, bypassing the scoring engine entirely.

Getting started

SDKs

AI tools

How Tool Optimization Works

Configuration

What happens in this example?

Configuration Properties

The `toolOptimization` Object

​How Tool Optimization Works

​Configuration

​What happens in this example?

​Configuration Properties

​The toolOptimization Object

How Tool Optimization Works

Configuration

What happens in this example?

Configuration Properties

The `toolOptimization` Object