Intro<\/p>\n
applications powered by Large Language Models (LLMs) require integration with external services, for example integration with Google Calendar to set up meetings or integration with PostgreSQL to get access to some data.\u00a0<\/p>\n
Function calling<\/p>\n
Initially these kinds of integrations were implemented through function calling: we were building some special functions that can be called by an LLM through some specific tokens (LLM was generating some special tokens to call the function, following patterns we defined), parsing and execution. To make it work we were implementing authorization and API calling methods for each of the tools. Importantly, we had to manage all the instructions for these tools to be called and build internal logic of these functions including default or user-specific parameters. But the hype around \u201cAI\u201d required fast, sometimes brute-force solutions to keep the pace, that is where MCPs were introduced by the Anthropic company.\u00a0<\/p>\n
MCPs<\/p>\n
MCP stands for Model Context Protocol and today it is a standard way of providing tools to the majority of the agentic pipelines. MCPs basically manage both integration functions and LLM instructions to use tools. At this point some may argue that Skills and Code execution that were also introduced by the Anthropic lately have killed MCPs, but in fact these features also tend to use MCPs for integration and instruction management (Code execution with MCP \u2014 Anthropic<\/a>). Skills and Code execution are focused on the context management problem and tools orchestration, that is a different problem from what MCPs are focused on<\/a>.<\/p>\n MCPs provide a standard way to integrate different services (tools) with LLMs and also provide instructions LLMs use to call the tools. However, here are a couple of problems:\u00a0<\/p>\n Current model context protocol supposes all the tool calling parameters to be exposed to the LLM, and all their values are supposed to be generated by the LLM. For example, that means the LLM has to generate user id value if function calling requires it. That is an overhead because the system, application knows user id value without the need for LLM to generate it, moreover to make LLM informed about the user id value we have to put it to the prompt (there is a \u201chiding arguments\u201d approach in FastMCP from gofastmcp<\/a> that is focused specifically on this problem, but I haven\u2019t seen it in the original MCP implementation from Anthropic).<\/p>\n No out-of-the-box control over instructions. MCPs provide description for each tool and description for each argument of a tool so these values are just used blindly in the agentic pipelines as an LLM API calling parameters.\u00a0And the description are provided by the each separate MCP server developer.<\/p>\n System prompt and tools<\/p>\n When you are calling LLMs you usually provide tools to the LLM call as an API call parameter. The value of this parameter is retrieved from the MCP\u2019s list_tools function that returns JSON schema for the tools it has.<\/p>\n At the same time this \u201ctools\u201d parameter is used to put additional information to the model\u2019s system prompt. For example, the Qwen3-VL model has chat_template<\/a> that manages tools insertion to the system prompt the following way:<\/p>\n \u201c…You are provided with function signatures within XML tags:\\\\n\\” }}\\n\u00a0 \u00a0 {%- for tool in tools %}\\n\u00a0 \u00a0 \u00a0 \u00a0 {{- \\”\\\\n\\” }}\\n\u00a0 \u00a0 \u00a0 \u00a0 {{- tool | tojson }}\\n\u00a0 \u00a0 {%- endfor %}…\u201d<\/p>\n So the tools descriptions end up in the system prompt of the LLM you are calling.<\/p>\n The first problem is actually partially solved by the mentioned \u201chiding arguments\u201d approach from the FastMCP, but still I saw some solutions where values like \u201cuser id\u201d were pushed to the model\u2019s system prompt to use it in the tool calling \u2014 it is just faster and much simpler to implement from the engineering point of view (actually no engineering required to just put it to the system prompt and rely on a LLM to use it). So here I am focused on the second problem.<\/p>\n At the same time I am leaving aside the problems related to tons of rubbish MCPs on the market \u2014 some of them do not work, some have generated tools description that can be confusing to the model. The problem I focus here on \u2014 non-standardised tools and their parameter descriptions that can be the reason why LLMs misbehave with some tools.<\/p>\n Instead of the conclusion for the introduction part:<\/p>\n If your agentic LLM-powered pipeline fails with the tools you have, you can:<\/p>\n Just choose a more powerful, modern and expensive LLM API;<\/p>\n Revisit your tools and the instructions overall.<\/p>\n Both can work. Make your decision or ask your AI-assistant to make a decision for you\u2026<\/p>\n Formal part of the work \u2014 research<\/p>\n 1. Examples of different descriptions<\/p>\n Based on the search through the real MCPs on the market, checking their tools lists and the descriptions, I could find many examples of the mentioned issue. Here I am providing just a single example from two different MCPs that have different domains as well (in the real life cases the list of MCPs a model uses tend to have different domains):<\/p>\n Example 1:\u00a0<\/p>\n Tool description: \u201cGenerate a area chart to show data trends under continuous independent variables and observe the overall data trend, such as, displacement = velocity (average or instantaneous) \u00d7 time: s = v \u00d7 t. If the x-axis is time (t) and the y-axis is velocity (v) at each moment, an area chart allows you to observe the trend of velocity over time and infer the distance traveled by the area\u2019s size.\u201d,<\/p>\n \u201cData\u201d property description: \u201cData for area chart, it should be an array of objects, each object contains a `time` field and a `value` field, such as, [{ time: \u20182015\u2019, value: 23 }, { time: \u20182016\u2019, value: 32 }], when stacking is needed for area, the data should contain a `group` field, such as, [{ time: \u20182015\u2019, value: 23, group: \u2018A\u2019 }, { time: \u20182015\u2019, value: 32, group: \u2018B\u2019 }].\u201d<\/p>\n Example 2:<\/p>\n Tool description: \u201cSearch for Airbnb listings with various filters and pagination. Provide direct links to the user\u201d,<\/p>\n \u201cLocation\u201d property description: \u201cLocation to search for (city, state, etc.)\u201d<\/p>\n Here I am not saying that any of these descriptions is incorrect, they are just very different from the format and details perspective.<\/p>\n 2. Dataset and benchmark<\/p>\n To prove that different tools descriptions can change model\u2019s behavior I used NVidia\u2019s \u201cWhen2Call\u201d<\/a> dataset. From this dataset I took test samples that have multiple tools for the model to choose from and one tool is the correct choice (it is correct to call a specific tool rather than any other or than to provide a text answer without any tool call, according to the dataset). The idea of the benchmark is to count correct and incorrect tool calls, I also count \u201cno tool calling\u201d cases as an incorrect answer. For the LLM I selected OpenAI\u2019s \u201cgpt-5-nano\u201d.<\/p>\n 3. Data generation<\/p>\n The original dataset provides just a single tool description. To create alternative descriptions for each tool and parameter I used \u201cgpt-5-mini\u201d to generate it based on the current one with the following instruction to complicate it (after generation there was an additional step of validation and re-generation when necessary):<\/p>\n \u00a0\u201c\u201d\u201dYou will receive the tool definition in JSON format. Your task is to make the tool description more detailed, so it can be used by a weak model.<\/p>\n One of the ways to complicate \u2014 insert detailed description of how it works and examples of how to use.<\/p>\n Example of detailed descriptions:<\/p>\n Tool description: \u201cGenerate a area chart to show data trends under continuous independent variables and observe the overall data trend, such as, displacement = velocity (average or instantaneous) \u00d7 time: s = v \u00d7 t. If the x-axis is time (t) and the y-axis is velocity (v) at each moment, an area chart allows you to observe the trend of velocity over time and infer the distance traveled by the area\u2019s size.\u201d,<\/p>\n Property description: \u201cData for area chart, it should be an array of objects, each object contains a `time` field and a `value` field, such as, [{ time: \u20182015\u2019, value: 23 }, { time: \u20182016\u2019, value: 32 }], when stacking is needed for area, the data should contain a `group` field, such as, [{ time: \u20182015\u2019, value: 23, group: \u2018A\u2019 }, { time: \u20182015\u2019, value: 32, group: \u2018B\u2019 }].\u201d<\/p>\n Return the updated detailed description strictly in JSON format (just change the descriptions, do not change the structure of the inputted JSON). Start your answer with:<\/p>\n \u201cNew JSON-formatted: \u2026\u201d<\/p>\n \u201c\u201d\u201d<\/p>\n 4. Experiments<\/p>\n To test the hypothesis I did a couple of tests, namely:<\/p>\n Measure the baseline of the model performance on the selected benchmark (Baseline);<\/p>\n Replace correct tool descriptions (including both tool description itself and parameters descriptions \u2014 the same for all the experiments) with the generated one (Correct tool replaced);<\/p>\n Replace incorrect tools descriptions with the generated (Incorrect tool replaced);<\/p>\n Replace all tools description with the generated (All tools replaced).<\/p>\n Here is a table with the results of these experiments (for each of the experiments 5 evaluations were executed, so in addition to accuracy standard deviation (std) is provided):<\/p>\n MethodMean accuracyAccuracy stdMaximum accuracy over 5 experimentsBaseline76.5%0.0379.0%Correct tool replaced80.5%0.0385.2%Incorrect tool replaced75.1%0.0176.5%All tools replaced75.3%0.0482.7%Table 1. Results of the experiments. Table prepared by the author.<\/p>\n Conclusion<\/p>\n From the table above it is evident that tools complication introduce bias to the model, selected LLM tends to choose the tool with more detailed description. At the same time we can see that extended description can confuse the model (in the case of all tools replaced).<\/p>\n The table shows that tools description provides mechanisms to manipulate and significantly adjust model\u2019s behaviour \/ accuracy, especially taking into account that the selected benchmark operates with a small number of tools at each model call, the average number of used tools at each sample is 4.35.<\/p>\n At the same time it clearly indicates that LLMs can have tools biases that potentially can be misused by MCP providers, that can be similar biases to those I reported before \u2014 style biases<\/a>. Research of the biases and their misuse can be important for further studies.<\/p>\n Engineering a solution<\/p>\n I\u2019ve prepared a PoC of tooling to address the mentioned issue in practice \u2014 Master-MCP. Master-MCP is a proxy MCP server that can be connected to any number of MCPs and also can be connected to an agent \/ LLM as a single MCP-server itself (currently stdio-transport MCP server). Default features of the Master-MCP I\u2019ve implemented:<\/p>\n Ignore some parameters. The implemented mechanics exclude all the parameters that start with \u201c_\u201d symbol from the tool\u2019s parameters schema. Later this parameter can be inserted programmatically or use default value (if provided).<\/p>\n Tool description adjustments. Master-MCP collects all the tool\u2019s and their descriptions from the connected MCP servers and provide a user a way to adjust it. It exposes a method with the simple UI to edit this list (JSON-schema), so the user can experiment with different tools\u2019 descriptions.<\/p>\n I invite everyone interested to join the project. With the community support the plans can include Master-MCP\u2019s functionality extension, for example:<\/p>\n Logging and monitoring followed by the advanced analytics;<\/p>\n Tools hierarchy and orchestration (including ML powered) to combine both modern context management techniques and smart algorithms.<\/p>\n