Skip to content

Instantly share code, notes, and snippets.

@mcvaha
Created April 27, 2025 23:54
Show Gist options
  • Save mcvaha/d534052a5eead07e2e6f1b47b267aab4 to your computer and use it in GitHub Desktop.
Save mcvaha/d534052a5eead07e2e6f1b47b267aab4 to your computer and use it in GitHub Desktop.
# Tools
Smolagents is an experimental API which is subject to change at any time. Results returned by the agents can vary as the APIs or underlying models are prone to change.
To learn more about agents and tools make sure to read the [introductory guide](../index). This page contains the API docs for the underlying classes.
[](#tools)Tools
---------------
### [](#smolagents.load_tool)load\_tool
#### smolagents.load\_tool
[](#smolagents.load_tool)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/tools.py#L732)
( repo\_id model\_repo\_id: typing.Optional\[str\] = None token: typing.Optional\[str\] = None trust\_remote\_code: bool = False \*\*kwargs )
Parameters
* [](#smolagents.load_tool.repo_id)**repo\_id** (`str`) — Space repo ID of a tool on the Hub.
* [](#smolagents.load_tool.model_repo_id)**model\_repo\_id** (`str`, _optional_) — Use this argument to use a different model than the default one for the tool you selected.
* [](#smolagents.load_tool.token)**token** (`str`, _optional_) — The token to identify you on hf.co. If unset, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`).
* [](#smolagents.load_tool.trust_remote_code)**trust\_remote\_code** (`bool`, _optional_, defaults to False) — This needs to be accepted in order to load a tool from Hub.
* [](#smolagents.load_tool.kwargs)**kwargs** (additional keyword arguments, _optional_) — Additional keyword arguments that will be split in two: all arguments relevant to the Hub (such as `cache_dir`, `revision`, `subfolder`) will be used when downloading the files for your tool, and the others will be passed along to its init.
Main function to quickly load a tool from the Hub.
Loading a tool means that you’ll download the tool and execute it locally. ALWAYS inspect the tool you’re downloading before loading it within your runtime, as you would do when installing a package using pip/npm/apt.
### [](#smolagents.tool)tool
#### smolagents.tool
[](#smolagents.tool)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/tools.py#L908)
( tool\_function: typing.Callable )
Parameters
* [](#smolagents.tool.tool_function)**tool\_function** (`Callable`) — Function to convert into a Tool subclass. Should have type hints for each input and a type hint for the output. Should also have a docstring including the description of the function and an ‘Args:’ part where each argument is described.
Convert a function into an instance of a dynamically created Tool subclass.
### [](#smolagents.Tool)Tool
### class smolagents.Tool
[](#smolagents.Tool)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/tools.py#L86)
( \*args \*\*kwargs )
A base class for the functions used by the agent. Subclass this and implement the `forward` method as well as the following class attributes:
* **description** (`str`) — A short description of what your tool does, the inputs it expects and the output(s) it will return. For instance ‘This is a tool that downloads a file from a `url`. It takes the `url` as input, and returns the text contained in the file’.
* **name** (`str`) — A performative name that will be used for your tool in the prompt to the agent. For instance `"text-classifier"` or `"image_generator"`.
* **inputs** (`Dict[str, Dict[str, Union[str, type, bool]]]`) — The dict of modalities expected for the inputs. It has one `type`key and a `description`key. This is used by `launch_gradio_demo` or to make a nice space from your tool, and also can be used in the generated description for your tool.
* **output\_type** (`type`) — The type of the tool output. This is used by `launch_gradio_demo` or to make a nice space from your tool, and also can be used in the generated description for your tool.
You can also override the method [setup()](about:/docs/smolagents/v1.14.0/en/reference/tools#smolagents.Tool.setup) if your tool has an expensive operation to perform before being usable (such as loading a model). [setup()](about:/docs/smolagents/v1.14.0/en/reference/tools#smolagents.Tool.setup) will be called the first time you use your tool, but not at instantiation.
#### from\_dict
[](#smolagents.Tool.from_dict)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/tools.py#L279)
( tool\_dict: dict \*\*kwargs ) → `Tool`
Parameters
* [](#smolagents.Tool.from_dict.tool_dict)**tool\_dict** (`dict[str, Any]`) — Dictionary representation of the tool.
* [](#smolagents.Tool.from_dict.*kwargs)\***\*kwargs** — Additional keyword arguments to pass to the tool’s constructor.
Tool object.
Create tool from a dictionary representation.
Creates a [Tool](about:/docs/smolagents/v1.14.0/en/reference/tools#smolagents.Tool) from a gradio tool.
#### from\_hub
[](#smolagents.Tool.from_hub)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/tools.py#L421)
( repo\_id: str token: typing.Optional\[str\] = None trust\_remote\_code: bool = False \*\*kwargs )
Parameters
* [](#smolagents.Tool.from_hub.repo_id)**repo\_id** (`str`) — The name of the Space repo on the Hub where your tool is defined.
* [](#smolagents.Tool.from_hub.token)**token** (`str`, _optional_) — The token to identify you on hf.co. If unset, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`).
* [](#smolagents.Tool.from_hub.trust_remote_code\(str,)**trust\_remote\_code(`str`,** _optional_, defaults to False) — This flags marks that you understand the risk of running remote code and that you trust this tool. If not setting this to True, loading the tool from Hub will fail.
* [](#smolagents.Tool.from_hub.kwargs)**kwargs** (additional keyword arguments, _optional_) — Additional keyword arguments that will be split in two: all arguments relevant to the Hub (such as `cache_dir`, `revision`, `subfolder`) will be used when downloading the files for your tool, and the others will be passed along to its init.
Loads a tool defined on the Hub.
Loading a tool from the Hub means that you’ll download the tool and execute it locally. ALWAYS inspect the tool you’re downloading before loading it within your runtime, as you would do when installing a package using pip/npm/apt.
Creates a [Tool](about:/docs/smolagents/v1.14.0/en/reference/tools#smolagents.Tool) from a langchain tool.
#### from\_space
[](#smolagents.Tool.from_space)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/tools.py#L500)
( space\_id: str name: str description: str api\_name: typing.Optional\[str\] = None token: typing.Optional\[str\] = None ) → [Tool](about:/docs/smolagents/v1.14.0/en/reference/tools#smolagents.Tool)
Parameters
* [](#smolagents.Tool.from_space.space_id)**space\_id** (`str`) — The id of the Space on the Hub.
* [](#smolagents.Tool.from_space.name)**name** (`str`) — The name of the tool.
* [](#smolagents.Tool.from_space.description)**description** (`str`) — The description of the tool.
* [](#smolagents.Tool.from_space.api_name)**api\_name** (`str`, _optional_) — The specific api\_name to use, if the space has several tabs. If not precised, will default to the first available api.
* [](#smolagents.Tool.from_space.token)**token** (`str`, _optional_) — Add your token to access private spaces or increase your GPU quotas.
The Space, as a tool.
Creates a [Tool](about:/docs/smolagents/v1.14.0/en/reference/tools#smolagents.Tool) from a Space given its id on the Hub.
Examples:
```
>>> image_generator = Tool.from_space(
... space_id="black-forest-labs/FLUX.1-schnell",
... name="image-generator",
... description="Generate an image from a prompt"
... )
>>> image = image_generator("Generate an image of a cool surfer in Tahiti")
```
```
>>> face_swapper = Tool.from_space(
... "tuan2308/face-swap",
... "face_swapper",
... "Tool that puts the face shown on the first image on the second image. You can give it paths to images.",
... )
>>> image = face_swapper('./aymeric.jpeg', './ruth.jpg')
```
#### push\_to\_hub
[](#smolagents.Tool.push_to_hub)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/tools.py#L326)
( repo\_id: str commit\_message: str = 'Upload tool' private: typing.Optional\[bool\] = None token: typing.Union\[bool, str, NoneType\] = None create\_pr: bool = False )
Parameters
* [](#smolagents.Tool.push_to_hub.repo_id)**repo\_id** (`str`) — The name of the repository you want to push your tool to. It should contain your organization name when pushing to a given organization.
* [](#smolagents.Tool.push_to_hub.commit_message)**commit\_message** (`str`, _optional_, defaults to `"Upload tool"`) — Message to commit while pushing.
* [](#smolagents.Tool.push_to_hub.private)**private** (`bool`, _optional_) — Whether to make the repo private. If `None` (default), the repo will be public unless the organization’s default is private. This value is ignored if the repo already exists.
* [](#smolagents.Tool.push_to_hub.token)**token** (`bool` or `str`, _optional_) — The token to use as HTTP bearer authorization for remote files. If unset, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`).
* [](#smolagents.Tool.push_to_hub.create_pr)**create\_pr** (`bool`, _optional_, defaults to `False`) — Whether to create a PR with the uploaded files or directly commit.
Upload the tool to the Hub.
#### save
[](#smolagents.Tool.save)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/tools.py#L295)
( output\_dir: str | pathlib.Path tool\_file\_name: str = 'tool' make\_gradio\_app: bool = True )
Parameters
* [](#smolagents.Tool.save.output_dir)**output\_dir** (`str` or `Path`) — The folder in which you want to save your tool.
* [](#smolagents.Tool.save.tool_file_name)**tool\_file\_name** (`str`, _optional_) — The file name in which you want to save your tool.
* [](#smolagents.Tool.save.make_gradio_app)**make\_gradio\_app** (`bool`, _optional_, defaults to True) — Whether to also export a `requirements.txt` file and Gradio UI.
Saves the relevant code files for your tool so it can be pushed to the Hub. This will copy the code of your tool in `output_dir` as well as autogenerate:
* a `{tool_file_name}.py` file containing the logic for your tool. If you pass `make_gradio_app=True`, this will also write:
* an `app.py` file providing a UI for your tool when it is exported to a Space with `tool.push_to_hub()`
* a `requirements.txt` containing the names of the modules used by your tool (as detected when inspecting its code)
Overwrite this method here for any operation that is expensive and needs to be executed before you start using your tool. Such as loading a big model.
Returns a dictionary representing the tool
### [](#smolagents.launch_gradio_demo)launch\_gradio\_demo
#### smolagents.launch\_gradio\_demo
[](#smolagents.launch_gradio_demo)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/tools.py#L686)
( tool: Tool )
Parameters
* [](#smolagents.launch_gradio_demo.tool)**tool** (`Tool`) — The tool for which to launch the demo.
Launches a gradio demo for a tool. The corresponding tool class needs to properly implement the class attributes `inputs` and `output_type`.
[](#default-tools)Default tools
-------------------------------
### [](#smolagents.PythonInterpreterTool)PythonInterpreterTool
### class smolagents.PythonInterpreterTool
[](#smolagents.PythonInterpreterTool)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/default_tools.py#L38)
( \*args authorized\_imports = None \*\*kwargs )
### [](#smolagents.FinalAnswerTool)FinalAnswerTool
### class smolagents.FinalAnswerTool
[](#smolagents.FinalAnswerTool)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/default_tools.py#L80)
( \*args \*\*kwargs )
### [](#smolagents.UserInputTool)UserInputTool
### class smolagents.UserInputTool
[](#smolagents.UserInputTool)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/default_tools.py#L90)
( \*args \*\*kwargs )
### [](#smolagents.DuckDuckGoSearchTool)DuckDuckGoSearchTool
### class smolagents.DuckDuckGoSearchTool
[](#smolagents.DuckDuckGoSearchTool)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/default_tools.py#L101)
( max\_results = 10 \*\*kwargs )
### [](#smolagents.GoogleSearchTool)GoogleSearchTool
### class smolagents.GoogleSearchTool
[](#smolagents.GoogleSearchTool)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/default_tools.py#L126)
( provider: str = 'serpapi' )
### [](#smolagents.VisitWebpageTool)VisitWebpageTool
### class smolagents.VisitWebpageTool
[](#smolagents.VisitWebpageTool)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/default_tools.py#L213)
( max\_output\_length: int = 40000 )
### [](#smolagents.SpeechToTextTool)SpeechToTextTool
### class smolagents.SpeechToTextTool
[](#smolagents.SpeechToTextTool)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/default_tools.py#L360)
( \*args \*\*kwargs )
[](#smolagents.ToolCollection)ToolCollection
--------------------------------------------
### class smolagents.ToolCollection
[](#smolagents.ToolCollection)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/tools.py#L787)
( tools: typing.List\[smolagents.tools.Tool\] )
Tool collections enable loading a collection of tools in the agent’s toolbox.
Collections can be loaded from a collection in the Hub or from an MCP server, see:
* [ToolCollection.from\_hub()](about:/docs/smolagents/v1.14.0/en/reference/tools#smolagents.ToolCollection.from_hub)
* [ToolCollection.from\_mcp()](about:/docs/smolagents/v1.14.0/en/reference/tools#smolagents.ToolCollection.from_mcp)
For example and usage, see: [ToolCollection.from\_hub()](about:/docs/smolagents/v1.14.0/en/reference/tools#smolagents.ToolCollection.from_hub) and [ToolCollection.from\_mcp()](about:/docs/smolagents/v1.14.0/en/reference/tools#smolagents.ToolCollection.from_mcp)
#### from\_hub
[](#smolagents.ToolCollection.from_hub)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/tools.py#L801)
( collection\_slug: str token: typing.Optional\[str\] = None trust\_remote\_code: bool = False ) → ToolCollection
Parameters
* [](#smolagents.ToolCollection.from_hub.collection_slug)**collection\_slug** (str) — The collection slug referencing the collection.
* [](#smolagents.ToolCollection.from_hub.token)**token** (str, _optional_) — The authentication token if the collection is private.
* [](#smolagents.ToolCollection.from_hub.trust_remote_code)**trust\_remote\_code** (bool, _optional_, defaults to False) — Whether to trust the remote code.
A tool collection instance loaded with the tools.
Loads a tool collection from the Hub.
it adds a collection of tools from all Spaces in the collection to the agent’s toolbox
> \[!NOTE\] Only Spaces will be fetched, so you can feel free to add models and datasets to your collection if you’d like for this collection to showcase them.
Example:
```
>>> from smolagents import ToolCollection, CodeAgent
>>> image_tool_collection = ToolCollection.from_hub("huggingface-tools/diffusion-tools-6630bb19a942c2306a2cdb6f")
>>> agent = CodeAgent(tools=[*image_tool_collection.tools], add_base_tools=True)
>>> agent.run("Please draw me a picture of rivers and lakes.")
```
#### from\_mcp
[](#smolagents.ToolCollection.from_mcp)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/tools.py#L841)
( server\_parameters: typing.Union\[ForwardRef('mcp.StdioServerParameters'), dict\] trust\_remote\_code: bool = False ) → ToolCollection
Parameters
* [](#smolagents.ToolCollection.from_mcp.server_parameters)**server\_parameters** (`mcp.StdioServerParameters` or `dict`) — The server parameters to use to connect to the MCP server. If a dict is provided, it is assumed to be the parameters of `mcp.client.sse.sse_client`.
* [](#smolagents.ToolCollection.from_mcp.trust_remote_code)**trust\_remote\_code** (`bool`, _optional_, defaults to `False`) — Whether to trust the execution of code from tools defined on the MCP server. This option should only be set to `True` if you trust the MCP server, and undertand the risks associated with running remote code on your local machine. If set to `False`, loading tools from MCP will fail.
A tool collection instance.
Automatically load a tool collection from an MCP server.
This method supports both SSE and Stdio MCP servers. Look at the `sever_parameters` argument for more details on how to connect to an SSE or Stdio MCP server.
Note: a separate thread will be spawned to run an asyncio event loop handling the MCP server.
Example with a Stdio MCP server:
```
>>> from smolagents import ToolCollection, CodeAgent
>>> from mcp import StdioServerParameters
>>> server_parameters = StdioServerParameters(
>>> command="uv",
>>> args=["--quiet", "[email protected]"],
>>> env={"UV_PYTHON": "3.12", **os.environ},
>>> )
>>> with ToolCollection.from_mcp(server_parameters, trust_remote_code=True) as tool_collection:
>>> agent = CodeAgent(tools=[*tool_collection.tools], add_base_tools=True)
>>> agent.run("Please find a remedy for hangover.")
```
Example with an SSE MCP server:
```
>>> with ToolCollection.from_mcp({"url": "http://127.0.0.1:8000/sse"}, trust_remote_code=True) as tool_collection:
>>> agent = CodeAgent(tools=[*tool_collection.tools], add_base_tools=True)
>>> agent.run("Please find a remedy for hangover.")
```
[](#smolagents.mcp_client.MCPClient)MCP Client
----------------------------------------------
### class smolagents.mcp\_client.MCPClient
[](#smolagents.mcp_client.MCPClient)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/mcp_client.py#L35)
( server\_parameters: mcp.client.stdio.StdioServerParameters | dict\[str, typing.Any\] | list\[mcp.client.stdio.StdioServerParameters | dict\[str, typing.Any\]\] )
Parameters
* [](#smolagents.mcp_client.MCPClient.server_parameters)**server\_parameters** (StdioServerParameters | dict\[str, Any\] | list\[StdioServerParameters | dict\[str, Any\]\]) — MCP server parameters (stdio or sse). Can be a list if you want to connect multiple MCPs at once.
Manages the connection to an MCP server and make its tools available to SmolAgents.
Note: tools can only be accessed after the connection has been started with the `connect()` method, done during the init. If you don’t use the context manager we strongly encourage to use “try … finally” to ensure the connection is cleaned up.
Example:
```
with MCPClient(...) as tools:
with MCPClient({"url": "http://localhost:8000/sse"}) as tools:
try:
mcp_client = MCPClient(...)
tools = mcp_client.get_tools()
finally:
mcp_client.stop()
```
Connect to the MCP server and initialize the tools.
#### disconnect
[](#smolagents.mcp_client.MCPClient.disconnect)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/mcp_client.py#L81)
( exc\_type: typing.Optional\[typing.Type\[BaseException\]\] = None exc\_value: typing.Optional\[BaseException\] = None exc\_traceback: typing.Optional\[traceback\] = None )
Disconnect from the MCP server
#### get\_tools
[](#smolagents.mcp_client.MCPClient.get_tools)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/mcp_client.py#L90)
( ) → list\[Tool\]
The SmolAgents tools available from the MCP server.
* `ValueError` — If the MCP server tools is None (usually assuming the server is not started).
The SmolAgents tools available from the MCP server.
Note: for now, this always returns the tools available at the creation of the session, but it will in a future release return also new tools available from the MCP server if any at call time.
[](#agent-types)Agent Types
---------------------------
Agents can handle any type of object in-between tools; tools, being completely multimodal, can accept and return text, image, audio, video, among other types. In order to increase compatibility between tools, as well as to correctly render these returns in ipython (jupyter, colab, ipython notebooks, …), we implement wrapper classes around these types.
The wrapped objects should continue behaving as initially; a text object should still behave as a string, an image object should still behave as a `PIL.Image`.
These types have three specific purposes:
* Calling `to_raw` on the type should return the underlying object
* Calling `to_string` on the type should return the object as a string: that can be the string in case of an `AgentText` but will be the path of the serialized version of the object in other instances
* Displaying it in an ipython kernel should display the object correctly
### [](#smolagents.AgentText)AgentText
Text type returned by the agent. Behaves as a string.
### [](#smolagents.AgentImage)AgentImage
Image type returned by the agent. Behaves as a PIL.Image.Image.
#### save
[](#smolagents.AgentImage.save)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/agent_types.py#L163)
( output\_bytes format: str = None \*\*params )
Parameters
* [](#smolagents.AgentImage.save.output_bytes)**output\_bytes** (bytes) — The output bytes to save the image to.
* [](#smolagents.AgentImage.save.format)**format** (str) — The format to use for the output image. The format is the same as in PIL.Image.save.
* [](#smolagents.AgentImage.save.*params)\***\*params** — Additional parameters to pass to PIL.Image.save.
Saves the image to a file.
Returns the “raw” version of that object. In the case of an AgentImage, it is a PIL.Image.Image.
Returns the stringified version of that object. In the case of an AgentImage, it is a path to the serialized version of the image.
### [](#smolagents.AgentAudio)AgentAudio
### class smolagents.AgentAudio
[](#smolagents.AgentAudio)
[< source \>](https://github.com/huggingface/smolagents/blob/v1.14.0/src/smolagents/agent_types.py#L175)
( value samplerate = 16000 )
Audio type returned by the agent.
Returns the “raw” version of that object. It is a `torch.Tensor` object.
Returns the stringified version of that object. In the case of an AgentAudio, it is a path to the serialized version of the audio.
[< \> Update on GitHub](https://github.com/huggingface/smolagents/blob/main/docs/source/en/reference/tools.mdx)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment