PoC: InferenceClient is also a McpClient

[julien-c]

Required reading

https://modelcontextprotocol.io/quickstart/client

TL;DR: MCP is a standard API to expose sets of Tools that can be hooked to LLMs

Summary of how to use this

You can either use McpClient, or you can run an example Agent directly:

Tiny Agent

We now have a tiny Agent (a while loop, really) in this PR, built on top of the MCP Client. You can run it like this:

cd packages/mcp-client
pnpm run agent

McpClient i.e. the underlying class

const client = new McpClient({
	provider: "together",
	model: "Qwen/Qwen2.5-72B-Instruct",
	apiKey: process.env.HF_TOKEN,
});

await client.addMcpServer({
	// Filesystem "official" mcp-server with access to your Desktop
	command: "npx",
	args: ["-y", "@modelcontextprotocol/server-filesystem", join(homedir(), "Desktop")],
});

Variant where we call a custom, local MCP server

await client.addMcpServer(
	"node",
	["--disable-warning=ExperimentalWarning", join(homedir(), "Desktop/hf-mcp/index.ts")],
	{
		HF_TOKEN: process.env.HF_TOKEN,
	}
);

const response = await client.processQuery(`
	find an app that generates 3D models from text,
	and also get the best paper about transformers
`);

Where to find the MCP Server used here as an example

https://gist.github.com/julien-c/0500ba922e1b38f2dc30447fb81f7dc6

(Note that you can replace it with any MCP Server, from this doc for instance: https://modelcontextprotocol.io/examples)

Python version

Python version will be implemented in huggingface_hub in this PR: huggingface/huggingface_hub#2986

Contributions are welcome!

[grll]

huggingface/huggingface_hub#2986 few of my comments there are also relevant here. Probably major one is I think you could add SSE support quite easily.

Also very minor but first time I see typescript with 4 indents, but why not.

[coyotte508]

Also very minor but first time I see typescript with 4 indents, but why not.

It uses tabs, you can set your tab width to whatever you want :) (2, 4 or 8)

[grll]

Ah the good old tab vs space conundrum then :)

[julien-c]

At this point we should let LLMs decide between tabs and spaces once and for all

[grll]

At this point we should let LLMs decide between tabs and spaces once and for all

Claude has spoken

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[julien-c]

Ok this is ready for review!

Note, there's now a small Agent.ts class, and an example agent you can run with:

cd packages/mcp-client
pnpm run agent

[Wauplin]

Not sure I am using this the expected way but I had to tweak a bit to run hf-mcp.ts:

1. Added some dev dependencies to the project

	"devDependencies": {
		"@huggingface/hub": "workspace:^",
		"ts-node": "^10.9.2",
		"typescript": "^5.8.3",
		"zod": "^3.24.3"
	}

2. Fixed a compile error in hf-mcp.ts (see https://gist.github.com/julien-c/0500ba922e1b38f2dc30447fb81f7dc6?permalink_comment_id=5553466#gistcomment-5553466)
3. Added "hf-mcp": "npx ts-node hf-mcp.ts" as official script to package.json (this is only for easier testing for me)
4. Updated in cli.ts to run the command:

		command: "pnpm",
		args: ["run", "hf-mcp"],

5. (also had to hardcode the authorized path for filesystem MCP server-filesystem)

Then with everything setup I ran pnpm agent with prompt

List packages implemented under ~/projects/huggingface.js and make a nice looking HTML page listing them shortly. No need to provide further information per package. Once you have that, publish this HTML page as a static Space named "list-huggingface.js-packages"

(yup, not very inspired here)

=> resulting in https://huggingface.co/spaces/Wauplin/list-huggingface.js-packages

---

Some random feedback:

• once the model context went too big (600k tokens in a single query). I requested something like "what are the projects" and it listed all files recursively and gave this to the model... Maybe have something to truncate in such a case? (for now it crashes the pnpm agent command)
• I feel the output of the agent/cli is too bloated. Hard to follow what's interesting and what's not. Nothing much we can do though

[julien-c]

@Wauplin I should have updated the PR description because hf-mcp.ts (an example MCP server) is quite orthogonal to the PR. I'll update the PR desc. right now.

Your agent's Generated Space looks cool though 😍 cc @gary149:

[julien-c]

missing type in our spec probably @hanouticelina

[hanouticelina]

isn't that ChatCompletionOutputTextMessage ?

[julien-c]

well:

• it's not in our generated types (inference.ts)
• and it's weird to me to use a output message type as an input message
  wdyt?

[hanouticelina]

ok ok you're right, I'd rather define another type indeed, but I'd keep this type defined here instead of putting it in our specs if that's okay for you

[julien-c]

yes that works

[hanouticelina]

do we need to include the tool name here?

[julien-c]

maybe not for the LLM inference, but i'm using it in my agent's outer loop (in cli.ts) and i think it can't hurt anyways