feat: adds default tools that can use MCP resources #619

salman1993 · 2025-01-16T00:11:26Z

adds 2 default "platform" tools to list and read MCP resources
dynamically added when any of the connected extensions have resource capabilities

github-actions · 2025-01-16T00:33:24Z

Desktop App for this PR

The following build is available for testing:

📱 macOS Desktop App (Universal, signed)

The app is signed and notarized for macOS. After downloading, unzip the file and drag the Goose.app to your Applications folder.

This link is provided by nightly.link and will work even if you're not logged into GitHub.

github-actions · 2025-01-16T18:30:28Z

Desktop App for this PR

The following build is available for testing:

📱 macOS Desktop App (Universal, signed)

The app is signed and notarized for macOS. After downloading, unzip the file and drag the Goose.app to your Applications folder.

This link is provided by nightly.link and will work even if you're not logged into GitHub.

github-actions · 2025-01-17T17:56:08Z

Desktop App for this PR

The following build is available for testing:

📱 macOS Desktop App (Universal, signed)

The app is signed and notarized for macOS. After downloading, unzip the file and drag the Goose.app to your Applications folder.

This link is provided by nightly.link and will work even if you're not logged into GitHub.

github-actions · 2025-01-17T18:02:09Z

Desktop App for this PR

The following build is available for testing:

📱 macOS Desktop App (Universal, signed)

The app is signed and notarized for macOS. After downloading, unzip the file and drag the Goose.app to your Applications folder.

This link is provided by nightly.link and will work even if you're not logged into GitHub.

github-actions · 2025-01-17T19:06:10Z

Desktop App for this PR

The following build is available for testing:

📱 macOS Desktop App (Universal, signed)

The app is signed and notarized for macOS. After downloading, unzip the file and drag the Goose.app to your Applications folder.

This link is provided by nightly.link and will work even if you're not logged into GitHub.

salman1993 · 2025-01-17T19:44:08Z

crates/goose/src/prompts/system.md

+The format of a tool is "{system_name}__{tool_name}", i.e. the system 
+name followed by the tool name with '__' as the separator.
+
+By default, we add a tool called "platform__read_resource", which can 


can briefly describe platform__list_resources here

salman1993 · 2025-01-17T21:46:45Z

this looks good to me but i think we should test this out a bit more with MCP servers that have resources, after merging in #636

curious how you've found this so far for gdrive?

kalvinnchau · 2025-01-17T21:51:52Z

curious how you've found this so far for gdrive?

only a tiny useful for gdrive imo, the list returns the last 10 most recently viewed files.
the search + read workflow works well for me

depending on how other mcp servers implement resources, since we don't do pagination could be hit or miss

usage

salman1993 · 2025-01-17T22:42:52Z

lgtm - github won't let me approve it since i am a co-author. @baxen can you do a quick skim on this one?

baxen

Looks good! I would do the conditional enable though if possible

baxen · 2025-01-17T23:00:28Z

crates/goose/src/agents/reference.rs

+            }),
+        );
+
+        tools.push(read_resource_tool);


i would conditionally append these tools only if one of the systems has a resource capability. I'd also update the tooltip to include which systems have resources

baxen · 2025-01-17T23:01:25Z

crates/goose/src/prompts/system.md

@@ -10,6 +10,14 @@ to interactions with sytems that are not currently active. The currently
 active systems are below. Each of these systems provides tools that are
 in your tool specification.

+The format of a tool is "{system_name}__{tool_name}", i.e. the system


nit: i think these additions are excludable? not sure if you saw any performance issues with this but i have not seen any problems or a need to warn it about tools beyond the tool descriptions

wendytang · 2025-01-17T23:41:36Z

crates/goose/src/agents/capabilities.rs

+        // If system name is not provided, we need to search for the resource across all systems
+        // Loop through each system and try to read the resource, don't raise an error if the resource is not found
+        for system_name in self.clients.keys() {
+            let result = self.read_resource_from_system(uri, system_name).await;


nit: either make a note that we assume no risk of collision for generic names or ensure num_matches == 1 before proceeding?

hmm good point, if two systems have the same resource uri, it would find the first match and exit the loop
i'll put a TODO comment, not sure if we'd want to find the uri across all systems or not

wendytang

one interesting follow up idea would be to append some metadata about the resources like token count or file size and make that transparent to the user

developer2 only returns an Vec::new() for list_resources anyway, we should just report we don't support resources

kalvinnchau · 2025-01-18T00:20:24Z

@baxen added the conditional adding of the tools, I removed the with_resource(false, false) from the developer2 system, since that reported that it supports resources but just returns an empty Vec all the time.

baxen

Looks good! Let's try this out with some real mcp servers before merging i think

baxen · 2025-01-18T01:01:10Z

crates/goose/src/agents/capabilities.rs

+                let mut futures = FuturesUnordered::new();
+
+                // Create futures for each system
+                for (system_name, client) in &self.clients {


can skip some steps by iterating over self.resource_capable_systems instead here now?

baxen · 2025-01-18T01:02:11Z

crates/goose/src/agents/capabilities.rs

+        // Loop through each system and try to read the resource, don't raise an error if the resource is not found
+        // TODO: do we want to find if a provided uri is in multiple systems?
+        // currently it will reutrn the first match and skip any systems
+        for system_name in self.clients.keys() {


nit: can skip some steps by searching only self.resource_capable_system (like below)

baxen · 2025-01-18T01:03:40Z

crates/goose/src/prompts/system.md

@@ -10,12 +10,21 @@ to interactions with sytems that are not currently active. The currently
 active systems are below. Each of these systems provides tools that are
 in your tool specification.

+When a system is added that supports Resources,, we add two tools,


nit: i think we can just delete this entirely. the tool descriptions and the below hint per system should cover this i think!

baxen · 2025-01-18T01:06:38Z

crates/goose/src/agents/reference.rs

+                resource URI in the provided system, and reads in the resource content. If no system
+                is provided, the tool will search all systems for the resource.
+
+                The read_resource tool is typically used with a search query (can be before or after). Here are two examples:


nit: i'd remove these extra tips unless we find they are needed tips? not sure if it was tested. i've been noticing these specific examples don't necessarily make it better at using the tools but can confuse it into hallucinating usage that isn't relevant, with frontier models at least

baxen · 2025-01-18T01:06:46Z

crates/goose/src/agents/reference.rs

+                in the provided system, and returns a list for the user to browse. If no system
+                is provided, the tool will search all systems for the resource.
+
+                The list_resources tool is typically used before a read_resource tool call. Here are two examples:


nit: i'd remove these extra tips unless we find they are needed tips? not sure if it was tested. i've been noticing these specific examples don't necessarily make it better at using the tools but can confuse it into hallucinating usage that isn't relevant, with frontier models at least

reuse list_resources_from_system instead of reimplmenting it for iterating over all systems

salman1993 · 2025-01-20T15:25:52Z

crates/goose/src/prompts/system.md

@@ -15,7 +15,10 @@ in your tool specification.

 ## {{system.name}}
 {{system.description}}


does the description get populated anywhere? i think we can remove this

* v1.0: fix: add missing type param, separate block for signed/unsigned (#661) feat: adds default tools that can use MCP resources (#619)

* v1.0: use an npx and uvx shim with hermit for MCPs (#655) fix: add missing type param, separate block for signed/unsigned (#661) feat: adds default tools that can use MCP resources (#619)

* adds 2 default "platform" tools to list and read MCP resources * dynamically added when any of the connected extensions have resource capabilities Co-authored-by: Kalvin Chau <[email protected]>

salman1993 requested review from kalvinnchau, baxen and wendytang January 16, 2025 00:11

salman1993 changed the base branch from main to v1.0 January 16, 2025 00:11

kalvinnchau force-pushed the sm/read-resource-tool branch from f936f67 to 06388d2 Compare January 17, 2025 17:56

salman1993 commented Jan 17, 2025

View reviewed changes

kalvinnchau marked this pull request as ready for review January 17, 2025 20:07

salman1993 and others added 7 commits January 17, 2025 14:03

add read resource tool def and function

b2820b6

add read_resource tool info to system.md

16ab2f9

fix: check full tool_call name and move client tools in branch

35ef405

feat: add list_resources too to use in conjunction with read_resource

003c02a

refactor: split out list_resources_from_system and refactor futures

1eacaa1

usage

style: implement clippy suggestions

a6875da

feat: add platform__list_resources into the system prompt

03fe456

kalvinnchau force-pushed the sm/read-resource-tool branch from c52b367 to 03fe456 Compare January 17, 2025 22:04

baxen approved these changes Jan 17, 2025

View reviewed changes

wendytang reviewed Jan 17, 2025

View reviewed changes

wendytang approved these changes Jan 17, 2025

View reviewed changes

kalvinnchau added 3 commits January 17, 2025 16:07

feat: conditionally add support for listing/reading resources

39752e5

docs: add comment about matching uris across systems

a2dfeda

feat: remove resource capability from developer2

46b704f

developer2 only returns an Vec::new() for list_resources anyway, we should just report we don't support resources

kalvinnchau added 2 commits January 17, 2025 16:20

style: cargo fmt on developer2

fcec628

feat: switch to HashSet for system resource support check

00b4203

kalvinnchau force-pushed the sm/read-resource-tool branch from 4b1425f to 00b4203 Compare January 18, 2025 00:34

baxen approved these changes Jan 18, 2025

View reviewed changes

refactor: use resource_capable_system to skip incapable systems

428e03d

reuse list_resources_from_system instead of reimplmenting it for iterating over all systems

salman1993 commented Jan 20, 2025

View reviewed changes

salman1993 added 2 commits January 20, 2025 18:23

merge origin/v1.0

377f51f

remove unused description field in system_info

c588729

salman1993 changed the title ~~feat: add read_resource tool which is always present in reference agent~~ feat: adds default tools that can use MCP resources Jan 20, 2025

salman1993 merged commit a22daca into v1.0 Jan 20, 2025

michaelneale added a commit that referenced this pull request Jan 21, 2025

Merge branch 'v1.0' into micn/hermitize-npx-mcp

fd6fec7

* v1.0: fix: add missing type param, separate block for signed/unsigned (#661) feat: adds default tools that can use MCP resources (#619)

yingjiehe-xyz deleted the sm/read-resource-tool branch February 5, 2025 21:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: adds default tools that can use MCP resources #619

feat: adds default tools that can use MCP resources #619

salman1993 commented Jan 16, 2025 •

edited

Loading

github-actions bot commented Jan 16, 2025

github-actions bot commented Jan 16, 2025

github-actions bot commented Jan 17, 2025

github-actions bot commented Jan 17, 2025

github-actions bot commented Jan 17, 2025

salman1993 Jan 17, 2025

salman1993 commented Jan 17, 2025

kalvinnchau commented Jan 17, 2025

salman1993 commented Jan 17, 2025 •

edited

Loading

baxen left a comment

baxen Jan 17, 2025

baxen Jan 17, 2025

wendytang Jan 17, 2025

kalvinnchau Jan 18, 2025

wendytang left a comment

kalvinnchau commented Jan 18, 2025 •

edited

Loading

baxen left a comment

baxen Jan 18, 2025

baxen Jan 18, 2025

baxen Jan 18, 2025

baxen Jan 18, 2025

baxen Jan 18, 2025

salman1993 Jan 20, 2025

		@@ -15,7 +15,10 @@ in your tool specification.

		## {{system.name}}
		{{system.description}}

feat: adds default tools that can use MCP resources #619

feat: adds default tools that can use MCP resources #619

Conversation

salman1993 commented Jan 16, 2025 • edited Loading

github-actions bot commented Jan 16, 2025

Desktop App for this PR

github-actions bot commented Jan 16, 2025

Desktop App for this PR

github-actions bot commented Jan 17, 2025

Desktop App for this PR

github-actions bot commented Jan 17, 2025

Desktop App for this PR

github-actions bot commented Jan 17, 2025

Desktop App for this PR

Choose a reason for hiding this comment

salman1993 commented Jan 17, 2025

kalvinnchau commented Jan 17, 2025

salman1993 commented Jan 17, 2025 • edited Loading

baxen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wendytang left a comment

Choose a reason for hiding this comment

kalvinnchau commented Jan 18, 2025 • edited Loading

baxen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

salman1993 commented Jan 16, 2025 •

edited

Loading

salman1993 commented Jan 17, 2025 •

edited

Loading

kalvinnchau commented Jan 18, 2025 •

edited

Loading