feat(web): add Keenable search provider and bundled MCP integration - #1286
Open
ilya-bogin-keenable wants to merge 1 commit into
Open
ilya-bogin-keenable wants to merge 1 commit into
ilya-bogin-keenable wants to merge 1 commit into
Conversation
Keenable (https://keenable.ai) works without an API key, so it is the first web_search provider here that returns API results on a fresh install. Two pieces: - Native provider in web_search.rs: `search_provider = "keenable"`. Without KEENABLE_API_KEY the request goes to /v1/search/public with the required X-Keenable-Title header; with the key it goes to /v1/search with X-API-Key. The key only lifts the per-IP rate limit. In auto mode Keenable runs after every keyed provider and SearXNG, and before the DuckDuckGo HTML scrape, so paid keys keep priority. 429 surfaces as an error (naming the env var), never as an empty result. Snippets are read from `snippet` with `description` as the fallback, and `snippet_max_length` (default 500) is sent as a hint. - Bundled MCP integration `keenable` pointing at the hosted server https://api.keenable.ai/mcp over Streamable HTTP. No credentials, so `openfang add keenable` is Ready immediately. Default provider is unchanged (auto). Tests cover the parser, keyless and keyed request shape (endpoint, headers, body), 429 and empty responses via a local one-shot HTTP server, plus the bundled template count and a credential-free install round trip.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Keenable web search in two places: a native
keenableprovider for the built-inweb_searchtool, and a bundled MCP integration (openfang add keenable) for the hosted Keenable MCP server. Keenable works without an API key, so this is the first provider here that returns API results on a fresh install;KEENABLE_API_KEYis optional and only lifts the per-IP rate limit. I work at Keenable.Default provider is unchanged (
auto). In auto mode Keenable runs after every keyed provider (Tavily, Brave, Perplexity) and SearXNG, and before the DuckDuckGo HTML scrape, so anyone with a paid key configured sees no change in which provider answers first. The practical change is for installs with no search key at all: they get a search API instead of the scraper, and the scraper stays as the last fallback if Keenable fails.Changes
crates/openfang-types/src/config.rs:SearchProvider::Keenable,[web.keenable](KeenableSearchConfig:api_key_envdefaultKEENABLE_API_KEY,max_results5,snippet_max_length500), no validation warning when the key is unset.crates/openfang-runtime/src/web_search.rs:search_keenable. No key:POST https://api.keenable.ai/v1/search/publicwith the requiredX-Keenable-Title: openfangheader. Key set:POST /v1/searchwithX-API-Key. Body is{query, max_results (clamped 1..50), snippet_max_length}. Readssnippet, falls back todescription. 429 is an error that names the env var; other non-2xx follow the Brave/Tavily shape. Output goes throughwrap_external_content("keenable-search", ...)like the other providers. Added toall_providers()and to the auto cascade before DuckDuckGo.crates/openfang-runtime/src/tool_runner.rs:web_searchtool description lists Keenable.crates/openfang-extensions/integrations/keenable.toml+bundled.rs/registry.rs/installer.rs/lib.rs: bundled template forhttps://api.keenable.ai/mcp(Streamable HTTP, norequired_env, no OAuth). Counts 25 to 26; the transport test now accepts HTTPS remote endpoints. The template exposesmcp_keenable_search_web_pagesandmcp_keenable_fetch_page_content.docs/configuration.md,docs/mcp-a2a.md,docs/security.md,docs/cli-reference.md: provider table,[web.keenable]section, env var tables, and a bundled-integration example.No new dependencies.
Cargo.lockuntouched.Testing
cargo clippy -p openfang-types -p openfang-runtime -p openfang-extensions --all-targets -- -D warningspasses (the three touched crates; the full--workspacerun was not done locally because of disk space, CI covers it)cargo test -p openfang-runtime web_search12 passed;cargo test -p openfang-extensions55 passed;cargo test -p openfang-types386 passedcargo fmt --all -- --checkcleanWebSearchEngine::searchwithsearch_provider = "keenable"and no key in the environment, queryrust programming language: 5 results, 5 non-empty snippets, 3,390 chars of output, 330 ms. The hosted MCP endpoint answers anonymousinitializeandtools/list(JSON, no session header required) with the two tools above.New tests: parser (snippet first, description fallback, missing URL skipped,
maxhonoured, malformed bodies), and four request-shape tests against a local one-shot TCP server: keyless hits/v1/search/publicwithX-Keenable-Titleand noX-API-Key; keyed hits/v1/searchwithX-API-Keyin the header and not in the body,max_resultsclamped to 50; 429 returns an error naming the env var; emptyresultsreturns the "No results" error. Extensions:install_keenable_without_credentials(install isReadywith no key, survives a registry reload as anHttpentry with emptyenv/headers, removes cleanly).Security
max_resultsclamped to the API range; provider output wrapped as external content like the other providers)New egress:
api.keenable.ai, only when the provider is selected or reached in the auto cascade, or when thekeenableintegration is installed. The keyless request carries the query,max_results,snippet_max_length, and a fixed app title; the API key, when set, travels only in a request header via the existingZeroizingresolve_api_keypath.