Skip to content

Add Raindrop observability integration#10965

Open
sestinj wants to merge 8 commits intomainfrom
wt-8
Open

Add Raindrop observability integration#10965
sestinj wants to merge 8 commits intomainfrom
wt-8

Conversation

@sestinj
Copy link
Contributor

@sestinj sestinj commented Mar 2, 2026

Summary

  • Integrates Raindrop.ai observability into the CLI via @raindrop-ai/ai-sdk, which auto-instruments Vercel AI SDK generateText/streamText/embed calls
  • Adds an injectable AI module override in AiSdkApi so the CLI can swap in a wrapped module without touching adapter internals
  • Activates only when RAINDROP_WRITE_KEY env var is set; zero impact otherwise

Test plan

  • RAINDROP_WRITE_KEY=test cn -p "hello" — no errors, no startup delay
  • With valid key, verify interactions appear in Raindrop dashboard
  • Without env var — zero impact, no Raindrop imports loaded
  • Run existing tests to confirm nothing breaks

🤖 Generated with Claude Code


Continue Tasks: ✅ 7 no changes — View all


Summary by cubic

Adds Raindrop observability to the CLI by wrapping Vercel AI SDK calls. Opt-in via RAINDROP_WRITE_KEY; initializes before services with a hashed anonymous user ID, only routes through the AI SDK after successful init, shuts down cleanly, and improves smoke-test reliability with retry/backoff helpers (model unchanged due to CI constraints).

  • New Features

    • Auto-instrumentation via @raindrop-ai/ai-sdk for generateText/streamText/embed.
    • New setAiModuleOverride hook and getAiModule in AiSdkApi to inject the wrapped AI module.
    • Gate CONTINUE_USE_AI_SDK until Raindrop initializes to avoid breaking completions/FIM when disabled.
  • Bug Fixes

    • Reduce smoke-api flakiness with exponential backoff retries (runHeadlessWithRetry, sendMessageAndWait); revert model change and keep claude-3-haiku-20240307 for CI.

Written for commit 9b9bf16. Summary will update on new commits.

Wraps the Vercel AI SDK's generateText/streamText/embed calls with
Raindrop instrumentation when RAINDROP_WRITE_KEY is set. Uses an
injectable AI module override in AiSdkApi so the CLI can swap in the
wrapped module without touching the adapter internals.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@sestinj sestinj requested a review from a team as a code owner March 2, 2026 00:49
@sestinj sestinj requested review from Patrick-Erichsen and removed request for a team March 2, 2026 00:49
@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Mar 2, 2026
@continue
Copy link
Contributor

continue bot commented Mar 2, 2026

Documentation Review

No documentation updates needed.

This PR adds an opt-in Raindrop observability integration that:

  • Activates only when RAINDROP_WRITE_KEY environment variable is set
  • Has zero impact otherwise (no imports loaded, no startup delay)
  • Is infrastructure tooling for observability rather than a core user-facing feature

The existing Telemetry docs cover Continue's own anonymous telemetry collection. Third-party observability integrations like Raindrop are internal/enterprise features that don't need public documentation at this time.

If Raindrop observability becomes a more prominent, documented feature with config.yaml support in the future, documentation can be added then.

sestinj and others added 2 commits March 1, 2026 16:52
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 7 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="extensions/cli/src/telemetry/raindropService.ts">

<violation number="1" location="extensions/cli/src/telemetry/raindropService.ts:13">
P2: Set CONTINUE_USE_AI_SDK only after Raindrop initialization succeeds (or clear it on failure). As written, a Raindrop init failure still forces OpenAI/Anthropic through AiSdkApi, which does not support completions/FIM and can break those paths.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

sestinj and others added 2 commits March 1, 2026 17:07
Prevents routing through AiSdkApi if Raindrop initialization fails,
which would break completions/FIM paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…xity

The function exceeded the max complexity of 30 (was 36). Extracted
PostHog reporting into a separate function.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Mar 2, 2026
sestinj and others added 3 commits March 1, 2026 23:17
Switch default smoke model from claude-3-haiku-20240307 to
claude-3-5-haiku-latest for better rate limits. Add retry helpers
(runHeadlessWithRetry, sendMessageAndWait) that detect rate-limit
errors in responses and retry with exponential backoff (10s, 20s, 40s).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The -latest alias isn't recognized by the CI Anthropic API key.
Use claude-3-5-haiku-20241022 explicitly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The CI Anthropic API key only has access to claude-3-haiku-20240307.
The rate-limit resilience comes from the retry/backoff helpers instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

1 participant