Skip to content

Conversation

ascandone
Copy link
Contributor

@ascandone ascandone commented Oct 7, 2025

Just an initial working MPV of an MCP integration for numscript. There are endless improvement that can be made, by improving current prompts and adding many other tools, but for now that's 80% of the value we can get out of it, with not so much complexity

Copy link
Contributor

coderabbitai bot commented Oct 7, 2025

Walkthrough

Adds an MCP server exposing evaluate and check tools for Numscript, a hidden Cobra subcommand to run it, registers the subcommand with the root command, and adds a helper to convert diagnostic severities to plain strings.

Changes

Cohort / File(s) Summary
MCP server and tools
internal/mcp_impl/handlers.go
Adds RunServer() error and registers evaluate and check tools. Implements parseBalancesJson and parseVarsJson, tool handlers that validate inputs, run the interpreter or analysis, and return ToolResultJSON or ToolResultError. Starts server over stdio.
CLI command wiring
internal/cmd/mcp.go, internal/cmd/root.go
Adds a hidden Cobra subcommand mcp that calls RunServer() (silencing Cobra's auto error/usage output) and registers it with the root command.
Diagnostics utilities
internal/analysis/diagnostic_kind.go
Adds SeverityToString(Severity) string mapping severity enum values to plain strings ("Error", "Warning", "Info", "Hint").

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor User
  participant CLI as CLI (cobra root)
  participant MCP as mcpCmd
  participant Server as MCP Server
  participant Tools as Tools (evaluate/check)
  participant Engine as Numscript Engine
  participant Anal as Analyzer

  User->>CLI: run app mcp
  CLI->>MCP: Execute mcpCmd
  MCP->>Server: RunServer()
  Server-->>User: serve over stdio (MCP)

  rect rgb(230,245,255)
    note right of User: evaluate flow
    User->>Server: tool:evaluate {script, balances, vars}
    Server->>Tools: invoke evaluate handler
    Tools->>Tools: parse balances & vars
    Tools->>Engine: interpreter.RunProgram(...)
    alt success
      Engine-->>Tools: result
      Tools-->>Server: ToolResultJSON
      Server-->>User: success response
    else error
      Engine-->>Tools: error
      Tools-->>Server: ToolResultError
      Server-->>User: error response
    end
  end

  rect rgb(240,255,240)
    note right of User: check flow
    User->>Server: tool:check {script}
    Server->>Tools: invoke check handler
    Tools->>Anal: parse + analyze(script)
    Anal-->>Tools: diagnostics [kind,severity,span]
    Tools-->>Server: ToolResultJSON {errors:[...]}
    Server-->>User: response
  end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~55 minutes

Poem

I hop to the server, nose in the air,
Scripts and balances handled with care.
I parse, I run, I check every string,
Plain severities make my heart sing.
Stdio hums; Numscript dances—hip hop, spring! 🐇✨

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Title Check ✅ Passed The title “feat: MCP first MVP” clearly and concisely summarizes the primary change of introducing a minimal viable MCP integration and matches the PR’s objectives and scope. It highlights the main feature addition without unnecessary detail or noise.
Description Check ✅ Passed The description succinctly states that this is an initial working MVP of the MCP integration for numscript and notes potential future improvements, directly relating to the implemented changes.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/mcp

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 5720b1b and a5e51da.

📒 Files selected for processing (1)
  • internal/mcp_impl/handlers.go (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
internal/mcp_impl/handlers.go (3)
internal/interpreter/interpreter.go (1)
  • RunProgram (228-285)
internal/analysis/check.go (1)
  • CheckSource (296-304)
internal/analysis/diagnostic_kind.go (2)
  • SeverityToString (39-52)
  • Severity (12-12)
🔇 Additional comments (1)
internal/mcp_impl/handlers.go (1)

139-172: LGTM!

The check tool implementation is correct with proper error handling and diagnostic conversion.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

server.WithInstructions(`
You're a numscript expert AI assistant. Numscript is a DSL that allows modelling finantial transaction in an easy and declarative way. Numscript scripts alwasy terminate.
`),
// TODO add prompt
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can add a more complete prompt in future releases

interpreter.StaticStore{
Balances: balances,
},
map[string]struct{}{},
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we'll add feature flags in future releases

@ascandone ascandone requested a review from a team October 7, 2025 08:16
@ascandone ascandone changed the title feat: MCP first MCP feat: MCP first MVP Oct 7, 2025
Copy link

codecov bot commented Oct 7, 2025

Codecov Report

❌ Patch coverage is 0% with 172 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.49%. Comparing base (952c1a7) to head (5720b1b).

Files with missing lines Patch % Lines
internal/mcp_impl/handlers.go 0.00% 151 Missing ⚠️
internal/analysis/diagnostic_kind.go 0.00% 12 Missing ⚠️
internal/cmd/mcp.go 0.00% 8 Missing ⚠️
internal/cmd/root.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #97      +/-   ##
==========================================
- Coverage   70.89%   68.49%   -2.41%     
==========================================
  Files          42       44       +2     
  Lines        4821     4993     +172     
==========================================
+ Hits         3418     3420       +2     
- Misses       1247     1418     +171     
+ Partials      156      155       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (1)
internal/cmd/mcp.go (1)

8-22: Set Silence flags on the command itself (optional).

Avoids duplicate error/usage output without toggling inside RunE.

 var mcpCmd = &cobra.Command{
   Use:    "mcp",
   Short:  "Run the mcp server",
   Hidden: true,
+  SilenceErrors: true,
+  SilenceUsage:  true,
   RunE: func(cmd *cobra.Command, args []string) error {
     err := mcp_impl.RunServer()
     if err != nil {
-      cmd.SilenceErrors = true
-      cmd.SilenceUsage = true
       return err
     }
     return nil
   },
 }
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 952c1a7 and de2bf29.

⛔ Files ignored due to path filters (2)
  • go.mod is excluded by !**/*.mod
  • go.sum is excluded by !**/*.sum, !**/*.sum
📒 Files selected for processing (4)
  • internal/analysis/diagnostic_kind.go (1 hunks)
  • internal/cmd/mcp.go (1 hunks)
  • internal/cmd/root.go (1 hunks)
  • internal/mcp_impl/handlers.go (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (3)
internal/analysis/diagnostic_kind.go (1)
internal/utils/utils.go (1)
  • NonExhaustiveMatchPanic (8-10)
internal/mcp_impl/handlers.go (3)
internal/interpreter/interpreter.go (1)
  • RunProgram (228-285)
internal/analysis/check.go (1)
  • CheckSource (296-304)
internal/analysis/diagnostic_kind.go (2)
  • SeverityToString (39-52)
  • Severity (12-12)
internal/cmd/mcp.go (1)
internal/mcp_impl/handlers.go (1)
  • RunServer (175-196)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: Tests
  • GitHub Check: Dirty
🔇 Additional comments (3)
internal/analysis/diagnostic_kind.go (1)

39-52: Helper looks correct and consistent with existing mapping.

internal/cmd/root.go (1)

27-27: Wiring the hidden mcp subcommand looks good.

internal/mcp_impl/handlers.go (1)

12-14: Confirmed valid mcp-go version pin
go.mod is pinned to v0.41.1, which matches an existing release.

Comment on lines +33 to +41
for asset, amountRaw := range assets {
amount, ok := amountRaw.(float64)
if !ok {
return interpreter.Balances{}, mcp.NewToolResultError(fmt.Sprintf("Expected float for amount: %v", amountRaw))
}

n, _ := big.NewFloat(amount).Int(new(big.Int))
iBalances[account][asset] = n
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Amounts parsing risks precision loss and silently truncates fractions. Enforce integer semantics and accept safe types.

  • Converting via big.NewFloat(amount).Int ignores fractional parts and float64 precision limits (money correctness risk).
  • Validate integer-ness, reject NaN/Inf, and optionally accept string integers.

Apply:

-    for asset, amountRaw := range assets {
-      amount, ok := amountRaw.(float64)
-      if !ok {
-        return interpreter.Balances{}, mcp.NewToolResultError(fmt.Sprintf("Expected float for amount: %v", amountRaw))
-      }
-
-      n, _ := big.NewFloat(amount).Int(new(big.Int))
-      iBalances[account][asset] = n
-    }
+    for asset, amountRaw := range assets {
+      var n *big.Int
+      switch v := amountRaw.(type) {
+      case float64:
+        if math.IsNaN(v) || math.IsInf(v, 0) {
+          return interpreter.Balances{}, mcp.NewToolResultError(fmt.Sprintf("Invalid number for amount at %s/%s: %v", account, asset, v))
+        }
+        if v != math.Trunc(v) {
+          return interpreter.Balances{}, mcp.NewToolResultError(fmt.Sprintf("Amount must be an integer at %s/%s, got: %v", account, asset, v))
+        }
+        if v > float64(math.MaxInt64) || v < float64(math.MinInt64) {
+          return interpreter.Balances{}, mcp.NewToolResultError(fmt.Sprintf("Amount out of 64-bit range at %s/%s: %v", account, asset, v))
+        }
+        n = big.NewInt(int64(v))
+      case string:
+        n = new(big.Int)
+        if _, ok := n.SetString(v, 10); !ok {
+          return interpreter.Balances{}, mcp.NewToolResultError(fmt.Sprintf("Invalid integer string for amount at %s/%s: %q", account, asset, v))
+        }
+      default:
+        return interpreter.Balances{}, mcp.NewToolResultError(fmt.Sprintf("Expected integer (number or string) for amount at %s/%s, got: %T", account, asset, amountRaw))
+      }
+      iBalances[account][asset] = n
+    }

Additionally, add import:

@@
-import (
+import (
   "context"
   "fmt"
+  "math"
   "math/big"
   "strings"

Committable suggestion skipped: line range outside the PR's diff.

Comment on lines +55 to +58
value, ok := rawValue.(string)
if !ok {
return map[string]string{}, mcp.NewToolResultError(fmt.Sprintf("Expected stringified var, got: %v", key))
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Fix error message: report the offending value/type, not the key.

-    value, ok := rawValue.(string)
-    if !ok {
-      return map[string]string{}, mcp.NewToolResultError(fmt.Sprintf("Expected stringified var, got: %v", key))
-    }
+    value, ok := rawValue.(string)
+    if !ok {
+      return map[string]string{}, mcp.NewToolResultError(fmt.Sprintf("Variable %q must be a string, got: %T", key, rawValue))
+    }
🤖 Prompt for AI Agents
In internal/mcp_impl/handlers.go around lines 55 to 58, the error message
currently reports the key when the cast to string fails; change it to report the
offending value and its type instead. Replace the fmt.Sprintf argument so it
includes rawValue and reflect.TypeOf(rawValue) (or use %T) in the message, e.g.
"Expected stringified var, got value=%v (type=%T)", rawValue, rawValue, and
return that in mcp.NewToolResultError.

Comment on lines 98 to 137
s.AddTool(tool, func(ctx context.Context, request mcp.CallToolRequest) (*mcp.CallToolResult, error) {
script, err := request.RequireString("script")
if err != nil {
return mcp.NewToolResultError(err.Error()), nil
}

parsed := parser.Parse(script)
if len(parsed.Errors) != 0 {
// TODO return all errors
out := make([]string, len(parsed.Errors))
for index, err := range parsed.Errors {
out[index] = err.Msg
}
mcp.NewToolResultError(strings.Join(out, ", "))
}

balances, mcpErr := parseBalancesJson(request.GetArguments()["balances"])
if mcpErr != nil {
return mcpErr, nil
}

vars, mcpErr := parseVarsJson(request.GetArguments()["vars"])
if mcpErr != nil {
return mcpErr, nil
}

out, iErr := interpreter.RunProgram(
context.Background(),
parsed.Value,
vars,
interpreter.StaticStore{
Balances: balances,
},
map[string]struct{}{},
)
if iErr != nil {
mcp.NewToolResultError(iErr.Error())
}
return mcp.NewToolResultJSON(*out)
})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Evaluate handler: missing returns on error paths and potential nil deref; also use ctx and avoid variable shadowing.

  • On parse errors, you create a ToolResultError but don’t return it.
  • On interpreter errors, same issue; returning JSON with a nil deref is possible.
  • Use the provided ctx instead of context.Background.
  • Avoid reusing the name out.

Apply:

@@
-    parsed := parser.Parse(script)
-    if len(parsed.Errors) != 0 {
-      // TODO return all errors
-      out := make([]string, len(parsed.Errors))
-      for index, err := range parsed.Errors {
-        out[index] = err.Msg
-      }
-      mcp.NewToolResultError(strings.Join(out, ", "))
-    }
+    parsed := parser.Parse(script)
+    if len(parsed.Errors) != 0 {
+      msgs := make([]string, len(parsed.Errors))
+      for i, pe := range parsed.Errors {
+        msgs[i] = pe.Msg
+      }
+      return mcp.NewToolResultError(strings.Join(msgs, ", ")), nil
+    }
@@
-    out, iErr := interpreter.RunProgram(
-      context.Background(),
+    execRes, iErr := interpreter.RunProgram(
+      ctx,
       parsed.Value,
       vars,
       interpreter.StaticStore{
         Balances: balances,
       },
       map[string]struct{}{},
     )
-    if iErr != nil {
-      mcp.NewToolResultError(iErr.Error())
-    }
-    return mcp.NewToolResultJSON(*out)
+    if iErr != nil {
+      return mcp.NewToolResultError(iErr.Error()), nil
+    }
+    return mcp.NewToolResultJSON(execRes)
🤖 Prompt for AI Agents
internal/mcp_impl/handlers.go around lines 98-137: the handler builds
ToolResultErrors but doesn’t return them, may dereference a nil interpreter
result, uses context.Background instead of the provided ctx, and reuses variable
names (out, mcpErr) causing shadowing. Fix by returning the error results
immediately (e.g. return mcp.NewToolResultError(strings.Join(...)), nil) on
parse failures and interpreter errors, check the interpreter output before
dereferencing so you don’t return mcp.NewToolResultJSON on a nil pointer,
replace context.Background() with the incoming ctx, and rename shadowed
variables (e.g. parsedErrs/outResult/parseErr) to avoid reuse. Ensure every
branch returns a valid (*mcp.CallToolResult, error).

Comment on lines +182 to +184
server.WithInstructions(`
You're a numscript expert AI assistant. Numscript is a DSL that allows modelling finantial transaction in an easy and declarative way. Numscript scripts alwasy terminate.
`),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Typos in instructions string.

Correct spelling/grammar to avoid user-facing nits:

-    server.WithInstructions(`
-    You're a numscript expert AI assistant. Numscript is a DSL that allows modelling finantial transaction in an easy and declarative way. Numscript scripts alwasy terminate.
-    `),
+    server.WithInstructions(`
+    You're a Numscript expert AI assistant. Numscript is a DSL that allows modeling financial transactions in an easy and declarative way. Numscript scripts always terminate.
+    `),
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
server.WithInstructions(`
You're a numscript expert AI assistant. Numscript is a DSL that allows modelling finantial transaction in an easy and declarative way. Numscript scripts alwasy terminate.
`),
server.WithInstructions(`
You're a Numscript expert AI assistant. Numscript is a DSL that allows modeling financial transactions in an easy and declarative way. Numscript scripts always terminate.
`),
🤖 Prompt for AI Agents
In internal/mcp_impl/handlers.go around lines 182 to 184, the instruction string
contains typos and grammar errors; replace the existing text with a corrected
sentence such as: "You are a Numscript expert AI assistant. Numscript is a DSL
that models financial transactions in an easy, declarative way. Numscript
scripts always terminate." Ensure spelling (Numscript, financial, always) and
punctuation are fixed and the string remains wrapped as a single
server.WithInstructions(...) argument.

}

out, iErr := interpreter.RunProgram(
context.Background(),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might want to use the proper context or a derived one ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one should be ok: we are only doing in-memory operations, and fetching data in a go struct
(We need context because the implementor of the interface might be sql queries )

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

David is right.
Always use the correct context.
There is no good reason to not use it.

@ascandone ascandone requested review from Dav-14 and gfyrag October 7, 2025 15:18
@ascandone ascandone enabled auto-merge (squash) October 8, 2025 06:49
@ascandone ascandone disabled auto-merge October 8, 2025 06:49
@ascandone ascandone enabled auto-merge (squash) October 8, 2025 06:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants