Skip to content

[dart_tooling_mcp_server] add tool for getting the selected widget #71

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 17, 2025
+286 −148
Merged

[dart_tooling_mcp_server] add tool for getting the selected widget #71

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1df4fcc
[dart_tooling_mcp_server] add tool for getting the selected widget
kenzieschmoll Apr 16, 2025
ab27e16
fix
kenzieschmoll Apr 16, 2025
513689f
json encode
kenzieschmoll Apr 16, 2025
ef1107d
Merge branch 'main' of github.com:dart-lang/ai into getSelectedWidget
kenzieschmoll Apr 17, 2025
c036d68
formatting
kenzieschmoll Apr 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 62 additions & 9 deletions pkgs/dart_tooling_mcp_server/lib/src/mixins/dtd.dart
View file
Original file line number Diff line number Diff line change
@@ -3,6 +3,7 @@
// BSD-style license that can be found in the LICENSE file.

import 'dart:async';
import 'dart:convert';

import 'package:dart_mcp/server.dart';
import 'package:dds_service_extensions/dds_service_extensions.dart';
@@ -39,11 +40,25 @@ base mixin DartToolingDaemonSupport on ToolsSupport {
@visibleForTesting
static bool debugAwaitVmServiceDisposal = false;

/// The id for the object group used when calling Flutter Widget
/// Inspector service extensions from DTD tools.
@visibleForTesting
static const inspectorObjectGroup = 'dart-tooling-mcp-server';

/// The prefix for Flutter Widget Inspector service extensions.
///
/// See https://github.com/flutter/flutter/blob/master/packages/flutter/lib/src/widgets/service_extensions.dart#L126
/// for full list of available Flutter Widget Inspector service extensions.
static const _inspectorServiceExtensionPrefix = 'ext.flutter.inspector';

/// Called when the DTD connection is lost, resets all associated state.
Future<void> _resetDtd() async {
_dtd = null;
_getDebugSessionsReady = false;

// TODO: determine whether we need to dispose the [inspectorObjectGroup] on
// the Flutter Widget Inspector for each VM service instance.

final future = Future.wait(
activeVmServices.values.map((vmService) => vmService.dispose()),
);
@@ -94,6 +109,7 @@ base mixin DartToolingDaemonSupport on ToolsSupport {
registerTool(screenshotTool, takeScreenshot);
registerTool(hotReloadTool, hotReload);
registerTool(getWidgetTreeTool, widgetTree);
registerTool(getSelectedWidgetTool, selectedWidget);

return super.initialize(request);
}
@@ -357,14 +373,12 @@ base mixin DartToolingDaemonSupport on ToolsSupport {
callback: (vmService) async {
final vm = await vmService.getVM();
final isolateId = vm.isolates!.first.id;
final groupId = 'dart-tooling-mcp-server';
const inspectorExtensionPrefix = 'ext.flutter.inspector';
try {
final result = await vmService.callServiceExtension(
'$inspectorExtensionPrefix.getRootWidgetTree',
'$_inspectorServiceExtensionPrefix.getRootWidgetTree',
isolateId: isolateId,
args: {
'groupName': groupId,
'groupName': inspectorObjectGroup,
// TODO: consider making these configurable or using defaults that
// are better for the LLM.
'isSummaryTree': 'true',
@@ -384,7 +398,7 @@ base mixin DartToolingDaemonSupport on ToolsSupport {
],
);
}
return CallToolResult(content: [TextContent(text: tree.toString())]);
return CallToolResult(content: [TextContent(text: jsonEncode(tree))]);
} catch (e) {
return CallToolResult(
isError: true,
@@ -394,11 +408,41 @@ base mixin DartToolingDaemonSupport on ToolsSupport {
),
],
);
} finally {
await vmService.callServiceExtension(
'$inspectorExtensionPrefix.disposeGroup',
}
},
);
}

/// Retrieves the selected widget from the currently running app.
///
/// If more than one debug session is active, then it just uses the first one.
// TODO: support passing a debug session id when there is more than one debug
// session.
Future<CallToolResult> selectedWidget(CallToolRequest request) async {
return _callOnVmService(
callback: (vmService) async {
final vm = await vmService.getVM();
final isolateId = vm.isolates!.first.id;
try {
final result = await vmService.callServiceExtension(
'$_inspectorServiceExtensionPrefix.getSelectedSummaryWidget',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this determine what things are "user code"? Do we need to call addPubRootDirectories? Or do we assume something else is?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Dart Code extension / Flutter IntelliJ plugin already do this today. Otherwise we assume the pub root directories is the root of the running app. CC @elliette to confirm this is how things still work today.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that matches my understanding. Though it looks like the call to addPubRootDirectories was removed from the IntelliJ plugin: flutter/flutter-intellij@1c1389f

FYI @pq @jwren this is another piece of code that might need to be added back

isolateId: isolateId,
args: {'objectGroup': groupId},
args: {'objectGroup': inspectorObjectGroup},
);

final widget = result.json?['result'];
if (widget == null) {
return CallToolResult(
content: [TextContent(text: 'No Widget selected.')],
);
}
return CallToolResult(
content: [TextContent(text: jsonEncode(widget))],
);
} catch (e) {
return CallToolResult(
isError: true,
content: [TextContent(text: 'Failed to get selected widget: $e')],
);
}
},
@@ -477,6 +521,15 @@ base mixin DartToolingDaemonSupport on ToolsSupport {
inputSchema: ObjectSchema(),
);

@visibleForTesting
static final getSelectedWidgetTool = Tool(
name: 'get_selected_widget',
description:
'Retrieves the selected widget from the active Flutter application. '
'Requires "${connectTool.name}" to be successfully called first.',
inputSchema: ObjectSchema(),
);

static final _dtdNotConnected = CallToolResult(
isError: true,
content: [
363 changes: 224 additions & 139 deletions pkgs/dart_tooling_mcp_server/test/tools/dtd_test.dart
View file
Original file line number Diff line number Diff line change
@@ -2,6 +2,8 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

import 'dart:convert';

import 'package:dart_mcp/server.dart';
import 'package:dart_tooling_mcp_server/src/mixins/dtd.dart';
import 'package:test/test.dart';
@@ -13,155 +15,238 @@ void main() {
late TestHarness testHarness;

group('dart tooling daemon tools', () {
// TODO: Use setUpAll, currently this fails due to an apparent TestProcess
// issue.
setUp(() async {
testHarness = await TestHarness.start();
await testHarness.connectToDtd();
});
group('[compiled server]', () {
// TODO: Use setUpAll, currently this fails due to an apparent TestProcess
// issue.
setUp(() async {
testHarness = await TestHarness.start();
await testHarness.connectToDtd();
});

test('can take a screenshot', () async {
await testHarness.startDebugSession(
counterAppPath,
'lib/main.dart',
isFlutter: true,
);
final tools = (await testHarness.mcpServerConnection.listTools()).tools;
final screenshotTool = tools.singleWhere(
(t) => t.name == DartToolingDaemonSupport.screenshotTool.name,
);
final screenshotResult = await testHarness.callToolWithRetry(
CallToolRequest(name: screenshotTool.name),
);
expect(screenshotResult.content.single, {
'data': anything,
'mimeType': 'image/png',
'type': ImageContent.expectedType,
test('can take a screenshot', () async {
await testHarness.startDebugSession(
counterAppPath,
'lib/main.dart',
isFlutter: true,
);
final tools = (await testHarness.mcpServerConnection.listTools()).tools;
final screenshotTool = tools.singleWhere(
(t) => t.name == DartToolingDaemonSupport.screenshotTool.name,
);
final screenshotResult = await testHarness.callToolWithRetry(
CallToolRequest(name: screenshotTool.name),
);
expect(screenshotResult.content.single, {
'data': anything,
'mimeType': 'image/png',
'type': ImageContent.expectedType,
});
});
});

test('can perform a hot reload', () async {
await testHarness.startDebugSession(
counterAppPath,
'lib/main.dart',
isFlutter: true,
);
final tools = (await testHarness.mcpServerConnection.listTools()).tools;
final hotReloadTool = tools.singleWhere(
(t) => t.name == DartToolingDaemonSupport.hotReloadTool.name,
);
final hotReloadResult = await testHarness.callToolWithRetry(
CallToolRequest(name: hotReloadTool.name),
);

expect(hotReloadResult.isError, isNot(true));
expect(hotReloadResult.content, [
TextContent(text: 'Hot reload succeeded.'),
]);
});
test('can perform a hot reload', () async {
await testHarness.startDebugSession(
counterAppPath,
'lib/main.dart',
isFlutter: true,
);
final tools = (await testHarness.mcpServerConnection.listTools()).tools;
final hotReloadTool = tools.singleWhere(
(t) => t.name == DartToolingDaemonSupport.hotReloadTool.name,
);
final hotReloadResult = await testHarness.callToolWithRetry(
CallToolRequest(name: hotReloadTool.name),
);

expect(hotReloadResult.isError, isNot(true));
expect(hotReloadResult.content, [
TextContent(text: 'Hot reload succeeded.'),
]);
});

test('can get runtime errors', () async {
await testHarness.startDebugSession(
counterAppPath,
'lib/main.dart',
isFlutter: true,
args: ['--dart-define=include_layout_error=true'],
);
final tools = (await testHarness.mcpServerConnection.listTools()).tools;
final runtimeErrorsTool = tools.singleWhere(
(t) => t.name == DartToolingDaemonSupport.getRuntimeErrorsTool.name,
);
final runtimeErrorsResult = await testHarness.callToolWithRetry(
CallToolRequest(name: runtimeErrorsTool.name),
);

expect(runtimeErrorsResult.isError, isNot(true));
final errorCountRegex = RegExp(r'Found \d+ errors?:');
expect(
(runtimeErrorsResult.content.first as TextContent).text,
contains(errorCountRegex),
);
expect(
(runtimeErrorsResult.content[1] as TextContent).text,
contains('A RenderFlex overflowed by'),
);
});
test('can get runtime errors', () async {
await testHarness.startDebugSession(
counterAppPath,
'lib/main.dart',
isFlutter: true,
args: ['--dart-define=include_layout_error=true'],
);
final tools = (await testHarness.mcpServerConnection.listTools()).tools;
final runtimeErrorsTool = tools.singleWhere(
(t) => t.name == DartToolingDaemonSupport.getRuntimeErrorsTool.name,
);
final runtimeErrorsResult = await testHarness.callToolWithRetry(
CallToolRequest(name: runtimeErrorsTool.name),
);

expect(runtimeErrorsResult.isError, isNot(true));
final errorCountRegex = RegExp(r'Found \d+ errors?:');
expect(
(runtimeErrorsResult.content.first as TextContent).text,
contains(errorCountRegex),
);
expect(
(runtimeErrorsResult.content[1] as TextContent).text,
contains('A RenderFlex overflowed by'),
);
});

test('can get the widget tree', () async {
await testHarness.startDebugSession(
counterAppPath,
'lib/main.dart',
isFlutter: true,
);
final tools = (await testHarness.mcpServerConnection.listTools()).tools;
final getWidgetTreeTool = tools.singleWhere(
(t) => t.name == DartToolingDaemonSupport.getWidgetTreeTool.name,
);
final getWidgetTreeResult = await testHarness.callToolWithRetry(
CallToolRequest(name: getWidgetTreeTool.name),
);

expect(getWidgetTreeResult.isError, isNot(true));
expect(
(getWidgetTreeResult.content.first as TextContent).text,
contains('MyHomePage'),
);
test('can get the widget tree', () async {
await testHarness.startDebugSession(
counterAppPath,
'lib/main.dart',
isFlutter: true,
);
final tools = (await testHarness.mcpServerConnection.listTools()).tools;
final getWidgetTreeTool = tools.singleWhere(
(t) => t.name == DartToolingDaemonSupport.getWidgetTreeTool.name,
);
final getWidgetTreeResult = await testHarness.callToolWithRetry(
CallToolRequest(name: getWidgetTreeTool.name),
);

expect(getWidgetTreeResult.isError, isNot(true));
expect(
(getWidgetTreeResult.content.first as TextContent).text,
contains('MyHomePage'),
);
});
});
});

group('$VmService management', () {
setUp(() async {
DartToolingDaemonSupport.debugAwaitVmServiceDisposal = true;
addTearDown(
() => DartToolingDaemonSupport.debugAwaitVmServiceDisposal = false,
);
group('[in process]', () {
setUp(() async {
DartToolingDaemonSupport.debugAwaitVmServiceDisposal = true;
addTearDown(
() => DartToolingDaemonSupport.debugAwaitVmServiceDisposal = false,
);

testHarness = await TestHarness.start(inProcess: true);
await testHarness.connectToDtd();
});
testHarness = await TestHarness.start(inProcess: true);
await testHarness.connectToDtd();
});

test('persists vm services', () async {
final server = testHarness.serverConnectionPair.server!;
expect(server.activeVmServices, isEmpty);

await testHarness.startDebugSession(
counterAppPath,
'lib/main.dart',
isFlutter: true,
);
await server.updateActiveVmServices();
expect(server.activeVmServices.length, 1);

// Re-uses existing VM Service when available.
final originalVmService = server.activeVmServices.values.single;
await server.updateActiveVmServices();
expect(server.activeVmServices.length, 1);
expect(originalVmService, server.activeVmServices.values.single);

await testHarness.startDebugSession(
counterAppPath,
'lib/main.dart',
isFlutter: true,
);
await server.updateActiveVmServices();
expect(server.activeVmServices.length, 2);
});
group('$VmService management', () {
test('persists vm services', () async {
final server = testHarness.serverConnectionPair.server!;
expect(server.activeVmServices, isEmpty);

await testHarness.startDebugSession(
counterAppPath,
'lib/main.dart',
isFlutter: true,
);
await server.updateActiveVmServices();
expect(server.activeVmServices.length, 1);

// Re-uses existing VM Service when available.
final originalVmService = server.activeVmServices.values.single;
await server.updateActiveVmServices();
expect(server.activeVmServices.length, 1);
expect(originalVmService, server.activeVmServices.values.single);

await testHarness.startDebugSession(
counterAppPath,
'lib/main.dart',
isFlutter: true,
);
await server.updateActiveVmServices();
expect(server.activeVmServices.length, 2);
});

test('automatically removes vm services upon shutdown', () async {
final server = testHarness.serverConnectionPair.server!;
expect(server.activeVmServices, isEmpty);

final debugSession = await testHarness.startDebugSession(
counterAppPath,
'lib/main.dart',
isFlutter: true,
);
await server.updateActiveVmServices();
expect(server.activeVmServices.length, 1);

await testHarness.stopDebugSession(debugSession);
await server.updateActiveVmServices();
expect(server.activeVmServices, isEmpty);
});
});

test('automatically removes vm services upon shutdown', () async {
final server = testHarness.serverConnectionPair.server!;
expect(server.activeVmServices, isEmpty);

final debugSession = await testHarness.startDebugSession(
counterAppPath,
'lib/main.dart',
isFlutter: true,
);
await server.updateActiveVmServices();
expect(server.activeVmServices.length, 1);

await testHarness.stopDebugSession(debugSession);
await server.updateActiveVmServices();
expect(server.activeVmServices, isEmpty);
group('get selected widget', () {
test('when a selected widget exists', () async {
final server = testHarness.serverConnectionPair.server!;
final tools =
(await testHarness.mcpServerConnection.listTools()).tools;

await testHarness.startDebugSession(
counterAppPath,
'lib/main.dart',
isFlutter: true,
);
await server.updateActiveVmServices();

final getWidgetTreeTool = tools.singleWhere(
(t) => t.name == DartToolingDaemonSupport.getWidgetTreeTool.name,
);
final getWidgetTreeResult = await testHarness.callToolWithRetry(
CallToolRequest(name: getWidgetTreeTool.name),
);

// Select the first child of the [root] widget.
final widgetTree =
jsonDecode(
(getWidgetTreeResult.content.first as TextContent).text,
)
as Map<String, Object?>;
final children = widgetTree['children'] as List<Object?>;
final firstWidgetId =
(children.first as Map<String, Object?>)['valueId'];
final appVmService = server.activeVmServices.values.first;
final vm = await appVmService.getVM();
await appVmService.callServiceExtension(
'ext.flutter.inspector.setSelectionById',
isolateId: vm.isolates!.first.id,
args: {
'objectGroup': DartToolingDaemonSupport.inspectorObjectGroup,
'arg': firstWidgetId,
},
);

// Confirm we can get the selected widget from the MCP tool.
final getSelectedWidgetTool = tools.singleWhere(
(t) =>
t.name == DartToolingDaemonSupport.getSelectedWidgetTool.name,
);
final getSelectedWidgetResult = await testHarness.callToolWithRetry(
CallToolRequest(name: getSelectedWidgetTool.name),
);
expect(getSelectedWidgetResult.isError, isNot(true));
expect(
(getSelectedWidgetResult.content.first as TextContent).text,
contains('MyApp'),
);
});

test('when there is no selected widget', () async {
await testHarness.startDebugSession(
counterAppPath,
'lib/main.dart',
isFlutter: true,
);
final tools =
(await testHarness.mcpServerConnection.listTools()).tools;
final getSelectedWidgetTool = tools.singleWhere(
(t) =>
t.name == DartToolingDaemonSupport.getSelectedWidgetTool.name,
);
final getSelectedWidgetResult = await testHarness.callToolWithRetry(
CallToolRequest(name: getSelectedWidgetTool.name),
);

expect(getSelectedWidgetResult.isError, isNot(true));
expect(
(getSelectedWidgetResult.content.first as TextContent).text,
contains('No Widget selected.'),
);
});
});
});
});
}