🚀 Roadmap to v1.0: Stability, Scalability, and Live Observability

[apiad]

Description:

This tracking issue consolidates the architectural improvements required to move Ferret from v0.1.0 to a production-ready v1.0. The primary goals are to remove fragile instrumentation logic, ensure non-blocking writes to BeaverDB, and transform Ferret into a real-time observability platform.

Core Architecture & Stability

• Issue Implement Adaptive Rate Limiting (Throughput-Based Sampling) #2: Replace fragile builtins injection with a robust singleton accessor pattern.
• Issue Implement Streaming Analysis to Prevent OOM on Large Datasets #3: Move BeaverDB writes to a background thread to prevent blocking the application loop.
• Issue Implement Non-Blocking Write Buffer with Background Worker #6: Implement throughput-based adaptive rate limiting to control overhead.

Analysis & Visualization

• Issue Enable Profiling of Installed Modules via CLI (-m flag) #4: Refactor reporting to use streaming iterators to handle large datasets without OOM.
• Issue Refactor Instrumentation to Remove "Magical" Builtins Injection #5: Export traces to Speedscope JSON format for flame graph visualization.

Live Observability

• Issue Implement Live Monitoring System (ferret watch) #8: Create ferret watch TUI for real-time monitoring of active spans and metrics.
• Issue Deadlock & Stall Detection (The Watchdog) #9: Add watchdog to detect and report stalled spans or deadlocks.
• Issue Capture Thread and Process Context in Spans #10: Capture PID and Thread ID in spans to visualize concurrency.

Developer Experience

• Issue Implement Flame Graph Export (Speedscope Support) #7: Support ferret run -m <module> to profile installed CLI tools easily.