Language-Agnostic LSP Source Concept

Creating a generic `LSPSource` type that can connect to any Language Server Protocol (LSP) server would be much more versatile and language-agnostic. This approach would make the Context Generator even more powerful by supporting multiple programming languages through their respective LSP servers.

The idea would be to create a source that:

1. Connects to any LSP-compliant server
2. Sends standardized LSP requests for symbols, definitions, references, etc.
3. Provides filtering capabilities for the returned symbols
4. Formats the results for context generation, regardless of language

## Architecture

Here's a planned file structure for the implementation:

```
src/
└── Source/
    └── Lsp/
        ├── LspSource.php                 # Main source class
        ├── LspSourceFetcher.php          # Fetcher for LSP sources
        ├── Client/
        │   ├── LspClientInterface.php    # Interface for LSP clients
        │   ├── TcpLspClient.php          # TCP implementation
        │   ├── StdioLspClient.php        # STDIO implementation
        │   └── LspRequest.php            # Value object for LSP requests
        ├── Model/
        │   ├── LspSymbol.php             # Represents a symbol from LSP
        │   ├── LspLocation.php           # Represents a location in a file
        │   └── LspResponse.php           # Wraps LSP server responses
        └── Filter/
            ├── LspFilterInterface.php    # Interface for LSP-specific filters
            ├── SymbolKindFilter.php      # Filter by symbol kind
            ├── SymbolNameFilter.php      # Filter by symbol name
            └── LocationFilter.php        # Filter by file location
```

## Class Hierarchy

```
SourceInterface
└── BaseSource
    └── SourceWithModifiers
        └── LspSource

SourceFetcherInterface
└── LspSourceFetcher

FilterableSourceInterface
└── LspSource
```

## Key Components

### `LspSource` Class

The main source class would be language-agnostic and focus on:

- LSP server connection parameters (URI, transport method)
- Root workspace path
- Symbol query parameters
- Filter criteria for symbols
- Formatting options

### `LspClientInterface` and Implementations

These would handle the communication protocol details:

- `TcpLspClient`: Connects to LSP servers over TCP
- `StdioLspClient`: Connects to LSP servers via standard I/O
- Both would implement JSON-RPC protocol required by LSP

### LSP Requests

The source would primarily use these LSP methods:

- `initialize`: Set up connection with server
- `workspace/symbol`: Get symbols across the workspace
- `textDocument/documentSymbol`: Get symbols in a specific document
- `textDocument/references`: Find all references to a symbol
- `textDocument/definition`: Find the definition of a symbol

### Symbol Filtering

Filtering would be based on LSP's standard symbol properties:

- Symbol kind (class, method, variable, etc.)
- Symbol name (exact match, pattern, regex)
- Container name (parent class/namespace)
- File location (path patterns)

## Configuration Example

Here's how the configuration might look in a `context.json` file:

```json
{
  "type": "lsp",
  "description": "All service classes in the project",
  "lspServer": {
    "type": "tcp",
    "uri": "localhost:12345",
    "language": "php",
    "initializationOptions": {
      "storagePath": "/tmp/lsp"
    }
  },
  "rootPath": "${PROJECT_ROOT}",
  "query": {
    "type": "workspace/symbol",
    "query": "*Service"
  },
  "filters": {
    "symbolKinds": ["class"],
    "exclude": ["*Test*", "*Mock*"],
    "include": ["src/Service/**"]
  },
  "format": {
    "includeDefinition": true,
    "includeReferences": false,
    "showDocumentation": true
  },
  "modifiers": []
}
```

## Benefits of This Approach

1. **Language Agnostic**: Works with any LSP-compliant server (PHP, JavaScript, Python, etc.)
2. **Rich Semantic Information**: Gets symbol data with full type information
3. **Standardized Protocol**: LSP is widely adopted and well-documented
4. **Future-Proof**: New LSP features can be incorporated as they become available
5. **IDE Integration**: Works with the same tools many developers already use in their IDEs

## Implementation Considerations

1. **Server Management**:
    - Should the source start/stop LSP servers, or connect to existing ones?
    - How to handle server crashes or disconnections?

2. **Performance**:
    - LSP operations can be expensive for large workspaces
    - Consider implementing caching mechanisms

3. **Authentication**:
    - Some LSP servers might require authentication
    - How to securely store and use credentials?

4. **Transport Mechanisms**:
    - Support TCP, STDIO, and possibly WebSockets
    - Handle different serialization formats (JSON-RPC is standard)

5. **Cross-Platform Compatibility**:
    - Ensure it works on different operating systems
    - Handle path differences between OS's

### Diagram

A sequence diagram that illustrates the interactions between our LSP source components and an LSP server. This will help visualize the flow of operations during the context generation process.

```mermaid
sequenceDiagram
    participant User
    participant ContextGenerator
    participant LspSource
    participant LspSourceFetcher
    participant LspClient
    participant LSPServer
    
    User->>ContextGenerator: Run with LSP source config
    ContextGenerator->>LspSource: Create source
    LspSource->>LspSourceFetcher: Register for fetching
    
    ContextGenerator->>LspSourceFetcher: Fetch content
    
    LspSourceFetcher->>LspClient: Initialize client
    LspClient->>LSPServer: initialize request
    LSPServer->>LspClient: initialize response
    
    LspClient->>LSPServer: workspace/symbol request
    Note over LspClient,LSPServer: Query symbols based on criteria
    LSPServer->>LspClient: workspace/symbol response
    
    alt For each interesting symbol
        LspClient->>LSPServer: textDocument/definition request
        LSPServer->>LspClient: textDocument/definition response
        
        opt If includeReferences is true
            LspClient->>LSPServer: textDocument/references request
            LSPServer->>LspClient: textDocument/references response
        end
    end
    
    LspSourceFetcher->>LspSourceFetcher: Apply filters to symbols
    
    LspSourceFetcher->>LspClient: Get file contents
    LspClient->>LSPServer: textDocument/documentSymbol request
    LSPServer->>LspClient: textDocument/documentSymbol response
    
    LspSourceFetcher->>ContextGenerator: Return formatted content
    
    LspSource->>LspClient: Shutdown connection
    LspClient->>LSPServer: shutdown request
    LSPServer->>LspClient: shutdown response
    LspClient->>LSPServer: exit notification
    
    ContextGenerator->>User: Return generated context
```

# LSP Source Features

Here's a comprehensive list of features that could be implemented in the `LspSource` type, along with descriptions and practical use cases for each feature:

## 1. Symbol Discovery

**Description**: Query and retrieve symbols (classes, functions, variables) from the entire workspace based on name patterns or other criteria.

**Use Cases**:

- Find all service classes (`*Service`) in a project
- Extract all controller actions for documentation
- Locate all implementation classes of a specific interface

## 2. Definition Resolution

**Description**: Locate precise definitions of symbols, including their exact file location, content, and documentation.

**Use Cases**:

- Extract the complete implementation of a specific class
- Generate documentation for library APIs
- Find the original declaration of an overridden method

## 3. Reference Tracking

**Description**: Find all usages and references to a specific symbol throughout the codebase.

**Use Cases**:

- Identify all places where a specific service is used
- Gauge the impact of changing a method signature
- Extract all files involved in a particular feature

## 4. Inheritance Analysis

**Description**: Determine inheritance relationships between classes, interfaces, and traits.

**Use Cases**:

- Generate class hierarchy diagrams
- Extract all classes in an inheritance tree
- Document interface implementations

## 5. Symbol Filtering

**Description**: Apply various filters to refine the set of symbols included in the context.

**Use Cases**:

- Include only public methods for documentation
- Focus on recently modified symbols
- Filter by visibility, return type, parameters, etc.

## 6. Multi-language Support

**Description**: Connect to LSP servers for different programming languages to support polyglot projects.

**Use Cases**:

- Generate context for full-stack features (PHP backend + JS frontend)
- Document interactions between different language components
- Extract symbols from configuration files (JSON, YAML) using their LSP servers

## 7. Dependency Resolution

**Description**: Analyze and extract dependencies between files and symbols.

**Use Cases**:

- Generate a complete feature context with all relevant dependencies
- Identify tight coupling between components
- Document service injection patterns

## 8. Documentation Extraction

**Description**: Extract documentation comments (PHPDoc, JSDoc, etc.) for symbols.

**Use Cases**:

- Generate API documentation
- Provide context on parameter usage and return values
- Include examples and usage notes for LLM prompts

## 9. Code Fragment Extraction

**Description**: Retrieve specific fragments of code rather than entire files.

**Use Cases**:

- Extract only the relevant method implementations
- Focus on hot spots of complex logic
- Provide specific algorithms without surrounding boilerplate

## 10. Semantic Search

**Description**: Search for symbols based on semantic properties rather than just text patterns.

**Use Cases**:

- Find all methods that return a specific type
- Locate classes with certain attributes or annotations
- Identify functions with specific parameter types

## 11. Impact Analysis

**Description**: Analyze the potential impact of changes to specific symbols.

**Use Cases**:

- Generate context for all code affected by a planned change
- Identify potential breaking changes
- Document ripple effects of API modifications

## 12. Cross-reference Visualization

**Description**: Visualize relationships between symbols and files.

**Use Cases**:

- Generate dependency graphs for context
- Visualize call hierarchies
- Map data flow between components

## 13. Custom Query Support

**Description**: Allow custom LSP queries for specialized information retrieval.

**Use Cases**:

- Retrieve language-specific metadata
- Utilize extensions provided by specific LSP implementations
- Support specialized static analysis results

## 14. Content Formatting

**Description**: Format extracted content according to different templates or styles.

**Use Cases**:

- Generate markdown documentation
- Create code snippets with proper syntax highlighting
- Produce structured overview documents

## 15. Workspace Navigation

**Description**: Navigate and explore the workspace structure through LSP.

**Use Cases**:

- Generate directory structure overviews
- Provide context about project organization
- Document module boundaries


## Config example

```json
{  
  "settings": {  
    "lspServers": {  
      "default": "php-default",  
      "php-default": {  
        "type": "tcp",  
        "uri": "127.0.0.1:6009",  
        "language": "php",  
        "initializationOptions": {  
          "storagePath": "/tmp/lsp"  
        }  
      }  
    }  
  },  
  "documents": [  
    {  
      "description": "Generator loaders",  
      "outputPath": "api/loaders.md",  
      "modifiers": [  
        "api-docs"  
      ],  
      "sources": [  
        {  
          "type": "lsp",  
          "description": "Related Classes and Interfaces",  
          "query": {  
            "type": "custom/referenceAnalysis",  
            "targetFile": "file://${PROJECT_ROOT}/src/Console/DisplayCommand.php",  
            "maxDepth": 1,  
            "includeImports": true  
          },  
          "filters": {  
            "symbolKinds": [  
              "class",  
              "interface"  
            ],  
            "include": [  
              "src/**"  
            ]  
          },  
          "format": {  
            "includeDefinition": false,  
            "includeReferences": false  
          },  
          "postProcessing": {  
            "filterByReference": "src/Console/DisplayCommand.php"  
          }  
        }  
      ]  
    }  
  ]  
}
```

Language-Agnostic LSP Source Concept #60

Description

Architecture

Class Hierarchy

Key Components

LspSource Class

LspClientInterface and Implementations

LSP Requests

Symbol Filtering

Configuration Example

Benefits of This Approach

Implementation Considerations

Diagram

LSP Source Features

1. Symbol Discovery

2. Definition Resolution

3. Reference Tracking

4. Inheritance Analysis

5. Symbol Filtering

6. Multi-language Support

7. Dependency Resolution

8. Documentation Extraction

9. Code Fragment Extraction

10. Semantic Search

11. Impact Analysis

12. Cross-reference Visualization

13. Custom Query Support

14. Content Formatting

15. Workspace Navigation

Config example

Activity

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Participants

Issue actions

`LspSource` Class

`LspClientInterface` and Implementations