Skip to content

Conversation

azzazzel
Copy link

Problem

The index creation process was complex and required users to manually specify many configuration parameters. It also required users to know the interdependencies between different flags, making it difficult for new users to get started quickly. There was no way to discover available embedding models, and users had to know the exact model names and their requirements. Additionally, there was no global option to skip confirmation prompts, making automation and scripting cumbersome.

Solution

This PR introduces intelligent index creation with automatic value inference and comprehensive model management capabilities. The changes include:

  • Smart Value Inference: Automatically infers missing index creation parameters based on provided values and available models from the API
  • Model Discovery: Added pc models command to list available embedding models with their specifications and requirements
  • Intelligent Defaults: Provides sensible defaults for common configurations (AWS us-east-1, cosine metric, etc.) while allowing full customization
  • Model-Based Configuration: Automatically configures index parameters based on selected embedding models, including dimensions, vector types, and field mappings
  • Global --assume-yes Flag: Added -y/--assume-yes global flag to skip all confirmation prompts for automation and scripting
  • Enhanced Validation: Improved validation with better error messages and support for inferred values
  • Caching System: Implemented local caching for model data to improve performance and reduce API calls
  • Simplified Command Structure: Streamlined index creation with clearer flag organization and better examples

The implementation maintains full backward compatibility while significantly improving the user experience for both beginners and advanced users.

Type of Change

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update
  • Bug fix (non-breaking change which fixes an issue)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infrastructure change (CI configs, etc)
  • Non-code change (docs, etc)
  • None of the above: (explain here)

Test Plan

  1. Model Discovery Testing: Test the new models command:

    # List available models
    pc models
    
    # List models with JSON output
    pc models --json
    
    # Force refresh from API
    pc models --no-cache
  2. Intelligent Index Creation Testing: Test automatic value inference:

    # Minimal serverless index (should infer defaults)
    pc index create my-index
    
    # With model specification (should infer model-specific settings)
    pc index create my-index --model llama-text-embed-v2
    
    # With default model aliases
    pc index create my-index --model dense
    pc index create my-index --model sparse
    
    # Pod index with minimal config (should infer defaults)
    pc index create my-pod --pod
  3. Global Assume-Yes Flag Testing: Test automation-friendly behavior:

    # Create index without confirmation prompts
    pc index create my-index -y
    
    # Delete index without confirmation
    pc index delete my-index -y
    
    # Test with other commands that have confirmations
    pc project delete <project> -y
  4. Validation Testing: Test improved validation and error handling:

    # Test invalid model names
    pc index create test --model invalid-model
    
    # Test conflicting configurations
    pc index create test --pod --serverless
  5. Backward Compatibility Testing: Ensure existing commands still work:

    # Test explicit configuration (should work as before)
    pc index create my-index --dimension 1536 --metric cosine --cloud aws --region us-east-1

This PR builds upon the previous #36 style improvements and contains all changes in that branch as well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant