feat(workers): Implement comprehensive test coverage for all background workers

[jwaldrip]

Summary

Implements comprehensive test coverage for all three CrystalShards background workers:

• IndexShardWorker
• BuildDocsWorker
• UpdateDependenciesWorker

All tests use mocks to avoid external dependencies (GitHub API, MinIO, Git), enabling fast, reliable test execution without network or infrastructure requirements.

Test Coverage Details

IndexShardWorker (12 tests)

Success Paths:

• ✅ Processes shard metadata from shard.yml correctly
• ✅ Fetches and updates GitHub metadata (stars, forks, sync time)
• ✅ Enqueues UpdateDependenciesWorker after indexing
• ✅ Enqueues BuildDocsWorker after indexing

Error Handling:

• ✅ Handles non-existent shards gracefully
• ✅ Handles non-existent shard versions gracefully
• ✅ Handles missing shard.yml gracefully
• ✅ Handles provider fetch errors gracefully

Idempotency:

• ✅ Can be run multiple times safely without data duplication

BuildDocsWorker (14 tests)

Success Paths:

• ✅ Builds docs and updates documentation_url
• ✅ Uploads documentation files to MinIO
• ✅ Handles different repository versions correctly
• ✅ Checks out specific commit SHA when available

Error Handling:

• ✅ Handles non-existent shards gracefully
• ✅ Handles non-existent shard versions gracefully
• ✅ Handles git clone failures gracefully
• ✅ Handles crystal docs build failures gracefully
• ✅ Handles MinIO upload failures gracefully
• ✅ Cleans up temporary directories after build

Idempotency:

• ✅ Can rebuild documentation multiple times safely

Edge Cases:

• ✅ Handles repositories without shard dependencies
• ✅ Handles specific version checkout with commit SHA

UpdateDependenciesWorker (16 tests)

Success Paths:

• ✅ Parses and stores dependencies from shard.yml metadata
• ✅ Handles dependencies with different version specification formats
• ✅ Links dependencies to dependent shards when they exist
• ✅ Correctly separates runtime and development dependencies

Error Handling:

• ✅ Handles non-existent shards gracefully
• ✅ Handles non-existent shard versions gracefully
• ✅ Handles shards with no dependencies
• ✅ Handles shard versions with no metadata
• ✅ Handles shards with only development dependencies

Idempotency:

• ✅ Can be run multiple times safely
• ✅ Replaces old dependencies when re-run

Edge Cases:

• ✅ Handles empty dependencies object
• ✅ Handles dependencies with special characters in names
• ✅ Handles large number of dependencies (25+)
• ✅ Handles version requirements without constraint operators
• ✅ Builds correct dependency relationships

Mock Infrastructure

Created spec/support/worker_mocks.cr with:

• MockProvider: Simulates GitHub/GitLab/etc providers without network calls
• MockStorageService: Simulates MinIO operations
• Test Helpers: For stubbing git commands and tracking worker enqueuing

Files Changed

• spec/support/worker_mocks.cr - New mock infrastructure
• spec/workers/index_shard_worker_spec.cr - Comprehensive tests (12 cases)
• spec/workers/build_docs_worker_spec.cr - Comprehensive tests (14 cases)
• spec/workers/update_dependencies_worker_spec.cr - Comprehensive tests (16 cases)
• spec/spec_helper.cr - Added worker_mocks require

Testing Approach

All tests follow these principles:

1. No External Dependencies: Mocks for GitHub API, MinIO, Git operations
2. Fast Execution: No network calls or file system operations (where possible)
3. Comprehensive Coverage: Success paths, error handling, edge cases
4. Idempotency Verification: Workers can safely run multiple times
5. Worker Chaining: Verify workers enqueue downstream workers correctly

Test Execution

Tests can be run with:

crystal spec spec/workers/

All tests use database transactions for isolation and cleanup.

Quality Assurance

• ✅ All tests have clear, descriptive names
• ✅ Both success and failure scenarios tested
• ✅ Edge cases covered (empty data, special characters, large datasets)
• ✅ No always-passing tests - all assertions verify real behavior
• ✅ Code formatted with crystal tool format

Closes #31

🤖 Generated with Claude Code

Co-Authored-By: Claude [email protected]