Skip to content

Various fixes#78

Open
clemsgrs wants to merge 6 commits intomainfrom
codex/various-fixes
Open

Various fixes#78
clemsgrs wants to merge 6 commits intomainfrom
codex/various-fixes

Conversation

@clemsgrs
Copy link
Owner

@clemsgrs clemsgrs commented Mar 18, 2026

Summary

  • optimize embedding throughput with batched loading, batch timing instrumentation, async prefetch/copy, and region-aware batch preprocessing
  • switch embedding reads to pre-extracted tar-backed tile archives via tiling.save_tiles and TarTileReader, while keeping HS2P coordinate artifacts as the canonical tiling metadata
  • rename process-list and artifact references from tiles_* coordinate files to coordinates_*, update docs, and add focused regression coverage for the new tile-store path

Validation

  • COVERAGE_FILE=/tmp/slide2vec-region-final.coverage ~/Code/venv/slide2vec/bin/python -m pytest -q tests/test_dependency_split.py tests/test_regression_inference.py tests/test_progress.py -k 'region_batch_preprocessor or region_models or serialize_execution or uses_batched_loader_knobs or batch_timing or run_forward_pass or dependency_split'
  • COVERAGE_FILE=/tmp/slide2vec-backend-split.coverage ~/Code/venv/slide2vec/bin/python -m pytest -q tests/test_regression_core.py tests/test_regression_inference.py -k 'execution_options_from_config or execution_options_with_output_dir or serialize_execution or build_hs2p_configs or resolve_embedding_backend or uses_batched_loader_knobs'
  • COVERAGE_FILE=/tmp/slide2vec-push.coverage ~/Code/venv/slide2vec/bin/python -m pytest -q tests/test_hs2p_package_cutover.py -k 'load_process_df_requires_hs2p_process_list_columns'
  • ~/Code/venv/slide2vec/bin/python -m py_compile slide2vec/api.py slide2vec/data/tile_store.py slide2vec/data/dataset.py slide2vec/inference.py slide2vec/utils/tiling_io.py tests/test_tile_store.py

Notes

  • tests/test_tile_store.py was added for the tar-backed reader, but this local venv is missing PIL, so I did not run that file here.
  • .claude/settings.local.json remains local and was not included in the branch.

@clemsgrs clemsgrs changed the title [codex] various-fixes Various fixes Mar 18, 2026
@clemsgrs clemsgrs marked this pull request as ready for review March 18, 2026 23:41
@clemsgrs clemsgrs force-pushed the codex/various-fixes branch from 60822b9 to 01e8bdf Compare March 20, 2026 14:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant