Allow passing custom estimation_options in localize_inloc.py

[ayoussf]

Allows passing custom estimation_options to localize_inloc.py, while maintaining the default behaviour if not provided.

I have also modified the Markdown in pipeline_InLoc.ipynb to specify additional details. If this is unwanted, I can revert to the original markdown.

[ayoussf]

Hi @Phil26AT,

I wanted to share a few observations regarding recent PyColmap updates (starting around v3.10.0 and later). Since these updates, I’ve been unable to reproduce InLoc's results, not only for SuperGlue and LightGlue but across other models I’ve tested (e.x. LoFTR)

As an example, after freshly cloning the InLoc repository and running the provided notebook without making any changes, I obtained the following results with SuperGlue:

• DUC1: 44.4 / 66.7 / 79.8
• DUC2: 50.4 / 73.3 / 77.1

These numbers were fairly consistent across multiple machines.

For context:

• PyColmap was installed via pip install, not from source.
• I tested versions v3.10.0, v3.12.0, v3.12.1, and v3.12.3.
• Python version: 3.9
• InLoc dataset downloaded from (Only DUC1 and DUC2 data): https://data.ciirc.cvut.cz/public/projects/2020VisualLocalization/InLoc/
• Alignment data downloaded from (Only DUC1 and DUC2 transformations): https://cvpr17.wijmans.xyz/data/

Interestingly, when replacing pycolmap.estimate_and_refine_absolute_pose with poselib.estimate_absolute_pose, I saw improved results on DUC2:

• DUC1: 44.9 / 67.7 / 79.3
• DUC2: 56.5 / 77.1 / 78.6

Moreover, I can still reliably reproduce results on Aachen v1.1, where I got the following with default settings:

SuperGlue:

• Day: 90.3 / 96.2 / 99.4
• Night: 76.4 / 90.1 / 100.0

LightGlue:

• Day: 90.3 / 96.2 / 99.2
• Night: 77.5 / 91.6 / 99.5

Thus, I am not entirely sure the reason for this behaviour on the InLoc dataset specifically.

I hope this helps and I’m happy to run additional tests if needed.

[Phil26AT]

Hi @ayoussf, thank you for reopening this PR.

Thank you for reporting detailed statistics on InLoc, and great that Aachen v1.1 is reproducible again. I was rerunning the pipeline (SP+SG) without changes, and also got similar results. However, results with the temporal pairs (used in the leaderboard) were reproducible up to ~2%:

Methods	InLoc DUC1	InLoc DUC2	Retrieval
Leaderboard	46.5 / 65.7 / 78.3	52.7 / 72.5 / 79.4	NetVLAD top 40
Pycolmap / 3.12.0	43.9 / 66.2 / 79.3	51.1 / 74.0 / 77.1	NetVLAD top 40
Leaderboard	49.0 / 68.7 / 80.8	53.4 / 77.1 / 82.4	NetVLAD top 40 (temporal)
Pycolmap / 3.12.0	47.0 / 68.2 / 80.3	52.7 / 77.9 / 80.2	NetVLAD top 40 (temporal)

Note that this dataset is fairly small and that pose estimation is non-deterministic (I get fluctuations of 1-2%), so small differences might not be significant.

I checked the backlog, and realized that some default parameters in the estimation options changed, e.g.: old vs. new. It might be worth trying those.

Could you maybe post the last pycolmap version where results were reproducible for you, together with the actual numbers?

I will also try to run the pipeline with older versions over the next days.

[ayoussf]

@Phil26AT Thank you for the detailed response.

Unfortunately, I cannot pinpoint the exact pycolmap version I used, as I have since switched machines and no longer have access to the old environment to fully reproduce it. What I can say with certainty is that it was prior to v3.11.0.

Over the coming days, I will rerun the evaluations for SP+SG and SP+LG across pycolmap versions 0.6.0 through 3.12.3. Starting from v3.11.0, I will also include results using both the old and new estimation options for consistency. I will share the updated results in a new comment on this PR.

Lastly, I am aware there is a notebook for InLoc evaluation; however, if you prefer, I can create \hloc\pipelines\InLoc and add a pipeline.py script to stay consistent with the other dataset evaluations.