Skip to content

fix(workflow): resolve FileLock path from file URI correctly - #2337

Open
Misty_Cirque (Cyber-Marty) wants to merge 1 commit into
microsoft:mainfrom
Cyber-Marty:fix/filelock-uri-path
Open

fix(workflow): resolve FileLock path from file URI correctly#2337
Misty_Cirque (Cyber-Marty) wants to merge 1 commit into
microsoft:mainfrom
Cyber-Marty:fix/filelock-uri-path

Conversation

@Cyber-Marty

@Cyber-Marty Misty_Cirque (Cyber-Marty) commented Sep 2, 2026

Copy link
Copy Markdown

MLflowExpManager._get_or_create_exp (qlib/workflow/expm.py:236) built its FileLock path as:

python
Path(os.path.join(pr.netloc, pr.path.lstrip("/"), "filelock"))

For any absolute file:// URI — including the default URI that qlib.config.QSettings constructs ("file:" + str(Path(os.getcwd()).resolve() / "mlruns")) — urlparse returns netloc="" and path="/absolute/...", so lstrip("/") turns the absolute path into a CWD-relative one. The lock then points under whatever directory the process happens to be in instead of the URI's location, and no longer serializes runs that share the same URI from different working directories.

On Windows, file:///C:/... drive-letter URIs only kept working by accident: after lstrip("/") the remaining C:/... is still absolute under NT path semantics, but the construction is fragile.

Fix

Build the lock path with urllib.request.url2pathname(pr.path), which converts a URI path to a filesystem path correctly on every platform (including Windows drive letters) and leaves relative URIs (file:mlruns) relative, preserving today's behavior for that shape.

URI shape: file:/Users/me/mlruns
before (POSIX): Users/me/mlruns/filelock (relative, CWD-dependent)
after (POSIX): /Users/me/mlruns/filelock
after (Windows): —
────────────────────────────────────────
URI shape: file:///Users/me/mlruns
before (POSIX): Users/me/mlruns/filelock (relative)
after (POSIX): /Users/me/mlruns/filelock
after (Windows): —
────────────────────────────────────────
URI shape: file:///C:/Users/me/mlruns
before (POSIX): C:/Users/me/mlruns/filelock (absolute by accident)
after (POSIX): /C:/Users/me/mlruns/filelock*
after (Windows): C:\Users\me\mlruns\filelock
────────────────────────────────────────
URI shape: file:mlruns
before (POSIX): mlruns/filelock
after (POSIX): unchanged
after (Windows): unchanged

*on POSIX the drive-letter form resolves as an absolute POSIX path; on Windows url2pathname produces the correct drive path (verified in the new tests).

Tests

New tests/workflow_tests/test_exp_manager_lock_path.py:

  • absolute file URIs produce an absolute lock path equal to /filelock (POSIX semantics, exercised on Linux CI)
  • Windows drive-letter URIs map to the correct drive path (Windows-only test)
  • relative file:mlruns URIs stay relative (behavior preserved)
  • end-to-end: the lock resolves inside the directory the URI points to, on the running platform

All new and existing workflow tests pass locally (tests/test_workflow.py + new file, 5 passed).

Fixes #2252 (primary defect). The _log_uncommitted_code stderr/shell=True issues from the same report will be addressed in a follow-up PR.

MLflowExpManager._get_or_create_exp built the FileLock path with
os.path.join(pr.netloc, pr.path.lstrip('/'), 'filelock'). For absolute
file URIs (including the default URI QSettings generates), lstrip('/')
stripped the leading slash and turned the lock path CWD-relative, so
the lock no longer serialized runs sharing the same URI from different
working directories. On Windows, drive-letter URIs (file:///C:/...)
only worked by accident under NT path semantics.

Build the lock path with urllib.request.url2pathname(pr.path) instead,
which converts the URI path to a filesystem path correctly on all
platforms and keeps relative URIs (file:mlruns) relative as before.

Fixes microsoft#2252 (primary defect; the _log_uncommitted_code issues in the
same report will be addressed separately).
@Cyber-Marty

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

qlib.workflow: FileLock and _log_uncommitted_code both unsafe vs CWD

1 participant