Refactor #line processing - keep the original positions in the AST

[Martin521]

Description

Fixes #18553.
Breaking change.

This PR is non-breaking for two of the three #line use cases mentioned in #18553.
It keeps the original ranges in the AST, stores the #line directives in a table and provides a method range.ApplyLineDirectives(). This method is applied in the following places

• construction of runtime diagnostics in pattern match compilation
• emitting debug information in quotation translation
• creating IL source markers (debug information)
• formatting fsc compiler diagnostics (CompilerDiagnostics.FormatDiagnosticLocation)
• creating service diagnostics (FSharpDiagnostics.CreateFromException)

This should make sure that all debugging information (in .pdb) and all diagnostics stay unchanged.

The third use case, symbol positions for IDE editors, is more tricky. FSharpSymbol has no range field or property that I could apply the transformation to, but just contains (and makes public) the AST items with their original ranges.
So, I propose we leave it like this and call ApplyLineDirectives in the appropriate places in FSAC / the editors.
This will also enable better line directive support in the editors (for some use cases you need the original ranges).
For Ionide / FSAC I found that calling ApplyLineDirectives in a single place (in FSAC's fcsRangeToLspLocation) will most probably be sufficient to support the current functionality.
I don't know what it means for the other editors.
Any thoughts or recommendations here? (@TheAngryByrd @auduchinok @abonie )

List of breaking changes

• For fsc and fcs users
  • __LINE__ and __SOURCE_FILE__ show original locations now
• For fcs users only (tooling)
  • AST has original ranges now (ignoring #line directives) (so, also FSharpSymbol points to its original range)

Checklist

• Test cases added
• Performance benchmarks added in case of performance changes
• Release notes entry updated

[github-actions]

❗ Release notes required

---

✅ Found changes and release notes in following paths:

Change path	Release notes path	Description
src/Compiler	docs/release-notes/.FSharp.Compiler.Service/10.0.100.md

[Martin521]

This is ready for review. See updated PR description.
It introduces no changes for compiler users (except for the hardly used __LINE__ / __SOURCE_FILE__ in combination with #line).
For tooling, see the remarks above.

[Martin521]

@T-Gro
Is there anything I can do to support reviewing this PR?

[TheAngryByrd]

I expect this to not create any changes to the fsc output. I also assume there are no consumers of the compiler service that rely on having the target ranges in the AST. For them, this would be a breaking change. They would need to apply the map when dealing with these ranges.

Sorry for not getting around to responding sooner. We probably do with FSAC QuickFixes or possibly some analyzers but with all changes to FCS, we adapt. It might be enlightening to see what tests fail with a build of this against FSAC or FSharp.Analyzers.SDK/Ionide-Analyzers/GResearch's analyzers.

[Martin521]

Sorry for not getting around to responding sooner. We probably do with FSAC QuickFixes or possibly some analyzers but with all changes to FCS, we adapt. It might be enlightening to see what tests fail with a build of this against FSAC or FSharp.Analyzers.SDK/Ionide-Analyzers/GResearch's analyzers.

Good to hear. Thanks for looking into it. I will try to run the tests and report back.

[Martin521]

Here is what I found out.

Good news is that goto-symbol is working correctly with the FCS of this PR and the fix in FSAC's fcsRangeToLspLocation that I mentioned in the OP.

I also ran FsAutoComplete.Tests.Lsp

• In my environment (fresh dev container), this always (also with the current unchanged FSAC from the repo) ends up hanging in test FSAC.lsp.Ionide WorkspaceLoader.TransparentCompiler.empty file features.tests.type 'c' for checking error and autocompletion starts with 'abs'. So I can only report about the first 5486 tests. Running with the unchanged FSAC, they all pass.
• When building FSAC with the FCS from the current dotnet/fsharp repo's main branch and running the tests, I see random failures of one or more of the CodeFix-tests.AdjustConstant tests. I see the same when using the FCS of this PR. No extra failures from the changes of this PR. Which is somehow to be expected because there is no #line in any of the tests.

[edgarfgp]

@T-Gro @abonie Can we please get a review here. ?

[Copilot]

Pull Request Overview

This PR refactors #line directive processing to fix issue #18553. It changes the system to keep original positions in the AST while storing line directives separately, providing a new ApplyLineDirectives() method for when transformed positions are needed.

Key changes:

• Original ranges are preserved in the AST, with line directives stored in a separate table
• The __LINE__ and __SOURCE_FILE__ intrinsics now show original positions instead of transformed ones
• Debug information and diagnostics continue to use transformed positions via ApplyLineDirectives()

Reviewed Changes

Copilot reviewed 30 out of 30 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/fsharpqa/Source/Conformance/SpecialAttributesAndTypes/Imported/CallerInfo/*.fs	Update test expectations to reflect that CallerLineNumber now uses original line numbers
tests/fsharp/core/syntax/test.fsx	Remove extensive line directive tests that no longer apply to the new behavior
tests/ILVerify/*.bsl	Update IL verification baselines to reflect line number changes in generated code
tests/FSharp.Compiler.Service.Tests/ProjectAnalysisTests.fs	Update test to use ApplyLineDirectives() for transformed ranges
tests/FSharp.Compiler.Service.Tests/*.bsl	Add ApplyLineDirectives method to surface area
tests/FSharp.Compiler.ComponentTests/CompilerDirectives/*.fs	Update/add tests for new line directive behavior
src/Compiler/lex.fsl	Replace WarnScopes.RegisterLineDirective with LineDirectiveStore.SaveLineDirective
src/Compiler/Utilities/range.*	Add ApplyLineDirectives method and LineDirectives module for storing directives
src/Compiler/SyntaxTree/WarnScopes.*	Remove line directive tracking and mapping logic
src/Compiler/SyntaxTree/*.fs	Add LineDirectiveStore module and update position handling
src/Compiler/Facilities/prim-lexing.*	Remove OriginalLine and ApplyLineDirective from Position type
src/Compiler/Driver/*.fs	Update to call ApplyLineDirectives for diagnostics and add line directive storage
src/Compiler/Checking/*.fs	Apply line directives for debug info and runtime diagnostics
docs/release-notes/	Document the breaking change