Skip to content

Conversation

@guibou
Copy link
Collaborator

@guibou guibou commented May 26, 2025

(only latests commit are relevant, the other are from other MR which should be merged before this work is merged. Or we can rebase)

This is related to #4598.

This changes the file to module associating logic done during dependency
graph building.

Before, each time a module Foo.Bar is found, HLS is testing inside all
the import path for the existence of a relevant fiel.. It means that for
i import paths and m modules to locate, m * n filesystem
operations are done. Note also that this involves a lot of complex
string concatenation primitive to build the FilePath.

A module is tested for each import for each of the file of the
project. We also test for boot files, doubling the number of test.

In #4598 we have a project with 1100 modules, in more than 250 import
paths and we count more than 17000 import statments, resulting on
over 6 millions test for file existences. This project was blocking for
more than 3 minutes during HLS startup.

This commit changes the way this is computed:

  • At startup, a Map ModuleName FilePath (the real type is a bit more
    involved for performance, multiples unit and boot files handling) is
    built by scanning all the import paths for files representing the
    different modules.
  • Directory scanning is efficient and if import path only contains
    haskell module, this will never do more job that listing the files of
    the project.
  • The lookup is now simplify a Map lookup.

The performance improvement is as follows:

  • The number of IO operation is dramatically reduced, from multiples
    millions to a few recursive directories listing.
  • A lot of the boilerplate of converting path had be removed.
  • TODO: add an RTS stats before / after with number of allocations
  • On my project, the graph building time is reduced from a few minutes
    to 3s.

Limitations:

  • How to rebuild the Map if the content of one directory change?
  • If one directory is filled with millions of files which are not of
    interested, performance can be damaged. TODO: add a diagnostic during
    this phase so the user can learn about this issue.

Code status:

  • The lookup is not fully restored, especially it does not include the
    handling of home unit as well as reexport.
  • The initialisation phase is cached inside a TVar stored as a top
    level identifier using unsafePerformIO. This is to be improved.

@guibou
Copy link
Collaborator Author

guibou commented May 28, 2025

I'm running this code in production on our large codebase for the 3rd day.

a) My users are super happy, a few quotes:

image

(it was more than 5 minutes before ;)

image

image

Ok, that's all for the shameless plug ;)

I think I observed a few reload / performance issues which are not related and would definitely do some followup issues. I'm investigating (this is difficult to describe right now. Looks like HLS is not able to give diagnostic back if it is typechecking a file which depends on the file I just modified and if this typechecking takes time).

I also observed a surprising behavior, I need to investigate if that's linked to my MR or to something else:

  • I had started HLS in a context where one of the dependencies (from the package db) was version X (which actually exposes a reexported symbol. e.g. Database.Selda which reexports Text).
  • I then had changed my working copy to a version of my code which uses a new version of Database.Selda, without the reexport of Text, and my code was updated to import Data.Text (Text) everywhere it was previously using the reexport.
  • From my editor, I restarted hls
  • Warnings started to appear in HLS about redundant import for import Data.Text (Text) (which was true with the previous version of Selda). Note that the warning appeared on files which are NOT open in the editor, but HLS got notified about the change and reevaluated them.
  • Opening the file in the editor, the warning disappears.

It looks like a multi-home issue, see. #3237, which would make sense considering that I had done nothing to fix multiple home. But this is still rather surprising because HLS was supposed to restart and hence should have completely ignored the old selda version. Maybe it picked some information from the cache.

@soulomoon
Copy link
Collaborator

soulomoon commented May 28, 2025

Maybe it picked some information from the cache

We handle FOI(File of interest) and none-FOI differently.
See getModIfaceRule. This might be causing what you see @guibou

@guibou
Copy link
Collaborator Author

guibou commented May 29, 2025

We handle FOI(File of interest) and none-FOI differently.
See getModIfaceRule. This might be causing what you see @guibou

@soulomoon Thank you. This is indeed interesting. I will confirm that the bug I'm observing is not linked to my change and open an issue. This is not a dramatic issue (e.g. it is not everyday that you restart the editor with breaking changes in the packages from the package database), but it is not convenient and can be confusing.

@guibou
Copy link
Collaborator Author

guibou commented May 31, 2025

I've found a bug, when looking for module in the import path, an exception is raised if the import path is empty.

@guibou guibou force-pushed the optimise_module_to_filename branch from 2d8386c to fdf3085 Compare September 21, 2025 11:59
@guibou
Copy link
Collaborator Author

guibou commented Sep 21, 2025

I've found a bug, when looking for module in the import path, an exception is raised if the import path is empty.

Fixed in latest commit.

@guibou guibou force-pushed the optimise_module_to_filename branch from fdf3085 to c856c6a Compare October 25, 2025 19:50
@guibou
Copy link
Collaborator Author

guibou commented Oct 25, 2025

Improved the support for currently edit files but not yet saved to the disk. The implementation will be improved for performance but that's a good first draft.

This fixs a lot of unit test, so it is good.

I do have a few (around 20) tests faliling in ghcide-tests, but otherwise, most of the tests are ok.

Still todo:

  • remove the unsafePerformIO
  • Handle multi home units (some tests are failing, that's a good news)
  • Handle reexport (same, some tests are failing, that's a good news)
  • Handle reloading of the cache if files are added / removed

Sorry, something went wrong.

@guibou guibou added the performance Issues about memory consumption, responsiveness, etc. label Nov 1, 2025
@guibou
Copy link
Collaborator Author

guibou commented Nov 1, 2025

Multiple-home-unit seems to be fixed now (maybe I've not pushed the MR), but the tests are fine. I have 3 failing tests related to -boot files and then tests will be ok.

I'll work on removing the unsafePerformIO cache (which breaks hlint test, this is great ;), maybe this cache is also responsible for the remainirg failing tests.

@guibou guibou force-pushed the optimise_module_to_filename branch from c856c6a to b20c634 Compare November 1, 2025 12:38
@guibou
Copy link
Collaborator Author

guibou commented Nov 1, 2025

Latest push should fix hlint and actually remove the unsafePerformIO: the cache is now stored in the ShakeExtras.

I have 3 (2 are flaky) remaining issues with -boot tests:

    cyclic module dependency with hs-boot:                             FAIL (0.63s)
      ghcide-test/exe/DiagnosticTests.hs:246:
      Got diagnostics for Uri {getUri = "file:///tmp/nix-shell.1Fwv9H/hls-test-root/extra-dir-80836663907422/ModuleA.hs"} but only expected diagnostics for [NormalizedUri 5530269853045372639 "
file:///tmp/nix-shell.1Fwv9H/hls-test-root/extra-dir-80836663907422/ModuleB.hs"] got [Diagnostic {_range = Range {_start = Position {_line = 1, _character = 22}, _end = Position {_line = 1, _c
haracter = 29}}, _severity = Just DiagnosticSeverity_Error, _code = Nothing, _codeDescription = Nothing, _source = Just "not found", _message = "Could not find module \8216ModuleB\8217.\nIt is
 not a module in the current program, or in any known package.", _tags = Nothing, _relatedInformation = Just [DiagnosticRelatedInformation {_location = Location {_uri = Uri {getUri = "file:///
tmp/nix-shell.1Fwv9H/hls-test-root/extra-dir-80836663907422/ModuleA.hs"}, _range = Range {_start = Position {_line = 1, _character = 22}, _end = Position {_line = 1, _character = 29}}}, _messa
ge = "GetLocatedImports"}], _data_ = Nothing}]
      Use -p '/cyclic module dependency with hs-boot/' to rerun this test only.
    bidirectional module dependency with hs-boot:                      FAIL (0.63s)
      ghcide-test/exe/DiagnosticTests.hs:269:
      Got diagnostics for Uri {getUri = "file:///tmp/nix-shell.1Fwv9H/hls-test-root/extra-dir-80836663907424/ModuleA.hs"} but only expected diagnostics for [NormalizedUri 5396647782368786105 "
file:///tmp/nix-shell.1Fwv9H/hls-test-root/extra-dir-80836663907424/ModuleB.hs"] got [Diagnostic {_range = Range {_start = Position {_line = 1, _character = 22}, _end = Position {_line = 1, _c
haracter = 29}}, _severity = Just DiagnosticSeverity_Error, _code = Nothing, _codeDescription = Nothing, _source = Just "not found", _message = "Could not find module \8216ModuleB\8217.\nIt is
 not a module in the current program, or in any known package.", _tags = Nothing, _relatedInformation = Just [DiagnosticRelatedInformation {_location = Location {_uri = Uri {getUri = "file:///
tmp/nix-shell.1Fwv9H/hls-test-root/extra-dir-80836663907424/ModuleA.hs"}, _range = Range {_start = Position {_line = 1, _character = 22}, _end = Position {_line = 1, _character = 29}}}, _messa
ge = "GetLocatedImports"}], _data_ = Nothing}]
      Use -p '/bidirectional module dependency with hs-boot/' to rerun this test only.
    correct reference used with hs-boot:                               FAIL (0.84s)
      ghcide-test/exe/DiagnosticTests.hs:295:
      Got diagnostics for Uri {getUri = "file:///tmp/nix-shell.1Fwv9H/hls-test-root/extra-dir-80836663907426/ModuleB.hs"} but only expected diagnostics for [NormalizedUri 4327465349865231065 "
file:///tmp/nix-shell.1Fwv9H/hls-test-root/extra-dir-80836663907426/ModuleC.hs"] got [Diagnostic {_range = Range {_start = Position {_line = 1, _character = 22}, _end = Position {_line = 1, _c
haracter = 29}}, _severity = Just DiagnosticSeverity_Error, _code = Nothing, _codeDescription = Nothing, _source = Just "not found", _message = "Could not find module \8216ModuleA\8217.\nIt is
 not a module in the current program, or in any known package.", _tags = Nothing, _relatedInformation = Just [DiagnosticRelatedInformation {_location = Location {_uri = Uri {getUri = "file:///
tmp/nix-shell.1Fwv9H/hls-test-root/extra-dir-80836663907426/ModuleB.hs"}, _range = Range {_start = Position {_line = 1, _character = 22}, _end = Position {_line = 1, _character = 29}}}, _messa
ge = "GetLocatedImports"}], _data_ = Nothing}]
      Use -p '/correct reference used with hs-boot/' to rerun this test only.
      ```

@guibou guibou force-pushed the optimise_module_to_filename branch from b20c634 to 98fc906 Compare November 1, 2025 13:45
guibou added 13 commits November 1, 2025 19:11
shell
This is related to #4598.

This changes the file to module associating logic done during dependency
graph building.

Before, each time a module `Foo.Bar` is found, HLS is testing inside all
the import path for the existence of a relevant fiel.. It means that for
`i` import paths and `m` modules to locate, `m * n` filesystem
operations are done. Note also that this involves a lot of complex
string concatenation primitive to build the `FilePath`.

A module is tested for each `import` for each of the file of the
project. We also test for `boot` files, doubling the number of test.

In #4598 we have a project with `1100` modules, in more than 250 import
paths and we count more than `17000` `import` statments, resulting on
over 6 millions test for file existences. This project was blocking for
more than 3 minutes during HLS startup.

This commit changes the way this is computed:

- At startup, a `Map ModuleName FilePath` (the real type is a bit more
  involved for performance, multiples unit and boot files handling) is
  built by scanning all the import paths for files representing the
  different modules.
- Directory scanning is efficient and if import path only contains
  haskell module, this will never do more job that listing the files of
  the project.
- The lookup is now simplify a `Map` lookup.

The performance improvement is as follows:

- The number of IO operation is dramatically reduced, from multiples
  millions to a few recursive directories listing.
- A lot of the boilerplate of converting path had be removed.
- TODO: add an RTS stats before / after with number of allocations
- On my project, the graph building time is reduced from a few minutes
  to 3s.

Limitations:

- How to rebuild the `Map` if the content of one directory change?
- If one directory is filled with millions of files which are not of
  interested, performance can be damaged. TODO: add a diagnostic during
  this phase so the user can learn about this issue.

Code status:

- The `lookup` is not fully restored, especially it does not include the
  handling of home unit as well as reexport.
- The initialisation phase is cached inside a `TVar` stored as a top
  level identifier using `unsafePerformIO`. This is to be improved.

A note about performance

Most users won't see the benefits of these change, but I think they
apply to everbody:

- We are still doing 1 lookup per `import` per module. But the lookup
  result is not multiples IO, so it should be faster by a large amount.
- Most project only have 1 (or a few) import paths so won't benefit as
  dramatically as me from this.

TODO for allocations
It does not seem to be used and apparently had never been used (after a
quick look in the repository diff).

Ask m.pickering about this.
The cache is now stored inside the `ShakeExtra`. I'm unsure that's the
right location for it.

This should fix hlint. Also, not having a cache stored as top level
entry may makes it not survive session reinit, which seams sane (for
example, during tests).

Still 3 `-boot` related tests are failing. 2 are however flaky.
@guibou guibou force-pushed the optimise_module_to_filename branch from 98fc906 to 2270264 Compare November 2, 2025 10:04
@guibou
Copy link
Collaborator Author

guibou commented Nov 2, 2025

I will need a bit of help here.

I'm blocked with the following errors locally:

cabal run test:ghcide-tests -- -p "hs-boot"
ghcide
  diagnostics
    cyclic module dependency with hs-boot:        OK (0.69s)
    bidirectional module dependency with hs-boot: OK (0.64s)
    correct reference used with hs-boot:          FAIL (0.99s)
      ghcide-test/exe/DiagnosticTests.hs:295:
      Got diagnostics for Uri {getUri = "file:///tmp/hls-test-root/extra-dir-35586222804939/ModuleB.hs"} but only expected diagnostics for [NormalizedUri 2644126075305381495 "file:///tmp/hls-test-root/extra-dir-35586222804939/ModuleC.hs"] got [Diagnostic {_range = Range {_start = Position {_line = 1, _character = 22}, _end = Position {_line = 1, _character = 29}}, _severity = Just DiagnosticSeverity_Error, _code = Nothing, _codeDescription = Nothing, _source = Just "not found", _message = "Could not find module \8216ModuleA\8217.\nIt is not a module in the current program, or in any known package.", _tags = Nothing, _relatedInformation = Just [DiagnosticRelatedInformation {_location = Location {_uri = Uri {getUri = "file:///tmp/hls-test-root/extra-dir-35586222804939/ModuleB.hs"}, _range = Range {_start = Position {_line = 1, _character = 22}, _end = Position {_line = 1, _character = 29}}}, _message = "GetLocatedImports"}], _data_ = Nothing}]
      Use -p '/hs-boot/&&/correct reference used with hs-boot/' to rerun this test only.

1 out of 3 tests failed (2.32s)

Note that out of the 3 tests related to -boot files, one seems to always fail, but 2 of them seems to be flaky.

In the context of this test, ModuleA cannot be found in the targetMap.

My understanding here is limited and especially, it is difficult for me to understand why my changes may have any impact on these tests. In such context, it is tempting to think that there is another problem such as a race condition / timing.

During my investigation, I found the code in sessionOpts:

          pPrint ("sessionOpts", cfp, v)
          case HM.lookup (toNormalizedFilePath' cfp) v of
            Just (opts, old_di) -> do
              deps_ok <- checkDependencyInfo old_di
              pPrint ("deps_ok", deps_ok, old_di)
              if not deps_ok
                then do
                  -- If the dependencies are out of date then clear both caches and start
                  -- again.
                  modifyVar_ fileToFlags (const (return Map.empty))
                  modifyVar_ filesMap (const (return HM.empty))
                  -- Keep the same name cache
                  modifyVar_ hscEnvs (return . Map.adjust (const []) hieYaml )
                  pPrint ("consultCradle not deps_ok", cfp)
                  consultCradle hieYaml cfp
                else return (opts, Map.keys old_di)
            Nothing -> do
              pPrint ("consultCradle Nothing", cfp)
              consultCradle hieYaml cfp

(the pPrint are my addition).

I observed that the pPrint("sessionOpts") is called multiple time for all the files in the "workspace" (for the test I care about, module{A, B}.hs[-boot].

However, consultCradle is only called once for any moduleX.hs[-boot], e.g. if called for ModuleA.hs, it won't be called for moduleA.hs-boot, or the opposite. Depending on for which "version" of the module it had been called, it will lead to a test success or failure.

It seems that constultCradle will populate the v map with both the .hs-boot and .hs when evaluating one module. For example, this is one trace (when I had a bit less pPrint), additionnal comments with #

# session opts is first called for ModuleB.hs-boot, followed by a consultCradle
( "sessionOpts"
, "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-80639723522145/ModuleB.hs-boot"
)
( "consultCradle Nothing"
, "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-80639723522145/ModuleB.hs-boot"
)
# Same for ModuleA.hs-boot
( "sessionOpts"
, "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-80639723522145/ModuleA.hs-boot"
)
( "consultCradle Nothing"
, "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-80639723522145/ModuleA.hs-boot"
)
# Then for "ModuleA.hs"
( "sessionOpts"
, "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-80639723522145/ModuleA.hs"
)
# Which does not call "consultCradle" because deps seems to be ok
( "deps_ok"
, True
, fromList []
)
# Same for ModuleB.hs
( "sessionOpts"
, "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-80639723522145/ModuleB.hs"
)
( "deps_ok"
, True
, fromList []
)
# Then the hs-boot files are called again due to the cyclic dependency...
( "sessionOpts"
, "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-80639723522145/ModuleB.hs-boot"
)
( "deps_ok"
, True
, fromList []
)
( "sessionOpts"
, "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-80639723522145/ModuleB.hs"
)
( "deps_ok"
, True
, fromList []
)
( "sessionOpts"
, "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-80639723522145/ModuleA.hs-boot"
)
( "deps_ok"
, True
, fromList []
)
( "sessionOpts"
, "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-80639723522145/ModuleA.hs"
)
( "deps_ok"
, True
, fromList []
)
( "sessionOpts"
, "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-80639723522145/ModuleB.hs-boot"
)
( "deps_ok"
, True
, fromList []
)
( "sessionOpts"
, "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-80639723522145/ModuleA.hs"
)
( "deps_ok"
, True
, fromList []
)
( "sessionOpts"
, "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-80639723522145/ModuleA.hs-boot"
)
( "deps_ok"
, True
, fromList []
)
( "sessionOpts"
, "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-80639723522145/ModuleB.hs"
)
( "deps_ok"
, True
, fromList []
)

The result of this is that when calling extendKnownTargets, only twice (for each ModuleX), it calls them with different targetTarget. See two trace here:

    [ TargetDetails
        { targetTarget = TargetFile NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-73114015013655/ModuleA.hs-boot"
        , targetEnv =
            ( []
            , Just HscEnvEq 10
            )
        , targetDepends = fromList []
        , targetLocations =
            [ NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-73114015013655/ModuleA.hs-boot"
            , NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-73114015013655/ModuleA.hs"
            ]
        }
    , TargetDetails
        { targetTarget = TargetFile NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-73114015013655/ModuleB.hs-boot"
        , targetEnv =
            ( []
            , Just HscEnvEq 10
            )
        , targetDepends = fromList []
        , targetLocations =
            [ NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-73114015013655/ModuleB.hs-boot"
            , NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-73114015013655/ModuleB.hs"
            ]
        }
    ]

And

  [ TargetDetails
        { targetTarget = TargetFile NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-911475559555/ModuleB.hs"
        , targetEnv =
            ( []
            , Just HscEnvEq 10
            )
        , targetDepends = fromList []
        , targetLocations =
            [ NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-911475559555/ModuleB.hs"
            , NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-911475559555/ModuleB.hs-boot"
            ]
        }
    , TargetDetails
        { targetTarget = TargetFile NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-911475559555/ModuleA.hs"
        , targetEnv =
            ( []
            , Just HscEnvEq 10
            )
        , targetDepends = fromList []
        , targetLocations =
            [ NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-911475559555/ModuleA.hs"
            , NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-911475559555/ModuleA.hs-boot"
            ]
        }
    ]
    [ TargetDetails
        { targetTarget = TargetFile NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-26070923945755/ModuleB.hs-boot"
        , targetEnv =
            ( []
            , Just HscEnvEq 10
            )
        , targetDepends = fromList []
        , targetLocations =
            [ NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-26070923945755/ModuleB.hs-boot"
            , NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-26070923945755/ModuleB.hs"
            ]
        }
    , TargetDetails
        { targetTarget = TargetFile NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-26070923945755/ModuleA.hs"
        , targetEnv =
            ( []
            , Just HscEnvEq 10
            )
        , targetDepends = fromList []
        , targetLocations =
            [ NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-26070923945755/ModuleA.hs"
            , NormalizedFilePath "/tmp/nix-shell.hE81CE/hls-test-root/extra-dir-26070923945755/ModuleA.hs-boot"
            ]
        }
    ]
    ```
    
    So how each time there is only two entries for each Module, but the `targetTarget` contains either the `.hs` or the `.hs-boot` file, but `targetLocations` always contain boths.
    
    Later, in `extendKnownTargets`:
    
    ```
    let extendKnownTargets newTargets = do
          let checkAndLogFile path = do
                 exists' <- IO.doesFileExist (fromNormalizedFilePath path)
                 pure $ exists'
          knownTargets <- concatForM  newTargets $ \TargetDetails{..} ->
            case targetTarget of
              TargetFile f -> do
                -- If a target file has multiple possible locations, then we
                -- assume they are all separate file targets.
                -- This happens with '.hs-boot' files if they are in the root directory of the project.
                -- GHC reports options such as '-i. A' as 'TargetFile A.hs' instead of 'TargetModule A'.
                -- In 'fromTargetId', we dutifully look for '.hs-boot' files and add them to the
                -- targetLocations of the TargetDetails. Then we add everything to the 'knownTargetsVar'.
                -- However, when we look for a 'Foo.hs-boot' file in 'FindImports.hs', we look for either
                --
                --  * TargetFile Foo.hs-boot
                --  * TargetModule Foo
                --
                -- If we don't generate a TargetFile for each potential location, we will only have
                -- 'TargetFile Foo.hs' in the 'knownTargetsVar', thus not find 'TargetFile Foo.hs-boot'
                -- and also not find 'TargetModule Foo'.
                fs <- filterM checkAndLogFile targetLocations
                pure $ map (\fp -> (TargetFile fp, Set.singleton fp)) (nubOrd (f:fs))
              TargetModule _ -> do
                found <- filterM checkAndLogFile targetLocations
                return [(targetTarget, Set.fromList found)]
          hasUpdate <- atomically $ do
            known <- readTVar knownTargetsVar
            let known' = flip mapHashed known $ \k -> unionKnownTargets k (mkKnownTargets knownTargets)
                hasUpdate = if known /= known' then Just (unhashed known') else Nothing
            writeTVar knownTargetsVar known'
            pure hasUpdate
          for_ hasUpdate $ \x ->
            logWith recorder Debug $ LogKnownFilesUpdated (targetMap x)
          return $ toNoFileKey GetKnownTargets

For each targetfile, we will create a TargetFile associated with actually, 0 files because all files are filtered out by the file exist test.

I'll continue this investigation later.

@guibou guibou force-pushed the optimise_module_to_filename branch from 2270264 to cce8f01 Compare November 2, 2025 15:05
add all alternate file to knownFiles

This need to be investigated.

The different `-boot` tests are flaky / failing in this MR because of
this.

WIP: remove debug statments
@guibou guibou force-pushed the optimise_module_to_filename branch from cce8f01 to f817d89 Compare November 3, 2025 05:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Issues about memory consumption, responsiveness, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

None yet

3 participants