Regression: Cabal-3.14.1.0: v1-test, Setup.hs test: test suites of alex-3.4.0.1 and happy-1.20.1.1 unable to find data files

[sternenseemann]

Describe the bug

When building the mentioned versions of alex and happy with --enable-tests via Setup.hs and Cabal 3.14, Setup.hs test will yield a lot of errors like this and ultimately fail. This is a regression, as downgrading to Cabal 3.12 solves the issue. v1-test is also affected, but v2-test works as it should.

../dist/build/happy/happy --strict -agc issue91.y -o issue91.agc.hs
happy: Uncaught exception ghc-internal:GHC.Internal.IO.Exception.IOException:

data//HappyTemplate-arrays-coerce: openFile: does not exist (No such file or directory)

HasCallStack backtrace:
  ioError, called at libraries/ghc-internal/src/GHC/Internal/Foreign/C/Error.hs:291:5 in ghc-internal:GHC.Internal.Foreign.C.Error

make: *** [Makefile:74: issue91.agc.hs] Error 1

This is pretty annoying since alex and happy are used to build Cabal, so it's best to be able to use the GHC bundled version.

To Reproduce

For v1-test:

1. cabal get happy-1.20.1.1 && cd happy*
2. cabal v1-test

For Setup.hs:

• Use Setup.hs to build the mentioned alex/happy versions and run their tests using GHC 9.12.1 and the version of Cabal it ships.
• You can use Nix to reproduce it a bit easier, just build Cabal_3_14.alex, Cabal_3_14.happy for the error and
  observe that the same builds work in the Cabal_3_12 set.

{ pkgs ? import (builtins.fetchTarball {
    url = "https://github.com/sternenseemann/nixpkgs/archive/263f0a1f6557b6849a4ae83e8f391efe9b289ab3.tar.gz";
    sha256 = "0xqdd6cch8fg5zghknvvxy4kpxz2fimn8czci6clj2p8r1qvwj7r";
  }) { }
}:

with pkgs;
with haskell.lib.compose;

let
  hpkgs = haskell.packages.ghc9121;
in

rec {
  Cabal_3_14 = {
    inherit (hpkgs) alex happy;
  };

  Cabal_3_12 = lib.mapAttrs (_: drv: overrideCabal {
    preCompileBuildDriver = ''
      export setupCompileFlags="-ignore-package Cabal-3.14.1.0 $setupCompileFlags";
    '';
    setupHaskellDepends = [
      # We still need alex and happy to build Cabal-syntax to get Cabal-3.12
      ((hpkgs.override {
        overrides = self: super: { alex = dontCheck super.alex; happy = dontCheck super.happy; };
      }).Cabal_3_12_1_0)
    ];
  } (drv.override {
    ${if drv.pname == "alex" then "happy" else null} = Cabal_3_12.happy;
  })) Cabal_3_14;
}

Expected behavior
The tests succeed.

System information

• NixOS
• Cabal 3.14.1.0, GHC 9.12.1

Additional context

Since this happens in the test suite which is executed from a subdirectory of the main project, it seems to me that Cabal stopped setting the happy_datadir and alex_datadir environment variables in test. This is just a hunch though, I haven't tried to verify it yet since someone here probably knows whether this changed or not.

[phadej]

3.14 seems to set alex_datadir=data/, the 3.12 (from GHC-9.10) sets alex_datadir=/codetmp/alex-3.5.2.0/data/

I.e. 3.12 used absolute paths for Paths_ variables (which needed for using data-files when package is not installed), but 3.14 doesn't. I suspect this is another side-effect of #9702, the patch seems to change

-      dataDirPath = pwd </> PD.dataDir pkg_descr
+      rawDataDir = PD.dataDir pkg_descr
+      dataDirPath
+        | null $ getSymbolicPath rawDataDir =
+            interpretSymbolicPath mbWorkDir sameDirectory
+        | otherwise =
+            interpretSymbolicPath mbWorkDir rawDataDir

interpretSymbolicPath documentations says

-- | Interpret a symbolic path with respect to the given directory.
--
-- Use this function before directly interacting with the file system in order
-- to take into account a working directory argument.
--
-- NB: when invoking external programs (such as @GHC@), it is preferable to set
-- the working directory of the process and use 'interpretSymbolicPathCWD'
-- rather than calling this function, as this function will turn relative paths
-- into absolute paths if the working directory is an absolute path.
-- This can degrade error messages, or worse, break the behaviour entirely
-- (because the program might expect certain paths to be relative).
--

but as far as I can tell, in this case the dataDirPath, i.e. alex_datadir environment variable contents SHOULD be an absolute path, so use of interpretSymbolicPath is incorrect.

For more context, alex's test suite changes the current working directory, which is non-unexpected thing to do for tests. So relative alex_datadir breaks.

IMO this is clear regression in 3.14

[mpickering]

Fix is in #10830