From 069876b8300f733930edae3146b9126baf2e01f3 Mon Sep 17 00:00:00 2001 From: Brett Viren Date: Thu, 30 May 2024 15:41:57 -0400 Subject: [PATCH] Update to Spack 0.22.0 and WCT 0.28.0 --- README.org | 208 +++++++++++---------- packages/paal/package.py | 19 +- packages/wire-cell-dependencies/package.py | 70 +++++-- packages/wire-cell-toolkit/package.py | 141 +++++++++++--- 4 files changed, 299 insertions(+), 139 deletions(-) diff --git a/README.org b/README.org index 629d2d9..0e0fef7 100644 --- a/README.org +++ b/README.org @@ -1,13 +1,23 @@ #+title: Spack packages for Wire-Cell +#+LATEX_HEADER: \usepackage[margin=1in]{geometry} #+options: toc:t -This repo holds a Spack ~wire-cell-toolkit/package.py~ and a few -additional Spack "recipes" to build WCT dependencies not already in -Spack. +This repo provides Spack packaging for building the Wire-Cell toolkit (~wire-cell-toolkit/package.py~) and a few additional Spack packaging to build WCT dependencies not already in Spack. It may be used to build development and production runtime environments. + +Other Spack packaging of the Wire-Cell Toolkit exists. Of note is the one that exists as part of the [[https://github.com/FNALssi/fnal_art/tree/develop/packages/wirecell][FNALssi Spack suite]] used to build the software ecosystem found in FNAL's CVMFS space. + +* Versions + +The version name of a release of this package is based on the version name of the most recent release of ~wire-cell-toolkit~ that the release of ~wire-cell-spack~ builds. A forth digit starting at ~.0~ is appended to the WCT version name and is incremented to indicate new releases of this package that do not update the latest version of WCT itself. + +Versions relevant to this package are currently: + +- Spack 0.22.0 +- Wire-Cell toolkit 0.28.0 + +Generally, a release of ~wire-cell-spack~ can build past released versions of ~wire-cell-toolkit~. +Versions strings used in the examples below may differ. -It builds Wire-Cell Toolkit [[https://github.com/WireCell/wire-cell-toolkit/releases/tag/0.24.1][0.24.1]] and requires exactly [[https://github.com/spack/spack/releases/tag/v0.20.1][Spack 0.20.1]]. -YMMV for any other mixes of versions. It may be used to build a -development environment for newer versions of Wire-Cell Toolkit. * Quick start @@ -18,7 +28,7 @@ Here is one way to do it. #+begin_example $ git clone -c feature.manyFiles=true https://github.com/spack/spack.git /srv/spack/local $ cd /srv/spack/local -$ git checkout -b v0.20.1 v0.20.1 +$ git checkout -b v0.22.0 v0.22.0 #+end_example See [[https://spack.readthedocs.io/en/latest/getting_started.html#installation][Installation section of Spack docs]] for details and options. @@ -33,10 +43,11 @@ See [[https://spack.readthedocs.io/en/latest/getting_started.html#shell-support] ** Add WCT Spack repo -This repo can live anywhere. For example, under ~/srv/spack~ is fine. +Clone this repo somewhere and register it with Spack. #+begin_example -$ cd /srv/spack +$ mkdir -p /srv/spack/repos +$ cd /srv/spack/repos $ git clone https://github.com/WireCell/wire-cell-spack.git wct-spack $ spack repo add wct-spack #+end_example @@ -50,9 +61,13 @@ $ spack install wire-cell-toolkit #+end_example This will install the "core" WCT code and its dependencies. -Alternatively, you may wish to install a "variant" build that provides -additional features requiring more dependencies. Discover what -variants are available with: + + +* Build options + +** Variants + +Alternatively, you may wish to install a "variant" build that provides additional features requiring more dependencies. Discover what variants are available with: #+begin_example $ spack info wire-cell-toolkit @@ -64,7 +79,7 @@ For example one may install support for ROOT and HDF5: $ spack install wire-cell-toolkit+root+hdf #+end_example -** Other compilers +** Compilers One can build software with more than one compiler as long as it is known to Spack: @@ -75,25 +90,22 @@ $ spack compiler list $ spack install wire-cell-toolkit%clang@14.0.6 #+end_example -** WCT shell environment -Spack provides support for "environment modules" that manages the -users shell environment. For example, so that ~PATH~ like variables are -set. +* Shell environment variable runtime + +Spack provides support for "environment modules" that manages the users shell environment. For example, so that ~PATH~ like variables are set. #+begin_example $ spack load wire-cell-toolkit $ wire-cell --help #+end_example -** WCT view -Note, developers should read this section but skip the example command -and see section [[Developer guide]] below. +* File-system based environment + +Note, developers should read this section but skip the example command and see section [[Developer guide]] below. -A "Spack view" provides an alternative to shell environment variables -for using packages. This method requires far smaller shell -environment mangling by providing a single area akin to ~/usr/local/~. +A "Spack view" provides an alternative to shell environment variables for using packages. This method requires far smaller shell environment mangling by providing a single area akin to ~/usr/local/~. #+begin_example $ spack view add -i myview wire-cell-toolkit @@ -115,33 +127,35 @@ $ export LD_LIBRARY_PATH=$myview/lib:$LD_LIBRARY_PATH * Developer guide -This section describes how to leverage Spack views to create simple -and tidy developer environments. Follow above up to an including the -~spack install~ step. +This section describes one way to provide a developer environment. -** Make a Spack view for development +** Make a Spack view -We will make a Spack view slightly differently than a user would. We -will explicitly exclude WCT itself as we will next build WCT -ourselves. +After installing at least one instance of ~wire-cell-toolkit~ we make a ~spack view~. This provides a single "tidy" directory similar to ~/usr/local~ with all the files for a set of packages. #+begin_example $ cd /path/to/my/dev/area -$ spack view -e wire-cell-toolkit add -i local wire-cell-toolkit +$ spack view -e wire-cell-toolkit add -i local wire-cell-toolkit +#+end_example + +This builds the view into a directory called ~local/~ with WCT as the "seed" package but excludes wire-cell-toolkit itself. + +** A view without a toolkit + +The above requires the requested instance of ~wire-cell-toolkit~ to have been installed by Spack. It is possible to create an equivalent ~view~ without building WCT with Spack. This can be needed in rare cases where Spack fails to build WCT but the developer wants Spack to provide dependencies so WCT source can be corrected so Spack build succeeds. + +#+begin_example +$ spack install wire-cell-dependencies +$ spack view add -i local wire-cell-dependencies #+end_example -This populates the directory named ~local/~. +This ~wire-cell-dependencies~ "bundle" package takes the same variants as ~wire-cell-toolkit~. ** Manage shell environment with ~direnv~ -Typically we developers want to form a strong binding between one of -our development areas on disk and the shell environment that is -required to build and run the software. This can be managed by hand -but that is tedious and error prone. Use of ~direnv~ overcomes these -problems and is well worth the one-time effort to install. +Typically we developers want to form a strong binding between one of our development areas on disk and the shell environment that is required to build and run the software. This can be managed by hand but that is tedious and error prone. Use of ~direnv~ overcomes these problems and is well worth the one-time effort to install. -To configure ~direnv~ to use the Spack view, first set up ~direnv~ -[[https://direnv.net/docs/hook.html][for your particular shell]] and then perform this one time setup: +To configure ~direnv~ to use the Spack view, first set up ~direnv~ [[https://direnv.net/docs/hook.html][for your particular shell]] and then perform this one time setup: #+begin_example $ cd /path/to/my/dev/area @@ -149,16 +163,15 @@ $ echo 'load_prefix local' > .envrc $ direnv allow #+end_example -This last command will need repeating if/when you edit the ~.envrc~ -file. With your shell set up to use ~direnv~ it will automatically warn -you when it detects a change. +This last command will need repeating if/when you edit the ~.envrc~ file. With your shell set up to +use ~direnv~ it will automatically warn you when it detects a change. + ** Installing ~wire-cell-python~ -The ~wire-cell-python~ package provides various useful, ancillary -facilities but is not (yet) officially a dependency of WCT. However, -developers are strongly recommended to install it as it may be used by -some of the tests provided by ~wire-cell-toolkit~. +The ~wire-cell-python~ package provides various useful, ancillary facilities but is not (yet) +officially a dependency of WCT. However, developers are strongly recommended to install it as it +is used by some of the tests provided by ~wire-cell-toolkit~. After the above ~direnv~ set up, check what Python we have: @@ -167,8 +180,7 @@ $ which python $ python --version #+end_example -This should show a modern version of Python provided under the Spack -*view* directory ~local/~. In general, do not use the system Python. +This should show a modern version of Python provided under the Spack *view* directory ~local/~. In general, do not use the system Python. #+begin_example $ cd /path/to/my/dev/area @@ -177,23 +189,18 @@ $ cd python/ $ pip install -e . #+end_example -Note, ~wire-cell-python~ packaging is still a work in progress and the -above will only install core dependencies. Depending on the command -line programs and Python modules to be used, the developer may need to -install more Python dependencies via ~pip~. This can be done using one -or more of the requirements files. For example: +Note, ~wire-cell-python~ packaging is still a work in progress and the above will only install core dependencies. Depending on the command line programs and Python modules to be used, the developer may need to install more Python dependencies via ~pip~. This can be done using one or more of the requirements files. For example: #+begin_example $ pip install -r requirements/base.txt #+end_example -Future improvements in the ~wire-cell-python~ Python packaging will make -this unnecessary. +Future improvements in the ~wire-cell-python~ Python packaging will make this unnecessary. + ** View-based development environment -Finally, the developer can get, build and run WCT. Start by getting a -copy of the source: +Finally, the developer can get, build and run WCT. Start by getting a copy of the source: #+begin_example $ cd /path/to/my/dev/area @@ -201,9 +208,7 @@ $ git clone https://github.com/WireCell/wire-cell-toolkit.git toolkit $ cd toolkit #+end_example -We are now set to configure and build. The Spack *view* and ~direnv~ -helps us here ~wcb~ will still need help to find some dependencies -by giving it ~--with-*~ command line options: +We are now set to configure and build. The Spack *view* and ~direnv~ helps us here ~wcb~ will still need help to find some dependencies by giving it ~--with-*~ command line options: #+begin_example $ export myview=/path/to/my/dev/area/local # just for brevity here @@ -216,17 +221,14 @@ $ ./wcb install $ ./wcb --tests # optional, run many tests #+end_example -Giving ~--prefix=$myview~ to ~wcb~ will cause WCT to be installed right -into the Spack *view* directory. Since ~direnv~ has minimally set up our -environment we (almost) ready for full usage: +Giving ~--prefix=$myview~ to ~wcb~ will cause WCT to be installed right into the Spack *view* directory. Since ~direnv~ has minimally set up our environment we (almost) ready for full usage: #+begin_example $ wire-cell --version 0.18.0-5-g8513660 #+end_example -The ~.envrc~ file is ideal for capturing the few shell environment -variable settings that are needed for developers: +The ~.envrc~ file is ideal for capturing the few shell environment variable settings that are needed for developers: #+begin_example $ cd /path/to/my/dev/area @@ -246,9 +248,7 @@ EOF ** Automate creation of development areas -Creating the patterns described above can be automated using the -provided [[./scripts/wct-make-dev]] script. This script is particularly -useful for developer that frequently create new WCT development areas. +Creating the patterns described above can be automated using the provided [[./scripts/wct-make-dev]] script. This script is particularly useful for developer that frequently create new WCT development areas. #+begin_example $ ./scripts/wct-make-dev /path/to/work [] @@ -256,8 +256,7 @@ $ ls -a /path/to/work . .. .direnv .envrc local python toolkit #+end_example -The script will guess the newest ~wire-cell-dependencies~ available if -~~ is not given and it produces: +The script will guess the newest version of WCT available unless ~~ is given. It produces: - ~.direnv/~ direnv's area holding a Python virtual env. - ~.envrc~ a direnv config file. @@ -265,8 +264,7 @@ The script will guess the newest ~wire-cell-dependencies~ available if - ~python/~ holding git clone of ~wire-cell-python~ - ~toolkit/~ holding git clone of ~wire-cell-toolkit~ -At the end the script prints a few lines which can be copy-pasted to -build and install the latter two. Eg: +At the end the script prints a few lines which can be copy-pasted to build and install the latter two. Eg: #+begin_example cd /path/to/work && direnv allow @@ -274,23 +272,19 @@ cd python && pip install -e . && cd - bash -c "$WIRECELL_CONFIG" && bash -c "$WIRECELL_BUILD" #+end_example -The developer is free to subsequently modify the generated ~.envrc~ file -as desired. +The developer is free to subsequently modify the generated ~.envrc~ file as desired. + * Details ** Other compilers -When using a non-default compiler one must set ~CC~ and ~CXX~ before -configuring ~wire-cell-toolkit~ build with ~wcb~. +When using a non-default compiler one must set ~CC~ and ~CXX~ before configuring ~wire-cell-toolkit~ build with ~wcb~. + ** More than one ~wire-cell-toolkit~ installed -Spack allows multiple packages of the same name to be installed. -Commands like ~spack load~ or ~spack view~ require a package to be -identified uniquely. One way to do that is to specify the *version* -and/or set of *variants* for the desired package. You can discover what -is available with: +Spack allows multiple packages of the same name to be installed. Commands like ~spack load~ or ~spack view~ require a package to be identified uniquely. One way to do that is to specify the *version* and/or set of *variants* for the desired package. You can discover what is available with: #+begin_example $ spack find -v wire-cell-toolkit @@ -300,8 +294,7 @@ wire-cell-toolkit@0.18.0~cppjsonnet~hdf+root+tbb wire-cell-toolkit@0.20.0~cppjsonnet+hdf+root+tbb #+end_example -In this example, adding =+root= or =~root= is enough to make the name -unique. You may test uniqueness, eg: +In this example, adding =+root= or =~root= is enough to make the name unique. You may test uniqueness, eg: #+begin_example $ spack find -v wire-cell-toolkit~root @@ -310,9 +303,7 @@ $ spack find -v wire-cell-toolkit~root wire-cell-toolkit@0.18.0~cppjsonnet~hdf~root+tbb #+end_example -If specifying a version and variants are not enough to uniquely -determine a package then a "slash-hash" may be used. To discover -these you may run: +If specifying a version and variants are not enough to uniquely determine a package then a "slash-hash" may be used. To discover these you may run: #+begin_example $ spack find -vl wire-cell-toolkit @@ -335,17 +326,46 @@ $ spack find -vl /55c7krd ** Using a Spack "scope" -To better keep multiple Spack instances you use separate, a Spack -"scope" may be used. Some example ones are provided under [[file:./scopes/][scopes/]] -directory. +To better keep multiple Spack instances you use separate, a Spack "scope" may be used. Some example ones are provided under [[file:./scopes/][scopes/]] directory. - ~debian/~ :: use Debian packages to provide packages otherwise built - by Spack. For use, see example [[scripts/wcspack]]. + by Spack. For use, see example [[file:scripts/wcspack]]. + + +** Choosing versions + +By default, Spack will use preferred versions, usually the newest. You may specify versions of WCT and dependencies at arbitrary detail by spelling out the Spack "spec". + +#+begin_example +spack install wire-cell-toolkit@0.20.0+root ^root@6.28.04 +#+end_example +See more in the [[https://spack.readthedocs.io/en/latest/basic_usage.html#specs-dependencies][Spack docs]]. + + +** Updating this package +This package is updated infrequently to rely on new releases of Spack and add support for new releases of WCT and its dependencies at versions supported by that new Spack. This section describes how to update. + +Either re-clone the Spack repository as in section [[Install Spack]] or fetch updates. +#+begin_example +$ cd /srv/spack/local/ +$ git tag|tail +$ git checkout -b v0.21.2 v0.21.2 +#+end_example + +Update ~wire-cell-toolkit~ versions for recent [[https://github.com/WireCell/wire-cell-toolkit/releases][Releases]] beyond what are currently captured. +#+begin_example +$ spack info wire-cell-toolkit +$ spack checksum wire-cell-toolkit 0.25.3 0.26.0 0.27.0 0.27.1 +#+end_example +Copy-paste the new ~version(...)~ lines into the ~package.py~. + +Do a test build. +#+begin_example +$ spack install wire-cell-toolkit+root+hdf+tbb +#+end_example -* More info +** Containers -See also https://github.com/wirecell/wire-cell-spack-containers for -idea on how to make and use Docker and Singularity containers built -via ~wire-cell-spack~ methods. +See [[https://github.com/wirecell/wire-cell-spack-containers][wire-cell-spack-containers]] for ideas on how to make and use Docker and Singularity (now Apptainer) containers built via ~wire-cell-spack~ methods. diff --git a/packages/paal/package.py b/packages/paal/package.py index ac26bda..76ced89 100644 --- a/packages/paal/package.py +++ b/packages/paal/package.py @@ -3,13 +3,13 @@ class Paal(Package): """ - A header-only, generic library consisting of approximation algorithms, - data structures and several complete solutions for the various optimization - problems. + Upstream blurb: A header-only, generic library consisting of + approximation algorithms, data structures and several complete solutions for + the various optimization problems. - Note, PAAL has suffered bitrot over the ages, particularly due to depending - on "boost" code that never went into boost such as "numeric bindings". - Maybe that refers to these: + Commentary:, PAAL has suffered bitrot over the ages, particularly due to + depending on "boost" code that never went into boost such as "numeric + bindings". Maybe that refers to these: https://github.com/uBLAS/numeric_bindings https://mathema.tician.de/software/boost-numeric-bindings/ @@ -25,11 +25,16 @@ class Paal(Package): maintainers = ["bv@bnl.gov"] # for this file, not paal # there are neither branches nor tags or other version identifiers. version("2017-01-30", - sha256sum="83487de59e8f7ded1a488ad8ad5359c358fb45a59e548d499f6dc44d712c82ad", + sha256="83487de59e8f7ded1a488ad8ad5359c358fb45a59e548d499f6dc44d712c82ad", commit="e537b58d50e93d4a72709821b9ea413008970c6b") version("master", branch="master") + variant('glpk', default=True, + description='PAAL is mainly a header only library. ' + 'If you use linear programming you must link your executable against glpk.') + depends_on(Boost.with_default_variants) + depends_on('glpk', when='+glpk') def install(self, spec, prefix): mkdirp(prefix.include) diff --git a/packages/wire-cell-dependencies/package.py b/packages/wire-cell-dependencies/package.py index 32c4d38..df9d336 100644 --- a/packages/wire-cell-dependencies/package.py +++ b/packages/wire-cell-dependencies/package.py @@ -14,30 +14,65 @@ class WireCellDependencies(BundlePackage): maintainers = ['brettviren'] - version('0.0.0') + # Only define versions to the extent needed for versioned dependencies. + version('0.27.1') # pre fmt v9 + version('0.28.0') # post fmt v9 - # optional, default=True + # the rest is copy-paste from wire-cell-toolkit/package.py + + # ---------- + + # see eg https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/boost/package.py + variant( + "cxxstd", + default="17", + values=("17", "20", "23"), + multi=False, + sticky=True, + description="C++ standard", + ) + + # Suggested. optional but default=True variant('tbb', default=True, - description='Support multithread execution') + description='Add support for multi-thread execution using TBB') - # optional, default=False + # Optional, default=False variant('cppjsonnet', default=False, description='Use slower C++ Jsonnet instead of Go Jsonnet') variant('hdf', default=False, description='Add support for HDF5') variant('root', default=False, description='Add support for ROOT') - + variant('glpk', default=False, + description='Add support for GLPK') # just for build time (for waf/wcb) depends_on('python', type=('build',)) depends_on('pkgconfig', type=('build',)) - # required + # Required + depends_on('eigen @3.4.0:') + depends_on('spdlog @1.9.2:') + + # Spack builds spdlog against an "external" fmtlib and does not build + # against spdlog's bundled fmtlib. Thie means the version of fmtlib is + # exposed to the application and spack will tend to use the most recent + # available. fmtlib made an API change at v9 that breaks the ability for it + # to implicitly use ostream insertion operator<<'s. The new API requires + # new application code and that code is not compatible with pre-v9 fmtlib. + # So a mutually-exclusive line in the version sand is required. + + # use of operator<<() is implicit + depends_on('fmt @:8.1.1', when='@:0.27.1') + # need specialization based on ostream_formatter + depends_on('fmt @9.0.0:', when='@0.28.0:') + depends_on('fftw @3.3.10: ~mpi') - depends_on('jsoncpp @1.9.4: cxxstd=17') + ## seems jsoncpp builder only supports this variant up to 1.9.2??? + #depends_on('jsoncpp @1.9.4: cxxstd=17') + depends_on('jsoncpp @1.9.4:') # In principle, we can form the list of Boost libs to build based # on the list that WCT's waf-tools/wcb.py uses to check for Boost @@ -50,15 +85,26 @@ class WireCellDependencies(BundlePackage): boost_variants = '+'.join(boost_libs) depends_on('boost @1.80.0: cxxstd=17 +'+boost_variants) - # we need one or the other + # We need one or the other. depends_on('jsonnet @0.19.1: +python', when='+cppjsonnet') depends_on('go-jsonnet @0.19.1: +python', when='~cppjsonnet') - # optional + + # Suggested: + depends_on('intel-tbb @2021.7.0: cxxstd=17', when='+tbb') - # fixme: may need to tell root to use same TBB - depends_on('root @6.28.04 cxxstd=17', when='+root') + + + # Optional: + + # ROOT is needed for wire-cell-toolkit/root + depends_on('root @6.28.04: cxxstd=17', when='+root') depends_on('hdf5 ~mpi+threadsafe', when='+hdf') depends_on('h5cpp ~mpi', when='+hdf') - depends_on('hdf5 ~mpi', when='+hdf') + # glpk is needed for wire-cell-toolkit/patrec + depends_on('glpk', when='+glpk') + + # TODO: cuda, torch, zmq + + # ---------- diff --git a/packages/wire-cell-toolkit/package.py b/packages/wire-cell-toolkit/package.py index 7ec38ff..c65c378 100644 --- a/packages/wire-cell-toolkit/package.py +++ b/packages/wire-cell-toolkit/package.py @@ -5,17 +5,27 @@ from spack import * - class WireCellToolkit(Package): """Toolkit for Liquid Argon TPC Reconstruction and Visualization .""" # FIXME: Add a proper url for your package's homepage here. homepage = "https://wirecell.github.io/" url = "https://github.com/WireCell/wire-cell-toolkit/archive/refs/tags/0.23.0.tar.gz" + git = "https://github.com/WireCell/wire-cell-toolkit.git" + + license("LGPLv3") maintainers = ['brettviren'] + + version("master", branch="master") + version("0.28.0", sha256="62f07ad8bf726ef8aaec428a84cae0ca61ca7b33d5c58f35d2c056f342fdc22c") + version("0.27.1", sha256="a8410a9e0524570e811f5cca2ea9fc636e48c048a5e67c5cee567b935515e176") + version("0.27.0", sha256="c4d1dc438b685bc54004425922f9435d8cb7f928a6b080b910cff021392571b2") + version("0.26.0", sha256="8ad495ef1fb8ca94ee02d77ab9b3873b278bfdce87fc3c2d4dccb8afb8aef41a") + version("0.25.3", sha256="e92b39335bc3faa1548db95a17fac5462363b4b9b636aaa427044e4a9b51d194") version("0.25.2", sha256="aac849da29092d3397f0130f45c499703496188ad74bd347a3bd154fe23f245d") version("0.25.1", sha256="c236758a70fd459824f2f769b1e4e146e8f75965833a5aa8a81f0cf08044f9d6") + version("0.24.3", sha256="040d819a3a81b953a42c8b4bb898acf6978cee45beea0361a2f3cdb602a6028c") version("0.24.2", sha256="ee8fe70f8ead7b7456bbcd791d8bf1a1cf22c6674df503fb9f93c1d33bfd2a1a") version("0.24.1", sha256="0467a4dff51abac3661aa99c5f3cc5de1ba1607a7f357631a2fbf7dcdf01c8a9") version("0.24.0", sha256="2a3a62089b40ee1baccdfaf320d3730eed8d301337a616eeb3186097996f3431") @@ -28,28 +38,59 @@ class WireCellToolkit(Package): version('0.18.0', sha256='9a659d2ac96cff0166c1e78574734981ed1fabe039a2f02376a8b3f04653b6b3') version('0.17.1', sha256='4abb1bf16a59815e1702c6c53238c6790ec0872a58f51fd2fc44866419b597e5') - # optional, default=True + # ---------- + + # see eg https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/boost/package.py + variant( + "cxxstd", + default="17", + values=("17", "20", "23"), + multi=False, + sticky=True, + description="C++ standard", + ) + + # Suggested. optional but default=True variant('tbb', default=True, - description='Support multithread execution') + description='Add support for multi-thread execution using TBB') - # optional, default=False + # Optional, default=False variant('cppjsonnet', default=False, description='Use slower C++ Jsonnet instead of Go Jsonnet') variant('hdf', default=False, description='Add support for HDF5') variant('root', default=False, description='Add support for ROOT') - + variant('glpk', default=False, + description='Add support for GLPK') # just for build time (for waf/wcb) depends_on('python', type=('build',)) depends_on('pkgconfig', type=('build',)) - # required + # Required + depends_on('eigen @3.4.0:') + depends_on('spdlog @1.9.2:') + + # Spack builds spdlog against an "external" fmtlib and does not build + # against spdlog's bundled fmtlib. Thie means the version of fmtlib is + # exposed to the application and spack will tend to use the most recent + # available. fmtlib made an API change at v9 that breaks the ability for it + # to implicitly use ostream insertion operator<<'s. The new API requires + # new application code and that code is not compatible with pre-v9 fmtlib. + # So a mutually-exclusive line in the version sand is required. + + # use of operator<<() is implicit + depends_on('fmt @:8.1.1', when='@:0.27.1') + # need specialization based on ostream_formatter + depends_on('fmt @9.0.0:', when='@0.28.0:') + depends_on('fftw @3.3.10: ~mpi') - depends_on('jsoncpp @1.9.4: cxxstd=17') + ## seems jsoncpp builder only supports this variant up to 1.9.2??? + #depends_on('jsoncpp @1.9.4: cxxstd=17') + depends_on('jsoncpp @1.9.4:') # In principle, we can form the list of Boost libs to build based # on the list that WCT's waf-tools/wcb.py uses to check for Boost @@ -62,32 +103,80 @@ class WireCellToolkit(Package): boost_variants = '+'.join(boost_libs) depends_on('boost @1.80.0: cxxstd=17 +'+boost_variants) - # we need one or the other + # We need one or the other. depends_on('jsonnet @0.19.1: +python', when='+cppjsonnet') depends_on('go-jsonnet @0.19.1: +python', when='~cppjsonnet') - # optional + + # Suggested: + depends_on('intel-tbb @2021.7.0: cxxstd=17', when='+tbb') - # fixme: may need to tell root to use same TBB - depends_on('root @6.28.04 cxxstd=17', when='+root') + + + # Optional: + + # ROOT is needed for wire-cell-toolkit/root + depends_on('root @6.28.04: cxxstd=17', when='+root') depends_on('hdf5 ~mpi+threadsafe', when='+hdf') depends_on('h5cpp ~mpi', when='+hdf') - depends_on('hdf5 ~mpi', when='+hdf') + + # glpk is needed for wire-cell-toolkit/patrec + depends_on('glpk', when='+glpk') + + # TODO: cuda, torch, zmq + + # ---------- def install(self, spec, prefix): - cfg = ["./wcb", "configure", "--prefix={0}".format(prefix)] - # fixme: honor cppjsonnet variant + + cfg = ["wcb", "configure", "--prefix="+prefix] + + # Prior to and including 0.27.1 the C++ std was hard-wired in the + # wscript. After finding out that FNALssi folk were forced to grub + # around in build/c4che/_cache.py to change it, a nicer way was added + # that can be more easily hooked in to the quasi standard "cxxstd" + # variant. + # if not spec.satisfies("@:0.27.1"): + # cfg += ["--cxxstd="+spec.variants["cxxstd"].value] + cfg += [ - "--with-jsonnet={0}".format(spec['go-jsonnet'].prefix), - "--with-jsonnet-lib={0}".format(spec['go-jsonnet'].prefix.lib), - "--with-jsonnet-libs=gojsonnet", - "--with-jsonnet-include={0}".format(spec['go-jsonnet'].prefix.include), "--boost-mt", - "--boost-libs={0}".format(spec['boost'].prefix.lib), - "--boost-includes={0}".format(spec['boost'].prefix.include), - ] - bld = ["./wcb", "build", "--notests", "install"] - - bash = which("bash") - bash("-c", ' '.join(cfg)) - bash("-c", ' '.join(bld)) + "--boost-libs=" + spec['boost'].prefix.lib, + "--boost-includes=" + spec['boost'].prefix.include, + ] + + if 'tbb' in spec: + cfg += ['--with-tbb=' + spec['tbb'].prefix ] + + # one or the other assured by +/~cppjsonnet? + if 'jsonnet' in spec: + jsonnet = 'jsonnet' + jsonnet_libs = 'jsonnet' + else: + jsonnet = 'go-jsonnet' + jsonnet_libs = 'gojsonnet' + cfg += [ + "--with-jsonnet=" + spec[jsonnet].prefix, + "--with-jsonnet-lib=" + spec[jsonnet].prefix.lib, + "--with-jsonnet-libs=" + jsonnet_libs, + "--with-jsonnet-include=" + spec[jsonnet].prefix.include, + ] + + # for now, explicitly turn this off to make sure some random cuda + # install on the host isn't picked up. + cfg += [ + "--with-cuda=no" + ] + + # The --notests flag is vestigial in more recent versions but doesn't + # hurt to keep it in order to avoid making version dependency code here. + # Tests are always built. To actually run them: ./wcb --tests . + # Developers should be running tests. Running them during an install + # does not add much value. + bld = ["wcb", "--notests"] + ins = ["wcb", "install"] + + python = which("python") + python(*cfg) + python(*bld) + python(*ins)