Merge remote-tracking branch 'origin/main' into indexer_multicrystal_multiorientation_bug

Author: ndevenish

doc/sphinx/_static/dials-styles.css

@@ -159,3 +159,11 @@ div.leftside {
 div.rightside {
   margin-left: 10px;
 }
+
+div.leftside-nih {
+  color: white;
+  background-color: black;
+  height: 80px;
+  padding: 0px 10px 0px 10px;
+  float: left;
+}

doc/sphinx/about.rst

@@ -35,19 +35,19 @@ reuse both within DIALS and by third parties.
 Development Teams
 =================
 
-DIALS East
+DIALS UK
 ----------
 
 Development of DIALS in the UK is funded by the `Wellcome Trust`_,
 `Diamond Light Source`_ and `CCP4`_, and led by `Dr Gwyndaf Evans`_.
 
-DIALS West
+DIALS US
 ----------
 
 Development of DIALS at `Lawrence Berkeley National Laboratory`_, USA is led by
-`Dr Nicholas Sauter`_ and supported by `National Institutes of Health`_ /
-`National Institute of General Medical Sciences`_ grant R01-GM117126: *DIALS:
-New Computational Methods to Enable Challenging Crystallographic Experiments*.
+`Dr Aaron Brewster`_ and supported by `National Institutes of Health`_ /
+`National Institute of General Medical Sciences`_ grant R24GM154040: *DIALS:
+supporting structural biology through open source diffraction processing software*.
 Work at LBNL is performed under `Department of Energy`_ contract
 DE-AC02-05CH11231.
 
@@ -83,11 +83,11 @@ Matthew Webber.
 
 .. _`Wellcome Trust`: https://wellcome.ac.uk/
 .. _`Boost.Python`: http://www.boost.org/doc/libs/1_59_0/libs/python/doc/index.html
-.. _`cctbx`: http://cctbx.sourceforge.net/
+.. _`cctbx`: https://github.com/cctbx
 .. _`CCP4`: http://www.ccp4.ac.uk/
 .. _`Diamond Light Source`: http://www.diamond.ac.uk/Home.html
 .. _`Dr Gwyndaf Evans`: http://www.diamond.ac.uk/Beamlines/Mx/VMXm/Staff/Evans.html
-.. _`Dr Nicholas Sauter`: https://biosciences.lbl.gov/profiles/nicholas-sauter/
+.. _`Dr Aaron Brewster`: https://biosciences.lbl.gov/profiles/aaron-brewster/
 .. _`Lawrence Berkeley National Laboratory`: http://www.lbl.gov/
 .. _`National Institutes of Health`: http://www.nih.gov/
 .. _`National Institute of General Medical Sciences`: http://www.nigms.nih.gov/

doc/sphinx/documentation/tutorials/diamond.rst

@@ -169,7 +169,7 @@ Feedback
 
 If you encounter any problems using DIALS or xia2, or indeed have any other
 feedback (positive or negative - we love to hear from you either way),
-please contact dials-support@lists.sourceforge.net or xia2.support@gmail.com
-respectively.
+please contact log feedback at https://github.com/dials/dials/issues or
+https://github.com/xia2/xia2/issues respectively.
 
 .. _xia2: https://xia2.github.io/

doc/sphinx/documentation/tutorials/multi_crystal_analysis.rst

@@ -95,7 +95,7 @@ for each dataset. This script could be written in BASH, tcsh, perl,
 ruby - whatever you feel most comfortable with. However here we will use Python,
 or more specifically :program:`dials.python` because we will take advantage of
 features in the cctbx to make it easy to write scripts that take advantage
-of `parallel execution <http://cctbx.sourceforge.net/current/python/libtbx.easy_mp.html>`_.
+of `parallel execution <https://cctbx.github.io/libtbx/libtbx.easy_mp.html>`_.
 Also we would like to read :samp:`imported.expt` with the DIALS API rather than
 extracting the sequence templates using something like :program:`grep`.
 

doc/sphinx/index.rst

@@ -17,6 +17,7 @@ DIALS: Diffraction Integration for Advanced Light Sources
    How-to <howto>
    Knowledge Base <https://dials.github.io/kb>
    workshops/index
+   US National Resource <national_resource>
    publications
    projects
    License <license>
@@ -41,8 +42,6 @@ For feature requests, bug reports or any other information, please contact
 the DIALS developers at dials-support@lists.sourceforge.net, or post an
 `issue`_.
 
-Mirror sites: `DIALS-East`_, `DIALS-West`_, `dials.github.io`_.
-
 Funding
 =======
 
@@ -69,8 +68,29 @@ DIALS development at `Diamond Light Source`_ is performed by employees of
 
 DIALS development at `Lawrence Berkeley National Laboratory`_ is
 supported by `National Institutes of Health`_ / `National Institute of General Medical Sciences`_
-grant R01-GM117126.  Work at LBNL is performed under
-`Department of Energy`_ contract DE-AC02-05CH11231.
+grant R24GM154040.
+
+.. container:: twocol
+
+   .. container:: leftside-nih
+
+        .. image:: https://www.nigms.nih.gov/Style%20Library/NIGMS2/images/nih.svg
+           :alt: NIH NIGMS
+           :target: `NIH`_
+           :height: 70px
+
+   .. container:: leftside-nih
+
+        | National Institute of
+        | General Medical Sciences
+
+.. rst-class:: clear-both
+
+DIALS is an NIH NIGMS US National Resource with the goal of supporting both users and
+facilities in using DIALS to analyse their diffraction data.
+:ref:`Learn more <national_resource>`.
+
+Work at LBNL is performed under `Department of Energy`_ contract DE-AC02-05CH11231.
 
 .. _`issue`: https://github.com/dials/dials/issues
 .. _`Wellcome`: https://wellcome.ac.uk/
@@ -82,6 +102,4 @@ grant R01-GM117126.  Work at LBNL is performed under
 .. _`National Institute of General Medical Sciences`: http://www.nigms.nih.gov/
 .. _`Department of Energy`: http://www.energy.gov/
 .. _dials.github.io: https://dials.github.io/
-.. _`DIALS-East`: http://dials.diamond.ac.uk/
-.. _`DIALS-West`: http://dials.lbl.gov/
-
+.. _`NIH`: https://www.nigms.nih.gov/

doc/sphinx/national_resource.rst

@@ -0,0 +1,79 @@
+++++++++++++++++++++++++++
+US DIALS National Resource
+++++++++++++++++++++++++++
+
+.. _national_resource:
+
+In the US, DIALS is supported through a R24 National Resource grant (R24GM154040)
+from the `National Institutes of Health`_ / `National Institute of General Medical Sciences`_
+at `Lawrence Berkeley National Laboratory`_.  The goal of the resource is to support both
+users and facilities in using DIALS to analyse their diffraction data. This is part of a
+larger international software project that includes support from staff in the US, UK, and
+worldwide.
+
+.. container:: twocol
+
+   .. container:: leftside-nih
+
+        .. image:: https://www.nigms.nih.gov/Style%20Library/NIGMS2/images/nih.svg
+           :alt: NIH NIGMS
+           :target: `NIGMS`_
+           :height: 70px
+
+   .. container:: leftside-nih
+
+        | National Institute of
+        | General Medical Sciences
+
+.. rst-class:: clear-both
+
+Goals
+=====
+
+The goals of the Resource are two-fold: 1) support users, instrument scientists,
+and beamline scientists in using DIALS to analyze their data and 2) maintain and upgrade DIALS
+as a software product.
+
+Community support activities
+----------------------------
+
+DIALS has many use cases. Just a few of a variety of possible examples:
+
+-  The individual scientist processing data on their laptop
+-  The synchrotron beamline processing thousands of datasets a week an a local cluster
+-  The electron diffraction team at a university solving structures using FIB milling single crystal methods
+-  The XFEL researcher processing tens of thousands of images a minute on a nation-leading supercomputer
+
+In each case, different support is may be needed.  The individual scientist may need help with parameters for indexing.  The synchroton may need to minimize memory load.  The electron micrscope may require different spotfinder paramters than default.  The XFEL data may need new super-computing submission arguments.  For all of this and more, the Resource can help.  Simply email a staff member or join the community!
+
+Additionally, workshops and training for DIALS will be available and direct support can be arranged by way of site visits by Resource staff.  Please be in touch!
+
+Software activities
+-------------------
+
+DIALS is built on cctbx, and in turn many packages depend on DIALS.  The Resource conducts the following activities to maintain and upgrade DIALS:
+
+- Codebase optimization: as processing needs increase and data volumes accelerate, DIALS staff refactors and optimizes the codebase, profiling code, bringing critical algoritms into C++ or onto GPUs, and increasing code stability.
+- Codebase maintenance: the Resource tracks modern coding standards and upgrades the code to support them, be they Python, C++, or HDF5 standards.  Additionally as external libraries change and evolve, the Resource continually runs regression tests to monitor for changes that affect code function and performance
+- Provide user interfaces: through GUIs and command line programs, DIALS staff build tools that allow data visualization and statistical analysis.
+
+Resource administration
+=======================
+
+Staff
+-----
+
+- Aaron Brewster: Director, Principal Investigator, representative of DIALS to standards bodies and user communities
+- Daniel Paley: senior staff and lead for outreach, maintenance, and optimization
+- David Mittan-Moreau: synchrotron and MicroED integration
+- Herbert Bernstein: CBF and NeXus support and maintenance
+
+Board
+-----
+
+The DIALS advisory board will include crystallographic software developers, beamline facility staff, and representatives of standards bodies to ensure the needs of all aspects of the crystallographic data collection and processing community are being addressed in the DIALS development efforts.  Annual board meetings will include discussions of metrics, improvements, and assessment of progress, and will charge developers to move towards completing specific milestones and deliverables. The board is currently being assembled.
+
+.. _`Lawrence Berkeley National Laboratory`: http://www.lbl.gov/
+.. _`National Institutes of Health`: http://www.nih.gov/
+.. _`National Institute of General Medical Sciences`: https://www.nigms.nih.gov/
+.. _`NIGMS`: https://www.nigms.nih.gov/

doc/sphinx/publications.rst

@@ -6,6 +6,7 @@ Publications
 DIALS Journal Articles
 ----------------------
 
+* .. pubmed:: 39386198 Laue-DIALS
 * .. pubmed:: 35647922 xia2.multiplex
 * .. pubmed:: 34747533 DIALS as a toolkit
 * .. pubmed:: 32254063 dials.scale
@@ -28,6 +29,7 @@ DIALS Journal Articles
 DIALS Application to XFEL Crystallography
 -----------------------------------------
 
+* .. pubmed:: 35046599 Chemical Crystallography at XFELs
 * .. pubmed:: 26352473 synuclein
 * .. pubmed:: 26280336 synaptotagmin-SNARE
 * .. pubmed:: 26057680 Ginn 2015b

doc/sphinx/workshops/index.rst

@@ -3,34 +3,42 @@
 Workshops
 +++++++++
 
-.. toctree::
-
-   DIALS-1
-
-* `DIALS West-1`_: July 2012 (Boston, MA)
+* `Serial Crystallography Methods Workshop`_: July 2023 (Ithaca, NY)
 
-.. _`DIALS West-1`: http://cci.lbl.gov/dials/jul_2012_boston.htm
+.. _`Serial Crystallography Methods Workshop`: https://indico.classe.cornell.edu/event/2248/
 
-* `DIALS-2`_: February 2013 (Berkeley, CA)
+* `Berkeley Lab Serial Crystallography Workshop`_: Feburary 2017 (Berkeley, CA)
 
-.. _`DIALS-2`: http://cci.lbl.gov/dials/feb_2013_berkeley.htm
+.. _`Berkeley Lab Serial Crystallography Workshop`: http://cci.lbl.gov/xfel/index.php/2017_Tutorials
 
-.. toctree::
+* `DIALS-6`_: May 2015 (Berkeley, CA)
 
-   dials3d
+.. _`DIALS-6`: http://cci.lbl.gov/dials/may_2015_berkeley.htm
 
-* DIALS-4A: September 2013 (Diamond Light Source, UK)
+* DIALS-5:  November 2014 (Diamond Light Source, UK)
 
 .. _`xxx`: xxx
 
 * `DIALS-4B`_: October 2013 (Berkeley, CA)
 
 .. _`DIALS-4B`: http://cci.lbl.gov/dials/oct_2013_berkeley.htm
 
-* DIALS-5:  November 2014 (Diamond Light Source, UK)
+* DIALS-4A: September 2013 (Diamond Light Source, UK)
 
 .. _`xxx`: xxx
 
-* `DIALS-6`_: May 2015 (Berkeley, CA)
+.. toctree::
 
-.. _`DIALS-6`: http://cci.lbl.gov/dials/may_2015_berkeley.htm
+   dials3d
+
+* `DIALS-2`_: February 2013 (Berkeley, CA)
+
+.. _`DIALS-2`: http://cci.lbl.gov/dials/feb_2013_berkeley.htm
+
+* `DIALS West-1`_: July 2012 (Boston, MA)
+
+.. _`DIALS West-1`: http://cci.lbl.gov/dials/jul_2012_boston.htm
+
+.. toctree::
+
+   DIALS-1

installer/dials_installer.py

@@ -80,7 +80,7 @@ def reconfigure(self, log=None, *args, **kwargs):
             traceback.print_exc()
             print("\n")
             sys.exit(
-                "Please report this installation error to dials-support@lists.sourceforge.net"
+                "Please report this installation error at https://github.com/dials/dials/issues"
             )
 
     def reconfigure_as_libtbx_does(self, log):

newsfragments/2528.misc

@@ -0,0 +1 @@
+Update documentation to remove most references to sourceforge.

newsfragments/2780.misc

@@ -0,0 +1 @@
+Add new R24 National Resource for DIALS to the website

newsfragments/2787.bugfix

@@ -0,0 +1 @@
+``dials.generate_distortion_maps``: Correct definitions for creating elliptical distortion correction maps.

src/dials/algorithms/image/distortion/boost_python/distortion_ext.cc

@@ -1,25 +1,25 @@
 #include <boost/python.hpp>
 #include <boost/python/def.hpp>
-#include <dials/algorithms/image/distortion/ellipse.h>
+#include <dials/algorithms/image/distortion/plane.h>
 
 namespace dials { namespace algorithms { namespace boost_python {
 
   using namespace boost::python;
 
-  void export_create_elliptical_distortion_maps() {
-    class_<CreateEllipticalDistortionMaps>("CreateEllipticalDistortionMaps", no_init)
+  void export_create_plane_distortion_maps() {
+    class_<PlaneLinearTransformationMaps>("PlaneLinearTransformationMaps", no_init)
       .def(init<const Panel &,
                 scitbx::mat2<double>,
                 scitbx::vec3<double>,
                 scitbx::vec3<double>,
                 scitbx::vec3<double>>(
         (arg("panel"), arg("ellipse_matrix"), arg("fast"), arg("slow"), arg("mid"))))
-      .def("get_dx", &CreateEllipticalDistortionMaps::get_dx)
-      .def("get_dy", &CreateEllipticalDistortionMaps::get_dy);
+      .def("get_dx", &PlaneLinearTransformationMaps::get_dx)
+      .def("get_dy", &PlaneLinearTransformationMaps::get_dy);
   }
 
   BOOST_PYTHON_MODULE(dials_algorithms_image_distortion_ext) {
-    export_create_elliptical_distortion_maps();
+    export_create_plane_distortion_maps();
   }
 
 }}}  // namespace dials::algorithms::boost_python

src/dials/algorithms/image/distortion/ellipse.h renamed to src/dials/algorithms/image/distortion/plane.h

@@ -14,13 +14,13 @@ namespace dials { namespace algorithms {
   using scitbx::vec2;
   using scitbx::vec3;
 
-  class CreateEllipticalDistortionMaps {
+  class PlaneLinearTransformationMaps {
   public:
-    CreateEllipticalDistortionMaps(const Panel &panel,
-                                   const mat2<double> ellipse_matrix,
-                                   const vec3<double> fast,
-                                   const vec3<double> slow,
-                                   const vec3<double> mid) {
+    PlaneLinearTransformationMaps(const Panel &panel,
+                                  const mat2<double> matrix,
+                                  const vec3<double> fast,
+                                  const vec3<double> slow,
+                                  const vec3<double> mid) {
       std::size_t xsize = panel.get_image_size()[0];
       std::size_t ysize = panel.get_image_size()[1];
       dx_.resize(scitbx::af::c_grid<2>(ysize, xsize));
@@ -33,7 +33,7 @@ namespace dials { namespace algorithms {
           double x = offset * fast;  // undistorted X coordinate (mm)
           double y = offset * slow;  // undistorted Y coordinate (mm)
           vec2<double> distort =
-            ellipse_matrix * vec2<double>(x, y);  // distorted by ellipse matrix
+            matrix * vec2<double>(x, y);  // distorted by transformation matrix
 
           // store correction in units of the pixel size
           dx_(j, i) = (x - distort[0]) / panel.get_pixel_size()[0];