Skip to content

Latest commit

 

History

History
352 lines (213 loc) · 14.8 KB

general_qa_setup.rst

File metadata and controls

352 lines (213 loc) · 14.8 KB

QA ToolBox Setup

Setup Qa-Toolbox

  1. Clone qa-toolbox

    $ git clone  https://github.com/TDycores-Project/qa-toolbox.git

Each test for the qa-toolbox consists of input decks for simulators, a configuration file, and an options file.

Tests can be run to produce a time slice (all locations at one time) and/or an observation point (one location at all times).

Running the Test Suite

  1. Cd into qa-toolbox and set up simulators.sim and config_files.txt. If running the default test this step can be skipped.

    1. Create a file called simulators.sim and set local paths to executables of the simulators. See default_simulators.sim as an example.
    2. Create a file called config_files.txt and set the local path to the configuration file for the desired tests. See default_config_files.txt as an example.

  2. Setup the documentation directory

    1. Make a new documentation directory. The documentation directory can be created anywhere. However, the default folder the qa-toolbox looks for is a /docs in the toolbox. If creating the documentation directory elsewhere the location must be specified when running the test suites (see Step 3 below). To make the default directory make sure you are in qa-toolbox and make a new directory call docs.
    $ mkdir docs
    1. Setup sphinx in the documentation directory and follow setup instructions.
    $ cd docs
$ sphinx-quickstart

  3. Move back in the qa-toolbox main directory and run the test suite using the Makefile.

    $ cd ..
$ make

    The optional parameter DOC_DIR can be set when running the makefile to indicate the location of the documentation directory. If DOC_DIR is not set the qa-toolbox will automatically assume the documentation directory is qa-toolbox/docs.

    $ make DOC_DIR=Users/username/Documents/documentation_folder

    Below is a full list of optional parameters that may be set when running the makefile:

    • DOC_DIR: (Default: qa-toolbox/docs) Specifies location of documentation directory
    • CURRENT_DIR: (Default: qa-toolbox) Specifies location where a user is running the qa-toolbox from (see setup QA repository below).
    • CONFIG_FILE: (Default: config_files.txt) Specifies filename which lists the configuration files for tests. (If config_files.txt does not exsist qa-toolbox will run default test).
    • SIMULATORS_FILE: (Default: simulators.sim) Specifies filename which lists the filepaths to executables. (If simulators.sim does not exsist will default to python).

  4. As the simulation is running, output files, saved figure results, and documentation files will be generated. They will be located in the same location as your input files.

    $ cd tracer
$ ls

1.0_Tracer_tracer_run1.png
1.0_Tracer_tracer_run1_error.png
1.0_Tracer_tracer_run1_error.stat
solute_transport.rst

    Image files are named based on the time (if time slice) or location (if observation), the template name, the run number, and the variable being plotted.

  5. To look at the documentation generated by the test suite, cd into the documentation directory and create the html files. For example, if the default doc_dir was chosen:

    $ cd docs
$ make clean
$ make html

Adding Tests to Suite

  1. To create a new test, create a new folder and cd into the folder. All tests placed in the same folder will be grouped together in the documentation. The default title of the group will be the folder name. This can be overwritten in the configuration file (see Step 4 below). If underscores are used in the folder name they will be replaced with spaces in the title for the documentation.

    $ mkdir my_test
$ cd my_test

  2. Create two or more input files for the desired simulators you wish to test. The input file has a file extension based on the simulator you wish to run, such as filename.pflotran, filename.python. The filename will be specified in the configuration file and must be the same for all simulators. For example, you can browse the input decks within the qa-toolbox tests. Note: If working in 2D, 3D, or calculating error only two simulators may be run at a time.

  3. The QA toolbox reads in an options file specified by the user in a standard .opt extension. The options file consists of a series of sections with key-value pairs.

    [section-name]
key = value

    Section names are all lower case with an underscore between words. Required section names are:

    • ouput_options

    Optional section names include:

    • swap_options
    • mapping_options
    • solution_convergence

    An example output_options section is as follows:

    [output_options]
times = 10.0 y, 50.0 y, 100.0 y
locations = 1.0 1.0 1.0, 5.0 1.0 1.0
plot_time_units = years
plot_dimension = 1D
plot_x_label = Time [yr], Distance X [m]
plot_y_label = Liquid Pressure, Liquid Pressure
plot_title = Pflotran Test
variables = liquid_pressure
plot_type = observation, time slice
plot_to_screen = True
plot_error = True
print_error = True
    • times: (Required for time slice) List of times to plot and compare solutions at. Must match the times of outputs created by simulators. Must specify units on last time or all times listed. If running a steady state problem use time: '-999' (No unit required for steady state).
    • locations: (Required for observation point) List of locations (x y z) where specified observation point(s) is indicated in simulator file. Units in [m].
    • plot_time_units: (Required) Units of time to be displayed on plot. (If steady state this variable will be ignored).
    • plot_dimension: (Required) Dimension of simulation. Options include: 1D, 2D, 3D. If plotting in 2D or 3D only two simulators may be tested at a time.
    • plot_x_label: (Required) Label to be put on x axis of plot. If plotting both a time slice and an observation file, two values must be specified here separted by a comma and order must match order of plot_type.
    • plot_y_label: (Required) Label to be put on y axis of plot. If plotting both a time slice and an observation file, two values must be specified here separted by a comma and order must match order of plot_type.
    • plot_title: (Required) Title to be displayed on plot.
    • variables: (Required) Variable to be plotted from the output files. Must match the simulator output format. Certain simulators have built in mapping. In PFLOTRAN and Tough3, variables liquid pressure and saturation are built in to match keywords Liquid Pressure and Liquid Saturation. If you wish to create a variable name different from what is outputted by the simulator custom mapping of variables can be specified in optional section mapping_options.
    • plot_type: (Optional, default: time slice) Observation if plotting observation point, time slice if plotting time slice. If plotting both order must match plot_x_label and plot_y_label.
    • plot_error: (Optional, default: False) True if plotting relative and absolute error, False if not. If True only two simulatos may be run at a time.
    • print_error: (Optional, default: False) When set to True a .stat file will be created with list of error metrics.
    • plot_to_screen: (Optional, default: False) When set to True images will pop up as python script is being run. Close images to continue running test.

    Optional section swap_options defines values of variables in input decks to be tested. Each value will correspond to a different run number when outputting figures.

    [swap_options]
method = list
nx = 20, 40
ny = 30, 50
    • method: (Default: list) Options: list, iterative.
      • List: Specifies list of values for different variables. All variables must have the same number of values. The length for each variable should be equal.
      • Iterative: Variables will be increased incrementally for an amount specified by max_attempts. A starting value and an increment should be specified sepearted by a comma. (For example: nx = 12,2 will start nx with a value of 12 and will multiply the value by 2 until max_attempts is reached.)
    • max_attemps: (Required if iterative) Maximum number of iterations to take with iterative method.

    Variables names are listed based on what is defined in the input decks. When defining the variable within the input deck the following format must be used swap{nx,10}.

    An example is shown in pflotran:

    GRID
  TYPE structured
  NXYZ swap{nx,10} 1 1
  BOUNDS
    0.d0 0.d0 0.d0
    100.d0 1.d0 1.d0
  END
END

    The optional section mapping_options can be used when trying to plot unconvential variables and when simulator output names do not match.

    [mapping_options]
Free X1 [M] = X1
Free_X1 [M] = X1

    where Free X1 [M] is the variable name outputted by the simulator and X1 is the variable listed under the variables key in output_options. As many key and value pairs can be listed as needed.

  4. Create the configuration file as a standard .cfg and specify the option file, input deck filenames, and simulators. The title variable is optional and will be displayed as the title for the test in the documentaiton. If no title is specified the title will be the input field for template.

    [OPTIONSFILENAME]
title = Test
template = filename
simulators = pflotran, python

    For example:

    [richards]
title = Kolditz 2.2.9 Test
template = kolditz_2_2_9
simulators = python, pflotran

    Where richards.opt is the options file and input decks are named kolditz_2_2_9.pflotran and kolditz_2_2_9.pflotran.

    Available simulators the toolbox can run include:

    • pflotran
    • tdycore
    • python
    • crunchflow
    • tough3

    If running multiple tests in the same folder, the title for the collection of tests can be set (default is name of folder tests are located in) with an optional info section in the configuration file:

    [info]
title = Kolditz Tests

  5. Move back into the qa-toolbox and update config_files.txt to point to the config file of the desired test and simulators.sim if new simulators were added and run the test suite.

Setup QA Repository

It may be desirable to create a new repository with the documentation folder and/or tests and run the qa-toolbox from the new repository. This can be done using the following steps:

  1. Make a new folder for the QA repository

    $ mkdir new-qa-repository

  2. Create tests either in the qa repository or else where and update qa-toolbox config_files.txt accordingly.

  3. Cd into the qa repository and create a documentation directory

    $ cd new-qa-repository
$ mkdir docs

  4. Setup sphinx in documentation directory and follow setup instructions.

    $ cd docs
$ sphinx-quickstart

  5. Setup makefile in qa-repository to run the toolbox.

    1. Cd out of documentation folder and open up new makefile in main directory

      $ cd ..
$ emacs makefile

    2. In makefile set python and directory to qa_toolbox path.

      PYTHON = python3
QA_TOOLBOX_DIR = ../qa-toolbox

    3. Run the qa_tests in the makefile by setting the directory and documentation directory.

      $(MAKE) --directory=$(QA_TOOLBOX_DIR) DOC_DIR=${PWD}/docs all;

Run Tests Incrementally

When running the tests as listed above the toolbox will automatically delete files generated by the toolbox. However, sometime it may be useful to simply add tests to current documentation instead of rerunning an entire list of tests. To overwrite this run the following command:

$ make run_incremental_tests

Documenting Tests

Documentation will automatically be generated when running the toolbox. Titles for tests and a collection of tests can be set in the configuration file (see above).

A problem description can be added to the documentation. Problem descriptions can be added to individual tests or to a collection of tests within the same folder. To document a problem description create a text file and name using the following file format:

  • template.description for individual test, where template is what is defined in the configuration file.
  • folder.description for collection of tests, where folder is the name of the folder the tests are located in.

Type in what you wish to be displayed under problem description in the documentation, such as references for test, grid spacing, initial conditions, material properties etc. Equations can be specified using sphinx math format.

An example problem description:

This problem is adapted from Kolditz, et al. (2015), Thermo-Hydro-Mechanical-Chemical Processes in Fractured Porous Media: Modelling and Benchmarking, Closed Form Solutions, Springer International Publishing, Switzerland. Section 2.2.9, pg.35, "A Transient 1D Pressure Distribution, Non-Zero Initial Pressure, Boundary Conditions of 1st and 2nd Kind."

The domain is a 100x1x1 meter rectangular beam extending along the positive x-axis. Two different grid spacings were tested. The first is made up of 10x1x1 hexahedral grid cells with dimensions 10x1x1 meters. The second is made up of 50x1x1 hexahedral grid cells with dimensions 2x1x1 meters.

The domain is composed of a single material and is assigned the following properties: porosity \phi = 0.20; permeability k = 1.0e-14 m^2; rock density \rho = 2,000 kg/m^3; fluid compressibility K = 1.0e-9 1/Pa; fluid viscosity \mu = 1.728e-3 Pa-sec.

The pressure is initially distributed according to p(x,t=0)=f(x), where f(x) is defined (in units of MPa) as

f(x) = 0  \hspace{0.25in} 0 \leq x < {L \over 10}

f(x) = {{10x} \over {3L}}-{1 \over 3}  \hspace{0.25in} {L \over 10} \leq x < {{4L} \over 10}

f(x) = 1  \hspace{0.25in} {{4L} \over 10} \leq x < {{6L} \over 10}

f(x) = 3-{{10x} \over {3L}}  \hspace{0.25in} {{6L} \over 10} \leq x < {{9L} \over 10}

f(x) = 0  \hspace{0.25in} {{9L} \over 10} \leq x \leq L

At the two boundaries, a no fluid flux condition is applied,

q(0,t) = 0

q(L,t) = 0

where L = 100 m. The transient pressure distribution is governed by,

{\phi K} {{\partial p} \over {\partial t}} = {k \over \mu} {{\partial^{2} p} \over {\partial x^{2}}}

With the initial pressure given, the solution is defined by,

 p(x,t) = {1 \over 2} + \sum_{n=1}^{\infty} exp\left({-\chi n^2 \pi^2 {t \over L^2}}\right)\left({80 \over {3(n\pi)^2}}\right) cos{{n \pi y} \over L} cos{{n\pi} \over 2} sin{{n\pi} \over 4} sin{{3n\pi} \over 20}

\chi = {{k} \over {\phi \mu K}}