Experimental clib from doxygen

[ischoegl]

Changes proposed in this pull request

This PR seeks to implement a basis for future auto-generated Cantera APIs that are fully documented based on information provided in C++ docstrings.

• New sourcegen scaffolder generates an experimental clib version ("clib-experimental")
• The clib scaffolder is based on doxygen tag files and associated XML anchorfiles
• A proposed minimal YAML syntax is used to reference C++ functions and methods (for details, see file interfaces/sourcegen/sourcegen/_data/README.md)
• A portion of clib googletests is adopted for illustration and code coverage as test-clib-experimental
• Partial CI is added to the clang-compiler action
• API information is obtained without having to parse pre-existing CLib headers (i.e. the original approach for the .NET API)

This proof of concept re-implements approaches of the traditional CLib library and is newr feature-complete, although the conversion of zeroD and oneD objects is currently omitted. One other currently omitted feature involves the inclusion of code that is difficult to automate, although it is expected to be rare; in many instances, tweaks to the C++ interface may be the go-to approach instead. Edit: custom code is now supported.

The configuration involves YAML markup that only uses minimal information for each entry, e.g.

- name: newSolution
  implements: newSolution(const string&, const string&, const string&)
  uses: [thermo, kinetics, transport]
- name: name

which produces fully documented declarations

/**
 *  Create and initialize a new Solution manager from an input file.
 *
 *  @param infile       name of the input file
 *  @param name         name of the phase in the file. If this is blank, the first phase in the file is used.
 *  @param transport    name of the transport model.
 *  @returns            Handle to stored Solution object or -1 for exception handling.
 *
 *  @implements constructor: shared_ptr<Solution> newSolution(const string&, const string&, const string&)
 *  @relates Solution::thermo(), Solution::kinetics(), Solution::transport()
 */
int sol3_newSolution(const char* infile, const char* name, const char* transport);

/**
 *  Return the name of this Solution object.
 *
 *  @param handle       Handle to queried Solution object.
 *  @param[in] bufLen   Length of reserved array.
 *  @param[out] buf     Returned string value.
 *  @returns            Actual length of string or -1 for exception handling.
 *
 *  @implements getter: string Solution::name()
 */
int sol3_name(int handle, int bufLen, char* buf);

and implementations (click to expand)

int sol3_newSolution(const char* infile, const char* name, const char* transport)
{
    // constructor: shared_ptr<Solution> newSolution(const string&, const string&, const string&)
    try {
        auto obj = newSolution(infile, name, transport);
        int id = SolutionCabinet::add(obj);
        // add all associated objects
        if (obj->thermo()) {
            ThermoPhaseCabinet::add(obj->thermo(), id);
        }
        if (obj->kinetics()) {
            KineticsCabinet::add(obj->kinetics(), id);
        }
        if (obj->transport()) {
            TransportCabinet::add(obj->transport(), id);
        }
        return id;
    } catch (...) {
        return handleAllExceptions(-2, ERR);
    }
}

int sol3_name(int handle, int bufLen, char* buf)
{
    // getter: string Solution::name()
    try {
        string out = SolutionCabinet::at(handle)->name();
        copyString(out, buf, bufLen);
        return int(out.size()) + 1;
    } catch (...) {
        return handleAllExceptions(-1, ERR);
    }
}

Here is an example for a full YAML input used for code generation

sol3_auto.yaml (click to expand)

# Auto-generated CLib API for %Cantera's Solution class.
# Partially implements a replacement for CLib's traditional @c ct library.

# This file is part of Cantera. See License.txt in the top-level directory or
# at https://cantera.org/license.txt for license and copyright information.

prefix: sol3
base: Solution
parents: []  # List of parent classes
derived: [Interface]  # List of specializations
recipes:
- name: newSolution
  implements: newSolution(const string&, const string&, const string&)
  uses: [thermo, kinetics, transport]
- name: newInterface
  implements:
    newInterface(const string&, const string&, const vector<shared_ptr<Solution>>&)
  uses: [thermo, kinetics]
- name: del
  uses: [thermo, kinetics, transport]
- name: name
- name: setName
- name: thermo
- name: kinetics
- name: transport
- name: setTransportModel
  code: |-
    try {
        auto obj = SolutionCabinet::at(handle);
        TransportCabinet::del(
            TransportCabinet::index(*(obj->transport()), handle));
        obj->setTransportModel(model);
        return TransportCabinet::add(obj->transport(), handle);
    } catch (...) {
        return handleAllExceptions(-2, ERR);
    }
- name: nAdjacent
- name: adjacent
  implements: Solution::adjacent(size_t)
  uses: [thermo, kinetics, transport]
  what: constructor  # registers object in CLib storage
# - name: adjacentName
- name: cabinetSize

If applicable, fill in the issue number this pull request is fixing

See Cantera/enhancements#39
Partially addresses Cantera/enhancements#220

If applicable, provide an example illustrating new features this pull request is introducing

Run the following command from the Cantera root folder:

scons doxygen
python3 interfaces/sourcegen/run.py --api=clib --output=.
scons build clib_experimental=y

A partial test suite based on the traditional CLib ensures that code performs as expected:

scons test-clib-experimental

Checklist

• The pull request includes a clear description of this code change
• Commit messages have short titles and reference relevant issues
• Build passes (scons build & scons test) and unit tests address code coverage
• Style & formatting of contributed code follows contributing guidelines
• The pull request is ready for review

[bryanwweber]

Thanks Ingmar! I know this is still in draft but I wanted to leave a small comment that might be intrusive later

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.39%. Comparing base (10d7754) to head (83442a2).
Report is 41 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1835   +/-   ##
=======================================
  Coverage   74.39%   74.39%           
=======================================
  Files         382      382           
  Lines       53330    53330           
  Branches     9022     9021    -1     
=======================================
  Hits        39676    39676           
  Misses      10607    10607           
  Partials     3047     3047           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ischoegl]

@speth and @bryanwweber ... this is ready for a review.

As an aside, there is a minor caveat for the .NET workflow on Windows, where a Python 3.10 version apparently does not respect PEP 604. I'll address this together with other comments. Edit: nevermind ... windows-2022 appears to use an older system Python and my initial attempt to override failed.

[bryanwweber]

Thanks for all this work Ingmar! Nearly all of my comments here are suggestions to improve the type hinting. In some cases, you'll need to add an import (I think just for Iterable). Obviously these don't affect the runtime, but my thought was that if we're going to have type hints they should at least be complete 😄

[ischoegl]

Thanks for all this work Ingmar! Nearly all of my comments here are suggestions to improve the type hinting. In some cases, you'll need to add an import (I think just for Iterable). Obviously these don't affect the runtime, but my thought was that if we're going to have type hints they should at least be complete 😄

Thanks, @bryanwweber, for the prompt review! I agree that having done most of the type hints already, it makes sense to take it all the way. I adopted almost all of the suggestions directly and added some conversions, so the googletest suite is almost fully comparable to the traditional CLib version for the parts that I have converted..

[ischoegl]

[...] I'm looking forward to the opportunity to continue expanding this code generation beyond clib.

Yes, I believe this can be leveraged in the near future (especially for MATLAB, as the hand-coded part covers only a portion of the actual API). Fortran could be done in analogy to CLib (without depending on it).

One question I have is on the logistics of replacing the existing manual clib. Are you expecting that we will be able to complete the transition before the next Cantera release, such that we can avoid the need for the parallel set of functions and header files with the "3" suffix?

I had already created a tracking issue for this: Cantera/enhancements#220. For now, the "3"s are needed; a future removal is imho trivial. I can likely handle the transition, but am not planning on investing in code generation for other APIs.

[ischoegl]

@speth ... please let me know in case there's anything else. At this point, I have taken care of all review comments.