Diagnostic Packages

[njordr]

• Feature Name: diagnostic-packages
• Start Date: 2024-05-08

Summary

During the last three years, we created and published the OCP Test & Validation Output specifications. They are based on a json schema to standardize the output of a diag that would like to be OCP compliant (https://github.com/opencomputeproject/ocp-diag-core/blob/main/json_spec/README.md).
We could improve the diags replicability describing the setup to use, even if we work in different environments.

Motivation

This proposal aims to create a standard to setup the environment achieving a replicable and comparable results.

These are the goals to achieve:

• Describe the diag running environment (including main and accessories dependencies)
• Provide descriptive steps to setup, configure and run the diag
• Identify why this package could be useful
• Explain how to manage the output

Guide-level explanation

This new spec works with different objects inside a main container: the "Package". This container holds all the information on how to replicate the environment, running the diag/s, and work with the output. It also has some key informations about the diag/s and why the package is useful to the user (Diagnosis Path).

The objects allowed in a package are:

• Units: this object define the diag/s
• Diagnosis Paths: what the package is trying to solve (e.g. memory debug)
• Dependencies: all the resources the units need to run, except for those related to the environment
• Executors: all the information to run the units
• Worksets: the environments in which the units are able to provide the results expected
• Processors: how to run the units (sequentially, in parallel, etc.)
• Collectors: how to deal with the output

An example root structure

{
    "PackageDescription": "Diagnose silicon components",
    "PackageID": "awesome-package",
    "PackageName": "My wonderful package",
    "Units": [...],
    "DiagnosisPaths": [...],
    "Dependencies": [...],
    "Executors": [...],    
    "Worksets": [...],
    "Processors": [...],
    "Collectors": [...]
}

Reference-level explanation

Unit

The core component of the spec. A Unit is a definition of a diag in its aspects.

Example

"Units":
    [{
            "IdCard": {
                "Component": "Memory",
                "Disruptiveness": "D1",
                "Id": "unit-1",
                "Purpose": "Diagnose Memory",
                "Runtime": "R1"
            },
            "Dependencies":
            [
                "hardware-meta-asic"
            ],
            "Worksets": [
                "docker-centos-9"
            ],
            "Collectors": [
                "ocptv-collector"
            ],
            "Executors": [
                "awesome-diag-ddr",
                "awesome-diag-lls"
            ]
        }
]

Diagnosis Paths

This object define which is the diagnosis scope of the package.

Example

"DiagnosisPaths": [{
            "Name": "Memory",
            "Description": "Debug silicon memory",
            "Children": [{
                    "Name": "DDR",
                    "Description": "Debug DDR memory",
                    "Children": [{
                            "Name": "DDR-CCP",
                            "Description": "Debug DDR memory",
                            "Units": ["unit-1"],
                        }, {
                            "Name": "DDR-CPU",
                            "Description": "Debug DDR memory interaction with CPU",
                            "Units": ["unit-2"],
                        }
                    ]
                }
            ]
        }
    ]

Dependencies

These are the resources needed to run the unit (excluding OS, common libraries, etc.). They could be:

• Vendor: if the unit should run for a single or a list of vendors only
• Unit: list of other units this unit depends on
• Hardware: description of the hardware needed to run the unit. This could be similar to the HardwareInfo object in the output specs.

Example

"Dependencies":
    [{
            "Identifier": "vendor-meta"
            "Type": "vendor",
            "Name": "Meta"
        }, {
            "Identifier": "hardware-meta-asic",
            "Type": "hardware",
            "Model": "meta-asic"
        }, {
            "Identifier": "hardware-meta-nic",
            "Type": "hardware",
            "Model": "meta-nic"
        }
    ]

Executors

These are the definition of the units binaries and their input parameters.

Example

"Executors": [{
            "Identifier": "awesome-diag-ddr",
            "Cmd": "/usr/local/bin/awesome-diag",
            "Parameters": ["--ram-ddr", "--output", "/tmp/logs.json"]
        }, {
            "Identifier": "awesome-diag-lls",
            "Cmd": "/usr/local/bin/awesome-diag",
            "Parameters": ["--ram-lls", "--output", "/tmp/logs.json"]
        }, {
            "Identifier": "big-diag-pcie-dram",
            "Cmd": "/usr/local/bin/big-diag",
            "Parameters": ["--pcie-dram", "--output", "/tmp/logs.json"]
        }, {
            "Identifier": "big-diag-pcie-lls",
            "Cmd": "/usr/local/bin/big-diag",
            "Parameters": ["--pcie-lls", "--output", "/tmp/logs.json"]
        }, {
            "Identifier": "small-diag-pe",
            "Cmd": "/usr/local/bin/small-diag",
            "Parameters": ["--pe", "--output", "/tmp/logs.json"]
        }
    ]

Worksets

These are the definitions of the environment in which to run the units. They could be Public or not and belongs to any type of infrastructure as a code definition (Docker, MS Cloud, Google cloud, AWS Cloud, etc.).

Example

"Worksets": [{
            "Identifier": "docker-centos-9":
            "Type": "docker",
            "Public": true,
            "Url": "https://hub.docker.com/r/dokken/centos-stream-9"
        }, {
            "Identifier": "docker-ubuntu-2204":
            "Type": "docker",
            "Public": true,
            "Tag": "ubuntu:22.04"
        }
    ]

Processors

This section define how to run the units, their order and parallelism.

Example

"Processors": [{
            "Identifier": "proc-diag",
            "Threads": [{
                    "Identifier": "thread-ddr",
                    "Units": [
                        "unit-1"
                    ]
                }, {
                    "Identifier": "thread-pcie-pe",
                    "Units": [
                        "unit-2",
                        "unit-3"
                    ]
                }
            }
        ]
    }

Collectors

The collectors define how to retrieve and parse the output. It will make the result comparable, regardless of the computation eventually applied.

Example

"Collectors": [{
            "Identifier": "ocptv-collector",
            "Cmd": "/usr/local/bin/ocptv-collector.py",
            "Parameters": ["--input", "/tmp/logs.json"],
            "Worksets": [
                "docker-ubuntu-2204"
            ]
        }
    ]

Drawbacks

No drawbacks identified at the moment.

Rationale and alternatives

At the moment there is nothing else that I know is comparable to these specs.

Prior art

NA

Unresolved questions

NA

Future possibilities

These specs open the doors to additional tools and services:

• A public registry of packages
• Tools to parser and create the running environment starting from the worksets
• Different vendors could provide help on their website or in their manuals using the "Diagnostic Packages" definitions