Creating Diffing Rules

Link to proposed schema

(CURRENT) Code-based Rules

Currently all rules are provided by writing TypeScript/Javascript code. Rules must satisfy the following signature:

type RuleSignature = (
  data: Diff<any, any>,
  lhs?: OpenAPIV2.Document,
  rhs?: OpenAPIV2.Document
) => RuleResult | [RuleResult, string] | undefined;

Where RuleResult look like:

export enum RuleResult {
  /** The rule applies and flags a verified violation. Stop processing other rules for this diff. */
  FlaggedViolation = "F",
  /** Rule applies and verifies this is not a violation. Stop processing other rules for this diff. */
  NoViolation = "N",
}

Rules that implicitly or explicitly return undefined will continue processing other rules. If all rules have been processed and no rule could classify the diff as a violation or not, it is assumed to be a violation. Rules may also return a tuple with the result and a custom error message, though generally the diff alone should be sufficient. A rule may require the lhs and/or rhs documents to use in their logic, though most rules do not.

Flow

The intended flow is:

1. Service team runs tsp-client convert to convert their Swagger to TypeSpec.
2. Service team runs tsp-client compare and gets its initial set of diffs.
3. Service team iterates on the TypeSpec to drive the diff down to (ideally) zero by any of the following:
   1. Fixing the TypeSpec so there is no relevant diff with the TypeSpec-generated Swagger
   2. Adding rules to the base tool to ignore patterns that are semantically equivalent
   3. Adding custom rules to either group diffs into more manageable categories or suppress them from the diff
4. Once tsp-client compare says the two specs are equivalent, have the spec reviewed.
5. Reviewers must approve any new filtering rules.

Examples Rules

The goals of this are to provide tool users a way to quickly create rules (for suppression or grouping) without having to write code using the most common kinds of rule patterns. The expressiveness of the schema is not intended to enable the creation of all kinds of rules, but the key scenarios that are most commonly used. For more complex scenarios, rules written in TypeScript are the way to go.

1. Ignore Parameter Foo Added:

This is a very common type of "ignore" or "suppress" rule. These would be used to drive down the violations detected by the tool, and these rule additions would be reviewed and approved as needed.

{
  "name": "ignoreParameterFooAdded",
  "reason": "Ignores when we add a Foo parameter to RHS.",
  "result": "noViolation",
  "conditions": [
    {
      "pathIndex": 4,
      "equals": "parameters"
    },
    {
      "pathIndex": -1,
      "equals": "Foo"
    },
    {
      "kind": "N"
    }
  ]
}

- name: ignoreParameterFooAdded
  reason: Ignores when we add a Foo parameter to RHS.
  result: noViolation
  conditions:
    - pathIndex: 4
      equals: parameters
    - pathIndex: -1
      equals: Foo
    - kind: 'N'

2. Create a Custom Violation Grouping

This is not a suppression but is also a very common use for rules. It allows the tool to group violations and pull them out of the massive heap of "assumed" violations. 100 "assumed" violations may in fact be duplications of 1 issue, so grouping violations (using the --group-violations flag) in conjunction with these kinds of rules can help users sort and more quickly address issues in their spec.

{
  "name": "flagBarChanged",
  "reason": "Flags when Bar is the target of a diff.",
  "result": "flagged",
  "conditions": [
    {
      "pathIndex": 4,
      "equals": "parameters"
    },
    {
      "pathIndex": -1,
      "equals": "Bar"
    }
  ]
}

- name: flagBarChanged
  reason: Flags when Bar is the target of a diff.
  result: flagged
  conditions:
    - pathIndex: 4
      equals: parameters
    - pathIndex: -1
      equals: Bar

3. Avoid Repetition with OR Logic

Just an example of some of the OR logic. Usually, everything is treated as AND.

{
  "name": "ignoreIrrelevantParameters",
  "reason": "Ignores when irrelevant documentation parameters change.",
  "result": "noViolation",
  "conditions": [
    {
      "pathIndex": -1,
      "equals": ["description", "title", "summary"]
    }
  ]
}

- name: ignoreIrrelevantParameters
  reason: Ignores when irrelevant documentation parameters change.
  result: noViolation
  conditions:
  - pathIndex: -1
    equals:
    - description
    - title
    - summary

4. More Use of OR Logic

Here, pathIndex uses a slice (-2, -1) to target the last two elements of the path. If either of the last two elements are '$fooboo', then flag a violation (again, useful for grouping). This would trigger if "$fooboo" were the last added (in which case it would be the last path segement) or a property within "$fooboo" were changed (in which case it would be the second to last path segment).

{
  "name": "flagAnyOfChanged",
  "reason": "Flags when $fooboo is the target of a diff.",
  "result": "flagged",
  "conditions": [
    {
      "pathIndex": 4,
      "equals": "parameters"
    },
    {
      "pathIndex": [
        -2,
        -1
      ],
      "equals": "$fooboo"
    }
  ]
}

- name: flagAnyOfChanged
  reason: Flags when $fooboo is the target of a diff.
  result: flagged
  conditions:
    - pathIndex: 4
      equals: parameters
    - pathIndex: [-2, -1]
      equals: "$fooboo"