Skip to content

Commit 836a890

Browse files
SBGoodsSBGoods
authored
Rewrite Validate Command (#341)
* Initial validate implementation * Add function tests to TestFileMismatchCheck * Add MixedDirectoryCheck and refactor relevant tests * Add functions and guides to glob pattern * Update README.md * Update documentationDirGlob to pass tests * Add copyright headers * Resolve linting errors * Update README.md with a link to provider documentation guidelines * Resolve linting errors * Add changelog entries * Add `NumberOfFilesCheck` to validate * Make file paths OS-agnostic * Fix file path in test error assertion * Update Go version to `1.21` * Change wording to past tense on changelog entries * Remove unused file * Remove `filecheck` interface * Refactor `schemamd` and `functionmd` packages to `internal/` * Remove unused method * Revert "Refactor `schemamd` and `functionmd` packages to `internal/`" This reverts commit a1b156e. * Remove go compatability section * Switch to `yuin/goldmark` for frontmatter parsing * Remove unused test * Move global variables to top of file * Refactor `testcases` to use `map[string]struct` * Refactor to use `t.tempDir()` * Fix documentationGlobPattern and add more debug and info logs * Add negative scenario acceptance tests
1 parent 221ea1b commit 836a890

File tree

108 files changed

+4271
-214
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

108 files changed

+4271
-214
lines changed

‎.changes/unreleased/FEATURES-20240305-135426.yaml

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
kind: FEATURES
2+
body: 'validate: Added support for Provider-defined Function documentation to all checks'
3+
time: 2024-03-05T13:54:26.307742-05:00
4+
custom:
5+
Issue: "341"

‎.changes/unreleased/FEATURES-20240305-135726.yaml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
kind: FEATURES
2+
body: 'validate: Added `InvalidDirectoriesCheck` which checks for valid provider documentation
3+
folder structure'
4+
time: 2024-03-05T13:57:26.273538-05:00
5+
custom:
6+
Issue: "341"

‎.changes/unreleased/FEATURES-20240305-135933.yaml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
kind: FEATURES
2+
body: 'validate: Added `MixedDirectoriesCheck` which throws an error if both legacy
3+
documentation and registry documentation are found'
4+
time: 2024-03-05T13:59:33.741601-05:00
5+
custom:
6+
Issue: "341"

‎.changes/unreleased/FEATURES-20240305-140106.yaml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
kind: FEATURES
2+
body: 'validate: Added `NumberOfFilesCheck` which checks the number of provider
3+
documentation files against the registry limit'
4+
time: 2024-03-05T14:01:06.742843-05:00
5+
custom:
6+
Issue: "341"

‎.changes/unreleased/FEATURES-20240305-140234.yaml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
kind: FEATURES
2+
body: 'validate: Added `FileSizeCheck` which checks the provider documentation file
3+
size against the registry limit'
4+
time: 2024-03-05T14:02:34.112782-05:00
5+
custom:
6+
Issue: "341"

‎.changes/unreleased/FEATURES-20240305-140346.yaml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
kind: FEATURES
2+
body: 'validate: Added `FileExtensionCheck` which checks for valid provider documentation
3+
file extensions'
4+
time: 2024-03-05T14:03:46.816256-05:00
5+
custom:
6+
Issue: "341"

‎.changes/unreleased/FEATURES-20240305-140451.yaml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
kind: FEATURES
2+
body: 'validate: Added `FrontMatterCheck` which checks the YAML frontmatter of provider
3+
documentation for missing required fields or invalid fields'
4+
time: 2024-03-05T14:04:51.781688-05:00
5+
custom:
6+
Issue: "341"

‎.changes/unreleased/FEATURES-20240305-140622.yaml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
kind: FEATURES
2+
body: 'validate: Added `FileMismatchCheck` which checks the names/number of provider
3+
documentation files against the provider schema'
4+
time: 2024-03-05T14:06:22.168518-05:00
5+
custom:
6+
Issue: "341"

‎README.md

+22-1
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ Available commands are:
5252
the generate command is run by default
5353
generate generates a plugin website from code, templates, and examples
5454
migrate migrates website files from either the legacy rendered website directory (`website/docs/r`) or the docs rendered website directory (`docs/resources`) to the tfplugindocs supported structure (`templates/`).
55-
validate validates a plugin website for the current directory
55+
validate validates a plugin website
5656
5757
```
5858
@@ -81,6 +81,11 @@ Usage: tfplugindocs generate [<args>]
8181
$ tfplugindocs validate --help
8282
8383
Usage: tfplugindocs validate [<args>]
84+
85+
--provider-dir <ARG> relative or absolute path to the root provider code directory; this will default to the current working directory if not set
86+
--provider-name <ARG> provider name, as used in Terraform configurations
87+
--providers-schema <ARG> path to the providers schema JSON file, which contains the output of the terraform providers schema -json command. Setting this flag will skip building the provider and calling Terraform CLI
88+
--tf-version <ARG> terraform binary version to download. If not provided, will look for a terraform binary in the local environment. If not found in the environment, will download the latest version of Terraform
8489
```
8590
8691
`migrate` command:
@@ -145,6 +150,22 @@ Otherwise, the provider developer can set an arbitrary description like this:
145150
// ...
146151
```
147152

153+
#### Validate subcommand
154+
155+
The `validate` subcommand can be used to validate the provider website documentation against the [Terraform Registry's provider documentation guidelines](https://developer.hashicorp.com/terraform/registry/providers/docs) and provider documentation best practices. The current checks in the `validate` command are:
156+
157+
| Check | Description |
158+
|---------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
159+
| `InvalidDirectoriesCheck` | Checks for valid subdirectory structure and throws an error if an invalid Terraform Provider documentation subdirectory is found. |
160+
| `MixedDirectoriesCheck` | Throws an error if both legacy documentation (`/website/docs`) and registry documentation (`/docs`) are found. |
161+
| `NumberOfFilesCheck` | Throws an error if the number of files in a directory is larger than the registry limit. |
162+
| `FileSizeCheck` | Throws an error if the documentation file is above the registry storage limit. |
163+
| `FileExtensionCheck` | Throws an error if the extension of the given file is not a valid registry documentation extension. |
164+
| `FrontMatterCheck` | Checks the YAML frontmatter of documentation for missing required fields or invalid fields. |
165+
| `FileMismatchCheck` | Throws an error if the names/number of resources/datasources/functions in the provider schema does not match the names/number of files in the corresponding documentation directory |
166+
167+
All check errors are wrapped and returned as a single error message to stderr.
168+
148169
#### Migrate subcommand
149170
150171
The `migrate` subcommand can be used to migrate website files from either the legacy rendered website directory (`website/docs/r`) or the docs

‎cmd/tfplugindocs/main_test.go

+8
Original file line numberDiff line numberDiff line change
@@ -54,3 +54,11 @@ func Test_SchemaJson_MigrateAcceptanceTests(t *testing.T) {
5454
Dir: "testdata/scripts/schema-json/migrate",
5555
})
5656
}
57+
58+
func Test_SchemaJson_ValidateAcceptanceTests(t *testing.T) {
59+
t.Parallel()
60+
61+
testscript.Run(t, testscript.Params{
62+
Dir: "testdata/scripts/schema-json/validate",
63+
})
64+
}

‎cmd/tfplugindocs/testdata/scripts/schema-json/validate/framework_provider_error_file_mismatch.txtar

+127
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
# Copyright (c) HashiCorp, Inc.
2+
# SPDX-License-Identifier: MPL-2.0
3+
4+
# Run of tfplugindocs validate command with a misnamed file
5+
[!unix] skip
6+
! exec tfplugindocs validate --provider-name=terraform-provider-scaffolding --providers-schema=schema.json
7+
stderr 'Error executing command: validation errors found:'
8+
stderr 'matching resource for documentation file \(resource2.md\) not found, file is extraneous or incorrectly named'
9+
stderr 'missing documentation file for resource: scaffolding_example'
10+
11+
-- docs/data-sources/example.md --
12+
---
13+
subcategory: "Example"
14+
page_title: "Example: example_thing"
15+
description: |-
16+
Example description.
17+
---
18+
# Data Fields
19+
20+
Name: {{.Name}}
21+
Type: {{.Type}}
22+
-- docs/resources/resource2.md --
23+
---
24+
subcategory: "Example"
25+
page_title: "Example: example_thing"
26+
description: |-
27+
Example description.
28+
---
29+
# Data Fields
30+
31+
Name: {{.Name}}
32+
Type: {{.Type}}
33+
-- schema.json --
34+
{
35+
"format_version": "1.0",
36+
"provider_schemas": {
37+
"registry.terraform.io/hashicorp/scaffolding": {
38+
"provider": {
39+
"version": 0,
40+
"block": {
41+
"attributes": {
42+
"endpoint": {
43+
"type": "string",
44+
"description": "Example provider attribute",
45+
"description_kind": "markdown",
46+
"optional": true
47+
}
48+
},
49+
"description": "Example provider",
50+
"description_kind": "markdown"
51+
}
52+
},
53+
"resource_schemas": {
54+
"scaffolding_example": {
55+
"version": 0,
56+
"block": {
57+
"attributes": {
58+
"configurable_attribute": {
59+
"type": "string",
60+
"description": "Example configurable attribute",
61+
"description_kind": "markdown",
62+
"optional": true
63+
},
64+
"defaulted": {
65+
"type": "string",
66+
"description": "Example configurable attribute with default value",
67+
"description_kind": "markdown",
68+
"optional": true,
69+
"computed": true
70+
},
71+
"id": {
72+
"type": "string",
73+
"description": "Example identifier",
74+
"description_kind": "markdown",
75+
"computed": true
76+
}
77+
},
78+
"description": "Example resource",
79+
"description_kind": "markdown"
80+
}
81+
}
82+
},
83+
"data_source_schemas": {
84+
"scaffolding_example": {
85+
"version": 0,
86+
"block": {
87+
"attributes": {
88+
"configurable_attribute": {
89+
"type": "string",
90+
"description": "Example configurable attribute",
91+
"description_kind": "markdown",
92+
"optional": true
93+
},
94+
"id": {
95+
"type": "string",
96+
"description": "Example identifier",
97+
"description_kind": "markdown",
98+
"computed": true
99+
}
100+
},
101+
"description": "Example data source",
102+
"description_kind": "markdown"
103+
}
104+
}
105+
},
106+
"functions": {
107+
"example": {
108+
"description": "Given a string value, returns the same value.",
109+
"summary": "Echo a string",
110+
"return_type": "string",
111+
"parameters": [
112+
{
113+
"name": "input",
114+
"description": "Value to echo.",
115+
"type": "string"
116+
}
117+
],
118+
"variadic_parameter": {
119+
"name": "variadicInput",
120+
"description": "Variadic input to echo.",
121+
"type": "string"
122+
}
123+
}
124+
}
125+
}
126+
}
127+
}

‎cmd/tfplugindocs/testdata/scripts/schema-json/validate/framework_provider_error_mixed_directories.txtar

+127
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
# Copyright (c) HashiCorp, Inc.
2+
# SPDX-License-Identifier: MPL-2.0
3+
4+
# Run of tfplugindocs validate command with mixed directory structure
5+
[!unix] skip
6+
! exec tfplugindocs validate --provider-name=terraform-provider-scaffolding --providers-schema=schema.json
7+
stderr 'Error executing command: validation errors found:'
8+
stderr 'mixed Terraform Provider documentation directory layouts found, must use only legacy or registry layout'
9+
10+
-- website/docs/d/example.html.md --
11+
---
12+
subcategory: "Example"
13+
layout: "example"
14+
page_title: "Example: example_thing"
15+
description: |-
16+
Example description.
17+
---
18+
# Data Fields
19+
20+
Name: {{.Name}}
21+
Type: {{.Type}}
22+
-- docs/data-sources/example.md --
23+
---
24+
subcategory: "Example"
25+
page_title: "Example: example_thing"
26+
description: |-
27+
Example description.
28+
---
29+
# Data Fields
30+
31+
Name: {{.Name}}
32+
Type: {{.Type}}
33+
-- schema.json --
34+
{
35+
"format_version": "1.0",
36+
"provider_schemas": {
37+
"registry.terraform.io/hashicorp/scaffolding": {
38+
"provider": {
39+
"version": 0,
40+
"block": {
41+
"attributes": {
42+
"endpoint": {
43+
"type": "string",
44+
"description": "Example provider attribute",
45+
"description_kind": "markdown",
46+
"optional": true
47+
}
48+
},
49+
"description": "Example provider",
50+
"description_kind": "markdown"
51+
}
52+
},
53+
"resource_schemas": {
54+
"scaffolding_example": {
55+
"version": 0,
56+
"block": {
57+
"attributes": {
58+
"configurable_attribute": {
59+
"type": "string",
60+
"description": "Example configurable attribute",
61+
"description_kind": "markdown",
62+
"optional": true
63+
},
64+
"defaulted": {
65+
"type": "string",
66+
"description": "Example configurable attribute with default value",
67+
"description_kind": "markdown",
68+
"optional": true,
69+
"computed": true
70+
},
71+
"id": {
72+
"type": "string",
73+
"description": "Example identifier",
74+
"description_kind": "markdown",
75+
"computed": true
76+
}
77+
},
78+
"description": "Example resource",
79+
"description_kind": "markdown"
80+
}
81+
}
82+
},
83+
"data_source_schemas": {
84+
"scaffolding_example": {
85+
"version": 0,
86+
"block": {
87+
"attributes": {
88+
"configurable_attribute": {
89+
"type": "string",
90+
"description": "Example configurable attribute",
91+
"description_kind": "markdown",
92+
"optional": true
93+
},
94+
"id": {
95+
"type": "string",
96+
"description": "Example identifier",
97+
"description_kind": "markdown",
98+
"computed": true
99+
}
100+
},
101+
"description": "Example data source",
102+
"description_kind": "markdown"
103+
}
104+
}
105+
},
106+
"functions": {
107+
"example": {
108+
"description": "Given a string value, returns the same value.",
109+
"summary": "Echo a string",
110+
"return_type": "string",
111+
"parameters": [
112+
{
113+
"name": "input",
114+
"description": "Value to echo.",
115+
"type": "string"
116+
}
117+
],
118+
"variadic_parameter": {
119+
"name": "variadicInput",
120+
"description": "Variadic input to echo.",
121+
"type": "string"
122+
}
123+
}
124+
}
125+
}
126+
}
127+
}

0 commit comments

Comments
 (0)