Skip to content

Commit d4bb0f5

Browse files
nhtruongNaarcha-AWSdblocknatebower
authored
[Jekyll] Spec Insert Plugin (#8692)
* Spec Insert A program that insert API Components generated from the OpenSearch OpenAPI Spec into markdown files Signed-off-by: Theo Truong <[email protected]> * # Sentence casing Signed-off-by: Theo Truong <[email protected]> * # vale:reviewdog Signed-off-by: Theo Truong <[email protected]> * # vale:reviewdog Signed-off-by: Theo Truong <[email protected]> * # vale:reviewdog Signed-off-by: Theo Truong <[email protected]> * # Correction on cron job run time. Signed-off-by: Theo Truong <[email protected]> * Apply suggestions from code review Co-authored-by: Naarcha-AWS <[email protected]> Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]> Signed-off-by: Theo N. Truong <[email protected]> * # More clarity in method documentation Signed-off-by: Theo Truong <[email protected]> * Update DEVELOPER_GUIDE.md * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> --------- Signed-off-by: Theo Truong <[email protected]> Signed-off-by: Theo N. Truong <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> Co-authored-by: Naarcha-AWS <[email protected]> Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]> Co-authored-by: Nathan Bower <[email protected]>
1 parent 872a495 commit d4bb0f5

37 files changed

+1345
-17
lines changed

‎.github/workflows/jekyll-spec-insert.yml

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
name: Lint and Test Jekyll Spec Insert
2+
on:
3+
push:
4+
paths:
5+
- 'spec-insert/**'
6+
pull_request:
7+
paths:
8+
- 'spec-insert/**'
9+
jobs:
10+
lint-and-test:
11+
runs-on: ubuntu-latest
12+
steps:
13+
- uses: actions/checkout@v4
14+
- uses: ruby/setup-ruby@v1
15+
with: { ruby-version: 3.3.0 }
16+
- run: bundle install
17+
- working-directory: spec-insert
18+
run: |
19+
bundle exec rubocop
20+
bundle exec rspec

‎.github/workflows/update-api-components.yml

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
name: Update API Components
2+
on:
3+
workflow_dispatch:
4+
schedule:
5+
- cron: "0 0 * * 0" # Every Sunday at midnight GMT
6+
jobs:
7+
update-api-components:
8+
if: ${{ github.repository == 'opensearch-project/documentation-website' }}
9+
runs-on: ubuntu-latest
10+
permissions:
11+
contents: write
12+
pull-requests: write
13+
steps:
14+
- uses: actions/checkout@v4
15+
with:
16+
submodules: recursive
17+
fetch-depth: 0
18+
19+
- run: git config --global pull.rebase true
20+
21+
- uses: ruby/setup-ruby@v1
22+
with: { ruby-version: 3.3.0 }
23+
24+
- run: bundle install
25+
26+
- name: Download spec and insert into documentation
27+
run: bundle exec jekyll spec-insert
28+
29+
- name: Get current date
30+
id: date
31+
run: echo "date=$(date +'%Y-%m-%d')" >> $GITHUB_ENV
32+
33+
- name: GitHub App token
34+
id: github_app_token
35+
uses: tibdex/[email protected]
36+
with:
37+
app_id: ${{ secrets.APP_ID }}
38+
private_key: ${{ secrets.APP_PRIVATE_KEY }}
39+
40+
- name: Create pull request
41+
uses: peter-evans/create-pull-request@v6
42+
with:
43+
token: ${{ steps.github_app_token.outputs.token }}
44+
commit-message: "Updated API components to reflect the latest OpenSearch API spec (${{ env.date }})"
45+
title: "[AUTOCUT] Update API components to reflect the latest OpenSearch API spec (${{ env.date }})"
46+
body: |
47+
Update API components to reflect the latest [OpenSearch API spec](https://github.com/opensearch-project/opensearch-api-specification/releases/download/main-latest/opensearch-openapi.yaml).
48+
Date: ${{ env.date }}
49+
branch: update-api-components-${{ env.date }}
50+
base: main
51+
signoff: true
52+
labels: autocut

‎DEVELOPER_GUIDE.md

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
# Developer guide
2+
- [Introduction](#introduction)
3+
- [Starting the Jekyll server locally](#starting-the-jekyll-server-locally)
4+
- [Using the spec-insert Jekyll plugin](#using-the-spec-insert-jekyll-plugin)
5+
- [Inserting query parameters](#inserting-query-parameters)
6+
- [Inserting path parameters](#inserting-path-parameters)
7+
- [Inserting paths and HTTP methods](#inserting-paths-and-http-methods)
8+
- [Ignoring files and folders](#ignoring-files-and-folders)
9+
- [CI/CD](#cicd)
10+
11+
## Introduction
12+
13+
The `.md` documents in this repository are rendered into HTML pages using [Jekyll](https://jekyllrb.com/). These HTML pages are hosted on [opensearch.org](https://opensearch.org/docs/latest/).
14+
15+
## Starting the Jekyll server locally
16+
You can run the Jekyll server locally to view the rendered HTML pages using the following steps:
17+
18+
1. Install [Ruby](https://www.ruby-lang.org/en/documentation/installation/) 3.1.0 or later for your operating system.
19+
2. Install the required gems by running `bundle install`.
20+
3. Run `bundle exec jekyll serve` to start the Jekyll server locally (this can take several minutes to complete).
21+
4. Open your browser and navigate to `http://localhost:4000` to view the rendered HTML pages.
22+
23+
## Using the `spec-insert` Jekyll plugin
24+
The `spec-insert` Jekyll plugin is used to insert API components into Markdown files. The plugin downloads the [latest OpenSearch specification](https://github.com/opensearch-project/opensearch-api-specification) and renders the API components from the spec. This aims to reduce the manual effort required to keep the documentation up to date.
25+
26+
To use this plugin, make sure that you have installed Ruby 3.1.0 or later and the required gems by running `bundle install`.
27+
28+
Edit your Markdown file and insert the following snippet where you want render an API component:
29+
30+
```markdown
31+
<!-- spec_insert_start
32+
api: <API_NAME>
33+
component: <COMPONENT_NAME>
34+
other_param: <OTHER_PARAM>
35+
-->
36+
37+
This is where the API component will be inserted.
38+
Everything between the `spec_insert_start` and `spec_insert_end` tags will be overwritten.
39+
40+
<!-- spec_insert_end -->
41+
```
42+
43+
Then run the following Jekyll command to render the API components:
44+
```shell
45+
bundle exec jekyll spec-insert
46+
```
47+
48+
If you are working on multiple Markdown files and do not want to keep running the `jekyll spec-insert` command, you can add the `--watch` (or `-W`) flag to the command to watch for changes in the Markdown files and automatically render the API components:
49+
50+
```shell
51+
bundle exec jekyll spec-insert --watch
52+
```
53+
54+
Depending on the text editor you are using, you may need to manually reload the file from disk to see the changes applied by the plugin if the editor does not automatically reload the file periodically.
55+
56+
The plugin will pull the newest OpenSearch API spec from its [repository](https://github.com/opensearch-project/opensearch-api-specification) if the spec file does not exist locally or if it is older than 24 hours. To tell the plugin to always pull the newest spec, you can add the `--refresh-spec` (or `-R`) flag to the command:
57+
58+
```shell
59+
bundle exec jekyll spec-insert --refresh-spec
60+
```
61+
62+
### Inserting query parameters
63+
64+
To insert the API query parameters table, use the following snippet:
65+
66+
```markdown
67+
<!-- spec_insert_start
68+
api: cat.indices
69+
component: query_parameters
70+
-->
71+
<!-- spec_insert_end -->
72+
```
73+
74+
This will insert the query parameters of the `cat.indices` API into the `.md` file with three default columns: `Parameter`, `Type`, and `Description`. There are five columns that can be inserted: `Parameter`, `Type`, `Description`, `Required`, and `Default`. When `Required`/`Default` is not chosen, the information will be written in the `Description` column.
75+
76+
You can customize the query parameters table with the following columns:
77+
78+
- `Parameter`
79+
- `Type`
80+
- `Description`
81+
- `Required`
82+
- `Default`
83+
84+
You can also customize this component with the following settings:
85+
86+
- `include_global` (Boolean; default is `false`): Includes global query parameters in the table.
87+
- `include_deprecated` (Boolean; default is `true`): Includes deprecated parameters in the table.
88+
- `pretty` (Boolean; default is `false`): Renders the table in the pretty format instead of the compact format.
89+
90+
The following snippet inserts the specified columns into the query parameters table:
91+
92+
```markdown
93+
<!-- spec_insert_start
94+
api: cat.indices
95+
component: query_parameters
96+
include_global: true
97+
include_deprecated: false
98+
pretty: true
99+
-->
100+
<!-- spec_insert_end -->
101+
```
102+
103+
### Inserting path parameters
104+
105+
To insert the `indices.create` API path parameters table, use the following snippet:
106+
107+
```markdown
108+
<!-- spec_insert_start
109+
api: indices.create
110+
component: path_parameters
111+
-->
112+
<!-- spec_insert_end -->
113+
```
114+
115+
This table behaves identically to the query parameters table except that it does not accept the `include_global` argument.
116+
117+
### Inserting paths and HTTP methods
118+
119+
To insert paths and HTTP methods for the `search` API, use the following snippet:
120+
121+
```markdown
122+
<!-- spec_insert_start
123+
api: search
124+
component: paths_and_http_methods
125+
-->
126+
<!-- spec_insert_end -->
127+
```
128+
129+
### Ignoring files and folders
130+
131+
The `spec-insert` plugin ignores all files and folders listed in the [./_config.yml#exclude](./_config.yml) list, which is also the list of files and folders that Jekyll ignores.
132+
133+
### CI/CD
134+
135+
The `spec-insert` plugin is run as part of the CI/CD pipeline to ensure that the API components are up to date in the documentation. This is performed through the [update-api-components.yml](.github/workflows/update-api-components.yml) GitHub Actions workflow, which creates a pull request containing the updated API components every Sunday.

‎Gemfile

Lines changed: 29 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,9 @@
1-
source "http://rubygems.org"
1+
# frozen_string_literal: true
2+
3+
source 'https://rubygems.org'
4+
5+
# Manually add csv gem since Ruby 3.4.0 no longer includes it
6+
gem 'csv', '~> 3.0'
27

38
# Hello! This is where you manage which Jekyll version is used to run.
49
# When you want to use a different version, change it below, save the
@@ -8,12 +13,12 @@ source "http://rubygems.org"
813
#
914
# This will help ensure the proper Jekyll version is running.
1015
# Happy Jekylling!
11-
gem "jekyll", "~> 4.3.2"
16+
gem 'jekyll', '~> 4.3.2'
1217

1318
# This is the default theme for new Jekyll sites. You may change this to anything you like.
14-
gem "just-the-docs", "~> 0.3.3"
15-
gem "jekyll-remote-theme", "~> 0.4"
16-
gem "jekyll-redirect-from", "~> 0.16"
19+
gem 'jekyll-redirect-from', '~> 0.16'
20+
gem 'jekyll-remote-theme', '~> 0.4'
21+
gem 'just-the-docs', '~> 0.3.3'
1722

1823
# If you want to use GitHub Pages, remove the "gem "jekyll"" above and
1924
# uncomment the line below. To upgrade, run `bundle update github-pages`.
@@ -22,21 +27,31 @@ gem "jekyll-redirect-from", "~> 0.16"
2227

2328
# If you have any plugins, put them here!
2429
group :jekyll_plugins do
25-
gem "jekyll-last-modified-at"
26-
gem "jekyll-sitemap"
30+
gem 'jekyll-last-modified-at'
31+
gem 'jekyll-sitemap'
32+
gem 'jekyll-spec-insert', :path => './spec-insert'
2733
end
2834

2935
# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
30-
gem "tzinfo-data", platforms: [:mingw, :mswin, :x64_mingw, :jruby]
36+
gem 'tzinfo-data', platforms: %i[mingw mswin x64_mingw jruby]
3137

3238
# Performance-booster for watching directories on Windows
33-
gem "wdm", "~> 0.1.0" if Gem.win_platform?
39+
gem 'wdm', '~> 0.1.0' if Gem.win_platform?
3440

3541
# Installs webrick dependency for building locally
36-
gem "webrick", "~> 1.7"
37-
42+
gem 'webrick', '~> 1.7'
3843

3944
# Link checker
40-
gem "typhoeus"
41-
gem "ruby-link-checker"
42-
gem "ruby-enum"
45+
gem 'ruby-enum'
46+
gem 'ruby-link-checker'
47+
gem 'typhoeus'
48+
49+
# Spec Insert
50+
gem 'activesupport', '~> 7'
51+
gem 'mustache', '~> 1'
52+
53+
group :development, :test do
54+
gem 'rspec'
55+
gem 'rubocop', '~> 1.44', require: false
56+
gem 'rubocop-rake', require: false
57+
end

‎README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
# About the OpenSearch documentation repo
44

55
The `documentation-website` repository contains the user documentation for OpenSearch. You can find the rendered documentation at [opensearch.org/docs](https://opensearch.org/docs).
6+
The markdown files in this repository are rendered into HTML pages using [Jekyll](https://jekyllrb.com/). Check the [DEVELOPER_GUIDE](DEVELOPER_GUIDE.md) for more information about how to use Jekyll for this repository.
67

78

89
## Contributing

‎_config.yml

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -311,6 +311,7 @@ plugins:
311311
- jekyll-remote-theme
312312
- jekyll-redirect-from
313313
- jekyll-sitemap
314+
- jekyll-spec-insert
314315

315316
# This format has to conform to RFC822
316317
last-modified-at:
@@ -320,13 +321,21 @@ last-modified-at:
320321
# The following items will not be processed, by default. Create a custom list
321322
# to override the default setting.
322323
exclude:
324+
- README.md
325+
- DEVELOPER_GUIDE.md
323326
- Gemfile
324327
- Gemfile.lock
325328
- node_modules
326329
- vendor/bundle/
327330
- vendor/cache/
328331
- vendor/gems/
329332
- vendor/ruby/
330-
- README.md
331-
- .idea
332-
- templates
333+
- templates/
334+
- .sass-cache/
335+
- .jekyll-cache/
336+
- .idea/
337+
- .github/
338+
- .bundle/
339+
- _site/
340+
- spec-insert
341+
- release-notes

‎spec-insert/.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
opensearch-openapi.yaml
2+
rspec_examples.txt

‎spec-insert/.rspec

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
--require spec_helper

‎spec-insert/.rubocop.yml

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
require: rubocop-rake
2+
AllCops:
3+
Include:
4+
- 'lib/**/*.rb'
5+
- 'Rakefile'
6+
NewCops: enable
7+
8+
Metrics/CyclomaticComplexity:
9+
Enabled: false
10+
Metrics/MethodLength:
11+
Enabled: false
12+
Metrics/ParameterLists:
13+
Enabled: false
14+
Metrics/AbcSize:
15+
Enabled: false
16+
Metrics/PerceivedComplexity:
17+
Enabled: false
18+
19+
Layout/EmptyLineAfterGuardClause:
20+
Enabled: false
21+
22+
Style/MultilineBlockChain:
23+
Enabled: false
24+
Style/SingleLineMethods:
25+
Enabled: false
26+
27+
Naming/FileName:
28+
Exclude:
29+
- 'lib/jekyll-spec-insert.rb' # For Jekyll to recognize the plugin

‎spec-insert/jekyll-spec-insert.gemspec

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# frozen_string_literal: true
2+
3+
Gem::Specification.new do |spec|
4+
spec.name = 'jekyll-spec-insert'
5+
spec.version = '0.1.0'
6+
spec.authors = ['Theo Truong']
7+
spec.email = ['[email protected]']
8+
9+
spec.summary = 'A Jekyll plugin for inserting OpenSearch OpenAPI specifications into Jekyll sites.'
10+
11+
spec.files = Dir['lib/**/*.rb']
12+
spec.require_paths = ['lib']
13+
14+
spec.metadata['rubygems_mfa_required'] = 'true'
15+
spec.required_ruby_version = '>= 3.1.0'
16+
end

0 commit comments

Comments
 (0)