Establish `template-formula` as the go-to resource for all formula development

[myii]

This issue is a collection of ideas initially taken from the log of discussions^[1][2][3][4] captured in the wiki.

Visibility

• Make this template-formula a sticky formula for the whole SaltStack Formulas project.
  • Probably best to wait until most of these changes have been made, to prevent premature use of the template.

Release tagging (for each merged PR)

• Apply semantic versioning to this formula -- @aboe76.
• Prepare bump:... labels to use when evaluating the extent of change each PR will have on the formula. Helps when bumping version numbers for releases.
  • bump:major
  • bump:minor
  • bump:patch
  • Note: If this works well, there are methods available to sync labels across all of the repos, such as this search or using the same keywords in a search engine.
• Modify the CHANGELOG to use the format proposed by https://keepachangelog.com/en/1.0.0/:
  • If doing this, the relevant section in the main documentation must be updated as well.
  • An alternative method is to use https://github.com/vaab/gitchangelog; this automates the process but it does go completely against the advice at https://keepachangelog.com/en/1.0.0/.
  • @aboe76 looking into an automation method, with this as an example committed to the django-formula repo.
• Add an AUTHORS file, with the primary purpose of identifying the key maintainers of the repo.
  • @aboe76 looking into an automation method, with this as an example produced for the salt-formula repo.
• Configure semantic-release for this formula, including updating the changelog and updating the FORMULA file -- feat(semantic-release): configure for this formula #35.
• Configure commitlint to warn when commit messages aren't appropriate for bumping the version number -- ci(travis): include commitlint stage #41.
• Update authors file alongside bumping the version number -- feat(authors): update automatically alongside semantic-release #42.
• Update the table of contents in Markdown files alongside bumping the version number -- feat(toc): use markdown-toc directly to update inline #43.
• Convert Markdown files to RST alongside bumping the version number -- docs(tofs): replace existing .md with .rst and add to RTD #69.
• Collect feedback from the following PRs, which have implemented semantic-release (keeping this list in chronological order for the most part):
  • vault-formula
  • collectd-formula
  • nginx-formula
  • cert-formula
  • chrony-formula
  • rkhunter-formula
  • postgres-formula -- specific commit
  • keepalived-formula
  • systemd-formula
  • ufw-formula
  • salt-formula
  • fail2ban-formula
  • bind-formula
  • syslog-ng-formula
  • apt-formula
  • locale-formula
  • sudoers-formula
  • postfix-formula
  • iptables-formula
  • golang-formula -- with follow-up
  • logrotate-formula
  • php-formula -- with follow-up
  • deepsea-formula
  • mysql-formula
  • libvirt-formula
  • openvpn-formula
  • sysstat-formula
  • dhcpd-formula
  • ...
  • (Numerous other formulas converted after this point but tracked in the documentation instead).
  • In progress:
    • packages-formula
  • Other (direct, non-PR) implementations include:
    • prometheus-formula
    • grafana-formula
• The script pre-commit_semantic-release.sh could check for the existence of the docs dir.
• Investigate how the semantic-release (and/or the TOFS pattern?) can be used to mitigate the use of formula:ng states:
  • Most of those can be integrated and removed and only keep one formula.
  • Also refer to the discussions at:
    • Feature (rhel7/httpd 2.4) : hardening apache and code refactoring apache-formula#251
    • docs: remove obsolete CHANGELOG.rst file nginx-formula#233
  • Most complete conversion method (in stages) done for the php-formula:
    • fix: warn formula users ng states will be promoted in v1.0.0 php-formula#185
    • fix: update deprecation version number in semantic-release run php-formula#187
    • feat(ng): promote NG formula php-formula#183

YAML & Jinja

• Update to new map.jinja style -- update formula with map.jinja and style guide references, improve REA… #20.
• Issue with new map.jinja style due to defaults.merge vs. salt-ssh -- Upstream bug report: 'defaults.merge' is not available when using salt-ssh saltstack/salt#51605.
• Use intermediate map.jinja style for the time being -- map.jinja: use grains.filter_by instead of defaults.merge #25.
• Update YAML to show all possible OSes that can be supported -- Fix map.jinja and add more OSes #23.
• Add other used OS grains-based YAML files; also update map.jinja accordingly.
  • oscodename -- e.g. postgres-formula.
  • osfinger -- e.g. apache-formula -- map.jinja: use grains.filter_by instead of defaults.merge #25.
• Add a macros.jinja template with common macros that "Any formulae would find useful":
  • Repeatable patterns deserve repeatable code.
• Evaluate whether the 'masterless' extra_yamls approach outlined by @javierbertoli is another approach to solve scalability issues (awaiting sample PR).
• Decide a consistent method to work with jinja_lstrip_blocks and more specfically jinja_trim_blocks until the upstream issue is resolved.
  • Example of the problem and the side-effects of the workaround.
• Confirm (and then globally apply) standard of always using ~ for string concatenation instead of +.
  • Two sides of the argument starting here: php.ng.modules php-formula#172 (comment).
• Use | json instead of | tojson, at least until 2017.7 support is required -- refer back to this comment.
  • Another comment: Use cmd_check to validate config files consul-formula#37 (comment)
  • Commit showing 2018.3.3: saltstack/salt@1499c6a
  • Note that there are issues with both | json and | tojson, so | yaml may be preferable (at least in certain situations), see:
    • Unable to locate package on debian stretch postgres-formula#253 (comment)
    • Unable to locate package on debian stretch postgres-formula#253 (comment)
• Prefer {%- ... %} as a standard due to linked issues raised in this comment.
• Consider tracking "custom" grain values to the map to cover for situations when the actual grain
  is unavailable, such as osmajorrelease and osfinger on Arch Linux.
  • PR mentioning the issue: feat(linux): archlinux support (no osfinger grain) firewalld-formula#29
  • Potential solution at the end of map.jinja:
    • {%- do template.update({'osmajorrelease': salt['grains.get']('osmajorrelease', 0)|int }) %}
• Determine strategy regarding YAML keys with no given value such as comp_lzo: -- insist on explicit values at all times by modifying our yamllint configurations based on https://yamllint.readthedocs.io/en/stable/rules.html#module-yamllint.rules.empty_values?
• Discuss the use of yaml_dump.sls (via. ssf-formula) in all formulas so that changes to the map before and after can easily be picked up in the CI logs.

TOFS pattern vs. pillar

• Switch to the TOFS pattern (files_switch) -- Feature/template override and files switch #19.
• If required, update TOFS pattern to allow working with custom files -- Improve TOFS implementation for increased configurability #28.
  • See top of discussion on 2019-02-11 log of discussions.
• Use source_files instead of files in the pillar (otherwise there are two files, which will cause confusion) -- fix(tofs): use source_files instead of files #76.
• Use libtofs.jinja instead of macros.jinja, to work towards the simplest drop-in solution for other formulas -- refactor(tofs): move “files_switch” macro to “libtofs.jinja” #79.

Other pillar-avoidance methods to consider

• Consider changing all uses of pillar.get to config.get globally (as in, for all formulas eventually).
• Work through discussions captured from Slack where alternatives are discussed (TODO: capture the discussions in a wiki page here).

Make the formula portable

• Find a solution for tpldir not working in template files, see Can't use tpldir or slspath in template file saltstack/salt#41195.
  • There is the workaround of sending it through as a context variable.
• Test using tplroot throughout the formula and then changing the root directory to ensure the whole formula is using the new namespace.
• Consider this comment by @noelmcloughlin re: how to support old and new functionality and having a deprecation scheme.

Documentation

• Set up Read the Docs for documentation, while maintaining readability from GitHub (i.e. using the docs sub-directory.
• Set up Antora (Asciidoctor-based) documentation, to be deployed on GitHub pages.
  • Introducing Antora:
    • https://matthewsetter.com/why-antora-is-the-leading-technical-writing-platform/
    • https://matthewsetter.com/antoras-three-core-concepts/
  • List of showcase sites: https://gitlab.com/antora/antora.org/issues/20
  • Prepare template module structure for each of deploying new modules.
    • https://docs.antora.org/antora/2.0/modules/#pages-dir
    • Use .keep to push directory structure to Git.
    • Ensure entry in .gitignore to prevent assets being committed (also need the proper solution for these).
  • Collect reference links to all of the existing documentation sites out there, for gaining insights in how to perform org-wide configurations for ourselves (inc. Fedora, couchbase, ...)
    • https://github.com/couchbase/docs-site/blob/master/staging-antora-playbook.yml
    • https://github.com/couchbase/docs-site/blob/master/production-antora-playbook.yml
  • Later on, consider: https://www.algolia.com/for-open-source/
• Add documentation: guides, proposals, etc.
  • Use the template-formula wiki the central location for planning/organisation/reference.
  • Idea: @javierbertoli explaining a method that can be used for managing numerous formulas, when formulas are lacking version numbers (near top of 2019-02-12 log of discussions).
• Document how to contribute to the RTD-based documentation (e.g. template for new pages).
• Document how to create a formula from this template one, inside this repository or/and in saltstack docs -- @daks looking into this.
• Document how to apply all of the changes made to this formula to existing formulas.
• Document best practices for testing locally during development; current useful examples to build upon:
  • tmux-formula
  • postgres-formula
• Document the deprecation process for formulas, such as when promoting formula:ng to formula -- see fix: prevent running of states deprecated in v1.0.0 nginx-formula#236.

Tests and CI

• Add a test scaffold with kitchen-ci and inspec (or testinfra) examples for the basic tasks (add/remove package, check service, etc) -- want to replace old serverspec tests -- Add a testing scaffold with kitchen-ci, inspec and travis support #34.
  • Discussions so far seem to be leaning towards inspec.
  • Goss and saltcheck also mentioned -- perhaps alongside for the latter?
  • Also consider saltscaffold.
• Compare this with the formula-test-harness introduced to the nginx-formula in this pull.
  • See discussion from here to see why no longer considering this.
• Find solution for opensuse not working -- ci(kitchen): use salt-minion version of opensuse to ensure tests run #45.
• For instances that requiring restarting (e.g. opensuse usually works the second time its run), evaluate ways of minimising these delays:
  • Try travis_retry.
  • Try matrix: allow_failures: - os: opensuse-leap.
  • Resolved by ci(kitchen): use salt-minion version of opensuse to ensure tests run #45 -- restarts no longer required.
• Add tests for all clean stages -- @javierbertoli looking into this.
• Work through this list, closing (and even reverting?) the PRs for the formula-test-harness, since it only has 1 contributor and is no longer maintained.
• Add tests for TOFS, as suggested by @daks in fix(tofs): use source_files instead of files #76.
• Use specific versions in Gemfile, to avoid test failures -- fix(travis): use version numbers in Gemfile to prevent failed builds #81.
• Consider merging (locale-specific) provision_command for each platform, as used in the systemd-formula -- only if actually useful, not just for the sake of it.
• Use pre-salted images for testing for improved performance and reliability -- @javierbertoli has provided the image and @myii has made PR ci(kitchen): use pre-salted images instead #92.
• Update: .kitchen.yml => kitchen.yml -- done in refactor(kitchen): prefer kitchen.yml to .kitchen.yml #40.
• Update: pillars-from-files => pillars_from_files -- done in refactor(kitchen): pillars-from-files => pillars_from_files #74.

States

• Make the formula "actually functional" so that all parts can be run, to aid understanding and quality assurance.
• Adapt template-formula to not be too focussed on pkg:
  • Must also consider archives, tarball releases, GitHub repos, etc.
• Add the remove state advocated by @noelmcloughlin.
• Ensure that each file does only one thing and do it well (instead of monolithic files):
  • E.g. pkg/install.sls, pkg/clean.sls, repo/install.sls, repo/clean.sls, etc.
• Add 'formula components' that can be reused in a formula as suggested by @javierbertoli, such as:
  • 'service-template'
  • 'pkg-template'
  • 'ini-config-template'
  • 'yaml-config-template'
  • Etc.
• Prevent the formula from running on unsupported minions -- WIP: feat(unsupported): prevent formula running on unsupported minions #91.

Repo standards

• Complete all guidelines at https://github.com/saltstack-formulas/template-formula/community -- issue Complete the remaining items in the Community profile #70 opened for this for tracking progress.
• Consider how Security Advisories apply to us.
• Consider having a develop branch to allow PRs with the very latest changes, that can be merged back into master at the appropriate times.
  • As mentioned in the 4+ comments below starting from: Establish template-formula as the go-to resource for all formula development #21 (comment).
  • A real example of where this would be useful is 'gid_from_name' is an invalid keyword argument for 'user.present' (=> usergroup) users-formula#198.

Other resources to evaluate for inclusion

• Evaluate https://github.com/claudekenni/stack-formula.
• Evaluate https://github.com/hbokh/awesome-saltstack.
• Evaluate https://github.com/ministryofjustice/salt-shaker.
• Evaluate Improvements (enhancement backlog?) needed #18 to see if it can be merged into this issue.
• Evaluate https://github.com/salt-formulas to learn from their ecosystem as discussed in this comment and this comment.
• Need to determine the right processes for ensuring formulas work for all officially supported versions of Salt, while also clearing out redundant content for version that are no longer within the support lifecyle.
  • This also includes updating/removing for supported/unsupported platforms as well.
  • See this PR discussion as an example of the issues faced.

[daks]

Just an idea: document how to create a formula from this template one, inside this repository or/and in saltstack docs

[myii]

@daks I've added that to the task list above -- thanks for that suggestion.

[alxwr]

@myii I just wanted to say „Thank you!“ :-)
This formula enabled me to implement https://github.com/saltstack-formulas/prometheus-formula really fast. I found the structure and especially your turnkey Travis/kitchen configuration very helpful.