Warning

Morio is currently early-access alpha code.

We are building this in the open, so feel free to look around, try it out, or leave feedback.

That being said, you probably don't want to run this in production yet.

Morio Template Hub

This repository holds a curated collection of templates for Morio.

There are four top-level folders here:

• bundles: These bundle both client templates and collector overlays
• modules: These Morio client modules
• overlays: These are Moriod settings overlays
• processors: These are stream processors for the tap service
• watchers: These are monitors for the watcher service (heartbeat)

Note

This is a work in progress. YMMV.

bundles

These are moriod configuration overlays that bundle multiple moriohub entries.

modules

The Morio client modules are stored in the modules folder.

These (Morio client) modules provide configuration for the different agents that are bundled by the Morio client. These agents gather different types of data:

• audit: Audit info is collected by Auditbeat
• logs: Collected by Filebeat
• metrics: Collected by Metricbeat

Great observability requires meticulous configuration of each of these agents. When you have more than a handful of system to configure -- whether it is a couple dozen, hundreds, or even several thousands -- you will want to apply some automation.

Maintainability and (facilitating) automation are some of Morio's design goals, so naturally we want to come up with a way to take the gruntwork out of this, while still giving you the flexibility to fine-tune the configuration of your systems.

This repository exists to facilitate that, by providing a library of client modules that you can use.

Tip

Refer to the Preseeding Guide for all details

To learn how you can ensure these modules are bundled with the Morio client, refer to the preseeding guide.

Rules governing client modules

Below are some rules to ensure each module plays nice within the Morio ecosystem:

• Modules shall have a unique name that is descriptive
• Modules that are platform-specific shall be prefixed by the platform they support followed by a dash.. One of linux-, macos-, or windows-.
• Modules that are platform-agnostic shall not have a platform prefix
• Module names shall only use [a-z][0-9]-
• Modules can provide one or more of the following files:
  • morio/audit/module-templates.d/[module-name].yaml: The module configuration for Auditbeat
  • morio/audit/rule-templates.d/[module-name].rules: A single rules file for auditd
  • morio/audit/rule-templates.d/[module-name]-*.rules: If a module utilzes multiple rules files, prefix them with the module name and a dash
  • morio/logs/module-templates.d/[module-name].yaml: The module configuration for Filebeat. Create an empty placeholder file if your module only provide inputs.
  • morio/logs/input-templates.d/[module-name].yaml: The input configuration for Filebeat
  • morio/metrics/module-templates.d/[module-name].yaml: The configuration for Metricbeat

Morio client module

Each of the various beats agents takes a YAML file as configuration.

The rules files used by Audutbeat are an exception, and this section does not apply to them.

To balance the ease-of-use of having a library of templates you can re-use with the requirement to be able to adapt the configuration to your specific needs, this repository does not host YAML files but rather templates that can be converted to YAML files by the Morio client (or by anyone utilizing Mustache templates.

In addition, the documentation for each template is included in the module file itself.

Module file structure

Let's look at an example of a module file structure to make this all a bit more tangible::

# This: {{ EXAMPLE }}
# is a mustache tag.
# The Morio client will replace it with whatever is stored in the EXAMPLE variable.
#
# The # prefix indicates that this block will only be rendered
# when the variable following it is set.
# So this: {{#MORIO_DOCS}}
# Means that whatever follows will only be rendered when MORIO_DOCS is set.
# To close such a block, use: {{/MORIO_DOCS}}
{{#MORIO_DOCS}}
# We are now inside a block that will only rendered when MORIO_DOCS is set.
# We use this to extract the documentation info which is included in this block.
# Your module files should follow this same structure.
#
# The 'about' key holds a description of the module. Multi-line is ok.
about: |-
  A metricbeat module for Linux system

  This leverages the `system` metricbeat module to gather basic data from a
  Linux system.
# The 'vars' key holds information about the vars used in this module.
vars:
  # The 'vars.local' key holds an object/map with name/description pairs
  # for the vars that are specific to this file.
  local:
    LINUX_SYSTEM_FILESYSTEM_INTERVAL: The interval to use for filesystem data
    LINUX_SYSTEM_INVENTORY_INTERVAL: The interval to use for inventory data
    LINUX_SYSTEM_METRICSETS: The metricsets to collect on every tick
    LINUX_SYSTEM_MOUNTPOINTS_IGNORE_REGEX: A regular expression of mountpoints for which to drop events
  # The 'vars.global' key holds an array/slice of names of global Morio vars
  # that are used by the module.
  global:
    - MORIO_TICK
  # The 'vars.defaults' key holds on object/map with the name/values that should
  # be set as default values for the module. This should only include local vars
  # as the defaults of the global vars cannot be changed by a module.
  # Note: This is not mere documentation. This will be used by the Morio client
  # to set the defaults for these vars
  defaults:
    LINUX_SYSTEM_FILESYSTEM_INTERVAL: 10m
    LINUX_SYSTEM_INVENTORY_INTERVAL: 8h
    # Vars will typically hold a string, but it can also be an array:
    LINUX_SYSTEM_METRICSETS_ALWAYS:
      - cpu
      - diskio
      - load
      - memory
      - network
      - service
    # Make sure to quote your strings if there's a risk they will cause issues when parsing YAML
    LINUX_SYSTEM_MOUNTPOINTS_IGNORE_REGEX: '^/(snap|sys|cgroup|proc|dev|etc|host|lib)($|/)'
{{/MORIO_DOCS}}
#
# Just like the # prefix indicates that this block will only be rendered
# when the variable following it is set, the ^ prefix does the opposite:
# Only render this block when the variable following it is NOT set.
# so this: {{^MORIO_DOCS}}
# Means that whatever follows will only be rendered when MORIO_DOCS is NOT set.
# When the Morio client templates out the configuration, MORIO_DOCS will not
# be set, and thus the entire block above will be ignored, and this is where the
# actual configuration starts.
{{^MORIO_DOCS}}
- module: system
  # This is a global variable that controls the minimal time interval
  # between subsequent collections of data.
  period: {{ MORIO_TICK }}
  # And this is a local variable that controls what metricsets should
  # be collects.
  # Using module variables like this allows people to use this template as-is
  # without losing the ability to fine-tune the configuration.
  metricsets: {{ LINUX_SYSTEM_METRICSETS_ALWAYS }}
  service.state_filter: [ failed ]
  processors:
    - add_fields:
        target: morio
        fields:
          # This variable will be set by the Morio client when templating out
          # the configuration. It is a best practice to use this and avoid
          # hard-coding the module name.
          module: {{ MORIO_MODULE_NAME }}
#
# Here, this entire block is made conditional.
# It will only be included when MORIO_TRACK_INVENTORY is set.
{{#MORIO_TRACK_INVENTORY}}
- module: system
  # This uses a non-standard (slower) interval for which the module created a local var.
  period: {{ LINUX_SYSTEM_INVENTORY_INTERVAL }}
  metricsets:
    - load
    - memory
  processors:
    - add_host_metadata:
        netinfo.enabled: true
        cache.ttl: 60m
    - add_fields:
        target: morio
        fields:
          module: {{ MORIO_MODULE_NAME }}
          inventory_update: true
{{/MORIO_TRACK_INVENTORY}}
{{/MORIO_DOCS}}
# Do not forget to close your conditional blocks.

overlays

The overlays are stored in the overlays folder.

These overlays extend the settings of a Morio collector. Refer to the documentation on overlays for all details.

processors

These are stram processors. They are stored in the processors folder.

watchers

The watchers are stored in hte watchers folder.

FIXME: Write documentation on this.