Skip to content

Add experimental filtered IoT Agent installer - #448

Draft
thieman wants to merge 12 commits into
DataDog:mainfrom
thieman:thieman/iot-filtered-install-script
Draft

Add experimental filtered IoT Agent installer#448
thieman wants to merge 12 commits into
DataDog:mainfrom
thieman:thieman/iot-filtered-install-script

Conversation

@thieman

@thieman thieman commented Aug 25, 2026

Copy link
Copy Markdown

Summary

Add an experimental install_script_agent7_iot.sh variant that installs the normal Agent 7 DEB with a reduced IoT filesystem profile instead of installing the separately packaged datadog-iot-agent artifact.

The generated script:

  • installs the normal datadog-agent package;
  • forces infrastructure_mode: iot;
  • writes a persistent, ordered dpkg path-filter policy before package unpack;
  • retains the main Agent, Agent Data Plane, rtloader shim, support views, SSL/support assets, and IoT core-check configurations;
  • filters Python, packaged integrations, companion agents, system-probe/eBPF assets, JMX, ODBC, and security/compliance payloads;
  • validates the final retained and excluded filesystem contract;
  • writes a root-owned, versioned /etc/datadog-agent/install_profile marker only after package, configuration, layout, and service handling succeed;
  • rolls back a transient filter if APT or package post-installation fails.

Existing generated installers remain byte-for-byte unchanged. The new script is generated as a CI artifact but is intentionally absent from the public deploy matrix while this remains a prototype.

Disk measurement

Same-version Agent 7.82.2 comparison on Ubuntu 22.04:

Layout Installed logical bytes
Normal Agent 797,357,886
Filtered IoT profile 184,336,138
Saved 613,021,748 (76.88%)

A prior same-version allocated-block comparison measured a 77.81% reduction. These are experimental measurements rather than CI thresholds and vary with package version and filesystem.

Validation

  • 76 main installer shunit tests pass.
  • 19 filtered-IoT orchestration tests pass.
  • Generated-script contract, Bash syntax, ShellCheck, and CI YAML parsing pass.
  • Fresh real installs pass on Ubuntu 22.04 and Debian 12.
  • Pinned Agent 7.82.2 install passes package/version, filter policy, configuration, marker, filesystem layout, checksum, and Agent CLI checks.
  • Ordinary generated Agent 6, Agent 7, legacy, and Docker-injection scripts remain byte-identical.
  • Controlled APT failures restore or remove the transient dpkg policy and do not publish a completed marker.

Prototype limitations

  • Debian and Ubuntu only.
  • Fresh installs only; existing Agent packages/configuration and ordinary APT upgrades are rejected/unsupported.
  • RPM support is not included. A real Rocky 9 attempt showed that the Agent RPM %post requires the embedded installer before the final filtered state; reconciling that with RPM's recorded file state needs a separate design.
  • The current Agent does not yet consume install_profile. A companion Agent change should require infrastructure_mode: iot and strict capability guards whenever profile: iot-filtered is present.
  • Runtime daemon/payload behavior beyond the Agent CLI is follow-up validation against an Agent build containing the IoT runtime mode.

thieman added 12 commits August 24, 2026 16:26
Generate install_script_agent7_iot.sh from the common Agent template with Agent 7 metadata and a target-specific IoT mode flag. Existing common-template targets explicitly disable the mode so every generated script resolves the placeholder.

Wire the ignored artifact into Make clean/all and tagged manual deployment, document its future filtered normal-Agent purpose, and teach local installation checks its package, major version, and install-info identity. Package filtering remains intentionally deferred to the follow-up tasks.
Run a lightweight contract check immediately after script generation so unresolved placeholders, incorrect IoT modes, metadata, or executable bits fail in CI. Keep the IoT output available as a generated artifact while removing it from the public deploy matrix until package filtering is implemented.
Add extracted helpers that validate the generated IoT mode, render deterministic dpkg filters, and derive bounded RPM exclusion prefixes from the package manifest. The helpers retain only the required Agent, data-plane, rtloader, support-view, and IoT check assets without changing any live package-manager branch.\n\nValidate filtered layouts, query installed package versions by family, and atomically write the durable install profile. Add shunit coverage for compatible and rejected options, filter ordering and retention, RPM quoting and bounds, aggregated layout failures, marker replacement failures, and package queries.
Validate the full retained IoT payload and every representative pruned class while tolerating empty dpkg-preserved directories and dangling links. Use targeted DEB and RPM exclusions so Agent SSL and support assets remain available, with explicit DEB parent re-includes.

Reject RPM exclusion-prefix collisions before producing output, and cover missing retained content, pruned payloads, parent rules, support retention, and collision cases with shunit tests.
Write the dpkg filter and install profile through same-directory temporary files, replace exact destinations atomically, and reject unsafe post-replacement types.

Broaden filtered layout validation across retained file types, disallowed payload classes, and live links while keeping RPM exclusion deduplication compatible with Bash 3. Extend shunit coverage for each failure mode.
Apply the same Python-script, requirements, and shared MS ODBC pruning contract to DEB filters, RPM exclusion fixtures, and post-install validation.

Capture every validator find traversal in a checked NUL-delimited temporary listing so inspection errors are aggregated and validation fails closed without relying on process-substitution status.
Wire the generated Agent 7 IoT variant into fresh-install-only option validation and force the normal datadog-agent package with IoT infrastructure mode.

Install the dpkg path filter atomically before package unpack, retain the package installer only for post-install configuration, then persist the final filter, validate the retained layout, and capture the installed package version.

Add orchestration, generated-script, local container, checksum, and dedicated Ubuntu/Debian CI coverage while keeping every existing generated installer byte-identical and excluding the draft from deploy.
Pass the installer-facing DD_AGENT_MINOR_VERSION variable so the filtered Debian job actually selects and verifies Agent 7.82. Extend the generated-script contract to catch an unconnected pinned-version variable.
Exercise the persistent Debian filter through the localtest decision helper so retained Agent, ADP, rtloader, views, and IoT check paths stay included while pruned payload classes stay excluded. Also reject the transient installer override and any retained installer binary, with a generated-script contract requiring the real-install check.
Force IoT infrastructure mode into the Debian package environment and normalize the fresh Agent configuration after postinst, while leaving generated ordinary installers byte-for-byte unchanged.

Treat every queryable dpkg state as pre-existing and add a rollback transaction around the transient dpkg path filter. The transaction snapshots prior content, ownership, and mode in private storage, restores or removes the filter on APT and EXIT failures, and is disarmed only after the persistent policy and filtered layout validate.

Strengthen filtered checksum evidence with a required dpkg md5 manifest and accurate retained-file reporting. Run orchestration regressions in GitLab, keep the pinned 7.82 job push-only, and document fresh-install and upgrade constraints plus the experimental same-version disk result.

Tests cover unset infrastructure input, all dpkg states, empty checksum evidence, silent debsums success, and APT rollback with and without a prior filter.
Clarify that the experimental filtered installer currently supports Debian and Ubuntu only. Record the RPM post-install installer dependency that prevents retaining a verified RPM path in this draft, and add the feature to the unreleased changelog.
Publish the reviewed filtered IoT profile only after Debian package, layout, configuration, and service handling complete. Stage the profile outside the root context, install it atomically as root:root mode 0644, and verify its exact content against the captured package version and installer identity before success telemetry.\n\nUpdate generated-script, shunit, local container, and documentation coverage for publication ordering, controlled pre-marker failure, prior-marker preservation, and the Agent companion-guard contract while keeping ordinary generated installers byte-identical.
@datadog-official

Copy link
Copy Markdown

Pipelines

Unblock PR with BitsAI

⚠️ Warnings

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 1 Pipeline job failed

Shellcheck | shellcheck — 🔧 Needs a code fix, caused by this PR

View more details · View in GitHub Actions

Multiple commands in test_iot_deb_orchestration.sh are unreachable due to improper function definitions as indicated by SC2317.

📋 Copy fix prompt
CI on my pull request is failing. Help me find and fix the root cause of each failing job below — they were flagged as caused by changes in this PR, so focus on the diff. For each job, explain the failure and propose a fix.

Before you start, set up the Datadog software-delivery tooling so you can
query the CI data yourself:

1. Check whether you already have the Datadog software-delivery MCP tools
   (e.g. a `search_datadog_ci_pipeline_events` tool) and the `unblock-pr` skill.
2. If either is missing, STOP and ask me for permission before installing
   anything. Do not install or run anything until I have said yes.
3. Only with my explicit approval, set up the Datadog software-delivery MCP
   server and skills by following:
     https://docs.datadoghq.com/getting_started/software_delivery_mcp_tools/
   then restart so the skill is picked up.
4. If I decline, skip all of the above and work from the context below alone.

Then run /unblock-pr — it will pull the CI data itself. The job context below is what we already know.

If /unblock-pr is not available — because I declined the setup above, or it did not install — work from the context below instead.

Datadog has already classified this failure as caused by changes in this PR.
Take that as given and work the fix:

1. Locate the change. Diff this branch against its base and find the change
   that produces this error. Explain the mechanism, don't just name a file:
     git fetch origin && git diff $(git merge-base origin/main HEAD)...HEAD
2. Reproduce it locally. Run the failing job's command or test before
   proposing anything.
3. Propose the smallest fix that addresses the root cause — not a workaround,
   not a broadened assertion, not a disabled or skipped test.
4. Re-run the same command to confirm, and say exactly what you ran.
5. If the failure turns out to be intermittent rather than deterministic, say
   so plainly instead of "fixing" it — that is a flaky test, and patching it
   hides the problem.

If the right move is to re-run the job rather than change code, use the job
link in the context below. For GitHub Actions: `gh run rerun <run-id> --failed`,
where the run ID is the number after `/runs/` in that URL (not the trailing
number, which is the job ID).

Branch: thieman/iot-filtered-install-script

Shellcheck | shellcheck
Commit: 80012a26c86ad71634ff457250fb8d3e885424d9
Error (code / build):
Multiple commands in test_iot_deb_orchestration.sh are unreachable due to improper function definitions as indicated by SC2317.
CI job: https://github.com/DataDog/agent-linux-install-script/actions/runs/32873294758/job/97885198089

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 80012a2 | Docs | View more details | Give us feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant