[FR] [DAC] Add Arbitrary File location Support for Local Creation Date

[eric-forte-elastic]

Pull Request

Issue link(s):

Resolves #4891

Summary - What I changed

Added the ability to specify --load-rule-loading / -lr to the kibana export-rules and import-rules-to-repo commands. This enables one to make use of the local folders specified in one's config.yaml when exporting rules from Kibana to the repo.

For instance, if I specify

rule_dirs:
- rules
- another_rules_dir

And have two rules

• dac_test/rules/my_test_rule.toml
• dac_test/another_rules_dir/high_number_of_process_and_or_service_terminations.toml

Rule updates from Kibana (when using the new -lr option) will now use these additional paths instead of exporting directly to the specified directory.

So if I run python -m detection_rules kibana --space test_local export-rules -d dac_test/rules/ -sv -ac -e -lr in a space with those two rules, my_test_rule.toml will be written to dac_test/rules/ as it was already on disk there and high_number_of_process_and_or_service_terminations.toml will be written to dac_test/another_rules_dir/ for the same reason.

Note: This can be very slow with a high number of rules. For instance, it can take 5-10min to load 1-2k rules in this way depending on system resources. The reason for this is the need to load the rule in from disk prior to actually exporting the rules, in order to know the appropriate path for the rule to be written.

How To Test

Run the respective kibana export-rules and import-rules-to-repo commands with some rules already existing in different directories specified in the custom rules config yaml.

e.g.

Checklist

• Added a label for the type of pr: bug, enhancement, schema, maintenance, Rule: New, Rule: Deprecation, Rule: Tuning, Hunt: New, or Hunt: Tuning so guidelines can be generated
• Added the meta:rapid-merge label if planning to merge within 24 hours
• Secret and sensitive material has been managed correctly
• Automated testing was updated or added to match the most common scenarios
• Documentation and comments were added for features that require explanation

Contributor checklist

• Have you signed the contributor license agreement?
• Have you followed the contributor guidelines?

[github-actions]

Enhancement - Guidelines

These guidelines serve as a reminder set of considerations when addressing adding a feature to the code.

Documentation and Context

• Describe the feature enhancement in detail (alternative solutions, description of the solution, etc.) if not already documented in an issue.
• Include additional context or screenshots.
• Ensure the enhancement includes necessary updates to the documentation and versioning.

Code Standards and Practices

• Code follows established design patterns within the repo and avoids duplication.
• Ensure that the code is modular and reusable where applicable.

Testing

• New unit tests have been added to cover the enhancement.
• Existing unit tests have been updated to reflect the changes.
• Provide evidence of testing and validating the enhancement (e.g., test logs, screenshots).
• Validate that any rules affected by the enhancement are correctly updated.
• Ensure that performance is not negatively impacted by the changes.
• Verify that any release artifacts are properly generated and tested.
• Conducted system testing, including fleet, import, and create APIs (e.g., run make test-cli, make test-remote-cli, make test-hunting-cli)

Additional Checks

• Verify that the enhancement works across all relevant environments (e.g., different OS versions).
• Confirm that the proper version label is applied to the PR patch, minor, major.

[traut]

🙇🏻‍♂️