feat: add custom search command support

docs/custom_search_commands.md

@@ -0,0 +1,224 @@
+# Custom Search Command
+
+Custom search commands are user-defined [SPL](https://docs.splunk.com/Splexicon:SPL) (Splunk Search Processing Language) commands that enable users to add custom functionality to their Splunk searches.
+There are 4 types of Custom search commands that are:
+
+- Generating
+- Streaming
+- Transforming
+- Dataset processing
+
+> Note: Reporting commands are being referred as Transforming commands [reference](https://docs.splunk.com/Splexicon:Transformingcommand).
+
+> Note: Eventing commands are being referred as Dataset processing commands [reference](https://dev.splunk.com/enterprise/docs/devtools/customsearchcommands/).
+
+## Generation of custom search command
+
+A new tag has been introduced in globalConfig (same indent level as of `meta` tag) named `customSearchCommand` where you need to define the configuration for the custom search command.
+
+### Minimal definition
+
+```json
+"customSearchCommand": [
+    {
+        "commandName": "mycommandname",
+        "fileName": "mycommandlogic.py",
+        "commandType": "generating",
+        "arguments": [
+            {
+                "name": "argument_name",
+                "validate": {
+                    "type": "Fieldname"
+                },
+                "required": true
+            },
+            {
+                "name": "argument_two"
+            }
+        ]
+    }
+]
+```
+
+This configuration will generate a template Python file named `mycommandname.py`, which imports logic from the `mycommandlogic.py` file and automatically updates the `commands.conf` file as shown below:
+
+```
+[mycommandname]
+filename = mycommandname.py
+chunked = true
+python.version = python3
+```
+
+ **NOTE:**
+   If the file specified in the `fileName` field does not exist in the `<YOUR_ADDON/bin>` directory, the build will fail.
+
+### Attributes for `customSearchCommand` tag
+
+| Property                 | Type   | Description |
+| ------------------------ | ------ | ------------------------------------ |
+| commandName<span class="required-asterisk">\*</span>  | string  | Name of the custom search command |
+| fileName<span class="required-asterisk">\*</span>     | string  | Name of the Python file which contains logic of custom search command  |
+| commandType<span class="required-asterisk">\*</span>  | string  | Specify type of custom search command. Four types of commands are allowed, `streaming`,`generating`,`transforming` and `dataset processing`. |
+| arguments<span class="required-asterisk">\*</span>    | object  | Arguments which can be passed to custom search command. |
+| requiredSearchAssistant                               | boolean | Specifies whether search assistance is required for the custom search command. Default: false. |
+| usage                                                 | string  | Defines the usage of custom search command. It can be one of `public`, `private` and `deprecated`.  |
+| description                                           | string  | Provide description of the custom search command.   |
+| syntax                                                | string  | Provide syntax for custom search command   |
+
+To generate a custom search command, the following attributes must be defined in globalConfig: `commandName`, `commandType`, `fileName`, and `arguments`. Based on the provided commandType, UCC will generate a template Python file and integrate the user-defined logic into it.
+
+If `requiredSearchAssistant` is set to True, the `syntax`, `description`, and `usage` attributes are mandatory, as they are essential for generating `searchbnf.conf` file.
+
+**NOTE:**
+    The user-defined Python file must include specific functions based on the command type:
+
+- For `Generating` command, the Python file must include a `generate` function.
+- For `Streaming` command, the Python file must include a `stream` function.
+- For `Dataset processing` command, the Python file must include a `transform` function.
+- For `Transforming` command, the Python file must include a `reduce` function, and optionally a `map` function if a streaming pre-operation is required.
+
+## Arguments
+
+| Property                                                              | Type   | Description   |
+| --------------------------------------------------------------------- | ------ | ------------------------------------------------------- |
+| name<span class="required-asterisk">\*</span>                         | string | Name of the argument  |
+| defaultValue                                                          | string/number | Default value of the argument.  |
+| required                                                              | string |  Specify if the argument is required or not. |
+| validate                                                              | object | Specify validation for the argument. It can be any of `Integer`, `Float`, `Boolean`, `RegularExpression` or `FieldName`. |
+
+UCC currently supports five types of validations provided by `splunklib` library:
+
+- IntegerValidator
+    + you can optionally define `minimum` and `maximum` properties.
+- FloatValidator
+    + you can optionally define `minimum` and `maximum` properties.
+- BooleanValidator
+    + no additional properties required.
+- RegularExpressionValidator
+    + no additional properties required.
+- FieldnameValidator
+    + no additional properties required.
+
+For more information, refer [splunklib API docs](https://splunk-python-sdk.readthedocs.io/en/latest/searchcommands.html)
+
+For example:
+
+```json
+"arguments": [
+    {
+        "name": "count",
+        "required": true,
+        "validate": {
+            "type": "Integer",
+            "minimum": 1,
+            "maximum": 10
+        },
+        "default": 5
+    },
+    {
+        "name": "test",
+        "required": true,
+        "validate": {
+            "type": "Fieldname"
+        }
+    },
+    {
+        "name": "percent",
+        "validate": {
+            "type": "Float",
+            "minimum": "85.5"
+        }
+
+    }
+]
+
+```
+
+## Example
+
+``` json
+{
+    "meta": {...}
+    "customSearchCommand": [
+        {
+            "commandName": "generatetextcommand",
+            "fileName": "generatetext.py",
+            "commandType": "generating",
+            "requiredSearchAssistant": true,
+            "description": " This command generates COUNT occurrences of a TEXT string.",
+            "syntax": "generatetextcommand count=<event_count> text=<string>",
+            "usage": "public",
+            "arguments": [
+                {
+                    "name": "count",
+                    "required": true,
+                    "validate": {
+                        "type": "Integer",
+                        "minimum": 5,
+                        "maximum": 10
+                    }
+                },
+                {
+                    "name": "text",
+                    "required": true
+                }
+            ]
+        },
+    ],
+    "pages": {...}
+}
+```
+
+Generated python file named `generatetextcommand.py`:
+
+``` python
+mport sys
+import import_declare_test
+
+from splunklib.searchcommands import \
+    dispatch, GeneratingCommand, Configuration, Option, validators
+from generatetext import generate
+
+@Configuration()
+class GeneratetextcommandCommand(GeneratingCommand):
+    """
+
+    ##Syntax
+    generatetextcommand count=<event_count> text=<string>
+
+    ##Description
+     This command generates COUNT occurrences of a TEXT string.
+
+    """
+    count = Option(name='count', require=True, validate=validators.Integer(minimum=5, maximum=10))
+    text = Option(name='text', require=True)
+
+    def generate(self):
+       return generate(self)
+
+dispatch(GeneratetextcommandCommand, sys.argv, sys.stdin, sys.stdout, __name__)
+```
+
+Generated stanza in `commands.conf` file
+
+```
+[generatetextcommand]
+filename = generatetextcommand.py
+chunked = true
+python.version = python3
+```
+
+Generated stanza in `searchbnf.conf` file
+
+```
+[generatetextcommand]
+syntax = generatetextcommand count=<event_count> text=<string>
+description = This command generates COUNT occurrences of a TEXT string.
+usage = public
+```
+
+### Output
+
+This is how generatetextcommand search result looks like:
+
+![image](./images/custom_search_command_output.png)

docs/generated_files.md

@@ -7,6 +7,8 @@ The following table describes the files generated by UCC framework.
 | File Name  | File Location | File Description |
 | ------------ | ------------ | ----------------- |
 | app.conf | output/<YOUR_ADDON_NAME>/default | Generates `app.conf` with the details mentioned in globalConfig[meta] |
+| commands.conf | output/<YOUR_ADDON_NAME>/default | Generates `commands.conf` for custom commands provided in the globalConfig. |
+| searchbnf.conf | output/<YOUR_ADDON_NAME>/default | Generates `searchbnf.conf` for custom search commands provided in the globalConfig. |
 | inputs.conf | output/<YOUR_ADDON_NAME>/default | Generates `inputs.conf` and `inputs.conf.spec` file for the services mentioned in globalConfig |
 | server.conf | output/<YOUR_ADDON_NAME>/default | Generates `server.conf` for the custom conf files created as per configurations in globalConfig |
 | restmap.conf | output/<YOUR_ADDON_NAME>/default | Generates `restmap.conf` for the custom REST handlers that are generated based on configs from globalConfig |
@@ -22,5 +24,6 @@ The following table describes the files generated by UCC framework.
 | inputs.xml | output/<YOUR_ADDON_NAME>/default/data/ui/views | Generates inputs.xml based on inputs configuration present in globalConfig, in `default/data/ui/views/inputs.xml` folder |
 | _redirect.xml | output/<YOUR_ADDON_NAME>/default/data/ui/views | Generates ta_name_redirect.xml file, if oauth is mentioned in globalConfig, in `default/data/ui/views/` folder. |
 | _.html | output/<YOUR_ADDON_NAME>/default/data/ui/alerts | Generates `alert_name.html` file based on alerts configuration present in globalConfig, in `default/data/ui/alerts` folder. |
+| _.py | output/<YOUR_ADDON_NAME>/bin | Generates Python files for custom commands provided in the globalConfig. |
 | globalConfig.json | <source_dir> | Generates globalConfig.json file in the source code if globalConfig is not present in source directory at build time. |
 

mkdocs.yml

@@ -89,6 +89,7 @@ nav:
       - Modify fields On change: "entity/modifyFieldsOnValue.md"
       - Help property: "entity/help_message.md"
 
+  - Custom search commands: "custom_search_commands.md"
   - Table: "table.md"
   - Additional packaging: "additional_packaging.md"
   - UCC ignore: "uccignore.md"

splunk_add_on_ucc_framework/commands/build.py

@@ -24,7 +24,6 @@
 import colorama as c
 import fnmatch
 import filecmp
-
 from splunk_add_on_ucc_framework import (
     __version__,
     exceptions,
@@ -457,7 +456,7 @@ def generate(
     global_config_update.handle_global_config_update(global_config, gc_path)
     try:
         validator = global_config_validator.GlobalConfigValidator(
-            internal_root_dir, global_config
+            internal_root_dir, global_config, source=source
         )
         validator.validate()
         logger.info("globalConfig file is valid")
@@ -513,6 +512,7 @@ def generate(
         logger.error(str(e))
         sys.exit(1)
     logger.info(f"Installed add-on requirements into {ucc_lib_target} from {source}")
+
     generated_files.extend(
         begin(
             global_config=global_config,