update docs

Author: jonathanalgar

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,38 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Desktop (please complete the following information):**
+ - OS: [e.g. iOS]
+ - Browser [e.g. chrome, safari]
+ - Version [e.g. 22]
+
+**Smartphone (please complete the following information):**
+ - Device: [e.g. iPhone6]
+ - OS: [e.g. iOS8.1]
+ - Browser [e.g. stock browser, safari]
+ - Version [e.g. 22]
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.gitignore

@@ -404,5 +404,4 @@ FodyWeavers.xsd
 .env
 
 __WIP
-benchmark_results
 generated_solutions

README.md

@@ -1,9 +1,9 @@
 # OutSystems Developer Cloud (ODC) Custom Code Analyzer: Generator
 
-[![Lines of Code](https://sonarcloud.io/api/project_badges/measure?project=jonathanalgar_CustomCode-Analyzer&metric=ncloc&token=cdc14f785767dbdef568a43b914c07c5be4f2e69)](https://sonarcloud.io/summary/new_code?id=jonathanalgar_CustomCode-Analyzer) [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](https://makeapullrequest.com)
+[![Lines of Code](https://sonarcloud.io/api/project_badges/measure?project=jonathanalgar_CustomCode-Analyzer-Generator&metric=ncloc)](https://sonarcloud.io/summary/new_code?id=jonathanalgar_CustomCode-Analyzer-Generator) [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](https://makeapullrequest.com)
 
-[![Reliability Rating](https://sonarcloud.io/api/project_badges/measure?project=jonathanalgar_CustomCode-Analyzer&metric=reliability_rating&token=cdc14f785767dbdef568a43b914c07c5be4f2e69)](https://sonarcloud.io/summary/new_code?id=jonathanalgar_CustomCode-Analyzer) [![Maintainability Rating](https://sonarcloud.io/api/project_badges/measure?project=jonathanalgar_CustomCode-Analyzer&metric=sqale_rating&token=cdc14f785767dbdef568a43b914c07c5be4f2e69)](https://sonarcloud.io/summary/new_code?id=jonathanalgar_CustomCode-Analyzer)
- [![Vulnerabilities](https://sonarcloud.io/api/project_badges/measure?project=jonathanalgar_CustomCode-Analyzer&metric=vulnerabilities&token=cdc14f785767dbdef568a43b914c07c5be4f2e69)](https://sonarcloud.io/summary/new_code?id=jonathanalgar_CustomCode-Analyzer)
+[![Reliability Rating](https://sonarcloud.io/api/project_badges/measure?project=jonathanalgar_CustomCode-Analyzer-Generator&metric=reliability_rating)](https://sonarcloud.io/summary/new_code?id=jonathanalgar_CustomCode-Analyzer-Generator) [![Maintainability Rating](https://sonarcloud.io/api/project_badges/measure?project=jonathanalgar_CustomCode-Analyzer-Generator&metric=sqale_rating)](https://sonarcloud.io/summary/new_code?id=jonathanalgar_CustomCode-Analyzer-Generator)
+ [![Vulnerabilities](https://sonarcloud.io/api/project_badges/measure?project=jonathanalgar_CustomCode-Analyzer-Generator&metric=vulnerabilities)](https://sonarcloud.io/summary/new_code?id=jonathanalgar_CustomCode-Analyzer-Generator)
 
 > :information_source: This component is unofficial and unsupported by OutSystems.
 
@@ -48,7 +48,7 @@ _(Quick-start script for Mac coming soon)._
 1. Open the [`Terminal` app](https://support.apple.com/en-sg/guide/terminal/apd5265185d-f365-44cb-8b09-71a064a42125/mac).
 1. Create the required directories in the home directory or elsewhere:
 
-        mkdir ~/CustomCode-Analyzer-Generator && cd ~/CustomCode-Analyzer-Generator && mkdir generated-solutions
+        mkdir -p ~/CustomCode-Analyzer-Generator/generated-solutions && chmod 700 ~/CustomCode-Analyzer-Generator/generated-solutions && cd ~/CustomCode-Analyzer-Generator
 
 1. Create a `.env` file in `~/CustomCode-Analyzer-Generator` and edit with your text editor of choice. Use [`.env_example`](./.env_example) as template.
 1. Run the command:
@@ -65,6 +65,7 @@ This means a range of models and prompts can be programatically benchmarked. The
 
 Test cases covered include [binary data](./agents/evaluation/ground_truth/pdf.yml) and [network access through the secure gateway](./agents/evaluation/ground_truth/redis.yml). The test set is currently limited to single action (but with support for multi-parameter using an LLM to do name mapping) and single type output.
 
+
 _(More documentation coming soon.)_
 
 ## TODO
@@ -75,4 +76,4 @@ See [here](https://github.com/jonathanalgar/CustomCode-Analyzer/issues?q=is%3Aop
 
 Please report bugs and feature requests [here](https://github.com/jonathanalgar/CustomCode-Analyzer/issues/new/choose).
 
-PRs are welcome. In particular, code quality improvements, new unit/integration tests, new ground truths and documentation improvements are all welcome 🤗 All changes to code should pass all existing tests (which are lacking and need to be expanded!). Please format any new code with [Flake8 and Black for Python](./Makefile) and [Csharpier for C#](./Makefile).
+PRs are welcome. In particular, code quality improvements, new unit/integration tests, new ground truths and documentation improvements are all welcome 🤗 All changes to code should pass all existing tests (which are lacking and need to be expanded!). Please format any new code with [Mypy and Flake8 for Python](./Makefile) and [Csharpier for C#](./Makefile).

agents/evaluation/benchmark_analysis.ipynb

@@ -12,18 +12,18 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "7dff4b0fb2224390becb40e37a874d2d",
+       "model_id": "e6f3b966b81e44e497111cb30a58fd0f",
        "version_major": 2,
        "version_minor": 0
       },
       "text/plain": [
-       "VBox(children=(VBox(children=(Checkbox(value=True, description='run_06b4f | 2025-02-25 01:29:45 | gpt-4o (temp…"
+       "VBox(children=(VBox(children=(Checkbox(value=True, description='run_8cb9f | 2025-02-25 03:26:01 | o3-mini (rea…"
       ]
      },
      "metadata": {},