Small improvements to onboarding workflow (#11)

Author: jonathanalgar

.github/workflows/ci.yml

@@ -1,10 +1,13 @@
 name: CI
 on:
   push:
-    paths-ignore:
-      - '**.md'
+    branches:
+      - main
     tags-ignore:
       - 'v*'
+  pull_request:
+    branches:
+      - '**'
 
 concurrency:
   group: ${{ github.ref }}

README.md

@@ -9,11 +9,11 @@
 
 ![](./README_resources/header.png)
 
-⚡ **`CustomCode-Analyzer-Generator` is a command line tool to generate ODC External Library solutions complete with unit tests**. See examples of generated solutions [here](./README_resources/example_generations/RedisConnector/) (_**>** Connect to a redis database set up on my secure gateway (port 6379). password and key will be input params. return the key value_) and [here](./README_resources/example_generations/PythonRunner/) (_**>** Take a python program in a string as input and return the stdout in the output_). ⚡
+⚡ **`CustomCode-Analyzer-Generator` is a command-line tool to generate validated OutSytems Developer Cloud (ODC) External Library projects complete with unit tests.**. See examples of generated solutions [here](./README_resources/example_generations/RedisConnector/) (_**>** Connect to a redis database set up on my secure gateway (port 6379). password and key will be input params. return the key value_) and [here](./README_resources/example_generations/PythonRunner/) (_**>** Take a python program in a string as input and return the stdout in the output_). ⚡
 
 ## Overview
 
-The BSD-3 licensed [`CustomCode-Analyzer`](https://github.com/jonathanalgar/CustomCode-Analyzer) component was released to give you real-time feedback your C# code as you build an external library in your IDE of choice. But more generally `CustomCode-Analyzer` can be thought of as a way of locally validating an external library at the point of build. This makes it a powerful component in an end-to-end Large Language Model (LLM) pipeline for generating an ODC external library:
+The BSD-3 licensed [`CustomCode-Analyzer`](https://github.com/jonathanalgar/CustomCode-Analyzer) component was released to give you real-time feedback your C# code as you build an external library in your IDE of choice. But more generally `CustomCode-Analyzer` can be thought of as a way of locally validating an external library project at the point of build. This makes it a powerful component in an end-to-end Large Language Model (LLM) pipeline for generating an ODC external library:
 
 ![](./README_resources/diagram.png)
 
@@ -65,7 +65,7 @@ This means a range of models and prompts can be programatically benchmarked. The
 
 Test cases covered include [binary data](./agents/evaluation/ground_truth/pdf.yml) and [network access through the secure gateway](./agents/evaluation/ground_truth/redis.yml). The test set is currently limited to single action (but with support for multi-parameter using an LLM to do name mapping) and single type output.
 
-Consolidated no-repetition results for 2x OpenAI models with one-shot prompt (current default prompt):
+Summary of consolidated no-repetition results for 2x OpenAI models with one-shot prompt (current default prompt):
 
 ![](README_resources/runs.png)
 
@@ -89,4 +89,4 @@ See [here](https://github.com/jonathanalgar/CustomCode-Analyzer/issues?q=is%3Aop
 
 Please report bugs and feature requests [here](https://github.com/jonathanalgar/CustomCode-Analyzer/issues/new/choose).
 
-PRs are welcome. In particular, code quality improvements, new unit/integration tests, new ground truths, and documentation improvements are all welcome 🤗 All changes to code should pass all existing tests (which are lacking and need to be expanded!). Please format any new code with [Mypy and Flake8 for Python](./Makefile) and [Csharpier for C#](./Makefile).
+PRs are welcome. In particular, code quality improvements, new unit/integration tests, new ground truths, and documentation improvements are all welcome 🤗 All changes to code should pass all existing tests (which are lacking and need to be expanded!). Please format any new code with [Mypy and Flake8 for Python](https://github.com/jonathanalgar/CustomCode-Analyzer-Generator/blob/d3cd06bd8f3fb962a1bb5459a15f098f9fc4b0c1/Makefile#L10-L12) and [Csharpier for C#](https://github.com/jonathanalgar/CustomCode-Analyzer-Generator/blob/d3cd06bd8f3fb962a1bb5459a15f098f9fc4b0c1/Makefile#L14-L15).

agents/generate_and_validate.py

@@ -34,13 +34,13 @@ def generate_and_validate(
     logger.info(f"Generating solution for: {use_case}")
 
     env_retain = os.getenv("RETAIN_ON_FAILURE")
-    if env_retain is not None:
+    if env_retain and env_retain.strip():
         retain_on_failure = env_retain.lower() in ["true", "1", "yes", "y"]
         logger.info(f"Using RETAIN_ON_FAILURE={retain_on_failure} from .env")
     else:
         retain_input = input("Retain solution even if build fails? (y/n) [default: y]: ")
         retain_on_failure = retain_input.lower() in ["y", "yes"] or retain_input == ""
-        print("(Tip: add RETAIN_ON_FAILURE=true or RETAIN_ON_FAILURE=true to .env to skip this input prompt)")
+        print("(Tip: add RETAIN_ON_FAILURE=true or RETAIN_ON_FAILURE=true to .env to skip this input prompt in future)")
 
     llm_inputs = LLMInputs(
         use_case=use_case,

agents/generate_library.py

@@ -30,7 +30,7 @@ def get_user_input() -> str:
 `---._.--- CustomCode-Analyzer-Generator, an experimental project -ja---._.---'"""
 
     print(banner)
-    print("\n\nPlease describe what functionality you want the ODC external library to provide.")
+    print("\n\nPlease describe what functionality you want the ODC External Library to provide.")
     print("Example: 'take a string and return the sha1 hash'")
     print("\nEnter the functionality:")
 
@@ -56,9 +56,7 @@ def _get_model_from_env_or_prompt(available_models: dict, model_type: str, env_v
                 print(f"Using {env_model_name} for {model_type} (from environment variable)")
                 return models[env_model_name]
         print(f"Warning: Model '{env_model_name}' specified in {env_var_name} not found. Prompting for selection.")
-
-    print(f"(Tip: add {env_var_name} to .env to skip this input prompt)")
-    return select_model(available_models, model_type)
+    return select_model(available_models, model_type, env_var_name)
 
 
 def main() -> None:
@@ -68,9 +66,10 @@ def main() -> None:
         raise EnvironmentError("No OpenAI API key found in .env file. Please add OPENAI_API_KEY")
 
     available_models = get_available_models(api_keys)
+
     use_case = get_user_input()
 
-    print("\nSelect models for library generation:")
+    print("\nNow let's select which LLMs to use to generate the library..")
     search_term_llm = _get_model_from_env_or_prompt(available_models, "NuGet package search", "SEARCH_TERM_LLM")
 
     code_generation_llm = _get_model_from_env_or_prompt(available_models, "code generation", "CODE_GENERATION_LLM")

agents/utils/model_definitions.py

@@ -45,17 +45,19 @@ def get_model(model_name: str) -> BaseChatModel:
         raise KeyError(f"Model {model_name} not found in any provider.")
 
 
-def select_model(available_models: Dict[str, Dict[str, Any]], purpose: str) -> Any:
+def select_model(available_models: Dict[str, Dict[str, Any]], purpose: str, env_var_name: str) -> Any:
     """Prompts the user to select a model from the available models based on the given purpose."""
-    print(f"\nSelect model for {purpose}:")
+    print(f"\nSelect model for {purpose} (Tip: add {env_var_name} to .env to skip this input prompt in future):")
     options = _get_model_options(available_models, purpose)
     default_option = options[0]
-
     for idx, (name, _) in enumerate(options, start=1):
         if idx == 1:
             print(f"{idx}. {name} (default - press Enter)")
         else:
             print(f"{idx}. {name}")
+    print(
+        "! If you are using the free tier of the OpenAI API, only gpt4o-mini model will work (see https://platform.openai.com/docs/guides/rate-limits)"
+    )
 
     while True:
         choice = input("Enter your choice (number or press Enter for default): ").strip()

agents/validation/utils.py

@@ -54,5 +54,3 @@ def extract_class_name(code: str) -> str:
             return class_name
         finally:
             tmp_path.unlink(missing_ok=True)
-
-

windows_setup.ps1

@@ -264,9 +264,41 @@ $envContent = $envLines -join "`n"
 $utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $false
 [System.IO.File]::WriteAllText($envPath, $envContent, $utf8NoBomEncoding)
 
-# Write file with UTF8 encoding without BOM
-$utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $false
-[System.IO.File]::WriteAllText($envPath, $envContent, $utf8NoBomEncoding)
+# Function to create Desktop shortcut
+function Create-DesktopShortcut {
+    param(
+        [string]$TargetPath,
+        [string]$ShortcutName
+    )
+    
+    $desktopPath = [Environment]::GetFolderPath("Desktop")
+    $shortcutPath = Join-Path $desktopPath "$ShortcutName.lnk"
+    
+    $WshShell = New-Object -ComObject WScript.Shell
+    $Shortcut = $WshShell.CreateShortcut($shortcutPath)
+    $Shortcut.TargetPath = $TargetPath
+    $Shortcut.WorkingDirectory = Split-Path -Parent $TargetPath
+    $Shortcut.IconLocation = "powershell.exe,0"
+    $Shortcut.Description = "Run CustomCode-Analyzer-Generator"
+    $Shortcut.Save()
+    
+    return $shortcutPath
+}
+
+# Ask user if they want to create a desktop shortcut
+$createShortcut = Read-Host "`nDo you want to create a desktop shortcut? (y/n)"
+if ($createShortcut -eq 'y' -or $createShortcut -eq 'Y') {
+    $shortcutPath = Create-DesktopShortcut -TargetPath "powershell.exe" -ShortcutName "CustomCode-Analyzer-Generator"
+    
+    # Update shortcut properties to run the script
+    $WshShell = New-Object -ComObject WScript.Shell
+    $Shortcut = $WshShell.CreateShortcut($shortcutPath)
+    $Shortcut.Arguments = "-ExecutionPolicy Bypass -File `"$mainScriptPath`""
+    $Shortcut.WorkingDirectory = $installPath
+    $Shortcut.Save()
+    
+    Write-Host "Desktop shortcut created at: $shortcutPath" -ForegroundColor Green
+}
 
 Write-Host "`nInstallation Complete!"
 Write-Host "-------------------------"
@@ -284,19 +316,28 @@ Write-Host "2. Navigate to the installation directory:"
 Write-Host "   cd `"$installPath`""
 Write-Host "3. Run the script:"
 Write-Host "   .\customcode-analyzer-generator.ps1"
-
-$effectivePolicy = Get-ExecutionPolicy -Scope Process
-if ($effectivePolicy -eq "Undefined") {
-    $effectivePolicy = Get-ExecutionPolicy -Scope CurrentUser
+if ($createShortcut -eq 'y' -or $createShortcut -eq 'Y') {
+    Write-Host "`nOr simply double-click the desktop shortcut that was created."
 }
-if ($effectivePolicy -eq "Restricted" -or $effectivePolicy -eq "AllSigned") {
-    Write-Host "`nPowerShell is blocking script execution!" -ForegroundColor Yellow
-    Write-Host "Detected Execution Policy: $effectivePolicy"
-    Write-Host "`nTo allow the script to run, choose one of these two options:"
-    Write-Host "   - (Recommended) Change policy to allow local scripts:"
-    Write-Host "     Set-ExecutionPolicy RemoteSigned -Scope CurrentUser"
-    Write-Host "   - (One-time use) Run the script with temporary permission:"
-    Write-Host "     powershell -ExecutionPolicy Bypass -File .\customcode-analyzer-generator.ps1"
+
+# Check execution policy for all scopes
+$processPolicy = Get-ExecutionPolicy -Scope Process
+$userPolicy = Get-ExecutionPolicy -Scope CurrentUser
+$machinePolicy = Get-ExecutionPolicy -Scope LocalMachine
+
+# If any relevant scope is Restricted or AllSigned, warn the user
+if ($processPolicy -eq "Restricted" -or $processPolicy -eq "AllSigned" -or
+    ($processPolicy -eq "Undefined" -and ($userPolicy -eq "Restricted" -or $userPolicy -eq "AllSigned" -or
+    ($userPolicy -eq "Undefined" -and ($machinePolicy -eq "Restricted" -or $machinePolicy -eq "AllSigned"))))) {
+    
+    Write-Host "`nPowerShell is blocking script execution!" -ForegroundColor Red
+    Write-Host "Current effective policy prevents running scripts." -ForegroundColor Yellow
+    
+    Write-Host "`nTo allow the script to run, choose one of these options:"
+    Write-Host "   - (Recommended) Change policy to allow local scripts:" -ForegroundColor Cyan
+    Write-Host "      Set-ExecutionPolicy RemoteSigned -Scope CurrentUser"
+    Write-Host "   - (One-time use) Run the script with this command:" -ForegroundColor Cyan
+    Write-Host "      powershell -ExecutionPolicy Bypass -File .\customcode-analyzer-generator.ps1"
     Write-Host "`nAfter fixing the execution policy, return to Step 3 and run the script."
 }