Improve CLI subcommand organisation and convert docs to Material for Mkdocs

Contents (#738)

IMPORTANT: There is 1 breaking change.

Enhancements

• Update version compatibility table script for markdown format

Dependencies

• Add mkdocs-material
• Add mkdocs-glightbox to provide image zooming to docs
• Remove dependencies and config from old documentation
• Add tabulate for version table script

Refactoring

• 💥 BREAKING CHANGE: Replace octue twined start CLI subcommand with octue twined service start subcommand

Reversions

• Revert "REF: Remove google_crc32c warning suppression"

Other

• Convert documentation from restructured text (RST) to markdown (MD)
• Use mkdocs and Material for Mkdocs to build and theme documentation
• Document using TWINED_SERVICES_TOPIC_NAME envvar
• Disable sphinx pre-commit step
• Update CLI printout in readme
• Replace "Octue SDK" with "Twined" in docs
• Move CONTRIBUTING.md to repo root and update it

---

Upgrade instructions

💥 Replace `octue twined start` CLI subcommand with `octue twined service start` subcommand

Use octue twined service start instead of octue twined start

Move current Octue SDK code into `twined` subpackage

Contents (#737)

IMPORTANT: There are 4 breaking changes.

Fixes

• Update command in dockerfiles to use new octue twined CLI

Refactoring

• 💥 BREAKING CHANGE: Move Twined-related octue code into twined subpackage
• 💥 BREAKING CHANGE: Rename OCTUE_SERVICES_TOPIC_NAME to TWINED_SERVICES_TOPIC_NAME
• 💥 BREAKING CHANGE: Move octue.runner to octue.twined.runner
• 💥 BREAKING CHANGE: Remove unused folders, isfolder, and isfile utils
• Merge TwinedEncoder with OctueJSONEncoder
• Rename scripts

Testing

• Restructure test directories
• Remove unused base test method

---

Upgrade instructions

💥 Move Twined-related `octue` code into `twined` subpackage

Import Twined-related code that was formerly in octue from octue.twined

💥 Rename `OCTUE_SERVICES_TOPIC_NAME` to `TWINED_SERVICES_TOPIC_NAME`

Use the TWINED_SERVICES_TOPIC_NAME envvar instead of OCTUE_SERVICES_TOPIC_NAME and ensure its value ends in octue.twined.services

💥 Move `octue.runner` to `octue.twined.runner`

Use the new import path: from octue.twined.runner import Runner (note that from octue.twined import Runner will not work)

💥 Remove unused `folders`, `isfolder`, and `isfile` utils

Stop using the get_file_name_from_strand, isfolder, and isfile functions

Move twined package into octue

Summary

This release moves the twined package into octue in preparation for the rebranding of Octue SDK into Twined.

Contents (#735)

Dependencies

• Remove twined library dependency

Refactoring

• Move twined package inside octue

Switch to octue twined CLI

Summary

This release moves the octue CLI commands down into the octue twined subcommand in preparation for the rebranding of Octue SDK into Twined.

Contents (#734)

IMPORTANT: There are 3 breaking changes.

New features

• 💥 BREAKING CHANGE: Move CLI commands into twined command

Operations

• Set pre-commit checks to not fail fast (this results in fewer runs for developers)

Refactoring

• 💥 BREAKING CHANGE: Remove deprecated octue get-diagnostics CLI command
• 💥 BREAKING CHANGE: Remove no-op CLI options

---

Upgrade instructions

💥 Move CLI commands into `twined` command

Type octue twined for the CLI instead of just octue

💥 Remove deprecated `octue get-diagnostics` CLI command

Use octue twined question diagnostics instead

💥 Remove no-op CLI options

Stop providing the --id and --force-reset/--no-force-reset options to the CLI

Use new docker base images

Contents (#732)

New features

• Add new python3.12 and python3.13 dockerfiles

Enhancements

• Use new rational-swordtail base images in dockerfiles

Operations

• Update devcontainer base image
• Use python3.12 for CI tests

Refactoring

• Remove google_crc32c warning suppression

Style

• Run isort
• Ignore D401 (first line of docstring not in imperative)

Other

• Update contribution docs python version

Rename `project_name` to `project_id`

Contents (#731)

IMPORTANT: There is 1 breaking change.

Refactoring

• 💥 BREAKING CHANGE: Rename project_name to project_id
• Remove unused test environment variable TEST_PROJECT_ID

Operations

• Stop using tox for CI tests

Dependencies

• Use version 0.16.0 of service communication schema
• Use twined=^0.7.0
• Remove tox

---

Upgrade instructions

💥 Rename `project_name` to `project_id`

Rename the project_name argument to project_id

Move app configuration into service configuration

Contents (#730)

IMPORTANT: There is 1 breaking change.

New features

• 💥 BREAKING CHANGE: Move app configuration into service configuration

---

Upgrade instructions

💥 Move app configuration into service configuration

• Move the contents of the app_configuration.json file into the octue.yaml file under the first item in the services list (make sure to convert from JSON to YAML)
• Remove the app_configuration_path argument from the octue.yaml file
• Use ServiceConfiguration.from_file instead of load_service_and_app_configuration
• Replace any uses of the app_configuration argument/variable with service_configuration

Fix google auth library imports and `grpcio` version

Contents (#729)

Fixes

• Fix google auth library imports

Dependencies

• Use new version of grpcio to avoid spurious warning

Switch to Kueue service backend

Summary

This release switches to running Twined services on Kubernetes + Kueue. This brings the following features:

• Queue questions so they're not just dropped if the service backend is overwhelmed
• Run questions that take any amount of time (specifically opening us up to runs > 1 hour)
• Request arbitrary compute resources per question (CPU, memory, storage etc.)
• Stop extraneous question reruns by allowing us to control when we acknowledge question events
• Monitor running questions individually
• Make it easier to run questions on providers other than Google in the future (i.e. on any Kubernetes cluster)

Contents (#723)

IMPORTANT: There are breaking changes.

New features

• #709 (see PR for list of breaking changes)
• Authenticate requests to service registries

Enhancements

• Add allow_not_found option to ServiceConfiguration.from_file
• Add default event store ID to get_events
• Increase default maximum heartbeat interval to 360s

Refactoring

• #721

Dependencies

• Remove gunicorn and Flask dependencies

Operations

• Replace Terraform configuration with new terraform-octue-twined-core module

Testing

• Move deployment test to octue/example-service-kueue repository

Improve output validation/upload logging

Contents (#691)

Enhancements

• Improve output validation/upload logging

Fixes

• Avoid attempting to upload output manifest if it's None