Add ability to get latest revision of a service

Contents (#587)

New features

• Add method for getting latest SRUID for a service
• Use latest service revision in Service.ask when revision tag not provided or latest is used
• Allow specification of service registries in service configuration and via Child and Service

Enhancements

• Add function for splitting SRUIDs

Refactoring

• Rename functions in service_id module to remove redundant "service" from name

Dependencies

• Swap deprecated pkg_resources for importlib.metadata

Testing

• Factor out mock service revision tag and use instead of latest in tests

Style

• Use latest commit message checker

Move crash diagnostics logic into class

Contents (#586)

Enhancements

Improve app crash log handling by:

• When an app crashes, always raise the error
• When crash diagnostics fail to upload, only log the error instead of raising it

Refactoring

• Factor crash diagnostics handling out into a class

Make crash diagnostics compatible with serialised datasets

Contents (#585)

Fixes

• Ensure crash diagnostics can deal with serialised values
• Ensure crash diagnostics can deal with manifests containing serialised datasets (as opposed to containing just paths to datasets)

Dependencies

• Update requests and protobuf to avoid security issues

Allow sets and datetimes as service input/output values

Contents (#581)

Fixes

• Allow sets and datetimes as service input/output values

Dependencies

• Update flask to avoid security issue

Remove Cloud Run deployer and associated CLI command

Contents (#580)

IMPORTANT: There is 1 breaking change.

Operations

• Add infrastructure needed to deploy example Cloud Run service

Testing

• Test latest deployed version of cloud run example service
• Test that app submodule log messages are handled

Reversions

• 💥 BREAKING CHANGE: Remove Cloud Run deployer and associated CLI command

---

Upgrade instructions

💥 Remove Cloud Run deployer and associated CLI command

Use terraform to create the Google Cloud infrastructure needed and the Octue GitHub actions and workflows to set up automatic deployment to Cloud Run. See here for an example workflow and here for an example terraform config.

Improve analysis logging

Summary

This PR refines the approach taken to adding metadata about an analysis to log messages emitted during its processing. It fixes an issue where developers couldn't test their apps based on the log messages emitted during an analysis run.

Contents (#578)

New features

• Enable Octue log handler by default

Enhancements

• Avoid using deprecated datafile property in runner
• Add default Octue service Dockerfile that uses python3.10

Fixes

• Swap out formatters instead of handlers during analyses
• Only use Octue formatter in analysis if envvar present

Refactoring

• Rename AnalysisLogHandlerSwitcher to AnalysisLogFormatterSwitcher
• Move run_logged_subprocess to octue.utils.processes

Testing

• Add tests for AnalysisLogFormatterSwitcher

Remove delivery acknowledgement timeout

Summary

This release removes the delivery acknowledgement timeout used when waiting for an answer from a child. The timeout had become mostly redundant since heartbeats were introduced and was causing difficult-to-debug errors. Heartbeats are now solely relied upon to determine whether a child is alive and responding to the parent. Delivery acknowledgement messages are still sent from the child to the parent, but their handling now only involves logging them.

Contents (#571)

IMPORTANT: There is 1 breaking change.

Fixes

• 💥 BREAKING CHANGE: Remove delivery acknowledgement timeout

---

Upgrade instructions

💥 Remove delivery acknowledgement timeout

Remove the delivery_acknowledgement_timeout argument from Child.ask and Service.wait_for_answer

Deprecate the Datafile.path property

Contents (#568)

Enhancements

• Deprecate the Datafile.path property

Fixes

• Ensure log records sent via Pub/Sub have a valid level number
• Handle empty expiration time in create-push-subscription CLI command
• Only log local file deletion in Runner if there are files to be deleted

Cache local metadata files and improve default push subscription creation behaviour

Summary

Add some performance and reliability improvements, including a local metadata file memory cache to speed up local metadata access for datafiles and datasets. Also set push subscriptions made via the CLI to never expire by default.

Contents (#567)

Enhancements

• Cache local metadata files' contents in memory to avoid wasted time repeatedly access the disk
• Break from datafile finding loop earlier if not in recursive mode

Fixes

• Set push subscriptions created via CLI to never expire by default

Refactoring

• Add and use local metadata context manager when updating local metadata files

Allow ignoring stored metadata when instantiating manifests

Contents (#563)

Enhancements

• Allow ignoring stored dataset and datafile metadata when instantiating manifests