Skip to content

Improve local developer experience when submitting pipelines #138

Open
@xinyi-joffre

Description

@xinyi-joffre

Since most developers using this repo will iterate on the scripts for the pipeline steps (to update their models or evaluation), it would be great to make the process of submitting those changes smoother.

Current experience:

In the current repo, these 4 steps need to be followed each time a small change in made in one of the steps (train, register, etc.). This is in part due to the requirement to have a unique build id for each pipeline.

  • Make changes in a file (such as train.py)
  • Update build id in .env
  • Run python ml_service/pipelines/build_train_pipeline.py
  • Run python ml_service/pipelines/run_train_pipeline.py

Desired experience:

Since there is no "build_id" when submitting locally, it would be best to allow the user to use a junk or default value.

  • Make changes in a file (such as train.py)
  • Run one script to build and submit the pipeline to run.

Metadata

Metadata

Assignees

No one assigned

    Labels

    docdocumentation update neededenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions