Implement a working ML batch prediciton use case with Azure
- Model Training
- Scheduled data movement from local db to cloud storage
- Batch Prediction
-
Load training data into storage (x)
-
Register data store (x)
-
Train model & register with MLFlow (x)
-
Create scoring script & create docker image
-
Deploy docker to batch prediction endpoint
-
Azure Data Factory for scheduling data movement from db to storage
-
ADF should trigger batch endpoint
-
automatize as much as possible with pulumi & github actions
- training and logging model to right experiment works
- Next steps:
- try in jupyter notebook to retrieve the model and make predictions
- deploy model to batch endpoint with custom batch_driver