Operationalizing Machine Learning

The running project code is in the file operationalizing_ml.ipynb which is based on the starter code file starter_files/aml-pipelines-with-automated-machine-learning-step.ipynb

This project comprises of operationalizing Machine Learning in Microsoft Azure. A classification model is trained and deployed. The deployed model endpoint URI can later be used to make predictions A pipeline is also created, published and consumed. The published pipeline endpoint URI can later be used to initiate a new run.

Architectural Diagram

Suggestions for improvement

1. Consider implementing one or more stand out suggestions made in the project rubric
2. Research and use balanced data to improve the outcome of the predictions
3. Version the pipelines. This would allow development of new models while customers continue to use the existing version

Key Steps

Step 2: Automated ML Experiment

This step allows a model to be trained across different algorithms/parameters and highlights the best model

1. An Azure ML Dataset is created and registered if it does not already exist. The data is retrieved from the URL provided.

2. A Compute cluster is created if it does not already exist
3. An AtoML configuration is specified with key information such as the best metric to use, the column label, etc.
4. A run is then submitted to train the model. Once the experiment/run is completed, a number of models including the best model may be inspected.

5. The best model is associated with the run and it is the VotingEnsemble for this run

Step 3: Deploy the best model

In this step, we use the Azure ML Studio user interface to deploy the best model. We use ACI (Azure Container Instance) with authentication enabled.

Step 4. Enable logging

Logging (Application Insights) helps troubleshoot and understand the workflow. It also helps with quantifying performance at various stages of the execution. The WebServcie is used to enable/disable Application Insights.

Display output from running logs.py. logs.py retrieves the logs from the WebService.

Step 5. Swagger Doc

Once a model is published, Azure ML exposes a swagger.json file. This can be consumed by Swagger and helps with the documentation of the methods that are exposed together with the JSON payloads for input/output. This makes it much easier to start consuming the endpoint.

Step 6. Consume model endpoints

The model endpoint is consumed by making a REST API call (endpoint.py). If authentication is enabled, the key must also be provide. The output of such a model invocation is displayed below.

Step 7. Create and publish pipeline

1. A pipeline is created when it is "run" in the context of an experiment. Pipelines are visualized in the Pipelines section of Azure ML Studio

2. Once a pipeline is published, a pipeline endpoint is generated and may be reviewed from Azure ML Studio under the 'Pipeline Endpoint' tab

3. Review of the pipeline from Azure ML studio showing the bank marketing dataset and the AutoML module

4. Once published, the Pipeline Details tab will show the published pipeline status (Active in this case) and also the pipeline REST endpoint that can be called to "run" the pipeline.

5. The RunDetails widget asynchronously displays the run details in the Notebook as the pipeline run progresses.

6. A pipeline run status (Scheduled, Completed, etc.) can be reviewed in Azure ML Studio in the Pipelines section

Screen Recording

Project Screencast