You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
An inactive model deployment can be activated using a ``ModelDeployment`` object. See `Activate Model Deployments <https://docs.oracle.com/en-us/iaas/data-science/using/model_dep_manage.htm#model_dep_deactivate>`_ for more details.
5
+
6
+
If you have a ``ModelDeployment`` object, you can use the ``.activate()`` method to activate the model that is associated with that object.
7
+
8
+
In the following code snippets, the variable ``deployment`` is a ``ModelDeployment`` object. This object can be obtained from a call to ``.deploy()`` or ``.from_id()``.
An activate model deployment can be deactivated using a ``ModelDeployment`` object. Deactivating a model deployment shuts down the instances that are associated with your deployment. See `Deactivate Model Deployments <https://docs.oracle.com/en-us/iaas/data-science/using/model_dep_manage.htm#model_dep_deactivate>`_ for more details.
5
+
6
+
If you have a ``ModelDeployment`` object, you can use the ``.deactivate()`` method to deactivate the model that is associated with that object.
7
+
8
+
In the following code snippets, the variable ``deployment`` is a ``ModelDeployment`` object. This object can be obtained from a call to ``.deploy()`` or ``.from_id()``.
Copy file name to clipboardExpand all lines: docs/source/user_guide/model_deployment/delete.rst
+2-2Lines changed: 2 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -5,11 +5,11 @@ A model deployment can be deleted using a ``ModelDeployment`` object.
5
5
6
6
When a model deployment is deleted, it deletes the load balancer instances associated with it. However, it doesn't delete other resources like log group, log, or model.
7
7
8
-
If you have a ``ModelDeployment`` object, you can use the ``.delete()`` method to delete the model that is associated with that object. The optional ``wait_for_completion`` parameter accepts a Boolean and determines if the process is blocking or not.
8
+
If you have a ``ModelDeployment`` object, you can use the ``.delete()`` method to delete the model that is associated with that object.
9
9
10
10
In the following code snippets, the variable ``deployment`` is a ``ModelDeployment`` object. This object can be obtained from a call to ``.deploy()`` or ``.from_id()``.
The ``.deploy()`` method of the ``ModelDeployment`` class is used to create a model deployment. It has the following parameters:
4
+
To deploy a model deployment, you'll need to define a ``ModelDeployment`` object and call the ``.deploy()`` of it. You could either use API or YAML to define the ``ModelDeployment`` object.
5
5
6
-
* ``max_wait_time``: The timeout limit, in seconds, for the deployment process to wait until it is active. Defaults to 1200 seconds.
7
-
* ``poll_interval``: The interval between checks of the deployment status in seconds. Defaults to 30 seconds.
8
-
* ``wait_for_completion``: Blocked process until the deployment has been completed. Defaults to ``True``.
6
+
Deploy a ModelDeployment on Docker Container Runtime
To deploy a ``ModelDeployment``, you need to define a ``ModelDeployment`` object first and then call ``.deploy()``. There are two ways to define a ``ModelDeployment`` object.
9
+
The ADS ``ModelDeploymentContainerRuntime`` class allows you to run a container image using OCI data science model deployment.
11
10
12
-
Builder Pattern
13
-
===============
11
+
To use the ``ModelDeploymentContainerRuntime``, you need to first build a docker container image. See `<build_container_image>` for the end-to-end example. Once you have the image, push it to `OCI container registry <https://docs.oracle.com/en-us/iaas/Content/Registry/Concepts/registryoverview.htm>`_. See `Creating a Repository <https://docs.oracle.com/en-us/iaas/Content/Registry/Tasks/registrycreatingarepository.htm>`_ and `Pushing Images Using the Docker CLI <https://docs.oracle.com/en-us/iaas/Content/Registry/Tasks/registrycreatingarepository.htm>`_ for more details.
14
12
15
-
Infrastructure
16
-
--------------
13
+
To configure ``ModelDeploymentContainerRuntime``, you must specify the container ``image``. You can optionally specify the `entrypoint` and `cmd` for running the container (See `Understand how CMD and ENTRYPOINT interact <https://docs.docker.com/engine/reference/builder/#understand-how-cmd-and-entrypoint-interact>`_).
17
14
18
-
You define the model deployment infrastructure by passing the following properties to ``ModelDeploymentInfrastructure``:
15
+
Below is an example of deploying model on docker container runtime:
19
16
20
-
* ``access_log``: Log group OCID and log OCID for the access logs.
21
-
* ``bandwidth_mbps``: The bandwidth limit on the load balancer in Mbps.
22
-
* ``compartment_id``: Compartment OCID that the model deployment belongs to.
23
-
* ``replica``: The number of instances to deploy.
24
-
* ``shape_name``: The instance shape name to use. For example, "VM.Standard.E4.Flex".
25
-
* ``shape_config_details``: The instance shape configure details to use if flexible shape is selected for ``shape_name``.
26
-
* ``predict_log``: Log group OCID and log OCID for the predict logs.
27
-
* ``project_id``: Project OCID that the model deployment will belong to.
28
-
* ``web_concurrency``: The web concurrency to use.
17
+
.. tabs::
29
18
30
-
Below is an example to define a ``ModelDeploymentInfrastructure`` object
31
-
32
-
.. code-block:: python3
19
+
.. code-tab:: Python3
20
+
:caption: Python
33
21
34
22
from ads.model.deployment.model_deployment_infrastructure import ModelDeploymentInfrastructure
23
+
from ads.model.deployment.model_deployment_runtime import ModelDeploymentContainerRuntime
24
+
from ads.model.deployment import ModelDeployment
35
25
36
26
infrastructure = (
37
27
ModelDeploymentInfrastructure()
@@ -55,40 +45,6 @@ Below is an example to define a ``ModelDeploymentInfrastructure`` object
55
45
)
56
46
)
57
47
58
-
59
-
Runtime
60
-
-------
61
-
62
-
The Data Science Model Deployment supports service managed conda runtime and customized docker container runtime.
63
-
64
-
* ``ModelDeploymentContainerRuntime`` allows you to deploy model deployment on customized docker container runtime.
65
-
* ``ModelDeploymentCondaRuntime`` allows you to deploy model deployment on service-managed conda runtime.
66
-
67
-
ModelDeploymentContainerRuntime
68
-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
69
-
70
-
To use the ``ModelDeploymentContainerRuntime``, you need to first push the image to `OCI container registry <https://docs.oracle.com/en-us/iaas/Content/Registry/Concepts/registryoverview.htm>`_. See `Creating a Repository <https://docs.oracle.com/en-us/iaas/Content/Registry/Tasks/registrycreatingarepository.htm>`_ and `Pushing Images Using the Docker CLI <https://docs.oracle.com/en-us/iaas/Content/Registry/Tasks/registrycreatingarepository.htm>`_ for more details.
71
-
72
-
You can define the model deployment container runtime by passing the following properties to ``ModelDeploymentContainerRuntime`` object:
73
-
74
-
* ``model_uri``: The model ocid or path to model artifacts directory that is used in the model deployment.
75
-
* ``deployment_mode``: The mode of model deployment. Allowed deployment modes are ``HTTPS_ONLY`` and ``STREAM_ONLY``. Optional.
76
-
* ``input_stream_ids``: The input stream ids for model deployment. Required when deployment mode is ``STREAM_ONLY``.
77
-
* ``output_stream_ids``: The output stream ids for model deployment. Required when deployment mode is ``STREAM_ONLY``.
78
-
* ``env``: The environment variables. Optional.
79
-
* ``image``: The full path of docker container image to the OCIR registry. The acceptable formats are: ``<region>.ocir.io/<registry>/<image>:<tag>`` and ``<region>.ocir.io/<registry>/<image>:<tag>@digest``. Required.
80
-
* ``image_digest``: The docker container image digest. Optional.
81
-
* ``entrypoint``: The entrypoint to docker container image. Optional.
82
-
* ``server_port``: The server port of docker container image. Optional.
83
-
* ``health_check_port``: The health check port of docker container image. Optional.
84
-
* ``cmd``: The additional commands to docker container image. The commands can be args to the entrypoint or the only command to execute in the absence of an entrypoint. Optional.
85
-
86
-
Below is an example to define a ``ModelDeploymentContainerRuntime`` object
87
-
88
-
.. code-block:: python3
89
-
90
-
from ads.model.deployment.model_deployment_runtime import ModelDeploymentContainerRuntime
91
-
92
48
container_runtime = (
93
49
ModelDeploymentContainerRuntime()
94
50
.with_image("<IMAGE_PATH_TO_OCIR>")
@@ -101,94 +57,129 @@ Below is an example to define a ``ModelDeploymentContainerRuntime`` object
101
57
.with_model_uri("<MODEL_URI>")
102
58
)
103
59
60
+
deployment = (
61
+
ModelDeployment()
62
+
.with_display_name("Model Deployment Demo using ADS")
63
+
.with_description("The model deployment description")
64
+
.with_freeform_tags({"key1":"value1"})
65
+
.with_infrastructure(infrastructure)
66
+
.with_runtime(container_runtime)
67
+
)
104
68
105
-
ModelDeploymentCondaRuntime
106
-
~~~~~~~~~~~~~~~~~~~~~~~~~~~
69
+
deployment.deploy()
107
70
108
-
You can define the model deployment conda runtime by passing the following properties to ``ModelDeploymentCondaRuntime`` object:
71
+
.. code-tab:: Python3
72
+
:caption: YAML
109
73
110
-
* ``model_uri``: The model ocid or path to model artifacts that is used in the model deployment.
111
-
* ``deployment_mode``: The deployment mode. The allowed deployment modes are ``HTTPS_ONLY`` and ``STREAM_ONLY``. Optional.
112
-
* ``input_stream_ids``: The input stream ids for model deployment. Required when deployment mode is ``STREAM_ONLY``.
113
-
* ``output_stream_ids``: The output stream ids for model deployment. Required when deployment mode is ``STREAM_ONLY``.
114
-
* ``env``: The environment variables. Optional.
74
+
from ads.model.deployment import ModelDeployment
115
75
116
-
Below is an example to define a ``ModelDeploymentCondaRuntime`` object
from ads.model.deployment.model_deployment_runtime import ModelDeploymentCondaRuntime
121
120
122
-
conda_runtime = (
123
-
ModelDeploymentCondaRuntime()
124
-
.with_env({"key":"value"})
125
-
.with_deployment_mode("HTTPS_ONLY")
126
-
.with_model_uri("<MODEL_URI>")
127
-
)
121
+
Deploy a ModelDeployment on Conda Runtime
122
+
=========================================
128
123
124
+
To deploy a model deployment on conda runtime, you need to configure ``ModelDeploymentCondaRuntime``.
129
125
130
-
ModelDeployment
131
-
~~~~~~~~~~~~~~~
126
+
Below is an example of deploying model on conda runtime:
132
127
133
-
You can define the model deployment by passing the following properties to ``ModelDeployment`` object:
128
+
.. tabs::
134
129
135
-
* ``defined_tags``: A dictionary of defined tags to be attached to the model deployment. Optional.
136
-
* ``description``: A description of the model deployment. Optional.
137
-
* ``display_name``: A name that identifies the model deployment in the Console.
138
-
* ``freeform_tags``: A dictionary of freeform tags to be attached to the model deployment. Optional.
139
-
* ``runtime``: The runtime configuration to be attached to the model deployment.
140
-
* ``infrastructure``: The infrastructure configuration to be attached to the model deployment.
130
+
.. code-tab:: Python3
131
+
:caption: Python
141
132
142
-
Below is an example to define and deploy a ``ModelDeployment`` object with custom docker container runtime
133
+
from ads.model.deployment.model_deployment_infrastructure import ModelDeploymentInfrastructure
134
+
from ads.model.deployment.model_deployment_runtime import ModelDeploymentCondaRuntime
135
+
from ads.model.deployment import ModelDeployment
143
136
144
-
.. code-block:: python3
137
+
infrastructure = (
138
+
ModelDeploymentInfrastructure()
139
+
.with_project_id("<PROJECT_OCID>")
140
+
.with_compartment_id("<COMPARTMENT_OCID>")
141
+
.with_shape_name("VM.Standard.E4.Flex")
142
+
.with_shape_config_details(
143
+
ocpus=1,
144
+
memory_in_gbs=16
145
+
)
146
+
.with_replica(1)
147
+
.with_bandwidth_mbps(10)
148
+
.with_web_concurrency(10)
149
+
.with_access_log(
150
+
log_group_id="<ACCESS_LOG_GROUP_OCID>",
151
+
log_id="<ACCESS_LOG_OCID>"
152
+
)
153
+
.with_predict_log(
154
+
log_group_id="<PREDICT_LOG_GROUP_OCID>",
155
+
log_id="<PREDICT_LOG_OCID>"
156
+
)
157
+
)
145
158
146
-
from ads.model.deployment import ModelDeployment
159
+
conda_runtime = (
160
+
ModelDeploymentCondaRuntime()
161
+
.with_env({"key":"value"})
162
+
.with_deployment_mode("HTTPS_ONLY")
163
+
.with_model_uri("<MODEL_URI>")
164
+
)
147
165
148
166
deployment = (
149
167
ModelDeployment()
150
168
.with_display_name("Model Deployment Demo using ADS")
151
169
.with_description("The model deployment description")
152
170
.with_freeform_tags({"key1":"value1"})
153
171
.with_infrastructure(infrastructure)
154
-
.with_runtime(container_runtime)
172
+
.with_runtime(conda_runtime)
155
173
)
156
174
157
-
deployment.deploy(wait_for_completion=False)
158
-
159
-
160
-
YAML Serialization
161
-
==================
162
-
163
-
A ``ModelDeployment`` object can be serialized to a YAML file by calling ``to_yaml()``, which returns the YAML as a string. You can easily share the YAML with others, and reload the configurations by calling ``from_yaml()``. The ``to_yaml()`` and ``from_yaml()`` methods also take an optional ``uri`` argument for saving and loading the YAML file. This argument can be any URI to the file location supported by `fsspec <https://filesystem-spec.readthedocs.io/en/latest/>`__, including Object Storage. For example:
164
-
165
-
.. code-block:: python3
166
-
167
-
# Save the model deployment configurations to YAML file
0 commit comments