Skip to content

Commit 08e4652

Browse files
lu-ohailu-ohai
committed
2 parents 2bf51e1 + 84e633e commit 08e4652

File tree

315 files changed

+16477
-9995
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

315 files changed

+16477
-9995
lines changed

‎.github/workflows/run-unittests.yml

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
name: Unit Tests
2+
3+
on:
4+
workflow_dispatch:
5+
push:
6+
branches:
7+
- main
8+
- 'release/**'
9+
- develop
10+
paths:
11+
- '!docs/**'
12+
13+
pull_request:
14+
15+
# Cancel in progress workflows on pull_requests.
16+
# https://docs.github.com/en/actions/using-jobs/using-concurrency#example-using-a-fallback-value
17+
concurrency:
18+
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
19+
cancel-in-progress: true
20+
21+
permissions:
22+
contents: read
23+
24+
jobs:
25+
test:
26+
name: ${{ matrix.tests-type }}, python ${{ matrix.python-version }}
27+
runs-on: ubuntu-latest
28+
timeout-minutes: 45
29+
30+
strategy:
31+
fail-fast: false
32+
matrix:
33+
python-version: ["3.7","3.8","3.9","3.10"]
34+
tests-type: ["DefaultSetup"]
35+
36+
steps:
37+
- uses: actions/checkout@v3
38+
# - uses: actions/cache@v3
39+
# with:
40+
# path: ~/.cache/pip
41+
# key: ${{ runner.os }}-pip-${{ hashFiles('**/test-requirements.txt') }}
42+
# restore-keys: |
43+
# ${{ runner.os }}-pip-
44+
- uses: actions/setup-python@v4
45+
with:
46+
python-version: ${{ matrix.python-version }}
47+
48+
- name: "Setup test env"
49+
run: |
50+
pip install coverage pytest-codecov tox==4.2.8
51+
52+
- name: "Run unit tests"
53+
timeout-minutes: 45
54+
shell: bash
55+
run: |
56+
set -x # print commands that are executed
57+
# coverage erase
58+
# ./scripts/runtox.sh "${{ matrix.python-version }}-${{ matrix.tests-type }}" --cov --cov-report=
59+
# coverage combine .coverage-*
60+
# coverage html -i
61+
62+
# Uploading test artifacts
63+
# https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts#uploading-build-and-test-artifacts
64+
# - name: "Upload artifact"
65+
# uses: actions/upload-artifact@v3
66+
# with:
67+
# name: code-coverage-report
68+
# path: htmlcov/
69+
# retention-days: 10

‎README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -166,7 +166,7 @@ This example uses SQL injection safe binding variables.
166166

167167
## Contributing
168168

169-
This project welcomes contributions from the community. Before submitting a pull request, please review our contribution guide [CONTRIBUTING.md](https://github.com/oracle/accelerated-data-science/blob/main/CONTRIBUTING.md).
169+
This project welcomes contributions from the community. Before submitting a pull request, please [review our contribution guide](./CONTRIBUTING.md)
170170

171171
Find Getting Started instructions for developers in [README-development.md](https://github.com/oracle/accelerated-data-science/blob/main/README-development.md)
172172

‎SECURITY.md

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,10 @@ and privacy of all our users.
66

77
Please do NOT raise a GitHub Issue to report a security vulnerability. If you
88
believe you have found a security vulnerability, please submit a report to
9-
[[email protected]](mailto:[email protected]) preferably with a proof of concept.
10-
Please review some additional information on
11-
[how to report security vulnerabilities to Oracle](https://www.oracle.com/corporate/security-practices/assurance/vulnerability/reporting.html).
9+
[[email protected]][1] preferably with a proof of concept. Please review
10+
some additional information on [how to report security vulnerabilities to Oracle][2].
1211
We encourage people who contact Oracle Security to use email encryption using
13-
[our encryption key](https://www.oracle.com/security-alerts/encryptionkey.html).
12+
[our encryption key][3].
1413

1514
We ask that you do not use other channels or contact the project maintainers
1615
directly.
@@ -22,15 +21,18 @@ security features are welcome on GitHub Issues.
2221

2322
Security updates will be released on a regular cadence. Many of our projects
2423
will typically release security fixes in conjunction with the
25-
[Oracle Critical Patch Update](https://www.oracle.com/security-alerts/encryptionkey.html) program.
26-
Security updates are released on the Tuesday closest to the 17th day of January, April, July and October.
27-
A pre-release announcement will be published on the Thursday preceding each release. Additional
28-
information, including past advisories, is available on our
29-
[security alerts](https://www.oracle.com/security-alerts/) page.
24+
[Oracle Critical Patch Update][3] program. Additional
25+
information, including past advisories, is available on our [security alerts][4]
26+
page.
3027

3128
## Security-related information
3229

3330
We will provide security related information such as a threat model, considerations
3431
for secure use, or any known security issues in our documentation. Please note
3532
that labs and sample code are intended to demonstrate a concept and may not be
3633
sufficiently hardened for production use.
34+
35+
[1]: mailto:[email protected]
36+
[2]: https://www.oracle.com/corporate/security-practices/assurance/vulnerability/reporting.html
37+
[3]: https://www.oracle.com/security-alerts/encryptionkey.html
38+
[4]: https://www.oracle.com/security-alerts/

‎ads/ads_version.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
{
2-
"version": "2.8.0"
2+
"version": "2.8.2"
33
}

‎ads/automl/driver.py

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,19 @@
11
#!/usr/bin/env python
22
# -*- coding: utf-8 -*--
33

4-
# Copyright (c) 2020, 2022 Oracle and/or its affiliates.
4+
# Copyright (c) 2020, 2023 Oracle and/or its affiliates.
55
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
66

77
from __future__ import absolute_import, division, print_function
88

9+
import copy
10+
911
import numpy as np
1012
import pandas as pd
1113
import scipy
12-
import copy
14+
1315
from ads.common import logger, utils
16+
from ads.common.decorator.deprecate import deprecated
1417
from ads.common.model import ADSModel
1518
from ads.dataset import helper
1619
from ads.dataset.classification_dataset import (
@@ -37,6 +40,10 @@
3740
}
3841

3942

43+
@deprecated(
44+
details="Working with AutoML has moved from within ADS to working directly with the AutoMLx library. AutoMLx are preinstalled in conda pack automlx_p38_cpu_v2 and later, and can now be updated independently of ADS. AutoMLx documentation may be found at https://docs.oracle.com/en-us/iaas/tools/automlx/latest/html/multiversion/v23.1.1/index.html. Notebook examples are in Oracle's samples repository: https://github.com/oracle-samples/oci-data-science-ai-samples/tree/master/notebook_examples and a migration tutorial can be found at https://accelerated-data-science.readthedocs.io/en/latest/user_guide/model_training/automl/quick_start.html .",
45+
raise_error=True,
46+
)
4047
def get_ml_task_type(X, y, classes):
4148
"""
4249
Gets the ML task type and returns it.
@@ -79,6 +86,10 @@ def get_ml_task_type(X, y, classes):
7986

8087

8188
class AutoML:
89+
@deprecated(
90+
details="Working with AutoML has moved from within ADS to working directly with the AutoMLx library. AutoMLx are preinstalled in conda pack automlx_p38_cpu_v2 and later, and can now be updated independently of ADS. AutoMLx documentation may be found at https://docs.oracle.com/en-us/iaas/tools/automlx/latest/html/multiversion/v23.1.1/index.html. Notebook examples are in Oracle's samples repository: https://github.com/oracle-samples/oci-data-science-ai-samples/tree/master/notebook_examples and a migration tutorial can be found at https://accelerated-data-science.readthedocs.io/en/latest/user_guide/model_training/automl/quick_start.html .",
91+
raise_error=True,
92+
)
8293
def __init__(
8394
self,
8495
training_data,
@@ -87,7 +98,7 @@ def __init__(
8798
baseline="dummy",
8899
client=None,
89100
):
90-
r"""
101+
"""
91102
Creates an Automatic machine learning object.
92103
93104
Parameters

‎ads/automl/provider.py

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
#!/usr/bin/env python
22
# -*- coding: utf-8; -*-
33

4-
# Copyright (c) 2020, 2022 Oracle and/or its affiliates.
4+
# Copyright (c) 2020, 2023 Oracle and/or its affiliates.
55
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
66

77
import logging
@@ -41,6 +41,10 @@ class AutoMLProvider(ABC):
4141
implement train() and get_transformer_pipeline().
4242
"""
4343

44+
@deprecated(
45+
details="Working with AutoML has moved from within ADS to working directly with the AutoMLx library. AutoMLx are preinstalled in conda pack automlx_p38_cpu_v2 and later, and can now be updated independently of ADS. AutoMLx documentation may be found at https://docs.oracle.com/en-us/iaas/tools/automlx/latest/html/multiversion/v23.1.1/index.html. Notebook examples are in Oracle's samples repository: https://github.com/oracle-samples/oci-data-science-ai-samples/tree/master/notebook_examples and a migration tutorial can be found at https://accelerated-data-science.readthedocs.io/en/latest/user_guide/model_training/automl/quick_start.html .",
46+
raise_error=True,
47+
)
4448
def __init__(self):
4549
self.X_train = None
4650
self.y_train = None
@@ -143,6 +147,10 @@ class BaselineModel(object):
143147
interface. Labels (y) are encoded using DataFrameLabelEncoder.
144148
"""
145149

150+
@deprecated(
151+
details="Working with AutoML has moved from within ADS to working directly with the AutoMLx library. AutoMLx are preinstalled in conda pack automlx_p38_cpu_v2 and later, and can now be updated independently of ADS. AutoMLx documentation may be found at https://docs.oracle.com/en-us/iaas/tools/automlx/latest/html/multiversion/v23.1.1/index.html. Notebook examples are in Oracle's samples repository: https://github.com/oracle-samples/oci-data-science-ai-samples/tree/master/notebook_examples and a migration tutorial can be found at https://accelerated-data-science.readthedocs.io/en/latest/user_guide/model_training/automl/quick_start.html .",
152+
raise_error=True,
153+
)
146154
def __init__(self, est):
147155
self.est = est
148156
self.df_label_encoder = DataFrameLabelEncoder()
@@ -250,6 +258,10 @@ def get_transformer_pipeline(self):
250258
msg = "Baseline"
251259
return [("automl_preprocessing", AutoMLPreprocessingTransformer(msg))]
252260

261+
@deprecated(
262+
details="Working with AutoML has moved from within ADS to working directly with the AutoMLx library. AutoMLx are preinstalled in conda pack automlx_p38_cpu_v2 and later, and can now be updated independently of ADS. AutoMLx documentation may be found at https://docs.oracle.com/en-us/iaas/tools/automlx/latest/html/multiversion/v23.1.1/index.html. Notebook examples are in Oracle's samples repository: https://github.com/oracle-samples/oci-data-science-ai-samples/tree/master/notebook_examples and a migration tutorial can be found at https://accelerated-data-science.readthedocs.io/en/latest/user_guide/model_training/automl/quick_start.html .",
263+
raise_error=True,
264+
)
253265
def __init__(self, est):
254266
"""
255267
Generates a baseline model using the Zero Rule algorithm by default. For a classification
@@ -309,6 +321,7 @@ class OracleAutoMLProvider(AutoMLProvider, ABC):
309321
@deprecated(
310322
"2.6.7",
311323
details="Oracle AutoML is recommended to be directly instantiated by importing automlx package",
324+
raise_error=True,
312325
)
313326
def __init__(
314327
self, n_jobs=-1, loglevel=None, logger_override=None, model_n_jobs: int = 1
@@ -870,6 +883,10 @@ def visualize_tuning_trials(self, ylabel=None):
870883

871884

872885
class AutoMLPreprocessingTransformer(object): # pragma: no cover
886+
@deprecated(
887+
details="Working with AutoML has moved from within ADS to working directly with the AutoMLx library. AutoMLx are preinstalled in conda pack automlx_p38_cpu_v2 and later, and can now be updated independently of ADS. AutoMLx documentation may be found at https://docs.oracle.com/en-us/iaas/tools/automlx/latest/html/multiversion/v23.1.1/index.html. Notebook examples are in Oracle's samples repository: https://github.com/oracle-samples/oci-data-science-ai-samples/tree/master/notebook_examples and a migration tutorial can be found at https://accelerated-data-science.readthedocs.io/en/latest/user_guide/model_training/automl/quick_start.html .",
888+
raise_error=True,
889+
)
873890
def __init__(self, msg):
874891
self.msg = msg
875892

@@ -915,6 +932,10 @@ def __repr__(self):
915932

916933

917934
class AutoMLFeatureSelection(object): # pragma: no cover
935+
@deprecated(
936+
details="Working with AutoML has moved from within ADS to working directly with the AutoMLx library. AutoMLx are preinstalled in conda pack automlx_p38_cpu_v2 and later, and can now be updated independently of ADS. AutoMLx documentation may be found at https://docs.oracle.com/en-us/iaas/tools/automlx/latest/html/multiversion/v23.1.1/index.html. Notebook examples are in Oracle's samples repository: https://github.com/oracle-samples/oci-data-science-ai-samples/tree/master/notebook_examples and a migration tutorial can be found at https://accelerated-data-science.readthedocs.io/en/latest/user_guide/model_training/automl/quick_start.html .",
937+
raise_error=True,
938+
)
918939
def __init__(self, msg):
919940
self.msg = msg
920941

‎ads/catalog/model.py

Lines changed: 15 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
#!/usr/bin/env python
22
# -*- coding: utf-8; -*-
33

4-
# Copyright (c) 2020, 2022 Oracle and/or its affiliates.
4+
# Copyright (c) 2020, 2023 Oracle and/or its affiliates.
55
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
66

77
import warnings
@@ -49,7 +49,7 @@
4949
OCI_REGION_METADATA,
5050
PROJECT_OCID,
5151
)
52-
from ads.dataset.progress import DummyProgressBar, TqdmProgressBar
52+
from ads.dataset.progress import TqdmProgressBar
5353
from ads.feature_engineering.schema import Schema
5454
from ads.model.model_version_set import ModelVersionSet, _extract_model_version_set_id
5555
from ads.model.deployment.model_deployer import ModelDeployer
@@ -1045,7 +1045,7 @@ def _download_large_artifact(
10451045
model_id: str,
10461046
target_dir: str,
10471047
bucket_uri: str,
1048-
progress: Union[TqdmProgressBar, DummyProgressBar],
1048+
progress: TqdmProgressBar,
10491049
remove_existing_artifact: Optional[bool] = True,
10501050
) -> None:
10511051
"""
@@ -1062,7 +1062,7 @@ def _download_large_artifact(
10621062
The OCI Object Storage URI where model artifacts will be copied to.
10631063
The `bucket_uri` is only necessary for downloading large artifacts with
10641064
size is greater than 2GB. Example: `bucket_uri=oci://<bucket_name>@<namespace>/prefix/`.
1065-
progress: Union[TqdmProgressBar, DummyProgressBar]
1065+
progress: TqdmProgressBar
10661066
The progress bar.
10671067
remove_existing_artifact: (bool, optional). Defaults to `True`.
10681068
Whether artifacts uploaded to object storage bucket need to be removed or not.
@@ -1125,7 +1125,7 @@ def _download_small_artifact(
11251125
self,
11261126
model_id: str,
11271127
target_dir: str,
1128-
progress: Union[TqdmProgressBar, DummyProgressBar],
1128+
progress: TqdmProgressBar,
11291129
) -> None:
11301130
"""
11311131
Downloads the model artifacts from model catalog to target_dir based on `model_id`.
@@ -1137,7 +1137,7 @@ def _download_small_artifact(
11371137
The OCID of the model to download.
11381138
target_dir: str
11391139
The target location of model artifacts.
1140-
progress: Union[TqdmProgressBar, DummyProgressBar]
1140+
progress: TqdmProgressBar
11411141
The progress bar.
11421142
11431143
Returns
@@ -1422,9 +1422,7 @@ def upload_model(
14221422
progress.update("Done")
14231423
return self.get_model(model.data.id)
14241424

1425-
def _prepare_model_artifact(
1426-
self, model_artifact, progress: Union[TqdmProgressBar, DummyProgressBar]
1427-
) -> str:
1425+
def _prepare_model_artifact(self, model_artifact, progress: TqdmProgressBar) -> str:
14281426
"""Prepares model artifacts to save in the Model Catalog.
14291427
14301428
Returns
@@ -1559,15 +1557,21 @@ def _wait_for_work_request(
15591557
work_request_logs = self.ds_client.list_work_request_logs(
15601558
work_request_id
15611559
).data
1562-
new_work_request_logs = work_request_logs[i:]
1560+
if work_request_logs:
1561+
new_work_request_logs = work_request_logs[i:]
15631562

15641563
for wr_item in new_work_request_logs:
15651564
progress.update(wr_item.message)
15661565
i += 1
15671566

15681567
if work_request.data.status in STOP_STATE:
15691568
if work_request.data.status != WorkRequest.STATUS_SUCCEEDED:
1570-
raise Exception(work_request_logs[-1].message)
1569+
if work_request_logs:
1570+
raise Exception(work_request_logs[-1].message)
1571+
else:
1572+
raise Exception(
1573+
"An error occurred in attempt to perform the operation. Check the service logs to get more details."
1574+
)
15711575
else:
15721576
break
15731577
return work_request

0 commit comments

Comments
 (0)