Skip to content

Commit 6a1fb78

Browse files
committed
Update README.md
1 parent 1c3572c commit 6a1fb78

File tree

1 file changed

+25
-60
lines changed

1 file changed

+25
-60
lines changed

README.md

Lines changed: 25 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
Cortex is an open source platform that takes machine learning models—trained with nearly any framework—and turns them into production web APIs in one command. <br>
44

55
<!-- Set header Cache-Control=no-cache on the S3 object metadata (see https://help.github.com/en/articles/about-anonymized-image-urls) -->
6-
![Demo](https://cortex-public.s3-us-west-2.amazonaws.com/demo/gif/v0.8.gif)<br>
6+
![Demo](https://cortex-public.s3-us-west-2.amazonaws.com/demo/gif/v0.8.gif)
77

88
<br>
99

@@ -25,108 +25,73 @@ Cortex is an open source platform that takes machine learning models—trained w
2525

2626
<br>
2727

28-
## Quickstart
28+
## Usage
2929

30-
Below, we'll walk through how to use Cortex to deploy OpenAI's GPT-2 model as a service on AWS. You'll need to [install Cortex](https://www.cortex.dev/install) on your AWS account before getting started.
30+
### Step 1: define your API
3131

32-
<br>
32+
```python
33+
# predictor.py
3334

34-
### Step 1: Define your deployment
35+
model = download_my_model()
36+
37+
def predict(sample, metadata):
38+
return model.predict(sample["text"])
39+
```
3540

36-
The configuration below will download the model from the `cortex-examples` S3 bucket and deploy it as a web service that can serve real-time predictions.
41+
### Step 2: configure your deployment
3742

3843
```yaml
3944
# cortex.yaml
4045

4146
- kind: deployment
42-
name: text
47+
name: sentiment
4348

4449
- kind: api
45-
name: generator
46-
tensorflow:
47-
model: s3://cortex-examples/tensorflow/text-generator/gpt-2/124M
48-
request_handler: handler.py
50+
name: classifier
51+
predictor:
52+
path: predictor.py
4953
compute:
5054
gpu: 1
5155
```
5256
53-
<!-- CORTEX_VERSION_README_MINOR -->
54-
You can run the code that generated the model [here](https://colab.research.google.com/github/cortexlabs/cortex/blob/0.10/examples/tensorflow/text-generator/gpt-2.ipynb).
55-
56-
<br>
57-
58-
### Step 2: Add request handling
59-
60-
The model requires encoded data for inference, but the API should accept strings of natural language as input. It should also decode the inference output as human-readable text.
61-
62-
```python
63-
# handler.py
64-
65-
from encoder import get_encoder
66-
encoder = get_encoder()
67-
68-
def pre_inference(sample, signature, metadata):
69-
context = encoder.encode(sample["text"])
70-
return {"context": [context]}
71-
72-
def post_inference(prediction, signature, metadata):
73-
response = prediction["sample"]
74-
return encoder.decode(response)
75-
```
76-
77-
<br>
78-
79-
### Step 3: Deploy to AWS
80-
81-
`cortex deploy` takes the declarative configuration from `cortex.yaml` and creates it on the cluster.
57+
### Step 3: deploy to AWS
8258
8359
```bash
8460
$ cortex deploy
8561

8662
deployment started
87-
```
8863

89-
You can track the status of a deployment using `cortex get`.
9064

91-
```bash
92-
$ cortex get generator --watch
65+
$ cortex get classifier --watch
9366

9467
status up-to-date available requested last update avg latency
9568
live 1 1 1 8s 123ms
9669

97-
url: http://***.amazonaws.com/text/generator
70+
url: http://***.amazonaws.com/sentiment/classifier
9871
```
9972
100-
Cortex will automatically launch more replicas if the load increases and spin down replicas if there is unused capacity.
101-
102-
<br>
103-
104-
### Step 4: Serve real-time predictions
105-
106-
Once you have your endpoint, you can make requests.
73+
### Step 4: serve real-time predictions
10774
10875
```bash
109-
$ curl http://***.amazonaws.com/text/generator \
76+
$ curl http://***.amazonaws.com/sentiment/classifier \
11077
-X POST -H "Content-Type: application/json" \
111-
-d '{"text": "machine learning"}'
78+
-d '{"text": "the movie was great!"}'
11279

113-
Machine learning, with more than one thousand researchers around the world today, are looking to create computer-driven machine learning algorithms that can also be applied to human and social problems, such as education, health care, employment, medicine, politics, or the environment...
80+
positive
11481
```
11582

116-
Any questions? [chat with us](https://gitter.im/cortexlabs/cortex).
117-
11883
<br>
11984

12085
## How Cortex works
12186

122-
The CLI sends configuration and code to the cluster every time you run `cortex deploy`. Each model is loaded from S3 into a Docker container, along with any Python packages and request handling code. The model is exposed as a web service using Elastic Load Balancing (ELB), Flask, TensorFlow Serving, and ONNX Runtime. The containers are orchestrated on Elastic Kubernetes Service (EKS) while logs and metrics are streamed to CloudWatch.
87+
The CLI sends configuration and code to the cluster every time you run `cortex deploy`. Each model is loaded into a Docker container, along with any Python packages and request handling code. The model is exposed as a web service using Elastic Load Balancing (ELB), Flask, TensorFlow Serving, and ONNX Runtime. The containers are orchestrated on Elastic Kubernetes Service (EKS) while logs and metrics are streamed to CloudWatch.
12388

12489
<br>
12590

12691
## More examples
12792

12893
<!-- CORTEX_VERSION_README_MINOR x4 -->
12994
- [Sentiment analysis](https://github.com/cortexlabs/cortex/tree/0.10/examples/tensorflow/sentiment-analysis) in TensorFlow with BERT
130-
- [Image classification](https://github.com/cortexlabs/cortex/tree/0.10/examples/tensorflow/image-classifier) in TensorFlow with Inception v3
131-
- [Text Generation](https://github.com/cortexlabs/cortex/tree/0.10/examples/pytorch/text-generator) in PyTorch with Hugging Face's DistilGPT2
95+
- [Image classification](https://github.com/cortexlabs/cortex/tree/0.10/examples/tensorflow/image-classifier) in TensorFlow with Inception
96+
- [Text generation](https://github.com/cortexlabs/cortex/tree/0.10/examples/pytorch/text-generator) in PyTorch with DistilGPT2
13297
- [Iris classification](https://github.com/cortexlabs/cortex/tree/0.10/examples/xgboost/iris-classifier) in XGBoost / ONNX

0 commit comments

Comments
 (0)