Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add jumpstart and bedrock deployments #2663

Merged
merged 2 commits into from
Feb 13, 2025
Merged

add jumpstart and bedrock deployments #2663

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6e05f82
add jumpstart and bedrock
pagezyhf Feb 11, 2025
48589a9
Update deepseek-r1-aws.md
pagezyhf Feb 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion deepseek-r1-aws.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ We collaborate with Amazon Web Services to make it easier for developers to depl
Let’s review how you can deploy and fine-tune DeepSeek R1 models with Hugging Face on AWS.
- [Deploy DeepSeek R1 models](#deploy-deepseek-r1-models)
- [Deploy on AWS with Hugging Face Inference Endpoints](#deploy-on-aws-with-hugging-face-inference-endpoints)
- [Deploy on Amazon Bedrock Marketplace]
- [Deploy on Amazon SageMaker AI with Hugging Face LLM DLCs](#deploy-on-amazon-sagemaker-ai-with-hugging-face-llm-dlcs)
- [DeepSeek R1 on GPUs](#deepseek-r1-on-gpus)
- [Distilled models on GPUs](#distilled-models-on-gpus)
Expand All @@ -48,6 +49,12 @@ You can find DeepSeek R1 and distilled models, as well as other popular open LLM

| **Note:** The team is working on enabling DeepSeek models deployment on Inferentia instances. Stay tuned!

### Deploy on Amazon Bedrock Marketplace

You can deploy the Deepseek distilled models on Amazon Bedrock via the marketplace, which will deploy an endpoint in Amazon SageMaker AI under the hood. Here is a video of how you can navigate through the AWS console:

![bedrock-deployment.gif](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/deepseek-aws/bedrock-deployment.gif)

### Deploy on Amazon Sagemaker AI with Hugging Face LLM DLCs

#### DeepSeek R1 on GPUs
Expand All @@ -56,7 +63,12 @@ You can find DeepSeek R1 and distilled models, as well as other popular open LLM

#### Distilled models on GPUs

Let’s walk through the deployment of DeepSeek-R1-Distill-Llama-70B.
You can deploy the Deepseek distilled models on Amazon Sagemaker AI with Hugging Face LLM DLCs using Jumpstart directly or using the Python Sagemaker SDK.
Here is a video of how you can navigate through the AWS console:

![bedrock-deployment.gif](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/deepseek-aws/jumpstart-deployment.gif)

Now we have seen how to deploy usig Jumpstart, let’s walk through the Python Sagemaker SDK deployment of DeepSeek-R1-Distill-Llama-70B.

Code snippets are available on the model page under the Deploy button!

Expand Down