Skip to content

Commit 2432932

Browse files
fpagnyLaure-di
fpagny
authored and
Laure-di
committed
Update function-calling-support.mdx (scaleway#4372)
1 parent 2481c5f commit 2432932

File tree

5 files changed

+174
-3
lines changed

5 files changed

+174
-3
lines changed

‎menu/navigation.json

+8
Original file line numberDiff line numberDiff line change
@@ -817,6 +817,14 @@
817817
"label": "Llama-3.3-70b-instruct model",
818818
"slug": "llama-3.3-70b-instruct"
819819
},
820+
{
821+
"label": "DeepSeek-R1-Distill-Llama-70B model",
822+
"slug": "deepseek-r1-distill-llama-70b"
823+
},
824+
{
825+
"label": "DeepSeek-R1-Distill-Llama-8B model",
826+
"slug": "deepseek-r1-distill-llama-8b"
827+
},
820828
{
821829
"label": "Mistral-7b-instruct-v0.3 model",
822830
"slug": "mistral-7b-instruct-v0.3"

‎pages/managed-inference/reference-content/deepseek-r1-distill-llama-70b.mdx

+81
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
---
2+
meta:
3+
title: Understanding the DeepSeek-R1-Distill-Llama-70B model
4+
description: Deploy your own secure DeepSeek-R1-Distill-Llama-70B model with Scaleway Managed Inference. Privacy-focused, fully managed.
5+
content:
6+
h1: Understanding the DeepSeek-R1-Distill-Llama-70B model
7+
paragraph: This page provides information on the DeepSeek-R1-Distill-Llama-70B model
8+
tags:
9+
dates:
10+
validation: 2025-02-06
11+
posted: 2025-02-06
12+
categories:
13+
- ai-data
14+
---
15+
16+
## Model overview
17+
18+
| Attribute | Details |
19+
|-----------------|------------------------------------|
20+
| Provider | [Deepseek](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-70B) |
21+
| License | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) |
22+
| Compatible Instances | H100-2 (BF16) |
23+
| Context Length | up to 56k tokens |
24+
25+
## Model names
26+
27+
```bash
28+
deepseek/deepseek-r1-distill-llama-70b:bf16
29+
```
30+
31+
## Compatible Instances
32+
33+
| Instance type | Max context length |
34+
| ------------- |-------------|
35+
| H100-2 | 56k (BF16) |
36+
37+
## Model introduction
38+
39+
Released January 21, 2025, Deepseek’s R1 Distilled Llama 70B is a distilled version of the Llama model family based on Deepseek R1.
40+
DeepSeek R1 Distill Llama 70B is designed to improve the performance of Llama models on reasoning use case such as mathematics and coding tasks.
41+
42+
## Why is it useful?
43+
44+
It is great to see Deepseek improving open(weight) models, and we are excited to fully support their mission with integration in the Scaleway ecosystem.
45+
46+
- DeepSeek-R1-Distill-Llama was optimized to reach accuracy close to Deepseek-R1 in tasks like mathematics and coding, while keeping inference costs limited and tokens speed efficient.
47+
- DeepSeek-R1-Distill-Llama supports a context window of up to 56K tokens and tool calling, keeping interaction with other components possible.
48+
49+
## How to use it
50+
51+
### Sending Managed Inference requests
52+
53+
To perform inference tasks with your DeepSeek R1 Distill Llama deployed at Scaleway, use the following command:
54+
55+
```bash
56+
curl -s \
57+
-H "Authorization: Bearer <IAM API key>" \
58+
-H "Content-Type: application/json" \
59+
--request POST \
60+
--url "https://<Deployment UUID>.ifr.fr-par.scaleway.com/v1/chat/completions" \
61+
--data '{"model":"deepseek/deepseek-r1-distill-llama-70b:fp8", "messages":[{"role": "user","content": "There is a llama in my garden, what should I do?"}], "max_tokens": 500, "temperature": 0.7, "stream": false}'
62+
```
63+
64+
Make sure to replace `<IAM API key>` and `<Deployment UUID>` with your actual [IAM API key](/iam/how-to/create-api-keys/) and the Deployment UUID you are targeting.
65+
66+
<Message type="note">
67+
Ensure that the `messages` array is properly formatted with roles (user, assistant) and content.
68+
</Message>
69+
70+
<Message type="tip">
71+
This model is better used without `system prompt`, as suggested by the model provider.
72+
</Message>
73+
74+
### Receiving inference responses
75+
76+
Upon sending the HTTP request to the public or private endpoints exposed by the server, you will receive inference responses from the Managed Inference server.
77+
Process the output data according to your application's needs. The response will contain the output generated by the LLM model based on the input provided in the request.
78+
79+
<Message type="note">
80+
Despite efforts for accuracy, the possibility of generated text containing inaccuracies or [hallucinations](/managed-inference/concepts/#hallucinations) exists. Always verify the content generated independently.
81+
</Message>

‎pages/managed-inference/reference-content/deepseek-r1-distill-llama-8b.mdx

+82
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
---
2+
meta:
3+
title: Understanding the DeepSeek-R1-Distill-Llama-8B model
4+
description: Deploy your own secure DeepSeek-R1-Distill-Llama-8B model with Scaleway Managed Inference. Privacy-focused, fully managed.
5+
content:
6+
h1: Understanding the DeepSeek-R1-Distill-Llama-8B model
7+
paragraph: This page provides information on the DeepSeek-R1-Distill-Llama-8B model
8+
tags:
9+
dates:
10+
validation: 2025-02-06
11+
posted: 2025-02-06
12+
categories:
13+
- ai-data
14+
---
15+
16+
## Model overview
17+
18+
| Attribute | Details |
19+
|-----------------|------------------------------------|
20+
| Provider | [Deepseek](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B) |
21+
| License | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) |
22+
| Compatible Instances | L4, H100 (BF16) |
23+
| Context Length | up to 131k tokens |
24+
25+
## Model names
26+
27+
```bash
28+
deepseek/deepseek-r1-distill-llama-8b:bf16
29+
```
30+
31+
## Compatible Instances
32+
33+
| Instance type | Max context length |
34+
| ------------- |-------------|
35+
| L4 | 39k (BF16) |
36+
| H100 | 131k (BF16) |
37+
38+
## Model introduction
39+
40+
Released January 21, 2025, Deepseek’s R1 Distilled Llama 8B is a distilled version of the Llama model family based on Deepseek R1.
41+
DeepSeek R1 Distill Llama 8B is designed to improve the performance of Llama models on reasoning use cases such as mathematics and coding tasks.
42+
43+
## Why is it useful?
44+
45+
It is great to see Deepseek improving open(weight) models, and we are excited to fully support their mission with integration in the Scaleway ecosystem.
46+
47+
- DeepSeek-R1-Distill-Llama was optimized to reach accuracy close to Deepseek-R1 in tasks like mathematics and coding, while keeping inference costs limited and tokens speed efficient.
48+
- DeepSeek-R1-Distill-Llama supports a context window of up to 131K tokens and tool calling, keeping interaction with other components possible.
49+
50+
## How to use it
51+
52+
### Sending Managed Inference requests
53+
54+
To perform inference tasks with your DeepSeek R1 Distill Llama deployed at Scaleway, use the following command:
55+
56+
```bash
57+
curl -s \
58+
-H "Authorization: Bearer <IAM API key>" \
59+
-H "Content-Type: application/json" \
60+
--request POST \
61+
--url "https://<Deployment UUID>.ifr.fr-par.scaleway.com/v1/chat/completions" \
62+
--data '{"model":"deepseek/deepseek-r1-distill-llama-8b:fp8", "messages":[{"role": "user","content": "There is a llama in my garden, what should I do?"}], "max_tokens": 500, "temperature": 0.7, "stream": false}'
63+
```
64+
65+
Make sure to replace `<IAM API key>` and `<Deployment UUID>` with your actual [IAM API key](/iam/how-to/create-api-keys/) and the Deployment UUID you are targeting.
66+
67+
<Message type="note">
68+
Ensure that the `messages` array is properly formatted with roles (user, assistant) and content.
69+
</Message>
70+
71+
<Message type="tip">
72+
This model is better used without `system prompt`, as suggested by the model provider.
73+
</Message>
74+
75+
### Receiving inference responses
76+
77+
Upon sending the HTTP request to the public or private endpoints exposed by the server, you will receive inference responses from the managed Managed Inference server.
78+
Process the output data according to your application's needs. The response will contain the output generated by the LLM model based on the input provided in the request.
79+
80+
<Message type="note">
81+
Despite efforts for accuracy, the possibility of generated text containing inaccuracies or [hallucinations](/managed-inference/concepts/#hallucinations) exists. Always verify the content generated independently.
82+
</Message>

‎pages/managed-inference/reference-content/function-calling-support.mdx

+2
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,8 @@ The following models in Scaleway's Managed Inference library can call tools as p
3232
* mistral/mistral-nemo-instruct-2407
3333
* mistral/pixtral-12b-2409
3434
* nvidia/llama-3.1-nemotron-70b-instruct
35+
* deepseek/deepseek-r1-distill-llama-70b
36+
* deepseek/deepseek-r1-distill-llama-8b
3537

3638
## Understanding function calling
3739

‎pages/managed-inference/reference-content/llama-3-8b-instruct.mdx

+1-3
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,6 @@ meta/llama-3-8b-instruct:fp8
3030

3131
## Compatible Instances
3232

33-
## Compatible Instances
34-
3533
| Instance type | Max context length |
3634
| ------------- |-------------|
3735
| L4 | 8192 (FP8, BF16) |
@@ -86,4 +84,4 @@ Process the output data according to your application's needs. The response will
8684

8785
<Message type="note">
8886
Despite efforts for accuracy, the possibility of generated text containing inaccuracies or [hallucinations](/managed-inference/concepts/#hallucinations) exists. Always verify the content generated independently.
89-
</Message>
87+
</Message>

0 commit comments

Comments
 (0)