Create deepseek-r1-distill-llama-8b

Author: fpagny

pages/managed-inference/reference-content/deepseek-r1-distill-llama-8b

@@ -0,0 +1,84 @@
+---
+meta:
+  title: Understanding the DeepSeek-R1-Distill-Llama-8B model
+  description: Deploy your own secure DeepSeek-R1-Distill-Llama-8B model with Scaleway Managed Inference. Privacy-focused, fully managed.
+content:
+  h1: Understanding the DeepSeek-R1-Distill-Llama-8B model
+  paragraph: This page provides information on the DeepSeek-R1-Distill-Llama-8B model
+tags:
+dates:
+  validation: 2025-02-06
+  posted: 2025-02-06
+categories:
+  - ai-data
+---
+
+## Model overview
+
+| Attribute       | Details                            |
+|-----------------|------------------------------------|
+| Provider        | [Deepseek](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B)  |
+| License        | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md)  |
+| Compatible Instances | L4, H100, H100-2 (FP8, BF16) |
+| Context Length | up to 32k tokens |
+
+## Model names
+
+```bash
+meta/deepseek-r1-distill-llama-8b:fp8
+meta/deepseek-r1-distill-llama-8b:bf16
+```
+
+## Compatible Instances
+
+| Instance type  | Max context length |
+| ------------- |-------------|
+| L4      | 32k (FP8, BF16) | 
+| H100      | 32k (FP8, BF16) |
+| H100-2      | 32k (FP8, BF16) |
+
+## Model introduction
+
+Released January 21, 2025, Deepseek’s R1 Distilled Llama 8B is a distilled version of Llama model family based on Deepseek R1.
+DeepSeek R1 Distill Llama 8B is designed to improve performance of Llama models on reasoning use case such as mathematics and coding tasks.
+
+## Why is it useful?
+
+It is great to see Deepseek improving open(weight) models, and we are excited to fully support their mission with integration in the Scaleway ecosystem.
+
+- DeepSeek-R1-Distill-Llama was optimized to reach accuracy close to Deepseek-R1 in tasks like mathematics and coding, while keeping inference costs limited and tokens speed efficient. 
+- DeepSeek-R1-Distill-Llama supports a context window up to 32K tokens and tool calling, keeping interaction with other components possible.
+
+## How to use it
+
+### Sending Managed Inference requests
+
+To perform inference tasks with your DeepSeek R1 Distill Llama deployed at Scaleway, use the following command:
+
+```bash
+curl -s \
+-H "Authorization: Bearer <IAM API key>" \
+-H "Content-Type: application/json" \
+--request POST \
+--url "https://<Deployment UUID>.ifr.fr-par.scaleway.com/v1/chat/completions" \
+--data '{"model":"deepseek/deepseek-r1-distill-llama-8b:fp8", "messages":[{"role": "user","content": "There is a llama in my garden, what should I do?"}], "max_tokens": 500, "temperature": 0.7, "stream": false}'
+```
+
+Make sure to replace `<IAM API key>` and `<Deployment UUID>` with your actual [IAM API key](/iam/how-to/create-api-keys/) and the Deployment UUID you are targeting.
+
+<Message type="note">
+  Ensure that the `messages` array is properly formatted with roles (user, assistant) and content.
+</Message>
+
+<Message type="tip">
+  This model is better used without `system prompt`, as suggested by the model provider.
+</Message>
+
+### Receiving Inference responses
+
+Upon sending the HTTP request to the public or private endpoints exposed by the server, you will receive inference responses from the managed Managed Inference server. 
+Process the output data according to your application's needs. The response will contain the output generated by the LLM model based on the input provided in the request.
+
+<Message type="note">
+  Despite efforts for accuracy, the possibility of generated text containing inaccuracies or [hallucinations](/managed-inference/concepts/#hallucinations) exists. Always verify the content generated independently.
+</Message>