Skip to content

Commit cb5138e

Browse files
mfhansentsavina
andauthored
Updated External Benchmarking details for OV-2025.4 (#33029)
Updated landing page with new date and release number. Updated brief system description with new machines (PDF) Updated detailed system description with new machines (XLSX) Updated FAQ file with new models. Updated Accuracy table to reflect new number and new models. ### Details: - *item1* - *...* ### Tickets: - *ticket-id* --------- Co-authored-by: Tatiana Savina <[email protected]>
1 parent 3ac45ad commit cb5138e

File tree

5 files changed

+40
-59
lines changed

5 files changed

+40
-59
lines changed

docs/articles_en/about-openvino/performance-benchmarks.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -158,7 +158,7 @@ For a listing of all platforms and configurations used for testing, refer to the
158158
**Disclaimers**
159159

160160
* System configurations used for Intel® Distribution of OpenVINO™ toolkit performance results
161-
are based on release 2025.3, as of September 3rd, 2025.
161+
are based on release 2025.4, as of December 1st, 2025.
162162

163163
* OpenVINO Model Server performance results are based on release 2025.3, as of September 3rd, 2025.
164164

docs/articles_en/about-openvino/performance-benchmarks/model-accuracy-int8-fp32.rst

Lines changed: 36 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ the table for more information.
4141
* - mobilenet-v2
4242
- ImageNet2012
4343
- accuracy @ top1
44-
- -0.93%
44+
- -0.91%
4545
- -0.93%
4646
- -0.91%
4747
- -1.03%
@@ -96,28 +96,28 @@ the table for more information.
9696
- 0.00%
9797
- 0.00%
9898
- 0.02%
99-
- 0.01%
99+
- 0.02%
100100
* - resnet-50
101101
- ImageNet2012
102102
- accuracy @ top1
103103
- 0.00%
104104
- 0.00%
105105
- 0.00%
106-
- -0.04%
106+
- -0.01%
107107
* - ssd-resnet34-1200
108108
- COCO2017_detection_80cl_bkgr
109109
- map
110110
- 0.02%
111111
- 0.02%
112112
- 0.02%
113-
- 0.06%
113+
- -0.23%
114114
* - yolo_v11
115115
- COCO2017_detection_80cl
116116
- [email protected]:0.05:0.95
117-
- 0.00%
118-
- 0.00%
119-
- 0.00%
120-
-
117+
- 0.03%
118+
- -2.21%
119+
- -2.21%
120+
- -2.21%
121121
.. list-table:: Model Accuracy for AMX-FP16, AMX-INT4, Arc-FP16 and Arc-INT4 (Arc™ B-series)
122122
:header-rows: 1
123123

@@ -134,69 +134,62 @@ the table for more information.
134134
- 98.1%
135135
- 94.4%
136136
- 99.5%
137-
- 92.6%
137+
- 94.0%
138138
* - DeepSeek-R1-Distill-Qwen-1.5B
139139
- Data Default WWB
140140
- Similarity
141141
- 96.5%
142142
- 92.4%
143143
- 99.7%
144-
- 92.1%
145-
* - Gemma-3-1B-it
144+
- 92.3%
145+
* - Gemma-3-4B-it
146146
- Data Default WWB
147147
- Similarity
148-
- 97.3%
149148
- 92.0%
150-
- 99.2%
151-
- 91.5%
152-
* - GLM4-9B-Chat
153-
- Data Default WWB
154-
- Similarity
155-
- 98.8%
156-
- 93.3%
157-
- %
158-
- 95.0%
149+
- 83.9%
150+
-
151+
- 84.9%
159152
* - Llama-2-7B-chat
160153
- Data Default WWB
161154
- Similarity
162155
- 99.3%
163156
- 93.4%
164157
- 99.8%
165-
- 91.9%
158+
- 93.4%
166159
* - Llama-3-8B
167160
- Data Default WWB
168161
- Similarity
169162
- 98.8%
170163
- 94.3%
171-
- %
164+
- 99.7%
172165
- 94.5%
173166
* - Llama-3.2-3b-instruct
174167
- Data Default WWB
175168
- Similarity
176-
- 98.2%
177-
- 93.2%
178-
- 98.4%
179-
- 94.0%
180-
* - Mistral-7b-instruct-V0.3
181-
- Data Default WWB
182-
- Similarity
183-
- 98.3%
184-
- 92.8%
185-
- 99.9%
186-
- 93.6%
169+
- 97.9%
170+
- 94.2%
171+
- 99.7%
172+
- 94.1%
187173
* - Phi4-mini-instruct
188174
- Data Default WWB
189175
- Similarity
190-
- 96.4%
191-
- 92.0%
192-
- 99.3%
193-
- 91.7%
176+
- 89.1%
177+
- 92.1%
178+
- 99.5%
179+
- 92.4%
194180
* - Qwen2-VL-7B
195181
- Data Default WWB
196182
- Similarity
197-
- 97.8%
198-
- 92.4%
183+
- 97.5%
184+
- 88.1%
199185
- 99.8%
186+
- 91.4%
187+
* - Qwen3-8B
188+
- Data Default WWB
189+
- Similarity
190+
- 97.8%
191+
- 92.3%
192+
-
200193
- 93.0%
201194
* - Flux.1-schnell
202195
- Data Default WWB
@@ -208,10 +201,10 @@ the table for more information.
208201
* - Stable-Diffusion-V1-5
209202
- Data Default WWB
210203
- Similarity
211-
- 97.3%
212-
- 95.1%
204+
- 96.3%
205+
- 93.3%
213206
- 99.5%
214-
- 91.5%
207+
- 93.7%
215208

216209
Notes: For all accuracy metrics a "-", (minus sign), indicates an accuracy drop.
217210
The Similarity metric is the distance from "perfect" and as such always positive.

docs/articles_en/about-openvino/performance-benchmarks/performance-benchmarks-faq.rst

Lines changed: 1 addition & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -55,11 +55,7 @@ Performance Information F.A.Q.
5555
- DeepSeek, HF
5656
- Auto regressive language
5757
- 128K
58-
* - `GLM4-9B-chat <https://huggingface.co/THUDM/glm-4-9b-chat/tree/main>`__
59-
- THUDM
60-
- Transformer
61-
- 128K
62-
* - `Gemma-3-1B-it <https://huggingface.co/google/gemma-3-1b-it>`__
58+
* - `Gemma-3-4B-it <https://huggingface.co/google/gemma-3-4b-it>`__
6359
- Hugginface
6460
- Text-To-Text Decoder-only
6561
- 128K
@@ -75,14 +71,6 @@ Performance Information F.A.Q.
7571
- Meta AI
7672
- Auto regressive language
7773
- 128K
78-
* - `Mistral-7b-Instruct-V0.3 <https://huggingface.co/mistralai/Mistral-7B-v0.3>`__
79-
- Mistral AI
80-
- Auto regressive language
81-
- 32K
82-
* - `Phi3-4k-mini-Instruct <https://huggingface.co/microsoft/Phi-3-mini-4k-instruct>`__
83-
- Huggingface
84-
- Auto regressive language
85-
- 4096
8674
* - `Phi4-mini-Instruct <https://huggingface.co/microsoft/Phi-4-mini-instruct>`__
8775
- Huggingface
8876
- Auto regressive language
Binary file not shown.
Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
version https://git-lfs.github.com/spec/v1
2-
oid sha256:a2a90960620f45e00c60adba9c29d6c7fd75219d4a3bddc103ebb5001f1c1e35
3-
size 74279
2+
oid sha256:224d139c3c4267d4d6459a61580d17fe9bd1aa39453bd83eafc512a9af2650d3
3+
size 61867

0 commit comments

Comments
 (0)