Skip to content

Commit

Permalink
fix inference client tests
Browse files Browse the repository at this point in the history
  • Loading branch information
@hanouticelina
hanouticelina committed Feb 11, 2025
1 parent 449342d commit bd8c8e4
Show file tree
Hide file tree
Showing 10 changed files with 51,643 additions and 85 deletions.
95 changes: 95 additions & 0 deletions ...ferenceClient.test_automatic_speech_recognition[fal-ai,automatic-speech-recognition].yaml
View file

Large diffs are not rendered by default.

256 changes: 256 additions & 0 deletions ...ssettes/TestInferenceClient.test_chat_completion_no_stream[sambanova,conversational].yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -151,4 +151,260 @@ interactions:
status:
code: 200
message: OK
- request:
body: null
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate
Connection:
- keep-alive
X-Amzn-Trace-Id:
- 861fe0f5-4b40-432b-8845-ebad7187842f
method: GET
uri: https://huggingface.co/api/models/meta-llama/Llama-3.1-8B-Instruct?expand=inferenceProviderMapping
response:
body:
string: '{"_id":"6698d8a0653e4babe21e1e7d","id":"meta-llama/Llama-3.1-8B-Instruct","inferenceProviderMapping":{"sambanova":{"status":"live","providerId":"Meta-Llama-3.1-8B-Instruct","task":"conversational"}}}'
headers:
Access-Control-Allow-Origin:
- https://huggingface.co
Access-Control-Expose-Headers:
- X-Repo-Commit,X-Request-Id,X-Error-Code,X-Error-Message,X-Total-Count,ETag,Link,Accept-Ranges,Content-Range,X-Xet-Access-Token,X-Xet-Token-Expiration,X-Xet-Refresh-Route,X-Xet-Cas-Url,X-Xet-Hash
Connection:
- keep-alive
Content-Length:
- '199'
Content-Type:
- application/json; charset=utf-8
Date:
- Tue, 11 Feb 2025 10:13:49 GMT
ETag:
- W/"c7-TZgW++vzRWhy4flOcxqeqWyrc+g"
Referrer-Policy:
- strict-origin-when-cross-origin
Vary:
- Origin
Via:
- 1.1 e47c282d2c53705a367f9e376a2eab28.cloudfront.net (CloudFront)
X-Amz-Cf-Id:
- KHBtO6BXmMOxnzrXJRREr_jCrEid_MRcRmy79pAMVD2J0XZkjaEWCQ==
X-Amz-Cf-Pop:
- CDG52-P4
X-Cache:
- Miss from cloudfront
X-Powered-By:
- huggingface-moon
X-Request-Id:
- Root=1-67ab22dd-227de3605c0bed333116b5d0;861fe0f5-4b40-432b-8845-ebad7187842f
cross-origin-opener-policy:
- same-origin
status:
code: 200
message: OK
- request:
body: '{"messages": [{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is deep learning?"}], "model": "Meta-Llama-3.1-8B-Instruct",
"stream": false}'
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate
Connection:
- keep-alive
Content-Length:
- '188'
Content-Type:
- application/json
X-Amzn-Trace-Id:
- 15a16020-b6c5-4530-b348-9c27dbcef3ce
method: POST
uri: https://api.sambanova.ai/v1/chat/completions
response:
body:
string: '{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"Deep
learning is a subset of machine learning that involves the use of artificial
neural networks with multiple layers to analyze and interpret data. These
neural networks are inspired by the structure and function of the human brain,
with each layer processing the input data in a hierarchical manner.\n\nThe
key characteristics of deep learning are:\n\n1. **Multiple layers**: Deep
learning models consist of multiple layers of interconnected nodes or \"neurons,\"
which process the input data in a hierarchical manner.\n2. **Non-linear transformations**:
Each layer applies a non-linear transformation to the input data, allowing
the model to learn complex relationships between the input features.\n3. **Large
amounts of data**: Deep learning models require large amounts of data to train,
as they need to learn the patterns and relationships in the data.\n4. **Computational
power**: Deep learning models require significant computational power to train,
as they involve complex matrix operations and optimization algorithms.\n\nDeep
learning has been successful in a wide range of applications, including:\n\n1.
**Computer vision**: Image recognition, object detection, segmentation, and
generation.\n2. **Natural language processing**: Text classification, sentiment
analysis, language translation, and text generation.\n3. **Speech recognition**:
Speech-to-text systems and voice assistants.\n4. **Robotics**: Control and
navigation of robots.\n5. **Healthcare**: Disease diagnosis, patient outcome
prediction, and personalized medicine.\n\nSome of the popular deep learning
techniques include:\n\n1. **Convolutional Neural Networks (CNNs)**: Used for
image and video analysis.\n2. **Recurrent Neural Networks (RNNs)**: Used for
sequential data, such as speech and text.\n3. **Long Short-Term Memory (LSTM)
networks**: A type of RNN that can learn long-term dependencies.\n4. **Generative
Adversarial Networks (GANs)**: Used for generating new data samples that resemble
existing data.\n\nOverall, deep learning has revolutionized the field of machine
learning and has enabled the development of sophisticated AI systems that
can learn and adapt to complex data.","role":"assistant"}}],"created":1739268830,"id":"d9de7139-0715-4275-8126-20aa43711c50","model":"Meta-Llama-3.1-8B-Instruct","object":"chat.completion","system_fingerprint":"fastcoe","usage":{"completion_tokens":409,"completion_tokens_after_first_per_sec":968.7091369463583,"completion_tokens_after_first_per_sec_first_ten":1019.1976283624523,"completion_tokens_per_sec":909.4776237067355,"end_time":1739268830.1627614,"is_last_response":true,"prompt_tokens":46,"start_time":1739268829.7130527,"stop_reason":"stop","time_to_first_token":0.028529644012451172,"total_latency":0.4497087001800537,"total_tokens":455,"total_tokens_per_sec":1011.7660606028475}}
'
headers:
Access-Control-Allow-Credentials:
- 'true'
Access-Control-Allow-Headers:
- DNT,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Authorization
Access-Control-Allow-Methods:
- GET, PUT, POST, DELETE, PATCH, OPTIONS
Access-Control-Allow-Origin:
- '*'
Access-Control-Max-Age:
- '1728000'
Connection:
- keep-alive
Content-Type:
- application/json; charset=utf-8
Date:
- Tue, 11 Feb 2025 10:13:50 GMT
Strict-Transport-Security:
- max-age=31536000; includeSubDomains
Transfer-Encoding:
- chunked
status:
code: 200
message: OK
- request:
body: null
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate
Connection:
- keep-alive
X-Amzn-Trace-Id:
- 19fd9fcc-1495-4c0f-8c1b-b9142d3585ff
method: GET
uri: https://huggingface.co/api/models/meta-llama/Llama-3.1-8B-Instruct?expand=inferenceProviderMapping
response:
body:
string: '{"_id":"6698d8a0653e4babe21e1e7d","id":"meta-llama/Llama-3.1-8B-Instruct","inferenceProviderMapping":{"sambanova":{"status":"live","providerId":"Meta-Llama-3.1-8B-Instruct","task":"conversational"}}}'
headers:
Access-Control-Allow-Origin:
- https://huggingface.co
Access-Control-Expose-Headers:
- X-Repo-Commit,X-Request-Id,X-Error-Code,X-Error-Message,X-Total-Count,ETag,Link,Accept-Ranges,Content-Range,X-Xet-Access-Token,X-Xet-Token-Expiration,X-Xet-Refresh-Route,X-Xet-Cas-Url,X-Xet-Hash
Connection:
- keep-alive
Content-Length:
- '199'
Content-Type:
- application/json; charset=utf-8
Date:
- Tue, 11 Feb 2025 10:15:16 GMT
ETag:
- W/"c7-TZgW++vzRWhy4flOcxqeqWyrc+g"
Referrer-Policy:
- strict-origin-when-cross-origin
Vary:
- Origin
Via:
- 1.1 4587dd93b6f56d2b3f35f25ef2cabe70.cloudfront.net (CloudFront)
X-Amz-Cf-Id:
- Zddt4GOQt5pfgHwZLBowNtSQh5QzuMmpDHanYG9QcKK7elNzrs2jgQ==
X-Amz-Cf-Pop:
- CDG52-P4
X-Cache:
- Miss from cloudfront
X-Powered-By:
- huggingface-moon
X-Request-Id:
- Root=1-67ab2334-2463bc050e153a831c340cdb;19fd9fcc-1495-4c0f-8c1b-b9142d3585ff
cross-origin-opener-policy:
- same-origin
status:
code: 200
message: OK
- request:
body: '{"messages": [{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is deep learning?"}], "model": "Meta-Llama-3.1-8B-Instruct",
"stream": false}'
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate
Connection:
- keep-alive
Content-Length:
- '188'
Content-Type:
- application/json
X-Amzn-Trace-Id:
- 5b4ed482-50c7-4f8c-bd26-6142881c54dd
method: POST
uri: https://api.sambanova.ai/v1/chat/completions
response:
body:
string: '{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"Deep
learning is a subset of machine learning that involves the use of artificial
neural networks with multiple layers to analyze and interpret data. These
neural networks are inspired by the structure and function of the human brain,
with each layer processing the input data in a hierarchical manner.\n\nThe
key characteristics of deep learning are:\n\n1. **Multiple layers**: Deep
learning models consist of multiple layers of interconnected nodes or \"neurons,\"
which process the input data in a hierarchical manner.\n2. **Non-linear transformations**:
Each layer applies a non-linear transformation to the input data, allowing
the model to learn complex relationships between the input features.\n3. **Large
amounts of data**: Deep learning models require large amounts of data to train,
as they need to learn the patterns and relationships in the data.\n4. **Computational
power**: Deep learning models require significant computational power to train,
as they involve complex matrix operations and optimization algorithms.\n\nDeep
learning has been successful in a wide range of applications, including:\n\n1.
**Computer vision**: Image recognition, object detection, segmentation, and
generation.\n2. **Natural language processing**: Text classification, sentiment
analysis, language translation, and text generation.\n3. **Speech recognition**:
Speech-to-text systems and voice assistants.\n4. **Robotics**: Control and
navigation of robots.\n5. **Healthcare**: Disease diagnosis, patient outcome
prediction, and personalized medicine.\n\nSome of the popular deep learning
techniques include:\n\n1. **Convolutional Neural Networks (CNNs)**: Used for
image and video analysis.\n2. **Recurrent Neural Networks (RNNs)**: Used for
sequential data, such as speech and text.\n3. **Long Short-Term Memory (LSTM)
networks**: A type of RNN that can learn long-term dependencies.\n4. **Generative
Adversarial Networks (GANs)**: Used for generating new data samples that resemble
existing data.\n\nOverall, deep learning has revolutionized the field of machine
learning and has enabled the development of sophisticated AI systems that
can learn and adapt to complex data.","role":"assistant"}}],"created":1739268917,"id":"108c6d9a-9a8a-47ae-89e5-ab8d2fdd1cfa","model":"Meta-Llama-3.1-8B-Instruct","object":"chat.completion","system_fingerprint":"fastcoe","usage":{"completion_tokens":409,"completion_tokens_after_first_per_sec":1085.097647474337,"completion_tokens_after_first_per_sec_first_ten":1141.4159171621786,"completion_tokens_per_sec":1017.4866819929916,"end_time":1739268917.083342,"is_last_response":true,"prompt_tokens":46,"start_time":1739268916.6813712,"stop_reason":"stop","time_to_first_token":0.025967836380004883,"total_latency":0.40197086334228516,"total_tokens":455,"total_tokens_per_sec":1131.9228369359687}}
'
headers:
Access-Control-Allow-Credentials:
- 'true'
Access-Control-Allow-Headers:
- DNT,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Authorization
Access-Control-Allow-Methods:
- GET, PUT, POST, DELETE, PATCH, OPTIONS
Access-Control-Allow-Origin:
- '*'
Access-Control-Max-Age:
- '1728000'
Connection:
- keep-alive
Content-Type:
- application/json; charset=utf-8
Date:
- Tue, 11 Feb 2025 10:15:17 GMT
Strict-Transport-Security:
- max-age=31536000; includeSubDomains
Transfer-Encoding:
- chunked
status:
code: 200
message: OK
version: 1
Loading

0 comments on commit bd8c8e4

Please sign in to comment.