latest-aio-gpu-intel-f32 problems in kubernetes #4428

urbaman · 2024-12-18T12:01:11Z

urbaman
Dec 18, 2024

Hi,

I'm trying to deploy in kubernetes, but it seems not to be working as expected.
Here's the image I'm using in the helm chart:

deployment:
  image:
    repository: quay.io/go-skynet/local-ai
    tag: latest-aio-gpu-intel-f32 #latest-gpu-intel-f32    # Use the appropriate tag. E.g. for cpu only: "latest-cpu"
  env:
    threads: 4
    context_size: 512

And here's the pod's logs:

kubectl logs -n local-ai local-ai-6756d5c4d8-fhvq4
Defaulted container "local-ai" out of: local-ai, k8tz (init)
===> LocalAI All-in-One (AIO) container starting...
Intel GPU detected
===> Starting LocalAI[intel] with the following models: /aio/intel/embeddings.yaml,/aio/intel/rerank.yaml,/aio/intel/text-to-speech.yaml,/aio/intel/image-gen.yaml,/aio/intel/text-to-text.yaml,/aio/intel/speech-to-text.yaml,/aio/intel/vision.yaml
@@@@@
Skipping rebuild
@@@@@
If you are experiencing issues with the pre-compiled builds, try setting REBUILD=true
If you are still experiencing issues with the build, try setting CMAKE_ARGS and disable the instructions set as needed:
CMAKE_ARGS="-DGGML_F16C=OFF -DGGML_AVX512=OFF -DGGML_AVX2=OFF -DGGML_FMA=OFF"
see the documentation at: https://localai.io/basics/build/index.html
Note: See also https://github.com/go-skynet/LocalAI/issues/288
@@@@@
CPU info:
model name      : QEMU Virtual CPU version 2.5+
flags           : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc nopl xtopology cpuid tsc_known_freq pni ssse3 cx16 sse4_1 sse4_2 x2apic popcnt aes hypervisor lahf_lm cpuid_fault pti
CPU: no AVX    found
CPU: no AVX2   found
CPU: no AVX512 found
@@@@@
1:01PM INF env file found, loading environment variables from file envFile=.env
1:01PM INF Setting logging to info
1:01PM INF Starting LocalAI using 4 threads, with models path: /models
1:01PM INF LocalAI version: v2.24.1 (184fbc26bf11599a11da8de0df14a885f1c7d2c0)
1:01PM INF Preloading models from /models
1:01PM INF Downloading "https://huggingface.co/cjpais/llava-1.6-mistral-7b-gguf/resolve/main/llava-v1.6-mistral-7b.Q5_K_M.gguf"
1:01PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 170.4 MiB/4.8 GiB (1.74%) ETA: 4m42.235360783s
1:01PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 376.1 MiB/4.8 GiB (3.84%) ETA: 4m10.270804123s
1:01PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 575.7 MiB/4.8 GiB (5.88%) ETA: 4m0.049067805s
1:01PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 774.4 MiB/4.8 GiB (7.91%) ETA: 3m52.781766375s
1:01PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 970.5 MiB/4.8 GiB (9.92%) ETA: 3m47.144919565s
1:01PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 1.1 GiB/4.8 GiB (12.01%) ETA: 3m39.888406255s
1:02PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 1.3 GiB/4.8 GiB (13.89%) ETA: 3m37.077534738s
1:02PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 1.5 GiB/4.8 GiB (15.98%) ETA: 3m30.378858463s
1:02PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 1.7 GiB/4.8 GiB (18.04%) ETA: 3m24.480039938s
1:02PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 1.9 GiB/4.8 GiB (20.15%) ETA: 3m18.098535431s
1:02PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 2.1 GiB/4.8 GiB (22.21%) ETA: 3m12.676647663s
1:02PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 2.3 GiB/4.8 GiB (24.26%) ETA: 3m7.384738989s
1:02PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 2.5 GiB/4.8 GiB (26.03%) ETA: 3m4.710677618s
1:02PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 2.7 GiB/4.8 GiB (28.13%) ETA: 2m58.85519586s
1:02PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 2.9 GiB/4.8 GiB (30.24%) ETA: 2m52.990820714s
1:02PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 3.1 GiB/4.8 GiB (32.35%) ETA: 2m47.280659981s
1:02PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 3.3 GiB/4.8 GiB (34.45%) ETA: 2m41.733806176s
1:02PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 3.5 GiB/4.8 GiB (36.50%) ETA: 2m36.56473882s
1:03PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 3.7 GiB/4.8 GiB (38.40%) ETA: 2m32.414384382s
1:03PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 3.9 GiB/4.8 GiB (40.49%) ETA: 2m26.975837304s
1:03PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 4.1 GiB/4.8 GiB (42.56%) ETA: 2m21.751377296s
1:03PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 4.3 GiB/4.8 GiB (44.50%) ETA: 2m17.235909828s
1:03PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 4.4 GiB/4.8 GiB (46.55%) ETA: 2m12.071672464s
1:03PM INF Downloading /models/llava-v1.6-mistral-7b.Q5_K_M.gguf.partial: 4.7 GiB/4.8 GiB (48.65%) ETA: 2m6.651854575s
1:03PM INF File "/models/llava-v1.6-mistral-7b.Q5_K_M.gguf" downloaded and verified
1:03PM INF Downloading "https://huggingface.co/cjpais/llava-1.6-mistral-7b-gguf/resolve/main/mmproj-model-f16.gguf"
1:03PM INF Downloading /models/llava-v1.6-7b-mmproj-f16.gguf.partial: 57.0 MiB/595.5 MiB (4.78%) ETA: 41m28.925847998s
1:03PM INF Downloading /models/llava-v1.6-7b-mmproj-f16.gguf.partial: 256.9 MiB/595.5 MiB (21.57%) ETA: 7m52.730764046s
1:03PM INF Downloading /models/llava-v1.6-7b-mmproj-f16.gguf.partial: 456.6 MiB/595.5 MiB (38.34%) ETA: 3m37.138806417s
1:03PM INF File "/models/llava-v1.6-7b-mmproj-f16.gguf" downloaded and verified

  Model name: gpt-4o



  curl http://localhost:8080/v1/chat/completions -H "Content-Type:
  application/json" -d '{ "model": "gpt-4-vision-preview", "messages": [{"role":
  "user", "content": [{"type":"text", "text": "What is in the image?"},
  {"type": "image_url", "image_url": {"url":
  "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-
  madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-
  boardwalk.jpg" }}], "temperature": 0.9}]}'



  Model name: jina-reranker-v1-base-en



  You can test this model with curl like this:

  curl http://localhost:8080/v1/rerank  -H "Content-Type: application/json"  -d
  '{ "model": "jina-reranker-v1-base-en", "query": "Organic skincare products for
1:03PM INF Downloading "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.bin"
  sensitive skin", "documents": [ "Eco-friendly kitchenware for modern homes",
  "Biodegradable cleaning supplies for eco-conscious consumers", "Organic
  cotton baby clothes for sensitive skin", "Natural organic skincare range for
  sensitive skin", "Tech gadgets for smart homes: 2024 edition", "Sustainable
  gardening tools and compost solutions", "Sensitive skin-friendly facial
  cleansers and toners", "Organic food wraps and storage solutions", "All-
  natural pet food for dogs with allergies", "Yoga mats made from recycled
  materials" ], "top_n": 3 }'


1:03PM INF Downloading /models/ggml-whisper-base.bin.partial: 103.3 MiB/141.1 MiB (73.21%) ETA: 51.239242391s
1:03PM INF File "/models/ggml-whisper-base.bin" downloaded and verified

  Model name: whisper-1



  ## example audio file

  wget --quiet --show-progress -O gb1.ogg
  https://upload.wikimedia.org/wikipedia/commons/1/1f/George_W_Bush_Columbia_FINAL.ogg

  ## Send the example audio file to the transcriptions endpoint

  curl http://localhost:8080/v1/audio/transcriptions  -H "Content-Type:
  multipart/form-data"  -F file="@$PWD/gb1.ogg" -F model="whisper-1"



  Model name: stablediffusion



  curl http://localhost:8080/v1/images/generations  -H "Content-Type:
  application/json"  -d '{ "prompt": "|", "step": 25, "size": "512x512" }'


1:03PM INF Downloading "https://github.com/rhasspy/piper/releases/download/v0.0.2/voice-en-us-amy-low.tar.gz"
1:03PM INF File "/models/voice-en-us-amy-low.tar.gz" downloaded and verified
1:03PM INF File "/models/voice-en-us-amy-low.tar.gz" is an archive, uncompressing to /models

  Model name: tts-1



  To test if this model works as expected, you can use the following curl
  command:

  curl http://localhost:8080/tts -H "Content-Type: application/json" -d '{
  "model":"tts-1", "input": "Hi, this is a test." }'



  Model name: text-embedding-ada-002



  You can test this model with curl like this:

  curl http://localhost:8080/embeddings -X POST -H "Content-Type:
  application/json" -d '{ "input": "Your text string goes here", "model": "text-
  embedding-ada-002" }'


1:03PM INF Downloading "https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf"
1:03PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 72.0 MiB/4.6 GiB (1.53%) ETA: 2h35m8.162309045s
1:03PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 278.4 MiB/4.6 GiB (5.93%) ETA: 39m39.16461259s
1:04PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 463.2 MiB/4.6 GiB (9.87%) ETA: 23m35.536831895s
1:04PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 671.0 MiB/4.6 GiB (14.30%) ETA: 15m59.262796342s
1:04PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 878.2 MiB/4.6 GiB (18.71%) ETA: 11m56.875060012s
1:04PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 1.1 GiB/4.6 GiB (23.03%) ETA: 9m28.307253359s
1:04PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 1.2 GiB/4.6 GiB (26.89%) ETA: 7m55.850836502s
1:04PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 1.4 GiB/4.6 GiB (31.21%) ETA: 6m36.892958363s
1:04PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 1.6 GiB/4.6 GiB (35.61%) ETA: 5m34.62427485s
1:04PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 1.8 GiB/4.6 GiB (40.02%) ETA: 4m44.849001102s
1:04PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 2.0 GiB/4.6 GiB (44.27%) ETA: 4m5.531515219s
1:04PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 2.2 GiB/4.6 GiB (48.68%) ETA: 3m30.900041934s
1:04PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 2.4 GiB/4.6 GiB (53.07%) ETA: 3m1.322543137s
1:04PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 2.6 GiB/4.6 GiB (57.37%) ETA: 2m36.099182598s
1:05PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 2.8 GiB/4.6 GiB (61.16%) ETA: 2m16.736728018s
1:05PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 3.0 GiB/4.6 GiB (65.49%) ETA: 1m56.090627128s
1:05PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 3.2 GiB/4.6 GiB (69.87%) ETA: 1m37.188431787s
1:05PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 3.4 GiB/4.6 GiB (74.14%) ETA: 1m20.325134371s
1:05PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 3.6 GiB/4.6 GiB (78.55%) ETA: 1m4.267363659s
1:05PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 3.8 GiB/4.6 GiB (82.57%) ETA: 50.748672257s
1:05PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 4.0 GiB/4.6 GiB (86.87%) ETA: 37.082112108s
1:05PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 4.2 GiB/4.6 GiB (91.28%) ETA: 23.914920036s
1:05PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 4.4 GiB/4.6 GiB (95.66%) ETA: 11.571267212s
1:05PM INF Downloading /models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf.partial: 4.6 GiB/4.6 GiB (99.93%) ETA: 191.829436ms
1:05PM INF File "/models/Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf" downloaded and verified

  Model name: gpt-4


1:05PM INF core/startup process completed!
1:05PM INF LocalAI API is listening! Please connect to the endpoint for API documentation. endpoint=http://0.0.0.0:8080
1:07PM INF Success ip=10.1.238.140 latency=3.411591ms method=GET status=200 url=/
1:07PM INF Success ip=10.1.238.140 latency="61.964µs" method=GET status=200 url=/static/assets/highlightjs.js
1:07PM INF Success ip=10.1.238.140 latency="67.814µs" method=GET status=200 url=/static/assets/highlightjs.css
1:07PM INF Success ip=10.1.238.140 latency="84.23µs" method=GET status=200 url=/static/general.css
1:07PM INF Success ip=10.1.238.140 latency="25.4µs" method=GET status=200 url=/static/assets/font2.css
1:07PM INF Success ip=10.1.238.140 latency="22.35µs" method=GET status=200 url=/static/assets/font1.css
1:07PM INF Success ip=10.1.238.140 latency="22.062µs" method=GET status=200 url=/static/assets/tw-elements.css
1:07PM INF Success ip=10.1.238.140 latency="14.559µs" method=GET status=200 url=/static/assets/fontawesome/css/fontawesome.css
1:07PM INF Success ip=10.1.238.140 latency="14.635µs" method=GET status=200 url=/static/assets/fontawesome/css/brands.css
1:07PM INF Success ip=10.1.238.140 latency="25.161µs" method=GET status=200 url=/static/assets/fontawesome/css/solid.css
1:07PM INF Success ip=10.1.238.140 latency="31.651µs" method=GET status=200 url=/static/assets/tailwindcss.js
1:07PM INF Success ip=10.1.238.140 latency="10.223µs" method=GET status=200 url=/static/assets/flowbite.min.js
1:07PM INF Success ip=10.1.238.140 latency="11.087µs" method=GET status=200 url=/static/assets/htmx.js
1:07PM INF Success ip=10.1.238.140 latency="24.006µs" method=GET status=200 url=/static/assets/tw-elements.js
1:07PM INF Success ip=10.1.238.140 latency="27.804µs" method=GET status=200 url=/static/assets/alpine.js
1:07PM INF Success ip=10.1.238.140 latency="22.772µs" method=GET status=200 url=/static/assets/marked.js
1:07PM INF Success ip=10.1.238.140 latency="36.323µs" method=GET status=200 url=/static/assets/purify.js
1:07PM INF Success ip=10.1.238.140 latency=7.246368ms method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuFuYMZg.ttf
1:07PM INF Success ip=10.1.238.140 latency="22.771µs" method=GET status=200 url=/static/assets/fontawesome/webfonts/fa-solid-900.woff2
1:07PM INF Success ip=10.1.238.140 latency="24.578µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuLyfMZg.ttf
1:07PM INF Success ip=10.1.238.140 latency="21.344µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuGKYMZg.ttf
1:07PM INF Success ip=127.0.0.1 latency="304.28µs" method=GET status=200 url=/
1:07PM INF Success ip=127.0.0.1 latency="30.897µs" method=GET status=200 url=/static/assets/highlightjs.css
1:07PM INF Success ip=127.0.0.1 latency="56.202µs" method=GET status=200 url=/static/assets/highlightjs.js
1:07PM INF Success ip=127.0.0.1 latency="41.04µs" method=GET status=200 url=/static/assets/fontawesome/css/fontawesome.css
1:07PM INF Success ip=127.0.0.1 latency="33.408µs" method=GET status=200 url=/static/assets/fontawesome/css/brands.css
1:07PM INF Success ip=127.0.0.1 latency="29.751µs" method=GET status=200 url=/static/assets/fontawesome/css/solid.css
1:07PM INF Success ip=127.0.0.1 latency="47.215µs" method=GET status=200 url=/static/assets/font2.css
1:07PM INF Success ip=127.0.0.1 latency="30.261µs" method=GET status=200 url=/static/assets/tw-elements.css
1:07PM INF Success ip=127.0.0.1 latency="40.649µs" method=GET status=200 url=/static/assets/font1.css
1:07PM INF Success ip=127.0.0.1 latency="70.685µs" method=GET status=200 url=/static/general.css
1:07PM INF Success ip=127.0.0.1 latency="19.797µs" method=GET status=200 url=/static/assets/tailwindcss.js
1:07PM INF Success ip=127.0.0.1 latency="22.013µs" method=GET status=200 url=/static/assets/flowbite.min.js
1:07PM INF Success ip=127.0.0.1 latency="26.861µs" method=GET status=200 url=/static/assets/htmx.js
1:07PM INF Success ip=127.0.0.1 latency="23.628µs" method=GET status=200 url=/static/assets/tw-elements.js
1:07PM INF Success ip=127.0.0.1 latency="23.409µs" method=GET status=200 url=/static/assets/alpine.js
1:07PM INF Success ip=127.0.0.1 latency="9.448µs" method=GET status=200 url=/static/assets/marked.js
1:07PM INF Success ip=127.0.0.1 latency="22.317µs" method=GET status=200 url=/static/assets/purify.js
1:07PM INF Success ip=127.0.0.1 latency="31.609µs" method=GET status=200 url=/static/assets/fontawesome/webfonts/fa-solid-900.woff2
1:07PM INF Success ip=127.0.0.1 latency="26.591µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuFuYMZg.ttf
1:07PM INF Success ip=127.0.0.1 latency="14.556µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuLyfMZg.ttf
1:07PM INF Success ip=127.0.0.1 latency="32.722µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuGKYMZg.ttf
1:07PM INF Success ip=127.0.0.1 latency="37.068µs" method=GET status=200 url=/favicon.ico
1:07PM INF Success ip=10.1.238.140 latency="297.056µs" method=GET status=200 url=/
1:07PM INF Success ip=10.1.238.140 latency="21.323µs" method=GET status=200 url=/static/assets/highlightjs.css
1:07PM INF Success ip=10.1.238.140 latency="15.502µs" method=GET status=200 url=/static/assets/font1.css
1:07PM INF Success ip=10.1.238.140 latency="13.596µs" method=GET status=200 url=/static/general.css
1:07PM INF Success ip=10.1.238.140 latency="19.577µs" method=GET status=200 url=/static/assets/font2.css
1:07PM INF Success ip=10.1.238.140 latency="6.85µs" method=GET status=200 url=/static/assets/fontawesome/css/fontawesome.css
1:07PM INF Success ip=10.1.238.140 latency="37.844µs" method=GET status=200 url=/static/assets/tw-elements.css
1:07PM INF Success ip=10.1.238.140 latency="5.443µs" method=GET status=200 url=/static/assets/highlightjs.js
1:07PM INF Success ip=10.1.238.140 latency="7.45µs" method=GET status=200 url=/static/assets/fontawesome/css/brands.css
1:07PM INF Success ip=10.1.238.140 latency="24.307µs" method=GET status=200 url=/static/assets/tailwindcss.js
1:07PM INF Success ip=10.1.238.140 latency="29.538µs" method=GET status=200 url=/static/assets/flowbite.min.js
1:07PM INF Success ip=10.1.238.140 latency="24.875µs" method=GET status=200 url=/static/assets/fontawesome/css/solid.css
1:07PM INF Success ip=10.1.238.140 latency="9.251µs" method=GET status=200 url=/static/assets/htmx.js
1:07PM INF Success ip=10.1.238.140 latency="19.594µs" method=GET status=200 url=/static/assets/tw-elements.js
1:07PM INF Success ip=10.1.238.140 latency="25.759µs" method=GET status=200 url=/static/assets/alpine.js
1:07PM INF Success ip=10.1.238.140 latency="14.068µs" method=GET status=200 url=/static/assets/purify.js
1:07PM INF Success ip=10.1.238.140 latency="16.362µs" method=GET status=200 url=/static/assets/marked.js
1:07PM INF Success ip=10.1.238.140 latency="27.515µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuFuYMZg.ttf
1:07PM INF Success ip=10.1.238.140 latency="24.719µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuLyfMZg.ttf
1:07PM INF Success ip=10.1.238.140 latency="28.306µs" method=GET status=200 url=/static/assets/fontawesome/webfonts/fa-solid-900.woff2
1:07PM INF Success ip=10.1.238.140 latency="33.458µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuGKYMZg.ttf
1:07PM INF Success ip=10.1.238.140 latency="17.669µs" method=GET status=200 url=/favicon.ico
1:07PM INF Success ip=10.1.238.140 latency="277.334µs" method=GET status=200 url=/chat/
1:07PM INF Success ip=10.1.238.140 latency="26.406µs" method=GET status=200 url=/static/assets/highlightjs.css
1:07PM INF Success ip=10.1.238.140 latency="27.28µs" method=GET status=200 url=/static/assets/fontawesome/css/fontawesome.css
1:07PM INF Success ip=10.1.238.140 latency="20.134µs" method=GET status=200 url=/static/assets/font1.css
1:07PM INF Success ip=10.1.238.140 latency="26.727µs" method=GET status=200 url=/static/assets/fontawesome/css/brands.css
1:07PM INF Success ip=10.1.238.140 latency="37.259µs" method=GET status=200 url=/static/assets/fontawesome/css/solid.css
1:07PM INF Success ip=10.1.238.140 latency="12.483µs" method=GET status=200 url=/static/assets/highlightjs.js
1:07PM INF Success ip=10.1.238.140 latency="12.957µs" method=GET status=200 url=/static/assets/tw-elements.css
1:07PM INF Success ip=10.1.238.140 latency="8.639µs" method=GET status=200 url=/static/assets/font2.css
1:07PM INF Success ip=10.1.238.140 latency="11.781µs" method=GET status=200 url=/static/general.css
1:07PM INF Success ip=10.1.238.140 latency="10.177µs" method=GET status=200 url=/static/assets/flowbite.min.js
1:07PM INF Success ip=10.1.238.140 latency="6.352µs" method=GET status=200 url=/static/assets/htmx.js
1:07PM INF Success ip=10.1.238.140 latency="14.501µs" method=GET status=200 url=/static/assets/tailwindcss.js
1:07PM INF Success ip=10.1.238.140 latency="67.954µs" method=GET status=200 url=/static/assets/alpine.js
1:07PM INF Success ip=10.1.238.140 latency="41.242µs" method=GET status=200 url=/static/assets/marked.js
1:07PM INF Success ip=10.1.238.140 latency="21.515µs" method=GET status=200 url=/static/assets/purify.js
1:07PM INF Success ip=10.1.238.140 latency="10.965µs" method=GET status=200 url=/static/chat.js
1:07PM INF Success ip=10.1.238.140 latency="22.625µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuFuYMZg.ttf
1:07PM INF Success ip=10.1.238.140 latency="14.824µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuLyfMZg.ttf
1:07PM INF Success ip=10.1.238.140 latency="9.545µs" method=GET status=200 url=/static/assets/fontawesome/webfonts/fa-solid-900.woff2
1:07PM INF Success ip=10.1.238.140 latency="26.621µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuGKYMZg.ttf
1:08PM INF Success ip=10.1.238.140 latency=9.212337ms method=POST status=200 url=/v1/chat/completions
1:08PM INF Loading model 'whisper-1' with backend whisper
1:08PM ERR failed starting/connecting to the gRPC service error="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:46453: connect: connection refused\""
4:31PM INF Success ip=10.1.238.140 latency="262.206µs" method=GET status=200 url=/
4:31PM INF Success ip=10.1.238.140 latency="26.323µs" method=GET status=200 url=/static/assets/highlightjs.css
4:31PM INF Success ip=10.1.238.140 latency="9.613µs" method=GET status=200 url=/static/assets/highlightjs.js
4:31PM INF Success ip=10.1.238.140 latency="9.848µs" method=GET status=200 url=/static/general.css
4:31PM INF Success ip=10.1.238.140 latency="14.663µs" method=GET status=200 url=/static/assets/font1.css
4:31PM INF Success ip=10.1.238.140 latency="29.175µs" method=GET status=200 url=/static/assets/font2.css
4:31PM INF Success ip=10.1.238.140 latency="22.012µs" method=GET status=200 url=/static/assets/tw-elements.css
4:31PM INF Success ip=10.1.238.140 latency="8.464µs" method=GET status=200 url=/static/assets/fontawesome/css/fontawesome.css
4:31PM INF Success ip=10.1.238.140 latency="24.078µs" method=GET status=200 url=/static/assets/fontawesome/css/brands.css
4:31PM INF Success ip=10.1.238.140 latency="14.641µs" method=GET status=200 url=/static/assets/tailwindcss.js
4:31PM INF Success ip=10.1.238.140 latency="7.675µs" method=GET status=200 url=/static/assets/fontawesome/css/solid.css
4:31PM INF Success ip=10.1.238.140 latency="17.546µs" method=GET status=200 url=/static/assets/flowbite.min.js
4:31PM INF Success ip=10.1.238.140 latency="28.127µs" method=GET status=200 url=/static/assets/htmx.js
4:31PM INF Success ip=10.1.238.140 latency="34.22µs" method=GET status=200 url=/static/assets/tw-elements.js
4:31PM INF Success ip=10.1.238.140 latency="23.292µs" method=GET status=200 url=/static/assets/alpine.js
4:31PM INF Success ip=10.1.238.140 latency="27.906µs" method=GET status=200 url=/static/assets/marked.js
4:31PM INF Success ip=10.1.238.140 latency="23.154µs" method=GET status=200 url=/static/assets/purify.js
4:31PM INF Success ip=10.1.238.140 latency="20.6µs" method=GET status=200 url=/static/assets/fontawesome/webfonts/fa-solid-900.woff2
4:31PM INF Success ip=10.1.238.140 latency="97.017µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuFuYMZg.ttf
4:31PM INF Success ip=10.1.238.140 latency="28.388µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuLyfMZg.ttf
4:31PM INF Success ip=10.1.238.140 latency="11.97µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuGKYMZg.ttf

See this part in particular:

1:08PM INF Success ip=10.1.238.140 latency=9.212337ms method=POST status=200 url=/v1/chat/completions
1:08PM INF Loading model 'whisper-1' with backend whisper
1:08PM ERR failed starting/connecting to the gRPC service error="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:46453: connect: connection refused\""

There's no way to make it work, even changing the model or something else.

urbaman · 2024-12-18T20:57:43Z

urbaman
Dec 18, 2024
Author

I solved setting up the cpu version of the image, so putting away the intel gpu.

Is there a way to use local-ai with intel GPU in kubernetes? Has anybody deployed it?

Thank you.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

latest-aio-gpu-intel-f32 problems in kubernetes #4428

Uh oh!

{{title}}

Uh oh!

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

latest-aio-gpu-intel-f32 problems in kubernetes #4428

Uh oh!

urbaman Dec 18, 2024

Replies: 1 comment

Uh oh!

urbaman Dec 18, 2024 Author

urbaman
Dec 18, 2024

urbaman
Dec 18, 2024
Author