Search results

Support fall back across several providers

featureCategorizes issue or PR as related to a new feature.Categorizes issue or PR as related to a new feature.

important-longtermImportant over the long term, but may not be staffed and/or may need multiple releases to complete.Important over the long term, but may not be staffed and/or may need multiple releases to complete.

Status: Open.

#60 In InftyAI/llmlite;

· kerthcet opened on Jul 1, 2024

· 

v0.0.16

Support counting tokens

featureCategorizes issue or PR as related to a new feature.Categorizes issue or PR as related to a new feature.

Status: Open.

#59 In InftyAI/llmlite;

· kerthcet opened on Jul 1, 2024

Support deepspeed-fastgen as another backend

Status: Open.

#57 In InftyAI/llmlite;

· kerthcet opened on Jan 9, 2024

vLLM not working as expected with ChatGLM2

bugCategorizes issue or PR as related to a bug.Categorizes issue or PR as related to a bug.

Status: Open.

#55 In InftyAI/llmlite;

· kerthcet opened on Dec 27, 2023

Support Stream output

Status: Open.

#52 In InftyAI/llmlite;

· kerthcet opened on Dec 27, 2023

Support loading different adapter

Status: Open.

#51 In InftyAI/llmlite;

· kerthcet opened on Dec 19, 2023

Support serving via HTTP/RPC server

enhancementNew feature or requestNew feature or request

Status: Open.

#40 In InftyAI/llmlite;

· kerthcet opened on Nov 28, 2023

Support serving fine-tuning layers easily

enhancementNew feature or requestNew feature or request

Status: Open.

#39 In InftyAI/llmlite;

· kerthcet opened on Nov 28, 2023

Integration with langchain

enhancementNew feature or requestNew feature or request

Status: Open.

#38 In InftyAI/llmlite;

· kerthcet opened on Nov 28, 2023

Support quantization

enhancementNew feature or requestNew feature or request

Status: Open.

#37 In InftyAI/llmlite;

· kerthcet opened on Nov 28, 2023

support text-generation-inference

enhancementNew feature or requestNew feature or request

Status: Open.

#35 In InftyAI/llmlite;

· kerthcet opened on Nov 28, 2023

Add codeLlama example

documentationCategorizes issue or PR as related to documentation.Categorizes issue or PR as related to documentation.

good first issueGood for newcomersGood for newcomers

Status: Open.

#33 In InftyAI/llmlite;

· kerthcet opened on Nov 28, 2023