Skip to content

docs: add Qdrant, Weaviate, RAGFlow, Dify, CocoIndex, Feast integration guides - #192

Merged
majinghe merged 2 commits into
rustfs:mainfrom
majinghe:docs/integration-ai-db-batch
Nov 3, 2026
Merged

majinghe merged 2 commits into
rustfs:mainfrom
majinghe:docs/integration-ai-db-batch

Conversation

@majinghe

@majinghe majinghe commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Six new verified AI/RAG storage integration guides, each in all five locales (en/zh/de/fr/ja) with screenshots (Chinese UI screenshots for zh, English for the other locales):

  • database/qdrant — CNCF-ecosystem vector database: S3-backed collection snapshots with fresh-node recovery round-trip
  • database/weaviate — vector database: backup-s3 module with backup → restore round-trip against a RustFS bucket
  • ai/ragflow — open-source RAG engine: MinIO-compatible storage config pointing at RustFS, per-tenant buckets, upload confirm + read-back
  • ai/dify — LLM application platform: OpenDAL S3 storage for uploads, knowledge-base files, and bootstrap system state
  • ai/cocoindex — incremental AI indexing engine: S3 source flow that embeds RustFS-stored documents into pgvector with semantic query
  • ai/feast — open-source feature store: S3 feature registry plus Parquet offline source with materialization and online reads

Every guide documents the exact pitfalls hit during verification:

  • Qdrant's settings live under QDRANT__STORAGE__SNAPSHOTS_CONFIG__* in released versions (older config keys are silently ignored)
  • Weaviate 1.31's backup-s3 reads AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY (not BACKUP_S3_ACCESS_KEY_ID) and a scheme-less BACKUP_S3_ENDPOINT
  • RAGFlow needs a network alias for the storage container (Compose bridge isolation), renders minio: config from .env, and requires RSA-encrypted passwords via its UI
  • Dify needs stack recreation after .env changes, bind-mount cleanup for true resets, and its Compose file bind-mounts under docker/volumes/
  • Feast's registry reads FEAST_S3_ENDPOINT_URL while the FileSource carries its own s3_endpoint_override; boto3/s3fs are not base dependencies

Chroma was evaluated and skipped: its object-storage backend is a managed-cloud feature, not present in the open-source server.

Validation

  • npm run docs:check: PASS (1000 pages, locale tree parity, code-block parity, banned strings, image sizes)
  • npm run build: succeeds (4025 static files generated)
  • All six integrations verified end-to-end on a live RustFS instance: write → confirm object in RustFS → read back (Qdrant/Weaviate/Dify/CocoIndex/Feast full round-trips; RAGFlow verified at the storage layer — parsing additionally requires an embedding model, independent of storage)

Summary

Six new verified AI/RAG storage integration guides, each in all five locales (en/zh/de/fr/ja) with screenshots (Chinese UI screenshots for zh, English for the other locales):

  • database/qdrant — CNCF-ecosystem vector database: S3-backed collection snapshots with fresh-node recovery round-trip
  • database/weaviate — vector database: backup-s3 module with backup → restore round-trip against a RustFS bucket
  • ai/ragflow — open-source RAG engine: MinIO-compatible storage config pointing at RustFS, per-tenant buckets, upload confirm + read-back
  • ai/dify — LLM application platform: OpenDAL S3 storage for uploads, knowledge-base files, and bootstrap system state
  • ai/cocoindex — incremental AI indexing engine: S3 source flow that embeds RustFS-stored documents into pgvector with semantic query
  • ai/feast — open-source feature store: S3 feature registry plus Parquet offline source with materialization and online reads

Every guide documents the exact pitfalls hit during verification:

  • Qdrant's settings live under QDRANT__STORAGE__SNAPSHOTS_CONFIG__* in released versions (older config keys are silently ignored)
  • Weaviate 1.31's backup-s3 reads AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY (not BACKUP_S3_ACCESS_KEY_ID) and a scheme-less BACKUP_S3_ENDPOINT
  • RAGFlow needs a network alias for the storage container (Compose bridge isolation), renders minio: config from .env, and requires RSA-encrypted passwords via its UI
  • Dify needs stack recreation after .env changes, bind-mount cleanup for true resets, and its Compose file bind-mounts under docker/volumes/
  • Feast's registry reads FEAST_S3_ENDPOINT_URL while the FileSource carries its own s3_endpoint_override; boto3/s3fs are not base dependencies

Chroma was evaluated and skipped: its object-storage backend is a managed-cloud feature, not present in the open-source server.

Copy fixes

Review follow-up on the same guides:

  • Weaviate (option A): the restore sample returns two titles, so each locale now shows a second POST /v1/objects for Weaviate on object storage. The second insert reuses the verified vector from the first object; the walkthrough recorded the title, not a separate vector.
  • Qdrant (en): replaced Bucket and prefix must exist considerations: ... with Create the bucket first if it does not exist; the prefix may be empty. zh/de/fr/ja already said this.
  • CocoIndex: kept ./.venv/bin/python main.py "what does RustFS expose" and extended main.py with the query entrypoint from the CocoIndex text-embedding example (same model, pgvector <=> cosine distance, same print format). load_dotenv() runs before the environment reads so that command works when .env is present. The code block is identical in all five locales.
  • French CocoIndex: embeddings wording instead of « embarquer »; délais d'attente de connexion instead of expiration de connexion.
  • Nits: German Überprüfen Sie die wiederhergestellten Daten, Demo-Umgebung entfernen, and in der Web-UI; Japanese パススタイルアドレッシング, 正本(source of truth), and 統合テスト; French Schéma d’architecture, point de terminaison, and valeurs de connexion indicatives; Chinese Feast 执行 feast apply.

Validation

  • npm run docs:check: PASS (1005 pages, locale tree parity, code-block parity, banned strings, image sizes)
  • npm run build: succeeds (4045 static files generated)
  • All six integrations verified end-to-end on a live RustFS instance: write → confirm object in RustFS → read back (Qdrant/Weaviate/Dify/CocoIndex/Feast full round-trips; RAGFlow verified at the storage layer — parsing additionally requires an embedding model, independent of storage)
Open in Web Open in Cursor 

@vercel

vercel Bot commented Nov 1, 2026

Copy link
Copy Markdown

@majinghe is attempting to deploy a commit to the overtrue's projects Team on Vercel.

A member of the Team first needs to authorize it.

…on guides

Six verified AI/RAG storage integrations across five locales (en/zh/de/fr/ja):
- database/qdrant: S3-backed collection snapshots with fresh-node recovery
- database/weaviate: backup-s3 module with backup/restore round-trip
- ai/ragflow: MinIO-compatible config pointing at RustFS, per-tenant buckets
- ai/dify: OpenDAL S3 storage for uploads, knowledge files, and system state
- ai/cocoindex: S3 source flow embedding RustFS documents into pgvector
- ai/feast: S3 registry plus Parquet offline source with materialization

Each guide verified end-to-end against RustFS and documents the pitfalls hit
during verification (Qdrant snapshots_config env path, Weaviate 1.31 AWS_*
credential variables and scheme-less endpoint, RAGFlow compose network alias
and RSA-encrypted passwords, Dify bind-mount resets and OpenDAL endpoint
routing, Feast's FEAST_S3_ENDPOINT_URL vs FileSource override split).
@majinghe
majinghe force-pushed the docs/integration-ai-db-batch branch from 2a3b3b1 to 1b5d3c5 Compare November 1, 2026 17:06
Add the second Weaviate object insert so the backup matches the restore
sample, give CocoIndex main.py the query entrypoint the documented
command runs, and correct the locale wording from review.

Co-authored-by: hector <majinghe@users.noreply.github.com>
@majinghe
majinghe merged commit 7005ce2 into rustfs:main Nov 3, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants