Skip to content

[Feature]: Add SeekDB as an optional derived-index backend #448

Description

@knqiufan

Area

architecture methods

Problem or opportunity

EverOS currently uses LanceDB by default and supports Milvus as an optional
derived-index backend. Users who want SeekDB's MySQL-compatible scalar,
full-text, and vector search need a first-class adapter that preserves the
same repository contracts and keeps Markdown as the source of truth.

The adapter should support both an in-process database for local deployments
and a remote seekdb Server or OceanBase endpoint, without adding dependencies
or behavior changes to the default LanceDB installation.

Proposed solution

Add an optional seekdb derived-index backend with:

  • configuration and dependency extras for embedded and remote modes;
  • implementations of all seven typed derived-index repositories;
  • schema creation and strict drift detection for scalar, JSON, full-text, and
    cosine HNSW vector indexes;
  • safe SQL rendering, exact datetime conversion, batched writes, native
    pagination, BM25 search, and vector search;
  • explicit embedded-directory ownership plus remote connection health checks
    and reconnect-on-disconnect behavior;
  • lifecycle and cascade rebuild integration through the existing backend
    ports;
  • user-facing configuration and architecture documentation; and
  • unit, contract, embedded integration, remote integration, and tiered API
    coverage in CI.

LanceDB remains the default. Switching backends rebuilds the derived data from
Markdown rather than migrating authoritative content.

Alternatives considered

  • Keep LanceDB only: this does not serve deployments standardizing on SeekDB
    or OceanBase.
  • Use Milvus: it covers remote vector storage but not SeekDB's embedded mode or
    its unified SQL/full-text/vector operational model.
  • Add an application-specific storage path outside the index ports: this would
    duplicate routing and leak backend details into memory services.

Additional context

The implementation targets the current pyseekdb 1.4.x API. Embedded support
is available on Linux and macOS; all platforms can use remote mode. The
physical database is treated as rebuildable derived state, consistent with
the existing LanceDB and Milvus backends.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions