docs: Visualization example with Qdrant

[Anush008]

Hey 👋

Description

This PR adds an example Jupyter Notebook demonstrating usage of Embedding Atlas and Qdrant.

The example:

• Sets up a Qdrant collection instantly using snapshot restore.
• Computes the neighbours and projections using vectors in the collection, without running inference.
• Visualizes it with Embedding Atlas

[fredhohman]

Hi @Anush008, thanks for your interest in Embedding Atlas, and for the new example notebook! Since the notebook mostly focuses on the Qdrant API and use case of wrangling data into a pd.df, this feels like it would be better located in an example folder over there.

[Anush008]

Hey @fredhohman 👋

This example also demonstrates building a UMAP with your own vectors and computing its nearest neighbours for viz.

[domoritz]

Embedding atlas has code for computing nearest neighbors already (called in

embedding-atlas/packages/backend/embedding_atlas/projection.py

Line 202 in 96d617c

def compute_text_projection(

). I'd still be happy to merge another example if

• It used existing dependencies (or UV for anything we really need)
• Add some testing infrastructure to make sure the notebooks continue to run successfully (maybe even show them in the docs pages like in Jupyter books).

[Anush008]

Hey @domoritz.

Seems to me that

embedding-atlas/packages/backend/embedding_atlas/projection.py

Line 202 in 96d617c

def compute_text_projection(

finds the nearest neighbours after generating new vector embeddings using a SentenceTransformer model.

Whereas this new example uses vectors that are already available in the Qdrant collection.

[domoritz]

Ah, thanks for pointing out the difference. I still think we should not add this example at the moment given the explanations above and the fact that we don't want multiple ways to do the same thing at the moment. Maybe I'm missing it but qdrant needs a server running and can't just be used as a python library, right? Or is the approach you propose here much much faster? If so, please add more details to the description of the pull request. The original description made it seem like this is just another way to use embedding atlas but with a proprietary tool.

[Anush008]

Hey.

Or is the approach you propose here much much faster

This PR demonstrates a bring your own vectors approach when using Embedding Atlas.
Throughout the project, I couldn't find such an example. Hence this specific contribution.

Also note that Qdrant is not proprietary. It is an open-source project licensed under Apache 2.0.

[domoritz]

#21 adds a feature to the cli to bring your own embedding. Maybe this addresses a part of the concern you had with not having an example for how to use your own embeddings. Having a dedicated example in Jupyter makes sense but I wouldn't conflate it with Qdrant which seems orthogonal.