CODEWITHSUNDEEP
pythonartificial-intelligencelangchainvector-databasedeeplake
kano
kano

Reputation: 21

Delete Documents from Deeplake vector database

I have created a deeplake vector database where embeddings of several different files are stored. I would like to develop a functionality that deletes the documents based on their metadata and updates the vector database with it. Unfortunately I can't find much documentation on this method. Therefore I wanted to ask the community for suggestions.

Here is the code snippet in which I load my database:

embeddings = OpenAIEmbeddings(disallowed_special=())
db = DeepLake(dataset_path='./vectordbs/deeplake/deeplake_default', embedding=embeddings)

Upvotes: 0

Views: 509

Answers (1)

Adilkhan Sarsen
Adilkhan Sarsen

Reputation: 16

I didn't get the update part, could you please elaborate more on this? How would you like to update data based on your metadata?

For the deletion part it is pretty straightforward to do:

embeddings = OpenAIEmbeddings(disallowed_special=())
db = DeepLake(dataset_path='./vectordbs/deeplake/deeplake_default', embedding=embeddings)

db.delete(
    filter={'metadata': {'key': 'value'}}
)

if you want to have more customizability over what you want to delete, then you can leverage our tql which will select all necessary elements using SQL like syntax, to do the actual deletion you would need to run the following code:

db.delete(
    query=tql_query_str
)

Upvotes: 0

Related Questions