littlechad
littlechad

Reputation: 1228

Retrieving typesense collection as llama_index VectorStoreIndex

I am using llama_index typesense vector store to build from document and then store it in typesense, and currently this following code is working as expected

...

def typesense(data, name):
    tc = Client(
        {
            "api_key": ...,
            "nodes": ...,
            "connection_timeout_seconds": 2,
        }
    )

    store = TypesenseVectorStore(client=tc, collection_name=name)
    context = StorageContext.from_defaults(vector_store=store)
    index = VectorStoreIndex.from_documents(
        documents=data, storage_context=context, show_progress=True
    )

    return index

But, unfortunately this function will insert new documents every time it runs, i am trying to figure out how to do a check before creating the index, by doing the following

# pseudo code

1  col = typesense retrieve collection
2
3  if col not exists:
4    index = create new VectorStoreIndex
5  else:
6    index = turn col into VectorStoreIndex

I think i manage to do up to line 4 (create new VectorStoreIndex), line 6 got me this error ValueError: No index in storage context, check if you specified the right persist_dir.

def typesense(data, key):
    tc = Client(
        {
            "api_key": ...,
            "nodes": ...,
            "connection_timeout_seconds": 2,
        }
    )

    col_name = key.split("/")
    col_name = col_name[len(col_name) - 1]

    print(col_name)

    store = TypesenseVectorStore(client=tc, collection_name=col_name)
    context = StorageContext.from_defaults(vector_store=store)

    try:
        x = tc.collections[col_name].retrieve()
    except:
        print("except")
        index = VectorStoreIndex.from_documents(
            documents=data, storage_context=context, show_progress=True
        )
    else:
        print(col_name)
        index = load_index_from_storage(storage_context=context)

    return index

From the error message, i think load_index_from_storage is trying to load something from my local machine, what method should i use so that i can load the index from typesense?

Upvotes: 0

Views: 113

Answers (0)

Related Questions