CODEWITHSUNDEEP
large-language-modelllama-indexllamacpp
6core
6core

Reputation: 1

Persist VectorStoreIndex (LlamaIndex) locally

I am trying to run this

import logging
import sys
from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
import torch
from llama_index.llms import LlamaCPP
from llama_index.llms.llama_utils import messages_to_prompt, completion_to_prompt
from langchain.embeddings import HuggingFaceEmbeddings
from llama_index.embeddings import LangchainEmbedding
from llama_index import ServiceContext, set_global_service_context

logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

llm = LlamaCPP(
    model_url='https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF/resolve/main/mistral-7b-instruct-v0.1.Q4_K_M.gguf',
    model_path=None,
    temperature=0.1,
    max_new_tokens=256,
    context_window=3900,
    generate_kwargs={},
    model_kwargs={"n_gpu_layers": -1},
    messages_to_prompt=messages_to_prompt,
    completion_to_prompt=completion_to_prompt,
    verbose=True,
)

embed_model = LangchainEmbedding(
  HuggingFaceEmbeddings(model_name="thenlper/gte-large")
)

service_context = ServiceContext.from_defaults(
    chunk_size=256,
    llm=llm,
    embed_model=embed_model
)

documents = SimpleDirectoryReader("/content/Data/").load_data()

index = VectorStoreIndex.from_documents(documents, service_context=service_context)

Can someone help me persist the index locally so I don't have to redo this every time?

Tried doing this way, throwing an error -

from llama_index import StorageContext, load_index_from_storage
index.storage_context.persist("/content/pers")
storage_context2 = StorageContext.from_defaults(persist_dir="/content/pers")
new_index2 = load_index_from_storage(storage_context2)
new_query_engine2 = new_index2.as_query_engine()

ValueError Traceback (most recent call last) /usr/local/lib/python3.10/dist-packages/llama_index/llms/utils.py in resolve_llm(llm) 28 llm = OpenAI() ---> 29 validate_openai_api_key(llm.api_key) 30 except ValueError as e:

8 frames ValueError: No API key found for OpenAI. Please set either the OPENAI_API_KEY environment variable or openai.api_key prior to initialization. API keys can be found or created at https://platform.openai.com/account/api-keys

During handling of the above exception, another exception occurred:

ValueError Traceback (most recent call last) /usr/local/lib/python3.10/dist-packages/llama_index/llms/utils.py in resolve_llm(llm) 29 validate_openai_api_key(llm.api_key) 30 except ValueError as e: ---> 31 raise ValueError( 32 "\n******\n" 33 "Could not load OpenAI model. "

ValueError:

Could not load OpenAI model. If you intended to use OpenAI, please check your OPENAI_API_KEY. Original error: No API key found for OpenAI. Please set either the OPENAI_API_KEY environment variable or openai.api_key prior to initialization. API keys can be found or created at https://platform.openai.com/account/api-keys

To disable the LLM entirely, set llm=None.

I am new to this, any help will be much appreciated. Thanks in advance.

Upvotes: 0

Views: 1074

Answers (2)

林抿均
林抿均

Reputation: 43

This is how you persist VectorStoreIndex (LlamaIndex) locally:

# Imports
from llama_index.core import (
    load_index_from_storage,
    SimpleDirectoryReader,
    VectorStoreIndex,
    StorageContext,
    ServiceContext,
    Document,
    Settings,
)
from llama_index.core.tools import QueryEngineTool, ToolMetadata

# Load your files
lyft_docs = SimpleDirectoryReader(input_files=["./data/10k/lyft_2021.pdf"]).load_data()

# Create index
lyft_index = VectorStoreIndex.from_documents(lyft_docs)

# Persist your index locally
lyft_index.storage_context.persist(persist_dir="./storage/lyft")

Later, I load the index using the code below:

# Load the index from your storage location
storage_context = StorageContext.from_defaults(persist_dir="./storage/lyft")
lyft_index = load_index_from_storage(storage_context)

Upvotes: 0

Natalie A
Natalie A

Reputation: 3

To persist locally, one approach is setting up a local vector data store and using it to store the vectors. Then, in subsequent queries, you can just re-load the vector data store as the index instead of re-loading all the documents each time. There are a lot of vector databases out there (Pinecone, Weaviate, FAISSS, etc) - as an example, using Milvus vector store:

vector_store = MilvusVectorStore(address="localhost:19530",                                 
                             dim=1024)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(
                documents, embed_model=embeddings, storage_context=storage_context,
                show_progress=True)
query_engine = index.as_query_engine()

Then, once you've loaded the documents, when you want to query the Milvus database again, you can just use:

index=VectorStoreIndex.from_vector_store(vector_store)

Upvotes: 0

Related Questions