Reputation: 1
I am trying to run this
import logging
import sys
from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
import torch
from llama_index.llms import LlamaCPP
from llama_index.llms.llama_utils import messages_to_prompt, completion_to_prompt
from langchain.embeddings import HuggingFaceEmbeddings
from llama_index.embeddings import LangchainEmbedding
from llama_index import ServiceContext, set_global_service_context
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
llm = LlamaCPP(
model_url='https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF/resolve/main/mistral-7b-instruct-v0.1.Q4_K_M.gguf',
model_path=None,
temperature=0.1,
max_new_tokens=256,
context_window=3900,
generate_kwargs={},
model_kwargs={"n_gpu_layers": -1},
messages_to_prompt=messages_to_prompt,
completion_to_prompt=completion_to_prompt,
verbose=True,
)
embed_model = LangchainEmbedding(
HuggingFaceEmbeddings(model_name="thenlper/gte-large")
)
service_context = ServiceContext.from_defaults(
chunk_size=256,
llm=llm,
embed_model=embed_model
)
documents = SimpleDirectoryReader("/content/Data/").load_data()
index = VectorStoreIndex.from_documents(documents, service_context=service_context)
Can someone help me persist the index locally so I don't have to redo this every time?
Tried doing this way, throwing an error -
from llama_index import StorageContext, load_index_from_storage
index.storage_context.persist("/content/pers")
storage_context2 = StorageContext.from_defaults(persist_dir="/content/pers")
new_index2 = load_index_from_storage(storage_context2)
new_query_engine2 = new_index2.as_query_engine()
ValueError Traceback (most recent call last) /usr/local/lib/python3.10/dist-packages/llama_index/llms/utils.py in resolve_llm(llm) 28 llm = OpenAI() ---> 29 validate_openai_api_key(llm.api_key) 30 except ValueError as e:
8 frames ValueError: No API key found for OpenAI. Please set either the OPENAI_API_KEY environment variable or openai.api_key prior to initialization. API keys can be found or created at https://platform.openai.com/account/api-keys
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last) /usr/local/lib/python3.10/dist-packages/llama_index/llms/utils.py in resolve_llm(llm) 29 validate_openai_api_key(llm.api_key) 30 except ValueError as e: ---> 31 raise ValueError( 32 "\n******\n" 33 "Could not load OpenAI model. "
ValueError:
Could not load OpenAI model. If you intended to use OpenAI, please check your OPENAI_API_KEY. Original error: No API key found for OpenAI. Please set either the OPENAI_API_KEY environment variable or openai.api_key prior to initialization. API keys can be found or created at https://platform.openai.com/account/api-keys
To disable the LLM entirely, set llm=None.
I am new to this, any help will be much appreciated. Thanks in advance.
Upvotes: 0
Views: 1062
Reputation: 43
This is how you persist VectorStoreIndex (LlamaIndex) locally:
# Imports
from llama_index.core import (
load_index_from_storage,
SimpleDirectoryReader,
VectorStoreIndex,
StorageContext,
ServiceContext,
Document,
Settings,
)
from llama_index.core.tools import QueryEngineTool, ToolMetadata
# Load your files
lyft_docs = SimpleDirectoryReader(input_files=["./data/10k/lyft_2021.pdf"]).load_data()
# Create index
lyft_index = VectorStoreIndex.from_documents(lyft_docs)
# Persist your index locally
lyft_index.storage_context.persist(persist_dir="./storage/lyft")
Later, I load the index using the code below:
# Load the index from your storage location
storage_context = StorageContext.from_defaults(persist_dir="./storage/lyft")
lyft_index = load_index_from_storage(storage_context)
Upvotes: 0
Reputation: 3
To persist locally, one approach is setting up a local vector data store and using it to store the vectors. Then, in subsequent queries, you can just re-load the vector data store as the index instead of re-loading all the documents each time. There are a lot of vector databases out there (Pinecone, Weaviate, FAISSS, etc) - as an example, using Milvus vector store:
vector_store = MilvusVectorStore(address="localhost:19530",
dim=1024)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(
documents, embed_model=embeddings, storage_context=storage_context,
show_progress=True)
query_engine = index.as_query_engine()
Then, once you've loaded the documents, when you want to query the Milvus database again, you can just use:
index=VectorStoreIndex.from_vector_store(vector_store)
Upvotes: 0