CODEWITHSUNDEEP
langchainlarge-language-model
Amit Kharel
Amit Kharel

Reputation: 154

Langchain(HuggingFaceModel) - argument needs to be of type (SquadExample, dict)

I am trying to build an RAG Question Answering Model using Langchain and HuggingFacePipeline.

model_name = "Intel/dynamic_tinybert"

tokenizer = AutoTokenizer.from_pretrained(model_name, cache_dir="../assets")

question_answerer = pipeline(
    "question-answering",
    model=model_name,
    tokenizer=tokenizer
)

llm = HuggingFacePipeline(
    pipeline=question_answerer
)

While trying to get any query result as:

llm("What is the mass of the sun?")

The error is:

argument needs to be of type (SquadExample, dict)

Any solution or insights? Am I doing anything wrong?

The llm model should return me the answer. I am trying a RetreivalQA Chain to chain my prompts and get the output from vector store retreiver as:

qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever, return_source_documents=True, chain_type_kwargs={"prompt": prompt})

Upvotes: 3

Views: 3012

Answers (4)

Amit Kharel
Amit Kharel

Reputation: 154

Upon looking at the HuggingFacePipeline source code, I found it only supports below pipelines:
text-generation`, text2text-generation and summarization

I went with text2text-generation for my use case as it supports generating answers from context and question.

Hope it helps if someone encounters similar issue.

Edit: The model Bert no longer works, I had to use the model in category text2text-generation

Upvotes: 2

Wasikur Rahman
Wasikur Rahman

Reputation: 74

HuggingFacePipeline does only support text-generation, text2text-generation and summarization. But even with text2text-generation I was getting this error. I think there could be a bug in langchain >= 0.1.8. Downgrading to langchain==0.1.7 solved the problem for now.

Upvotes: 0

Amir P
Amir P

Reputation: 151

When using "question-answering" Pipeline the prompt should be a SquadExampleinstead of a String. Use create_sample to create a SquadExample.

Upvotes: 0

sajad pakdaman
sajad pakdaman

Reputation: 1

I believe the issue is that inter/dynamic_tinybert was fine-tuned for abstractive QA. It means that it will only support `task="question-answering".

Please see the docs below.

Question answering

Question answering is another token-level task that returns an answer to a question, sometimes with context (open-domain) and other times without context (closed-domain). This task happens whenever we ask a virtual assistant something like whether a restaurant is open. It can also provide customer or technical support and help search engines retrieve the relevant information you’re asking for.

There are two common types of question answering:

extractive: given a question and some context, the answer is a span of text from the context the model must extract.

abstractive: given a question and some context, the answer is generated from the context; this approach is handled by the Text2TextGenerationPipeline instead of the QuestionAnsweringPipeline shown below

from transformers import pipeline

question_answerer = pipeline(task="question-answering")
preds = question_answerer(
    question="What is the name of the repository?",
    context="The name of the repository is huggingface/transformers",
)
print(
    f"score: {round(preds['score'], 4)}, start: {preds['start']}, end: {preds['end']}, answer: {preds['answer']}"
)
score: 0.9327, start: 30, end: 54, answer: huggingface/transformers

Upvotes: 0

Related Questions