CODEWITHSUNDEEP
pythonartificial-intelligencelarge-language-modelhuggingfacellama
Luis Leal
Luis Leal

Reputation: 3524

huggingface model inference: ERROR: Flag 'minloglevel' was defined more than once (in files 'src/error.cc' and ..)

I'm trying to use llama 3.1 70b from huggingface(end goal is to quantize it and deploy it in amazon sagemaker), but I'm facing:

ERROR: Flag 'minloglevel' was defined more than once (in files 'src/error.cc' and 'home/conda/feedstock_root/build_artifacts/abseil-split_1720857154496/work/absl/log/flags.cc').

The code is super simple (quantization not applied yet), but I cannot make it work (originally I was using a notebook and the kernel died, after exporting the code to python file and running it in the command line I found the error mentioned previously).

import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig, AutoTokenizer
model_id = "meta-llama/Llama-3.1-8B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto"
)
input_text = """<|begin_of_text|><|start_header_id|>user<|end_header_id|>Translate the following English text to French:
'Hello, how are you?'<|eot_id|><|start_header_id|>assistant<|end_header_id|>"""
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
print(input_ids)
output = model.generate(**input_ids, max_new_tokens=10)

I have tried both the 70b and 8b versions, but none worked (so I know it is not something specific to 70). I have a feeling that this is related to package conflicts or similar and not necessarily the models.

Upvotes: 0

Views: 540

Answers (1)

Abdullah Ayad
Abdullah Ayad

Reputation: 21

I faced this issue with my studio space on SageMaker, too.
This is a compatibility issue when using sentencepiece v0.2.0 with the latest transformers version.

I solved this problem by downgrading sentencepiece to 0.1.99.

Upvotes: 2

Related Questions