CODEWITHSUNDEEP
pythonsyntax-errorllamaollamallama3
James Brittain
James Brittain

Reputation: 1

HFValidation Error for calling the repo-id incorrectly, what am I doing wrong?

HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': 'meta-llama/llama3.1/8b-instruct-fp16'. Use repo_type argument if needed.

tokenizer = AutoTokenizer.from_pretrained("meta-llama/llama3.1/8b-instruct-fp16")
model = AutoModelForCausalLM.from_pretrained("meta-llama/llama3.1/8b-instruct-fp16")

This is my error and the code that seems to be the issue but I'm not sure how to fix it.

I've tried entering meta-llama/llama3.1/8b-instruct-fp16 and I've also tried entering the direct path address. I'm not sure what I'm doing wrong. I've also tried giving the code to ChatGPT and that didn't work either.

Upvotes: 0

Views: 531

Answers (1)

D.lola
D.lola

Reputation: 2284

Try this instead, see ref: llama3, however, before that you try any solution provided, you can remove fp-16 from meta-llama/llama3.1/8b-instruct-fp16

from transformers import AutoTokenizer, AutoModelForCausalLM

# Use the correct repo name
tokenizer = AutoTokenizer.from_pretrained("meta-llama/llama3.1:8b-instruct-fp16")
model = AutoModelForCausalLM.from_pretrained("meta-llama/llama3.1:8b-instruct-fp16")

Alternatively you can follow this guide on hugging face .`

import transformers
import torch

model_id = "meta-llama/Meta-Llama-3.1-8B-Instruct"

pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
    {"role": "user", "content": "Who are you?"},
]

outputs = pipeline(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])

Upvotes: 0

Related Questions