CODEWITHSUNDEEP
huggingfacellama
Youshikyou
Youshikyou

Reputation: 417

fine tuned model being pushed to huggingface repo doesn't have config.json

I am using autotrain to train the llama model and push it to my huggingface repo.

!autotrain llm --train --project_name my-llm-test --model meta-llama/Llama-2-7b-hf --data_path test --use_peft --use_int4 --learning_rate 2e-4 --train_batch_size 12 --num_train_epochs 3 --trainer sft --push_to_hub --repo_id myrepo/test

After that I tried to use it

from transformers import AutoModelForSeq2SeqLM,AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("myrepo/test", use_auth_token=True)
test_model = AutoModelForSeq2SeqLM.from_pretrained("myrepo/test", use_auth_token=True)

But I got the error like below. I don't know why it doesn't have the config.json file because I just used the autotrain and pushed the model. I am quite new to huggingface. Would you mind giving me a help? Thanks!

OSError: myrepo/test does not appear to have a file named config.json.

Upvotes: 1

Views: 722

Answers (2)

Michael Gruner
Michael Gruner

Reputation: 640

PEFT just trains the adapter, you need to merge with the base model. You may use this space, for example: https://huggingface.co/spaces/autotrain-projects/llm-merge-adapter

Upvotes: 0

Rishabh Gupta
Rishabh Gupta

Reputation: 31

The problem is due to the restricted resources of the free Colab environment, which led to the script being terminated by Colab. However, upon running the script on a higher-capacity instance with a Colab Pro subscription, the script executed without any issues.

Upvotes: 1

Related Questions