CODEWITHSUNDEEP
nlphuggingface-transformersbert-language-modelhuggingfacehuggingface-hub
Pablo Messina
Pablo Messina

Reputation: 441

How can I make my Hugging Face fine-tuned model's config.json file reference a specific revision/commit from the original pretrained model?

I uploaded this model: https://huggingface.co/pamessina/CXRFE, which is a fine-tuned version of this model: https://huggingface.co/microsoft/BiomedVLP-CXR-BERT-specialized

Unfortunately, CXR-BERT-specialized has this issue: https://github.com/huggingface/transformers/issues/30412

I fixed the issue with this pull request: https://huggingface.co/microsoft/BiomedVLP-CXR-BERT-specialized/discussions/5

However, when I save my fine-tuned model, the config.json file doesn't point to that specific pull request, pointing instead to the main branch by default, but the main branch of CXR-BERT-specialized has the aforementioned issue. As a consequence, when I try to use my model, it triggers the bug from the main branch of the underlying model, which shouldn't happen it were using the version from my pull request.

I've tried explicitly enforcing the revision I want like this:

model = AutoModel.from_pretrained('microsoft/BiomedVLP-CXR-BERT-specialized', revision="6cfc310817fb7d86762d888ced1e3709c57ac578", trust_remote_code=True)
...
model.save_pretrained("/home/pamessina/huggingface_models/CXRFE/")

But the config file that gets saved doesn't reference the desired revision:

{
  "_name_or_path": "microsoft/BiomedVLP-CXR-BERT-specialized",
  "architectures": [
    "CXRBertModel"
  ],
  "attention_probs_dropout_prob": 0.25,
  "auto_map": {
    "AutoConfig": "microsoft/BiomedVLP-CXR-BERT-specialized--configuration_cxrbert.CXRBertConfig",
    "AutoModel": "microsoft/BiomedVLP-CXR-BERT-specialized--modeling_cxrbert.CXRBertModel"
  },
  "classifier_dropout": null,
  "gradient_checkpointing": false,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.25,
  "hidden_size": 768,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "layer_norm_eps": 1e-12,
  "max_position_embeddings": 512,
  "model_type": "cxr-bert",
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "pad_token_id": 0,
  "position_embedding_type": "absolute",
  "projection_size": 128,
  "torch_dtype": "float32",
  "transformers_version": "4.41.2",
  "type_vocab_size": 2,
  "use_cache": true,
  "vocab_size": 30522
}

And when I try to use my fine-tuned model from Hugging Face on Google Colab, I get this error:

enter image description here

As you can see, it's invoking the version associated with the commit id "b59c09e51ab2410b24f4be214bbb49043fe63fc2", when instead it should be using the commit id "6cfc310817fb7d86762d888ced1e3709c57ac578" with my pull request that fixes the bug.

What can I do?

Thanks in advance.

Upvotes: 0

Views: 272

Answers (1)

Yurkee
Yurkee

Reputation: 873

Have you tried adding revision argument to save_pretrained function?

Documentation says save_pretrained accepts push_to_hub arguments, one of which is revision.

huggingface_documentation

like:

model.save_pretrained("/home/pamessina/huggingface_models/CXRFE/", revision="6cfc310817fb7d86762d888ced1e3709c57ac578")

Upvotes: 1

Related Questions