Can't Run Transformer Fine Tuning With M1 Mac CPU

Question

Here's the code I use

from transformers import DistilBertForSequenceClassification, Trainer, TrainingArguments

training_args = TrainingArguments(
    output_dir='./results',          # output directory
    num_train_epochs=1,              # total number of training epochs
    per_device_train_batch_size=16,  # batch size per device during training
    per_device_eval_batch_size=64,   # batch size for evaluation
    warmup_steps=500,                # number of warmup steps for learning rate scheduler
    weight_decay=0.01,               # strength of weight decay
    logging_dir='./logs',            # directory for storing logs
    logging_steps=10,
    use_mps_device=False
)

model = DistilBertForSequenceClassification.from_pretrained("distilbert-base-uncased")

trainer = Trainer(
    model=model,                         # the instantiated 🤗 Transformers model to be trained
    args=training_args,                  # training arguments, defined above
    train_dataset=train_dataset,         # training dataset
    eval_dataset=val_dataset             # evaluation dataset
)
 
device = torch.device("cpu")
model.to(device)
  
trainer.train()

I get this error: message

RuntimeError: Placeholder storage has not been allocated on MPS device!

If I change the argument to use_mps_device=True, it will train with GPU, even though the model.to statement is pointing to CPU?

I could get the training to run with GPU but not with CPU.

Answer 1

I can verify the OP and the response: Trainer is not working properly on my M2 MPS, but does work (painfully slowly) with the MPS turned off with the no_cuda=True option under TrainingArguments.

FWIW, the errors that I am getting with the MPS set to be on seem to have to do with storage allocation on the MPS.

Answer 2

The Trainer automatically moves the model to the appropriate device if the place_model_on_device attribute is True.

My suggestion would be to remove any moving code & do it through the transformers library. You can specify no_cuda in TrainingArguments as False so that the training objects aren't moved to GPU.