trainer.train() in Kaggle: StdinNotImplementedError: getpass was called, but this frontend does not support input requests

Question

When saving a version in Kaggle, I get StdinNotImplementedError: getpass was called, but this frontend does not support input requests whenever I use the Transformers.Trainer class. The general code I use:

from transformers import Trainer, TrainingArguments
training_args = TrainingArguments(params)
trainer = Trainer(params)
trainer.train()

And the specific cell I am running now:

from transformers import Trainer, TrainingArguments,EarlyStoppingCallback

early_stopping = EarlyStoppingCallback()

training_args = TrainingArguments(
    output_dir=OUT_FINETUNED_MODEL_PATH,          
    num_train_epochs=20,              
    per_device_train_batch_size=16,  
    per_device_eval_batch_size=16,   
    warmup_steps=0,                
    weight_decay=0.01,               
    logging_dir='./logs',            
    logging_steps=100,
    evaluation_strategy="steps",
    eval_steps=100,
    load_best_model_at_end=True,
    metric_for_best_model="eval_loss",
    greater_is_better=False
    
)

trainer = Trainer(
    model=model,                         
    args=training_args,                  
    train_dataset=train_dataset,         
    eval_dataset=val_dataset,             
    callbacks=[early_stopping]

)

trainer.train()

When trainer.train() is called, I get the error below, which I do not get if I train with native PyTorch. I understood that the error arises since I am asked to input a password, but no password is asked when using native PyTorch code, nor when using the same code with trainer.train() on Google Colab. Any solution would be ok, like:

1. Avoid being asked the password.
2. Enable input requests when saving a notebook on Kaggle. After that, if I understood correctly, I would need to go to https://wandb.ai/authorize (after having created an account) and copy the generated key to console. However, I do not understand why wandb should be necessary since I never explicitly used it so far.

wandb: You can find your API key in your browser here: https://wandb.ai/authorize
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/wandb/sdk/wandb_init.py", line 741, in init
    wi.setup(kwargs)
  File "/opt/conda/lib/python3.7/site-packages/wandb/sdk/wandb_init.py", line 155, in setup
    wandb_login._login(anonymous=anonymous, force=force, _disable_warning=True)
  File "/opt/conda/lib/python3.7/site-packages/wandb/sdk/wandb_login.py", line 210, in _login
    wlogin.prompt_api_key()
  File "/opt/conda/lib/python3.7/site-packages/wandb/sdk/wandb_login.py", line 144, in prompt_api_key
    no_create=self._settings.force,
  File "/opt/conda/lib/python3.7/site-packages/wandb/sdk/lib/apikey.py", line 135, in prompt_api_key
    key = input_callback(api_ask).strip()
  File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 825, in getpass
    "getpass was called, but this frontend does not support input requests."
IPython.core.error.StdinNotImplementedError: getpass was called, but this frontend does not support input requests.
wandb: ERROR Abnormal program exit
---------------------------------------------------------------------------
StdinNotImplementedError                  Traceback (most recent call last)
/opt/conda/lib/python3.7/site-packages/wandb/sdk/wandb_init.py in init(job_type, dir, config, project, entity, reinit, tags, group, name, notes, magic, config_exclude_keys, config_include_keys, anonymous, mode, allow_val_change, resume, force, tensorboard, sync_tensorboard, monitor_gym, save_code, id, settings)
    740         wi = _WandbInit()
--> 741         wi.setup(kwargs)
    742         except_exit = wi.settings._except_exit

/opt/conda/lib/python3.7/site-packages/wandb/sdk/wandb_init.py in setup(self, kwargs)
    154         if not settings._offline and not settings._noop:
--> 155             wandb_login._login(anonymous=anonymous, force=force, _disable_warning=True)
    156 

/opt/conda/lib/python3.7/site-packages/wandb/sdk/wandb_login.py in _login(anonymous, key, relogin, host, force, _backend, _silent, _disable_warning)
    209     if not key:
--> 210         wlogin.prompt_api_key()
    211 

/opt/conda/lib/python3.7/site-packages/wandb/sdk/wandb_login.py in prompt_api_key(self)
    143             no_offline=self._settings.force,
--> 144             no_create=self._settings.force,
    145         )

/opt/conda/lib/python3.7/site-packages/wandb/sdk/lib/apikey.py in prompt_api_key(settings, api, input_callback, browser_callback, no_offline, no_create, local)
    134             )
--> 135             key = input_callback(api_ask).strip()
    136         write_key(settings, key, api=api)

/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py in getpass(self, prompt, stream)
    824             raise StdinNotImplementedError(
--> 825                 "getpass was called, but this frontend does not support input requests."
    826             )

StdinNotImplementedError: getpass was called, but this frontend does not support input requests.

The above exception was the direct cause of the following exception:

Exception                                 Traceback (most recent call last)
<ipython-input-82-4d1046ab80b8> in <module>
     42     )
     43 
---> 44     trainer.train()

/opt/conda/lib/python3.7/site-packages/transformers/trainer.py in train(self, resume_from_checkpoint, trial, **kwargs)
   1067         model.zero_grad()
   1068 
-> 1069         self.control = self.callback_handler.on_train_begin(self.args, self.state, self.control)
   1070 
   1071         # Skip the first epochs_trained epochs to get the random state of the dataloader at the right point.

/opt/conda/lib/python3.7/site-packages/transformers/trainer_callback.py in on_train_begin(self, args, state, control)
    338     def on_train_begin(self, args: TrainingArguments, state: TrainerState, control: TrainerControl):
    339         control.should_training_stop = False
--> 340         return self.call_event("on_train_begin", args, state, control)
    341 
    342     def on_train_end(self, args: TrainingArguments, state: TrainerState, control: TrainerControl):

/opt/conda/lib/python3.7/site-packages/transformers/trainer_callback.py in call_event(self, event, args, state, control, **kwargs)
    386                 train_dataloader=self.train_dataloader,
    387                 eval_dataloader=self.eval_dataloader,
--> 388                 **kwargs,
    389             )
    390             # A Callback can skip the return of `control` if it doesn't change it.

/opt/conda/lib/python3.7/site-packages/transformers/integrations.py in on_train_begin(self, args, state, control, model, **kwargs)
    627             self._wandb.finish()
    628         if not self._initialized:
--> 629             self.setup(args, state, model, **kwargs)
    630 
    631     def on_train_end(self, args, state, control, model=None, tokenizer=None, **kwargs):

/opt/conda/lib/python3.7/site-packages/transformers/integrations.py in setup(self, args, state, model, **kwargs)
    604                     project=os.getenv("WANDB_PROJECT", "huggingface"),
    605                     name=run_name,
--> 606                     **init_args,
    607                 )
    608             # add config parameters (run may have been created manually)

/opt/conda/lib/python3.7/site-packages/wandb/sdk/wandb_init.py in init(job_type, dir, config, project, entity, reinit, tags, group, name, notes, magic, config_exclude_keys, config_include_keys, anonymous, mode, allow_val_change, resume, force, tensorboard, sync_tensorboard, monitor_gym, save_code, id, settings)
    779             if except_exit:
    780                 os._exit(-1)
--> 781             six.raise_from(Exception("problem"), error_seen)
    782     return run

/opt/conda/lib/python3.7/site-packages/six.py in raise_from(value, from_value)

Exception: problem

Answer 1

I think the problem is from getpass itself, because when we tell Kaggle to run the notebook in background, there is no input box for us to enter value.

Answer 2

from tokenizers import ByteLevelBPETokenizer

from tokenizers.models import BPE

from transformers import GPT2Config,

GPT2LMHeadModel,GPT2Tokenizer,DataCollatorForLanguageModeling

from tokenizers import Tokenizer

tokenizer = Tokenizer(BPE(unk_token="[x]"))

from datasets import load_dataset

from transformers import Trainer,TrainingArguments

:::write down your api key to x

Answer 3

You may want to try adding report_to="tensorboard" or any other reasonable string array in your TrainingArguments https://huggingface.co/transformers/main_classes/trainer.html#transformers.TrainingArguments

If you have multiple logger that you want to use report_to="all" (the default value) try os.environ["WANDB_DISABLED"] = "true" such that wandb is always disabled.

see: https://huggingface.co/transformers/main_classes/trainer.html#transformers.TFTrainer.setup_wandb