CODEWITHSUNDEEP
tensorflownlpspacynamed-entity-recognitiontransformer-model
miladjurablu
miladjurablu

Reputation: 41

how to make colab use GPU for spacy training NER model

I have 40,000 records and I the training process is very slow, this is the line I use in colab for training

! python -m spacy train config.cfg --output /content/ --paths.train /content/training_data.spacy --paths.dev /content/training_data.spacy

when I run this cell, this show up

ℹ Saving to output directory: /content
ℹ Using CPU
ℹ To switch to GPU 0, use the option: --gpu-id 0

I want to use the GPU but when I use the --gpu-id 0 in end of cell command, I got ERROR

and I changed runtime of colab to GPU

Upvotes: 1

Views: 3902

Answers (1)

Luciano Martins
Luciano Martins

Reputation: 551

The key steps to make it happen are:

  1. enable the GPU (edit -> notebook settings -> hardware acceleration)

  2. install spacy with CUDA support (pip install spacy[cuda100])

  3. Validate if it is all set by running the following code (it must return True):

import spacy

gpu = spacy.prefer_gpu()
print(gpu)

If it is true, then your env is ready to run spacy with GPU support.

Upvotes: 5

Related Questions