thisintheway
Reputation: 123
Cuda and OpenAI Whisper : enforcing GPU instead of CPU not working?
tldr : Am I right in assuming torch.cuda.init(), device = "cuda" and result = model.transcribe(etc) should be enough to enforce gpu usage ?
I have checked on several forum posts and could not find a solution. Sorry if it's silly. I also posted on the whisper git but maybe it's not whisper-specific.
Here is my python script in a nutshell :
import whisper
import soundfile as sf
import torch
# specify the path to the input audio file
input_file = "H:\\path\\3minfile.WAV"
# specify the path to the output transcript file
output_file = "H:\\path\\transcript.txt"
# Cuda allows for the GPU to be used which is more optimized than the cpu
torch.cuda.init()
device = "cuda" # if torch.cuda.is_available() else "cpu"
# Load audio file
audio_data, sample_rate = sf.read(input_file, always_2d=True)
#load whisper model
model_size = "tiny"
print("loading model :", model_size)
model = whisper.load_model(model_size).to(device)
print(model_size, "model loaded")
# Initialize variables
results = []
language = "fr"
# Transcribe audio
with torch.cuda.device(device):
result = model.transcribe(audio_data, language=language, fp16=False, word_timestamps=True)
However, it is returning the following error on the last line, hinting that it's trying to run it on cpu :
RuntimeError: [enforce fail at C:\actions-runner_work\pytorch\pytorch\builder\windows\pytorch\c10\core\impl\alloc_cpu.cpp:72] data. DefaultCPUAllocator: not enough memory: you tried to allocate 30623038517864 bytes.
I am using Jupyter, and i checked that the pytorch version it's using was the cuda/gpu one and not a cpu-locked version :
print(torch.__version__)
2.0.0+cu117
So I really don't get it. Could there be a conflict of pythorch libraries ? Am I doing something wrong ? Is the transcribe() function indeed using cpu instead of gpu ?
I am using anaconda3, here is what conda list returns, in case it helps :
# Name Version Build Channel
_ipyw_jlab_nb_ext_conf 0.1.0 py39haa95532_0
alabaster 0.7.12 pyhd3eb1b0_0
anaconda 2022.10 py39_0
anaconda-client 1.11.0 py39haa95532_0
anaconda-navigator 2.3.1 py39haa95532_0
anaconda-project 0.11.1 py39haa95532_0
anyio 3.5.0 py39haa95532_0
appdirs 1.4.4 pyhd3eb1b0_0
argon2-cffi 21.3.0 pyhd3eb1b0_0
argon2-cffi-bindings 21.2.0 py39h2bbff1b_0
arrow 1.2.2 pyhd3eb1b0_0
astroid 2.11.7 py39haa95532_0
astropy 5.1 py39h080aedc_0
atomicwrites 1.4.0 py_0
attrs 21.4.0 pyhd3eb1b0_0
automat 20.2.0 py_0
autopep8 1.6.0 pyhd3eb1b0_1
babel 2.9.1 pyhd3eb1b0_0
backcall 0.2.0 pyhd3eb1b0_0
backports 1.1 pyhd3eb1b0_0
backports.functools_lru_cache 1.6.4 pyhd3eb1b0_0
backports.tempfile 1.0 pyhd3eb1b0_1
backports.weakref 1.0.post1 py_1
bcrypt 3.2.0 py39h2bbff1b_1
beautifulsoup4 4.11.1 py39haa95532_0
binaryornot 0.4.4 pyhd3eb1b0_1
bitarray 2.5.1 py39h2bbff1b_0
bkcharts 0.2 py39haa95532_1
black 22.6.0 py39haa95532_0
blas 1.0 mkl
bleach 4.1.0 pyhd3eb1b0_0
blosc 1.21.0 h19a0ad4_1
bokeh 2.4.3 py39haa95532_0
boto3 1.24.28 py39haa95532_0
botocore 1.27.28 py39haa95532_0
bottleneck 1.3.5 py39h080aedc_0
brotli 1.0.9 h2bbff1b_7
brotli-bin 1.0.9 h2bbff1b_7
brotlipy 0.7.0 py39h2bbff1b_1003
bzip2 1.0.8 he774522_0
ca-certificates 2022.07.19 haa95532_0
certifi 2022.9.14 py39haa95532_0
cffi 1.15.1 py39h2bbff1b_0
cfitsio 3.470 h2bbff1b_7
chardet 4.0.0 py39haa95532_1003
charls 2.2.0 h6c2663c_0
charset-normalizer 2.0.4 pyhd3eb1b0_0
click 8.0.4 py39haa95532_0
cloudpickle 2.0.0 pyhd3eb1b0_0
clyent 1.2.2 py39haa95532_1
colorama 0.4.5 py39haa95532_0
colorcet 3.0.0 py39haa95532_0
comtypes 1.1.10 py39haa95532_1002
conda 23.1.0 py39haa95532_0
conda-build 3.22.0 py39haa95532_0
conda-content-trust 0.1.3 py39haa95532_0
conda-env 2.6.0 haa95532_1
conda-pack 0.6.0 pyhd3eb1b0_0
conda-package-handling 2.0.2 py39haa95532_0
conda-package-streaming 0.7.0 py39haa95532_0
conda-repo-cli 1.0.27 py39haa95532_0
conda-token 0.4.0 pyhd3eb1b0_0
conda-verify 3.4.2 py_1
console_shortcut 0.1.1 4
constantly 15.1.0 pyh2b92418_0
cookiecutter 1.7.3 pyhd3eb1b0_0
cryptography 37.0.1 py39h21b164f_0
cssselect 1.1.0 pyhd3eb1b0_0
cuda-cccl 12.1.55 0 nvidia
cuda-cudart 11.8.89 0 nvidia
cuda-cudart-dev 11.8.89 0 nvidia
cuda-cupti 11.8.87 0 nvidia
cuda-libraries 11.8.0 0 nvidia
cuda-libraries-dev 11.8.0 0 nvidia
cuda-nvrtc 11.8.89 0 nvidia
cuda-nvrtc-dev 11.8.89 0 nvidia
cuda-nvtx 11.8.86 0 nvidia
cuda-profiler-api 12.1.55 0 nvidia
cuda-runtime 11.8.0 0 nvidia
cudatoolkit 10.1.243 h74a9793_0
curl 7.84.0 h2bbff1b_0
cycler 0.11.0 pyhd3eb1b0_0
cython 0.29.32 py39hd77b12b_0
cytoolz 0.11.0 py39h2bbff1b_0
daal4py 2021.6.0 py39h757b272_1
dal 2021.6.0 h59b6b97_874
dask 2022.7.0 py39haa95532_0
dask-core 2022.7.0 py39haa95532_0
dataclasses 0.8 pyh6d0b6a4_7
datashader 0.14.1 py39haa95532_0
datashape 0.5.4 py39haa95532_1
debugpy 1.5.1 py39hd77b12b_0
decorator 5.1.1 pyhd3eb1b0_0
defusedxml 0.7.1 pyhd3eb1b0_0
diff-match-patch 20200713 pyhd3eb1b0_0
dill 0.3.4 pyhd3eb1b0_0
distributed 2022.7.0 py39haa95532_0
docutils 0.18.1 py39haa95532_3
entrypoints 0.4 py39haa95532_0
et_xmlfile 1.1.0 py39haa95532_0
ffmpeg 1.4 pypi_0 pypi
ffmpeg-python 0.2.0 pypi_0 pypi
fftw 3.3.9 h2bbff1b_1
filelock 3.6.0 pyhd3eb1b0_0
flake8 4.0.1 pyhd3eb1b0_1
flask 1.1.2 pyhd3eb1b0_0
fonttools 4.25.0 pyhd3eb1b0_0
freetype 2.10.4 hd328e21_0
fsspec 2022.7.1 py39haa95532_0
future 0.18.2 py39haa95532_1
gensim 4.1.2 py39hd77b12b_0
giflib 5.2.1 h62dcd97_0
glob2 0.7 pyhd3eb1b0_0
greenlet 1.1.1 py39hd77b12b_0
h5py 3.7.0 py39h3de5c98_0
hdf5 1.10.6 h1756f20_1
heapdict 1.0.1 pyhd3eb1b0_0
holoviews 1.15.0 py39haa95532_0
hvplot 0.8.0 py39haa95532_0
hyperlink 21.0.0 pyhd3eb1b0_0
icc_rt 2022.1.0 h6049295_2
icu 58.2 ha925a31_3
idna 3.3 pyhd3eb1b0_0
imagecodecs 2021.8.26 py39hc0a7faf_1
imageio 2.19.3 py39haa95532_0
imagesize 1.4.1 py39haa95532_0
importlib-metadata 4.11.3 py39haa95532_0
importlib_metadata 4.11.3 hd3eb1b0_0
incremental 21.3.0 pyhd3eb1b0_0
inflection 0.5.1 py39haa95532_0
iniconfig 1.1.1 pyhd3eb1b0_0
intake 0.6.5 pyhd3eb1b0_0
intel-openmp 2021.4.0 haa95532_3556
intervaltree 3.1.0 pyhd3eb1b0_0
ipykernel 6.15.2 py39haa95532_0
ipython 7.31.1 py39haa95532_1
ipython_genutils 0.2.0 pyhd3eb1b0_1
ipywidgets 7.6.5 pyhd3eb1b0_1
isort 5.9.3 pyhd3eb1b0_0
itemadapter 0.3.0 pyhd3eb1b0_0
itemloaders 1.0.4 pyhd3eb1b0_1
itsdangerous 2.0.1 pyhd3eb1b0_0
jdcal 1.4.1 pyhd3eb1b0_0
jedi 0.18.1 py39haa95532_1
jellyfish 0.9.0 py39h2bbff1b_0
jinja2 2.11.3 pyhd3eb1b0_0
jinja2-time 0.2.0 pyhd3eb1b0_3
jmespath 0.10.0 pyhd3eb1b0_0
joblib 1.1.0 pyhd3eb1b0_0
jpeg 9e h2bbff1b_0
jq 1.6 haa95532_1
json5 0.9.6 pyhd3eb1b0_0
jsonschema 4.16.0 py39haa95532_0
jupyter 1.0.0 py39haa95532_8
jupyter_client 7.3.4 py39haa95532_0
jupyter_console 6.4.3 pyhd3eb1b0_0
jupyter_core 4.11.1 py39haa95532_0
jupyter_server 1.18.1 py39haa95532_0
jupyterlab 3.4.4 py39haa95532_0
jupyterlab_pygments 0.1.2 py_0
jupyterlab_server 2.10.3 pyhd3eb1b0_1
jupyterlab_widgets 1.0.0 pyhd3eb1b0_1
keyring 23.4.0 py39haa95532_0
kiwisolver 1.4.2 py39hd77b12b_0
lazy-object-proxy 1.6.0 py39h2bbff1b_0
lcms2 2.12 h83e58a3_0
lerc 3.0 hd77b12b_0
libaec 1.0.4 h33f27b4_1
libarchive 3.6.1 hebabd0d_0
libbrotlicommon 1.0.9 h2bbff1b_7
libbrotlidec 1.0.9 h2bbff1b_7
libbrotlienc 1.0.9 h2bbff1b_7
libcublas 11.11.3.6 0 nvidia
libcublas-dev 11.11.3.6 0 nvidia
libcufft 10.9.0.58 0 nvidia
libcufft-dev 10.9.0.58 0 nvidia
libcurand 10.3.2.56 0 nvidia
libcurand-dev 10.3.2.56 0 nvidia
libcurl 7.84.0 h86230a5_0
libcusolver 11.4.1.48 0 nvidia
libcusolver-dev 11.4.1.48 0 nvidia
libcusparse 11.7.5.86 0 nvidia
libcusparse-dev 11.7.5.86 0 nvidia
libdeflate 1.8 h2bbff1b_5
libiconv 1.16 h2bbff1b_2
liblief 0.11.5 hd77b12b_1
libnpp 11.8.0.86 0 nvidia
libnpp-dev 11.8.0.86 0 nvidia
libnvjpeg 11.9.0.86 0 nvidia
libnvjpeg-dev 11.9.0.86 0 nvidia
libpng 1.6.37 h2a8f88b_0
libsodium 1.0.18 h62dcd97_0
libspatialindex 1.9.3 h6c2663c_0
libssh2 1.10.0 hcd4344a_0
libtiff 4.4.0 h8a3f274_0
libuv 1.44.2 h2bbff1b_0
libwebp 1.2.2 h2bbff1b_0
libxml2 2.9.14 h0ad7f3c_0
libxslt 1.1.35 h2bbff1b_0
libzopfli 1.0.3 ha925a31_0
llvmlite 0.38.0 py39h23ce68f_0
locket 1.0.0 py39haa95532_0
lxml 4.9.1 py39h1985fb9_0
lz4 3.1.3 py39h2bbff1b_0
lz4-c 1.9.3 h2bbff1b_1
lzo 2.10 he774522_2
m2-msys2-runtime 2.5.0.17080.65c939c 3
m2-patch 2.7.5 2
m2w64-libwinpthread-git 5.0.0.4634.697f757 2
markdown 3.3.4 py39haa95532_0
markupsafe 2.0.1 py39h2bbff1b_0
matplotlib 3.5.2 py39haa95532_0
matplotlib-base 3.5.2 py39hd77b12b_0
matplotlib-inline 0.1.6 py39haa95532_0
mccabe 0.6.1 py39haa95532_2
menuinst 1.4.19 py39h59b6b97_0
mistune 0.8.4 py39h2bbff1b_1000
mkl 2021.4.0 haa95532_640
mkl-service 2.4.0 py39h2bbff1b_0
mkl_fft 1.3.1 py39h277e83a_0
mkl_random 1.2.2 py39hf11a4ad_0
mock 4.0.3 pyhd3eb1b0_0
more-itertools 9.1.0 pypi_0 pypi
mpmath 1.2.1 py39haa95532_0
msgpack-python 1.0.3 py39h59b6b97_0
msys2-conda-epoch 20160418 1
multipledispatch 0.6.0 py39haa95532_0
munkres 1.1.4 py_0
mypy_extensions 0.4.3 py39haa95532_1
navigator-updater 0.3.0 py39haa95532_0
nbclassic 0.3.5 pyhd3eb1b0_0
nbclient 0.5.13 py39haa95532_0
nbconvert 6.4.4 py39haa95532_0
nbformat 5.5.0 py39haa95532_0
nest-asyncio 1.5.5 py39haa95532_0
networkx 2.8.4 py39haa95532_0
nltk 3.7 pyhd3eb1b0_0
nose 1.3.7 pyhd3eb1b0_1008
notebook 6.4.12 py39haa95532_0
numba 0.55.1 py39hf11a4ad_0
numexpr 2.8.3 py39hb80d3ca_0
numpy 1.21.5 py39h7a0a035_3
numpy-base 1.21.5 py39hca35cd5_3
numpydoc 1.4.0 py39haa95532_0
olefile 0.46 pyhd3eb1b0_0
openai-whisper 20230314 pypi_0 pypi
openjpeg 2.4.0 h4fc8c34_0
openpyxl 3.0.10 py39h2bbff1b_0
openssl 1.1.1q h2bbff1b_0
packaging 21.3 pyhd3eb1b0_0
pandas 1.4.4 py39hd77b12b_0
pandocfilters 1.5.0 pyhd3eb1b0_0
panel 0.13.1 py39haa95532_0
param 1.12.0 pyhd3eb1b0_0
paramiko 2.8.1 pyhd3eb1b0_0
parsel 1.6.0 py39haa95532_0
parso 0.8.3 pyhd3eb1b0_0
partd 1.2.0 pyhd3eb1b0_1
pathlib 1.0.1 pyhd3eb1b0_1
pathspec 0.9.0 py39haa95532_0
patsy 0.5.2 py39haa95532_1
pep8 1.7.1 py39haa95532_1
pexpect 4.8.0 pyhd3eb1b0_3
pickleshare 0.7.5 pyhd3eb1b0_1003
pillow 9.2.0 py39hdc2b20a_1
pip 22.2.2 py39haa95532_0
pkginfo 1.8.2 pyhd3eb1b0_0
platformdirs 2.5.2 py39haa95532_0
plotly 5.9.0 py39haa95532_0
pluggy 1.0.0 py39haa95532_1
powershell_shortcut 0.0.1 3
poyo 0.5.0 pyhd3eb1b0_0
prometheus_client 0.14.1 py39haa95532_0
prompt-toolkit 3.0.20 pyhd3eb1b0_0
prompt_toolkit 3.0.20 hd3eb1b0_0
protego 0.1.16 py_0
psutil 5.9.0 py39h2bbff1b_0
ptyprocess 0.7.0 pyhd3eb1b0_2
py 1.11.0 pyhd3eb1b0_0
py-lief 0.11.5 py39hd77b12b_1
pyasn1 0.4.8 pyhd3eb1b0_0
pyasn1-modules 0.2.8 py_0
pycodestyle 2.8.0 pyhd3eb1b0_0
pycosat 0.6.3 py39h2bbff1b_0
pycparser 2.21 pyhd3eb1b0_0
pyct 0.4.8 py39haa95532_1
pycurl 7.45.1 py39hcd4344a_0
pydispatcher 2.0.5 py39haa95532_2
pydocstyle 6.1.1 pyhd3eb1b0_0
pyerfa 2.0.0 py39h2bbff1b_0
pyflakes 2.4.0 pyhd3eb1b0_0
pygments 2.11.2 pyhd3eb1b0_0
pyhamcrest 2.0.2 pyhd3eb1b0_2
pyjwt 2.4.0 py39haa95532_0
pylint 2.14.5 py39haa95532_0
pyls-spyder 0.4.0 pyhd3eb1b0_0
pynacl 1.5.0 py39h8cc25b3_0
pyodbc 4.0.34 py39hd77b12b_0
pyopenssl 22.0.0 pyhd3eb1b0_0
pyparsing 3.0.9 py39haa95532_0
pyqt 5.9.2 py39hd77b12b_6
pyrsistent 0.18.0 py39h196d8e1_0
pysocks 1.7.1 py39haa95532_0
pytables 3.6.1 py39h56d22b6_1
pytest 7.1.2 py39haa95532_0
python 3.9.13 h6244533_1
python-dateutil 2.8.2 pyhd3eb1b0_0
python-fastjsonschema 2.16.2 py39haa95532_0
python-libarchive-c 2.9 pyhd3eb1b0_1
python-lsp-black 1.0.0 pyhd3eb1b0_0
python-lsp-jsonrpc 1.0.0 pyhd3eb1b0_0
python-lsp-server 1.3.3 pyhd3eb1b0_0
python-slugify 5.0.2 pyhd3eb1b0_0
python-snappy 0.6.0 py39hd77b12b_3
pytorch 2.0.0 py3.9_cuda11.8_cudnn8_0 pytorch
pytorch-cuda 11.8 h24eeafa_3 pytorch
pytorch-mutex 1.0 cuda pytorch
pytz 2022.1 py39haa95532_0
pyviz_comms 2.0.2 pyhd3eb1b0_0
pywavelets 1.3.0 py39h2bbff1b_0
pywin32 302 py39h2bbff1b_2
pywin32-ctypes 0.2.0 py39haa95532_1000
pywinpty 2.0.2 py39h5da7b33_0
pyyaml 6.0 py39h2bbff1b_1
pyzmq 23.2.0 py39hd77b12b_0
qdarkstyle 3.0.2 pyhd3eb1b0_0
qstylizer 0.1.10 pyhd3eb1b0_0
qt 5.9.7 vc14h73c81de_0
qtawesome 1.0.3 pyhd3eb1b0_0
qtconsole 5.2.2 pyhd3eb1b0_0
qtpy 2.2.0 py39haa95532_0
queuelib 1.5.0 py39haa95532_0
regex 2022.7.9 py39h2bbff1b_0
requests 2.28.1 py39haa95532_0
requests-file 1.5.1 pyhd3eb1b0_0
rope 0.22.0 pyhd3eb1b0_0
rtree 0.9.7 py39h2eaa2aa_1
ruamel.yaml 0.17.21 py39h2bbff1b_0
ruamel.yaml.clib 0.2.6 py39h2bbff1b_1
ruamel_yaml 0.15.100 py39h2bbff1b_0
s3transfer 0.6.0 py39haa95532_0
scikit-image 0.19.2 py39hf11a4ad_0
scikit-learn 1.0.2 py39hf11a4ad_1
scikit-learn-intelex 2021.6.0 py39haa95532_0
scipy 1.9.1 py39he11b74f_0
scrapy 2.6.2 py39haa95532_0
seaborn 0.11.2 pyhd3eb1b0_0
send2trash 1.8.0 pyhd3eb1b0_1
service_identity 18.1.0 pyhd3eb1b0_1
setuptools 63.4.1 py39haa95532_0
sip 4.19.13 py39hd77b12b_0
six 1.16.0 pyhd3eb1b0_1
smart_open 5.2.1 py39haa95532_0
snappy 1.1.9 h6c2663c_0
sniffio 1.2.0 py39haa95532_1
snowballstemmer 2.2.0 pyhd3eb1b0_0
sortedcollections 2.1.0 pyhd3eb1b0_0
sortedcontainers 2.4.0 pyhd3eb1b0_0
soundfile 0.12.1 pypi_0 pypi
soupsieve 2.3.1 pyhd3eb1b0_0
sphinx 5.0.2 py39haa95532_0
sphinxcontrib-applehelp 1.0.2 pyhd3eb1b0_0
sphinxcontrib-devhelp 1.0.2 pyhd3eb1b0_0
sphinxcontrib-htmlhelp 2.0.0 pyhd3eb1b0_0
sphinxcontrib-jsmath 1.0.1 pyhd3eb1b0_0
sphinxcontrib-qthelp 1.0.3 pyhd3eb1b0_0
sphinxcontrib-serializinghtml 1.1.5 pyhd3eb1b0_0
spyder 5.2.2 py39haa95532_1
spyder-kernels 2.2.1 py39haa95532_0
sqlalchemy 1.4.39 py39h2bbff1b_0
sqlite 3.39.3 h2bbff1b_0
statsmodels 0.13.2 py39h2bbff1b_0
sympy 1.10.1 py39haa95532_0
tabulate 0.8.10 py39haa95532_0
tbb 2021.6.0 h59b6b97_0
tbb4py 2021.6.0 py39h59b6b97_0
tblib 1.7.0 pyhd3eb1b0_0
tenacity 8.0.1 py39haa95532_1
terminado 0.13.1 py39haa95532_0
testpath 0.6.0 py39haa95532_0
text-unidecode 1.3 pyhd3eb1b0_0
textdistance 4.2.1 pyhd3eb1b0_0
threadpoolctl 2.2.0 pyh0d69192_0
three-merge 0.1.1 pyhd3eb1b0_0
tifffile 2021.7.2 pyhd3eb1b0_2
tiktoken 0.3.1 pypi_0 pypi
tinycss 0.4 pyhd3eb1b0_1002
tk 8.6.12 h2bbff1b_0
tldextract 3.2.0 pyhd3eb1b0_0
toml 0.10.2 pyhd3eb1b0_0
tomli 2.0.1 py39haa95532_0
tomlkit 0.11.1 py39haa95532_0
toolz 0.11.2 pyhd3eb1b0_0
torch 2.0.0+cu117 pypi_0 pypi
torchaudio 2.0.1+cu117 pypi_0 pypi
torchvision 0.15.1+cu117 pypi_0 pypi
tornado 6.1 py39h2bbff1b_0
tqdm 4.64.1 py39haa95532_0
traitlets 5.1.1 pyhd3eb1b0_0
twisted 22.2.0 py39h2bbff1b_1
twisted-iocpsupport 1.0.2 py39h2bbff1b_0
typing-extensions 4.3.0 py39haa95532_0
typing_extensions 4.3.0 py39haa95532_0
tzdata 2022c h04d1e81_0
ujson 5.4.0 py39hd77b12b_0
unidecode 1.2.0 pyhd3eb1b0_0
urllib3 1.26.11 py39haa95532_0
vc 14.2 h21ff451_1
vs2015_runtime 14.27.29016 h5e58377_2
w3lib 1.21.0 pyhd3eb1b0_0
watchdog 2.1.6 py39haa95532_0
wcwidth 0.2.5 pyhd3eb1b0_0
webencodings 0.5.1 py39haa95532_1
websocket-client 0.58.0 py39haa95532_4
werkzeug 2.0.3 pyhd3eb1b0_0
wheel 0.37.1 pyhd3eb1b0_0
widgetsnbextension 3.5.2 py39haa95532_0
win_inet_pton 1.1.0 py39haa95532_0
win_unicode_console 0.5 py39haa95532_0
wincertstore 0.2 py39haa95532_2
winpty 0.4.3 4
wrapt 1.14.1 py39h2bbff1b_0
xarray 0.20.1 pyhd3eb1b0_1
xlrd 2.0.1 pyhd3eb1b0_0
xlsxwriter 3.0.3 pyhd3eb1b0_0
xlwings 0.27.15 py39haa95532_0
xz 5.2.6 h8cc25b3_0
yaml 0.2.5 he774522_0
yapf 0.31.0 pyhd3eb1b0_0
zeromq 4.3.4 hd77b12b_0
zfp 0.5.5 hd77b12b_6
zict 2.1.0 py39haa95532_0
zipp 3.8.0 py39haa95532_0
zlib 1.2.12 h8cc25b3_3
zope 1.0 py39haa95532_1
zope.interface 5.4.0 py39h2bbff1b_0
zstandard 0.19.0 py39h2bbff1b_0
zstd 1.5.2 h19a0ad4_0
Upvotes: 8
Views: 43650
Answers (2)
Etienne Jacquot
Reputation: 486
You need to make sure that you are installing PyTorch with CUDA support, to actually leverage your GPUs.
See here for reference documentation: https://pytorch.org/get-started/locally/#with-cuda-1
python3 -m pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 python3 -m pip3 install -U openai-whisperyou can then confirm in Python if CUDA is available
import torch torch.cuda.is_available()
Upvotes: 14
pythonista
Reputation: 410
When using Whisper, you can directly offload the model to the GPU during initialization. To do so, you have to specify the device parameter in the load_model method
So your corrected code would look like:
model = whisper.load_model(model_size, device="cuda")
You can now call the transcribe function directly, no need to use with torch.cuda.device(device)
Note that you actually do not need to specify the device parameter, Whisper attempts to use CUDA by default if it is present
Upvotes: 7
Related Questions
- OpenAI Whisper allows me to use cpu device on the command line, but forces cuda in interpreter and fails
- How to create a requirements.txt file in Django project?
- Segmentation fault when importing torch-sparse (installing pytorch-geometric)
- File not found by Python openai/whisper
- fbprophet/Prophet fit function aborts and restarts Anaconda/Spyder Python kernel
- dask distributed grabing all available memory + swap
- How do I install openCV via anaconda at mac ?
- importing python modules in openshift