CODEWITHSUNDEEP
pytorchcompiler-errorstorchvision
shushuno1
shushuno1

Reputation: 11

torchvision::nms does not exist

Im trying to run the llama3 models using nf4 data format on cpu using intel extension . I download the pytorch from https://github.com/pytorch/pytorch and use the following command to compile: DEBUG=1 python ./setup.py develop. The version is the same as before and other dependencies don't make a difference.

python run_generation_cpu_woq.py --model llama_3_8b_nf4 --woq --woq_algo GPTQ --benchmark --tasks lambada_openai,piqa,hellaswag --batch_size 4 --weight_dtype nf4 --load_in_4bit --bits 4 --iters 1
Traceback (most recent call last):
  File "/home/run_generation_cpu_woq.py", line 9, in <module>
    from intel_extension_for_transformers.transformers import (
  File "/root/miniconda3/envs/hyw2/lib/python3.10/site-packages/intel_extension_for_transformers/transformers/__init__.py", line 56, in <module>
    from .modeling import (
  File "/root/miniconda3/envs/hyw2/lib/python3.10/site-packages/intel_extension_for_transformers/transformers/modeling/__init__.py", line 21, in <module>
    from .modeling_auto import (AutoModel, AutoModelForCausalLM,
  File "/root/miniconda3/envs/hyw2/lib/python3.10/site-packages/intel_extension_for_transformers/transformers/modeling/modeling_auto.py", line 64, in <module>
    from ..llm.quantization.utils import (
  File "/root/miniconda3/envs/hyw2/lib/python3.10/site-packages/intel_extension_for_transformers/transformers/llm/quantization/utils.py", line 37, in <module>
    import intel_extension_for_pytorch as ipex
  File "/root/miniconda3/envs/hyw2/lib/python3.10/site-packages/intel_extension_for_pytorch/__init__.py", line 8, in <module>
    import torchvision
  File "/root/miniconda3/envs/hyw2/lib/python3.10/site-packages/torchvision/__init__.py", line 6, in <module>
    from torchvision import _meta_registrations, datasets, io, models, ops, transforms, utils
  File "/root/miniconda3/envs/hyw2/lib/python3.10/site-packages/torchvision/_meta_registrations.py", line 164, in <module>
    def meta_nms(dets, scores, iou_threshold):
  File "/home/files/pytorch1/pytorch/pytorch/pytorch/torch/library.py", line 467, in inner
    handle = entry.abstract_impl.register(func_to_register, source)
  File "/home/h30060564/files/pytorch1/pytorch/pytorch/pytorch/torch/_library/abstract_impl.py", line 30, in register
    if torch._C._dispatch_has_kernel_for_dispatch_key(self.qualname, "Meta"):
RuntimeError: operator torchvision::nms does not exist

Everything was normal until I compiled the debug version of Pytorch

trying to run llama3 and perf more function stack information

I have tried install the 2.4.0 develop pytorch and the same mistake appears again

torch                            2.3.0a0+git97ff6cf /home/files/pytorch1/pytorch/pytorch/pytorch
torchaudio                       2.3.0+cpu
torchvision                      0.18.0+cpu

is there other types of torchvision fit that

Upvotes: 1

Views: 2155

Answers (3)

lunar Rival
lunar Rival

Reputation: 1

Re-install torch and torchvision via your local .whl file.

Upvotes: 0

Sergio Mcfly PYK
Sergio Mcfly PYK

Reputation: 4331

I did solve it with:

pip install torchvision

Upvotes: 0

user26005124
user26005124

Reputation: 1

I tried to install torchvision from source [https://github.com/pytorch/vision.git][1], which is feasible, but it also requires the use of develop mode, similar to running "python setup.py develop".

Upvotes: 0

Related Questions