CODEWITHSUNDEEP
pythondeep-learninghuggingface-transformersnameerror
Octavio Mesa Varona
Octavio Mesa Varona

Reputation: 1

I cannot run the KOSMOS 2 model from huggingface in colab

I am trying to run the kosmos 2 model for image recognition and description. I am trying to execute the code provided in huggingface for its execution but I get an error:

name 'Kosmos2Tokenizer' is not defined

It seems to be related to the Kosmos2Tokenizer class. This error occurs because the AutoProcessor tries to instantiate the tokenizer for the model ydshieh/kosmos-2-patch14-224, but does not find the required tokenizer class. I tried to execute the code provided in hugging face (https://huggingface.co/ydshieh/kosmos-2-patch14-224).

Code:

import requests
from PIL import Image
from transformers import AutoProcessor, AutoModelForVision2Seq

# Load model and processor
model = AutoModelForVision2Seq.from_pretrained("ydshieh/kosmos-2-patch14-224", trust_remote_code=True)
processor = AutoProcessor.from_pretrained("ydshieh/kosmos-2-patch14-224", trust_remote_code=True)

Result:

NameError                                 Traceback (most recent call last)

<ipython-input-6-728393258997> in <cell line: 8>()
      6 
      7 model = AutoModelForVision2Seq.from_pretrained("ydshieh/kosmos-2-patch14-224", trust_remote_code=True)
----> 8 processor = AutoProcessor.from_pretrained("ydshieh/kosmos-2-patch14-224", trust_remote_code=True)
      9 
     10 prompt = "<grounding>An image of"

14 frames

~/.cache/huggingface/modules/transformers_modules/ydshieh/kosmos-2-patch14-224/ada61181d314e7e7c5ff5656322e03a3f1912ba8/tokenization_kosmos2_fast.py in Kosmos2TokenizerFast()
    104     max_model_input_sizes = PRETRAINED_POSITIONAL_EMBEDDINGS_SIZES
    105     model_input_names = ["input_ids", "attention_mask"]
--> 106     slow_tokenizer_class = Kosmos2Tokenizer
    107 
    108     def __init__(
NameError: name 'Kosmos2Tokenizer' is not defined

After following the setup steps mentioned on kosmos official repo, I am using this full code in colab but after waiting a long time (using CPU and als GPU) it does not work. I would like to check the description image feature of KOSMOS2 model with an image. This is the full pipeline.

Clone the repo

!git clone https://github.com/microsoft/unilm.git %cd unilm/kosmos-2

Install required packages

!pip install -r requirements.txt

Install NVIDIA/apex

!git clone https://github.com/NVIDIA/apex %cd apex

Install apex

!pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

Move back to the original directory

%cd ..

Set up the packages (if applicable)

!bash vl_setup_xl.sh

Upvotes: 0

Views: 593

Answers (1)

gil
gil

Reputation: 2552

in order to get it running you need to follow the setup steps mentioned on kosmos official repo:

from the repo, here are the steps:

  1. Clone the repo: bash git clone https://github.com/microsoft/unilm.git cd unilm/kosmos-2
  2. Create conda env and install packages: bash conda create -n kosmos-2 python=3.9 conda activate kosmos-2 pip3 install -r requirements.txt
  3. Install NVIDIA/apex
# if pip >= 23.1 (ref: https://pip.pypa.io/en/stable/news/#v23-1) which supports multiple `--config-settings` with the same key...  pip
install -v --disable-pip-version-check --no-cache-dir
--no-build-isolation --config-settings "--build-option=--cpp_ext" --config-settings "--build-option=--cuda_ext" ./
# otherwise pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --global-option="--cpp_ext" --global-option="--cuda_ext" ./ ```

4. Install the packages: ```bash bash vl_setup_xl.sh ```

Upvotes: 0

Related Questions