RuntimeError: The size of tensor a (1024) must match the size of tensor b (640) at non-singleton dimension 1

Question

I do the instructions below: We provide an example below for training an open-vocabulary object detector on COCO dataset, with pretrained RegionCLIP (ResNet50) as the initialization.

Before training, please prepare our pretrained RegionCLIP model and set up the dataset.

Check MODEL_ZOO.md to
download the pretrained RegionCLIP checkpoint regionclip_pretrained-cc_rn50.pth to the folder ./pretrained_ckpt/regionclip,
download the trained RPN checkpoint rpn_coco_48.pth to the folder ./pretrained_ckpt/rpn,
download the class embeddings coco_48_base_cls_emb.pth and coco_65_cls_emb.pth to the folder ./pretrained_ckpt/concept_emb.
Check datasets/README.md to set up COCO dataset.
After preparation, run the following script to train an open-vocabulary detector.

python3 ./tools/train_net.py \

--num-gpus 1 \
--config-file ./configs/COCO-InstanceSegmentation/CLIP_fast_rcnn_R_50_C4_ovd.yaml \
MODEL.WEIGHTS ./pretrained_ckpt/regionclip/regionclip_pretrained-cc_rn50.pth \
MODEL.CLIP.OFFLINE_RPN_CONFIG ./configs/COCO-InstanceSegmentation/mask_rcnn_R_50_C4_1x_ovd_FSD.yaml \
MODEL.CLIP.BB_RPN_WEIGHTS ./pretrained_ckpt/rpn/rpn_coco_48.pth \
MODEL.CLIP.TEXT_EMB_PATH ./pretrained_ckpt/concept_emb/coco_48_base_cls_emb.pth \
MODEL.CLIP.OPENSET_TEST_TEXT_EMB_PATH ./pretrained_ckpt/concept_emb/coco_65_cls_emb.pth \

I want to train my own dataset and categories, so I modified the dataset configuration in the configs and have already changed the `NUM_CLASSES`.

And I got error:
Traceback (most recent call last):
  File "/home/u9347822/RegionCLIP/./tools/train_net.py", line 173, in <module>
    launch(
  File "/home/u9347822/RegionCLIP/detectron2/engine/launch.py", line 67, in launch
    mp.spawn(
  File "/home/u9347822/.local/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 230, in spawn
    return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
  File "/home/u9347822/.local/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 188, in start_processes
    while not context.join():
  File "/home/u9347822/.local/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 150, in join
    raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 0 terminated with the following error:
Traceback (most recent call last):
  File "/home/u9347822/.local/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 59, in _wrap
    fn(i, *args)
  File "/home/u9347822/RegionCLIP/detectron2/engine/launch.py", line 125, in _distributed_worker
    main_func(*args)
  File "/home/u9347822/RegionCLIP/tools/train_net.py", line 161, in main
    trainer = Trainer(cfg)
  File "/home/u9347822/RegionCLIP/detectron2/engine/defaults.py", line 377, in __init__
    model = self.build_model(cfg)
  File "/home/u9347822/RegionCLIP/detectron2/engine/defaults.py", line 517, in build_model
    model = build_model(cfg)
  File "/home/u9347822/RegionCLIP/detectron2/modeling/meta_arch/build.py", line 22, in build_model
    model = META_ARCH_REGISTRY.get(meta_arch)(cfg)
  File "/home/u9347822/RegionCLIP/detectron2/config/config.py", line 174, in wrapped
    explicit_args = _get_args_from_config(from_config_func, *args, **kwargs)
  File "/home/u9347822/RegionCLIP/detectron2/config/config.py", line 229, in _get_args_from_config
    ret = from_config_func(*args, **kwargs)
  File "/home/u9347822/RegionCLIP/detectron2/modeling/meta_arch/clip_rcnn.py", line 146, in from_config
    roi_heads = build_roi_heads(cfg, backbone.output_shape())
  File "/home/u9347822/RegionCLIP/detectron2/modeling/roi_heads/roi_heads.py", line 43, in build_roi_heads
    return ROI_HEADS_REGISTRY.get(name)(cfg, input_shape)
  File "/home/u9347822/RegionCLIP/detectron2/config/config.py", line 174, in wrapped
    explicit_args = _get_args_from_config(from_config_func, *args, **kwargs)
  File "/home/u9347822/RegionCLIP/detectron2/config/config.py", line 229, in _get_args_from_config
    ret = from_config_func(*args, **kwargs)
  File "/home/u9347822/RegionCLIP/detectron2/modeling/roi_heads/clip_roi_heads.py", line 101, in from_config
    ret["box_predictor"] = FastRCNNOutputLayers(
  File "/home/u9347822/RegionCLIP/detectron2/config/config.py", line 175, in wrapped
    init_func(self, **explicit_args)
  File "/home/u9347822/RegionCLIP/detectron2/modeling/roi_heads/fast_rcnn.py", line 453, in __init__
    self.cls_score.weight.copy_(pre_computed_w)
RuntimeError: The size of tensor a (1024) must match the size of tensor b (640) at non-singleton dimension 1

I tried to adjust MIN_SIZE_TRAIN in the configs from (640, 672, 704, 736, 768, 800) to (1024, 672, 704, 736, 768, 800), but I still got the same error.