CODEWITHSUNDEEP
pythondeep-learningfast-ai
Hooked
Hooked

Reputation: 88238

How to directly infer an image with transforms using fastai?

I'm extremely new to fastai and I'm trying to adapt a prior model. I've been able to load the model and use inference only if the files are on disk. Since I'm trying to process files to a video stream, I'm saving each file and processing them one at a time, eg.

import cv2, imageio 
from fastai.vision import *

# ... load the model here

stream = cv2.VideoCapture('Die.Hard.1988.mkv')

while True:
    _, frame = stream.read()

    with tempfile.NamedTemporaryFile(suffix='.jpg') as FOUT:
        imageio.imwrite(FOUT.name, frame)
        FOUT.flush()

        x = open_image(FOUT.name)
        preds_num = learn.predict(x)[2].numpy()

This seems to work, but seems wasteful saving the image to disk. The library seems to use some transforms which means I can't directly push the image to the "learner". Here is how it's loaded:

from fastai.vision import *

f_model = "shot-type-classifier"
path = "shot_type_classifier/"

data = ImageDataBunch.from_folder(
    path,
    "train",
    "valid",
    size=(375, 666),
    ds_tfms=get_tfms(),
    bs=1,
    resize_method=ResizeMethod.SQUISH,
    num_workers=0,
).normalize(imagenet_stats)

learn = cnn_learner(data, models.resnet50, metrics=[accuracy], pretrained=True)
learn = learn.to_fp16()
learn.load(f_model)

Where the transforms seem to be defined here. I'd really just like to directly pipe images to the preprocesser and then the model without having to save them to disk. Is there a way to do that using fastai?

Upvotes: 1

Views: 3636

Answers (1)

juvian
juvian

Reputation: 16068

Try converting frame to a pillow image and then just use pil2tensor:

from PIL import Image as PImage
from fastai.vision import *

frame = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)
pil_im = PImage.fromarray(frame) 
x = pil2tensor(pil_im ,np.float32)
preds_num = learn.predict(Image(x))[2].numpy()

Upvotes: 1

Related Questions