How to generate video thumbnails from not displayed video in objective-C

Question

For an iOS project in objective-C, I need to display thumbnails of rtsp video streams that are not displayed on screen (I can't take a screenshot). My view display a video stream and three thumbnails (periodically refreshed) of other streams so that the user can switch between them.

So, how can I generate this thumbnails? I'm using MobileVLCKit to display my main video stream (and I would prefer to use this framework to generate my thumbnails... if it's possible).

I found many results in forums and Google results, but none matching my case: streaming + not displayed video.

Thanks for your help!

Edit :

I tried VLC's thumbnailer too but without success:

VLCMedia *media = [VLCMedia mediaWithURL:[NSURL URLWithString:<Rtsp Url>]];
thumbnailer = [VLCMediaThumbnailer thumbnailerWithMedia:media andDelegate:self];

But I always fall in mediaThumbnailerDidTimeOut delegate method.

Answer 1

basically all you need to do is use the VLCKit 'VLCMediaThumbnailer' class and implement the two functions required by the 'VLCMediaThumbnailerDelegate'

here is an Objective-C example:

@implementation DummyObject <VLCMediaThumbnailerDelegate>

- (void)workerMethod
{
    NSURL *url = [NSURL urlWithString:@""];
    VLCMedia *media = [VLCMedia mediaWithURL:url];

    VLCMediaThumbnailer *thumbnailer = [VLCMediaThumbnailer thumbnailerWithMedia:media delegate:self];

    CGSize thumbSize = CGSizeMake(800.,600.);
    thumbnailer.thumbnailWidth = thumbSize.width;
    thumbnailer.thumbnailHeight = thumbSize.height;

    [thumbnailer fetchThumbnail];
}

- (void)mediaThumbnailer:(VLCMediaThumbnailer *)mediaThumbnailer didFinishThumbnail:(CGImageRef)thumbnail
{
    if (thumbnail) {
        UIImage *thumbnailImage = [UIImage imageWithCGImage:thumbnail scale:[UIScreen mainScreen].scale orientation:UIImageOrientationUp];
        if (thumbnailImage) {
            // do something with the thumbnail
        }
    }
}

- (void)mediaThumbnailerDidTimeOut:(VLCMediaThumbnailer *)mediaThumbnailer
{
     // show error generating thumbnail
}

@end

credits for the Objective-C code

this is how it should look in swift and SwiftUI

func generateThumbnails()
{
    if mediaPlayer.media != nil
    {
        let vlcMediaThumbnailer: VLCMediaThumbnailer = VLCMediaThumbnailer(media: mediaPlayer.media, andDelegate: self)
        vlcMediaThumbnailer.thumbnailWidth = 440
        vlcMediaThumbnailer.thumbnailHeight = 225
        vlcMediaThumbnailer.snapshotPosition = 0.5 // get thumbs from the middle of the video

        vlcMediaThumbnailer.fetchThumbnail()
    }
}

func mediaThumbnailerDidTimeOut(_ mediaThumbnailer: VLCMediaThumbnailer!)
{
    print("failed generating thumbnail")
}

func mediaThumbnailer(_ mediaThumbnailer: VLCMediaThumbnailer!, didFinishThumbnail thumbnail: CGImage!)
{
    print("got another thumbnail - do something with it")
}

and here is a complete working demo on tvOS 14.7 and apple-tv in SwiftUI for thumbnails in VLCKit:

import SwiftUI
import AVFoundation

class VideoModel: ObservableObject
{
    @Published var thumbnails: [Image] = []
    var videoUrl: String = ""
    let thumbnailWidth: CGFloat = 400.0
    let thumbnailHeight: CGFloat = 225.0
}

struct VLCPlayerTestView: View
{
    @State var isPlayVideo: Bool = false
    
    let videoUrl: String = "https://upload.wikimedia.org/wikipedia/commons/transcoded/c/c0/Big_Buck_Bunny_4K.webm/Big_Buck_Bunny_4K.webm.480p.vp9.webm"
    var videoPlayer: VideoPlayer? = nil
    @ObservedObject var videoModel = VideoModel()
    
    init()
    {
        videoModel.videoUrl = videoUrl
        videoPlayer = VideoPlayer(videoModel: videoModel)
    }
    
    var body: some View
    {
        HStack(spacing: 20)
        {
            ForEach(0..<videoModel.thumbnails.count, id: \.self) { itemIndex in
                videoModel.thumbnails[itemIndex]
                    .resizable()
                    .aspectRatio(contentMode: .fill)
                    .frame(width: videoModel.thumbnailWidth, height: videoModel.thumbnailHeight, alignment: Alignment.center)
                    .focusable()
                    .onLongPressGesture(minimumDuration: 0.01) {
                        isPlayVideo = true
                    }
            }
        }
        .fullScreenCover(isPresented: $isPlayVideo)
        {
            videoPlayer
                .edgesIgnoringSafeArea(.all)
                .onAppear {
                    videoPlayer!.player.playVideo()
                }
                .onDisappear {
                    videoPlayer!.player.stopVideo()
                }
        }
    }
}

struct VideoPlayer: UIViewRepresentable
{
    var player: VLCPlayerView
    
    init(videoModel: VideoModel)
    {
        player = VLCPlayerView()
        player.videoModel = videoModel
        player.loadMedia()
        player.generateThumbnails()
    }

    func updateUIView(_ uiView: UIView, context: UIViewRepresentableContext<VideoPlayer>)
    {
    }

    func makeUIView(context: Context) -> UIView
    {
        return player
    }
}




class VLCPlayerView: UIView, VLCMediaPlayerDelegate, VLCMediaThumbnailerDelegate
{
    var videoModel: VideoModel?
    let mediaPlayer: VLCMediaPlayer = VLCMediaPlayer()

    override init(frame: CGRect)
    {
        super.init(frame: frame)
    }
    
    func loadMedia()
    {
        if !videoModel!.videoUrl.isEmpty
        {
            mediaPlayer.media = VLCMedia(url: URL(string: videoModel!.videoUrl)!)
            mediaPlayer.delegate = self
            mediaPlayer.drawable = self
            generateThumbnails()
        }
    }
    
    func playVideo()
    {
        if mediaPlayer.media != nil
        {
            mediaPlayer.play()
        }
    }
    
    func pauseVideo()
    {
        if mediaPlayer.media != nil && mediaPlayer.isPlaying
        {
            mediaPlayer.pause()
        }
    }
    
    func stopVideo()
    {
        if mediaPlayer.media != nil
        {
            mediaPlayer.stop()
        }
    }

    required init?(coder: NSCoder)
    {
      fatalError("init(coder:) has not been implemented")
    }

    override func layoutSubviews()
    {
      super.layoutSubviews()
    }
     
    func generateThumbnails()
    {
        if mediaPlayer.media != nil
        {
            let vlcMediaThumbnailer: VLCMediaThumbnailer = VLCMediaThumbnailer(media: mediaPlayer.media, andDelegate: self)
            vlcMediaThumbnailer.thumbnailWidth = videoModel!.thumbnailWidth
            vlcMediaThumbnailer.thumbnailHeight = videoModel!.thumbnailHeight
            vlcMediaThumbnailer.snapshotPosition = 0.5

            vlcMediaThumbnailer.fetchThumbnail()
        }
    }

    func mediaThumbnailerDidTimeOut(_ mediaThumbnailer: VLCMediaThumbnailer!)
    {
        print("failed generating thumbnail")
    }
    
    func mediaThumbnailer(_ mediaThumbnailer: VLCMediaThumbnailer!, didFinishThumbnail thumbnail: CGImage!)
    {
        print("got another thumbnail")
        if(thumbnail != nil)
        {
            let thumbnailUIImage = UIImage(cgImage: thumbnail)
            let image = Image(uiImage: thumbnailUIImage).renderingMode(.original)
            videoModel!.thumbnails.append(image)
        }
    }
}

Result: