CODEWITHSUNDEEP
iosswiftvideoavfoundation
user2232305
user2232305

Reputation: 337

Acessing Video Frames in Real Time iOS

I have to access video from a server and I want to be able to acess its frames in real time so that I can do some image processing on them.

I know how to do it offline using the AVAssetReader but this solution does not work for real time processing.

Upvotes: 4

Views: 2643

Answers (1)

Gordon Childs
Gordon Childs

Reputation: 36074

You can access frames of a remote asset in realtime using AVPlayerItemVideoOutput (with thanks to objc.io).

p.s. always check your remote asset isn't being blocked by App Transport Security.

p.p.s AVAssetReader's support for remote assets is complicated.

import UIKit
import AVFoundation

class ViewController: UIViewController {        
    let player = AVPlayer(url: URL(string: "http://0.s3.envato.com/h264-video-previews/80fad324-9db4-11e3-bf3d-0050569255a8/490527.mp4")!)
    let videoOutput = AVPlayerItemVideoOutput(pixelBufferAttributes: [String(kCVPixelBufferPixelFormatTypeKey): NSNumber(value: kCVPixelFormatType_32BGRA)])

    override func viewDidLoad() {
        super.viewDidLoad()

        player.currentItem!.add(videoOutput)
        player.play()

        let displayLink = CADisplayLink(target: self, selector: #selector(displayLinkDidRefresh(link:)))
        displayLink.add(to: RunLoop.main, forMode: RunLoopMode.commonModes)
    }

    func displayLinkDidRefresh(link: CADisplayLink) {
        let itemTime = videoOutput.itemTime(forHostTime: CACurrentMediaTime())

        if videoOutput.hasNewPixelBuffer(forItemTime: itemTime) {
            if let pixelBuffer = videoOutput.copyPixelBuffer(forItemTime: itemTime, itemTimeForDisplay: nil) {
                print("pixelBuffer \(pixelBuffer)") // yay, pixel buffer
                let image = CIImage(cvImageBuffer: pixelBuffer) // or maybe CIImage?
                print("CIImage \(image)")
            }
        }
    }
}

Upvotes: 9

Related Questions