CODEWITHSUNDEEP
iphoneiosvideo
Nico AD
Nico AD

Reputation: 1709

transparent video iPhone

I have a video which was recorded on blue screen.

Is there a way to make it transparent on iOS devices? I know I can do it with pixel shaders and openGL but I m afraid that the process of decoding video frame/ uploading openGL texture and eliminate fragments with pixel shader will be too slow.

Any suggestions?

Upvotes: 2

Views: 3165

Answers (2)

Brad Larson
Brad Larson

Reputation: 170319

It sounds like you want to do some sort of chroma keying with your video. I just added the capability to do this to my GPUImage framework, which as the name indicates uses GPU-based processing to perform these operations many times faster than CPU-bound filters could.

The SimpleVideoFileFilter example in the framework shows how to load a movie, filter it, and encode it back to disk. Modifying this to perform chroma keying gives the following:

NSURL *sampleURL = [[NSBundle mainBundle] URLForResource:@"sample" withExtension:@"m4v"];
movieFile = [[GPUImageMovie alloc] initWithURL:sampleURL];

filter = [[GPUImageChromaKeyBlendFilter alloc] init];
[filter setColorToReplaceRed:0.0 green:0.0 blue:1.0];
[filter setThresholdSensitivity:0.4];

[movieFile addTarget:filter];

UIImage *inputImage = [UIImage imageNamed:@"background.jpg"];
sourcePicture = [[GPUImagePicture alloc] initWithImage:inputImage smoothlyScaleOutput:YES];
[sourcePicture addTarget:filter];

NSString *pathToMovie = [NSHomeDirectory() stringByAppendingPathComponent:@"Documents/Movie.m4v"];
NSURL *movieURL = [NSURL fileURLWithPath:pathToMovie];
movieWriter = [[GPUImageMovieWriter alloc] initWithMovieURL:movieURL size:CGSizeMake(480.0, 640.0)];
[filter addTarget:movieWriter];

[movieWriter startRecording];
[movieFile startProcessing];

[movieWriter setCompletionBlock:^{
    [filter removeTarget:movieWriter];
    [movieWriter finishRecording];
}];

The above code will load a movie from the application's resources called sample.m4v, feed it into a chroma key filter that is set to key off of pure blue with a sensitivity of 0.4, attach a background image to use for the chroma keying, and then send all that to a movie encoder which writes Movie.m4v in the application's /Documents directory.

You can adjust the threshold and specific blue tint to match your needs, as well as replace the input image with another movie or other source as needed. This process can also be applied to live video from the iOS device's camera, and you can display the results to the screen if you'd like.

On an iPhone 4 running iOS 5.0, chroma keying takes 1.8 ms (500+ FPS) for 640x480 frames of video, 65 ms for 720p frames (15 FPS). The newer A5-based devices are 6-10X faster than that for these operations, so they can handle 1080p video without breaking a sweat. I use iOS 5.0's fast texture caches for both frame uploads and retrievals, which accelerates the processing on that OS version over 4.x.

The one caution I have about this is that I haven't quite gotten the audio to record right in my movie encoding, but I'm working on that right now.

Upvotes: 3

Tark
Tark

Reputation: 5173

If you mean you want to render the video, but set the blue pixels to transparent, then the only efficient way to do this is with OpenGL. This should be easily possible for iOS devices, video decoding is handled in hardware, and I have several projects where I transfer video frames to OpenGL using glTexSubImage2D, works fine.

Upvotes: 2

Related Questions