CODEWITHSUNDEEP
iosswiftmetal
JCutting8
JCutting8

Reputation: 774

How to resize/scale an MTLTexture

I have a MTLTexture which is the size of the MTLView it appears in. However, I am writing the texture to an AVAssetWriterInputPixelBufferAdaptor for the purpose of recording a video which requires the texture to be of a standard video size, i.e. 1280x720.

At the moment I am using MTLRegionMake2D to specify a region in the texture to extract. On a phone which is long in width, the effect of the crop is not that noticable. But on an iPad which is more square in shape and size, the crop is noticeable. Ideally, I would resize the texture to be the same width or height as the video first, and then crop the remainder (as opposed to just cropping).

What is the appropriate way to resize an MTLTexture?

Example code:

guard let pixelBufferPool = assetWriterPixelBufferInput.pixelBufferPool else { return }

var maybePixelBuffer: CVPixelBuffer? = nil
let status  = CVPixelBufferPoolCreatePixelBuffer(nil, pixelBufferPool, &maybePixelBuffer)
if status != kCVReturnSuccess { return }

guard let pixelBuffer = maybePixelBuffer else { return }

CVPixelBufferLockBaseAddress(pixelBuffer, [])
let pixelBufferBytes = CVPixelBufferGetBaseAddress(pixelBuffer)!

let bytesPerRow = CVPixelBufferGetBytesPerRow(pixelBuffer)
let region = MTLRegionMake2D(0, 0, Int(videoSize.width), Int(videoSize.height))

texture.getBytes(pixelBufferBytes, bytesPerRow: bytesPerRow, from: region, mipmapLevel: 0)

let frameTime = CACurrentMediaTime()
let presentationTime = CMTimeMakeWithSeconds(frameTime, 240)
assetWriterPixelBufferInput.append(pixelBuffer, withPresentationTime: presentationTime)

CVPixelBufferUnlockBaseAddress(pixelBuffer, [])

Upvotes: 4

Views: 2581

Answers (1)

JCutting8
JCutting8

Reputation: 774

In the end I used MPSImageLanczosScale to scale the texture, see:

Apple Docs: https://developer.apple.com/documentation/metalperformanceshaders/mpsimagelanczosscale

Example: Crop and scale MTLTexture

After re-scaling, I still needed to use a region to then crop the remainder. I had some logic to detect portrait/landscape, and tablet or phone to determine an appropriate crop. In my case (for portrait), I chose to center vertically for a skinny device (phone) cropping the top/bottom, and horizontally for a wide device (tablet) cropping the left/right. Had similar logic for landscape, but reversed.

Upvotes: 2

Related Questions