CODEWITHSUNDEEP
iosobjective-cairplay
user3861866
user3861866

Reputation: 320

USB Connected iPhone Screen Mirroring

Apples AirPlay protocol enable mirroring the device screen/audio to a remote device ( eg. a Desktop computer ) via WiFi.

Given an iOS device USB connected to a Desktop computer, Is it possible to route AirPlay payload through the USB connection rather then using WiFi ( eg. using usbmuxd ) ?

Upvotes: 5

Views: 6809

Answers (2)

nanoscopic
nanoscopic

Reputation: 21

There are different methods of getting video from an iOS device programmatically.

The first, which is used by the mentioned ios-minicap, is to use Apple AVFoundation libraries to fetch video content. This method works, but it is rather flaky in the way it starts up. I've found that it doesn't begin to work consistently every time you run code to use it.

Another example of code that uses AVFoundation to pull video is here: https://github.com/nanoscopic/ios_video_pull

The second method of pulling video is by utilizing the underlying usbmuxd communication used by AVFoundation itself. This was reverse engineered by Daniel Paulus and released as open source code here: https://github.com/danielpaulus/quicktime_video_hack/

The third method of pulling video is by using the "Upload Broadcast Extension" feature of iOS devices, specifically with either ReplayKit or ReplayKit 2. I am currently working on an iOS app that will be released on the app store to do this via ReplayKit 2. There are various examples of this on Github but few work well.

The fourth method of pulling video is by using am HDMI dongle on the iOS device and then capturing/streaming that HDMI feed using some hardware device.

The fifth method of pulling video is to use one of the various variants of the AirPlay protocol. There are open source projects on Github capable of handling the encryption and obfuscation of this protocol but it is messy and there is little to no guarantee that those projects will continue to work in the near future. I'd recommend against them.

Upvotes: 2

Frederik Carlier
Frederik Carlier

Reputation: 4796

Newer iOS devices allow you to enable mirroring the device screen/audio via USB as well. I believe the feature was introduced in iOS 8.

You can use it on macOS using QuickTime like this:

  1. Open QuickTime
  2. Click File > New Movie Recording
  3. In the dropdown menu next to the record button, select your iPhone as the Camera

QuickTime will now display the screen of your iOS device.

You can access this using the QuickTime API; see https://github.com/openstf/ios-minicap for an example of a project accessing the iOS screen and sharing it ovre the network.

Upvotes: 2

Related Questions