CODEWITHSUNDEEP
c++opencvreact-native
Abdoo Fathy
Abdoo Fathy

Reputation: 57

what's the best approach to design this simple ReactNative AR app?

I'm trying to write a simple AR app in ReactNative, it should simply see 4 predefined markers and draw a rectangle as a boundary on the live preview of the camera, the thing is I'm trying to do the processing in C++ using opencv so as to have the logic of the app in one place accessible to both Android & IOS. here's what I've been thinking

  1. write the OS dependent code to open the camera and get permissions in (java/ObjC) & the C++ part to do processing on each frame.

  2. call the C++ code (from within the native code) on each frame, and that should return lets say coordinates for the markers.

  3. draw the rect if 4 markers found on the preview in native code (No idea how to achieve this so far but I think it will be native code).

  4. expose that preview (the live preview with the drawn view) to ReactNative (Not sure about that or how to achieve it) I've looked at the react native camera component but it doesn't provide access to frames & if that's even possible, I'm not sure if it would be a good idea to send frames over the bridge between JS & java/ObjC. the problem is that I'm not sure of the performance or if that is even possible. if you know of any ReactNative library that would be great.

Upvotes: 3

Views: 339

Answers (1)

jpetrichsr
jpetrichsr

Reputation: 247

Your steps seem sound. After processing the frame in C++, you will need to set the application properties RCTRootView.appProperties in iOS, and emit an event using RCTDeviceEventEmitter on Android. So, you will need an Objective-C wrapper for your C++ code on iOS and a Java wrapper on Android. In either case, you should be able to use the same React Native code for actually drawing the rectangle on top of the camera preview. You're right that the React Native camera component does not have an API for getting individual frames from the camera, so you'll need to write that code natively for each platform.

Upvotes: 2

Related Questions