CODEWITHSUNDEEP
javascriptandroidcomputer-visionartoolkitar.js
tgpfeiffer
tgpfeiffer

Reputation: 1858

How to improve marker detection on mobile browser with AR.js

During development I'm testing marker detection and rendering with my laptop's webcam, which gives satisfactory results, but when using my Android tablet detection is not very good.

I have found out that one difference is that my webcam has a higher resolution (1280x720), while the tablet's back camera only gives 640x480 video data according to this website: https://webrtchacks.github.io/WebRTC-Camera-Resolution/

Note that the sourceWidth, sourceHeight, displayWidth, displayHeight parameters to a-scene's arjs attribute do not seem to have any effect, no matter how high or low I set them, my laptop always prints

Allocated videoFrameSize 1228800

my tablet always prints

*** Camera Parameter resized from 640, 480. ***  
Allocated videoFrameSize 307200

Since I understood that restriction I took a video of the moving marker and used sourceType: video instead of sourceType: webcam. However, even then the quality of marker detection on the tablet is worse than on the laptop. (Is the video resolution also limited by the browser in the same way as the camera?)

I see three main issues on the tablet:

  1. The marker is not detected at all if the size is small.
  2. Marker detection is unstable, so the marker flickers on/off or orientation changes for a single frame etc.
  3. The fps is lower, marker detection takes longer, and once a marker has been detected and the corresponding entities painted, the video stream has already significantly advanced.

Are there some parameters to AR.js that can improve the recognition results? I'm thinking of something like "lower confidence threshold" to detect even smaller markers, "recognize for N subsequent frames before showing" to improve stability, "reduce framerate" to reduce the load, or similar.

On a similar note, isn't some camera calibration needed to actually compute an accurate marker orientation from the image data.

Upvotes: 0

Views: 2573

Answers (1)

Frédéric Bussière
Frédéric Bussière

Reputation: 1

Working on that aspect too. Without being perfect, these parameters where not bad (i'm testing with the marker system, with 0.75 border, to display a video):

<a-scene embedded arjs='sourceType: webcam; patternRatio: 0.75 trackingMethod: best maxDetectionRate: 60 detectionMode: mono' renderer='antialias: true; alpha: true; precision: medium;' vr-mode-ui="enabled: false" stats>

and later on

<a-marker vidhandler preset="custom" type='pattern' url='...' smoothCount=0 smoothTolerance=0.0001 >

It definitely better than the default result but my phone (S9) is still at 60 fps and I can't find where to push the parameters to track better.

Few links where I'm digging information from:

https://jeromeetienne.github.io/AR.js/aframe/

https://gitlab.giondesign.com.au/ar-and-vr/ar-js/tree/dev/aframe

https://books.google.nl/books?id=JpLnDwAAQBAJ&pg=PA54&lpg=PA54&dq=ar.js+maxDetectionRate&source=bl&ots=9zs_0le60q&sig=ACfU3U014Kfw8Pt64OKa34Y0jzPCHppxFg&hl=en&sa=X&ved=2ahUKEwiF1JDmqpbqAhXN_aQKHYAmDTQQ6AEwAnoECBQQAQ#v=onepage&q=ar.js%20maxDetectionRate&f=false

Upvotes: 0

Related Questions