CODEWITHSUNDEEP
opencvunity-game-engineocrtesseractvuforia
hotshotiguana
hotshotiguana

Reputation: 1560

Unity3D - OCR Number Recognition

Our initial use case called for writing an application in Unity3D (write solely in C# and deploy to both iOS and Android simultaneously) that allowed a mobile phone user to hold their camera up to the title of a magazine article, use OCR to read the title, and then we would process that title on the backend to get related stories. Vuforia was far and away the best for this use case because of its fast native character recognition.

After the initial application was demoed a bit, more potential uses came up. Any use case that needed solely A-z characters recognized was easy in Vuforia, but the second it called for number recognition we had to look elsewhere because Vuforia does not support number recognition (now or anywhere in the near future).

Attempted Workarounds:

  1. Google Cloud Vision - works great, but not native and camera images are sometime quite large, so not nearly as fast as we require. Even thought about using the OpenCV Unity asset to identify the numbers and then send multiple much smaller API calls, but still not native and one extra step.
  2. Following instructions from SO to use a .Net wrapper for Tesseract - would probably work great, but after building and trying to bring the external dlls into Unity I receive this error .Net Assembly Not Found (most likely an issue with the version of .Net the dlls were compiled in).
  3. Install Tesseract from source on a server and then create our own API - honestly unclear why we tried this when Google's works so well and is actively maintained.

Has anyone run into this same problem in Unity and ultimately found a good solution?

Upvotes: 1

Views: 4330

Answers (1)

David S.
David S.

Reputation: 41

Vuforia on itself doesn't provide any system to detect numbers, just letters. To solve this problem I followed the next strategy (just for numbers near of a common image):

  1. Recognize the image.
  2. Capture a Screenshot just after the target image is recognized (this screenshot must contain the numbers).
  3. Send the Screenshot to an OCR web-service and get the response.
  4. Extract the numbers from the response.
  5. Use these numbers to do whatever you need and show AR info.

This approach solves this problem, but it doesn't work like a charm. Their success depends on the quality of the screenshot and the OCR service.

Upvotes: 1

Related Questions