CODEWITHSUNDEEP
iosreactjstensorflowcapacitorface-recognition
Omar
Omar

Reputation: 181

Error Loading face-api.js Models: Tensor Shape Mismatch on iOS App with React and Capacitor

I am building an iOS app using React and Capacitor that uses face-api.js for facial recognition. The app runs fine in a web environment both locally and on prod for both windows and IOS. However, I encounter issues when loading the face-api.js models on xcode in development mode. Specifically, I’m getting a tensor shape mismatch error that I can't seem to resolve.

Error:

This error occurs as soon as the models are loaded, even before I attempt to use them for scanning faces. Here’s the full error message:

Error loading face-api.js models:
Error: Based on the provided shape, [1,1,512,9], the tensor should have 4608 values but has 1324
    at lt — util.ts:108
    at $r — tensor_ops.ts:87
    at o — io_utils.ts:175
    at Yd — io_utils.ts:116
    ...
    at AddEmployeePage.js:38

What I’ve Tried:

1- Model Loading Code:

I’m using the following code to load models from a CDN. This works perfectly on the web and android apps, but throws the error above on iOS.

const loadModels = async () => {
    try {
        const modelUrl = "https://cdn.jsdelivr.net/gh/justadudewhohacks/[email protected]/weights";
        await faceapi.nets.ssdMobilenetv1.loadFromUri(modelUrl);
        await faceapi.nets.faceRecognitionNet.loadFromUri(modelUrl);
        await faceapi.nets.faceLandmark68Net.loadFromUri(modelUrl);
        console.log("Models loaded successfully");
    } catch (err) {
        console.error("Error loading face-api.js models:", err.message);
    }
};

2- Loading from Local Files:

I've tried loading models from the app’s local filesystem, but the same issue persists. I ensured that all model files are correctly placed and accessible.

3- Alternative Loading Methods:

I also attempted to load models via an offline CDN with no difference in outcome.

Additional Information:

Environment:

Objective:

I need the facial recognition to function offline as well as in web environments. The app will be deployed on multiple iOS devices and must handle real-time facial recognition efficiently.

If you have any questions for me, please don't hesitate to ask!

Any insights or workarounds would be greatly appreciated!

Upvotes: 0

Views: 129

Answers (1)

Omar
Omar

Reputation: 181

Alright so apparently there are two ways to fix this. First, is the long way (not recommended) which is to use FileZilla to convert your models to binary, but it's such a pain since you need to connect to your server; provide, host, username, password and port. Along with having to install the Application itself.

Instead what you can do is just simply renaming your "shard" models by adding a .bin at the end.

For example instead of naming it "face_recognition_model-shard1" you name it "face_recognition_model-shard1.bin"

Something that you might forget to do is changing the paths in you .json files to :

paths": ["face_landmark_68_model-shard1.bin"]

Thanks for reading and hope that helped!

Upvotes: 0

Related Questions