CODEWITHSUNDEEP
webpackthree.js3dmediapipe
shadowsandlights
shadowsandlights

Reputation: 163

How to feed Mediapipe data into Three.js

my problem is a Frontend issue. I use Three.js and @Mediapipe/Hands whereby I feed the data from the latter to Three.js. This works fine in simple applications, but as soon as I do some specific changes to the Three.js code, the Mediapipe framework throws some error and I can't follow the pattern. An example when this error occurs is when I use a bigger glTF file which is being loaded into the scene with the GLTFLoader, while the other smaller glTF does not lead to an error for Mediapipe and everything works fine. But also sometimes smaller changes like having three instead of two lights lead to an error although this only occurs sometimes and I could not figure out why. The error being thrown is visible in the images.

For Mediapipe I followed the instructions as on https://google.github.io/mediapipe/solutions/hands.html using CDN to get the necessary files: const hands = new Hands({locateFile: (file) => { return https://cdn.jsdelivr.net/npm/@mediapipe/hands/${file}; }});

This was leading to the error which you can see in figure Error with CDN This error I could read very well as only some file was missing and with the following approach this error was gone.

To solve this, I tried using the files of @Mediapipe/Hands locally with NPM and Webpack. This also first worked fine but as with the other solution, different Three.js implementations lead to an error which you can see here: Error with NPM

As both error occured for small changes in my Three.js app I assume that there may be some correlation. I don't know where to start debugging and can't put the error in context, so I was hoping that someone may have some idea here. My Three.js app does not include a lot which is what confuses me as my (rookie) assumption is that there is something with the Promises within the Mediapipe files which are being confused by the Three.js processing, although my Three.js app really does not have a lot of elements.

Upvotes: 0

Views: 544

Answers (1)

Riccardo Volpe
Riccardo Volpe

Reputation: 1633

Try with:

const hands = new Hands({
    locateFile: (file) => {
        return `https://cdn.jsdelivr.net/npm/@mediapipe/[email protected]/${file}`;
    },
});

or

const hands = new Hands({
    locateFile: (file) => {
        return `https://cdn.jsdelivr.net/npm/@mediapipe/[email protected]/${file}`;
    },
});

or replace url with https://cdn.jsdelivr.net/npm/@mediapipe/hands/hands.js

Upvotes: 0

Related Questions