CODEWITHSUNDEEP
node.jstensorflowmachine-learningautomationtensorflow.js
Dexter
Dexter

Reputation: 1796

Can we train tensorflow.js to get certain part of image?

My Client has requirement that user will upload image using that image i have to crop some part of image to do that first i have used tesseract.js node module but it's not worked so i moved to tensorflow.js to create my own model and train to crop certain part of images. I want to crop below image signature part and photo. So using object detection of tensorflow can we detect signature and get x and y coordinates to crop images.

Image to crop

if i get coordinates using below code i can crop certain part of images.

const sharp = require("sharp");
sharp("./pancard.jpg")
  .resize(500, 300, {
    fit: sharp.fit.fill
  })
  .toFile("./pancard_new.jpg", (err, info) => {
    sharp("./pancard_new.jpg")
      .extract({ left: 35, top: 220, width: 180, height: 28 })
      .toFile("./pancard_new2.jpg", function(err) {});
  });

Upvotes: 0

Views: 1608

Answers (1)

Jason Mayes
Jason Mayes

Reputation: 301

You would need to have a lot of training data if you wanted to train a machine learning model - maybe thousands of examples of cards which are human annotated with the regions you want to detect and labelled. You can then train something like a segmentation model to extract raw pixels that match those classes, or a multi-box classifier which can give you the bounding box (but may not do so well with things at angles).

The easiest way to train your own and output to TensorFlow.js format would be to try Cloud Auto ML Vision which exports built models to TensorFlow.js. See this tutorial for object detection which gives you bounding boxes back: https://cloud.google.com/vision/automl/object-detection/docs/tensorflow-js-tutorial and then click the Node.js code tab of that tutorial to see how to use in JavaScript on the server side instead of Python.

However as others have mentioned, it may be easier to figure out how to find out where the card is, straighten it, and crop it as that may just be a simple computer vision problem vs a machine learning one.

Upvotes: 3

Related Questions