CODEWITHSUNDEEP
computer-visionimage-segmentationvideo-tracking
Crazed
Crazed

Reputation: 117

Multi-Object Tracking from Segmentation Masks

Say I already have segmentation masks of some (biological) cells that I want to track; these cells are fairly similar to one another but do still vary slightly in shape and may also change shape as they are moving. Importantly, these cells do not divide (although they may overlap). My issue is most microscopy-focused tracking algorithms (e.g. trackpy, Ilastik) I've seen only use the cell's/object's centroid for tracking, which leads to mistakes when other cells are nearby. The mistakes are obvious to me since I can tell from the cells' shapes which one is which.

My question is, do people have recommendations for papers/packages that track multiple segmentation masks (preferably from cells but not necessarily) and also take into account the structure of the object (e.g. it's shape, brightness, etc.)? Preferably, these methods should be relatively robust to masks occasionally disappearing for a frame or two or being obscured.

Upvotes: 0

Views: 732

Answers (1)

Mike B
Mike B

Reputation: 3476

What you are looking for is a tracking algorithm that takes motion and appearance into consideration in the association process. The top two methods with these characteristics on paperswithcode, as of today, are:

But don't discard other advanced motion only solutions as they are usually good enough and less complex that than the combined ones.

DISCALIMER. I am the owner of this repo: https://github.com/mikel-brostrom/yolov8_tracking. There you will find such methods and hparam tracker tuning for your custom dataset in order to achieve optimal results

Upvotes: 0

Related Questions