Difference between CRF and Fully Connected CRF?

Question

Can anyone explain me the difference between Conditional Random Fields and Fully Connected Conditional Random Fields for semantic segmentation? I only understand so far, that with CRF you try to use two kinds of information to improve the segmentation mask:

1. Pixel Intensity: A good guess for a edge between to classes is pixel intensity. So with this we can weight the edges of objects
2. Pixel proximity: For pixel close together, there is a high chance that they blong to the same class. With out this, we would interpret edges inside a object at instances of another class, e.g. backround.

Are my assumptions right? Is this true for CRFs or Fully connected CRFs, or both?

Thanks!

Answer 1

You are kind of correct, but I will try to make it more precise and explain their differences.

First, Conditional Random Fields (CRFs) is a graphical model for classification where you have two penalties, one for the node classification (your item 1) and another for the edges, where the neighboring nodes disagreement are penalized (your item 2).

For image segmentation, it is usual to consider each pixel as a node in the graph, and their adjacent pixels are their neighbors (4 or 8-neighbors in a 2D image), the edges' weights will try to enforce those adjacenct pixels will have similar labels. The graph resulting is very sparse, and the CRF computation is fast.

When the CRF is fully connected, every node is adjacent to each other, this makes the computation much more expensive! However, it was found in [1], that the optimization can be done efficiently in image graphs with gaussian edge weights. In this case, you are not only considering each pixel's neighborhood to obtain its class but every other pixel in the image.

You can find more information about this in [1].

[1] Krähenbühl, P., & Koltun, V. (2011). Efficient inference in fully connected CRFs with gaussian edge potentials. In Advances in neural information processing systems (pp. 109-117).