Pattern Recognition 3 - Points and Neighbourhoods¶
Objectives:
- Detect points of interest in an image
- Extract neighbourhood descriptors
- Use descriptors for clustering and matching
Points of interest¶
In the previous labs, we have analyzed the images either using "object descriptors" (area, eccentricity, moments of segmented object) or "image descriptors" (based on all the pixels of the image). Another way of approaching an image analysis problem is to first look for "interesting points".
What makes a point interesting will highly depend on the application. Common examples are corners, or local extrema.
Starting from the mitosis.bmp image, find or create a point detector that will detect the nuclei of the cells.
Hint: it may be easier to do so in a different color space...
import numpy as np
from matplotlib import pyplot as plt
from skimage.io import imread
%matplotlib notebook
im = imread('mitosis.bmp')
plt.figure()
plt.imshow(im)
plt.show()
## -- Your code here -- ##
Descriptors¶
The goal of descriptors is to represent the region surrounding the point of interest in a way that allow our algorithm to best perform its task. For instance, the BRIEF descriptor will compare pairs of pixels in the given neighbourhood and create a binary descriptor representing the result of all these comparison. This kind of descriptor will tend to create a unique signature in a grayscale image, and is very useful for applications such as matching for stereo-vision or for video tracking.
Other types of descriptors can be based on the texture (for instance with local Fourier transforms) or on local histograms.
- Using the points detected before, extract neighbourhood patches and compute the concatenated color histogram as a descriptor (meaning a 256+256+256 = size 768 vector, with the first 256 values corresponding to the R histogram, etc...).
- Use the KMeans class from
scikit-learnto cluster the points of interest depending on their descriptor. Can you identify clusters representing the type of tissue of the cell?
from sklearn.cluster import KMeans
def compute_histogram(patch):
rgb_histogram = np.zeros((768,))
## TODO : compute histogram ##
return rgb_histogram
def extract_descriptors(im, coords, neighbourhood_size=15):
## TODO : extract neighbourhood patch for each point coordinate,
## compute histogram and compile everything in a "descriptor matrix"
return
## -- Your code here -- ##
- Apply the same point detection and descriptors on the
mitosis2.bmpimage, and use the already trained KMeans to predict the clusters of the points in the new image.
## -- Your code here -- ##