Machine Learning 2 - Neural Networks¶

In this lab, we will use simple Neural Networks to classify the images from the simplified CIFAR-10 dataset. We will compare our results with those obtained with Decision Trees and Random Forests.

Lab objectives¶

Classification with neural networks
Influence of hidden layers and of the selected features on the classifier results

In [1]:

Copied!

from lab_tools import CIFAR10, evaluate_classifier, get_hog_image
        
dataset = CIFAR10('../../extern_data/CIFAR10/')
from lab_tools import CIFAR10, evaluate_classifier, get_hog_image
        
dataset = CIFAR10('../../extern_data/CIFAR10/')

Pre-loading training data
Pre-loading test data

We will use the Multi-Layer Perceptron implementation from scikit-learn, which is only available since version 0.18. You can check which version of scikit-learn is installed by executing this :

In [2]:

Copied!

import sklearn
print(sklearn.__version__)
import sklearn
print(sklearn.__version__)

0.23.2

If you have version 0.17 or older, please update your scikit-learn installation (for instance, with the command pip install scikit-learn==0.19.1 in the terminal or Anaconda prompt)

Build a simple neural network¶

Using the MLPClassifier from scikit-learn, create a neural network with a single hidden layer.
Train this network on the CIFAR dataset.
Using cross-validation, try to find the best possible parameters.

In [3]:

Copied!

from sklearn.neural_network import MLPClassifier

## -- Your code here -- ##
from sklearn.neural_network import MLPClassifier

## -- Your code here -- ##

In [4]:

Copied!

from sklearn.model_selection import StratifiedKFold

kf = StratifiedKFold(5)

for train,test in kf.split(dataset.train['hog'], dataset.train['labels']):
    train_x = dataset.train['hog'][train]
    train_y = dataset.train['labels'][train]
    
    test_x = dataset.train['hog'][test]
    test_y = dataset.train['labels'][test]
from sklearn.model_selection import StratifiedKFold

kf = StratifiedKFold(5)

for train,test in kf.split(dataset.train['hog'], dataset.train['labels']):
    train_x = dataset.train['hog'][train]
    train_y = dataset.train['labels'][train]
    
    test_x = dataset.train['hog'][test]
    test_y = dataset.train['labels'][test]

Add hidden layers to the network.¶

Try to change the structure of the network by adding hidden layers. Using cross-validation, try to find the best architecture for your network.

In [5]:

Copied!

## -- Your code here -- ##

## -- Your code here -- ##