A short quiz on machine learning in computer vision

Adrian F. Clark

1. How goes a real-valued genetic algorithm work?

   • it multiplies together randomly-chosen parameters
   • it repeatedly sets parameters to random values
   • it adds together randomly-chosen parameters

   • it interpolates along lines in parameter space by a random amount

   Real-valued GAs work by choosing two 'genes' randomly to give two points in parameter space, then interpolating along the line joining them by a random amount.

2. What is the purpose of the RELU layer in a CNN?

   • it applies the activation function

   • it applies the convolution coefficients to an image
   • it down-samples regions of its input
   • it determines the final result

   The RELU layer applies the activation function to each element independently.

3. What is genetic programming?

   • a program that implements a genetic algorithm
   • a program that processes genetic data

   • the use of a genetic algorithm to 'write' a program

   • the execution of parse trees

   Genetic programming is the use of a genetic algorithm to 'write' programs. It combines program fragments in the form of parse trees into these programs, which it evaluates by running them.

4. What is transfer learning?

   • Using features from a pre-trained network as input to a ML algorithm

   • transforming the networks inputs to avoid over-fitting
   • the way in which a deep network is trained
   • adapting a network trained on one thing to recognise another

   As deep networks take so long to train from scratch, people now widely use transfer learning instead. This takes the values from some layers of a pre-trained network and uses them as the inputs to a separate machine learning stage, not necessarily even a neural network. In a very real sense, the deep network is being used as a feature detector.

5. What is the purpose of the POOL layer in a CNN?

   • it applies the convolution coefficients to an image

   • it down-samples regions of its input

   • it applies the activation function
   • it determines the final result

   The POOL layer reduces the size of its inputs, usually by down-sampling but sometimes by averaging.

6. In a deep neural network, what does the 'deep' refer to?

   • it takes many inputs
   • it allows many generations when training
   • it produces many outputs

   • it has many hidden layers

   A 'deep' network is one with many hidden layers. Modern networks often have >100 hidden layers...and take a Very Long time to train.

7. What is over-training in neural networks?

   • where the number of training epochs is too large
   • where the network recognizes only general patterns
   • where the network fails to converge

   • where the network recognizes specific patterns and does not generalize

   Over-training is when the neural network 'learns' each input pattern but is unable to classify correctly similar patterns: it has 'forgotten' the general pattern by focussing on the specific training ones.

8. k-means works by:

   • assigning data points to clusters and using a MLP on them
   • assigning data points to clusters and using SVM on them
   • assigning data points to clusters and performing PCA on them

   • alternating between assigning data points to clusters and finding centroids

   k-means is iterative, alternating between assigning points to clusters based on their curent centroids and then finding the centroids of the clusters. The other answers might actually work better but they aren't k-means!

9. At its simplest, WISARD consists of:

   • some RAM
   • image sampling, some RAM

   • image sampling, a register, some RAM

   • a shift register and some RAM

   WISARD works by sampling points in an image, storing zero or unity in corresponding points of a register, and then using that register to index into some RAM. The diagram in figure 10.2 of the lecture notes makes this clear.

10. Why might one "polish" the result of a genetic algorithm?

    • because the GA gives the wrong result
    • because the GA gives its result in the wrong coordinate system
    • because a numerical optimizer is too inaccurate

    • to improve the accuracy of the solution

    A GA is excellent at exploring the breadth of a search space and finding an approximate solution but it is less effective at determining the exact solution. Using the GA's result to initialize a numerical optimizer, which is what "polishing" means here, combines the best of both algorithms.