A short quiz on machine learning in computer vision

Adrian F. Clark

1. What is the purpose of the RELU layer in a CNN?

   • it applies the convolution coefficients to an image

   • it applies the activation function

   • it down-samples regions of its input
   • it determines the final result

   The RELU layer applies the activation function to each element independently.

2. In Genetic Programming, programs are commonly presented in which language?

   • Java

   • Lisp

   • English
   • Python

   Programs in GP are normally presented as Lisp. In fact, in the author's GP system, programs are converted to Lisp and executed in that form: the overhead of doing this is tiny.

3. At its simplest, WISARD consists of:

   • image sampling, some RAM

   • image sampling, a register, some RAM

   • a shift register and some RAM
   • some RAM

   WISARD works by sampling points in an image, storing zero or unity in corresponding points of a register, and then using that register to index into some RAM. The diagram in figure 10.2 of the lecture notes makes this clear.

4. k-means works by:

   • assigning data points to clusters and performing PCA on them
   • assigning data points to clusters and using a MLP on them

   • alternating between assigning data points to clusters and finding centroids

   • assigning data points to clusters and using SVM on them

   k-means is iterative, alternating between assigning points to clusters based on their curent centroids and then finding the centroids of the clusters. The other answers might actually work better but they aren't k-means!

5. Which of the following is a deterministic machine learning technique?

   • SVM

   • MLP
   • WISARD
   • GP

   All the techniques listed except SVM are stochasic, meaning they involve the use of random numbers and hence normally produce different results every time they are run on the same data. SVM doesn't do this and so is deterministic, producing the same output every time the same input is used.

6. What is the most common training techinque for a neural network?

   • back propagation

   • sideways propagation
   • back and forward propoagation
   • forward propagation

   The most widely-used training technique is called back-propagation. It is a way of adjusting the weights of each neuron to reduce the size of the error when evaluating inputs.

7. In a deep neural network, what does the 'deep' refer to?

   • it allows many generations when training
   • it produces many outputs

   • it has many hidden layers

   • it takes many inputs

   A 'deep' network is one with many hidden layers. Modern networks often have >100 hidden layers...and take a Very Long time to train.

8. Which of the following is NOT a well-known deep neural network architecture?

   • DNNarch

   • ResNet
   • AlexNet
   • GoogLeNet

   AlexNet was the first deep CNN to show real promise, and this was followed by GoogLeNet. ResNet is roughly the state of the art, and RecNet is one I make up!

9. How goes a real-valued genetic algorithm work?

   • it multiplies together randomly-chosen parameters
   • it adds together randomly-chosen parameters
   • it repeatedly sets parameters to random values

   • it interpolates along lines in parameter space by a random amount

   Real-valued GAs work by choosing two 'genes' randomly to give two points in parameter space, then interpolating along the line joining them by a random amount.

10. What is over-training in neural networks?

    • where the number of training epochs is too large
    • where the network fails to converge
    • where the network recognizes only general patterns

    • where the network recognizes specific patterns and does not generalize

    Over-training is when the neural network 'learns' each input pattern but is unable to classify correctly similar patterns: it has 'forgotten' the general pattern by focussing on the specific training ones.