\chapter{Conclusions}
\label{ch:conclusions}

Perhaps one of the reasons we enjoy so-called ``optical illusions'' is not because they shed some light on what might be going on inside our brains but because they demonstrate that our vision system can, just occasionally, fall short. Our vision system is usually so relentlessly flawless that its failure is something of a novelty! In emulating vision, by contrast, we encounter a seemingly endless number of difficulties, limitations and hidden complexities. In this thesis, the author has tackled a few of the classic problems in machine vision, and attempted to develop practical solutions in an automated fashion. In this final chapter, the author shall make a critical assessment of the applicability and value of the author's work. We shall start by revisiting some of the concepts that motivated the research in this thesis.

\paragraph{The Automated Production of Vision Systems}
So exceptional are our eyes and brains that certain tasks in vision feel so trivial as to be tedious. Accordingly, there are a great many tasks that we would like machines to attend to on our behalf. Indeed, if one were to sit down and enumerate all the things that we can do with our eyes, but which we'd prefer a computer to do for us, the list would rapidly become gargantuan. It appears that machine vision is one of those topics for which the number of expedient solutions is outnumbered by the volume of desirable applications. 

To develop a ``complete'' vision system on a par with our own is currently not achievable, so we must restrict our ambitions to solving one specific problem at a time. Indeed, although some commercial vision systems claim to be applicable for ``all situations'', they are in fact usually limited to a subset of tasks, usually involving industrial inspection \cite{visionsystems, cognex}. Faced with the huge amount of human expertise required to create any non-trivial vision system, comes a desire to automate production in some way, which was the primary motivation underlying the work in this thesis.

\paragraph{Extending GP Vision Research}
The author's work uses Genetic Programming (GP) as its principal learning component. Although Genetic Programming has been applied to vision problems throughout the history of its own evolution, a gap in GP machine vision research provided a second motivation for the author's study: work is usually concentrated on a single component of an overall vision system, for instance pre-processing operations, low-level components or object recognition. As such, those evolved components do not represent working vision systems; and until now, it has not been shown how a more complete, working vision system may be constructed using GP-evolved components. 

\paragraph{The Spirit of Automatic Programming}
A third aim was motivated by another feature of some GP research, which is generally to devise a highly task-specific set of features and parameters, and then show how they may be put together using Genetic Programming \cite{ebner:2001:colour-constancy, koppen:1996:image-exploring, spivak:2002:OCR, zhang:rockett:edge, graae:2000:steresoscopic}. In the author's opinion, this is somewhat contrary to the intention of machine learning, which is to have computers develop software themselves. If the learning system requires customisation for each different problem then it isn't learning enough! 

\section{Assessment}
Over the last three years, the author has developed a software framework in which the machine itself is almost entirely responsible for the technical aspects arising from the production of various vision systems. The human user may define the problem with the tools provided, but expertise in machine vision itself is generally not required. In an ideal world it would be left for a domain expert, such as a radiologist, geologist or zoologist, to define the ``truth'' in each case, leaving the task of identifying patterns to the software. In this section, the author shall critically assess his system in terms of its applicability, quality and competitiveness with other techniques, and determine how well the system meets the original motivations for this work.

In Chapter \ref{ch:gpsurvey}, we saw how many components and parameters may make up a Genetic Programming system, and in Chapter \ref{ch:gpcomparison} the author suggested how to avoid some of them! The author has tried to show how Genetic Programming can be rendered reasonably parameter-free for a variety of problems, and in certain respects this has been successful: all the results in Chapters  \ref{ch:low-level} and \ref{ch:vision-system}, for what they are worth, were obtained from a system that did not require any tuning. In the author's system, feature selection and extraction, training data creation, segmentation and shape classification are all performed automatically.

To what extent is the author's toolkit parameter-free? Is the quality of solutions compromised when the system cannot be tuned to each problem? Indeed, in many cases, such as those discussed in Chapter \ref{ch:gpcomparison} there isn't a straightforward ``best'' parameter for a particular solution. For this reason the author developed a classification system which attempts to make intelligent decisions and try out ideas, instead of delegating those decisions to the human operator. Although the binary decomposition process within the system can make this more accurate, trying our different ideas increases the learning time, although this too can be offset by using multiple CPUs. The author fancies that this is similar to the process that a human researcher might undertake while developing a system. It may well be possible to discover parameters that suit a particular problem better, but the time taken in doing so may outweigh the advantages! 

\subsection{Multi-Stage Vision Systems}
The author claims to have gone further than other GP researchers by developing vision systems that comprise two or more separate stages of processing, each using evolved components. Multi-stage evolution is not a new topic in GP, but is almost exclusively confined to refinements of the \emph{same task} \cite{roberts:1999:evdIRLI,EvoVision:Roberts-Claridge:2005,Teredesai2005}. In this thesis, the author has shown that Genetic Programming can evolve solutions to a number of image processing operations, and that each stage is sufficiently accurate as to provide meaningful data to the next stage. This would appear to accommodate the requirement of extending GP research, but the question is whether the author's systems are competitive.

\subsection{Competitiveness}
It may be that the machines have more to offer than cheap labour. Given sufficiently well-defined criteria, computers are able to try out possibilities more quickly than can a human, so systems whose development requires some degree of trial-and-error may benefit from more comprehensive exploration if studied by  computer. In general, machines are not biased towards a particular approach, so it could be submitted that their exploration of the search space is more fair. Of course, it could also be argued that they also lack the intuition necessary to concentrate their search in promising areas! Still, it may be the case, one day, that the state-of-the-art requires more work than can be accomplished by humans alone.

% To what extent is the author's vision system comparable to other vision systems?

%Evolution of a vision system by genetic algorithm, 1998
% http://uk.geocities.com/markcsinclair/ps/esnhs_sin.ps.gz

% Neural Network Shape Recognition
% http://www.ucs.louisiana.edu/~dxj2534/james_gecco05.pdf

But to what extent does the author's work represent the state-of-the-art in computer vision today? The author would readily acknowledge that the vision systems produced using his system are not as advanced as some of the approaches taken by vision researchers in recent years. It is the nature of generic systems to follow behind the leading edge to some degree. Still, the author would submit that his system has value: it can be used to generate working vision systems in a fraction of the time that  would be taken if being developed manually.

Other considerations, so elegantly managed by our own eyes and brain, such as geometric or illumination invariance, provide substantial stumbling blocks for vision systems and demand significant research in their own right. In some senses, therefore, the author's remit in this work is too broad, especially since it covers both a substantial swathe of computer vision and evolutionary computation; the reader will have noticed a relatively equal split between the two during the course of this thesis. Although issues such as colour constancy, noise reduction and others have been touched upon, the author does not claim to be an expert in these fields. Algorithms have been introduced and integrated into to the author's system, but these may be regarded as ``placeholders'' rather than advanced systems with which the author would claim satisfaction. Accordingly, the author's framework is designed such that other vision algorithms and learning techniques could be plugged in at a later date to improve its robustness. Still, one of the major benefits of using Genetic Programming is that the imperfections of other algorithms are inherently accounted for!

In some ways, to concentrate on vision \emph{systems} in general rather than vision \emph{components}, then to divide one's loyalties between machine vision and the field of evolutionary computation makes it difficult to examine everything to the detail desired. Perhaps it is better to concentrate more intently on a smaller area, although the challenge of taking on more is sometimes difficult to resist. The author would have preferred to spend more time investigating individual components in machine vision and evolutionary computation, but there is simply too much to cover during the course of three short years!

In Chapter \ref{ch:gpcomparison} we saw that the author's system was broadly competitive with other machine learning techniques on classification tasks, more so against classifiers published by GP researchers. In vision too, the author's system goes beyond what other GP researchers have published. Indeed, the author's generic system has been used to replicate and extend work recently published by other GP researchers. Still, it is acknowledged that Genetic Programming does not represent the current mainstream approach towards machine vision; indeed, some of the problems attacked by genetic programmers are simplistic. The ``edge'' offered by Genetic Programming is its ability to solve problems in an automated way; as already discussed, it appears the author's system does offer such an advantage in terms of the speed and ease with which vision systems can be produced. This benefit is most meaningful if the system can solve a host of \emph{different} tasks.

\subsection{Applicability}
As we saw in Chapter \ref{ch:low-level} and \ref{ch:vision-system}, the author's system has been applied to problems encompassing a wide variety of domains; the system was not developed with any particular application in mind. But to what extent can the author claim that his system is widely applicable? Despite claims toward genericity, and a system that doesn't require explicit tuning to particular problems, the reality is that the author's software can solve but a small fraction of the gamut of machine vision systems that exist. The field of computer vision is such that it is not easy (or perhaps possible) to find a set of metrics that can adequately describe all the possible information we might like to draw from an image. Nor is it straightforward to bolt them all together into a single architecture or interface. The further that one proceeds with computer vision, the more different applications come to mind; some can be accommodated, many cannot. There are a number of areas where the author, given time, would choose to expand his vision system with a view to increasing its applicability. Some of these are described later in Section \ref{future-directions}.

\subsection{Genetic Programming}
% How good is GP really?

Throughout the thesis, the reader may have questioned whether Genetic Programming is the most appropriate tool for developing \emph{all} domain-specific components of the system. As was discussed toward the end of Chapter \ref{ch:gpcomparison}, machine learning paradigms each exhibit their own strengths and weaknesses. Perhaps it is rather inflexible to use Genetic Programming exclusively, when other techniques may be more appropriate in certain situations or for certain tasks in general? Neural networks are widely used in machine vision for various tasks, and Genetic Algorithms are also used for feature selection and other optimisations. The author acknowledges this is a reasonable suggestion, which may benefit future versions of his software. Like much doctoral research, this work has concentrated on the development and use of a specific system, with the inevitable consequence that some more practical considerations are compromised.

That said, Genetic Programming remains a flexible and promising machine learning technique. There are few learning algorithms that are sufficiently generic to allow such a wide range of learning applications. As we saw in Chapter \ref{ch:gpcomparison}, the search space for all but the simplest tasks can grow to astronomical proportions and yet the evolutionary process can pick out competitive solutions within a matter of minutes or hours, which is quite remarkable. The fortunate thing is that while the search space grows, so does the solution space.

\section{Future Directions}
\label{future-directions}
Over the past three-and-a-half years, the author has somehow managed to write slightly over 100,000 lines of code. Still, even while putting the finishing touches to this final chapter, there are still many new possibilities and areas of endeavour that come to mind! In addition to those already hinted at so far in this chapter and previous ones, there are a number of directions for extending this work that can be recommended.

%The feature detector does have some drawbacks -- it cannot extract shapes if the objects in question are amorphous and rotatable. The author's system, for example, cannot be applied with great success to tasks such as galaxy classification \cite{galaxyzoo}. A window detection technique would be better suited to perform the feature extraction task in this instance. Later, classification of objects (in the case of galaxies, into either elliptical or spiral forms) could be performed by a shape classifier (although some of the features directly associated with shape could not be used). 

%Image of galaxies?

The most straightforward extension is to introduce additional imaging features and shape descriptors in order to enhance the system's ability to generalise. With a feature selection system and the inherent ability of GP to select useful terminals, additional descriptors could be added quite liberally. Such is the breadth of ideas conceived by computer vision researchers, that one could no doubt find many new ideas to take advantage of!

As we saw in Chapter \ref{ch:low-level}, there are certain features which work ``out of the box'', as it were; but other feature families, such as Haar-like features, need to have parameters chosen for them in order to be useful. Indeed, certain features may be combined together to produce more useful ones, a common application of principal components analysis. Genetic Programming has also been used for this purpose \cite{otero02}. Feature extraction, by which parameters for Haar-like features and others may be discovered, was touched upon in this thesis, and indeed appears to work satisfactorily, but like many other things, the author's system may benefit from a more comprehensive study.

The author's vision framework is designed to recognise two-dimensional objects, or at least objects which are captured from a consistent camera direction, although they can be rotated around an axis parallel to the camera's point-of-view. For many applications, this is sufficient, but as vision systems grow more complex, it becomes more practical to define an object by points and edges in 3D space. An intermediate layer, which takes the shapes identified by the feature detector and identifies corners and edges in 3D (perhaps using stereo image pairs), for use by the shape classifier, may help create an object recogniser that is more robust and more applicable. 

% TODO: image of 3D reconstruction by GP here? - take a stereo pair and identify the corners? Need image pairings in Jasmine - probably possible..

As has already been suggested, the author's system may be rendered more robust by more advanced colour constancy and exposure manipulation algorithms. The author's software would serve as a useful test bed for evaluating the reliability of such algorithms on a variety of different imaging problems. Although the author's system uses Genetic Programming exclusively, the nature of the system's dataset/classifier interface makes it straightforward to introduce different learning techniques to the fold. %It may be an interesting area of research to identify automatically the learning technique that best suits a particular problem.

Aside from purely vision related matters, there are a number of additions that could be made to the evolutionary system itself. The author has always considered \emph{co-evolution} to be a fine idea -- where features can be evolved alongside the algorithm itself. Binary decomposition, implemented by the author's classifier, helps guarantee that sub-solutions, once learned, can be protected, but this is dependent upon the user defining multiple classes, and does not assist in binary situations. Although the author has implemented a co-evolution system in SXGP there hasn't been enough time, alas, to investigate it thoroughly.

\subsection{Resources}
% URI is the identifier for some resource, but URL gives specific information as to obtain it. If the URL describes both the location and name of a resource, the correct term is URI.
For readers interested in implementing some of the ideas, the entirety of author's software is available online. You may either run the software directly from the web and generate vision systems of your own (a tutorial is provided) or download the source code and start on Version~2. The URI is:

\begin{quotation}
http://vase.essex.ac.uk/software/jasmine
\end{quotation}

\section{Closing Remarks}
% Mention the lack of conclusion in the field
It is clear that there are many different avenues that interested parties could explore. There is plenty more work to do on the author's system before the word ``towards'' on the title of this thesis could be safely removed. Therefore, the most suitable conclusion to this thesis is, perhaps, that there is no satisfactory conclusion so far. The field of computer vision remains open and, as yet, there is no truly generic means of approaching vision in general, short of building a complete emulation of the human brain and visual system. The author suspects this will be the ultimate realisation of machine vision. In the meantime, there is a need for more straightforward vision systems, perhaps of the type that author's software is able to create, until  such a time as a colossus can be constructed -- which may take some time yet. Don't despair though reader, in case you're recalling that your own vision system's evolution commenced some 540 million years before you started reading this thesis -- the rate of human science and discovery is such that the colossus may be here sooner than any of us can imagine.
 
\vspace{1in}

\begin{center}
\textbf{-- THE END --}
\end{center}
