Laboratory 5: Interaction and OpenCV

Introduction

The particular program you will review and extend here stems from EPIC, the Essex Plant Innovation Centre, which has a strong focus on agritech. When monitoring plant health or trying to develop crop strains that are resistant to environment factors such as drought, it is important to know how much a plant's leaves transpire ("breathe"). It does this through microscopic pores on leaves known as stomata, so counting the number of stomata in an image is an important requirement. A fairly typical leaf image showing stomata is shown below; the stomata are the lighter-coloured dots, arranged roughly in lines. Clearly, counting the number of stomata in the image by hand is possible but very tedious.

The task here is to produce a program which is able to segment out the stomata from the leaf background. If it can do that successfully, counting the number of them is simple.

The problem is that it turns out to be difficult to segment the stomata successfully across a range of plant types and imaging conditions. To produce a solution that works well in practice, it has been necessary to develop operator-guided software to control the segmentation. Now, the whole idea of computer vision is to produce software that is able to perform vision tasks without human involvement, so in that sense an operator-guided approach is a failure. On the other hand, it does save huge amounts of time when counting stomata. The interaction facilities in OpenCV are rather limited but adequate for this purpose.

The count-stomata program

The zip-file associated with this laboratory contains count-stomata.py, a program that interacts with the user to count the number of stomata in an image, and two example images for you to work on, imaginatively called stomata1.jpg and stomata2.jpg. (I wouldn't have chosen JPEG-format images myself but we are constrained by what the capture system does.) To process the supplied images with the program, you would use the command:

python count-stomata.py stomata*.jpg

as long as the images are in the current directory. Adding the -h qualifier to the command will print out its help message, as is usual in the Unix world. When the program is executed, it will pop up a user interface rather like the following screenshot:

There are sliders that control the colour thresholds for cv2.inRange to segment foreground (stomata) from background (leaf). Each individual stoma detected is marked with a black + and the number of them found is written onto the displayed image. You will see that not all the stomata are detected.

As well as adjusting the sliders, the program allows the stomata count to be adjusted:

• a click with the left mouse button adds a stoma, which is shown with a red +

• a click with the right mouse button deletes a stoma, which is then shown with a blue X

When the user is confident that they have finished working with an image, they can type n or (space) to move to the next one. The program prints out the image name, stomata count, and the locations of the stomata before loading the new image.

The first thing you should do with count-stomata is use it --- in particular, finding how adjusting the sliders affects the identification of stomata. You should review the code too, seeing how the functionality you use is provided by OpenCV calls, as well as the mechanisms in the program to ensure stomata are not detected too close together, and so on. There will be questions about how the program works in the second progress test.

Extending count-stomata

You will see that there are a number of 'hooks' inside count-stomata to support HSV processing: the -hsv command-line qualifier, converting the image to HSV, and so on. However, there is no code for applying the HSV limits obtained from the sliders to the image to perform segmentation in HSV colour space. Your job is to implement that, using the routines in sxcv.py that you have already written and tested. You will make count-stomata use your HSV code with a command like:

python count-stomata.py -hsv stomata*.jpg

When your code is written and working, see whether segmentation is better done in HSV or RGB space. If you cannot tell directly, identify how you would go about proving that one approach is better than the other. Do discuss your ideas with a demonstrator --- there will be questions about this in the second progress test.

All the laboratory scripts