As much of 25 percent of agricultural crops are left to rot in fields, and of those that are harvested, gathering the necessary human labour at the drop of a hat - usually last-minute high-volume demands from the supermarkets - can prove tricky. To help solve this, a team of engineers at Cambridge has developed Vegebot, a lettuce-spotting machine based on open source software that can slice the traditionally challenging salad staple of the daisy family.

"There's two major problems with lettuce-packing," said Simon Birrell of Cambridge's Department of Engineering, co-author of a report on the project that appeared in the Journal of Field Robotics. "The first is: 'where is the lettuce'? Which is not as obvious as it sounds if you look from a robots' eye's perspective; all you see is fields of green leaves. It's not obvious where the heads are even to human beings."

vegebot lettuce picker supplied art

The second problem is that once a lettuce head has been located, it must be removed very carefully, because the damage-prone crop must remain both intact and aesthetically suitable to ship to the supermarkets. Gripping too tightly also threatens damaging the plant, and so there are a "whole series" of constraints which have "resisted automation until now", said Birrell.

Robotics and agricultural technology are already deployed in the field to automate the lettuce lifecycle, but the actual harvesting continues to be a manual job. Machines are able to seed the lettuce and plant them in fields, while others can be programmed to spray the crops, but the harvesting typically requires a team of at least 10 people moving across the fields on rigs to cut the product.

Birrell describes the whole process as "extremely iterative". The team spent much of its time out in the field, with varying combinations of hardware and software, rather than lab experiments. For crafting the machine's ability to recognise the lettuce to harvest, the researchers began with standard computer vision classifiers before moving on to neural networks and AI.

They went through a series of different end effectors for the machine - the name for the devices at the end of a robot's arms - trying electric motors and rotary blades first, which "didn't work so well".

Linear electric actuators, devices that convert rotational motion in DC motors into push-pull movements, were better - however, they did not have the force required to achieve the cut. Pneumatic linear actuators, where compressed air drives a blade through the stalk, was found to provide a clean cut, and after that it was a "question of using soft silicone grippers" to "actually grip the lettuce head, and then coordinating all of that with software", said Birrell.

The prototype uses the open source Robotic Operating System in combination with the You Only Look Once computer vision tool for object detection, and is mostly written in Python and some C++. The hope is that eventually, with a full fleet of these Vegebots, they could learn from one another and automatically improve their future performance.

Another complication is that fields of crops are semi-unstructured environments. On one extreme, Birrell explains, would be a factory where everything is in its place. The opposite extreme is environments out in the real-world, like those that self-driving cars have to navigate. Lettuce fields are somewhat predictable, because they tend to be planted in neat rows, but machines must still be able to withstand unpredictable weather, for example, as well as any humans that are accompanying them - as with one terrifying experience when the team was out in a large empty field with a metal object during a thunderstorm replete with lightning.

Although the technology is not yet commercialised - and there are no plans to spin out the Vegebot as a startup as it currently exists - Cambridge is open to collaborating with organisations that would like to attempt to work on it further for real-world applications.

Birrel hopes that more widely, the combination of software, AI, and custom end effectors, could be a "powerful combination" that should be able to be applied to a wide variety of crops.

Some of the coverage so far has focused on the labour market, and how crop harvesting tends to be precarious work for low-income migrants. Some of the usual suspects have tied the robot to Brexit, but he says that this is not the whole story.

"It's not just the current labour shortage problem due to Brexit, it's more to do with ensuring the reliability of labour supply," he adds. "What happens is you get these sudden spikes in demand: everything is driven by the supermarkets in this country. They'll call up the grower and say: we need a delivery of 10,000 lettuces for tomorrow morning.

"The difficulty is marshalling a team of human beings to do that, at that short notice: even without Brexit, that's a problem that needs solving. How do you cope with spikes in demand? Having a fleet of lettuce-picking robots would definitely help that."

The high wastage ratio of 25 percent of crops rotting on fields is another problem that requires solving. Birrell says that by using data to predict accurately the weather, the demand, and automating the processes to picking it in time, this wastage could be cut down as well.