Have you ever wondered how a computer automatically identifies a picture of a koala or the Sahara Desert? To do this, humans train the computer to know what a koala or desert looks like. Later, the trained computer can recognize new images of koalas and deserts. Scientists also use this technique to teach computers to automatically identify objects in places like Mars, where humans can’t go themselves.
The NASA Mars Curiosity rover does this to drive around rocks and boulders without hitting them. It uses software called AEGIS, which powers the “brains” of the rover’s robotic cameras and tells it whether there’s a rock ahead. Just like humans constantly learn and update their own “software,” scientists regularly update rover software so that it works better. A team from Shandong University, China, proposed new software that adds to the rover’s abilities by teaching it to report back what type of rock it’s looking at.
The team built upon existing software models called Convolutional Neural Networks (CNNs) and VGG-16. These models provide a way for a computer to study information, like images of different kinds of Mars rocks. After enough studying, the computer can tell what type of rock it’s looking at in new images. The catch is that computers need a library of thousands of photos to learn well, but we don’t have enough Mars images.
Li’s team tackled this problem using transfer learning and data augmentation. Transfer learning is learning from a similar but different library, like a library with millions of Earth rocks, and applying what you learned to study and identify your target, like Mars rocks. Data augmentation is when you learn from synthetic data that is slightly different from the original data.
Samples of Martian rocks collected from the Analyst’s Notebook. Source: NASA/JPL-Caltech
The team used transfer learning by training a model on the massive online image library called ImageNet. Then they took the ImageNet-trained model and trained it again using the augmented Mars rock image library. This is similar to teaching a child the numbers 1-100 (training using the model and the “number library”). Then teaching them how to add and subtract using those numbers (training again using the model and the “arithmetic library”).
The team used data augmentation by flipping images of Mars rocks horizontally or combining two images to add new photos to the Mars rock library. From the original 620 pictures of Mars rocks, they were able to add 1240 synthetic pictures. This made the Mars rock library three times bigger than the original — big enough to train a computer.
What does training a computer mean? Individual image pixels are converted to numbers so that they can be fed into the CNN VGG-16 model. The model passes this information through equations that sort pixel colors, intensities, and coordinates. Patterns like rock edges or textures are flagged to learn what a rock is and how rocks are different from each other. At the end of the first pass, the model tries to identify the kind of rocks it’s looking at from a list of rock categories. It usually fails the first time, but learns to succeed through repetition. The model keeps studying the images over many passes until it consistently recognizes the rock type.
Li’s team recorded this learning process using loss and accuracy curves. Loss curves showed that the number of identification mistakes went down with each pass. Accuracy curves showed that the number of successful rock identifications went up with each pass. They lowered loss and raised accuracy as much as possible after 20 passes through the CNN VGG-16 model.
The software models identified four rock types in the Mars rock library. The rock types were dark-colored igneous rocks, rounded and fine-grained igneous rocks, layered sedimentary rocks, and clustered pebble conglomerate rocks. Rock images Group 1-4 respectively. (Figure 1) The authors compared their models to six other models and found that their software best identified Mars rocks 100% of the time.
Automatic rock identification on Mars can increase the speed at which we can investigate the surface of Mars and the amount of data we can collect. Li’s team accomplished this using automated CNN VGG-16 models without the need for massive Mars rock image libraries. Until scientists collect enough original Mars rock images to train computers on, current and future rovers may be able to upgrade their “brains” using software like this.
