Neural Nexus with Computers Speeds Imagery Analysis

Despite increasing computer capacity and speed, and more sophistication in software, efforts to automate information extraction have not succeeded.

SAIC, however, is helping pioneer a way to rapidly prioritize large volumes of imagery by measuring and discriminating brain signals that are triggered when analysts see something interesting in the images. The work has the potential to change the way people interact with computers.

Seeing before knowing

As it turns out, the brain's visual pattern recognition system spots things more quickly than the mind registers them cognitively. Humans notice things before they know it. In fact, neural responses occur within a few hundred milliseconds after an image is presented.

Laurie Gibson, a chief scientist at SAIC, was involved in an experiment that analyzed brain signals of experts who looked for ships in satellite imagery of coastal areas across the world. The images were flashed at a rate of five to 10 per second on a computer monitor, far more quickly than they would normally process imagery. The analysts' brain signals were collected with a neural sensor net and relayed to a computer. The brain signals were automatically classified based on whether or not the neural activity signaled that the imagery experts detected changes in the images. This classification prioritized the most relevant images for further review.

Very fast search rates

With this method, image analysts searched 65.8 square kilometers of imagery at a rate of 2.21 square kilometers per minute. To compare this to current image analysis techniques, Gibson ran a baseline experiment. This experiment required analysts to decide which images — shown on monitors in pairs — were potentially relevant and then use software tools to tag them for further review.

The results of the research, done in conjunction with the University of Colorado and sponsored by the Defense Advanced Research Projects Agency, were significant. Analysts using standard techniques searched only .21 square kilometers per minute. This meant that analysts examined 10 times more imagery using the neural change method.

According to Gibson, the research provided a good foundation for a tool to rapidly filter streams of images with significant content. In fact, depending on how fast the imagery can be presented to the analyst and how rapidly the neural response can be processed, this approach has the potential to greatly speed up the review of large volumes of data. The research also demonstrated the feasibility of an image system based on the neural response of experts to visual change.

"SAIC is funding us under IR&D [independent research and development] to look at a number of applications for this technology," Gibson said. "This has the potential to change the way that people and computers interact."