Just about every major country in the World has watched Robert Downey Jr. streak across the sky righting wrongs and taking out the bad guys. As is so often the case, movies plant the seed and the military folks start the step of transitioning from fantasy to reality.

The picture on the left is Russia’s entry and the picture on the right is a possible USA entry.

Really cool. They come with an exoskeleton — super human performance on demand. If it gets too hot, there is an on board air conditioner. No need to visit the gym — these folks simply strap on the suit and instantly look buff. There is one slight problem yet to be over come — they require batteries — very big batteries. And when the batteries bleed off, it is time to retreat. I have little doubt that the R&D folks will solve the problem. But I also suspect that before the suit becomes operational, we will see more robots wearing the suits as an outer skin than we will see humans.

I am real big on taking small steps — build something that satisfies a demand — build it so that works in a real environment — and use off the shelf hardware and software. We did just that. In the image above, you see Big Bubba. Big Bubba is collecting real time audio and video along with six different types of brain waves. The Bubba streams the information and gets back the current situation as seen from his perspective and the perspective of other team members. And all with credit card size batteries.

 

The picture above is from an article in Police Magazine. All about how you survive moving through a dark room.

“The Suspect’s Advantage

The simple fact is regardless of how skilled you are with a flashlight, how well you understand low-light operational tactics, and how skilled you are deploying those low-light skills, in a dark room the suspect always has the advantage. If the suspect has the capacity to be patient during your advance, the suspect’s advantage increases.”

 

Pretty obvious. In the picture above, the suspect has a great target and has a pretty good idea about the officer’s mental state.

When you started working on our first set of problems, we realized the dark room and a flash light as a problem. We thought off the shelf, long operational life, and minimize the flash light.

So in the picture above this is what we wanted the suspect to see. Nothing. Just above, you see a big black empty space. That is what we want the suspect to see.

First step, think FLIR. Think a thermal imaging camera. Then think cost. Then think interface. Develop some software. Then integrate the device into our other solutions.

We just turned the table. Instead of the suspect seeing a bright light, the suspect sees a black void, and the officer sees the suspect in blazing color. When I was first testing these cameras, I monitored myself moving through a room. I could see the activity of my brain and what lobes were most active. I am easily impressed.

I am not sure if it is evident, but capturing physiological data, audio, and video streams, analyzing suspect threats and intentions and feeding the conclusions to team members — even teams scattered around the world, requires a fair amount of technology.  Not to rest on our laurels, we are leveraging Google Cloud. With the cloud we are able to relatively easily tap into speech services so that we can translate speech to text. And because we are able to get the text, we can tap into Natural Language Processing services — translating languages on the fly. And since we can hear, and we can hear from multiple points, we can locate. We have been doing speech processing for over twenty five years in hostile environments. At the time, I did not understand the real potential. I was fortunate to be trained by some of the best folks — experts on how to gather information from anything that transmits. And then came Google and the cloud. The right time and the right place.

 

The collection of data from a high level:

1. Physiological, audio, video, and environment data are streamed to a server
2. Both Kafka and RabbitMQ are used to queue data
3. Apache Storm ingests the data. Apaches Storm instances runs in parallel authenticating and filtering data
4. Accumulo, Cassandra, and Hadoop are run in parallel to store the time series data. By using both, CAP is possible.
5. Data streams are applied to a series of AI and Machine Learning algorithms
6. Assumptions are made about the current context
7. Situation reports are sent back to team members