Border Patrol Control

As the United States expands surveillance of its 1,900-mile border with Mexico, operating the monitoring technologies involved becomes more challenging.

Systems and industrial engineers at the University of Arizona are building a framework for border surveillance that uses artificial intelligence, based on realistic computer simulations, to integrate data from multiple sources and respond in real time.

“Our goal is to devise a system to most efficiently and safely deploy border patrol resources,” says Young-Jun Son, professor, head of the UA’s Department of Systems and Industrial Engineering, and principal investigator of the project.

Funding for Focused Surveillance

With some unmanned aerial vehicles at the border costing $18 million apiece, their performance has implications for taxpayers as well as national security.

Son has received a three-year, $750,000 grant from the Air Force Office of Scientific Research to build an integrated and autonomous surveillance system for land and aerial vehicles monitoring the nation’s southern border.   

Son and his co-principal investigator, UA associate professor of systems and industrial engineering Jian Liu, specialize in helping manufacturers implement smart production systems. Son’s main expertise is in computer modeling and simulation and Liu’s is in statistics and data analysis.

With the Air Force funds, the researchers are applying their skills to help the federal government — ultimately, the U.S. Department of Homeland Security’s Customs and Border Protection unit — gain a clearer picture of border activities for swifter, better-coordinated responses.

Homeland Security has used unmanned aerial vehicles — UAVs, or drones — equipped with cameras and radar for border surveillance since 2005. The drones can cover broad swaths of land and quickly detect activities that might be missed by fixed or mobile ground sensors, particularly in remote or mountainous areas.

Ground-based vehicles have their own advantages. Their sensors better detect objects on cloudy days or beneath trees, and they produce higher-quality images for identifying individual objects or people.

The challenge for the UA researchers is to choose the right combination of aerial and ground vehicles, given different terrain and weather conditions, and activate them at just the right time.

“A major task of unmanned vehicles in patrol missions is to detect and find their targets’ locations in real time,” says research collaborator Sara Minaeian, a UA doctoral candidate in systems and industrial engineering. “This can be challenging for many reasons: For example, the surveillance vehicles and targets are all moving, and the landscape’s uneven nature may alter how targets appear.”

In a paper with Liu and Son published in the July 2016 IEEE Transactions on Systems, Man and Cybernetics: Systems, Minaeian describes their novel motion-detection and geo-localization algorithms for enabling aerial and ground vehicles to work in teams to precisely locate targets and decide how to respond.

The researchers also have been analyzing and testing different wireless network technologies for drones to communicate and cooperate over varied distances.

Delicate Balancing Act

Establishing when and where to send UAVs versus personnel on foot or in trucks is a delicate balancing act. Factors to consider include fuel consumption at different altitudes, accessibility, weather conditions and whether subjects may be armed.

“Once we have detected, located and identified our targets of interest, we must decide which vehicles to deploy, and how many of each, to best meet objectives while considering trade-offs of performance, cost and safety,” Son says.

“For example, to track a group of people moving in mountainous areas under clear blue skies, the optimal solution might be to deploy six UAVs and two trucks driven by border patrol agents, whereas for monitoring a group of the same size traveling in an urban area on a cloudy day, two UAVs and six ground patrol vehicles might be more effective.”

In their simulations, Son’s team also will be adding aerostats, lighter-than-air craft such as unpowered balloons that are increasingly used to track drug traffickers’ low-flying drones and intercept traffickers.

Using NASA geographical data from the border, the UA researchers have written hundreds of algorithms to simulate and predict how groups of people may move when traveling on flat desert and mountains, uninhabited areas and cities, in dry and dusty conditions or during monsoons.

While the UA researchers are not doing field tests at the U.S.-Mexico border, they are conducting experiments outside the lab. They have two quadcopter drones, one purchased and the other built with off-the-shelf parts, and a ground vehicle resembling a toy car. All are remote-controlled and carry a variety of sensors. 

In experiments this spring, the researchers used an aerial drone outside on the UA Mall and inside the Student Union Memorial Center to track 10 student volunteers walking in a group before randomly dispersing. They also deployed their unmanned ground vehicle to identify individual people and serve as a moving landmark to prevent the UAV from losing sight of its subjects. 

The researchers are using their experimental data to better understand various crowd behaviors, such as gathering and splitting, and to refine their algorithms to more accurately predict and track the crowd’s movements. From experiments with a few drones and students, the researchers are scaling up their simulation models to involve hundreds of drones and thousands of people.

“We believe that by integrating multiple surveillance technologies, we can far surpass their individual capabilities,” Son says. “In our integrated system, the sum is bigger than its parts.”