Artificial Intelligence / Machine Learning (AI/ML) is a foundational requirement for Multi-Domain Operations (MDO). To solve some of MDO's most critical problems—for example, penetrating and dis-integrating an adversary's Anti-Access/Area Denial systems—the future force requires the ability to converge capabilities from across multiple domains and services at speeds and scales beyond human cognitive abilities. This requires robust, interoperable AI/ML that operates in multiple layers: from optimizing technologies and platforms, to fusing data from multiple sources, to transferring knowledge across joint functions to accomplish critical MDO tactical tasks. The presentation provides an overview of ongoing work to operationalize AI/ML to address MDO problems with this layered approach. It includes insights and required AI/ML capabilities determined at learning events over the past 18 months, as well as vignettes that illustrate how AI/ML can be operationalized to enable successful Multi-Domain Operations against a near peer adversary.

The Army is evolving its warfighting concepts as part of a Multi-Domain Operations (MDO) Joint Force. Artificial Intelligence/Machine Learning (AI/ML) is critical to the Army’s vision, but has significant challenges/risks. This is especially challenging at the Tactical Edge that operates in settings that are dynamic, distributed, resource-constrained, fast-paced, contested, and often physically and virtually isolated. Federated machine learning is collaborative training where training data is not exchanged, and used to overcome constraints on training data sharing and/or insufficient network capacity. We describe the applicability of federated machine learning; identify when it is advantageous to be used; propose strategies for six AI/ML deployment regimes that are at the intersection of total uncertainty and operational timeliness; map AI/ML techniques to address these challenges/requirements; and highlight scientific research questions and ongoing research approaches to answer them.

The paper presents a research agenda on supporting machine intelligence at the tactical network edge, and overviews early results in that space developed under the Internet of Battlefield Things Collaborative Research Alliance (IoBT CRA); a collaboration between the US Army Research Labs and a consortium of academia and industry led by the University of Illinois. It is becoming evident today that the use of artificial intelligence and machine learning components in future military operations will be inevitable. Yet, the dependability limitations and failure modes of these components are poorly understood. Most civilian research investigates solutions that exceed the SWaP (Size, Weight, and Power) limitations of tactical edge devices. Resilience to adversarial inputs is not well studied. The need for significant labeling to train the machine slows down agility and adaptation. These gaps are filled by recent research emerging from the IoBT CRA, reviewed in this work.

Recent trends in artificial intelligence and machine learning (AI/ML), dynamic data driven application systems (DDDAS), and cloud computing provide opportunities for enhancing multidomain systems performance. The DDDAS framework utilizes models, measurements, and computation to enhance real-time sensing, performance, and analysis. One example the represents a multi-domain scenario is “fly-by-feel” avionics systems that can support autonomous operations. A "fly-by-feel" system measures the aerodynamic forces (wind, pressure, temperature) for physics-based adaptive flight control to increase maneuverability, safety and fuel efficiency. This paper presents a multidomain approach that identifies safe flight operation platform position needs from which models, data, and information are invoked for effective multidomain control. Concepts are presented to demonstrate the DDDAS approach for enhanced multi-domain coordination bringing together modeling (data at rest), control (data in motion)

We are developing an agent-based model that allows us to explore control and distributed machine learning concepts in a systems-of-systems framework. The model captures the complex interactions between vehicles with very different operational tempos (such as satellites and unmanned aerial vehicles (UAVs)) and a variety of environmental elements (communication towers, objects of interest, etc.). Treating the model as a complex adaptive system, we can explore issues of controllability and observability, such as the constraints needed to maintain multi-vehicle formations under diverse conditions, and scaling questions, such as the data rates among vehicles and control centers under diverse system parameters

In order to make sensible decisions during a multi domain battle, autonomous systems need to understand important military information such as: where are my dispersed friends and adversaries, and what is the state of my communication networks. The problem is that capturing, representing, and reasoning over this contextual information in distributed, dynamic, and congested multi domain battlespaces, is extremely challenging. This paper describes how the authors are developing an autonomous tactical network application that uses artificial intelligence and machine learning to reason over military context in a way that improves the prioritization and control of information across limited tactical networks.

The paper describes a vision and an accompanying testbed realization for dependable application of machine learning-based inferencing on resource-constrained edge devices. The high computational overhead of sophisticated deep learning learning techniques imposes a prohibitive overhead, both in terms of energy consumption and sustainable processing throughput, on such resource-constrained edge devices (e.g., audio or video sensors). To overcome these limitations, we propose a "collaborative inferencing" paradigm, where the edge devices collectively optimize and augment their individual ML-driven tasks. We provide early examples of how such collaborative operation for video-based object recognition tasks. We then describe a testbed deployment of such edge sensing devices at SMU in Singapore and explain how programmable APIs enable the testbed to be used by remote collaborators to test the effectiveness of novel edge-based ML algorithms and techniques.

Real-life AI based solutions consist of a complex chain of processing elements, which may include a mixture of machine learning based approaches an traditional programmed knowledge. The solution uses this chain of processing elements to convert input information into an output decision. When information is provided for a specific solution, the impact of the information decision on the decision can be measured quantitatively as a Value of Information (VoI) metric. In prior work, we have considered how the VoI metric can be defined for a single AI-based processing element. To be useful in real-life solution instances, the VoI metric needs to be enhanced to handle a complex chain of processors, and be extended to AI-based solutions, as well as supporting elements that may not necessarily used AI. In this paper, we propose a definition of VoI that can be used across AI-based processing, as well as non AI based processing, and show how the construct can be used to analyze and unders

It is readily evident that the U.S. Army must establish an Artificial Intelligence (AI) S&T strategy to rapidly capture, develop, and field the steady stream of discoveries and innovations derived from the global proliferation of AI. In order to focus on Army specific problem sets, the U.S. Army Research, Development and Engineering Command (RDECOM) Research Laboratory, U.S. Army Research Laboratory (ARL) intends to stand up the Army AI Innovation Institute (A2I2) in 2019. This paper discusses how A2I2 will coordinate, conduct, and accelerate basic research to address Army-specific challenges, with a focus on advancing general AI capabilities for Autonomous Maneuver in Multi-Domain Operations. ARL will leverage its existing distributed High Performance Computing (HPC) and network infrastructure, along with its regional laboratory extensions, to enable basic AI research with top-tier universities, small and large commercial businesses, and established DoD industrial partners. This paper also discusses how A2I2 will establish an accessible database of heterogeneous data, repository of AI and ML algorithms and software tools, and military relevant challenge problems.

This paper proposes a novel crowd-sourcing approach for accelerating prototype innovations in AI utilizing scores of multi-disciplinary student-teams recruited nationally to collaborate in our 3D Virtual Trusted Autonomy Campus (V-TAC) within our Deep HoriXons (DHX) virtual environment as avatars. These “crowds” of student-teams will work together to accelerate prototype innovations in AI / ML for supporting teams of highly-dispersed warfighters and agents (robotic and software) operating in distributed, dynamic, complex, cluttered environments. To inspire the student-teams that we recruit nationally, we created an integrated capabilities project framework that we call Project Beast Master with focus on prototyping innovations in Human Machine Teaming and Human Information Interfaces.

In preparations for Multi-Domain Operations and Battles, intelligence analysts gather, assess, and extract relevant information from operational databases as well as publicly available information. This data, often unstructured text documents, is noisy with relevant snippets buried within the document corpus. Our automated information foraging framework can automatically identify a wide breadth of topics for the user, extracted directly from a document corpus. A semantic network formed from the constituent entities within a document corpus contains inherently valuable topological structures that can be used to generate topics and also guide the analyst’s information exploration, facilitating the sensemaking process.

Rapid alignment of data from different sources is a critical data processing need in DoD and other environments. This session summarizes research performed to determine if Machine Learning (ML) is a viable alternative to existing built-in techniques for performing automated data tagging in the open source KARMA Mapping Tool. We discuss the experimental design, our initial results, and a path toward future work in broader applications, including the maritime intelligence domain.

Intelligent systems require perception capabilities that are flexible, dynamic, and robust to unstructured environments and new situations. However, current state-of-the-art algorithms are based on deep learning, require large amounts of data, and require a proportionally large human effort in collection and annotation. To help improve this situation, we define a method of comparing 3D environment reconstructions without ground truth based on the exploitation of available reflexive information, and use the method to evaluate existing RGB-D mapping algorithms in an effort to generate a fully-annotated data set for visual learning tasks. We describe algorithms and software that support rapid manual annotation of these reconstructed 3D environments for a variety of vision tasks. Our results show that we can use existing data sets as well as synthetic data to bootstrap tools that allow us to quickly and efficiently label larger data sets without ground truth, thus maximizing human effort.

Machine learning systems rely on data from many sources for training, input and ongoing feedback and validation Wherever data is used, its users need assurance that the data is as described, and to understand the terms under which it is made available. Similarly, data suppliers need to know that their data is being used legitimately, and that usage is appropriately recompensed. Both parties may agree to quality of service (QoS) metrics, which can be used to negotiate service quality based on cost, and then receive affirmation that data is being supplied appropriately. We present an architecture which enables data sharing agreements to be encoded, and provides a traceable view on data usage. The architecture uses blockchain-based technology, to facilitate transactions in situations where parties do not have an established trust relationship or centralised control, and is exemplified through a case study which uses multiple data sources to categorise traffic levels in urban locations.

Commercial Artificial Intelligence (AI), e.g., the self driving car industry, is often used in predictable settings, with structured surroundings. Significant AI and Machine Learning (ML) progress, particularly in visual perception, has been made in these settings with the use of large publicly available datasets. However, there still exists a prevalent domain mismatch between this data and military relevant environments. In this work we begin to analyze the importance of mobile robot platform design and heterogeneity to effectively collect data more representative of the military domain. The framework of our research is rooted in the importance of expressing constantly changing, yet repeated conditions, with disadvantageous lighting and perspectives in highly unstructured environments.

How to build a machine that can continuously learn from observations in its life and make accurate inference/prediction? This is one of the central questions in Artificial Intelligence. Many challenges are present, such as the difficulty of learning from infinitely many observations (data), the dynamic nature of the environments, noisy and sparse data, the intractability of posterior inference, etc. This tutorial will discuss how the Bayesian approach provides a natural and efficient answer. We will start from the basic of Bayesian models, and then the variational Bayes method for inference. Next, we will discuss how to learn a Bayesian model from an infinite sequence of data. Some challenges such as catastrophic forgetting phenomenon, concept drifts, and overfitting will be discussed.

Autonomous systems are expected to have a major impact in future coalition operations. These systems are enabled by a variety of AI learning algorithms that contextualize and adapt in varying, possibly unforeseen situations to assist humans in achieving complex tasks. Therefore, such systems require the capability to adapt and evolve so that they can behave autonomously at the edge of the network in new situations. Policies are typically used by systems to define their behavior and constraints and often these policies are manually configured and managed by humans. AI-enabled systems will require novel approaches to rapidly learn, create, augment, and model emerging policies to support real-time decision making. Motivated by this observation, in this paper, we propose to apply recent advances in explainable-AI to develop an approach which is agnostic to the learning algorithm, thus enabling seamless policy generation in the coalition environment.

Nowadays, intelligent unmanned vehicles, such as unmanned aircraft and tanks, are involved in many complex tasks in the modern battlefield. They compose the networked intelligent systems with varying degrees of operational autonomy, which will continue to be used increasingly on the future battlefield. To deal with such a highly unstable environment, intelligent agents need to collaborate to explore the information and achieve the entire goal. In this paper, we will establish a novel comprehensive cooperative deep deterministic policy gradients (C2DDPG) algorithm by designing a special reward function for each agent to help collaboration and exploration. The agents will receive states information from their neighboring teammates to achieve better teamwork. The method is demonstrated in a real-time strategy game, StarCraft micromanagement, which is similar to a battlefield with two groups of units.

Since the mid-1990s, a significant scientific literature has evolved regarding the outcomes from the use of what we now refer to as Clinical Virtual Reality (VR). This use of VR simulation technology has produced encouraging results when applied to address cognitive, psychological, motor, and functional impairments across a wide range of clinical health conditions. This presentation addresses the question, “Is Clinical VR Ready for Primetime?” After a brief description of the various forms of VR technology, I will discuss the trajectory of Clinical VR over the last 20 years and summarize the basic assets that VR offers for creating clinical applications.

Dr. Grayson will present on “Mosaic Warfare”, DARPA’s strategy for conducting joint multi-domain battle at speed. Instead of focusing on developing new, highly capable platforms, Mosaic Warfare focuses on speed and adaptation; networking sensors; command and control; and effects together across domains to form kill chains that adapt to dynamic threats and environments. Unlike today’s monolithic systems and rigid architectures that take decades to develop, Mosaic Warfare will utilize rapid machine-to-machine interoperability and AI to network manned and unmanned systems together, creating resilient and distributed architectures at campaign, and eventually, mission speeds. Dr. Grayson will introduce the audience to this Mosaic Warfare concept, and discuss both key technology enablers and challenges that will need to be addressed to make it a reality.

Situational understanding is impossible without causal reasoning and reasoning under and about uncertainty, i.e. probabilistic reasoning and reasoning about the confidence in the uncertainty assessment. We therefore consider the case of subjective (uncertain) Bayesian networks. In previous work we notice that when observations are out of the ordinary, confidence decreases because the relevant training data to determine the probabilities for unobserved variables is significantly smaller than the size of the training data. It is therefore of primary importance for the ultimate goal of situational understanding to be able to efficiently determine the reasoning paths that lead to low confidence whenever and wherever it occurs: this can guide specific data collection exercises to reduce such an uncertainty. We propose three methods to this end, and we evaluate them on the basis of a case-study developed in collaboration with professional intelligence analysts.

As humans and agents or machines are integrated to accomplish missions in multi domain battles there is the inevitable increase in artificial intelligence (AI) and machine learning (ML) to support the interaction. However, the techniques and algorithms within AI and ML are not without challenges. One key challenge is how uncertainty influences decision making. The work presented in this paper will define and scope uncertainty, linking it to selected components of decision making. We will discuss the generation of the measure of uncertainty that we are including in the simulation along with the logic that supports the measure for decision making.The outcomes of the simulations and the influences of the uncertainty will be discussed. Supporting data will be used to frame the results and plans for future investigations.

This session summarizes a study performed on: Application of Data Science within the Army Intelligence Warfighting Function. It includes a problem statement, research summary, key findings, and way forward for military intelligence (MI) to effectively employ data science and data scientists to reduce the burden on Army Intelligence Analysts, increase the effectiveness of data exploitation, and therefore maintain a competitive edge over our adversaries.

Disruptive change in the conduct of military operations occur not with the emergence of new technologies, but with the integration of the new technologies into processes of the military organization. To integrate new and emerging computational tools it is important to first to understand the computational ecosystem and how this ecosystem supports constant development of computational tools. Understanding this dynamic process and how each tool has unique strengths and weaknesses provides the perspective necessary to properly integrate them. Understanding this dynamic and the general categories of computational tools allows one to integrate each tool into the proper place in the military analysis and decision making processes. Critical to this integration however is not only the technology but the capability of those leveraging the technology. Using the tools at the right point and ensuring the capability of the users is necessary for essential integration.

When training data form machine learning is obtained from many different sources, not all of which may be trusted, it is difficult to determine which training data to accept and which one to reject. A policy-based approach for data curation, where the policies are generated after examining the properties of the offered data, can provide a way to only accept selected data for creating a machine learning model. In this paper, we discuss the challenges associated with generating policies that can manage training data from different sources. An efficient policy generation scheme needs to determine the order in which information is received, an approach to determine the trustworthiness of each partner, an approach to decide how to quickly assess which data subset can add value to a complex model, and several other issues. After providing an overview of the challenges, we propose approaches to solve them and study the properties of those approaches.

We applied Human-Machine Teaming Systems Engineering insights towards prototype development with research on supporting situational understanding in MDC2 Common Operating Picture interfaces. User engagements, data analysis, and design feedback sessions yielded user stories, capabilities, and interface requirements for future MDC2 systems. Application of machine learning in MDC2 should go beyond generating courses of action to offer access to system reasoning and data sources (observability), enable operator review and modification of actions (exploring the solution space), and generate non-traditional actions against adversaries. Equally critical will be supporting operator understanding of the implications and significance of the machine learning on C2.

While deep reinforcement learning techniques have led to agents that are successfully able to learn to perform a number of tasks that had been previously unlearnable, these techniques are still susceptible to the longstanding problem of reward sparsity. While this problem can be addressed through reward shaping, such approaches typically require a human expert with specialized knowledge. Inspired by the vision of enabling reward shaping through the more-accessible paradigm of natural-language narration, we investigate to what extent we can contextualize these narrations by grounding them to the goal-specific states. We present a mutual-embedding model using a multi-input deep-neural network that projects a sequence of natural language commands into the same high-dimensional representation space as corresponding goal states. We also discuss how this model can allow for the use of narrations as a robust form of reward shaping to improve RL performance and efficiency.

Head-mounted displays (HMDs) may prove useful for synthetic training and augmentation of C5ISR decision-making. Motion sickness caused by such HMD use is detrimental, resulting in decreased task performance or total user dropout. The genesis of sickness is often measured using surveys, which may not be usable outside the lab. We show a new way to track sickness severity using machine learning on multimodal data collected from users who navigated a virtual environment while remaining stationary in reality. We found that two models, one trained on multimodal sensor data and one trained only on EEG data, were able to classify sickness severity with over 95% accuracy. Greedy optimization was used to maximize accuracy while minimizing the feature subspace. We found that across models, the most important features have been shown to be related to motion sickness in the literature. Finally, we discuss how models constructed on uni- vs multimodal sensor data may be useful in tactical scenarios.

Collective intelligence is generally defined as the emergence and evolution of intelligence derived from the collective and collaborative efforts of several entities; to include humans and (dis)embodied intelligent agents. Recent advances in immersive technology have led to cost-effective tools that allow us to study and replicate interactions in a controlled environment. Combined together, immersive collective intelligence holds the promise of a symbiotic intelligence that could be greater than the sum of the individual parts. For the military, where the decision making process is typically characterized by high-stress and high-consequence, the concept of a distributive, immersive collective intelligence capability is game changing. Commanders and staff will now be able to remotely immerse themselves in their operational environment with subject matter expertise and advanced analytics.

Future weapon systems are likely to integrate artificial intelligence (AI)/machine learning (ML) and other intelligent decision-making aids into dismounted Soldier situational awareness and target acquisition systems. Due to the unique constraints of dismounted operations, successful implementation of AI/ML in small arms necessitates a human-in-the-loop approach where the Warfighter can provide feedback to the AI. We are investigating the influence of different display and input methods on the implementation of human-in-the-loop AI. In this presentation we will identify and discuss challenges unique to displaying and interacting with AI/ML in small arms and propose novel areas of research.

In recent and planned work, researchers at the Naval Surface Warfare Center Dahlgren Division will develop a method to create an explainable artificial intelligence application. Eye-tracking software will be used to record participants in an experiment as they watch and categorize video clips. Then, a deep neural network architecture will be trained using a combination of the eye-tracking data and class labels for the videos with the intention of producing the correct label as well as visual information about features in the video important to this determination. Findings and recommendations for further research will be discussed.

Mission success of future military operations in the multi-domain battlespace will be reliant on intelligent adaptive computational agents and effective human-agent teaming. Our study focuses on an attempt to enhance human-agent team relationships by incorporating the human counterpart(s) preferences and capabilities into an adaptive agent’s repertoire. Using a machine learning approach, statistical comparisons are made to assess population averaged Soldier responses to a questionnaire associated with a multitude of military occupational specialty (MOS) codes. The goal of this work is to identify Soldier related parameters that will account for Soldier characteristics and guide adaptive agent behaviors in collaborative military operations.

Machine Learning (ML) techniques are often inscrutable & hard for users to trust since they lack effective explanations for their outputs. A wide variety of explanation techniques are emerging for AI/ML across data modalities. We investigate which explanation to choose for a particular user & task, considering relevant contextual information such as time available to them, their level of skill, their level of access, & the device they are using, along with available bandwidth and currently usable sensors and services. The explanation techniques range across transparent & post-hoc mechanisms & form part of a conversation with the user to ascertain the explanation through dialogue with the system. Our generic techniques underpin useful explanations in a range of modalities for AI/ML services on multisensor data in a distributed, dynamic, contested & adversarial setting. We define a meta-model & examples of conversational explanation across a range of situations, datasets & modalities.

In human teams, teamwork processes such as effective communication and shared understanding underlie effective team performance. We propose a dynamical system model to understand how individualized, adaptive technology can facilitate teamwork in human-agent teams. The model reveals three scientific challenges: describing the dynamics of team state, understanding how technological interventions will manifest in team states, and observing latent teamwork states. Using this model, we motivate a problem in which we predict team outcomes from non-obtrusive observation of a military staff during a training exercise. Representing pairwise interactions between team members as a weighted adjacency matrix, we use low-rank matrix recovery techniques to identify communication patterns that predict external evaluations of three team processes during task completion: effective communication, shared understanding, and positive affect.

Machine learning systems can provide outstanding results, and understanding how the results are determined is especially important in military and security contexts. In this work, the reliability of LIME (Local Interpretable Model Agnostic Explanations), a method of interpretability, was analysed and developed. A simple Convolutional Neural Network model was trained using two classes of images of “gun-wielder” and “non-wielder". The sensitivity of LIME improved when multiple output weights for individual images were averaged and visualised. The resultant averaged images were compared to the individual images to analyse the variability and reliability of the two LIME methods. Without techniques such as those explored in this paper, LIME appears to be unstable because of the simple binary colouring. A closer inspection reveals that the significantly weighted regions are consistent and the lower weighted regions flip states due to inherent randomness of the method.

In recent work, we utilized convergent cross mapping (CCM) to quantify coordination in a multi-agent reinforcement learning (MARL) paradigm by measuring causal influence between pairs of agents in a joint task. CCM was originally developed to detect causal influences within ecological systems, and as we previously demonstrated, it can be used to measure causal dependencies between pairs of time-series data. While this work has provided important insight into the coordination between 2 teammates, it is not clear how such coordination scales with the number of agents working together with a shared goal. Within a predator-prey pursuit environment, the current study investigates the influence that an incremental increase in number of predator agents has on the inherently causal relationship between predators working together to pursue a single prey.

Tactical edge environments are highly distributed with a large number of sensing, computational, and communication nodes spread across large geographical regions. In such settings, a large number of observations and actions may occur across a large number of nodes but each node may only have a small number of these data locally. Recently proposed federated averaging approaches allow for learning a single model from data spread across multiple nodes and achieve good results on image classification tasks. However, this still assumes a sizable amount of data on each node and a small number of nodes. This paper investigates the properties of federated averaging for neural networks relative to batch sizes and number of nodes. Experimental results on a human activity dataset finds that (1) accuracy indeed drops as the number of nodes increase but only slightly, however (2) accuracy is highly sensitive to the batch size only in the federated averaging case.

Real-time object recognition systems are extremely critical for several UAV applications since it provides fundamental semantic information of the aerial scene. In this study, we describe how image quality limits object detection frame-works such as YOLO which can distinguish between cars, trucks and buses. Pristine high-resolution images are degraded using different blurring functions, spatial resolution, reduced image contrast, noise and lossy compression. Using real data will also investigate how image super resolution algorithms may improve object recognition results.

Recent technological advances provide the opportunities to bridge the physical world with cyber-space that leads to complex and multi-domain cyber physical systems (CPS) where physical systems are monitored and controlled using numerous smart sensors and cyber space to respond in real-time based on their operating environment. However, the rapid adoption of smart, adaptive and remotely accessible connected devices in CPS makes the cyberspace more complex and diverse as well as more vulnerable to multitude of cyber-attacks and adversaries. In this paper, we aim to design, develop and evaluate a distributed machine learning algorithm for adversarial resiliency where developed algorithm is expected to provide security in adversarial environment for critical mobile CPS.

Emerging cyber physical system (CPS) are expected to enhance the overall performance of the networked systems to provide reliable services and applications to their users. However, massive number of connectivities in CPS bring security vulnerabilities and the mobility adds more complexity for securing the mobile CPS. Any mobile CPS can be represented as a graph with connectivity as well as with interactions among a group of mobile CPS nodes that plays a major role as a medium for the propagation of wrong/right information, and influence its members in the mobile CPS. This problem has wide spread applications in viral information disseminating in mobile CPS, where a malicious mobile CPS node may wish to spread the rumor via the most influential individuals in mobile CPS. In this paper, we design, develop and evaluate a machine learning approach that is based on a graph theoretic approach for optimizing the influence in mobile CPS.

Target detection systems identify targets by localizing their coordinates on the input image of interest. This is ideally achieved by labeling each pixel in an image as a background or a potential target pixel. Deep Convolutional Neural Network (DCNN) classifiers have proven to be successful tools for computer vision applications. However, prior research confirms that even state of the art classifier models are susceptible to adversarial attacks. In this paper, we show how to generate adversarial infrared images by adding some small perturbations to the targets regions to deceive a DCNN-based target detector at remarkable levels. We demonstrate significant progress in developing visually imperceptible adversarial infrared images where the targets are visually recognizable by an expert but a DCNN-based target detector cannot detect the targets in the image.

Recently, deep networks have shown unparalleled capabilities in recognizing objects in images/videos. There is a need to deploy DNNs, to a battlefield to provide situational understanding. Soldiers can benefit from at-the-point-of-need analytics on heterogeneous platforms in distributed networks. This work aims to decompose DNNs and optimally distribute over edge nodes. We decompose DNNs into two stages: an initial stage on an edge device and the rest on a cloud. To divide DNNs into two stages, we analyze DNNs with respect to memory size, FLOPs, and amounts of data between layers. We will also develop splitting strategies to handle network constraints.

Commercial sensors are becoming more widely available and deployed in applications ranging from autonomous package delivery, emergency response, and tactical military operations. This leads to large data sets that cannot easily be analyzed by humans. This research proposes an automated method for frame-to-frame object tracking utilizing an image template-matching. A template image of an object of interest (a quadcopter UAS, for example) initializes the algorithm. Each image is then processed to maximize the likelihood of object detection before being passed into the machine learning algorithm. Real-time updates of the template allow frame-to-frame tracking of objects changing due to movement, orientation, or illumination differences (within frame-rate limitations), which improves the autonomous tracking capability. Applications include robust and autonomous object tracking, obstacle avoidance, and independent decision making for navigation and control algorithms.

Machine learning based perception algorithms are increasingly being used in the development of autonomous navigation systems of ground vehicles. For the commercial sector, these ML algorithms are primarily used for functionalities such as object tracking, lane detection and semantic understanding in structured environment. For military applications, Unmanned Ground Vehicles (UGVs) need to operate in unstructured, combat environment such as diverse off-road terrain and under inclement weather conditions. This paper presents an overview of Army-relevant perception challenges, state-of-the-art approaches in addressing these requirements, current limitations and future perspectives to address the gap.

We describe a methodology to optimize underwater seafloor detection for airborne bathymetric LIDAR, an application domain with sparse truth data, leveraging evolutionary algorithms and genetic programming. Our methodology uses the Evolutionary Multi-objective Algorithm Design Engine (EMADE) and a radiometric waveform simulator that generates millions of test cases from which genetic programming techniques select the optimal signal processing techniques and their parameters given the goal of reducing Total Propagated Uncertainty (TPU). The EMADE affords several benefits not found in existing automated machine-learning solutions, including the ability to stack machine learning models, process time-series data using dozens of signal-processing techniques, and efficiently evaluate algorithms on multiple objectives. Preliminary testing indicates successful reduction in overall TPU and foreshadows using EMADE in similar ways to assist in future multi-domain-battle algorithm development.

Better understanding of multi-domain battle challenges in complex military environments may start by gaining a basic scientific appreciation of the level of generalization and scalability offered by narrow AI/ML solutions for day/night performance robustness. We examine these levels of performance from modern deep ML solutions applied to a complex spatial-spectral dataset consisting of calibrated thermal radiance from a fixed target site containing three surrogate tanks deployed in a field of mixed vegetation. The complexity is due in part to the natural variations of the various types of vegetation, cloud presence, and the changing solar loading conditions over 24 hour periods.

Chemical detection library’s used to alert Soldiers of danger grow more complex as new threats emerge. Machine Learning (ML) is a technique to enable computers to learn and provides a method to enhance detection of threats despite interferents. Presented is an application of a ML that models the data from an ion-mobility spectrometer (IMS) and a photoionization detector to correctly identify two molecules with over-lapping IMS detection windows.

Modern warfare can combine actions in the cyber and physical domains to increase attack effect. In the cyber domain, attacks and defenses have a growing need to be automated in order to effectively combat enemy attacks, which themselves may occur at ‘Internet’ speed. The interconnection of the command systems is a logical next step, which can provide some immediate benefits. This paper discusses two Blackboard Architecture based command systems (for online and physical attacks) and describes a framework for their combined operations a shared blackboard space facilitating integrated decision making from the highest (strategic) to the lowest (operational) levels.

Research conducted at NRL has demonstrated the ability of attention management strategies to significantly improve operator performance in highly multitasking environments such as CICs. Viewing human attention as a limited resource, this research attempts to manage an operator’s attention through serialized guidance thus freeing the operator to completely attend to one task at a time. NRL has already shown the merit of mediated attention management for concurrent radio communications monitoring in CICs through an extended series of human performance studies. Measures of attention to data, comprehension, and effort improve dramatically when competing communications circuits are buffered and played sequentially at accelerated rates of speech to ensure that transmissions on each circuit are fully presented. An attention management strategy, however, must include a task prioritization model that can inform serialized guidance so that operators address the most critical amongst competing primary tasks. Additionally, this model must provide for operator-centered control that enables the operators to affect how tasks are prioritized based on their current operation needs. Finally, contingencies of an operator’s workflow must be accounted for with strategies for rapid recovery of task-level SA when an operator leaves and reenters attention management. The objective of this research is to design and evaluate operator-centric prioritization algorithms for chat and voice communications. Operator centric prioritization means that operators are able to specify topics of interest explicitly, and provide the system with feedback on how well the prioritization scheme reflects their informational priorities. The proposed approach will utilize three criteria for assessing the relative importance of a chat or voice message: relevance, urgency and precedence. Given these three measures, the Analytic Hierarchy Process (AHP) model will be utilized to derive a final priority for an incoming chat or voice message.

This paper shows that different artificial intelligence techniques can be combined to provide the commander with various forms of intelligence augmentation and to support the decision-making process. A warfare model has been developed where an AI system, representing a red unit, learns how to select the position for a target and for several improvised explosive devices (IEDs) in order to prevent the blue unit to locate the target. The blue unit is trained to reach the target by using deep reinforcement learning, while an evolutionary algorithm is used to train the red unit. These techniques do not rely on large amounts of historical data. Different approaches have been used and discussed to optimise the co-learning of the two agents, showing that optimal behaviour can be learned in an urban environment. The learned strategy can be used by the commander to provide intelligence augmentation while planning an operation.

In this paper, we describe a phenomenon, which we named ``super-convergence'', where neural networks can be trained up to an order of magnitude faster than with standard training methods. The existence of super-convergence is relevant to understanding why deep networks generalize well. One of the key elements of super-convergence is training with a learning rate cycle and a large maximum learning rate. A primary insight that allows super-convergence training is that large learning rates regularize the training, hence requiring a reduction of all other forms of regularization in order to preserve an optimal regularization balance. Experiments demonstrate super-convergence for Cifar-10/100, MNIST and Imagenet datasets, and resnet, wide-resnet, densenet, and inception architectures. In addition, we show that super-convergence provides a greater boost in performance relative to standard training when the amount of labeled training data is limited.

Barrier coverage problems in wireless camera sensor networks (WCSNs) have drawn the attention byacademic community because of their huge potential applications. Various versions of barrier coverage under WCSNs have been studied such as minimal exposure path, strong/weak barrier, 1/k barrier, full view barrier problems. In this paper, based on new (k − !) coverage model, we study how to achieve (k − !) barrier coverage problem under uniform random deployment scheme (hereinafter A(k − !)BC problem). This problem aims to juggle whether any given camera sensor networks is (k−!) barrier coverage. A camera sensor network is called (k−!) barrier coverage if any crossing path is (k − !) coverage. The A(k − !)BC problem is useful because it can make balance of the number of camera sensors used and the information retrieved by the camera sensors. Furthermore, this problem is vital for design and applications for camera sensor networks when camera sensor nodes were deployed randomly.

We identify strategies among recurring patterns within game-theoretic environments (e.g., predator-prey pursuit) through an algorithmic approach utilizing timeseries methodologies. Our approach leverages multi-agent adversarial data into a composite measure where strategies can be identified through breakpoint discover and Dynamic Time Warping. After verifying the selected algorithm in simple multi-agent game-theoretic environments with top-down strategies, we can extend this work by evaluating human-agent behaviors in more complex environments with emergent strategies to provide fundamental rules for human-agent collaboration.

This paper proposes a novel solution for the Traveling Salesman Problem, a NP (non-deterministic polynomial-time) hardness problem. The algorithm presented in this paper offers an innovative solution by combining the qualities of a Nearest Neighbor (NN) greedy algorithm and the Genetic Algorithm (GA), by overcoming their weaknesses. The paper analyses the algorithm features/improvements and presents this implementation on a FPGA based target board. The experimental results of the algorithm, tested in software (Matlab) and implemented on a portable hardware (FPGA for GA, Raspberry Pi 3 for NN) show a significant improvement: a shorter route, compared to NN , a shorter running time (less generations) compared to traditional GA , and reaching the optimal minimum (validated by Matlab). In real time, the algorithm runs on a handheld console, which can also act as a server, through a custom Android client application.

Varying resolution quality of operational data, size of targets, view occlusions, and large variation in sensors due to nature of overhead systems as compared to consumer devices contribute to degradation of the maritime vessel identification. We exploit the maritime domain characteristics to optimize and refine the deep learning Mask-RCNN framework for training generic maritime vessel classes. In this paper, we build upon this robust localization work, and extend our transfer learning work to new domains and datasets. We propose new approach for identifying specific category of maritime vessels and build a refined multi-label classifier that is based on deep Mask-RCNN features. The classifier is designed to be robust to domain transfer (e.g. different overhead maritime video feed), and to the noise in the data annotation (e.g. vessel is not correctly marked or label is ambiguous). We sub-classification results of this low shot learning approach on MarDCT dataset.

This paper describes coupling of Bayesian learning methods with realistic statistical models for randomly scattered signals in complex environments. Such a formulation enables, for example, efficient learning of signal properties observed at multiple sensor locations in an urban environment. In this formulation, the physics-based model for the random signal corresponds to the likelihood function, while the distribution for the uncertain signal parameters corresponds to the Bayesian prior/posterior. Although many models for random signals have been proposed in the literature, we focus here on the complex Wishart distribution, which describes strongly scattered signals along multiple transmission paths. The Wishart distribution possesses a convenient Bayesian conjugate prior, namely the inverse complex Wishart distribution. We derive and demonstrate appropriate Bayesian updating equations for an uncertain source power and uncertain transmission losses between the source and receivers.

Federated learning distributes model training among a multitude of agents, who, guided by privacy concerns, perform training using their local data but share only model parameter updates, for iterative aggregation at the server. In this talk, we will explore the threat of model poisoning attacks on federated learning initiated by a single, non-colluding malicious agent where the adversarial objective is to cause the model to misclassify a set of chosen inputs with high confidence. We will explore a number of strategies to carry out this attack, at each step enhancing the stealth of the attack while maintaining attack confidence. Finally, using state-of-the-art interpretability techniques we will demonstrate visual explanations of model decisions for both benign and malicious models and show that the explanations are nearly visually indistinguishable. Our results indicate that even a highly constrained adversary can carry out model poisoning attacks while simultaneously maintaining stealth, thus highlighting the vulnerability of the federated learning setting and the need to develop effective defense strategies.

Object detection from images captured by UAVs are widely used for surveillance, precision agricultural etc. Large performance drop is observed when current state-of-the-art object detection approaches are directly applied on the UAV datasets such as the recently released VisDrone2018, primarily due to varying altitude, weather conditions and camera viewing angles. To overcome the challenges, we propose to leverage a novel approach of adversarial training that aims to learn domain invariant features with respect to UAV specific variations. In this paper, we will demonstrate the effectiveness and robustness of the proposed object detection algorithm on two datasets, UAVDT and VisDrone2018.

We focus on defending against adversarial attacks in deep neural networks using signal analysis technology. The method employs a novel signal processing theory as a defense to adversarial perturbations. The method neither modifies the protected network nor requires knowledge of the process for generating adversarial examples. Extensive evaluation experiments demonstrate the efficiency and effectiveness of the proposed adversarial defending method.

Future military coalition operations will increasingly rely on machine learning (ML) methods to improve situational awareness. The coalition context presents unique challenges for ML: the tactical environment creates significant computing and communications limitations while also having to deal with an adversarial presence. Further, coalition operations must operate in a distributed manner, while coping with the constraints posed by the operational environment. Envisioned ML deployments in military assets must be resilient to these challenges. Here, we focus on the susceptibility of ML models to be poisoned (during training) or fooled (after training) by adversarial inputs. We review recent work on distributed adversarial ML, and present new results from our own investigations into model poisoning attacks on distributed learning systems without a central parameter aggregation node.

Future approaches to cyber defense must rely on automated reasoning and automated responses to ensure mission success. Automated reasoning about defensive cyber responses is essentially a sequential decision-making problem involving the projection of possible futures from a current situation under uncertainty. Our research tackles these challenges using the formal framework of partially observable Markov decision problems (POMDPs). We show how to break the dimensionality curse that makes these problems intractable by computing approximate solutions using the combination of Monte Carlo online planning with a computationally feasible simulation of the cyber security problem.

Ongoing research (DAIS-ITA) aims to enable secure, dynamic, semantically-aware, distributed analytics for deriving situational understanding in future coalitions. This paper sets out an example military scenario and operations in a future time frame, to capture the expected battlespace context and key future challenges. Key considerations involve complex multi-actor situations, high complexity information, high tempo processing, all within human-machine hybrid-teams. The coalition composition of these teams is critical and all resources will be constrained. A phased operation is proposed across rural and urban operation involving a range of ISR sensors and autonomous devices. All these are subject to enemy action and perturbation and must be used across a highly contested and congested electromagnetic spectrum. Agile command and control is required across the coalition with information arriving from multiple sources and partners that may also be utilised for learning

In cyber and threat intelligence areas, Indicators of Compromise (IOC) can be used as inputs to security controls to guide defense and mitigation activities. We propose a collaboration model in certain attributes in IOC model related to the (1) seriousness of the threat that the IOC triggers and (2) the confidence in the IOC detection or prediction are build based on a community or collaborative model. In this model, users can subscribe or introduce new IOCs based on their own/systems’ exposures or analysis. They can also assess IOCs created by others and vote to continuously change IOC seriousness and confidence values.

A Federated Learning approach consists of creating an AI model from multiple data sources, without moving large amounts of data across to a central environment. Federated learning can be very useful in a tactical coalition environment, where data can be collected individually by each of the coalition partners, but network connectivity is inadequate to move the data to a central environment. However, such data collected is often dirty and imperfect. The data can be imbalanced, and in some cases, some classes can be completely missing from some coalition partners. Under these conditions, traditional approaches for federated learning can result in models that are highly inaccurate. In this paper, we propose approaches that can result in good machine learning models even in the environments where the data may be highly skewed, classes may be missing, and study their relative performance in a coalition setting.