In progress – view active session
Conference 13051
Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications VI
22 - 25 April 2024 | Potomac 4
Conference Co-Sponsor
Show conference sponsors +
View Session 
Session 1:
Mission Planning I
22 April 2024 • 8:40 AM - 10:00 AM EDT | Potomac 4
Session Chairs:
Peter J. Schwartz, The MITRE Corp. (United States), Brayden Hollis, Air Force Research Lab. (United States)
Opening Remarks 8:40 AM to 8:50 AM
13051-201
DoD challenges and opportunities in the era of foundation models
(Invited Paper)
22 April 2024 • 8:50 AM - 9:20 AM EDT | Potomac 4
Show Abstract +
Foundation models, including Chat-GPT and its many variants, have come into prominence in the natural language processing (NLP) community thanks the ubiquity of text data readily available on the internet and the design of modern transformer architectures that can effectively learn from such data. However, popular DoD domains, such as autonomy and logistics, are faced with additional challenges not present in NLP and vision. In this talk, we discuss some of those challenges and opportunities for addressing them.
13051-3
22 April 2024 • 9:20 AM - 9:40 AM EDT | Potomac 4
Show Abstract +
Battlespace management requires rapid processing of large amounts of data to facilitate informed decision-making. Large Language Models (LLMs) have demonstrated near or above human performance on a wide range of cognitive tasks. Specifically, LLM capabilities to synthesize defense-relevant data and make decisions in a combat environment have been largely unexplored. The battlefield information and tactics engine (BITE) uses LLMs as observers and decision-makers in a military environment. A multiplayer video game focusing on modern mechanized combat, Squad by Offworld Industries Ltd., is used as an operating environment due to its moderate realism levels and focus on audio communication between players. BITE is tasked with ingesting tactical data, providing summaries of the current situation, and giving order to a squad of human players. The present work aims to qualitatively assess the suitability of BITE, and LLMs in general, for use in battlespace management
13051-4
CANCELED: From pixels to strategy: harnessing Chat-GPT (Vision) for Atari games and military operations
22 April 2024 • 9:40 AM - 10:00 AM EDT | Potomac 4
Show Abstract +
Modern artificial intelligence has witnessed a paradigm shift with the advent of Large Language Models (LLMs), with capabilities extending beyond text into multi-modal tasks involving images. This paper explores a novel application of these multi-modal LLMs, specifically leveraging the Chat-GPT (Vision) model, to act as game-playing agents for Atari video games.
Historically, Atari game-playing agents have been developed using reinforcement learning (RL). While effective, RL demands extensive training iterations, often spanning millions of timesteps, to master the intricacies of these games. LLMs, laden with pre-existing knowledge, present an intriguing alternative. Their inherent knowledge base might enable them to comprehend and enact Atari-game play policies via zero-shot learning or context-sensitive adaptations.
In this research, we challenge Chat-GPT (Vision) with raw visual images from various Atari games to determine its competency in interpreting, strategizing, and playing these games. Our methodology revolves around feeding screen captures into the model and analyzing its output for game tactics, understanding, and performance metrics.
Break
Coffee Break 10:00 AM - 10:30 AM
Session 2:
Computer Vision I
22 April 2024 • 10:30 AM - 11:30 AM EDT | Potomac 4
Session Chairs:
Christopher R. Ratto, Johns Hopkins Univ. Applied Physics Lab., LLC (United States), Tien Pham, The MITRE Corp. (United States)
13051-6
22 April 2024 • 10:30 AM - 10:50 AM EDT | Potomac 4
Show Abstract +
Image geo-registration is an essential technology and has wide range of applications in the geospatial intelligence space. Prior to the recent deep learning advancements, image geo-registration is primarily using area-based image correlation algorithms. The “line-photogrammetry” concept, which uses lines instead of points for geo-registration and triangulation, is not widely adapted due to the difficulties of reliably and accurately extract lines from geospatial images. With deep learning, this is no longer a challenge. We take advantage of the recent advancements in deep learning and have develop DeepTie, which uses both tie points and tie features, for image geo-registration and triangulation.
13051-7
22 April 2024 • 10:50 AM - 11:10 AM EDT | Potomac 4
Show Abstract +
Image-to-image correspondence being important in numerous remote sensing applications. While many local features used for these methods aim for robustness to changes in viewpoint/illumination, recent studies suggest that traditional feature extractors may lack stability in multi-temporal applications. We have discovered that this is especially true in multi-modal sensor contexts, such as corresponding high-resolution UAV images to satellite imagery. This paper explores the performance of various local feature extraction methods as they pertain to image-to-image correspondence. Experiments here specifically evaluate co-registration between low-altitude, nadir UAV frames, and imagery collected from satellite sources. Due to challenges in the localization of imagery with significantly different resolutions, spatial extents, and spectral characteristics, two further studies are presented beyond baseline evaluation. First, images undergo histogram matching to enhance feature similarity and second, semantic segmentation maps are used to refine and restrict key point detections and matches.
13051-8
22 April 2024 • 11:10 AM - 11:30 AM EDT | Potomac 4
Show Abstract +
The rise of the Internet of Things (IoT) has significantly influenced military science, leading to the development of the Internet of Battlefield Things (IoBT) for enhanced military operations. IoBT devices, especially camera-based ones, are crucial for soldiers to navigate complex battlefields. However, their limited field of view and resource constraints hinder prompt object recognition and labeling. This paper proposes a collaborative approach for efficient object labeling, utilizing multiple geo-distributed IoBT devices.
Break
Lunch Break 11:30 AM - 1:20 PM
Session 3:
Multi-Agent Systems
22 April 2024 • 1:20 PM - 2:50 PM EDT | Potomac 4
Session Chairs:
Jeffrey Hudack, Air Force Research Lab. (United States), Danda B. Rawat, Howard Univ. (United States)
13051-202
How to Accelerate AI Adoption
(Invited Paper)
22 April 2024 • 1:20 PM - 1:50 PM EDT | Potomac 4
Show Abstract +
Accelerating AI adoption necessitates a concerted focus on enhancing workforce skills, optimizing development environments, and prioritizing user-centered design. By upskilling employees and cultivating a culture of innovation, organizations can build a robust talent pool equipped to tackle AI challenges. Emphasizing accessible and flexible development platforms enables swift creation and deployment of AI solutions. Integrating user-centered design principles ensures these technologies meet real-world needs, fostering widespread acceptance and use.
13051-9
22 April 2024 • 1:50 PM - 2:10 PM EDT | Potomac 4
Show Abstract +
Reinforcement Learning (RL) has become a widely used approach for pursuit-evasion games. However, the behavior of such RL models is hard to analyze, often leading to a lack of trust. This paper describes a study in which we used machine learning (ML) approaches to develop metareasoning policies that control pursuers’ strategies. The proposed approach enables pursuer agents to capture a faster evader by choosing simple pursuit strategies collaboratively. The results show that some metareasoning policies perform better than any pursuer strategy combinations. Our approach provides an innovative way for the pursuer agents to reason about their opponents and adapt their strategy, which could have significant implications for the design of intelligent agents in real-world applications.
13051-11
22 April 2024 • 2:10 PM - 2:30 PM EDT | Potomac 4
Show Abstract +
Collections of autonomously behaving systems, or swarms, are predicted to be an important component of the US DoD strategy. Therefore, research into how to create swarms with suitable characteristics, behaviors, and function for these different purposes is in the interest of the US military. However, there are challenges in swarm research, including technical limitations of existing hardware, the need to address both individual drone level behavior as well as the complexities of the entire swarm behavior, and the sheer number of parameters that may be relevant to swarm performance in operations. This presentation proposes methodologies for the computer simulation research and analyses for experimentation on swarm behavior. Swarm performance data from computer simulation experimentations using simulation software were analyzed through multiple steps to investigate how individual and entire swarm characteristics might affect how well the swarm performed a mission.
13051-12
22 April 2024 • 2:30 PM - 2:50 PM EDT | Potomac 4
Show Abstract +
Within the landscape of military technology, the time has come for automated systems to assume mission-critical responsibilities demanding computational speeds that are beyond human capabilities. Swarm robotics exhibits a diversity of applications in many fields including surveillance, reconnaissance, and more. Regardless of the specific application, meticulous planning and optimization of traversal routes coupled with seamless communication among individual robots assumes a pivotal role. Optimized cooperative route planning enables heightened operational efficiency, yielding a reduction in operational costs while elevating overall productivity. The significance of optimization extends to each robotic ensemble and an understanding of the intricacies of route selection fosters enhanced collaborative synergy. Consequently, the swarm attains a heightened capability to undertake intricate and multifaceted missions, transcending the limits of individual capabilities. Traditional approaches have restricted the potential of swarm-based systems. There has been little focus on cooperative route planning for swarms, essentially eliminating the advantages that swarm systems provide. We propos
Break
Coffee Break 2:50 PM - 3:20 PM
Session 4:
Virtual, Augmented, and Mixed Reality
22 April 2024 • 3:20 PM - 4:40 PM EDT | Potomac 4
Session Chairs:
Bruce Swett, Northrop Grumman Corp. (United States), Ravi Ravichandran, BAE Systems (United States)
13051-15
22 April 2024 • 3:20 PM - 3:40 PM EDT | Potomac 4
Show Abstract +
Virtual Reality (VR) has improved greatly in recent years, offering various ways for users to
interact with virtual environments. Each extrasensory modality in VR provides different
sensory inputs and interactions, enhancing the user& #39;s immersion, presence, and decision-
making skills in the virtual world, which could lead to transferable skills in the real world.
However, the effectiveness of additional sensory modalities, such as haptic feedback and
360° locomotion, in improving decision-making performance has not been explored.
Additionally, longitudinal training methods have not been tested for their effectiveness in
improving complex decision-making tasks. This study aims to address this gap by evaluating
the impact of a 360° locomotion-integrated VR framework and longitudinal, heterogeneous
training on decision-making performance in a complex search-and-shoot simulation. The
study involved 32 participants (mean age = 25.8 years) from a defence simulation base in
India, who were randomly divided into two groups: experimental (haptic feedback, 360°
locomotion-integrated VR framework with longitudinal, heterogeneous training) and
placebo control (longitudinal, heterogeneous
13051-13
22 April 2024 • 3:40 PM - 4:00 PM EDT | Potomac 4
Show Abstract +
In the past decade, there has been tremendous growth in actuators, end-effectors, and grasping algorithms, that has resulted in low-cost robot arms. Employing such arms on ground, air, and space vehicles is called mobile manipulation.
The U.S. Department of Defense (DoD) has been interested in using mobile manipulation for applications like explosive ordnance disposal (EOD) and intelligence, surveillance, and reconnaissance (ISR) missions. The motive is to increase standoff distance between soldiers and hazards by introducing mobile manipulating robots for dangerous or dirty tasks.
One of the challenges in mobile manipulation is to complete the desired tasks beyond an operator’s line of sight. Immersive technologies like virtual reality and augmented reality (VR/AR) could address this challenge. VR/AR provides an immersive experience and enhanced situational awareness to the operator in a remote site. It can also more intuitively map human body motions to a robot manipulator. Integrating these two technologies results in tele-manipulation.
This paper presents a hardware and software system design, along with experimental results, for tele-manipulation.
13051-14
22 April 2024 • 4:00 PM - 4:20 PM EDT | Potomac 4
Show Abstract +
Digital augmentation of direct view optics like riflescopes allows viewing of the scene with near-zero latency while simultaneously offering increased situational awareness from fielded sensors and systems data. Existing solutions to digitally augment a direct view optic add a considerable amount of weight/size, require the digital projection system to be designed into each specific viewing optic, or interfere with operational function of the optic. Here we present an externally mounted, compact optical waveguide capable of displaying information such as ballistic data overlaid onto the field-of-view of a riflescope. Volume phase holograms form the input/output couplers and the pupil expander of the waveguide. The holographic waveguide satisfies the contrast ratio needed for daylight bright environments, allows for operability of the diopter adjustment, minimally impacts eye relief, and requires no refocusing to view the symbology.
13051-16
22 April 2024 • 4:20 PM - 4:40 PM EDT | Potomac 4
Show Abstract +
The information presented by AR systems may not be 100% accurate, and anomalies like tracker errors, lack of opacity compared to the background and reduced field of view (FOV) can make users feel like an AR training system is not believable. This lack of belief can lead to negative training, where trainees adjust how they train due to flaws in the training system and are therefore less prepared for actual battlefield situations.
We have completed an experiment to investigate trust, reliance, and human task performance in an augmented reality three-dimensional experimental scenario. Specifically, we used a methodology in which simulated real (complex) entities are supplemented by abstracted (basic) cues presented as overlays in an AR head mounted display (HMD) in a visual search and awareness task. We simulated properties of different AR displays to determine which of the properties most affect training efficacy. Results from our experiment will feed directly into the design of training systems that use AR/MR displays and will help increase the efficacy of training.
Symposium Plenary
22 April 2024 • 5:00 PM - 6:30 PM EDT | Potomac A
Session Chairs:
Tien Pham, The MITRE Corp. (United States), Douglas R. Droege, L3Harris Technologies, Inc. (United States)
View Full Details: spie.org/dcs/symposium-plenary
Chair welcome and introduction
22 April 2024 • 5:00 PM - 5:05 PM EDT
DoD's microelectronics for the defense and commercial sensing ecosystem (Plenary Presentation)
Presenter(s): Dev Shenoy, Principal Director for Microelectronics, Office of the Under Secretary of Defense for Research and Engineering (United States)
22 April 2024 • 5:05 PM - 5:45 PM EDT
NATO DIANA: a case study for reimagining defence innovation (Plenary Presentation)
Presenter(s): Deeph Chana, Managing Director, NATO Defence Innovation Accelerator for the North Atlantic (DIANA) (United Kingdom)
22 April 2024 • 5:50 PM - 6:30 PM EDT
Symposium Panel on Microelectronics Commercial Crossover
23 April 2024 • 8:30 AM - 10:00 AM EDT | Potomac A
View Full Details: spie.org/dcs/symposium-panel
The CHIPS Act Microelectronics Commons network is accelerating the pace of microelectronics technology development in the U.S. This panel discussion will explore opportunities for crossover from commercial technology into DoD systems and applications, discussing what emerging commercial microelectronics technologies could be most impactful on photonics and sensors and how the DoD might best leverage commercial innovations in microelectronics.
Moderator:
John Pellegrino, Electro-Optical Systems Lab., Georgia Tech Research Institute (retired) (United States)
Panelists:
Shamik Das, The MITRE Corporation (United States)
Erin Gawron-Hyla, OUSD (R&E) (United States)
Carl McCants, Defense Advanced Research Projects Agency (United States)
Kyle Squires, Ira A. Fulton Schools of Engineering, Arizona State Univ. (United States)
Anil Rao, Intel Corporation (United States)
Break
Coffee Break 10:00 AM - 10:30 AM
Session 5:
Causality
23 April 2024 • 10:30 AM - 12:10 PM EDT | Potomac 4
Session Chairs:
Tien Pham, The MITRE Corp. (United States), Bruce Swett, Northrop Grumman Corp. (United States)
Opening remarks 10:30 AM to 10:40 AM
13051-207
MITRE ATLAS & AI Assurance Overview
(Invited Paper)
23 April 2024 • 10:40 AM - 11:10 AM EDT | Potomac 4
Show Abstract +
MITRE Adversarial Threat Landscape for AI Systems (ATLAS™) is a globally accessible, living knowledge base of adversary tactics and techniques based on real-world attack observations and realistic demonstrations from artificial intelligence (AI) red teams and security groups. There are a growing number of vulnerabilities in AI-enabled systems as the incorporation of AI increases the attack surfaces of existing systems beyond those of traditional cyberattacks. We developed ATLAS to raise community awareness and readiness for these unique threats, vulnerabilities, and risks in the broader AI assurance landscape. Anna will discuss the latest ATLAS community efforts focused on capturing cross community data on real world AI incidents, growing understanding of vulnerabilities that can arise when using open-source models or data, building new open-source tools for threat emulation and AI red teaming, and developing mitigations to defend against AI security threats.
13051-17
23 April 2024 • 11:10 AM - 11:30 AM EDT | Potomac 4
Show Abstract +
Explainable AI (XAI) has been crucial in making artificial reasoning systems more comprehensible. Even though XAI can explain the decisions being made by the machine learning (ML) systems, these decisions are based on correlation and not causation. And for applications such as tumor classification in the medical field, this can have serious consequences as people’s lives are affected. A potential solution for this challenge is the application of causal learning, which goes beyond the limitations of correlation for machine learning systems. Causal learning can generate analysis based on cause-and-effect relations within the data. This study compares the results of explanations given by post-hoc XAI systems to the causal features derived from causal graphs via causal discover for image datasets. We investigate how well XAI explanations/interpretations are able to identify the pertinent features within images. Causal graphs are generated for image datasets to extract the causal features that have a direct cause and effect relation with the label. These features are then compared to the features highlighted by XAI via feature relevance.
13051-18
23 April 2024 • 11:30 AM - 11:50 AM EDT | Potomac 4
Show Abstract +
With the spread of powerful AI models, human-machine interaction has improved rapidly. In this situation, recognizing facial expressions of human emotion is essential for successful human-machine collaboration/teaming. While Convolutional Neural Network (CNN) models were widely used for facial emotion classification, Transformer-based models, known for excelling in NLP tasks, have demonstrated superior performance in areas like image classification, semantic segmentation, and object detection. This study investigates the effectiveness of using a transformer based model, the Face-transformer model a powerful tool for facial identification, that will be fine-tuned for the Facial Emotion Recognition (FER) challenge. We aim to modify the face transformer architecture to recognize emotional states from facial photos using the extensive facial emotion recognition datasets, opening the door for more natural and responsive machine interactions. Our initial findings suggest that the Face-Transformer model holds promise for bridging the gap between machine interpretability and human emotions, potentially paving the way for more natural and responsive human-computer interactions.
13051-19
23 April 2024 • 11:50 AM - 12:10 PM EDT | Potomac 4
Show Abstract +
Causal learning for image, audio, video, rf, other modalities still remain a major challenge. While there are open-source tools available for causal learning with tabular data, there is a lack of tools for other modalities. To this extent, this study proposes a causal learning method with image datasets by using existing tools and methodologies. In specific, we propose to use existing causal discovery toolboxes for investigating causal relations within image datasets by converting image datasets into tabular form via attribute extraction. The converted dataset can then be used to generate causal graphs by using tools such as the Causal Discovery Toolbox to highlight the specific cause and effect relations within the data.
Break
Lunch/Exhibition Break 12:10 PM - 2:20 PM
Session 6:
Decision Support
23 April 2024 • 2:20 PM - 3:00 PM EDT | Potomac 4
Session Chairs:
Brayden Hollis, Air Force Research Lab. (United States), Michael Wolmetz, Johns Hopkins Univ. Applied Physics Lab., LLC (United States)
13051-21
23 April 2024 • 2:20 PM - 2:40 PM EDT | Potomac 4
Show Abstract +
Artificial intelligence (AI) is quickly gaining relevance as a transformative technology. Its ability to rapidly fuse and synthesize data, accelerate processes, automate tasks, and augment decision-making has the potential to revolutionize multi-domain warfighting through data-centric operations and algorithmic warfare. As the military relies more on AI-enabled Decision Aids to increase the efficiency and effectiveness of decision-making, it highlights the need to effectively assess them before deployment. Modeling and simulation (M&S) environments are essential for assessing these rapidly evolving AI-enabled systems. Accepted analytical frameworks are needed to guide ways to represent and model AI sufficiently within M&S environments for accurate assessment. In this paper, we identify common characteristics within the main categories of AI and investigate how those characteristics can be best represented across the main categories of M&S. We provide two use cases to highlight an assessment of AI-enabled Decision Aids for cybersecurity and aeromedical evacuation problems. Our use cases demonstrate how to leverage a framework for analytic assessment of AI within M&S environments.
13051-22
23 April 2024 • 2:40 PM - 3:00 PM EDT | Potomac 4
Show Abstract +
In response to escalating complexity and uncertainty in decision-making, a novel framework is introduced to enhance multi-criteria decision-making under uncertain conditions. This framework integrates advanced data processing, dynamic decision plan generation, and adaptive mechanisms to offer robust, flexible, and responsive decision-making solutions. Methodological rigor ensures accurate decision-making through comprehensive data processing, including feature extraction and anomaly detection. The framework systematically generates initial decision plans, dynamically expanding them to incorporate co-requisite actions, thus ensuring comprehensive coverage. An adaptive feedback mechanism refines decision plans based on implemented actions and new data, facilitating continuous improvement. Advanced visualization techniques, such as Sankey diagrams, elucidate information flow and action interdependencies, enhancing transparency and stakeholder engagement. Demonstrating versatility across domains, from emergency response planning to business strategy development, the framework presents scalable solutions for multi-criteria decision-making under uncertainty.
Break
Coffee Break 3:00 PM - 3:30 PM
Session 7:
AI Assurance for MDO Applications: Joint Session with Conferences 13051 and 13054
23 April 2024 • 3:30 PM - 5:20 PM EDT | Potomac 4
13051-204
Overview of the responsible AI (RAI) toolkit
(Invited Paper)
23 April 2024 • 3:30 PM - 4:00 PM EDT | Potomac 4
Show Abstract +
The CDAO's Responsible AI team focuses on operationalizing the DoD AI Ethical Principles, sustaining the DoD's tactical edge through concrete actions, processes, and tools. This presentation provides a deep dive into a key piece of the DoD’s approach to Responsible AI: the Responsible AI Toolkit. The Toolkit is a voluntary process through which AI projects can identify, track, and mitigate RAI-related issues (and capitalize on RAI-related opportunities for innovation) via the use of tailorable and modular assessments, tools, and artifacts. The Toolkit rests on the twin pillars of the SHIELD Assessment and the Defense AI Guide on Risk (DAGR), which holistically address AI risk. The Toolkit enables risk management, traceability, and assurance of responsible AI practice, development, and use.
13051-24
23 April 2024 • 4:00 PM - 4:20 PM EDT | Potomac 4
Show Abstract +
Reinforcement learning (RL) agents offer significant value for military applications by effectively navigating
complex, dynamic environments typical of mission engineering and operational analysis. Once trained, these
agents can be employed to inform mission planners on optimal strategies, tactics, or even innovative utilization
of different military platforms within a given scenario. In recent years, RL has become a major research area for
automation and solving complex sequential decision-making problems. However, a notable challenge lies in the
inherent black-box nature of RL models and their inability to explain their decisions and actions. This limitation
serves as a major adoption barrier, especially in Defense. This paper aims to study EXplainable RL (XRL)
within an operational context. XRL is a distinct branch of Explainable Artificial Intelligence (XAI) techniques
that provides the necessary transparency to make AI models more transparent to address this challenge. This
research is an effort to gain insight into the behavior of RL agents in an operational environment and to discuss
explainability and interpretability through the lens of different roles within the decision.
13051-25
23 April 2024 • 4:20 PM - 4:40 PM EDT | Potomac 4
Show Abstract +
Large Language Models act as an all-inclusive ground for engagement. However, the output of these models is not always aligned with the correct human preferences or values. This can produce a large range of inappropriate and offensive content. In human culture/society, proverbs act as a way to transmit and shape human values and behavior. In this research, we experiment to determine whether or not proverbs can play the same role for LLMs in transmitting and aligning to human values. Additionally, we research to determine how we can prompt LLMs to generate their own set of rules for categories of human values and use that as a means to align future-generated content to said values.
13054-25
23 April 2024 • 4:40 PM - 5:00 PM EDT | Potomac 4
Show Abstract +
Artificial Intelligence (AI) systems are deployed to the tactical edge to enhance situational awareness and enable data-driven decision making. As more systems deploy, they are more likely to be captured and reversed engineered by our adversaries, making AI model security a critical aspect of edge AI. In this paper, we describe security features such as encryption and watermarking that are intimately integrated into the model runtime. The goal is to provide an additional layer of security features around the AI model to make reverse engineering difficult.
Show Abstract +
In this work we develop recommendations for best practices in machine learning model development, with an emphasis on risk mitigation methods that can be utilized at each stage in an MLOps development process. We evaluate both adversarial and naturally arising risks by cross-referencing the CRISP-ML(Q) model development architecture with the MITRE ATLAS framework and the AVID AI Vulnerability Database. From the case studies cited in ATLAS and AVID we identify common risks, recommend mitigation strategies, and organize the mitigations around relevant MLOps stages. Finally, we discuss next steps in building out an operational ML pipeline that utilizes our recommendations.
Poster Session
23 April 2024 • 6:00 PM - 7:30 PM EDT | Potomac C
Conference attendees are invited to attend the symposium-wide poster session on Tuesday evening. Come view the SPIE DCS posters, enjoy light refreshments, ask questions, and network with colleagues in your field. Authors of poster papers will be present to answer questions concerning their papers. Attendees are required to wear their conference registration badges to the poster session.
Poster Setup: Tuesday 12:00 PM - 5:30 PM
Poster authors, view poster presentation guidelines and set-up instructions at http://spie.org/DCSPosterGuidelines.
Poster Setup: Tuesday 12:00 PM - 5:30 PM
Poster authors, view poster presentation guidelines and set-up instructions at http://spie.org/DCSPosterGuidelines.
13051-54
On demand | Presented live 23 April 2024
Show Abstract +
In this article, a physics-informed neural networks is employed to address the issue of model identification for complex systems, focusing on its application to a magnetorheological damper setup. The research leverages the Bouc-Wen hysteresis model, a well-established representation of nonlinear behavior, to inform the training process of a series of cascaded neural networks. The objective of this research is to develop a surrogate model capable of accurately predicting the dynamic behavior of MR dampers under various operational conditions. The approach explored in this article combines physics-based insights with the capabilities of neural networks to resolve the complexity associated with the modelling process. This fusion of physical principles and machine learning enables the networks to inherently capture the underlying physics, resulting in enhanced accuracy and interpretability. Through experimentation, the effectiveness of the approach is demonstrated. The model developed exhibits decent predictive performance across a range of input parameters and excitation conditions, offering a promising alternative to conventional black-box machine learning and physics based methods.
13051-55
On demand | Presented live 23 April 2024
Show Abstract +
Adversarial attacks on machine learning models can prove to be a critical problem in cyber security, one example is the use of adversarial domain generation algorithms (DGAs). These adversarial DGAs claim to generate malicious domains that successfully evade deep learning-based DGA detectors. We test two state-of-the-art DGA detectors that make use of deep learning against four different adversarial DGAs. We find that both DGA detectors reach near-perfect performance on real malware domains, but see a dramatic decline in performance on adversarially generated domain names. To counteract the adversarial DGAs, we test two methods to improve adversarial robustness of the detectors: adversarial training and residual loss. The former results in a significant performance increase, whether the latter is not as effective. However, these results indicate that model robustness against adversarial attacks can be improved.
13051-56
On demand | Presented live 23 April 2024
Show Abstract +
The automotive industry must implement some of the best cybersecurity practices to protect electronic systems, communication networks, control algorithms, software, users, and data from malicious attacks. An artificial intelligence (AI) & machine learning (ML) based automotive cybersecurity system could help identify potential vulnerabilities in electronic vehicles as they are part of the vehicular cyber physical systems (CPS). There is a pressing need for better understanding of various attacks and defensive approaches for vehicular CPS to better protect them against any potential threats. This study investigates AI/ML attacks and defenses for vehicular CPS systems to extract a better comprehension of how cybersecurity affects computational components within the vehicular CPS, what the standards are & how they differ, the types of prominent attacks against these systems, and finally an overview of defensive approaches for these attacks .This paper provides a comprehensive overview of the attacks and defensive techniques/methodologies against vehicular cyber-physical systems.
13051-57
On demand | Presented live 23 April 2024
Show Abstract +
Physics-informed neural networks (PINNs) are a recently-developed scientific machine learning (SciML) approach useful for predicting behavior of systems governed by differential equations (DEs). Compared to classical methods like finite element analysis, PINNs can be easily set up and trained on general DEs and geometries. In this work, we will discuss uses of PINNs in different scientific domains. Our focus will be on the use of pinn-jax, an open-source library we have designed to enable easy development and training of PINNs on varied problems, including forward prediction and inverse estimation. We have designed pinn-jax to be easily extensible while also featuring implementations of some common techniques for enhancing PINNs, and we will demonstrate these on different problems. Particular attention will be paid to evaluating PINNs’ performance on problems that vary in behavior across different temporal and spatial scales.
13051-58
23 April 2024 • 6:00 PM - 7:30 PM EDT | Potomac C
Show Abstract +
In the fast-paced and intricate realm of military operations, the ability to generate and analyze Courses of Actions (COAs) rapidly is paramount. Swift and accurate decision-making can be the difference between mission success and failure, and, in many cases, between life and death. Traditional methodologies, reliant on human expertise alone, often grapple with the sheer complexity and dynamic nature of modern warfare. This is where Artificial Intelligence (AI) can be a game-changer.
Interactive Evolutionary Computation (IEC) traditionally thrives in areas where human evaluation forms the backbone of fitness determination, particularly where specific user preferences are crucial or explicit fitness functions are elusive. In the realm of military operations, the concept of evolving Courses of Actions (COAs) for command and control through IEC presents a novel application, but with unique challenges. This paper delves into the pivotal challenges of adapting IEC for military COA development such as: Visualization, Genotype-Phenotype Mapping, Efficiency, and Incorporating learning into IEC.
13051-62
On demand | Presented live 23 April 2024
Show Abstract +
Machine learning is a pervasive technique in contemporary applications, representing a subfield of artificial intelligence dedicated to machines emulating human behaviors. Neural networks, a prominent class of machine learning models, excel in decision-making tasks. Nevertheless, the empirical nature of designing a neural network structure poses challenges, with practitioners often facing the dilemma of incorporating excessive neurons, leading to prolonged training times, or insufficient neurons, resulting in training failures. This paper presents a solution by introducing a method that recommends an appropriate range of neuron numbers for a neural network, leveraging clustering methods to enhance structural design efficiency.
13051-63
On demand | Presented live 23 April 2024
Show Abstract +
Examining throughput of multi-input computer vision pipelines using Nvidia DeepStream SDK and a series of benchmark tests. Using the analysis blueprint to evaluate the point of channel saturation in search of a predictable degradation of the quality of service. Discussing the implications and making software architecture recommendations for the video pipelines implemented at the tactical edge.
13051-26
On demand | Presented live 23 April 2024
Show Abstract +
In the U.S. Army the variables of the Operational Environment (OE) defined as Political, Military, Economic, Social, Information, Infrastructure, Physical Environment, and Time (PMESII-PT) are used for mission analysis and course of action development. In this research, we discuss how to modify existing simulators in order to add non-kinetic operational variables and investigate how they may shape and influence mission outcomes. We use the StarCraft II (SC2) Learning Environment (LE) which provides an interface for Artificial Intelligent agents to control game entities, gather observations, and adjust actions algorithmically. We develop a military-relevant scenario in SC2LE that will be perturbed by the emergence of non-kinetic challenges that affect the simulation outcome. Finally, we investigate reward functions that leverage the OE context to discover an optimal course of action. By integrating PMESII-PT variables into simulators, this research significantly enhances the realism of military simulations, facilitating the development of algorithms that improve operational planning and decision-making for real-world scenarios.
13051-10
On demand | Presented live 23 April 2024
Show Abstract +
Robots moving in formations can serve as the core functionality of coordinated maneuvers, which have real-world applications for logistics-based tasks, mobile asset protection, environmental monitoring, or wide-area surveillance. In order for autonomous multi-robot systems to be useful in such applications, they must be able to seamlessly execute actions, strategies, and behaviors in a coordinated manner. An example of coordinated behavior in multi-robot systems is the movement of robots across terrain in various geometric formations. The general goal of this work is to leverage formations inspired by military mobility and maneuver to guide the behaviors of autonomous multi-robot systems. Generally, our approach aims to develop coordinated multi-robot behaviors by utilizing domain knowledge to guide multi-agent systems. In this work, we measure a global-planning based approach for control of a multi-robot system that is tasked with executing specific geometric formations in a Robot Operating System (ROS) based photo-realistic simulation environment (ARL Unity ROS Simulator (AURS)).
Symposium Plenary on AI/ML + Sustainability
24 April 2024 • 8:30 AM - 10:00 AM EDT | Potomac A
Session Chairs:
Latasha Solomon, DEVCOM Army Research Lab. (United States), Ann Marie Raynal, Sandia National Labs. (United States)
View Full Details: spie.org/dcs/plenary-ai-ml-sustainability
Welcome and opening remarks
24 April 2024 • 8:30 AM - 8:40 AM EDT
Army intelligence data and AI in modern warfare (Plenary Presentation)
Presenter(s): David Pierce, U.S. Army Intelligence (United States)
24 April 2024 • 8:40 AM - 9:20 AM EDT
FUTUR-IC: A three-dimensional optimization path towards building a sustainable microchip industry (Plenary Presentation)
Presenter(s): Anu Agarwal, Massachusetts Institute of Technology, Microphotonics Ctr. and Materials Research Lab. (United States)
24 April 2024 • 9:20 AM - 10:00 AM EDT
Break
Coffee Break 10:00 AM - 10:30 AM
Session 8:
Mission Planning II
24 April 2024 • 10:30 AM - 12:10 PM EDT | Potomac 4
Session Chairs:
Benjamin Jensen, Marine Corps Univ. (United States), Jeffrey Hudack, Air Force Research Lab. (United States)
Opening remarks 10:30 AM to 10:40 AM
13051-205
Modeling and reasoning AI course of action generation hub
(Invited Paper)
24 April 2024 • 10:40 AM - 11:10 AM EDT | Potomac 4
Show Abstract +
The Modeling and Reasoning Artificial Intelligence Course of Action Generation Hub (MRAI Hub) seeks to advance the research and development of artificial intelligence (AI) technologies in support of combat planning by improving sharing and collaboration across the Department of Defense. The MRAI Hub will include a data and information repository, test and evaluation capability, and common operational challenges. This talk will provide a high-level overview of course of action (COA) generation AI, present the MRAI Hub vision and mission, and discuss current status and efforts, including ways to engage.
13051-27
24 April 2024 • 11:10 AM - 11:30 AM EDT | Potomac 4
Show Abstract +
AFRL/RI formed an AI Agent Community of Practice to rapidly identify, evaluate, and engage the right community members with government stakeholders to address high priority operational problem sets and AI technology challenges. A key component of the AI Agent CoP is to use digital AI wargaming competitions to spur research and evaluate AI technology for transitioning to operational partners. This talk will go through the pilot competition for the CoP and future engagement opportunities.
13051-28
24 April 2024 • 11:30 AM - 11:50 AM EDT | Potomac 4
Show Abstract +
Remaining competitive in future conflicts with technologically-advanced competitors requires us to accelerate the development of artificial intelligence (AI) for wargaming. In this study, we contribute to the research of AI methods to develop intelligent agent behaviors in combat simulations by investigating the effectiveness of a multi-model approach as compared to a single-model approach. We find that a multi-model approach improved the mean score by 62.6% over the mean game score of the best-performing single-model alone. Additionally, we find that a multi-model with more embedded behavior models outperforms a multi-model with fewer behavior models.
13051-29
24 April 2024 • 11:50 AM - 12:10 PM EDT | Potomac 4
Show Abstract +
LegatusAI, an Air Force program supporting joint and coalition partners, is addressing the need for the creation of a common architecture of software and hardware for campaign level planning, empowering the efficient development of artificial intelligence (AI) and core AI capabilities. Building off lessons learned in previous command and control programs, LegatusAI seeks to close the capability gap by transforming AI development from non-reusable boutique solutions to an ecosystem of modular and reusable processes for configuring, training, evaluating, and deploying reinforcement learning and game theory agents on planning and wargaming environments. The maturation of this program is a key cornerstone for future collaborative research and development of adaptable AI and plug-in-play wargaming platforms.
Break
Lunch/Exhibition Break 12:10 PM - 2:00 PM
Session 9:
AI/ML and Unmanned Systems: Joint Session with Conferences 13051 and 13055
24 April 2024 • 2:00 PM - 3:20 PM EDT | National Harbor 10
Session Chairs:
Raja Suresh, Arizona State Univ. (United States), Myron E. Hohil, DEVCOM - Armaments Ctr. (United States)
13055-11
24 April 2024 • 2:00 PM - 2:20 PM EDT | National Harbor 10
Show Abstract +
Recently, reinforcement learning has been exhibited as being capable of providing base-level reasoning towards
agent-based intelligence. Agents have had applications of reinforcement learning applied from simpler problem
spaces (such as learning how to play with virtual cards), to learning how to make a physical robot walk. With
reinforcement learning exhibiting capabilities to provide intelligence towards an individual agent, a question
becomes how well could a reinforcement learning agent be able to manage multiple individual agents that
have their execution of tasks abstracted. This challenge is important to recognize when we consider more
advanced applications of reinforcement learning, such as leveraging reinforcement learning to conduct strategic
coordination. In our studies, we have developed a system that leveraged reinforcement learning in an abstracted
competitive strategic environment (currently, a real-time strategy (RTS) engine) to evaluate the effectiveness
of reinforcement learning in automating the strategic approach of individual agents.
13051-31
24 April 2024 • 2:20 PM - 2:40 PM EDT | National Harbor 10
Show Abstract +
In recent years, deep learning-based SLAM techniques have seen significant advancements, driven by a need for better feature extraction and stability against varying light. Emerging methods utilize either Visible, Thermal IR, or their combined forms. Building on insights from new studies, this paper introduces an approach applying deep learning to a Vis-LWIR data, creating a multi-modal and thermal world map. By comparing this map with current scene thermal data, we inform the potential for global thermal estimation. This research highlights the practicality of estimating real-world thermal data.
13055-12
24 April 2024 • 2:40 PM - 3:00 PM EDT | National Harbor 10
Show Abstract +
We explore the use of transfer learning to reduce the data and computing resources required for training convolutional neural networks used by autonomous vehicles for predicting target behavior and improving target tracking as the scenario/environment changes. We demonstrate the ability to adapt to four different changes to the baseline scenario: a new target behavior, mission, adversary, and environment. The results from all four scenarios demonstrate positive transfer learning with reduced training datasets and show that transfer learning is a robust approach to dealing with changing environments even when the input or output dimensions of the neural network are changed.
13051-32
24 April 2024 • 3:00 PM - 3:20 PM EDT | National Harbor 10
Show Abstract +
The evolution of robots from tools to teammates will require them to derive meaningful information about the world around them, translate knowledge and skill into effective planning and action based on stated goals, and communicate with human partners in a natural way. Recent advances in foundation models, large pre-trained models such as large language models and visual language models, will help enable these capabilities. We are using open-vocabulary 3D scene graphs based on foundation models to add scene understanding and natural language interaction to our human-robot teaming research. Semantic scene information can inform context-aware decision making to improve task performance and increase autonomy. We highlight teaming scenarios involving robotic casualty evacuation and stealthy movement through an environment that could benefit from enhanced scene understanding, describe our approach to enabling this capability, and present preliminary results. It is anticipated that advanced perception and planning provided by foundation models will give robots the ability to better understand their environment, share that information with humans, and generate novel courses of action.
Break
Coffee Break 3:20 PM - 3:50 PM
Session 10:
Edge Computing
24 April 2024 • 3:50 PM - 5:30 PM EDT | Potomac 4
Session Chairs:
Nathaniel D. Bastian, U.S. Military Academy (United States), Oscar Munoz, DEVCOM Army Research Lab. (United States)
Show Abstract +
Military operations invariably involve devices at the edge (e.g. sensors, drones, handsets of soldiers, etc.) In edge environments, good network connectivity cannot be assumed due to Denied, Degraded, Intermittent, or Low-bandwidth (DDIL) conditions. A DDIL environment poses unique challenges for deploying AI applications at the edge, particularly in the execution of Machine Learning Operations (MLOps). In this paper, we present a framework to address these challenges by considering three important dimensions: (i)the ML model lifecycle activities, (ii) specific DDIL induced challenges at the edge and (iii) the application stack. We discuss three realistic use cases in detail to explain the use of this approach to identify the underlying design patterns. We believe that use of this framework can lead to a responsive and reliable AI deployment under varying operational conditions.
13051-34
24 April 2024 • 4:10 PM - 4:30 PM EDT | Potomac 4
Show Abstract +
In the era of data-intensive edge computing, the orchestration of Data Distributed Inferencing (DDI) tasks poses a formidable challenge, demanding real-time adaptability to varying network conditions and compute resources. This study introduces an innovative approach to address this challenge, leveraging Gradient Boosting Regression (GBR) as the core predictive modeling technique. The primary objective is to estimate inferencing time based on crucial factors, including bandwidth, compute device type, and the number of compute nodes, allowing for dynamic task placement and optimization in a DDI environment. Our model employs an online learning framework, continuously updating itself as new data streams in, enabling it to swiftly adapt to changing conditions and consistently deliver accurate inferencing time predictions. This research marks a significant step forward in enhancing the efficiency and performance of DDI systems, with implications for real-world applications across various domains, including the Internet Of Things ( IoT), edge computing, and distributed machine learning.
13051-35
24 April 2024 • 4:30 PM - 4:50 PM EDT | Potomac 4
Show Abstract +
In tactical edge networks, the volatility of computational and communication resources complicates the consistent processing of data. Previous work has developed a system that allows execution of inference task applications throughout a network using a variety of adaptations to machine learning models that offer accuracy and latency trade-offs as conditions change in order adaptively perform deterministic resource allocation at the tactical edge. In this paper, we propose utilizing stochastic optimization to analyze the computational time and performance for inference tasks. Instead of adhering to deterministic averages, we use sample average approximation as a technique to optimize and analyze the inherent uncertainties of tactical edge environments, optimizing for totality of inference data rather than average-case scenarios. This paper verifies Jensen’s inequality gap within deterministic optimization and proposes an improved resource allocation algorithm that optimally places tasks throughout a network. We present initial results on a military relevant tactical edge network scenario.
13051-36
24 April 2024 • 4:50 PM - 5:10 PM EDT | Potomac 4
Show Abstract +
Machine learning models designed for edge computing are typically trained offline, before being loaded on embedded platforms. Deployed continual-learning systems have limited access to labeled data, which can negatively affect performance. We implement a modular approach to few-shot class-incremental learning, leveraging state-of-the-art models to improve classification performance while minimizing retraining.
We evaluate the Constrained Few-Shot Class Incremental Learning (C-FSCIL) framework and demonstrate incremental learning of 40 novel CIFAR100 classes under 5-shot sample constraints. A pre-trained ConvNeXt-L achieves a final accuracy of 79.9% over the 100 classes, sacrificing only 7.2% points of accuracy from the base session.
13051-37
24 April 2024 • 5:10 PM - 5:30 PM EDT | Potomac 4
Show Abstract +
Generalized additive models (GAMs) are statistical models that account for non-linear relationships between independent and dependent variables and can yield more accurate relationships compared to strictly linear models. GAMs are composed of smooth functions of independent variables, and possibly tensor product terms accounting for interactions between multiple independent variables, which are fitted using spline basis functions. However, computers onboard military assets with very limited computational power and resources are not able to quickly compute predictions for newly seen feature vectors due to the high computational cost of computing required spline bases functions from scratch. We show that approximating the smooth functions in a trained GAM yield very good estimates that result in minimal loss of accurate compared to the original trained GAM model while significantly decreasing computational cost. These two benefits have resulted in our method being adopted by a primary contractor handling high-fidelity analyses.
Session 11:
Computer Vision II
25 April 2024 • 8:20 AM - 10:00 AM EDT | Potomac 4
Session Chairs:
Christopher R. Ratto, Johns Hopkins Univ. Applied Physics Lab., LLC (United States), Latasha Solomon, DEVCOM Army Research Lab. (United States)
Opening Remarks 8:20-8:30 AM
Show Abstract +
The DARPA Robotic Autonomy in Complex Environment’s with Resiliency (RACER) program is focusing on developing highspeed offroad ground autonomy for military applications. The presentation will discuss the progress made to advance the state of the art in this domain and address the challenges that remain.
13051-38
Efficient fine-grained automatic target recognition through active learning for defense applications
25 April 2024 • 9:00 AM - 9:20 AM EDT | Potomac 4
Show Abstract +
Out of distribution (OOD) detection has shown immense promise to enable Automatic Target Recognition models for defense applications. However, many defense applications have constraints that make current best practices for training OOD detection models challenging. These include: the need to perform fine-grained classification of identified targets, low amounts of labeled data to train models, limited availability of Subject Matter Experts to accurately label new data, and the potential need to incorporate new classes of targets as they are discovered. Given these constraints, we propose to build a fine-grained classifier with robustness against OOD data through an active learning approach - designed to further classify objects after detection through some coarse-grained object detection model. This paper will explore active learning methods for Automatic Target Recognition applications, with experiments conducted using the fine-grained overhead imagery dataset, ShipsRSImageNet, along with samples from the DOTA dataset as an exposure set. Our contributions will include recommendations to achieve fine-grained Automatic Target Recognition with robustness against OOD data
13051-40
25 April 2024 • 9:20 AM - 9:40 AM EDT | Potomac 4
Show Abstract +
Deep learning based approaches, such as Convolutional Neural Nets (CNNs), have shown high performance in classifying contents of images. CNNs, however, have the notable drawbacks of poor explainability, as well as wide performance variance if the underlying imagery data changes from the training baseline. As advanced image processing capabilities are matured, the space-based, on-board detection of objects in space-based imagery is increasingly proposed. The on-board satellite processing applications, which may be resource limited, can drive the need for simpler models with greater explainability. This raises the question of how well can classic computer vision techniques compete with more modern approaches? This paper characterizes and compares the performance of multiple computer vision models for the application of distinguishing a maritime vessel from typical clutter in commercial electro-optical (EO) satellite imagery. A comparison is made between deep learning-based and heritage computer vision applications for maritime applications.
13051-41
25 April 2024 • 9:40 AM - 10:00 AM EDT | Potomac 4
Show Abstract +
In recent years, computer vision research has witnessed transformative changes with the integration of generative artificial intelligence (AI) models. The generative models have been widely researched in the field of semantic segmentation. In this survey paper, we present a comprehensive review of the generative models, with a specific focus on Generative Adversarial Networks (GANs), Diffusion Models (DMs), and Variational Autoencoders (VAEs), in the realm of semantic segmentation. We incorporate the study related to these generative models for generative semantic segmentation, image synthesis, image-annotation pair synthesis, domain adaptation, feature learning, and boundary localization. We also perform a thorough comparative analysis highlighting the approach, task, datasets involved, strengths, and weaknesses of the GANs, DMs, and VAEs-based semantic segmentation models.
Break
Coffee Break 10:00 AM - 10:30 AM
Session 12:
Mission Planning III
25 April 2024 • 10:30 AM - 11:30 AM EDT | Potomac 4
Session Chairs:
Benjamin Jensen, Marine Corps Univ. (United States), Oscar Munoz, DEVCOM Army Research Lab. (United States)
13051-42
25 April 2024 • 10:30 AM - 10:50 AM EDT | Potomac 4
Show Abstract +
We present a method for maritime platform defense using constrained deep reinforcement learning (DRL), showing how competing desires to reliably defend a fleet and conserve inventory may be managed through a dual optimization strategy. Against persistent and variable raids of threats, our agents minimize inventory expenditure subject to a constraint on the average time before a threat impacts the fleet being defended. We evaluate the performance of our method in a realistic simulation environment, exploring the effect of different constraint learning rates on agent behavior in multi-ship scenarios with variable fleet geometry. We speculate on the potential of this method to provide a tunable, trustworthy artificial assistant to human decision-makers tasked with defense scheduling.
13051-43
25 April 2024 • 10:50 AM - 11:10 AM EDT | Potomac 4
Show Abstract +
Flight planning in support of multi-domain operations is a complex and manpower intensive process. Changing constraints result in multiple updates and deviations to the flight plans which eventually impact mission objectives and timing. This work leverages voluminous sources of operational flight planning data and applies machine learning technologies to automatically determine the quality of generated flight plans to enable rapid verification and approval for plan filing. It also identifies preferred routes that are input into planner that results in higher flight plan acceptance rates. Supervised learning is leveraged to predict whether a generated plan will be filed or rejected. The unsupervised learning component identifies flight plan preferences for feedback to the planner system. These models could be deployed in a flight planning operational environment to reduce human effort, cost, and time to generate flyable plans. The results could also be used to improve planner rules and search algorithms.
13051-45
25 April 2024 • 11:10 AM - 11:30 AM EDT | Potomac 4
Show Abstract +
This project explores the application of reinforcement learning and game theory principles in modeling
strategic interactions within a conflict scenario. A Q-learning algorithm is developed to optimize
decision-making by learning from rewards and risks associated with different actions. The
analysis involves tracking the learning progress, evaluating the effectiveness of learned policies, and
testing the algorithm’s performance under varying consequences. Results indicate that incorporating
risk values into the Q-learning process enhances decision quality. Next, game theory is employed
to create a strategic interaction between an agent and an adversary. The agent strategically
manipulates the adversary’s decisions by altering risk values associated with potential actions. Findings
demonstrate the agent’s ability to influence the adversary’s choices, reduce their cumulative
reward, and introduce delays in the adversary’s decision-making. This experiment sheds light on
the dynamic interplay between reinforcement learning and strategic manipulation, offering insights
into decision-making processes within complex environments.
Break
Lunch/Exhibition Break 11:30 AM - 1:00 PM
Session 13:
Data Integration
25 April 2024 • 1:00 PM - 2:30 PM EDT | Potomac 4
Session Chairs:
Tarek Abdelzaher, Univ. of Illinois (United States), Michael Wolmetz, Johns Hopkins Univ. Applied Physics Lab., LLC (United States)
13051-203
Using AI to Support Critical Homeland Security Missions
(Invited Paper)
25 April 2024 • 1:00 PM - 1:30 PM EDT | Potomac 4
Show Abstract +
The responsible and trustworthy use of AI is critical to the safe and secure processing and understanding of the large volumes of data collected everyday by the Department of Homeland Security. For example, the Transportation Security Administration takes five million images a day and screens millions of air passengers, while U.S. Customs and Border Protection processes more than 1 million images per day and more than 1 billion packages per year in achieving their mission to protect the U.S. homeland. In April of 2023, the Department of Homeland (DHS) Secretary, Alejandro Mayorkas, established the department’s first ever artificial intelligence task force (AITF). The guiding mission of the AITF is to advance the use of AI to support critical homeland security missions. This presentation will focus on the findings and results of the DHS AITF including policies for the responsible use of AI within the department, the selection and initiation of AI pilot projects, and the development of a framework for the successful implementation of AI within DHS. Finally, multiple DHS inspired problems will be presented to introduce the breadth of the DHS missions that will be impacted through the use of AI.
13051-46
25 April 2024 • 1:30 PM - 1:50 PM EDT | Potomac 4
Show Abstract +
Vector symbolic architectures (VSA) support semantic information processing via highly compact binary vectors, typically 1-10k bits, making them suitable in resource-constrained settings. VSA can handle structured, semi-structured or unstructured data via a uniform vector space approach. Coalition operations require rapid data sharing with minimal cost in metadata curation and schema alignment; curation and alignment is particularly challenging or even infeasible for external sources not under the coalition's control, such as open source information (OSINF). Large language models (LLMs) facilitate semantic data and metadata alignment via vector representations but are inefficient in resource-constrained settings as the vectors tend to be large. We demonstrate a novel integration of VSA with LLMs, combining the compactness and representational structure of VSAs with the power of semantic matching of LLMs. The approach is demonstrated via an OSINF data discovery use case that allows partners in a coalition operation to share datasets with minimal metadata curation and low communications resource.
13051-47
25 April 2024 • 1:50 PM - 2:10 PM EDT | Potomac 4
Show Abstract +
In order to achieve a comprehensive situational awareness suite, there is a need to standardize how we access and
manipulate information streams. To enable distributed situational awareness, here we present an
approach that standardizes the management of information streams across contexts of varying sensor domains.
These information streams enable individual platforms to dynamically query any relevant information where
it will co-register the sensor streams and be ready for perception systems to rationalize the information. This
system is enabled through the Robot Operating System 2 (ROS 2), which provides a distributed publisher-
subscriber model for communication in addition to standardization for autonomous systems, such as positional
referencing, sensor processing, and dynamic mapping. Based on ROS 2’s system components, we show a system
that enables sensor information to retain its generalized form, which other platforms can manipulate for their
targeted need without the unnecessary destruction of data to perceive within its system.
13051-48
CANCELED: Synchronized analysis of video, imagery, and audio
25 April 2024 • 2:10 PM - 2:30 PM EDT | Potomac 4
Show Abstract +
SAVIA is designed as a layered system that is comprised of a web service and
library code: The web-service is responsible for parallelized and multi-threaded
operations utilizing an asynchronous event dispatching methodology to accelerate
concurrent request and the synchronization process on available data streams. The
library code – on the other hand – is the core functionality offering of the entire
ecosystem. Itself, composed of a tiered system whereby at the highest level, the
primary module, there exist 3 fundamental stream processors (i.e., Video Stream
Processor, Audio Stream Processor, and Sensor Stream Processor) that processes the
individuated streams using complex and reliable feature extraction processes to
perform synchronization on correlated multi-source data (i.e., anomaly detection,
trend analysis, regression analysis, causality analysis, pattern recognition, perceived
truth/ground truth comparison). Furthermore, by automating human inspection of
the data and composing the multi-stream event detection and low-level analysis
output on a global time scale, SAVIA speeds up end-user inference by
identifying and aligning anomalous activities.
Break
Coffee Break 2:30 PM - 3:00 PM
Session 14:
Cyber
25 April 2024 • 3:00 PM - 4:40 PM EDT | Potomac 4
Session Chairs:
Nathaniel D. Bastian, U.S. Military Academy (United States), Danda B. Rawat, Howard Univ. (United States)
13051-49
25 April 2024 • 3:00 PM - 3:20 PM EDT | Potomac 4
Show Abstract +
This paper shares insights from designing a competition in cyber reinforcement learning that focused on network attack simulation with an emphasis on lateral movement. The competition serves as a unique platform for assessing technical readiness and generating valuable data. Our experience reveals key design principles for future cyber reinforcement learning competitions, including the need for a realistic simulation environment, a transparent scoring system, and roles tailored to varied expertise levels in cybersecurity and reinforcement learning. This work serves as a guide for practitioners, researchers, and government sponsors interested in assessing and advancing the state-of-the-art in cyber reinforcement learning through similar competitions.
13051-50
25 April 2024 • 3:20 PM - 3:40 PM EDT | Potomac 4
Show Abstract +
We consider a scenario in multi-domain operations where the users in one domain need to perform searches on information hosted by a provider in another domain. It is very common in many scenarios that information cannot be shared openly across different domains, and users may want to obfuscate their searches and prevent the search provider from learning the intent of their searches. In scenarios where search privacy is important, the use of a large language model can help implement an obfuscation approach relying on generation of decoy queries to obfuscate the real query. In this paper, we consider different alternative approaches to use large language models for search privacy, compare their strengths and weaknesses, and discuss their effectiveness.
13051-51
25 April 2024 • 3:40 PM - 4:00 PM EDT | Potomac 4
Show Abstract +
In this work, we demonstrate the potential of dynamic reinforcement learning (RL) methods to revolutionize cybersecurity. The RL framework we develop is shown to be capable of shutting down an aggressive botnet, which initially uses spear phishing to establish itself in a Department of Defense (DoD) network. To ensure a suitable real-time response, we employ CP, a transformer model trained for network anomaly detection, to factorize the state space accessible to our RL agent. As the fidelity of our cyber scenario is of the utmost importance for meaningful RL training, we leverage the CyberVAN emulation environment to model an appropriate DoD enterprise network to attack and defend. Our work represents an important step towards harnessing the power of RL to automate general and fully-realistic Defensive Cyber Operations (DCOs).
13051-52
25 April 2024 • 4:00 PM - 4:20 PM EDT | Potomac 4
Show Abstract +
Deep learning (DL) has revolutionized machine learning tasks in various domains, but conventional DL methods often demand substantial amounts of labeled data. Semi-supervised learning (SSL) provides an effective solution by incorporating unlabeled data, offering significant advantages in terms of cost and data accessibility. While DL has shown promise with its integration as a component of modern network intrusion detection systems (NIDS), the majority of research in this field focuses on fully supervised learning. However, more recent SSL algorithms leveraging data augmentations do not perform optimally "out of the box" due to the absence of suitable augmentation schemes for packet-level network traffic data. Through the introduction of a novel data augmentation scheme tailored to packet-level network traffic datasets, this paper presents a comprehensive analysis of multiple SSL algorithms for multi-class network traffic detection in a few-shot learning scenario. We find that even relatively simple approaches like vanilla pseudo-labeling can achieve an F1-Score that is within 5% of fully supervised learning methods while utilizing less than 2% of the labeled data.
13051-53
25 April 2024 • 4:20 PM - 4:40 PM EDT | Potomac 4
Show Abstract +
In recognizing the importance of network traffic monitoring for cybersecurity, it is essential to acknowledge that most traditional machine learning models integrated in network intrusion detection systems encounter difficulty in training because acquiring labeled data involves an expensive and time-consuming process. This triggers an in-depth analysis into zero-shot learning techniques specifically designed for raw network traffic detection. Our innovative approach uses clustering combined with the instance-based method for zero-shot learning, enabling classification of network traffic without explicit training on labeled attack data and produces pseudo-labels for unlabeled data. This approach enables the development of accurate models with minimal limited labeled data for making network security more adaptable. Extensive computational experimentation is performed to evaluate our zero-shot learning approach using a real-world network traffic detection dataset. Finally, we offer insights into state-of-art developments and guiding efforts to enhance network security against ever-evolving cyber threats.
Digital Posters
The posters listed below are available exclusively for online viewing during the week of SPIE Defense + Commercial Sensing 2024.
13051-59
On demand | Presenting live 25 April 2024
Show Abstract +
In recent years, there has been a growing convergence between Particle Swarm Optimization (PSO) and Multi-Domain Operations (MDO). This study conducts a comprehensive bibliometric analysis to examine the relationship and academic development of Particle Swarm Optimization (PSO) within the context of Multidisciplinary Design Optimization (MDO). Through an analysis of more than 1,000 scholarly articles covering a period of three decades, we present a comprehensive chronology of research topics, influential publications, and prominent contributors. The research methodology utilized in this study comprises citation analysis, co-authorship networks, and keyword trend mapping. The analysis of our data indicates that the integration of Particle Swarm Optimization (PSO) into Multidisciplinary Design Optimization (MDO) started to gain popularity in the early 2000s. Subsequently, there has been a noticeable increase in research activities in this area, particularly after 2010.
13051-60
On demand | Presenting live 25 April 2024
Show Abstract +
This study delves into a comprehensive comparison between Fuzzy Logic (FL) and Artificial Neural Networks (ANN) in the context of the Inverse Depletion problem. Both methodologies, recognized for their distinct capabilities in handling complex problems, are assessed for their efficacy, accuracy, and computational efficiency. Initial observations highlighted the inherent flexibility of FL in managing uncertainty and the adaptive nature of ANN in recognizing patterns from intricate datasets. A series of benchmark scenarios were established to gauge the performance of each model. Results indicate that while FL offers more interpretable solutions, ANNs often outpace in terms of prediction accuracy. However, the choice between the two largely hinges on the specific requirements of the problem at hand, including the available data quality and the desired output precision. This research underscores the importance of understanding the nuances of each method and provides insights to practitioners on selecting the optimal approach for tackling the Inverse Depletion problem in the field of nuclear forensics.
13051-61
On demand | Presenting live 25 April 2024
Show Abstract +
This Chaos theory has long been a fascinating realm of study, offering insights into systems characterized by sensitivity to initial conditions and complex, unpredictable behaviour. Among these intriguing systems, the "Capsule-Shaped Equilibrium Curve Chaotic System" stands out due to its distinctive and intricate dynamics. In this paper, we present a novel approach to understanding and predicting the behaviour of this complex chaotic system through the application of Recurrent Neural Networks (RNNs). Our study begins with a comprehensive exploration of the capsule-shaped equilibrium curve chaotic system, elucidating its underlying principles and revealing its chaotic nature. By leveraging the power of RNNs, we propose an innovative framework for predicting the temporal evolution of this system. The RNN architecture, with its inherent ability to capture temporal dependencies, offers a promising avenue for modelling the dynamic behaviour of chaotic systems with a high degree of accuracy.
13051-64
On demand | Presenting live 25 April 2024
Show Abstract +
This article addresses the challenge of image restoration using generative adversarial networks (GANs) specifically tailored for images that include undesired objects. This task becomes crucial in scenarios where there is a need to eliminate incidental elements like random pedestrians or obstructive objects such as text, symbols, or drawings that hinder the main content's clarity. These objects can sometimes entirely obscure critical parts of the image, leading to a distortion of the intended information. In this study, we introduce an innovative approach to image reconstruction by leveraging a generative adversarial network (GAN) architecture enhanced with a two-path discriminator for distinct texture and color analysis. Our model, which integrates the stability advantages of Wasserstein GANs, effectively addresses common GAN challenges like mode collapse and training instability. The sophisticated design of our generator and discriminator results in superior image quality for reconstruction tasks, surpassing the performance of traditional single-path GANs in terms of accuracy and visual fidelity.
13051-66
On demand | Presenting live 25 April 2024
Show Abstract +
The growing sector of Extended Reality (XR), comprising VR, AR, and MR, requires successful approaches for localization and mapping. SLAM has gained more importance in XR uses. This research conducts an in-depth bibliometric examination of the utilization of SLAM in XR settings, analyzing publications from the last ten years. We investigate the developing trends, fields of knowledge, prominent contributors, and important publications that have influenced this intersection. Our study indicates a notable increase in research on incorporating SLAM in XR, especially in fields such as gaming, architectural visualization, and remote collaboration. A study of the region highlights North America and the Asia-Pacific as key players in this field of research. The research also discusses the difficulties involved in integrating SLAM into XR, such as handling the computational requirements, guaranteeing real-time processing, and accomplishing successful sensor fusion. This paper strives to provide a thorough overview of the current status of SLAM in XR, pinpointing gaps that already exist and suggesting directions for further research.
Program Committee
Johns Hopkins Univ. Applied Physics Lab., LLC (United States)