by jsendak | Mar 30, 2024 | Computer Science
arXiv:2403.19002v1 Announce Type: new
Abstract: This paper addresses the issue of active speaker detection (ASD) in noisy environments and formulates a robust active speaker detection (rASD) problem. Existing ASD approaches leverage both audio and visual modalities, but non-speech sounds in the surrounding environment can negatively impact performance. To overcome this, we propose a novel framework that utilizes audio-visual speech separation as guidance to learn noise-free audio features. These features are then utilized in an ASD model, and both tasks are jointly optimized in an end-to-end framework. Our proposed framework mitigates residual noise and audio quality reduction issues that can occur in a naive cascaded two-stage framework that directly uses separated speech for ASD, and enables the two tasks to be optimized simultaneously. To further enhance the robustness of the audio features and handle inherent speech noises, we propose a dynamic weighted loss approach to train the speech separator. We also collected a real-world noise audio dataset to facilitate investigations. Experiments demonstrate that non-speech audio noises significantly impact ASD models, and our proposed approach improves ASD performance in noisy environments. The framework is general and can be applied to different ASD approaches to improve their robustness. Our code, models, and data will be released.
Active Speaker Detection in Noisy Environments: A Robust Approach
Active speaker detection (ASD) is an essential task in multimedia information systems where the goal is to identify and track the speaker in a given audio or audio-visual stream. However, in real-world scenarios, the presence of ambient noise can significantly degrade the performance of ASD models. This paper introduces a robust approach, called robust active speaker detection (rASD), which addresses the challenge of detecting the active speaker accurately in noisy environments.
Existing ASD approaches leverage both audio and visual modalities to improve accuracy. However, non-speech sounds in the surrounding environment can interfere with the speaker’s voice, leading to performance degradation. To overcome this, the proposed rASD framework introduces a novel strategy that utilizes audio-visual speech separation as guidance to learn noise-free audio features. These features are then fed into an ASD model in an end-to-end framework, where both the speech separation and ASD tasks are jointly optimized.
This multi-disciplinary approach combines concepts from multimedia information systems, animations, artificial reality, augmented reality, and virtual realities. The integration of audio and visual modalities aligns with the principles of multimedia information systems, where the goal is to process and analyze different forms of media simultaneously. Additionally, the use of audio-visual speech separation techniques relates to the field of animations, as it involves separating speech from non-speech sounds, similar to isolating dialogues from background noises in animated films.
The proposed rASD framework also emphasizes the importance of addressing the audio quality reduction issues that can occur in a naive cascaded two-stage framework. By jointly optimizing the speech separation and ASD tasks, the framework mitigates residual noise and improves the overall audio quality. The dynamics weighted loss approach introduced to train the speech separator further enhances the robustness of the audio features, making the framework more resilient to inherent speech noises.
To validate the effectiveness of the rASD framework, the authors conducted experiments using a real-world noise audio dataset they collected. The experiments demonstrate that non-speech audio noises have a significant impact on ASD models, confirming the need for robust approaches. The proposed rASD framework outperforms existing methods in noisy environments, offering improved accuracy and robustness.
In conclusion, this paper presents a robust approach, the rASD framework, for active speaker detection in noisy environments. The integration of audio-visual speech separation and the joint optimization of both tasks contribute to its effectiveness. The paper’s contribution extends to the wider field of multimedia information systems, animations, artificial reality, augmented reality, and virtual realities by addressing the challenges posed by ambient noise in active speaker detection.
Read the original article
by jsendak | Mar 30, 2024 | AI
arXiv:2403.18827v1 Announce Type: new
Abstract: This article presents a theoretical framework for adapting the Common Model of Cognition to large generative network models within the field of artificial intelligence. This can be accomplished by restructuring modules within the Common Model into shadow production systems that are peripheral to a central production system, which handles higher-level reasoning based on the shadow productions’ output. Implementing this novel structure within the Common Model allows for a seamless connection between cognitive architectures and generative neural networks.
Adapting the Common Model of Cognition for Large Generative Network Models: A Theoretical Framework
Introduction
In the field of artificial intelligence, cognitive architectures play a crucial role in understanding and simulating human-like intelligence. One widely used cognitive architecture is the Common Model of Cognition (CMC), which provides a framework for representing and organizing various cognitive processes. However, as the field progresses and more advanced generative neural network models emerge, there is a need to adapt the CMC to seamlessly integrate with these models. This article presents a theoretical framework that achieves this integration by restructuring the CMC using shadow production systems.
The Common Model of Cognition
Before delving into the proposed framework, it is essential to understand the basics of the Common Model of Cognition. The CMC is a modular architecture that consists of several interconnected modules representing different cognitive processes such as attention, perception, memory, language, and reasoning. These modules interact with each other, allowing for a comprehensive representation of human cognition.
The Need for Integration
Generative neural network models, such as deep learning architectures, have shown remarkable success in various tasks, including image and speech recognition, natural language processing, and even creative tasks like music and art generation. However, these models often lack a higher-level reasoning component that is critical for human-like intelligence.
By adapting the CMC to integrate with generative neural network models, we can create a hybrid architecture that combines the strengths of both approaches. This integration enables the neural network models to handle lower-level perceptual and pattern generation tasks, while the CMC’s central production system utilizes the output from these models for higher-level reasoning and decision-making.
Shadow Production Systems
The key concept in this framework is the introduction of shadow production systems. These systems act as peripheral modules connected to the central production system of the CMC. The shadow production systems receive input from the generative neural network models and generate shadow productions based on their output.
Shadow productions are similar to the rule-based productions used in traditional cognitive architectures. They represent knowledge in the form of condition-action rules that govern behavior. By structuring the CMC with shadow production systems, we establish a seamless connection between the generative neural network models and the higher-level cognitive processes.
Multi-Disciplinary Nature
The proposed framework showcases the multi-disciplinary nature of this research. It draws inspiration from both cognitive psychology, particularly the Common Model of Cognition, and the advancements in generative neural network models within the field of artificial intelligence. By combining these disciplines, we progress towards a more comprehensive understanding and replication of human-like intelligence.
Furthermore, the successful integration of the CMC and generative neural network models requires expertise in cognitive science, machine learning, and computer science. Researchers with diverse backgrounds can collaborate to create a truly interdisciplinary approach that shapes the future of AI.
Future Implications
The theoretical framework presented in this article opens up exciting possibilities for future research and development. By utilizing shadow productions and integrating generative network models into the Common Model of Cognition, we may achieve significant advancements in AI systems with higher-level reasoning capabilities.
Further research can explore the optimization of the shadow production systems, fine-tuning the connection between the CMC and generative neural network models for enhanced performance. Additionally, investigating the transferability of knowledge learned by the neural network models to other domains can lead to more generalizable cognitive architectures.
Conclusion
In conclusion, the adaptation of the Common Model of Cognition to incorporate generative neural network models through the use of shadow production systems presents a promising theoretical framework. This integration combines the strengths of both approaches and paves the way for AI systems with more advanced cognitive capabilities. The multi-disciplinary nature of this work emphasizes the importance of collaboration between cognitive scientists and AI researchers in shaping the future of artificial intelligence.
Read the original article
by jsendak | Mar 30, 2024 | GR & QC Articles
arXiv:2403.18936v1 Announce Type: new
Abstract: The parametrized post-Einsteinian (ppE) framework and its variants are widely used to probe gravity through gravitational-wave tests that apply to a large class of theories beyond general relativity. However, the ppE framework is not truly theory-agnostic as it only captures certain types of deviations from general relativity: those that admit a post-Newtonian series representation in the inspiral of coalescencing compact objects. Moreover, each type of deviation in the ppE framework has to be tested separately, making the whole process computationally inefficient and expensive, possibly obscuring the theoretical interpretation of potential deviations that could be detected in the future. We here present the neural post-Einsteinian (npE) framework, an extension of the ppE formalism that overcomes the above weaknesses using deep-learning neural networks. The core of the npE framework is a variantional autoencoder that maps the discrete ppE theories into a continuous latent space in a well-organized manner. This design enables the npE framework to test many theories simultaneously and to select the theory that best describes the observation in a single parameter estimation run. The smooth extension of the ppE parametrization also allows for more general types of deviations to be searched for with the npE model. We showcase the application of the new npE framework to future tests of general relativity with the fifth observing run of the LIGO-Virgo-KAGRA collaboration. In particular, the npE framework is demonstrated to efficiently explore modifications to general relativity beyond what can be mapped by the ppE framework, including modifications coming from higher-order curvature corrections to the Einstein-Hilbert action at high post-Newtonian order, and dark-photon interactions in possibly hidden sectors of matter that do not admit a post-Newtonian representation.
Exploring Gravity Beyond General Relativity: The Neural Post-Einsteinian (npE) Framework
Gravity, one of the fundamental forces of nature, has been extensively studied and validated through the lens of general relativity. However, the limitations of this theory and the need to explore alternative explanations have led to the development of frameworks like the parametrized post-Einsteinian (ppE) framework. While the ppE framework has been valuable in probing deviations from general relativity, it has certain limitations that restrict its scope and efficiency.
The npE framework, introduced in this article, extends the ppE formalism by harnessing the power of deep-learning neural networks. By utilizing a variational autoencoder, the npE framework maps the discrete ppE theories into a continuous latent space. This innovative design allows for the simultaneous testing of multiple theories and the identification of the theory that best fits the observation through a single parameter estimation run.
Unlike the ppE framework, the npE framework is capable of exploring a broader range of deviations from general relativity. It can efficiently search for modifications beyond those captured by the post-Newtonian series representation, such as higher-order curvature corrections to the Einstein-Hilbert action and dark-photon interactions in hidden sectors of matter.
To demonstrate the potential of the npE framework, we showcase its application to future tests of general relativity with the fifth observing run of the LIGO-Virgo-KAGRA collaboration. This collaboration aims to detect gravitational waves and study their properties with unprecedented precision. The npE framework, with its enhanced capability to explore a wider range of theories, offers an invaluable tool in unraveling the mysteries of gravity.
Roadmap for Readers:
- Understanding the Limitations of the ppE Framework: Explore the constraints and computational inefficiencies associated with the ppE framework in capturing deviations from general relativity.
- Introducing the npE Framework: Learn about the neural post-Einsteinian framework and its core component, the variational autoencoder, which enables efficient testing of multiple theories.
- Advantages of the npE Framework: Understand how the npE framework overcomes the limitations of the ppE framework by exploring a broader range of deviations.
- Showcasing the npE Framework: Discover the application of the npE framework in future tests of general relativity with the LIGO-Virgo-KAGRA collaboration’s fifth observing run.
- Potential Challenges and Opportunities: Explore the challenges that may arise in implementing the npE framework and the opportunities it presents in uncovering new insights about gravity.
Challenges: The implementation of the npE framework may require substantial computational resources and expertise in deep learning. Additionally, the theoretical interpretation of potential deviations detected by the npE framework may pose challenges in understanding the underlying physics.
Opportunities: The npE framework offers a more comprehensive and efficient approach to exploring gravity beyond general relativity. It has the potential to uncover new phenomena and shed light on the mysteries of dark matter, dark energy, and other fundamental aspects of the universe.
Read the original article
by jsendak | Mar 30, 2024 | Computer Science
The present paper examines the effectiveness of code live-load models in accurately estimating vehicular loads on bridge substructures. The study utilizes realistic traffic vehicle data from four Weigh-in-Motion databases, which provide an authentic representation of vehicle information. This ensures that the examination of the bridges studied is based on real-world data.
The evaluation includes various bridge models, ranging from single-span girder bridges to two-, three-, and four-span continuous pinned-support girder bridges. By comparing the extreme force values obtained from the Weigh-in-Motion databases with those predicted by selected code live-load models, the study assesses the accuracy of the models.
The exceedance rates, which indicate the frequency with which the predicted forces exceed the actual forces, are presented in a spectra format, organized by span length. The analysis reveals significant variations in these exceedance rates, underscoring the need for improvements in code live-load models to achieve more accurate estimations of the forces transferred to bridge substructures.
Enhancing the accuracy of these models is crucial in achieving more consistent reliability levels for a range of limit states, such as resistance, fatigue, serviceability, and cracking. By refining code live-load models, engineers and policymakers can ensure that bridges are designed to withstand the actual loads they will experience, leading to improved bridge safety and longevity.
Read the original article
by jsendak | Mar 30, 2024 | Art
