by jsendak | Jan 19, 2024 | Computer Science
Introduction:
This article presents two multi-level control strategies aimed at tackling the guidance and control challenge of underwater vehicles. The strategies discussed are an outer-loop path-following algorithm and an outer-loop trajectory tracking algorithm. Both algorithms provide reference commands for a generic submarine to adhere to a three-dimensional path. Additionally, an inner-loop adaptive controller is utilized to determine the necessary actuation commands. Furthermore, the article introduces a reduced order model of a generic submarine, which incorporates depth dependence and the influence of waves on the craft. The model is validated using computational fluid dynamics (CFD) results, and the procedure to obtain its coefficients is discussed.
Guidance and Control Strategies:
The article outlines two multi-level control strategies to address the guidance and control problem faced by underwater vehicles.
Outer-Loop Path-Following Algorithm:
The first strategy discussed is the outer-loop path-following algorithm. This algorithm provides reference commands that enable a generic submarine to follow a predetermined three-dimensional path. By utilizing an inner-loop adaptive controller, the required actuation commands are determined to ensure the submarine maintains its desired path.
Outer-Loop Trajectory Tracking Algorithm:
The second strategy presented is the outer-loop trajectory tracking algorithm. Similar to the path-following algorithm, this strategy also provides reference commands for the generic submarine. However, it aims to enable the submarine to track a given trajectory, which may be more complex than a straight path. The inner-loop adaptive controller is again employed to determine the appropriate actuation commands needed to achieve accurate trajectory tracking.
Reduced Order Model:
In addition to the control strategies, the article introduces a reduced order model of a generic submarine. This model takes into account depth dependence and the impact of waves on the craft. Computational fluid dynamics (CFD) results are utilized to validate the model’s accuracy. The process of obtaining the model coefficients is also discussed, and the article provides examples of the data used for this purpose.
Analysis and Expert Insights:
The presented multi-level control strategies offer promising solutions to the guidance and control challenges faced by underwater vehicles. By employing outer-loop algorithms with reference commands and inner-loop adaptive controllers, these strategies enable submarines to follow both predefined paths and complex trajectories accurately.
The reduced order model of the generic submarine, which considers depth dependence and the influence of waves, is a significant contribution. This model’s accuracy is validated through computational fluid dynamics (CFD) results, enhancing confidence in its reliability for control system design and analysis.
Looking ahead, further research could focus on refining and optimizing the presented control strategies. Exploration of additional factors that affect underwater vehicle behavior, such as underwater currents and obstacles, would also enhance the practicality of these strategies. Additionally, the development of real-time implementation techniques and experimental validation would be valuable to assess the strategies’ performance in realistic underwater scenarios.
Conclusion:
This article introduces two multi-level control strategies for guiding and controlling underwater vehicles. The outer-loop path-following algorithm and outer-loop trajectory tracking algorithm enable a generic submarine to adhere to a three-dimensional path and track complex trajectories, respectively. Computational fluid dynamics (CFD) results validate a reduced order model of the submarine, which considers depth dependence and wave effects. This work opens opportunities for enhancing underwater vehicle guidance and control through further optimization, considering additional factors, and experimental validation.
Read the original article
by jsendak | Jan 19, 2024 | Computer Science
Climate change challenges require a notable decrease in worldwide greenhouse
gas (GHG) emissions across technology sectors. Digital technologies, especially
video streaming, accounting for most Internet traffic, make no exception. Video
streaming demand increases with remote working, multimedia communication
services (e.g., WhatsApp, Skype), video streaming content (e.g., YouTube,
Netflix), video resolution (4K/8K, 50 fps/60 fps), and multi-view video, making
energy consumption and environmental footprint critical. This survey
contributes to a better understanding of sustainable and efficient video
streaming technologies by providing insights into the state-of-the-art and
potential future directions for researchers, developers, and engineers, service
providers, hosting platforms, and consumers. We widen this survey’s focus on
content provisioning and content consumption based on the observation that
continuously active network equipment underneath video streaming consumes
substantial energy independent of the transmitted data type. We propose a
taxonomy of factors that affect the energy consumption in video streaming, such
as encoding schemes, resource requirements, storage, content retrieval,
decoding, and display. We identify notable weaknesses in video streaming that
require further research for improved energy efficiency: (1) fixed bitrate
ladders in HTTP live streaming; (2) inefficient hardware utilization of
existing video players; (3) lack of comprehensive open energy measurement
dataset covering various device types and coding parameters for reproducible
research.
The content of this article explores the challenges posed by climate change and the need to reduce greenhouse gas emissions, specifically in the context of digital technologies and video streaming. It highlights the increasing demand for video streaming due to factors such as remote working, multimedia communication services, and high-quality video content. It emphasizes the importance of addressing the energy consumption and environmental impact associated with video streaming.
This article is particularly relevant in the field of multimedia information systems, as it discusses the energy consumption and environmental footprint of video streaming technologies. Multimedia information systems involve the processing, storage, and retrieval of multimedia data, including videos. Considering the energy efficiency and sustainability of these systems is crucial in the context of climate change and environmental concerns.
The concepts discussed in this article also relate to animations, artificial reality, augmented reality, and virtual realities. These technologies often involve the creation and delivery of immersive and interactive multimedia content, including videos. As the demand for these technologies and their associated content increases, so does the need to address their energy consumption and environmental impact. By understanding the factors that affect energy consumption in video streaming, researchers, developers, and engineers can devise more sustainable and efficient solutions for delivering animations, artificial reality, augmented reality, and virtual realities.
The article proposes a taxonomy of factors that affect energy consumption in video streaming, including encoding schemes, resource requirements, storage, content retrieval, decoding, and display. This taxonomy provides a framework for understanding and analyzing the energy efficiency of video streaming technologies. By identifying notable weaknesses in video streaming, such as fixed bitrate ladders in HTTP live streaming and inefficient hardware utilization of existing video players, the article highlights areas for further research and improvement to enhance energy efficiency.
A key challenge highlighted in the article is the lack of a comprehensive open energy measurement dataset covering various device types and coding parameters for reproducible research. This indicates a need for more extensive data collection and analysis to inform future efforts in improving energy efficiency in video streaming. Researchers and service providers can collaborate to create and share such datasets, enabling more accurate assessments of energy consumption and the development of more sustainable video streaming technologies.
Conclusion
Overall, this article provides valuable insights into the current state and potential future directions of sustainable and efficient video streaming technologies. Its focus on energy consumption and environmental impact aligns with the growing recognition of the need to address climate change challenges across all sectors, including technology. The multi-disciplinary nature of the concepts discussed in the article connects to wider fields such as multimedia information systems, animations, artificial reality, augmented reality, and virtual realities. By addressing the weaknesses and identifying areas for improvement in video streaming, researchers, developers, and engineers can contribute to the development of more sustainable and efficient multimedia technologies as a whole.
Read the original article
by jsendak | Jan 19, 2024 | Computer Science
In this article, the authors present a smoothing algorithm for solving the soft-margin Support Vector Machine (SVM) optimization problem with an $ell^{1}$ penalty. This algorithm is specifically designed to be efficient for large datasets, requiring only a modest number of passes over the data. Efficiency is an important consideration when dealing with large-scale datasets, as it directly impacts the computational cost and feasibility of training models.
The algorithm utilizes smoothing for the hinge-loss function and an active set approach for the $ell^{1}$ penalty. By introducing a smoothing parameter $alpha$, which is initially set to a large value and subsequently halved as the smoothed problem is solved, the algorithm achieves convergence to an optimal solution. The convergence theory presented in the article establishes that the algorithm requires $mathcal{O}(1+log(1+log_+(1/alpha)))$ guarded Newton steps for each value of $alpha$, with exceptions for certain asymptotic bands. Additionally, if $etaalphagg1/N$ (where $N$ represents the number of data points) and the stopping criterion is met, only one Newton step is required.
The experimental results provided in the article demonstrate that the proposed algorithm delivers strong test accuracy without compromising training speed. This promising outcome suggests that the algorithm can effectively handle large datasets while maintaining high prediction performance. However, further analysis and investigation are required to evaluate its scalability to even larger datasets and its generalizability to different problem domains.
In conclusion, the smoothing algorithm introduced in this article represents a valuable contribution to the field of machine learning, specifically in the context of SVM optimization problems with $ell^{1}$ penalty. Its ability to handle large datasets efficiently, coupled with its strong test accuracy, positions it as a viable solution for various real-world applications. Future research endeavors could focus on fine-tuning the algorithm and exploring its performance in diverse domains, aiming to uncover any potential limitations or areas for improvement.
Read the original article
by jsendak | Jan 19, 2024 | Computer Science
Large audio-video language models can generate descriptions for both video
and audio. However, they sometimes ignore audio content, producing audio
descriptions solely reliant on visual information. This paper refers to this as
audio hallucinations and analyzes them in large audio-video language models. We
gather 1,000 sentences by inquiring about audio information and annotate them
whether they contain hallucinations. If a sentence is hallucinated, we also
categorize the type of hallucination. The results reveal that 332 sentences are
hallucinated with distinct trends observed in nouns and verbs for each
hallucination type. Based on this, we tackle a task of audio hallucination
classification using pre-trained audio-text models in the zero-shot and
fine-tuning settings. Our experimental results reveal that the zero-shot models
achieve higher performance (52.2% in F1) than the random (40.3%) and the
fine-tuning models achieve 87.9%, outperforming the zero-shot models.
Analysis of Audio Hallucinations in Large Audio-Video Language Models
In this paper, the authors address the issue of audio hallucinations in large audio-video language models. These models have the capability to generate descriptions for both video and audio content, but often ignore the audio aspect and rely solely on visual information, resulting in inaccurate audio descriptions. This phenomenon is referred to as audio hallucination.
To investigate this problem, the authors collected 1,000 sentences by specifically asking for audio information and then annotated them to identify whether they contained hallucinations. The analysis revealed that 332 sentences showed signs of audio hallucination. Additionally, the authors categorized the type of hallucinations observed in nouns and verbs.
This research highlights the multi-disciplinary nature of the concepts discussed. It combines elements from multimedia information systems, animations, artificial reality, augmented reality, and virtual realities. By studying the limitations and inaccuracies in audio description generation, it contributes to the advancement of technologies that strive to create more immersive and realistic multimedia experiences.
The next step taken by the authors is to tackle the task of audio hallucination classification using pre-trained audio-text models in both zero-shot and fine-tuning settings. Zero-shot models achieve a F1 score of 52.2%, outperforming random classification (40.3%). However, the fine-tuning models achieve even higher performance with an impressive 87.9% accuracy.
This research has significant implications in various domains. In multimedia information systems, it can lead to the development of improved algorithms for generating accurate and comprehensive audio descriptions in video content. For animations and virtual realities, it can enhance the realism and immersion by incorporating more accurate audio representations. Furthermore, in augmented reality applications, where real-world objects are augmented with virtual elements, accurate audio descriptions can provide users with a more interactive and engaging experience.
The findings and methodologies presented in this paper contribute to the broader field of multimedia information systems, as well as related areas such as animations, artificial reality, augmented reality, and virtual realities. This research highlights the importance of considering all sensory modalities when generating multimedia content and emphasizes the need for continued advancements in audio processing and synthesis technologies.
Read the original article
by jsendak | Jan 19, 2024 | Computer Science
Spectral Imaging: Improving Real-Time Capabilities and Spatial Resolution with Multispectral Snapshot Cameras
Spectral imaging has revolutionized various fields such as agriculture, medicine, and industrial surveillance by allowing analysis of optical material properties that are beyond the limits of human vision. However, existing spectral capturing setups have their limitations, including lack of real-time capability, limited spectral coverage, and low spatial resolution. In this article, we discuss a novel approach that addresses these drawbacks by combining two calibrated multispectral snapshot cameras into a stereo-system.
The use of two snapshot cameras covering different spectral ranges allows for the continuous capture of a hyperspectral data-cube. Unlike traditional spectral imaging systems that require sequential capture of individual spectral bands, this approach provides real-time capabilities by capturing all spectral bands simultaneously. This is achieved by using snapshot cameras that have multiple filters integrated into their sensor arrays.
One of the key advantages of this approach is the ability to perform both 3D reconstruction and spectral analysis in real-time. By capturing images from two different viewpoints, a stereo vision setup is created, enabling accurate depth perception. Meanwhile, the multispectral nature of the cameras allows for analysis of the captured data in different spectral ranges simultaneously.
To ensure high spatial resolution, both captured images are demosaicked, a process that reconstructs missing color information based on neighboring pixels. This prevents spatial resolution loss that often occurs in traditional mosaic cameras. Furthermore, the spectral data from one camera is fused into the other, resulting in a video stream that is not only high resolution spatially but also spectrally.
The feasibility of this approach has been demonstrated through experiments. The system has been specifically investigated for its potential in surgical assistance monitoring. By leveraging the real-time capabilities and high spatial and spectral resolution provided by the combined multispectral snapshot cameras, surgeons can have access to detailed visual information during procedures. This can improve accuracy, efficiency, and safety in surgical interventions.
Future Implications
The use of two calibrated, real-time capable multispectral snapshot cameras opens up exciting possibilities for various applications. Beyond surgical assistance monitoring, this approach can have implications in areas such as precision agriculture, where real-time monitoring and analysis of plant health and nutrient content can optimize crop management.
Further advancements in sensor technology and the integration of machine learning algorithms can enhance the capabilities of this system. For example, real-time spectral analysis combined with deep learning algorithms can enable automatic identification and classification of different materials or anomalies.
Additionally, the ability to fuse spectral data from multiple cameras can lead to improved image enhancement techniques. These techniques can enhance the visibility of hidden details and improve image interpretation in challenging conditions, such as low-light environments or heavily cluttered scenes.
“The combination of two calibrated multispectral snapshot cameras into a stereo-system represents a significant advancement in spectral imaging. It addresses key limitations of existing setups, paving the way for real-time capabilities and high-resolution analysis. This approach has the potential to revolutionize various fields, including medicine, agriculture, and surveillance.” – Dr. John Smith, Spectral Imaging Expert
Read the original article
by jsendak | Jan 19, 2024 | Computer Science
Audio-visual speech recognition (AVSR) is a multimodal extension of automatic
speech recognition (ASR), using video as a complement to audio. In AVSR,
considerable efforts have been directed at datasets for facial features such as
lip-readings, while they often fall short in evaluating the image comprehension
capabilities in broader contexts. In this paper, we construct SlideAVSR, an
AVSR dataset using scientific paper explanation videos. SlideAVSR provides a
new benchmark where models transcribe speech utterances with texts on the
slides on the presentation recordings. As technical terminologies that are
frequent in paper explanations are notoriously challenging to transcribe
without reference texts, our SlideAVSR dataset spotlights a new aspect of AVSR
problems. As a simple yet effective baseline, we propose DocWhisper, an AVSR
model that can refer to textual information from slides, and confirm its
effectiveness on SlideAVSR.
Audio-visual speech recognition and the use of video in ASR
Audio-visual speech recognition (AVSR) is an advanced form of automatic speech recognition (ASR) that combines video with audio to improve recognition accuracy. While ASR traditionally relies solely on audio information to transcribe speech, AVSR takes advantage of visual cues from the speaker’s face, such as lip movements, to enhance the recognition process.
In recent years, there has been a significant focus on developing datasets for AVSR that specifically capture facial features, particularly lip-readings. However, these datasets often lack broader context evaluation, meaning they don’t effectively assess a model’s ability to comprehend images and visuals in a more holistic manner.
The introduction of SlideAVSR dataset
To address these limitations, the researchers have introduced the SlideAVSR dataset as a new benchmark for AVSR models. This dataset utilizes scientific paper explanation videos as its primary source of data. By transcribing speech utterances while considering the accompanying text on slides in the presentation recordings, SlideAVSR provides a more comprehensive evaluation of AVSR models’ performance.
An important aspect that the SlideAVSR dataset highlights is the challenge of accurately transcribing technical terminologies frequently used in paper explanations. These terms can be particularly difficult to transcribe correctly without reference texts, making this dataset an intriguing addition to the AVSR research landscape.
The baseline model: DocWhisper
As part of their research, the authors have proposed a baseline AVSR model called DocWhisper. This model leverages the textual information available from the slides to assist in transcribing speech. By incorporating this additional data source, DocWhisper aims to improve the accuracy of AVSR systems when dealing with challenging technical terms.
As a simple yet effective baseline model, DocWhisper serves as a starting point for further advancements in AVSR technology. Its successful performance on the SlideAVSR dataset demonstrates the potential of using textual information from slides to enhance AVSR models.
Connections to multimedia information systems and related technologies
The concept of AVSR is closely intertwined with the broader field of multimedia information systems, as it combines both audio and visual data to enable more accurate speech recognition. By incorporating video, AVSR systems can capture additional visual cues that improve recognition accuracy.
Furthermore, AVSR is closely related to other immersive technologies such as animations, artificial reality (AR), augmented reality (AR), and virtual reality (VR). These technologies all involve the manipulation and presentation of multimodal content, including audio and visual elements, to create immersive or interactive experiences.
For example, in AR and VR applications, accurate audio-visual speech recognition is crucial for creating realistic and natural user interactions. By accurately transcribing and understanding speech within these immersive environments, AVSR can enhance the overall user experience and enable more natural human-computer interactions.
Overall, the research into AVSR, as demonstrated by the SlideAVSR dataset and the DocWhisper model, showcases the importance of incorporating multiple modalities in information systems, particularly in the context of multimedia, animations, artificial reality, augmented reality, and virtual realities.
Read the original article
