2025-03-06

Title: ClipGrader: Leveraging Vision-Language Models for Robust Label Quality Assessment in Object Detection

• Authors: Hong Lu, Yali Bian, Rahul C. Shah
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.02897
• Pdf URL: https://arxiv.org/pdf/2503.02897
• Copy Paste: [[2503.02897]] ClipGrader: Leveraging Vision-Language Models for Robust Label Quality Assessment in Object Detection(https://arxiv.org/abs/2503.02897)
• Keywords: quality assessment
• Abstract: High-quality annotations are essential for object detection models, but ensuring label accuracy - especially for bounding boxes - remains both challenging and costly. This paper introduces ClipGrader, a novel approach that leverages vision-language models to automatically assess the accuracy of bounding box annotations. By adapting CLIP (Contrastive Language-Image Pre-training) to evaluate both class label correctness and spatial precision of bounding box, ClipGrader offers an effective solution for grading object detection labels. Tested on modified object detection datasets with artificially disturbed bounding boxes, ClipGrader achieves 91% accuracy on COCO with a 1.8% false positive rate. Moreover, it maintains 87% accuracy with a 2.1% false positive rate when trained on just 10% of the COCO data. ClipGrader also scales effectively to larger datasets such as LVIS, achieving 79% accuracy across 1,203 classes. Our experiments demonstrate ClipGrader's ability to identify errors in existing COCO annotations, highlighting its potential for dataset refinement. When integrated into a semi-supervised object detection (SSOD) model, ClipGrader readily improves the pseudo label quality, helping achieve higher mAP (mean Average Precision) throughout the training process. ClipGrader thus provides a scalable AI-assisted tool for enhancing annotation quality control and verifying annotations in large-scale object detection datasets.
• 摘要：高质量的注释对于对象检测模型至关重要，但是确保标签准确性（尤其是边界框）仍然具有挑战性且昂贵。本文介绍了剪贴画，这是一种新型的方法，它利用视觉模型自动评估边界框注释的准确性。通过调整剪辑（对比性语言图像的预训练）来评估类标签的正确性和边界框的空间精度，Clipgrader为对象检测标签提供了有效的解决方案。在带有人工干扰边界框的修改对象检测数据集上测试，夹板以1.8％的假阳性速率在可可的可可速度上达到91％的精度。此外，当仅10％的可可数据训练时，它保持87％的准确性，误报率为2.1％。剪贴剂还有效地扩展到较大的数据集，例如LVI，在1,203个类别中达到79％的准确性。我们的实验表明，夹具在现有的可可注释中识别错误的能力，突出了其数据集细化的潜力。当集成到半监督对象检测（SSOD）模型中时，夹板很容易改善伪标签质量，从而帮助在整个训练过程中实现更高的MAP（平均平均精度）。因此，夹具提供了可扩展的AI辅助工具，用于增强注释质量控制和验证大规模对象检测数据集中的注释。

Title: Straight-Line Diffusion Model for Efficient 3D Molecular Generation

• Authors: Yuyan Ni, Shikun Feng, Haohan Chi, Bowen Zheng, Huan-ang Gao, Wei-Ying Ma, Zhi-Ming Ma, Yanyan Lan
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.02918
• Pdf URL: https://arxiv.org/pdf/2503.02918
• Copy Paste: [[2503.02918]] Straight-Line Diffusion Model for Efficient 3D Molecular Generation(https://arxiv.org/abs/2503.02918)
• Keywords: generation, generative
• Abstract: Diffusion-based models have shown great promise in molecular generation but often require a large number of sampling steps to generate valid samples. In this paper, we introduce a novel Straight-Line Diffusion Model (SLDM) to tackle this problem, by formulating the diffusion process to follow a linear trajectory. The proposed process aligns well with the noise sensitivity characteristic of molecular structures and uniformly distributes reconstruction effort across the generative process, thus enhancing learning efficiency and efficacy. Consequently, SLDM achieves state-of-the-art performance on 3D molecule generation benchmarks, delivering a 100-fold improvement in sampling efficiency. Furthermore, experiments on toy data and image generation tasks validate the generality and robustness of SLDM, showcasing its potential across diverse generative modeling domains.
• 摘要：基于扩散的模型在分子产生中表现出了很大的希望，但通常需要大量的采样步骤来生成有效的样品。在本文中，我们引入了一种新型的直线扩散模型（SLDM），以通过制定遵循线性轨迹的扩散过程来解决此问题。所提出的过程与分子结构的噪声灵敏度特征很好地对齐，并均匀地在整个生成过程中分布了重建工作，从而提高了学习效率和功效。因此，SLDM在3D分子生成基准测试中实现了最先进的性能，从而提高了100倍的采样效率。此外，关于玩具数据和图像生成任务的实验验证了SLDM的一般性和鲁棒性，展示了其在各种生成建模域中的潜力。

Title: Robust time series generation via Schrödinger Bridge: a comprehensive evaluation

• Authors: Alexandre Alouadi, Baptiste Barreau, Laurent Carlier, Huyên Pham
• Subjects: cs.LG, eess.SY
• Abstract URL: https://arxiv.org/abs/2503.02943
• Pdf URL: https://arxiv.org/pdf/2503.02943
• Copy Paste: [[2503.02943]] Robust time series generation via Schrödinger Bridge: a comprehensive evaluation(https://arxiv.org/abs/2503.02943)
• Keywords: generation, generative
• Abstract: We investigate the generative capabilities of the Schrödinger Bridge (SB) approach for time series. The SB framework formulates time series synthesis as an entropic optimal interpolation transport problem between a reference probability measure on path space and a target joint distribution. This results in a stochastic differential equation over a finite horizon that accurately captures the temporal dynamics of the target time series. While the SB approach has been largely explored in fields like image generation, there is a scarcity of studies for its application to time series. In this work, we bridge this gap by conducting a comprehensive evaluation of the SB method's robustness and generative performance. We benchmark it against state-of-the-art (SOTA) time series generation methods across diverse datasets, assessing its strengths, limitations, and capacity to model complex temporal dependencies. Our results offer valuable insights into the SB framework's potential as a versatile and robust tool for time series generation.
• 摘要：我们研究了时间序列的Schrödinger桥（SB）方法的生成能力。 SB框架将时间序列综合作为熵最佳插值传输问题，在路径空间上的参考概率度量与目标关节分布之间。这会在有限的地平线上产生随机微分方程，从而准确捕获目标时间序列的时间动力学。尽管SB方法在图像生成等领域进行了很大的探讨，但其在时间序列中的应用很少。在这项工作中，我们通过对SB方法的鲁棒性和生成性能进行全面评估来弥合这一差距。我们对其对不同数据集的最新时间（SOTA）时间序列生成方法进行基准测试，以评估其优势，局限性和对复杂时间依赖性建模的能力。我们的结果为SB Framework作为时间序列的多功能和强大工具提供了宝贵的见解。

Title: Integrating Predictive and Generative Capabilities by Latent Space Design via the DKL-VAE Model

• Authors: Boris N. Slautin, Utkarsh Pratiush, Doru C. Lupascu, Maxim A. Ziatdinov, Sergei V. Kalinin
• Subjects: cs.LG, cond-mat.mtrl-sci
• Abstract URL: https://arxiv.org/abs/2503.02978
• Pdf URL: https://arxiv.org/pdf/2503.02978
• Copy Paste: [[2503.02978]] Integrating Predictive and Generative Capabilities by Latent Space Design via the DKL-VAE Model(https://arxiv.org/abs/2503.02978)
• Keywords: generation, generative
• Abstract: We introduce a Deep Kernel Learning Variational Autoencoder (VAE-DKL) framework that integrates the generative power of a Variational Autoencoder (VAE) with the predictive nature of Deep Kernel Learning (DKL). The VAE learns a latent representation of high-dimensional data, enabling the generation of novel structures, while DKL refines this latent space by structuring it in alignment with target properties through Gaussian Process (GP) regression. This approach preserves the generative capabilities of the VAE while enhancing its latent space for GP-based property prediction. We evaluate the framework on two datasets: a structured card dataset with predefined variational factors and the QM9 molecular dataset, where enthalpy serves as the target function for optimization. The model demonstrates high-precision property prediction and enables the generation of novel out-of-training subset structures with desired characteristics. The VAE-DKL framework offers a promising approach for high-throughput material discovery and molecular design, balancing structured latent space organization with generative flexibility.
• 摘要：我们介绍了深内核学习变异自动编码器（VAE-DKL）框架，该框架将变量自动编码器（VAE）的生成能力与深内核学习（DKL）的预测性质相结合。 VAE了解了高维数据的潜在表示，从而实现了新结构的产生，而DKL通过通过高斯工艺（GP）回归以与目标性能的对准结构来完善了这一潜在空间。这种方法保留了VAE的生成能力，同时增强了其基于GP的财产预测的潜在空间。我们在两个数据集上评估了框架：一个带有预定义变异因子和QM9分子数据集的结构化卡数据集，其中焓是进行优化的目标函数。该模型展示了高精度的性质预测，并可以生成具有期望特征的新型训练子集结构。 VAE-DKL框架为高通量材料发现和分子设计提供了一种有希望的方法，平衡结构性潜在空间组织具有生成性灵活性。

Title: Out-of-Distribution Generalization on Graphs via Progressive Inference

• Authors: Yiming Xu, Bin Shi, Zhen Peng, Huixiang Liu, Bo Dong, Chen Chen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.02988
• Pdf URL: https://arxiv.org/pdf/2503.02988
• Copy Paste: [[2503.02988]] Out-of-Distribution Generalization on Graphs via Progressive Inference(https://arxiv.org/abs/2503.02988)
• Keywords: generation
• Abstract: The development and evaluation of graph neural networks (GNNs) generally follow the independent and identically distributed (i.i.d.) assumption. Yet this assumption is often untenable in practice due to the uncontrollable data generation mechanism. In particular, when the data distribution shows a significant shift, most GNNs would fail to produce reliable predictions and may even make decisions randomly. One of the most promising solutions to improve the model generalization is to pick out causal invariant parts in the input graph. Nonetheless, we observe a significant distribution gap between the causal parts learned by existing methods and the ground truth, leading to undesirable performance. In response to the above issues, this paper presents GPro, a model that learns graph causal invariance with progressive inference. Specifically, the complicated graph causal invariant learning is decomposed into multiple intermediate inference steps from easy to hard, and the perception of GPro is continuously strengthened through a progressive inference process to extract causal features that are stable to distribution shifts. We also enlarge the training distribution by creating counterfactual samples to enhance the capability of the GPro in capturing the causal invariant parts. Extensive experiments demonstrate that our proposed GPro outperforms the state-of-the-art methods by 4.91% on average. For datasets with more severe distribution shifts, the performance improvement can be up to 6.86%.
• 摘要：图形神经网络（GNN）的开发和评估通常遵循独立和相同分布的（I.I.D.）假设。然而，由于无法控制的数据生成机制，这种假设在实践中通常是站不住脚的。特别是，当数据分布显示重大变化时，大多数GNN将无法产生可靠的预测，甚至可能随机做出决策。改善模型概括的最有希望的解决方案之一是在输入图中挑选因果不变部分。尽管如此，我们观察到通过现有方法学到的因果部分与地面真理所学的因果部分之间的分布差距很大，从而导致了不良的绩效。为了响应上述问题，本文介绍了GPRO，该模型以渐进推理学习图形因果不变性。具体而言，复杂的图形因果不变学习被分解为从易于硬到硬的多个中间推理步骤，并且通过渐进的推理过程不断增强GPRO的感知，以提取出稳定的分布转移的因果特征。我们还通过创建反事实样本来扩大训练分布，以增强GPRO在捕获因果不变零件方面的能力。广泛的实验表明，我们提出的GPRO的表现平均超过了最新方法4.91％。对于更严重的分配变化的数据集，性能提高可能高达6.86％。

Title: Generative assimilation and prediction for weather and climate

• Authors: Shangshang Yang, Congyi Nai, Xinyan Liu, Weidong Li, Jie Chao, Jingnan Wang, Leyi Wang, Xichen Li, Xi Chen, Bo Lu, Ziniu Xiao, Niklas Boers, Huiling Yuan, Baoxiang Pan
• Subjects: cs.LG, physics.ao-ph
• Abstract URL: https://arxiv.org/abs/2503.03038
• Pdf URL: https://arxiv.org/pdf/2503.03038
• Copy Paste: [[2503.03038]] Generative assimilation and prediction for weather and climate(https://arxiv.org/abs/2503.03038)
• Keywords: generative
• Abstract: Machine learning models have shown great success in predicting weather up to two weeks ahead, outperforming process-based benchmarks. However, existing approaches mostly focus on the prediction task, and do not incorporate the necessary data assimilation. Moreover, these models suffer from error accumulation in long roll-outs, limiting their applicability to seasonal predictions or climate projections. Here, we introduce Generative Assimilation and Prediction (GAP), a unified deep generative framework for assimilation and prediction of both weather and climate. By learning to quantify the probabilistic distribution of atmospheric states under observational, predictive, and external forcing constraints, GAP excels in a broad range of weather-climate related tasks, including data assimilation, seamless prediction, and climate simulation. In particular, GAP is competitive with state-of-the-art ensemble assimilation, probabilistic weather forecast and seasonal prediction, yields stable millennial simulations, and reproduces climate variability from daily to decadal time scales.
• 摘要：机器学习模型在预测未来两个星期的天气方面已取得了巨大的成功，优于基于过程的基准测试。但是，现有方法主要集中在预测任务上，并且不纳入必要的数据同化。此外，这些模型会在长期推出中累积错误，从而将其适用性限制在季节性预测或气候预测中。在这里，我们引入了生成同化和预测（GAP），这是一个统一的深层生成框架，用于同化和预测天气和气候。通过学习量化观察性，预测性和外部强迫限制下大气状态的概率分布，差距在广泛的天气气候相关任务中出色，包括数据同化，无缝预测和气候模拟。特别是，GAP具有最新的合奏同化，概率天气预测和季节性预测的竞争力，产生稳定的千禧一代模拟，并从每天到衰减时间尺度再现气候变化。

Title: Hopfield Networks Meet Big Data: A Brain-Inspired Deep Learning Framework for Semantic Data Linking

• Authors: Ashwin Viswanathan Kannan, Johnson P Thomas, Abhimanyu Mukerji
• Subjects: cs.LG, cs.AI, cs.DC, cs.NE
• Abstract URL: https://arxiv.org/abs/2503.03084
• Pdf URL: https://arxiv.org/pdf/2503.03084
• Copy Paste: [[2503.03084]] Hopfield Networks Meet Big Data: A Brain-Inspired Deep Learning Framework for Semantic Data Linking(https://arxiv.org/abs/2503.03084)
• Keywords: generation
• Abstract: The exponential rise in data generation has led to vast, heterogeneous datasets crucial for predictive analytics and decision-making. Ensuring data quality and semantic integrity remains a challenge. This paper presents a brain-inspired distributed cognitive framework that integrates deep learning with Hopfield networks to identify and link semantically related attributes across datasets. Modeled on the dual-hemisphere functionality of the human brain, the right hemisphere assimilates new information while the left retrieves learned representations for association. Our architecture, implemented on MapReduce with Hadoop Distributed File System (HDFS), leverages deep Hopfield networks as an associative memory mechanism to enhance recall of frequently co-occurring attributes and dynamically adjust relationships based on evolving data patterns. Experiments show that associative imprints in Hopfield memory are reinforced over time, ensuring linked datasets remain contextually meaningful and improving data disambiguation and integration accuracy. Our results indicate that combining deep Hopfield networks with distributed cognitive processing offers a scalable, biologically inspired approach to managing complex data relationships in large-scale environments.
• 摘要：数据生成的指数增加导致了对预测分析和决策至关重要的广泛的，异质的数据集。确保数据质量和语义完整性仍然是一个挑战。本文提出了一个受脑启发的分布式认知框架，该框架将深度学习与Hopfield网络集成在一起，以识别和链接跨数据集的语义相关属性。右半球以人脑的双半球功能为模型，在左侧检索相关的表示表示时吸收了新信息。我们的体系结构是通过Hadoop分布式文件系统（HDFS）在MapReduce上实施的，它利用Deep Hopfield网络作为一种关联的内存机制来增强对经常共同发生的属性的回忆，并基于演变的数据模式，动态调整关系。实验表明，随着时间的流逝，Hopfield内存中的关联烙印会得到加强，以确保链接的数据集在上下文上仍然有意义，并改善了数据歧义和集成精度。我们的结果表明，将深层网络与分布式认知处理相结合，为在大规模环境中管理复杂的数据关系提供了可扩展的，具有生物学启发的方法。

Title: A Survey of Foundation Models for Environmental Science

• Authors: Runlong Yu, Shengyu Chen, Yiqun Xie, Xiaowei Jia
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03142
• Pdf URL: https://arxiv.org/pdf/2503.03142
• Copy Paste: [[2503.03142]] A Survey of Foundation Models for Environmental Science(https://arxiv.org/abs/2503.03142)
• Keywords: generation
• Abstract: Modeling environmental ecosystems is essential for effective resource management, sustainable development, and understanding complex ecological processes. However, traditional methods frequently struggle with the inherent complexity, interconnectedness, and limited data of such systems. Foundation models, with their large-scale pre-training and universal representations, offer transformative opportunities by integrating diverse data sources, capturing spatiotemporal dependencies, and adapting to a broad range of tasks. This survey presents a comprehensive overview of foundation model applications in environmental science, highlighting advancements in forward prediction, data generation, data assimilation, downscaling, model ensembling, and decision-making across domains. We also detail the development process of these models, covering data collection, architecture design, training, tuning, and evaluation. By showcasing these emerging methods, we aim to foster interdisciplinary collaboration and advance the integration of cutting-edge machine learning for sustainable solutions in environmental science.
• 摘要：建模环境生态系统对于有效的资源管理，可持续发展和了解复杂的生态过程至关重要。但是，传统方法经常与此类系统的固有复杂性，互连性和有限的数据相处。基金会模型及其大规模的预训练和普遍表示形式，通过整合多种数据源，捕获时空依赖性并适应广泛的任务来提供变革的机会。这项调查介绍了环境科学中基础模型应用的全面概述，强调了远期预测，数据生成，数据同化，缩小，模型结合，结合模型和跨领域的决策的进步。我们还详细介绍了这些模型的开发过程，涵盖了数据收集，体系结构设计，培训，调整和评估。通过展示这些新兴方法，我们旨在促进跨学科的合作，并促进尖端机器学习在环境科学领域的可持续解决方案。

Title: Position: Model Collapse Does Not Mean What You Think

• Authors: Rylan Schaeffer, Joshua Kazdan, Alvan Caleb Arulandu, Sanmi Koyejo
• Subjects: cs.LG, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2503.03150
• Pdf URL: https://arxiv.org/pdf/2503.03150
• Copy Paste: [[2503.03150]] Position: Model Collapse Does Not Mean What You Think(https://arxiv.org/abs/2503.03150)
• Keywords: generative
• Abstract: The proliferation of AI-generated content online has fueled concerns over \emph{model collapse}, a degradation in future generative models' performance when trained on synthetic data generated by earlier models. Industry leaders, premier research journals and popular science publications alike have prophesied catastrophic societal consequences stemming from model collapse. In this position piece, we contend this widespread narrative fundamentally misunderstands the scientific evidence. We highlight that research on model collapse actually encompasses eight distinct and at times conflicting definitions of model collapse, and argue that inconsistent terminology within and between papers has hindered building a comprehensive understanding of model collapse. To assess how significantly different interpretations of model collapse threaten future generative models, we posit what we believe are realistic conditions for studying model collapse and then conduct a rigorous assessment of the literature's methodologies through this lens. While we leave room for reasonable disagreement, our analysis of research studies, weighted by how faithfully each study matches real-world conditions, leads us to conclude that certain predicted claims of model collapse rely on assumptions and conditions that poorly match real-world conditions, and in fact several prominent collapse scenarios are readily avoidable. Altogether, this position paper argues that model collapse has been warped from a nuanced multifaceted consideration into an oversimplified threat, and that the evidence suggests specific harms more likely under society's current trajectory have received disproportionately less attention.
• 摘要：在线AI生成的内容的扩散引起了人们对\ emph {Model Collapse}的关注，这是对以前模型生成的合成数据培训的未来生成模型性能的退化。行业领导者，主要的研究期刊和流行科学出版物都预言了灾难性的社会后果，这是由于模型崩溃而造成的。在这个职位上，我们认为这种广泛的叙事从根本上误解了科学证据。我们强调，对模型崩溃的研究实际上包含了八个不同的，有时甚至是模型崩溃的定义相互矛盾的，并认为论文之间和论文之间的术语不一致，这阻碍了人们对模型崩溃的全面了解。为了评估模型崩溃的显着不同的解释威胁着未来的生成模型，我们认为我们认为是研究模型崩溃的现实条件，然后通过该镜头对文献方法进行严格的评估。尽管我们留出了合理分歧的空间，但我们对研究的分析是由每个研究忠实地与现实世界中的条件相匹配的加权，这使我们得出结论，某些预测模型崩溃的主张取决于与现实世界中不良情况相匹配的假设和条件，实际上可以轻松避免几种突出的崩溃方案。总的来说，该立场论文认为，模型崩溃已从细微的多方面考虑到过度简化的威胁中扭曲，并且证据表明，在社会目前的轨迹下，特定的危害更大的危害受到了不成比例的关注。

Title: SpinML: Customized Synthetic Data Generation for Private Training of Specialized ML Models

• Authors: Jiang Zhang, Rohan Xavier Sequeira, Konstantinos Psounis
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2503.03160
• Pdf URL: https://arxiv.org/pdf/2503.03160
• Copy Paste: [[2503.03160]] SpinML: Customized Synthetic Data Generation for Private Training of Specialized ML Models(https://arxiv.org/abs/2503.03160)
• Keywords: generation
• Abstract: Specialized machine learning (ML) models tailored to users needs and requests are increasingly being deployed on smart devices with cameras, to provide personalized intelligent services taking advantage of camera data. However, two primary challenges hinder the training of such models: the lack of publicly available labeled data suitable for specialized tasks and the inaccessibility of labeled private data due to concerns about user privacy. To address these challenges, we propose a novel system SpinML, where the server generates customized Synthetic image data to Privately traIN a specialized ML model tailored to the user request, with the usage of only a few sanitized reference images from the user. SpinML offers users fine-grained, object-level control over the reference images, which allows user to trade between the privacy and utility of the generated synthetic data according to their privacy preferences. Through experiments on three specialized model training tasks, we demonstrate that our proposed system can enhance the performance of specialized models without compromising users privacy preferences.
• 摘要：专门的机器学习（ML）模型量身定制了用户需求和请求，越来越多地通过相机部署在智能设备上，以利用相机数据提供个性化的智能服务。但是，两个主要挑战阻碍了此类模型的培训：由于对用户隐私的疑虑，缺乏适合专业任务的公开标记数据以及标记的私人数据的无法访问。为了应对这些挑战，我们提出了一个新颖的系统SpinML，在该系统中，服务器生成自定义的合成图像数据，以私下训练针对用户请求的专门ML模型，仅使用了几个用户的少数消毒参考图像。 SpinML为用户提供了对参考图像的细粒度，对象级的控制，这使用户可以根据其隐私偏好在生成的合成数据的隐私和实用性之间进行交易。通过对三项专业模型培训任务的实验，我们证明了我们提出的系统可以增强专用模型的性能，而不会损害用户的隐私偏好。

Title: Find Matching Faces Based On Face Parameters

• Authors: Setu A. Bhatt, Harshadkumar B. Prajapati, Vipul K. Dabhi, Ankush Tyagi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03204
• Pdf URL: https://arxiv.org/pdf/2503.03204
• Copy Paste: [[2503.03204]] Find Matching Faces Based On Face Parameters(https://arxiv.org/abs/2503.03204)
• Keywords: generation
• Abstract: This paper presents an innovative approach that enables the user to find matching faces based on the user-selected face parameters. Through gradio-based user interface, the users can interactively select the face parameters they want in their desired partner. These user-selected face parameters are transformed into a text prompt which is used by the Text-To-Image generation model to generate a realistic face image. Further, the generated image along with the images downloaded from the this http URL are processed through face detection and feature extraction model, which results in high dimensional vector embedding of 512 dimensions. The vector embeddings generated from the downloaded images are stored into vector database. Now, the similarity search is carried out between the vector embedding of generated image and the stored vector embeddings. As a result, it displays the top five similar faces based on the user-selected face parameters. This contribution holds a significant potential to turn into a high-quality personalized face matching tool.
• 摘要：本文提出了一种创新的方法，使用户能够根据用户选择的面部参数找到匹配面。通过基于Gradio的用户界面，用户可以在所需的合作伙伴中交互选择所需的面部参数。这些用户选择的面部参数被转换为文本提示，文本对图像生成模型将其用于生成逼真的面部图像。此外，生成的图像以及从此HTTP URL下载的图像通过面部检测和特征提取模型处理，从而导致高维矢量嵌入512个维度。从下载的图像生成的向量嵌入式存储在矢量数据库中。现在，在生成图像的向量嵌入和存储的向量嵌入之间进行相似性搜索。结果，它根据用户选择的面部参数显示了前五个相似的面孔。这项贡献具有变成高质量的个性化面部匹配工具的巨大潜力。

Title: An Analytical Theory of Power Law Spectral Bias in the Learning Dynamics of Diffusion Models

• Authors: Binxu Wang
• Subjects: cs.LG, cs.CV, math.ST, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.03206
• Pdf URL: https://arxiv.org/pdf/2503.03206
• Copy Paste: [[2503.03206]] An Analytical Theory of Power Law Spectral Bias in the Learning Dynamics of Diffusion Models(https://arxiv.org/abs/2503.03206)
• Keywords: generative
• Abstract: We developed an analytical framework for understanding how the learned distribution evolves during diffusion model training. Leveraging the Gaussian equivalence principle, we derived exact solutions for the gradient-flow dynamics of weights in one- or two-layer linear denoiser settings with arbitrary data. Remarkably, these solutions allowed us to derive the generated distribution in closed form and its KL divergence through training. These analytical results expose a pronounced power-law spectral bias, i.e., for weights and distributions, the convergence time of a mode follows an inverse power law of its variance. Empirical experiments on both Gaussian and image datasets demonstrate that the power-law spectral bias remains robust even when using deeper or convolutional architectures. Our results underscore the importance of the data covariance in dictating the order and rate at which diffusion models learn different modes of the data, providing potential explanations for why earlier stopping could lead to incorrect details in image generative models.
• 摘要：我们开发了一个分析框架，以了解扩散模型训练期间学习分布的演变。利用高斯等效原理，我们得出了具有任意数据的一层或两层线性denoiser设置中权重的梯度流动动力学的精确解。值得注意的是，这些解决方案使我们能够通过训练以封闭形式及其KL差异得出生成的分布。这些分析结果暴露了明显的幂律频谱偏差，即，对于权重和分布，模式的收敛时间遵循其方差的反向幂定律。高斯和图像数据集的经验实验表明，即使使用更深入或卷积的体系结构，幂律频谱偏置也仍然坚固。我们的结果强调了数据协方差在规定扩散模型学习不同模式的顺序和速率方面的重要性，从而提供了潜在的解释，以说明为什么早期停止可能导致图像生成模型中的细节不正确。

Title: Mocap-2-to-3: Lifting 2D Diffusion-Based Pretrained Models for 3D Motion Capture

• Authors: Zhumei Wang, Zechen Hu, Ruoxi Guo, Huaijin Pi, Ziyong Feng, Sida Peng, Xiaowei Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03222
• Pdf URL: https://arxiv.org/pdf/2503.03222
• Copy Paste: [[2503.03222]] Mocap-2-to-3: Lifting 2D Diffusion-Based Pretrained Models for 3D Motion Capture(https://arxiv.org/abs/2503.03222)
• Keywords: generative
• Abstract: Recovering absolute poses in the world coordinate system from monocular views presents significant challenges. Two primary issues arise in this context. Firstly, existing methods rely on 3D motion data for training, which requires collection in limited environments. Acquiring such 3D labels for new actions in a timely manner is impractical, severely restricting the model's generalization capabilities. In contrast, 2D poses are far more accessible and easier to obtain. Secondly, estimating a person's absolute position in metric space from a single viewpoint is inherently more complex. To address these challenges, we introduce Mocap-2-to-3, a novel framework that decomposes intricate 3D motions into 2D poses, leveraging 2D data to enhance 3D motion reconstruction in diverse scenarios and accurately predict absolute positions in the world coordinate system. We initially pretrain a single-view diffusion model with extensive 2D data, followed by fine-tuning a multi-view diffusion model for view consistency using publicly available 3D data. This strategy facilitates the effective use of large-scale 2D data. Additionally, we propose an innovative human motion representation that decouples local actions from global movements and encodes geometric priors of the ground, ensuring the generative model learns accurate motion priors from 2D data. During inference, this allows for the gradual recovery of global movements, resulting in more plausible positioning. We evaluate our model's performance on real-world datasets, demonstrating superior accuracy in motion and absolute human positioning compared to state-of-the-art methods, along with enhanced generalization and scalability. Our code will be made publicly available.
• 摘要：从单眼观点中恢复世界坐标系统中的绝对姿势提出了重大挑战。在这种情况下，出现了两个主要问题。首先，现有方法依赖于3D运动数据进行培训，这需要在有限的环境中收集。及时购买此类3D标签是不切实际的，严重限制了模型的概括能力。相比之下，2D姿势更容易获得，更容易获得。其次，从单个角度来估算一个人在公制空间中的绝对位置本质上更为复杂。为了应对这些挑战，我们介绍了MOCAP-2至3，这是一个新颖的框架，将复杂的3D运动分解为2D姿势，利用2D数据来增强不同场景中的3D运动重建，并准确预测世界坐标系统中的绝对位置。最初，我们为具有广泛的2D数据的单视图扩散模型预算，然后使用公开可用的3D数据对多视图扩散模型进行微调以查看一致性。该策略有助于有效使用大型2D数据。此外，我们提出了一种创新的人类运动表示形式，该表示将当地动作与全球运动脱离并编码地面的几何先验，从而确保生成模型从2D数据中学习准确的运动先验。在推断期间，这允许逐渐恢复全球运动，从而导致更合理的定位。我们评估了模型在现实世界数据集上的性能，与最新方法相比，运动的精度和绝对人类定位的精度以及增强的概括和可扩展性。我们的代码将公开可用。

Title: Computational Analysis of Degradation Modeling in Blind Panoramic Image Quality Assessment

• Authors: Jiebin Yan, Ziwen Tan, Jiale Rao, Lei Wu, Yifan Zuo, Yuming Fang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03255
• Pdf URL: https://arxiv.org/pdf/2503.03255
• Copy Paste: [[2503.03255]] Computational Analysis of Degradation Modeling in Blind Panoramic Image Quality Assessment(https://arxiv.org/abs/2503.03255)
• Keywords: quality assessment
• Abstract: Blind panoramic image quality assessment (BPIQA) has recently brought new challenge to the visual quality community, due to the complex interaction between immersive content and human behavior. Although many efforts have been made to advance BPIQA from both conducting psychophysical experiments and designing performance-driven objective algorithms, \textit{limited content} and \textit{few samples} in those closed sets inevitably would result in shaky conclusions, thereby hindering the development of BPIQA, we refer to it as the \textit{easy-database} issue. In this paper, we present a sufficient computational analysis of degradation modeling in BPIQA to thoroughly explore the \textit{easy-database issue}, where we carefully design three types of experiments via investigating the gap between BPIQA and blind image quality assessment (BIQA), the necessity of specific design in BPIQA models, and the generalization ability of BPIQA models. From extensive experiments, we find that easy databases narrow the gap between the performance of BPIQA and BIQA models, which is unconducive to the development of BPIQA. And the easy databases make the BPIQA models be closed to saturation, therefore the effectiveness of the associated specific designs can not be well verified. Besides, the BPIQA models trained on our recently proposed databases with complicated degradation show better generalization ability. Thus, we believe that much more efforts are highly desired to put into BPIQA from both subjective viewpoint and objective viewpoint.
• 摘要：由于沉浸式内容与人类行为之间的复杂相互作用，盲目的全景图像质量评估（BPIQA）最近对视觉质量社区带来了新的挑战。尽管已经做出了许多努力，从而从进行心理物理实验和设计以性能驱动的客观算法的方式进行BPIQA，但在这些封闭的集合中，\ textit {有限的内容}和\ textit {lime samples}不可避免地会导致摇摇欲坠的结论，从而使Bpiqa的发展造成更轻松的发展。在本文中，我们对BPIQA中的降解模型进行了足够的计算分析，以彻底探索\ textIt {easy-database essups}，在该{easy-database问题}中，我们通过研究BPIQA和盲图像质量评估（BIQA）之间的差距（BIXA），BPIQA模型的特定设计和BPIQA模型的特定设计和BPIQA的特定设计。从广泛的实验中，我们发现简单的数据库缩小了BPIQA和BIQA模型的性能之间的差距，这对于BPIQA的发展是无关的。而且，简单的数据库使BPIQA模型封闭于饱和度，因此无法很好地验证相关特定设计的有效性。此外，在我们最近提出的具有复杂降解的数据库中训练的BPIQA模型显示出更好的泛化能力。因此，我们认为，从主观的观点和客观观点上都非常希望将更多的努力纳入BPIQA。

Title: Optimizing for the Shortest Path in Denoising Diffusion Model

• Authors: Ping Chen, Xingpeng Zhang, Zhaoxiang Liu, Huan Hu, Xiang Liu, Kai Wang, Min Wang, Yanlin Qian, Shiguo Lian
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03265
• Pdf URL: https://arxiv.org/pdf/2503.03265
• Copy Paste: [[2503.03265]] Optimizing for the Shortest Path in Denoising Diffusion Model(https://arxiv.org/abs/2503.03265)
• Keywords: generation
• Abstract: In this research, we propose a novel denoising diffusion model based on shortest-path modeling that optimizes residual propagation to enhance both denoising efficiency and this http URL on Denoising Diffusion Implicit Models (DDIM) and insights from graph theory, our model, termed the Shortest Path Diffusion Model (ShortDF), treats the denoising process as a shortest-path problem aimed at minimizing reconstruction error. By optimizing the initial residuals, we improve the efficiency of the reverse diffusion process and the quality of the generated this http URL experiments on multiple standard benchmarks demonstrate that ShortDF significantly reduces diffusion time (or steps) while enhancing the visual fidelity of generated samples compared to prior this http URL work, we suppose, paves the way for interactive diffusion-based applications and establishes a foundation for rapid data generation. Code is available at this https URL.
• 摘要：在这项研究中，我们提出了一种基于最短路径模型的新型deno denoing扩散模型，该模型优化了剩余的传播，以提高降级效率和对deno deno deno deno扩散隐含模型（DDIM）的HTTP URL和图理论的见解，我们的模型将最短的路径扩散模型（ShortDF）称为简短的问题，并将其视为简短的问题。通过优化初始残差，我们提高了反向扩散过程的效率以及在多个标准基准上进行的HTTP URL实验生成的质量表明，与先前的HTTP URL相比，相比，互动的数据相比，ShortDF显着降低了扩散时间（或步骤），同时增强了生成的样品的视觉效率，并建立了相互作用的差异。代码可在此HTTPS URL上找到。

Title: Towards Effective and Sparse Adversarial Attack on Spiking Neural Networks via Breaking Invisible Surrogate Gradients

• Authors: Li Lun, Kunyu Feng, Qinglong Ni, Ling Liang, Yuan Wang, Ying Li, Dunshan Yu, Xiaoxin Cui
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03272
• Pdf URL: https://arxiv.org/pdf/2503.03272
• Copy Paste: [[2503.03272]] Towards Effective and Sparse Adversarial Attack on Spiking Neural Networks via Breaking Invisible Surrogate Gradients(https://arxiv.org/abs/2503.03272)
• Keywords: generation
• Abstract: Spiking neural networks (SNNs) have shown their competence in handling spatial-temporal event-based data with low energy consumption. Similar to conventional artificial neural networks (ANNs), SNNs are also vulnerable to gradient-based adversarial attacks, wherein gradients are calculated by spatial-temporal back-propagation (STBP) and surrogate gradients (SGs). However, the SGs may be invisible for an inference-only model as they do not influence the inference results, and current gradient-based attacks are ineffective for binary dynamic images captured by the dynamic vision sensor (DVS). While some approaches addressed the issue of invisible SGs through universal SGs, their SGs lack a correlation with the victim model, resulting in sub-optimal performance. Moreover, the imperceptibility of existing SNN-based binary attacks is still insufficient. In this paper, we introduce an innovative potential-dependent surrogate gradient (PDSG) method to establish a robust connection between the SG and the model, thereby enhancing the adaptability of adversarial attacks across various models with invisible SGs. Additionally, we propose the sparse dynamic attack (SDA) to effectively attack binary dynamic images. Utilizing a generation-reduction paradigm, SDA can fully optimize the sparsity of adversarial perturbations. Experimental results demonstrate that our PDSG and SDA outperform state-of-the-art SNN-based attacks across various models and datasets. Specifically, our PDSG achieves 100% attack success rate on ImageNet, and our SDA obtains 82% attack success rate by modifying only 0.24% of the pixels on CIFAR10DVS. The code is available at this https URL .
• 摘要：尖峰神经网络（SNN）表明了它们在处理基于空间事件的数据中具有低能消耗的能力。与常规的人工神经网络（ANN）相似，SNN也容易受到基于梯度的对抗攻击的影响，其中梯度是通过空间 - 周期性后反向传播（STBP）和替代梯度（SGS）计算的。但是，对于仅推理模型不影响推理结果，SGS可能是看不见的，并且当前基于梯度的攻击对动态视觉传感器（DVS）捕获的二进制动态图像无效。尽管某些方法通过通用SG解决了隐形SG的问题，但其SG与受害者模型缺乏相关性，导致了次优的性能。此外，现有基于SNN的二进制攻击的不可识别仍然不足。在本文中，我们引入了一种创新的潜在依赖性替代梯度（PDSG）方法，以在SG和模型之间建立牢固的联系，从而增强了具有无形SGS的各种模型的对抗性攻击的适应性。此外，我们提出稀疏的动态攻击（SDA）来有效攻击二进制动态图像。 SDA利用生成减少范式可以完全优化对抗扰动的稀疏性。实验结果表明，我们的PDSG和SDA胜过各种模型和数据集的基于SNN的最先进攻击。具体而言，我们的PDSG在ImageNet上达到了100％的攻击成功率，我们的SDA通过仅修改CIFAR10DVS上的0.24％的像素来获得82％的攻击成功率。该代码可在此HTTPS URL上找到。

Title: Video Super-Resolution: All You Need is a Video Diffusion Model

• Authors: Zhihao Zhan, Wang Pang, Xiang Zhu, Yechao Bai
• Subjects: cs.CV, cs.LG, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.03355
• Pdf URL: https://arxiv.org/pdf/2503.03355
• Copy Paste: [[2503.03355]] Video Super-Resolution: All You Need is a Video Diffusion Model(https://arxiv.org/abs/2503.03355)
• Keywords: super-resolution, generation
• Abstract: We present a generic video super-resolution algorithm in this paper, based on the Diffusion Posterior Sampling framework with an unconditional video generation model in latent space. The video generation model, a diffusion transformer, functions as a space-time model. We argue that a powerful model, which learns the physics of the real world, can easily handle various kinds of motion patterns as prior knowledge, thus eliminating the need for explicit estimation of optical flows or motion parameters for pixel alignment. Furthermore, a single instance of the proposed video diffusion transformer model can adapt to different sampling conditions without re-training. Due to limited computational resources and training data, our experiments provide empirical evidence of the algorithm's strong super-resolution capabilities using synthetic data.
• 摘要：我们在本文中提出了一种通用的视频超分辨率算法，该算法基于潜在的后验采样框架，该框架具有潜在空间中无条件的视频生成模型。视频生成模型是一种扩散变压器，用作时空模型。我们认为，了解现实世界的物理学的强大模型可以轻松处理各种运动模式作为先验知识，从而消除了对像素对齐的光流或运动参数的明确估计的需求。此外，所提出的视频扩散变压器模型的一个实例可以适应不同的采样条件，而无需重新训练。由于计算资源和培训数据有限，我们的实验提供了算法使用合成数据的强大超分辨率功能的经验证据。

Title: Simplicial SMOTE: Oversampling Solution to the Imbalanced Learning Problem

• Authors: Oleg Kachan, Andrey Savchenko, Gleb Gusev
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03418
• Pdf URL: https://arxiv.org/pdf/2503.03418
• Copy Paste: [[2503.03418]] Simplicial SMOTE: Oversampling Solution to the Imbalanced Learning Problem(https://arxiv.org/abs/2503.03418)
• Keywords: generation
• Abstract: SMOTE (Synthetic Minority Oversampling Technique) is the established geometric approach to random oversampling to balance classes in the imbalanced learning problem, followed by many extensions. Its idea is to introduce synthetic data points of the minor class, with each new point being the convex combination of an existing data point and one of its k-nearest neighbors. In this paper, by viewing SMOTE as sampling from the edges of a geometric neighborhood graph and borrowing tools from the topological data analysis, we propose a novel technique, Simplicial SMOTE, that samples from the simplices of a geometric neighborhood simplicial complex. A new synthetic point is defined by the barycentric coordinates w.r.t. a simplex spanned by an arbitrary number of data points being sufficiently close rather than a pair. Such a replacement of the geometric data model results in better coverage of the underlying data distribution compared to existing geometric sampling methods and allows the generation of synthetic points of the minority class closer to the majority class on the decision boundary. We experimentally demonstrate that our Simplicial SMOTE outperforms several popular geometric sampling methods, including the original SMOTE. Moreover, we show that simplicial sampling can be easily integrated into existing SMOTE extensions. We generalize and evaluate simplicial extensions of the classic Borderline SMOTE, Safe-level SMOTE, and ADASYN algorithms, all of which outperform their graph-based counterparts.
• 摘要：Smote（合成的少数族裔过采样技术）是既定的几何方法，用于随机过度采样以平衡不平衡的学习问题中的类别，然后进行许多扩展。它的想法是引入小类的合成数据点，每个新点都是现有数据点的凸组合，及其最初的邻居之一。在本文中，通过将Smote视为从几何邻域图的边缘进行采样，并从拓扑数据分析中借用工具，我们提出了一种新颖的技术，简单的Smote，从几何邻域简单型复合物的简单中取样。 Barycentric坐标W.R.T.定义了一个新的合成点。由任意数量的数据点跨越的单纯形，而不是一对。与现有的几何抽样方法相比，这种替换几何数据模型可以更好地覆盖基础数据分布，并允许在决策边界上更接近多数类的少数族裔类的合成点。我们在实验上证明，我们的简单Smote优于几种流行的几何抽样方法，包括原始Smote。此外，我们表明简单采样可以轻松地集成到现有的Smote扩展中。我们概括和评估经典边界线，安全级别的Smote和Adasyn算法的简单扩展，所有这些算法的表现都优于基于图形的对应物。

Title: Automatic Drywall Analysis for Progress Tracking and Quality Control in Construction

• Authors: Mariusz Trzeciakiewicz, Aleixo Cambeiro Barreiro, Niklas Gard, Anna Hilsmann, Peter Eisert
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03422
• Pdf URL: https://arxiv.org/pdf/2503.03422
• Copy Paste: [[2503.03422]] Automatic Drywall Analysis for Progress Tracking and Quality Control in Construction(https://arxiv.org/abs/2503.03422)
• Keywords: quality assessment
• Abstract: Digitalization in the construction industry has become essential, enabling centralized, easy access to all relevant information of a building. Automated systems can facilitate the timely and resource-efficient documentation of changes, which is crucial for key processes such as progress tracking and quality control. This paper presents a method for image-based automated drywall analysis enabling construction progress and quality assessment through on-site camera systems. Our proposed solution integrates a deep learning-based instance segmentation model to detect and classify various drywall elements with an analysis module to cluster individual wall segments, estimate camera perspective distortions, and apply the corresponding corrections. This system extracts valuable information from images, enabling more accurate progress tracking and quality assessment on construction sites. Our main contributions include a fully automated pipeline for drywall analysis, improving instance segmentation accuracy through architecture modifications and targeted data augmentation, and a novel algorithm to extract important information from the segmentation results. Our modified model, enhanced with data augmentation, achieves significantly higher accuracy compared to other architectures, offering more detailed and precise information than existing approaches. Combined with the proposed drywall analysis steps, it enables the reliable automation of construction progress and quality assessment.
• 摘要：建筑行业的数字化已成为必不可少的，可以使集中式，轻松访问建筑物的所有相关信息。自动化系统可以促进更改的及时和资源有效文档，这对于诸如进度跟踪和质量控制等关键过程至关重要。本文介绍了一种基于图像的自动干墙分析的方法，可通过现场摄像头系统进行施工进度和质量评估。我们提出的解决方案集成了一个基于深度学习的实例分割模型，以通过分析模块来检测和对各种干墙元素进行分类，以聚集单个壁段，估计摄像机的透视扭曲，并应用相应的校正。该系统从图像中提取有价值的信息，从而在施工站点上更准确地进行进度跟踪和质量评估。我们的主要贡献包括用于干墙分析的全自动管道，通过体系结构修改和目标数据增强提高实例分割精度，以及一种新颖的算法，以从分割结果中提取重要信息。与其他体系结构相比，随着数据扩展的增强，我们改进的模型的准确性明显更高，比现有方法提供了更详细和更精确的信息。结合拟议的干墙分析步骤，它可以可靠地自动化施工进度和质量评估。

Title: Rethinking Synthetic Data definitions: A privacy driven approach

• Authors: Vibeke Binz Vallevik, Serena Elizabeth Marshall, Aleksandar Babic, Jan Franz Nygaard
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03506
• Pdf URL: https://arxiv.org/pdf/2503.03506
• Copy Paste: [[2503.03506]] Rethinking Synthetic Data definitions: A privacy driven approach(https://arxiv.org/abs/2503.03506)
• Keywords: generation, generative
• Abstract: Synthetic data is gaining traction as a cost-effective solution for the increasing data demands of AI development and can be generated either from existing knowledge or derived data captured from real-world events. The source of the synthetic data generation and the technique used significantly impacts its residual privacy risk and therefore its opportunity for sharing. Traditional classification of synthetic data types no longer fit the newer generation techniques and there is a need to better align the classification with practical needs. We suggest a new way of grouping synthetic data types that better supports privacy evaluations to aid regulatory policymaking. Our novel classification provides flexibility to new advancements like deep generative methods and offers a more practical framework for future applications.
• 摘要：综合数据正在作为对AI开发的增加数据需求的一种成本效益的解决方案，可以从现有知识或从现实世界中捕获的数据中生成。合成数据生成和使用该技术的来源显着影响其剩余隐私风险，从而影响其共享机会。合成数据类型的传统分类不再适合新的一代技术，并且有必要更好地将分类与实际需求保持一致。我们建议一种对合成数据类型进行分组的新方法，以更好地支持隐私评估以帮助监管决策。我们的新颖分类为新的进步提供了灵活性，例如深层生成方法，并为将来的应用提供了更实用的框架。

Title: High-Quality Virtual Single-Viewpoint Surgical Video: Geometric Autocalibration of Multiple Cameras in Surgical Lights

• Authors: Yuna Kato, Mariko Isogawa, Shohei Mori, Hideo Saito, Hiroki Kajita, Yoshifumi Takatsume
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03558
• Pdf URL: https://arxiv.org/pdf/2503.03558
• Copy Paste: [[2503.03558]] High-Quality Virtual Single-Viewpoint Surgical Video: Geometric Autocalibration of Multiple Cameras in Surgical Lights(https://arxiv.org/abs/2503.03558)
• Keywords: generation
• Abstract: Occlusion-free video generation is challenging due to surgeons' obstructions in the camera field of view. Prior work has addressed this issue by installing multiple cameras on a surgical light, hoping some cameras will observe the surgical field with less occlusion. However, this special camera setup poses a new imaging challenge since camera configurations can change every time surgeons move the light, and manual image alignment is required. This paper proposes an algorithm to automate this alignment task. The proposed method detects frames where the lighting system moves, realigns them, and selects the camera with the least occlusion. This algorithm results in a stabilized video with less occlusion. Quantitative results show that our method outperforms conventional approaches. A user study involving medical doctors also confirmed the superiority of our method.
• 摘要：由于外科医生在摄像头视野中的障碍物，无遮挡视频生成具有挑战性。先前的工作已经通过在手术灯上安装多个摄像头解决了这一问题，希望某些摄像头能够以更少的遮挡观察手术场。但是，这种特殊的摄像头设置构成了新的成像挑战，因为相机配置每次外科医生都会移动光线时都会改变，并且需要手动图像对齐。本文提出了一种算法来自动执行此对齐任务。所提出的方法检测到照明系统移动，重新调整它们并以最小遮挡选择相机的框架。该算法导致稳定的视频，其阻塞较少。定量结果表明，我们的方法表现优于常规方法。一项涉及医生的用户研究也证实了我们方法的优势。

Title: Domain Consistent Industrial Decarbonisation of Global Coal Power Plants

• Authors: Waqar Muhammad Ashraf, Vivek Dua, Ramit Debnath
• Subjects: cs.LG, eess.SY
• Abstract URL: https://arxiv.org/abs/2503.03571
• Pdf URL: https://arxiv.org/pdf/2503.03571
• Copy Paste: [[2503.03571]] Domain Consistent Industrial Decarbonisation of Global Coal Power Plants(https://arxiv.org/abs/2503.03571)
• Keywords: generation
• Abstract: Machine learning and optimisation techniques (MLOPT) hold significant potential to accelerate the decarbonisation of industrial systems by enabling data-driven operational improvements. However, the practical application of MLOPT in industrial settings is often hindered by a lack of domain compliance and system-specific consistency, resulting in suboptimal solutions with limited real-world applicability. To address this challenge, we propose a novel human-in-the-loop (HITL) constraint-based optimisation framework that integrates domain expertise with data-driven methods, ensuring solutions are both technically sound and operationally feasible. We demonstrate the efficacy of this framework through a case study focused on enhancing the thermal efficiency and reducing the turbine heat rate of a 660 MW supercritical coal-fired power plant. By embedding domain knowledge as constraints within the optimisation process, our approach yields solutions that align with the plant's operational patterns and are seamlessly integrated into its control systems. Empirical validation confirms a mean improvement in thermal efficiency of 0.64\% and a mean reduction in turbine heat rate of 93 kJ/kWh. Scaling our analysis to 59 global coal power plants with comparable capacity and fuel type, we estimate a cumulative lifetime reduction of 156.4 million tons of carbon emissions. These results underscore the transformative potential of our HITL-MLOPT framework in delivering domain-compliant, implementable solutions for industrial decarbonisation, offering a scalable pathway to mitigate the environmental impact of coal-based power generation worldwide.
• 摘要：机器学习和优化技术（MLOPT）具有通过启用数据驱动的运营改进来加速工业系统脱碳的巨大潜力。但是，MLOPT在工业环境中的实际应用通常受到缺乏域合规性和特定于系统特定的一致性的阻碍，从而导致了现实世界中适用有限的次优解决方案。为了应对这一挑战，我们提出了一种基于基于数据驱动的方法的域专业知识的新型人类（HITL）约束优化框架，以确保解决方案在技术上是合理的，而且在操作上是可行的。我们通过案例研究的重点是提高热效率并降低660 MW超临界燃煤电厂的涡轮热速率，从而证明了该框架的功效。通过将域知识嵌入优化过程中的约束，我们的方法产生了与植物的运营模式保持一致的解决方案，并无缝集成到其控制系统中。经验验证证实，热效率的平均提高为0.64 \％，涡轮热速率的平均降低为93 kJ/kWh。将我们的分析扩展到具有可比容量和燃料类型的59个全球煤炭发电厂，我们估计累计终生减少了15640万吨碳排放。这些结果强调了我们的HITL-MLOPT框架在提供符合域名的，可实施的工业脱碳的解决方案方面的变革潜力，提供了可扩展的途径来减轻全球煤炭发电的环境影响。

Title: Towards Understanding Text Hallucination of Diffusion Models via Local Generation Bias

• Authors: Rui Lu, Runzhe Wang, Kaifeng Lyu, Xitai Jiang, Gao Huang, Mengdi Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03595
• Pdf URL: https://arxiv.org/pdf/2503.03595
• Copy Paste: [[2503.03595]] Towards Understanding Text Hallucination of Diffusion Models via Local Generation Bias(https://arxiv.org/abs/2503.03595)
• Keywords: generation
• Abstract: Score-based diffusion models have achieved incredible performance in generating realistic images, audio, and video data. While these models produce high-quality samples with impressive details, they often introduce unrealistic artifacts, such as distorted fingers or hallucinated texts with no meaning. This paper focuses on textual hallucinations, where diffusion models correctly generate individual symbols but assemble them in a nonsensical manner. Through experimental probing, we consistently observe that such phenomenon is attributed it to the network's local generation bias. Denoising networks tend to produce outputs that rely heavily on highly correlated local regions, particularly when different dimensions of the data distribution are nearly pairwise independent. This behavior leads to a generation process that decomposes the global distribution into separate, independent distributions for each symbol, ultimately failing to capture the global structure, including underlying grammar. Intriguingly, this bias persists across various denoising network architectures including MLP and transformers which have the structure to model global dependency. These findings also provide insights into understanding other types of hallucinations, extending beyond text, as a result of implicit biases in the denoising models. Additionally, we theoretically analyze the training dynamics for a specific case involving a two-layer MLP learning parity points on a hypercube, offering an explanation of its underlying mechanism.
• 摘要：基于得分的扩散模型在生成逼真的图像，音频和视频数据方面取得了令人难以置信的性能。尽管这些模型产生了具有令人印象深刻的细节的高质量样本，但它们经常引入不现实的文物，例如失真的手指或幻觉的文字，没有意义。本文着重于文本幻觉，其中扩散模型正确地生成了单个符号，但以荒谬的方式组装它们。通过实验探测，我们一贯观察到这种现象归因于网络的本地一代偏见。去核网络倾向于产生很大程度上依赖高度相关的本地区域的输出，尤其是当数据分布的不同维度几乎不独立时。这种行为导致一个生成过程，该过程将全局分布分解为每个符号的独立，独立的分布，最终未能捕获全球结构，包括潜在的语法。有趣的是，这种偏见持续存在着各种具有MLP和变形金刚的网络体系结构的偏见，这些网络体系结构具有模拟全局依赖性的结构。这些发现还为理解其他类型的幻觉提供了见解，这是由于剥夺模型中隐性偏见而扩展了文本的。此外，我们从理论上分析了涉及HyperCube上两层MLP学习奇偶校验点的特定情况的训练动力学，并提供了其潜在机制的解释。

Title: An Adaptive Underwater Image Enhancement Framework via Multi-Domain Fusion and Color Compensation

• Authors: Yuezhe Tian, Kangchen Yao, Xiaoyang Yu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03640
• Pdf URL: https://arxiv.org/pdf/2503.03640
• Copy Paste: [[2503.03640]] An Adaptive Underwater Image Enhancement Framework via Multi-Domain Fusion and Color Compensation(https://arxiv.org/abs/2503.03640)
• Keywords: restoration
• Abstract: Underwater optical imaging is severely degraded by light absorption, scattering, and color distortion, hindering visibility and accurate image analysis. This paper presents an adaptive enhancement framework integrating illumination compensation, multi-domain filtering, and dynamic color correction. A hybrid illumination compensation strategy combining CLAHE, Gamma correction, and Retinex enhances visibility. A two-stage filtering process, including spatial-domain (Gaussian, Bilateral, Guided) and frequency-domain (Fourier, Wavelet) methods, effectively reduces noise while preserving details. To correct color distortion, an adaptive color compensation (ACC) model estimates spectral attenuation and water type to combine RCP, DCP, and MUDCP dynamically. Finally, a perceptually guided color balance mechanism ensures natural color restoration. Experimental results on benchmark datasets demonstrate superior performance over state-of-the-art methods in contrast enhancement, color correction, and structural preservation, making the framework robust for underwater imaging applications.
• 摘要：水下光学成像会因光吸收，散射和颜色失真，阻碍可见性和准确的图像分析而严重降低。本文提出了一个自适应增强框架，该框架集成了照明补偿，多域过滤和动态颜色校正。结合Clahe，Gamma校正和Etinex的混合照明补偿策略增强了可见度。一个两阶段的过滤过程，包括空间域（高斯，双侧，指导）和频域（傅立叶，小波）方法，可以有效地减少噪声，同时保留细节。为了纠正颜色失真，自适应颜色补偿（ACC）模型估计光谱衰减和水类型，以动态结合RCP，DCP和MUDCP。最后，感知指导的色彩平衡机制可确保自然的色彩恢复。基准数据集的实验结果表明，在对比度增强，颜色校正和结构保存方面，表现出优于最先进方法的性能，从而使水下成像应用的框架可靠。

Title: DoraCycle: Domain-Oriented Adaptation of Unified Generative Model in Multimodal Cycles

• Authors: Rui Zhao, Weijia Mao, Mike Zheng Shou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03651
• Pdf URL: https://arxiv.org/pdf/2503.03651
• Copy Paste: [[2503.03651]] DoraCycle: Domain-Oriented Adaptation of Unified Generative Model in Multimodal Cycles(https://arxiv.org/abs/2503.03651)
• Keywords: generative
• Abstract: Adapting generative models to specific domains presents an effective solution for satisfying specialized requirements. However, adapting to some complex domains remains challenging, especially when these domains require substantial paired data to capture the targeted distributions. Since unpaired data from a single modality, such as vision or language, is more readily available, we utilize the bidirectional mappings between vision and language learned by the unified generative model to enable training on unpaired data for domain adaptation. Specifically, we propose DoraCycle, which integrates two multimodal cycles: text-to-image-to-text and image-to-text-to-image. The model is optimized through cross-entropy loss computed at the cycle endpoints, where both endpoints share the same modality. This facilitates self-evolution of the model without reliance on annotated text-image pairs. Experimental results demonstrate that for tasks independent of paired knowledge, such as stylization, DoraCycle can effectively adapt the unified model using only unpaired data. For tasks involving new paired knowledge, such as specific identities, a combination of a small set of paired image-text examples and larger-scale unpaired data is sufficient for effective domain-oriented adaptation. The code will be released at this https URL.
• 摘要：将生成模型调整为特定领域为满足专业要求提供了有效的解决方案。但是，适应某些复杂的域仍然具有挑战性，尤其是当这些域需要大量配对数据以捕获目标分布时。由于更容易获得来自视觉或语言等单个模式的不成对数据，因此我们利用统一生成模型学到的视觉和语言之间的双向映射来启用对域适应的未配对数据的培训。具体而言，我们提出了doracycle，它集成了两个多模式循环：文本到图像到文本和图像到文本到图像。该模型是通过在周期终点上计算出的横向渗透损失进行了优化的，其中两个端点共享相同的模态。这有助于模型的自我发展，而不依赖于注释的文本图像对。实验结果表明，对于独立于配对知识的任务（例如风格化），doracycle可以仅使用未配对的数据有效地调整统一模型。对于涉及新配对知识的任务，例如特定的身份，一组配对的图像文本示例和较大规模的不成对数据的组合足以有效地面向域名适应。该代码将在此HTTPS URL上发布。

Title: LION-FS: Fast & Slow Video-Language Thinker as Online Video Assistant

• Authors: Wei Li, Bing Hu, Rui Shao, Leyang Shen, Liqiang Nie
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03663
• Pdf URL: https://arxiv.org/pdf/2503.03663
• Copy Paste: [[2503.03663]] LION-FS: Fast & Slow Video-Language Thinker as Online Video Assistant(https://arxiv.org/abs/2503.03663)
• Keywords: generation
• Abstract: First-person video assistants are highly anticipated to enhance our daily lives through online video dialogue. However, existing online video assistants often sacrifice assistant efficacy for real-time efficiency by processing low-frame-rate videos with coarse-grained visual this http URL overcome the trade-off between efficacy and efficiency, we propose "Fast & Slow Video-Language Thinker" as an onLIne videO assistaNt, LION-FS, achieving real-time, proactive, temporally accurate, and contextually precise responses. LION-FS adopts a two-stage optimization strategy: 1)Fast Path: Routing-Based Response Determination evaluates frame-by-frame whether an immediate response is necessary. To enhance response determination accuracy and handle higher frame-rate inputs efficiently, we employ Token Aggregation Routing to dynamically fuse spatiotemporal features without increasing token numbers, while utilizing Token Dropping Routing to eliminate redundant features. 2)Slow Path: Multi-granularity Keyframe Augmentation optimizes keyframes during response generation. To provide comprehensive and detailed responses beyond atomic actions constrained by training data, fine-grained spatial features and human-environment interaction features are extracted through multi-granular pooling. These features are further integrated into a meticulously designed multimodal Thinking Template to guide more precise response generation. Comprehensive evaluations on online video tasks demonstrate that LION-FS achieves state-of-the-art efficacy and efficiency.
• 摘要：高度期望第一人称视频助理通过在线视频对话来增强我们的日常生活。但是，现有的在线视频助理通常通过用粗粒度视觉处理低框架的视频来牺牲助手效力来提高实时效率。这个HTTP URL克服了功效和效率之间的权衡，我们提出了“快速且慢的视频思想家”作为在线视频助理，lion-fs，Lion-fs，Lion-fs，实现实时，实用，实用，实用的，时间准确的精确响应，并响应于上下文。 Lion-FS采用了两阶段优化策略：1）快速路径：基于路由的响应确定评估是否需要立即响应是否需要立即响应。为了提高响应确定精度并有效地处理更高的帧速率输入，我们将令牌聚合路由采用动态融合时空特征而不增加令牌数字，同时利用令牌删除路由来消除冗余特征。 2）慢速路径：多粒性密钥帧增加在响应生成过程中优化了密钥帧。为了提供受训练数据限制的原子动作以外的全面和详细的响应，通过多粒子池提取了细粒度的空间特征和人类环境相互作用特征。这些功能进一步集成到精心设计的多模式思维模板中，以指导更精确的响应生成。对在线视频任务进行的全面评估表明，Lion-Fs实现了最先进的功效和效率。

Title: A Generative Approach to High Fidelity 3D Reconstruction from Text Data

• Authors: Venkat Kumar R, Deepak Saravanan
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03664
• Pdf URL: https://arxiv.org/pdf/2503.03664
• Copy Paste: [[2503.03664]] A Generative Approach to High Fidelity 3D Reconstruction from Text Data(https://arxiv.org/abs/2503.03664)
• Keywords: generation, generative
• Abstract: The convergence of generative artificial intelligence and advanced computer vision technologies introduces a groundbreaking approach to transforming textual descriptions into three-dimensional representations. This research proposes a fully automated pipeline that seamlessly integrates text-to-image generation, various image processing techniques, and deep learning methods for reflection removal and 3D reconstruction. By leveraging state-of-the-art generative models like Stable Diffusion, the methodology translates natural language inputs into detailed 3D models through a multi-stage workflow. The reconstruction process begins with the generation of high-quality images from textual prompts, followed by enhancement by a reinforcement learning agent and reflection removal using the Stable Delight model. Advanced image upscaling and background removal techniques are then applied to further enhance visual fidelity. These refined two-dimensional representations are subsequently transformed into volumetric 3D models using sophisticated machine learning algorithms, capturing intricate spatial relationships and geometric characteristics. This process achieves a highly structured and detailed output, ensuring that the final 3D models reflect both semantic accuracy and geometric precision. This approach addresses key challenges in generative reconstruction, such as maintaining semantic coherence, managing geometric complexity, and preserving detailed visual information. Comprehensive experimental evaluations will assess reconstruction quality, semantic accuracy, and geometric fidelity across diverse domains and varying levels of complexity. By demonstrating the potential of AI-driven 3D reconstruction techniques, this research offers significant implications for fields such as augmented reality (AR), virtual reality (VR), and digital content creation.
• 摘要：生成人工智能和先进的计算机视觉技术的融合引入了一种开创性的方法，可以将文本描述转化为三维表示。这项研究提出了一条完全自动化的管道，该管道无缝地整合了文本对象的生成，各种图像处理技术以及用于删除反射和3D重建的深度学习方法。通过利用稳定扩散（稳定扩散）的最新生成模型，该方法通过多阶段的工作流将自然语言输入转化为详细的3D模型。重建过程始于从文本提示中产生高质量的图像，然后通过增强型学习代理和使用稳定的Delight Model的反射去除。然后，应用高级图像放大和背景去除技术，以进一步增强视觉保真度。这些精制的二维表示随后使用复杂的机器学习算法转化为体积3D模型，从而捕获复杂的空间关系和几何特征。该过程达到了高度结构化和详细的输出，确保了最终的3D模型既反映了语义精度和几何精度。这种方法解决了生成重建中的关键挑战，例如保持语义连贯性，管理几何复杂性并保留详细的视觉信息。全面的实验评估将评估不同领域的重建质量，语义准确性和几何忠诚度以及各种复杂程度的水平。通过证明AI驱动的3D重建技术的潜力，这项研究对诸如增强现实（AR），虚拟现实（VR）和数字内容创建等领域产生了重要影响。

Title: DualDiff+: Dual-Branch Diffusion for High-Fidelity Video Generation with Reward Guidance

• Authors: Zhao Yang, Zezhong Qian, Xiaofan Li, Weixiang Xu, Gongpeng Zhao, Ruohong Yu, Lingsi Zhu, Longjun Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03689
• Pdf URL: https://arxiv.org/pdf/2503.03689
• Copy Paste: [[2503.03689]] DualDiff+: Dual-Branch Diffusion for High-Fidelity Video Generation with Reward Guidance(https://arxiv.org/abs/2503.03689)
• Keywords: generation
• Abstract: Accurate and high-fidelity driving scene reconstruction demands the effective utilization of comprehensive scene information as conditional inputs. Existing methods predominantly rely on 3D bounding boxes and BEV road maps for foreground and background control, which fail to capture the full complexity of driving scenes and adequately integrate multimodal information. In this work, we present DualDiff, a dual-branch conditional diffusion model designed to enhance driving scene generation across multiple views and video sequences. Specifically, we introduce Occupancy Ray-shape Sampling (ORS) as a conditional input, offering rich foreground and background semantics alongside 3D spatial geometry to precisely control the generation of both elements. To improve the synthesis of fine-grained foreground objects, particularly complex and distant ones, we propose a Foreground-Aware Mask (FGM) denoising loss function. Additionally, we develop the Semantic Fusion Attention (SFA) mechanism to dynamically prioritize relevant information and suppress noise, enabling more effective multimodal fusion. Finally, to ensure high-quality image-to-video generation, we introduce the Reward-Guided Diffusion (RGD) framework, which maintains global consistency and semantic coherence in generated videos. Extensive experiments demonstrate that DualDiff achieves state-of-the-art (SOTA) performance across multiple datasets. On the NuScenes dataset, DualDiff reduces the FID score by 4.09% compared to the best baseline. In downstream tasks, such as BEV segmentation, our method improves vehicle mIoU by 4.50% and road mIoU by 1.70%, while in BEV 3D object detection, the foreground mAP increases by 1.46%. Code will be made available at this https URL.
• 摘要：准确而高保真的驾驶现场重建需要有效利用全面的场景信息作为条件输入。现有方法主要依赖于3D边界框和BEV路线图来进行前景和背景控制，这些框和背景控制无法捕获驾驶场景的全部复杂性并充分整合了多模式信息。在这项工作中，我们提出了Dualdiff，这是一种双分支条件扩散模型，旨在增强跨多个视图和视频序列的驾驶场景的生成。具体而言，我们将占用射线形状采样（OR）作为条件输入引入，提供丰富的前景和背景语义以及3D空间几何形状，以精确控制这两个元素的产生。为了改善精细颗粒前景物体的合成，尤其是复杂和遥远的物体，我们提出了一个前景感知的掩码（FGM）脱氧损耗函数。此外，我们开发了语义融合注意力（SFA）机制，以动态优先考虑相关信息并抑制噪声，从而实现更有效的多模式融合。最后，为了确保高质量的图像到视频生成，我们介绍了奖励引导的扩散（RGD）框架，该框架在生成的视频中保持了全球一致性和语义连贯性。广泛的实验表明，Dualdiff在多个数据集中实现了最新的（SOTA）性能。在Nuscenes数据集上，Dualdiff将FID得分降低了4.09％，而不是最佳基线。在下游任务（例如BEV分割）中，我们的方法将MIOU的车辆MIOU提高了4.50％，MIOU将MIOU提高了1.70％，而在BEV 3D对象检测中，前景图增加了1.46％。代码将在此HTTPS URL上提供。

Title: Rethinking Video Tokenization: A Conditioned Diffusion-based Approach

• Authors: Nianzu Yang, Pandeng Li, Liming Zhao, Yang Li, Chen-Wei Xie, Yehui Tang, Xudong Lu, Zhihang Liu, Yun Zheng, Yu Liu, Junchi Yan
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03708
• Pdf URL: https://arxiv.org/pdf/2503.03708
• Copy Paste: [[2503.03708]] Rethinking Video Tokenization: A Conditioned Diffusion-based Approach(https://arxiv.org/abs/2503.03708)
• Keywords: generation, generative
• Abstract: Video tokenizers, which transform videos into compact latent representations, are key to video generation. Existing video tokenizers are based on the VAE architecture and follow a paradigm where an encoder compresses videos into compact latents, and a deterministic decoder reconstructs the original videos from these latents. In this paper, we propose a novel \underline{\textbf{C}}onditioned \underline{\textbf{D}}iffusion-based video \underline{\textbf{T}}okenizer entitled \textbf{\ourmethod}, which departs from previous methods by replacing the deterministic decoder with a 3D causal diffusion model. The reverse diffusion generative process of the decoder is conditioned on the latent representations derived via the encoder. With a feature caching and sampling acceleration, the framework efficiently reconstructs high-fidelity videos of arbitrary lengths. Results show that {\ourmethod} achieves state-of-the-art performance in video reconstruction tasks using just a single-step sampling. Even a smaller version of {\ourmethod} still achieves reconstruction results on par with the top two baselines. Furthermore, the latent video generation model trained using {\ourmethod} also shows superior performance.
• 摘要：将视频转换为紧凑的潜在表示的视频图形器是视频生成的关键。现有的视频令牌基于VAE架构，并遵循一个范式，编码器将视频压缩到紧凑的潜伏期中，而确定性解码器则重建了这些潜伏的原始视频。在本文中，我们提出了一部小说\下划线{\ textbf {c}} onditioned \下划线{\ textbf {d}} iffusion-iffusion video \下划线\ suespline {\ textbf {t}} okenizer okenizer okenizer okenizer intitled pretitd \ textbf {因果扩散模型。解码器的反向扩散生成过程由通过编码得出的潜在表示。通过功能缓存和采样加速度，该框架有效地重建了任意长度的高保真视频。结果表明，{\ oureMethod}仅使用单步采样即可在视频重建任务中实现最新的性能。即使是较小版本的{\ oureMethod}仍然可以与前两个基线相同地达到重建结果。此外，使用{\ oureMethod}训练的潜在视频生成模型也显示出卓越的性能。

Title: Handling Uncertainty in Health Data using Generative Algorithms

• Authors: Mahdi Arab Loodaricheh, Neh Majmudar, Anita Raja, Ansaf Salleb-Aouissi
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03715
• Pdf URL: https://arxiv.org/pdf/2503.03715
• Copy Paste: [[2503.03715]] Handling Uncertainty in Health Data using Generative Algorithms(https://arxiv.org/abs/2503.03715)
• Keywords: generative
• Abstract: Understanding and managing uncertainty is crucial in machine learning, especially in high-stakes domains like healthcare, where class imbalance can impact predictions. This paper introduces RIGA, a novel pipeline that mitigates class imbalance using generative AI. By converting tabular healthcare data into images, RIGA leverages models like cGAN, VQVAE, and VQGAN to generate balanced samples, improving classification performance. These representations are processed by CNNs and later transformed back into tabular format for seamless integration. This approach enhances traditional classifiers like XGBoost, improves Bayesian structure learning, and strengthens ML model robustness by generating realistic synthetic data for underrepresented classes.
• 摘要：理解和管理不确定性对于机器学习至关重要，尤其是在诸如医疗保健等高风险领域，阶级失衡会影响预测。本文介绍了里加（Riga），这是一种新型的管道，可以使用生成AI来减轻类失衡。通过将表格医疗保健数据转换为图像，Riga利用CGAN，VQVAE和VQGAN等模型生成平衡的样本，从而提高分类性能。这些表示由CNN处理，后来转换为表格格式以进行无缝集成。这种方法增强了传统的分类器，例如XGBoost，改善贝叶斯结构学习，并通过为代表性不足的类别生成现实的合成数据来增强ML模型的鲁棒性。

Title: GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control

• Authors: Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier-David, Thomas Müller, Alexander Keller, Sanja Fidler, Jun Gao
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2503.03751
• Pdf URL: https://arxiv.org/pdf/2503.03751
• Copy Paste: [[2503.03751]] GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control(https://arxiv.org/abs/2503.03751)
• Keywords: generation, generative
• Abstract: We present GEN3C, a generative video model with precise Camera Control and temporal 3D Consistency. Prior video models already generate realistic videos, but they tend to leverage little 3D information, leading to inconsistencies, such as objects popping in and out of existence. Camera control, if implemented at all, is imprecise, because camera parameters are mere inputs to the neural network which must then infer how the video depends on the camera. In contrast, GEN3C is guided by a 3D cache: point clouds obtained by predicting the pixel-wise depth of seed images or previously generated frames. When generating the next frames, GEN3C is conditioned on the 2D renderings of the 3D cache with the new camera trajectory provided by the user. Crucially, this means that GEN3C neither has to remember what it previously generated nor does it have to infer the image structure from the camera pose. The model, instead, can focus all its generative power on previously unobserved regions, as well as advancing the scene state to the next frame. Our results demonstrate more precise camera control than prior work, as well as state-of-the-art results in sparse-view novel view synthesis, even in challenging settings such as driving scenes and monocular dynamic video. Results are best viewed in videos. Check out our webpage! this https URL
• 摘要：我们提出了Gen3C，这是一种具有精确的相机控制和时间3D一致性的生成视频模型。先前的视频模型已经生成了现实的视频，但是它们倾向于利用少量3D信息，导致不一致之处，例如弹出和不存在的对象。相机控制（如果实现）根本不精确，因为相机参数仅是对神经网络的输入，然后必须推断视频如何依赖相机。相比之下，GEN3C由3D缓存：通过预测种子图像或先前生成的帧的像素深度获得的点云。在生成下一个帧时，Gen3C由用户提供的新相机轨迹在3D缓存的2D渲染上进行条件。至关重要的是，这意味着GEN3C既不需要记住它以前产生的内容，也不必须从相机姿势推断出图像结构。相反，该模型可以将其所有生成力重点放在以前未观察到的区域上，并将场景状态推向下一帧。我们的结果证明了比先前的工作更精确的相机控制，并且最新的摄像头会导致稀疏视图综合，即使在诸如驾驶场景和单眼动态视频之类的挑战性设置中也是如此。在视频中最好查看结果。查看我们的网页！此HTTPS URL