2025-05-07

Title: Rewriting Pre-Training Data Boosts LLM Performance in Math and Code

• Authors: Kazuki Fujii, Yukito Tajima, Sakae Mizuki, Hinari Shimada, Taihei Shiotani, Koshiro Saito, Masanari Ohi, Masaki Kawamura, Taishi Nakamura, Takumi Okamoto, Shigeki Ishida, Kakeru Hattori, Youmi Ma, Hiroya Takamura, Rio Yokota, Naoaki Okazaki
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.02881
• Pdf URL: https://arxiv.org/pdf/2505.02881
• Copy Paste: [[2505.02881]] Rewriting Pre-Training Data Boosts LLM Performance in Math and Code(https://arxiv.org/abs/2505.02881)
• Keywords: generation
• Abstract: The performance of large language models (LLMs) in program synthesis and mathematical reasoning is fundamentally limited by the quality of their pre-training corpora. We introduce two openly licensed datasets, released under the Llama 3.3 Community License, that significantly enhance LLM performance by systematically rewriting public data. SwallowCode (approximately 16.1 billion tokens) refines Python snippets from The-Stack-v2 through a novel four-stage pipeline: syntax validation, pylint-based style filtering, and a two-stage LLM rewriting process that enforces style conformity and transforms snippets into self-contained, algorithmically efficient examples. Unlike prior methods that rely on exclusionary filtering or limited transformations, our transform-and-retain approach upgrades low-quality code, maximizing data utility. SwallowMath (approximately 2.3 billion tokens) enhances Finemath-4+ by removing boilerplate, restoring context, and reformatting solutions into concise, step-by-step explanations. Within a fixed 50 billion token training budget, continual pre-training of Llama-3.1-8B with SwallowCode boosts pass@1 by +17.0 on HumanEval and +17.7 on HumanEval+ compared to Stack-Edu, surpassing the baseline model's code generation capabilities. Similarly, substituting SwallowMath yields +12.4 accuracy on GSM8K and +7.6 on MATH. Ablation studies confirm that each pipeline stage contributes incrementally, with rewriting delivering the largest gains. All datasets, prompts, and checkpoints are publicly available, enabling reproducible research and advancing LLM pre-training for specialized domains.
• 摘要：大型语言模型（LLM）在程序综合和数学推理中的性能在根本上受到其前培训语料库质量的限制。我们介绍了两个公开许可的数据集，这些数据集是根据Llama 3.3社区许可发布的，可通过系统地重写公共数据来大大提高LLM性能。通过新颖的四阶段管道，吞咽码（约161亿个令牌）从堆栈-V2中完善了Python摘要：语法验证，基于Pylint的样式过滤，以及两阶段的LLM重写过程，可以强制实现样式的配方，并将snippets转换为具有自动化的，高级，有效地具有高效的典范。与依靠排除过滤或有限转换的先前方法不同，我们的转换和退休临近升级低质量代码，从而最大化数据实用程序。通过删除样板，恢复上下文并将解决方案重新制定为简洁，逐步的解释，Swallowmath（约23亿个令牌）通过删除样板，恢复上下文并将其重新格式化解决方案增强了Finemath-4+。在固定的500亿代价培训预算中，与堆栈-EDU相比，HumaneVal上的Llama-3.1-8B具有燕代码增强的llama-3.1-8b在Humaneval上的1 +17.0通过+17.7，超过了基线模型的代码生成功能。类似地，替换吞咽在GSM8K上产生+12.4的精度，数学上的+7.6。消融研究证实，每个管道阶段都会逐步贡献，重写最大的收益。所有数据集，提示和检查站都可以公开使用，从而为专用域提供了可重现的研究并推进LLM的LLM预培训。

Title: Generating Narrated Lecture Videos from Slides with Synchronized Highlights

• Authors: Alexander Holmberg
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.02966
• Pdf URL: https://arxiv.org/pdf/2505.02966
• Copy Paste: [[2505.02966]] Generating Narrated Lecture Videos from Slides with Synchronized Highlights(https://arxiv.org/abs/2505.02966)
• Keywords: generation
• Abstract: Turning static slides into engaging video lectures takes considerable time and effort, requiring presenters to record explanations and visually guide their audience through the material. We introduce an end-to-end system designed to automate this process entirely. Given a slide deck, this system synthesizes a video lecture featuring AI-generated narration synchronized precisely with dynamic visual highlights. These highlights automatically draw attention to the specific concept being discussed, much like an effective presenter would. The core technical contribution is a novel highlight alignment module. This module accurately maps spoken phrases to locations on a given slide using diverse strategies (e.g., Levenshtein distance, LLM-based semantic analysis) at selectable granularities (line or word level) and utilizes timestamp-providing Text-to-Speech (TTS) for timing synchronization. We demonstrate the system's effectiveness through a technical evaluation using a manually annotated slide dataset with 1000 samples, finding that LLM-based alignment achieves high location accuracy (F1 > 92%), significantly outperforming simpler methods, especially on complex, math-heavy content. Furthermore, the calculated generation cost averages under $1 per hour of video, offering potential savings of two orders of magnitude compared to conservative estimates of manual production costs. This combination of high accuracy and extremely low cost positions this approach as a practical and scalable tool for transforming static slides into effective, visually-guided video lectures.
• 摘要：将静态滑动变成引人入胜的视频讲座需要大量的时间和精力，要求演示者记录解释并视觉指导听众通过材料。我们介绍了一个旨在完全自动化此过程的端到端系统。鉴于幻灯片甲板，该系统合成了一个视频演讲，其中包含AI生成的叙述与动态视觉亮点精确同步。这些亮点自动引起人们对所讨论的特定概念的关注，就像有效的演示者一样。核心技术贡献是一个新颖的重点对齐模块。该模块将使用各种策略（例如Levenshtein距离，基于LLM的语义分析）在特定幻灯片上精确地映射到给定幻灯片上的位置，并在可选粒度（线条或单词级别）上使用时间段提供时间段的文本对语音（TTS）来进行计时同步。我们使用具有1000个样本的手动注释的幻灯数据集通过技术评估来证明系统的有效性，发现基于LLM的对齐能够达到高位置的准确性（F1> 92％），尤其超过了更简单的方法，尤其是在复杂的，较重的数学内容上。此外，与保守的手动生产成本估算相比，计算出的一代平均每小时的视频成本低于每小时1美元的视频，可节省两个数量级。高精度和极低成本位置的这种结合将这种方法作为一种实用且可扩展的工具，可将静态滑梯转换为有效的，视觉引导的视频讲座。

Title: NTIRE 2025 Challenge on UGC Video Enhancement: Methods and Results

• Authors: Nikolay Safonov, Alexey Bryncev, Andrey Moskalenko, Dmitry Kulikov, Dmitry Vatolin, Radu Timofte, Haibo Lei, Qifan Gao, Qing Luo, Yaqing Li, Jie Song, Shaozhe Hao, Meisong Zheng, Jingyi Xu, Chengbin Wu, Jiahui Liu, Ying Chen, Xin Deng, Mai Xu, Peipei Liang, Jie Ma, Junjie Jin, Yingxue Pang, Fangzhou Luo, Kai Chen, Shijie Zhao, Mingyang Wu, Renjie Li, Yushen Zuo, Shengyun Zhong, Zhengzhong Tu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.03007
• Pdf URL: https://arxiv.org/pdf/2505.03007
• Copy Paste: [[2505.03007]] NTIRE 2025 Challenge on UGC Video Enhancement: Methods and Results(https://arxiv.org/abs/2505.03007)
• Keywords: quality assessment
• Abstract: This paper presents an overview of the NTIRE 2025 Challenge on UGC Video Enhancement. The challenge constructed a set of 150 user-generated content videos without reference ground truth, which suffer from real-world degradations such as noise, blur, faded colors, compression artifacts, etc. The goal of the participants was to develop an algorithm capable of improving the visual quality of such videos. Given the widespread use of UGC on short-form video platforms, this task holds substantial practical importance. The evaluation was based on subjective quality assessment in crowdsourcing, obtaining votes from over 8000 assessors. The challenge attracted more than 25 teams submitting solutions, 7 of which passed the final phase with source code verification. The outcomes may provide insights into the state-of-the-art in UGC video enhancement and highlight emerging trends and effective strategies in this evolving research area. All data, including the processed videos and subjective comparison votes and scores, is made publicly available at this https URL.
• 摘要：本文概述了在UGC视频增强上的NTIRE 2025挑战的概述。这项挑战构建了一组150个用户生成的内容视频，而没有参考地面真相，这些视频遭受了现实世界中的退化，例如噪声，模糊，褪色的颜色，压缩工件等。参与者的目的是开发一种能够提高此类视频质量的算法。鉴于UGC在短形式的视频平台上广泛使用，此任务具有重要的实际重要性。评估是基于众包中的主观质量评估，从8000多名评估者那里获得了投票。挑战吸引了25多个提交解决方案的团队，其中7个通过源代码验证通过了最后阶段。结果可能会提供有关UGC视频增强中最新技术的见解，并突出该研究领域的新兴趋势和有效的策略。所有数据，包括处理的视频和主观比较投票和分数，在此HTTPS URL上公开可用。

Title: GIF: Generative Inspiration for Face Recognition at Scale

• Authors: Saeed Ebrahimi, Sahar Rahimi, Ali Dabouei, Srinjoy Das, Jeremy M. Dawson, Nasser M. Nasrabadi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.03012
• Pdf URL: https://arxiv.org/pdf/2505.03012
• Copy Paste: [[2505.03012]] GIF: Generative Inspiration for Face Recognition at Scale(https://arxiv.org/abs/2505.03012)
• Keywords: generative
• Abstract: Aiming to reduce the computational cost of Softmax in massive label space of Face Recognition (FR) benchmarks, recent studies estimate the output using a subset of identities. Although promising, the association between the computation cost and the number of identities in the dataset remains linear only with a reduced ratio. A shared characteristic among available FR methods is the employment of atomic scalar labels during training. Consequently, the input to label matching is through a dot product between the feature vector of the input and the Softmax centroids. Inspired by generative modeling, we present a simple yet effective method that substitutes scalar labels with structured identity code, i.e., a sequence of integers. Specifically, we propose a tokenization scheme that transforms atomic scalar labels into structured identity codes. Then, we train an FR backbone to predict the code for each input instead of its scalar label. As a result, the associated computational cost becomes logarithmic w.r.t. number of identities. We demonstrate the benefits of the proposed method by conducting experiments. In particular, our method outperforms its competitors by 1.52%, and 0.6% at TAR@FAR$=1e-4$ on IJB-B and IJB-C, respectively, while transforming the association between computational cost and the number of identities from linear to logarithmic. See code at this https URL
• 摘要：旨在降低面部识别（FR）基准标签空间（FR）基准标签空间中SoftMax的计算成本，最近的研究使用一部分身份估算了输出。尽管很有希望，但计算成本与数据集中的身份数量之间的关联仅在降低比率的情况下保持线性。可用的FR方法中的共同特征是在训练过程中使用原子标量标签。因此，标签匹配的输入是通过输入的特征向量和软马克斯质心之间的点产物。受生成建模的启发，我们提出了一种简单而有效的方法，该方法用结构化身份代码（即一系列整数序列）代替标量标签。具体而言，我们提出了一个将原子标量标签转换为结构化身份代码的令牌化方案。然后，我们训练一个FR主链，以预测每个输入的代码，而不是其标量标签。结果，相关的计算成本成为对数W.R.T.身份数量。我们通过进行实验来证明该方法的好处。特别是，我们的方法在IJB-B和IJB-C上分别优于竞争对手1.52％，在Tar@far $ = 1e-4 $上的0.6％，同时改变了计算成本与从线性到和解的身份之间的关联。在此HTTPS URL上查看代码

Title: Lesion-Aware Generative Artificial Intelligence for Virtual Contrast-Enhanced Mammography in Breast Cancer

• Authors: Aurora Rofena, Arianna Manchia, Claudia Lucia Piccolo, Bruno Beomonte Zobel, Paolo Soda, Valerio Guarrasi
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.03018
• Pdf URL: https://arxiv.org/pdf/2505.03018
• Copy Paste: [[2505.03018]] Lesion-Aware Generative Artificial Intelligence for Virtual Contrast-Enhanced Mammography in Breast Cancer(https://arxiv.org/abs/2505.03018)
• Keywords: generative
• Abstract: Contrast-Enhanced Spectral Mammography (CESM) is a dual-energy mammographic technique that improves lesion visibility through the administration of an iodinated contrast agent. It acquires both a low-energy image, comparable to standard mammography, and a high-energy image, which are then combined to produce a dual-energy subtracted image highlighting lesion contrast enhancement. While CESM offers superior diagnostic accuracy compared to standard mammography, its use entails higher radiation exposure and potential side effects associated with the contrast medium. To address these limitations, we propose Seg-CycleGAN, a generative deep learning framework for Virtual Contrast Enhancement in CESM. The model synthesizes high-fidelity dual-energy subtracted images from low-energy images, leveraging lesion segmentation maps to guide the generative process and improve lesion reconstruction. Building upon the standard CycleGAN architecture, Seg-CycleGAN introduces localized loss terms focused on lesion areas, enhancing the synthesis of diagnostically relevant regions. Experiments on the CESM@UCBM dataset demonstrate that Seg-CycleGAN outperforms the baseline in terms of PSNR and SSIM, while maintaining competitive MSE and VIF. Qualitative evaluations further confirm improved lesion fidelity in the generated images. These results suggest that segmentation-aware generative models offer a viable pathway toward contrast-free CESM alternatives.
• 摘要：对比增强光谱乳房X线摄影（CESM）是一种双能乳腺X线摄影技术，可通过施用碘化对比剂来提高病变的可见性。它既获得了低能图像，与标准乳房摄影相当，又获得了高能量图像，然后将其组合起来，以产生双能量减去的图像突出显示病变对比增强。与标准乳房X线摄影相比，CESM具有优异的诊断精度，但其使用需要更高的辐射暴露和与对比介质相关的潜在副作用。为了解决这些局限性，我们提出了Seg-Cyclegan，这是CESM中虚拟对比度增强的生成深度学习框架。该模型合成了从低能量图像中减去高保真双能量的图像，利用病变分割图来指导生成过程并改善病变重建。 SEG-Cyclegan在标准Cyclegan架构的基础上引入了局部损失条款，该损失条款集中在病变区域，增强了诊断相关区域的综合。 CESM@UCBM数据集上的实验表明，SEG-Cyclegan在PSNR和SSIM方面优于基线，同时保持竞争性MSE和VIF。定性评估进一步证实了生成图像中的病变保真度。这些结果表明，分割感知的生成模型为无对比的CESM替代方案提供了可行的途径。

Title: 34 Examples of LLM Applications in Materials Science and Chemistry: Towards Automation, Assistants, Agents, and Accelerated Scientific Discovery

• Authors: Yoel Zimmermann, Adib Bazgir, Alexander Al-Feghali, Mehrad Ansari, L. Catherine Brinson, Yuan Chiang, Defne Circi, Min-Hsueh Chiu, Nathan Daelman, Matthew L. Evans, Abhijeet S. Gangan, Janine George, Hassan Harb, Ghazal Khalighinejad, Sartaaj Takrim Khan, Sascha Klawohn, Magdalena Lederbauer, Soroush Mahjoubi, Bernadette Mohr, Seyed Mohamad Moosavi, Aakash Naik, Aleyna Beste Ozhan, Dieter Plessers, Aritra Roy, Fabian Schöppach, Philippe Schwaller, Carla Terboven, Katharina Ueltzen, Shang Zhu, Jan Janssen, Calvin Li, Ian Foster, Ben Blaiszik
• Subjects: cs.LG, cond-mat.mtrl-sci
• Abstract URL: https://arxiv.org/abs/2505.03049
• Pdf URL: https://arxiv.org/pdf/2505.03049
• Copy Paste: [[2505.03049]] 34 Examples of LLM Applications in Materials Science and Chemistry: Towards Automation, Assistants, Agents, and Accelerated Scientific Discovery(https://arxiv.org/abs/2505.03049)
• Keywords: generation
• Abstract: Large Language Models (LLMs) are reshaping many aspects of materials science and chemistry research, enabling advances in molecular property prediction, materials design, scientific automation, knowledge extraction, and more. Recent developments demonstrate that the latest class of models are able to integrate structured and unstructured data, assist in hypothesis generation, and streamline research workflows. To explore the frontier of LLM capabilities across the research lifecycle, we review applications of LLMs through 34 total projects developed during the second annual Large Language Model Hackathon for Applications in Materials Science and Chemistry, a global hybrid event. These projects spanned seven key research areas: (1) molecular and material property prediction, (2) molecular and material design, (3) automation and novel interfaces, (4) scientific communication and education, (5) research data management and automation, (6) hypothesis generation and evaluation, and (7) knowledge extraction and reasoning from the scientific literature. Collectively, these applications illustrate how LLMs serve as versatile predictive models, platforms for rapid prototyping of domain-specific tools, and much more. In particular, improvements in both open source and proprietary LLM performance through the addition of reasoning, additional training data, and new techniques have expanded effectiveness, particularly in low-data environments and interdisciplinary research. As LLMs continue to improve, their integration into scientific workflows presents both new opportunities and new challenges, requiring ongoing exploration, continued refinement, and further research to address reliability, interpretability, and reproducibility.
• 摘要：大型语言模型（LLM）正在重塑材料科学和化学研究的许多方面，从而在分子财产预测，材料设计，科学自动化，知识提取等方面取得了进步。最近的发展表明，最新类模型能够整合结构化和非结构化数据，有助于假设产生以及简化研究工作流程。为了探索整个研究生命周期中LLM功能的前沿，我们回顾了LLM的应用到在第二届年度大型语言模型Hackathon中开发的34个项目，用于全球混合活动材料科学与化学中的应用。这些项目跨越了七个关键的研究领域：（1）分子和材料性质预测，（2）分子和材料设计，（3）自动化和新型界面，（4）科学交流和教育，（5）研究数据管理和自动化，（6）假设产生和评估，以及（7）从科学文献中提取和推理的知识。总的来说，这些应用程序说明了LLM如何用作通用的预测模型，用于域特异性工具的快速原型制作的平台等等。特别是，通过添加推理，其他培训数据和新技术的开源和专有LLM绩效的改进都扩大了有效性，尤其是在低数据表环境和跨学科研究中。随着LLM的继续改善，它们融入科学工作流程既提出了新的机遇和新挑战，需要进行持续的探索，继续进行，并进一步研究以解决可靠性，可解释性和可重复性。

Title: Not All Parameters Matter: Masking Diffusion Models for Enhancing Generation Ability

• Authors: Lei Wang, Senmao Li, Fei Yang, Jianye Wang, Ziheng Zhang, Yuhan Liu, Yaxing Wang, Jian Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.03097
• Pdf URL: https://arxiv.org/pdf/2505.03097
• Copy Paste: [[2505.03097]] Not All Parameters Matter: Masking Diffusion Models for Enhancing Generation Ability(https://arxiv.org/abs/2505.03097)
• Keywords: generation
• Abstract: The diffusion models, in early stages focus on constructing basic image structures, while the refined details, including local features and textures, are generated in later stages. Thus the same network layers are forced to learn both structural and textural information simultaneously, significantly differing from the traditional deep learning architectures (e.g., ResNet or GANs) which captures or generates the image semantic information at different layers. This difference inspires us to explore the time-wise diffusion models. We initially investigate the key contributions of the U-Net parameters to the denoising process and identify that properly zeroing out certain parameters (including large parameters) contributes to denoising, substantially improving the generation quality on the fly. Capitalizing on this discovery, we propose a simple yet effective method-termed ``MaskUNet''- that enhances generation quality with negligible parameter numbers. Our method fully leverages timestep- and sample-dependent effective U-Net parameters. To optimize MaskUNet, we offer two fine-tuning strategies: a training-based approach and a training-free approach, including tailored networks and optimization functions. In zero-shot inference on the COCO dataset, MaskUNet achieves the best FID score and further demonstrates its effectiveness in downstream task evaluations. Project page: this https URL
• 摘要：在早期阶段，扩散模型集中在构建基本图像结构上，而精致的细节（包括本地特征和纹理）是在较晚阶段生成的。因此，相同的网络层被迫同时学习结构和纹理信息，与传统的深度学习体系结构（例如Resnet或gans）有显着差异，这些架构（例如Resnet或gans）在不同层捕获或生成图像语义信息。这种差异激发了我们探索时间的扩散模型。我们最初研究了U-NET参数对降解过程的关键贡献，并确定正确归零某些参数（包括大参数）有助于降解，从而实质上提高了即时的生成质量。利用这一发现，我们提出了一个简单而有效的方法，``Maskunet'' - 通过可忽略不计的参数数字增强了发电质量。我们的方法完全利用时间段和样本依赖性有效的U-NET参数。为了优化Maskunet，我们提供了两种微调策略：一种基于培训的方法和一种无培训的方法，包括量身定制的网络和优化功能。在对可可数据集的零射推理中，Maskunet获得了最佳的FID分数，并进一步证明了其在下游任务评估中的有效性。项目页面：此HTTPS URL

Title: Deep Learning in Renewable Energy Forecasting: A Cross-Dataset Evaluation of Temporal and Spatial Models

• Authors: Lutfu Sua, Haibo Wang, Jun Huang
• Subjects: cs.LG, econ.GN
• Abstract URL: https://arxiv.org/abs/2505.03109
• Pdf URL: https://arxiv.org/pdf/2505.03109
• Copy Paste: [[2505.03109]] Deep Learning in Renewable Energy Forecasting: A Cross-Dataset Evaluation of Temporal and Spatial Models(https://arxiv.org/abs/2505.03109)
• Keywords: generation
• Abstract: Unpredictability of renewable energy sources coupled with the complexity of those methods used for various purposes in this area calls for the development of robust methods such as DL models within the renewable energy domain. Given the nonlinear relationships among variables in renewable energy datasets, DL models are preferred over traditional machine learning (ML) models because they can effectively capture and model complex interactions between variables. This research aims to identify the factors responsible for the accuracy of DL techniques, such as sampling, stationarity, linearity, and hyperparameter optimization for different algorithms. The proposed DL framework compares various methods and alternative training/test ratios. Seven ML methods, such as Long-Short Term Memory (LSTM), Stacked LSTM, Convolutional Neural Network (CNN), CNN-LSTM, Deep Neural Network (DNN), Multilayer Perceptron (MLP), and Encoder-Decoder (ED), were evaluated on two different datasets. The first dataset contains the weather and power generation data. It encompasses two distinct datasets, hourly energy demand data and hourly weather data in Spain, while the second dataset includes power output generated by the photovoltaic panels at 12 locations. This study deploys regularization approaches, including early stopping, neuron dropping, and L2 regularization, to reduce the overfitting problem associated with DL models. The LSTM and MLP models show superior performance. Their validation data exhibit exceptionally low root mean square error values.
• 摘要：可再生能源的不可预测性，加上用于此领域各种目的的方法的复杂性，要​​求开发可再生能源域内的诸如DL模型之类的强大方法。鉴于可再生能源数据集中变量之间的非线性关系，DL模型比传统机器学习（ML）模型优选，因为它们可以有效地捕获和建模变量之间的复杂相互作用。这项研究旨在确定负责DL技术准确性的因素，例如针对不同算法的采样，平稳性，线性和超参数优化。提出的DL框架比较了各种方法和替代训练/测试比率。在两个不同的数据集上评估了七种ML方法，例如长期任期内存（LSTM），堆叠的LSTM，堆积的LSTM，卷积神经网络（CNN），CNN-LSTM，深神经网络（DNN），多层perceptron（MLP）和Encoder-Decoder（ED）。第一个数据集包含天气和发电数据。它包括两个不同的数据集，每小时的能源需求数据和西班牙的每小时天气数据，而第二个数据集包括由光伏面板在12个位置生成的电源输出。这项研究部署了正则化方法，包括早期停止，神经元下降和L2正则化，以减少与DL模型相关的过度拟合问题。 LSTM和MLP模型显示出卓越的性能。他们的验证数据表现出异常低的均方根误差值。

Title: Plug-and-Play AMC: Context Is King in Training-Free, Open-Set Modulation with LLMs

• Authors: Mohammad Rostami, Atik Faysal, Reihaneh Gh. Roshan, Huaxia Wang, Nikhil Muralidhar, Yu-Dong Yao
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.03112
• Pdf URL: https://arxiv.org/pdf/2505.03112
• Copy Paste: [[2505.03112]] Plug-and-Play AMC: Context Is King in Training-Free, Open-Set Modulation with LLMs(https://arxiv.org/abs/2505.03112)
• Keywords: generation
• Abstract: Automatic Modulation Classification (AMC) is critical for efficient spectrum management and robust wireless communications. However, AMC remains challenging due to the complex interplay of signal interference and noise. In this work, we propose an innovative framework that integrates traditional signal processing techniques with Large-Language Models (LLMs) to address AMC. Our approach leverages higher-order statistics and cumulant estimation to convert quantitative signal features into structured natural language prompts. By incorporating exemplar contexts into these prompts, our method exploits the LLM's inherent familiarity with classical signal processing, enabling effective one-shot classification without additional training or preprocessing (e.g., denoising). Experimental evaluations on synthetically generated datasets, spanning both noiseless and noisy conditions, demonstrate that our framework achieves competitive performance across diverse modulation schemes and Signal-to-Noise Ratios (SNRs). Moreover, our approach paves the way for robust foundation models in wireless communications across varying channel conditions, significantly reducing the expense associated with developing channel-specific models. This work lays the foundation for scalable, interpretable, and versatile signal classification systems in next-generation wireless networks. The source code is available at this https URL
• 摘要：自动调制分类（AMC）对于有效的频谱管理和强大的无线通信至关重要。但是，由于信号干扰和噪声的复杂相互作用，AMC仍然具有挑战性。在这项工作中，我们提出了一个创新的框架，该框架将传统的信号处理技术与大型语言模型（LLMS）集成在一起以解决AMC。我们的方法利用高阶统计数据和累积估计将定量信号特征转换为结构化的自然语言提示。通过将示例性上下文纳入这些提示，我们的方法利用了LLM对经典信号处理的固有熟悉程度，从而实现了有效的一声分类，而无需额外的培训或预处理（例如，Denoising）。跨越无噪声和嘈杂条件的合成生成数据集的实验评估表明，我们的框架在不同的调制方案和信噪比（SNRS）之间实现了竞争性能。此外，我们的方法为在不同的通道条件下无线通信中的强大基础模型铺平了道路，从而大大降低了与开发特定渠道模型相关的费用。这项工作为下一代无线网络中的可扩展，可解释和多功能信号分类系统奠定了基础。源代码可在此HTTPS URL上找到

Title: StableMotion: Training Motion Cleanup Models with Unpaired Corrupted Data

• Authors: Yuxuan Mu, Hung Yu Ling, Yi Shi, Ismael Baira Ojeda, Pengcheng Xi, Chang Shu, Fabio Zinno, Xue Bin Peng
• Subjects: cs.CV, cs.AI, cs.GR
• Abstract URL: https://arxiv.org/abs/2505.03154
• Pdf URL: https://arxiv.org/pdf/2505.03154
• Copy Paste: [[2505.03154]] StableMotion: Training Motion Cleanup Models with Unpaired Corrupted Data(https://arxiv.org/abs/2505.03154)
• Keywords: generation
• Abstract: Motion capture (mocap) data often exhibits visually jarring artifacts due to inaccurate sensors and post-processing. Cleaning this corrupted data can require substantial manual effort from human experts, which can be a costly and time-consuming process. Previous data-driven motion cleanup methods offer the promise of automating this cleanup process, but often require in-domain paired corrupted-to-clean training data. Constructing such paired datasets requires access to high-quality, relatively artifact-free motion clips, which often necessitates laborious manual cleanup. In this work, we present StableMotion, a simple yet effective method for training motion cleanup models directly from unpaired corrupted datasets that need cleanup. The core component of our method is the introduction of motion quality indicators, which can be easily annotated through manual labeling or heuristic algorithms and enable training of quality-aware motion generation models on raw motion data with mixed quality. At test time, the model can be prompted to generate high-quality motions using the quality indicators. Our method can be implemented through a simple diffusion-based framework, leading to a unified motion generate-discriminate model, which can be used to both identify and fix corrupted frames. We demonstrate that our proposed method is effective for training motion cleanup models on raw mocap data in production scenarios by applying StableMotion to SoccerMocap, a 245-hour soccer mocap dataset containing real-world motion artifacts. The trained model effectively corrects a wide range of motion artifacts, reducing motion pops and frozen frames by 68% and 81%, respectively. See this https URL for more results.
• 摘要：运动捕获（MOCAP）数据通常由于传感器不准确和后处理而表现出视觉震撼的伪影。清理这些损坏的数据可能需要人类专家的大量手动努力，这可能是一个昂贵且耗时的过程。以前的数据驱动运动清理方法提供了自动化此清理过程的承诺，但通常需要内域配对损坏到清洁的培训数据。构建此类配对数据集需要访问高质量的，相对无伪影的运动夹，这通常需要手动清理。在这项工作中，我们提出了StableMotion，这是一种直接来自需要清理的未配对损坏数据集的简单而有效的方法清理模型。我们方法的核心组成部分是引入运动质量指标，可以通过手动标记或启发式算法轻松注释该指标，并能够在具有混合质量的原始运动数据上培训质量感知的运动生成模型。在测试时，可以提示该模型使用质量指标生成高质量的动作。我们的方法可以通过简单的基于扩散的框架来实现，从而导致统一动作生成歧视模型，该模型可用于识别和修复损坏的框架。我们证明，我们提出的方法可通过将稳定脱脂剂应用于Soccermocap（245小时的足球MOCAP数据集，其中包含现实世界动作文物的245小时，在生产方案中的原始MOCAP数据中有效清理模型。受过训练的模型有效地纠正了广泛的运动伪像，将运动弹出和冷冻帧分别减少了68％和81％。有关更多结果，请参见此HTTPS URL。

Title: Convergence Of Consistency Model With Multistep Sampling Under General Data Assumptions

• Authors: Yiding Chen, Yiyi Zhang, Owen Oertell, Wen Sun
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.03194
• Pdf URL: https://arxiv.org/pdf/2505.03194
• Copy Paste: [[2505.03194]] Convergence Of Consistency Model With Multistep Sampling Under General Data Assumptions(https://arxiv.org/abs/2505.03194)
• Keywords: generation
• Abstract: Diffusion models accomplish remarkable success in data generation tasks across various domains. However, the iterative sampling process is computationally expensive. Consistency models are proposed to learn consistency functions to map from noise to data directly, which allows one-step fast data generation and multistep sampling to improve sample quality. In this paper, we study the convergence of consistency models when the self-consistency property holds approximately under the training distribution. Our analysis requires only mild data assumption and applies to a family of forward processes. When the target data distribution has bounded support or has tails that decay sufficiently fast, we show that the samples generated by the consistency model are close to the target distribution in Wasserstein distance; when the target distribution satisfies some smoothness assumption, we show that with an additional perturbation step for smoothing, the generated samples are close to the target distribution in total variation distance. We provide two case studies with commonly chosen forward processes to demonstrate the benefit of multistep sampling.
• 摘要：扩散模型在各个领域的数据生成任务上取得了显着成功。但是，迭代抽样过程在计算上是昂贵的。提出了一致性模型来学习一致性功能，以直接从噪声映射到数据，这允许一步快速数据生成和多步骤采样以提高样本质量。在本文中，我们研究了一致性属性大约在训练分布下的一致性模型的融合。我们的分析仅需要温和的数据假设，并适用于远期过程的家族。当目标数据分布有界限或尾巴足够快地衰减时，我们表明，一致性模型产生的样品接近Wasserstein距离的目标分布。当目标分布满足某些平滑度的假设时，我们表明，通过额外的扰动步骤进行平滑，生成的样品接近总变化距离的目标分布。我们提供了两个案例研究，并具有通常选择的远期过程，以证明多步抽样的好处。

Title: PiCo: Enhancing Text-Image Alignment with Improved Noise Selection and Precise Mask Control in Diffusion Models

• Authors: Chang Xie, Chenyi Zhuang, Pan Gao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.03203
• Pdf URL: https://arxiv.org/pdf/2505.03203
• Copy Paste: [[2505.03203]] PiCo: Enhancing Text-Image Alignment with Improved Noise Selection and Precise Mask Control in Diffusion Models(https://arxiv.org/abs/2505.03203)
• Keywords: generation
• Abstract: Advanced diffusion models have made notable progress in text-to-image compositional generation. However, it is still a challenge for existing models to achieve text-image alignment when confronted with complex text prompts. In this work, we highlight two factors that affect this alignment: the quality of the randomly initialized noise and the reliability of the generated controlling mask. We then propose PiCo (Pick-and-Control), a novel training-free approach with two key components to tackle these two factors. First, we develop a noise selection module to assess the quality of the random noise and determine whether the noise is suitable for the target text. A fast sampling strategy is utilized to ensure efficiency in the noise selection stage. Second, we introduce a referring mask module to generate pixel-level masks and to precisely modulate the cross-attention maps. The referring mask is applied to the standard diffusion process to guide the reasonable interaction between text and image features. Extensive experiments have been conducted to verify the effectiveness of PiCo in liberating users from the tedious process of random generation and in enhancing the text-image alignment for diverse text descriptions.
• 摘要：先进的扩散模型在文本到图像构图生成方面取得了显着进展。但是，在面对复杂的文本提示时，现有模型要实现文本图像对齐仍然是一个挑战。在这项工作中，我们重点介绍了影响这一路线的两个因素：随机初始化噪声的质量以及生成的控制面膜的可靠性。然后，我们提出了Pico（Pick-control），这是一种新型的无训练方法，采用两个关键组件来解决这两个因素。首先，我们开发一个噪声选择模块来评估随机噪声的质量，并确定噪声是否适合目标文本。使用快速采样策略来确保噪声选择阶段的效率。其次，我们引入了一个引用掩码模块，以生成像素级掩码并精确调制跨注意图。引用掩码应用于标准扩散过程，以指导文本和图像特征之间的合理相互作用。已经进行了广泛的实验，以验证PICO从随机生成的繁琐过程中解放出用户的有效性，并增强文本图像的文本描述的文本图像对齐方式。

Title: Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insights

• Authors: Zhaiming Shen, Alex Havrilla, Rongjie Lai, Alexander Cloninger, Wenjing Liao
• Subjects: cs.LG, math.NA, math.ST
• Abstract URL: https://arxiv.org/abs/2505.03205
• Pdf URL: https://arxiv.org/pdf/2505.03205
• Copy Paste: [[2505.03205]] Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insights(https://arxiv.org/abs/2505.03205)
• Keywords: generation
• Abstract: Transformers serve as the foundational architecture for large language and video generation models, such as GPT, BERT, SORA and their successors. Empirical studies have demonstrated that real-world data and learning tasks exhibit low-dimensional structures, along with some noise or measurement error. The performance of transformers tends to depend on the intrinsic dimension of the data/tasks, though theoretical understandings remain largely unexplored for transformers. This work establishes a theoretical foundation by analyzing the performance of transformers for regression tasks involving noisy input data on a manifold. Specifically, the input data are in a tubular neighborhood of a manifold, while the ground truth function depends on the projection of the noisy data onto the manifold. We prove approximation and generalization errors which crucially depend on the intrinsic dimension of the manifold. Our results demonstrate that transformers can leverage low-complexity structures in learning task even when the input data are perturbed by high-dimensional noise. Our novel proof technique constructs representations of basic arithmetic operations by transformers, which may hold independent interest.
• 摘要：变形金刚是大型语言和视频生成模型的基础架构，例如GPT，BERT，SORA及其继任者。经验研究表明，现实世界中的数据和学习任务表现出低维结构，以及一些噪声或测量误差。变形金刚的性能倾向于取决于数据/任务的固有维度，尽管理论上的理解在很大程度上尚未探索变压器。这项工作通过分析变压器的性能来建立理论基础，以进行涉及多种噪声输入数据的回归任务。具体而言，输入数据位于歧管的管状邻域中，而地面真实函数取决于将噪声数据投射到歧管上。我们证明了近似和概括误差至关重要的取决于歧管的内在维度。我们的结果表明，即使输入数据受到高维噪声的干扰，变压器也可以利用学习任务中的低复杂性结构。我们的新颖证明技术构建了变形金刚基本算术操作的表示，这可能具有独立的兴趣。

Title: DiffVQA: Video Quality Assessment Using Diffusion Feature Extractor

• Authors: Wei-Ting Chen, Yu-Jiet Vong, Yi-Tsung Lee, Sy-Yen Kuo, Qiang Gao, Sizhuo Ma, Jian Wang
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.03261
• Pdf URL: https://arxiv.org/pdf/2505.03261
• Copy Paste: [[2505.03261]] DiffVQA: Video Quality Assessment Using Diffusion Feature Extractor(https://arxiv.org/abs/2505.03261)
• Keywords: quality assessment
• Abstract: Video Quality Assessment (VQA) aims to evaluate video quality based on perceptual distortions and human preferences. Despite the promising performance of existing methods using Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), they often struggle to align closely with human perceptions, particularly in diverse real-world scenarios. This challenge is exacerbated by the limited scale and diversity of available datasets. To address this limitation, we introduce a novel VQA framework, DiffVQA, which harnesses the robust generalization capabilities of diffusion models pre-trained on extensive datasets. Our framework adapts these models to reconstruct identical input frames through a control module. The adapted diffusion model is then used to extract semantic and distortion features from a resizing branch and a cropping branch, respectively. To enhance the model's ability to handle long-term temporal dynamics, a parallel Mamba module is introduced, which extracts temporal coherence augmented features that are merged with the diffusion features to predict the final score. Experiments across multiple datasets demonstrate DiffVQA's superior performance on intra-dataset evaluations and its exceptional generalization across datasets. These results confirm that leveraging a diffusion model as a feature extractor can offer enhanced VQA performance compared to CNN and ViT backbones.
• 摘要：视频质量评估（VQA）旨在根据知觉扭曲和人类偏好评估视频质量。尽管使用卷积神经网络（CNN）和视觉变形金刚（VIT）的现有方法表现有希望，但它们通常很难与人类的看法紧密保持一致，尤其是在各种现实世界中。可用数据集的规模有限和多样性加剧了这一挑战。为了解决这一限制，我们引入了一个新颖的VQA框架DIFFVQA，该框架可以利用在广泛的数据集中预先训练的扩散模型的强大概括能力。我们的框架使这些模型适应通过控制模块重建相同的输入帧。然后，使用调整的扩散模型分别从调整分支和裁剪分支中提取语义和失真特征。为了增强模型处理长期时间动力学的能力，引入了平行的MAMBA模块，该模块提取了时间连贯性增强功能，这些功能与扩散特征合并以预测最终分数。跨多个数据集的实验证明了DIFFVQA在数据集中评估及其在数据集跨数据集中的卓越概括方面的出色性能。这些结果证实，与CNN和VIT骨架相比，利用扩散模型作为特征提取器可以提供增强的VQA性能。

Title: Base-Detail Feature Learning Framework for Visible-Infrared Person Re-Identification

• Authors: Zhihao Gong, Lian Wu, Yong Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.03286
• Pdf URL: https://arxiv.org/pdf/2505.03286
• Copy Paste: [[2505.03286]] Base-Detail Feature Learning Framework for Visible-Infrared Person Re-Identification(https://arxiv.org/abs/2505.03286)
• Keywords: generation
• Abstract: Visible-infrared person re-identification (VIReID) provides a solution for ReID tasks in 24-hour scenarios; however, significant challenges persist in achieving satisfactory performance due to the substantial discrepancies between visible (VIS) and infrared (IR) modalities. Existing methods inadequately leverage information from different modalities, primarily focusing on digging distinguishing features from modality-shared information while neglecting modality-specific details. To fully utilize differentiated minutiae, we propose a Base-Detail Feature Learning Framework (BDLF) that enhances the learning of both base and detail knowledge, thereby capitalizing on both modality-shared and modality-specific information. Specifically, the proposed BDLF mines detail and base features through a lossless detail feature extraction module and a complementary base embedding generation mechanism, respectively, supported by a novel correlation restriction method that ensures the features gained by BDLF enrich both detail and base knowledge across VIS and IR features. Comprehensive experiments conducted on the SYSU-MM01, RegDB, and LLCM datasets validate the effectiveness of BDLF.
• 摘要：可见的红外人员重新识别（VIREID）为在24小时场景中提供了REID任务的解决方案；但是，由于可见（VIS）和红外（IR）方式之间存在很大的差异，因此在实现令人满意的表现方面存在重大挑战。现有方法不足以利用不同方式的信息，主要集中于挖掘与模态共享信息的区分，同时忽略了特定于模态的细节。为了充分利用差异化的细节，我们提出了一个基本范围的特征学习框架（BDLF），以增强基础和细节知识的学习，从而利用模式共享和特定于模态的信息。具体而言，提出的BDLF地雷细节和基本特征通过无损的细节特征提取模块和互补的基础嵌入生成机制，并由新型的相关限制方法支持，该方法确保了BDLF丰富的详细信息和基础知识所获得的特征，并且跨访问和IR特征。在SYSU-MM01，REGDB和LLCM数据集上进行的全面实验验证了BDLF的有效性。

Title: 3D Can Be Explored In 2D: Pseudo-Label Generation for LiDAR Point Clouds Using Sensor-Intensity-Based 2D Semantic Segmentation

• Authors: Andrew Caunes, Thierry Chateau, Vincent Frémont
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.03300
• Pdf URL: https://arxiv.org/pdf/2505.03300
• Copy Paste: [[2505.03300]] 3D Can Be Explored In 2D: Pseudo-Label Generation for LiDAR Point Clouds Using Sensor-Intensity-Based 2D Semantic Segmentation(https://arxiv.org/abs/2505.03300)
• Keywords: generation
• Abstract: Semantic segmentation of 3D LiDAR point clouds, essential for autonomous driving and infrastructure management, is best achieved by supervised learning, which demands extensive annotated datasets and faces the problem of domain shifts. We introduce a new 3D semantic segmentation pipeline that leverages aligned scenes and state-of-the-art 2D segmentation methods, avoiding the need for direct 3D annotation or reliance on additional modalities such as camera images at inference time. Our approach generates 2D views from LiDAR scans colored by sensor intensity and applies 2D semantic segmentation to these views using a camera-domain pretrained model. The segmented 2D outputs are then back-projected onto the 3D points, with a simple voting-based estimator that merges the labels associated to each 3D point. Our main contribution is a global pipeline for 3D semantic segmentation requiring no prior 3D annotation and not other modality for inference, which can be used for pseudo-label generation. We conduct a thorough ablation study and demonstrate the potential of the generated pseudo-labels for the Unsupervised Domain Adaptation task.
• 摘要：3D激光雷德点云的语义细分是自动驾驶和基础架构管理所必需的，可以通过监督学习来实现，这需要广泛的注释数据集并面临域转移的问题。我们介绍了一种新的3D语义分割管道，该管道利用了对齐场景和最新的2D分割方法，避免了需要直接3D注释或依赖其他方式（例如，在推理时摄像机图像）。我们的方法从传感器强度颜色的LIDAR扫描中产生2D视图，并使用摄像头预审预告片模型将2D语义分割应用于这些视图。然后将分段的2D输出回到3D点上，并具有简单的基于投票的估计器，该估计器合并了与每个3D点关联的标签。我们的主要贡献是用于3D语义分割的全球管道，不需要先前的3D注释，而不是推理的其他模式，可用于伪标签生成。我们进行了一项彻底的消融研究，并证明了生成的伪标签对无监督的域适应任务的潜力。

Title: Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuning

• Authors: Yibin Wang, Zhimin Li, Yuhang Zang, Chunyu Wang, Qinglin Lu, Cheng Jin, Jiaqi Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.03318
• Pdf URL: https://arxiv.org/pdf/2505.03318
• Copy Paste: [[2505.03318]] Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuning(https://arxiv.org/abs/2505.03318)
• Keywords: generation
• Abstract: Recent advances in multimodal Reward Models (RMs) have shown significant promise in delivering reward signals to align vision models with human preferences. However, current RMs are generally restricted to providing direct responses or engaging in shallow reasoning processes with limited depth, often leading to inaccurate reward signals. We posit that incorporating explicit long chains of thought (CoT) into the reward reasoning process can significantly strengthen their reliability and robustness. Furthermore, we believe that once RMs internalize CoT reasoning, their direct response accuracy can also be improved through implicit reasoning capabilities. To this end, this paper proposes UnifiedReward-Think, the first unified multimodal CoT-based reward model, capable of multi-dimensional, step-by-step long-chain reasoning for both visual understanding and generation reward tasks. Specifically, we adopt an exploration-driven reinforcement fine-tuning approach to elicit and incentivize the model's latent complex reasoning ability: (1) We first use a small amount of image generation preference data to distill the reasoning process of GPT-4o, which is then used for the model's cold start to learn the format and structure of CoT reasoning. (2) Subsequently, by leveraging the model's prior knowledge and generalization capabilities, we prepare large-scale unified multimodal preference data to elicit the model's reasoning process across various vision tasks. During this phase, correct reasoning outputs are retained for rejection sampling to refine the model (3) while incorrect predicted samples are finally used for Group Relative Policy Optimization (GRPO) based reinforcement fine-tuning, enabling the model to explore diverse reasoning paths and optimize for correct and robust solutions. Extensive experiments across various vision reward tasks demonstrate the superiority of our model.
• 摘要：多模式奖励模型（RMS）的最新进展在提供奖励信号以使视力模型与人类偏好相结合时表现出了巨大的希望。但是，当前的RMS通常仅限于提供直接响应或参与深度有限的浅水推理过程，通常导致奖励信号不准确。我们认为，将明确的思想链（COT）纳入奖励推理过程可以显着增强其可靠性和鲁棒性。此外，我们认为，一旦RMS内部化COT推理，它们的直接响应精度也可以通过隐式推理能力提高。为此，本文提出了Unifiedreward-Ink，这是第一个统一的基于多模式的奖励模型，能够具有多维，分步长链推理，以实现视觉理解和生成奖励任务。具体而言，我们采用探索驱动的加固方法来引起并激励该模型的潜在复杂推理能力：（1）我们首先使用少量图像生成偏好数据来提炼GPT-4O的推理过程，然后将其用于模型的冷启动，以学习COT推理的格式和结构。 （2）随后，通过利用模型的先验知识和概括功能，我们准备大规模的统一多模式偏好数据，以在各种视觉任务中启动模型的推理过程。在此阶段，保留了正确的推理输出以进行排斥采样以完善模型（3），而最终将不正确的预测样品用于基于组的相对策略优化（GRPO）基于基于的强化微调，从而使该模型能够探索多样的推理路径并优化了正确和可靠的解决方案。各种视觉奖励任务的广泛实验证明了我们模型的优势。

Title: FLUX-Text: A Simple and Advanced Diffusion Transformer Baseline for Scene Text Editing

• Authors: Rui Lan, Yancheng Bai, Xu Duan, Mingxing Li, Lei Sun, Xiangxiang Chu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.03329
• Pdf URL: https://arxiv.org/pdf/2505.03329
• Copy Paste: [[2505.03329]] FLUX-Text: A Simple and Advanced Diffusion Transformer Baseline for Scene Text Editing(https://arxiv.org/abs/2505.03329)
• Keywords: generation, generative
• Abstract: The task of scene text editing is to modify or add texts on images while maintaining the fidelity of newly generated text and visual coherence with the background. Recent works based on latent diffusion models (LDM) show improved text editing results, yet still face challenges and often generate inaccurate or unrecognizable characters, especially for non-Latin ones (\eg, Chinese), which have complex glyph structures. To address these issues, we present FLUX-Text, a simple and advanced multilingual scene text editing framework based on FLUX-Fill. Specifically, we carefully investigate glyph conditioning, considering both visual and textual modalities. To retain the original generative capabilities of FLUX-Fill while enhancing its understanding and generation of glyphs, we propose lightweight glyph and text embedding modules. Owning to the lightweight design, FLUX-Text is trained only with $100K$ training examples compared to current popular methods trained with 2.9M ones. With no bells and whistles, our method achieves state-of-the-art performance on text editing tasks. Qualitative and quantitative experiments on the public datasets demonstrate that our method surpasses previous works in text fidelity.
• 摘要：场景文本编辑的任务是在图像上修改或添加文本，同时保持新生成的文本和与背景的视觉连贯性的保真度。基于潜在扩散模型（LDM）的最新作品显示出改进的文本编辑结果，但仍然面临挑战，并且通常会产生不准确或无法识别的字符，尤其是对于非拉丁蛋白（\ eg，中文），它们具有复杂的字形结构。为了解决这些问题，我们提出了Flux-Text，这是一个基于Flux-Fill的简单而高级的多语言场景文本编辑框架。具体而言，我们仔细研究了视觉和文本方式，我们仔细研究了字形调节。为了保留通量填充的原始生成能力，同时增强了其对字形的理解和产生，我们建议使用轻质的字形和文本嵌入模块。对于轻巧的设计，Flux-Text仅接受了$ 10.K $培训的训练示例，与当前受到290万培训的流行方法相比。我们的方法没有铃铛和口哨声，可以在文本编辑任务上实现最先进的性能。公共数据集上的定性和定量实验表明，我们的方法超过了文本保真度以前的作品。

Title: Reinforced Correlation Between Vision and Language for Precise Medical AI Assistant

• Authors: Haonan Wang, Jiaji Mao, Lehan Wang, Qixiang Zhang, Marawan Elbatel, Yi Qin, Huijun Hu, Baoxun Li, Wenhui Deng, Weifeng Qin, Hongrui Li, Jialin Liang, Jun Shen, Xiaomeng Li
• Subjects: cs.CV, cs.AI, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.03380
• Pdf URL: https://arxiv.org/pdf/2505.03380
• Copy Paste: [[2505.03380]] Reinforced Correlation Between Vision and Language for Precise Medical AI Assistant(https://arxiv.org/abs/2505.03380)
• Keywords: generation
• Abstract: Medical AI assistants support doctors in disease diagnosis, medical image analysis, and report generation. However, they still face significant challenges in clinical use, including limited accuracy with multimodal content and insufficient validation in real-world settings. We propose RCMed, a full-stack AI assistant that improves multimodal alignment in both input and output, enabling precise anatomical delineation, accurate localization, and reliable diagnosis through hierarchical vision-language grounding. A self-reinforcing correlation mechanism allows visual features to inform language context, while language semantics guide pixel-wise attention, forming a closed loop that refines both modalities. This correlation is enhanced by a color region description strategy, translating anatomical structures into semantically rich text to learn shape-location-text relationships across scales. Trained on 20 million image-mask-description triplets, RCMed achieves state-of-the-art precision in contextualizing irregular lesions and subtle anatomical boundaries, excelling in 165 clinical tasks across 9 modalities. It achieved a 23.5% relative improvement in cell segmentation from microscopy images over prior methods. RCMed's strong vision-language alignment enables exceptional generalization, with state-of-the-art performance in external validation across 20 clinically significant cancer types, including novel tasks. This work demonstrates how integrated multimodal models capture fine-grained patterns, enabling human-level interpretation in complex scenarios and advancing human-centric AI healthcare.
• 摘要：医疗AI助理支持医生疾病诊断，医学图像分析和报告生成。但是，它们在临床使用方面仍然面临重大挑战，包括具有多模式内容的精度有限，在现实世界中的验证不足。我们提出了RCMED，这是一项全栈AI助手，可以改善输入和输出的多模式对齐，从而通过层次视觉语言接地实现精确的解剖学描述，准确的定位和可靠的诊断。一种自我强化的相关机制允许视觉特征为语言上下文提供信息，而语言语义指导Pixel的关注，形成了一种完善这两种方式的闭环。颜色区域描述策略可以增强这种相关性，从而将解剖结构转化为语义丰富的文本，以学习跨尺度的形状锁定文本关系。 RCMed接受了2000万张图像掩码描述的三胞胎，在上下文化的不规则病变和微妙的解剖学边界方面达到了最先进的精度，在9种方式中达到了165个临床任务。它从显微镜图像中的细胞分割相对提高了23.5％，而不是先前的方法。 RCMED的强烈视力语言对齐能够进行非凡的概括，并在20种临床上具有重要意义的癌症类型（包括新任务）中具有最新的外部验证表现。这项工作表明，综合多模型模型如何捕获细粒的模式，在复杂的场景中实现人类水平的解释并推进以人为中心的人工智能医疗保健。

Title: Improving Omics-Based Classification: The Role of Feature Selection and Synthetic Data Generation

• Authors: Diego Perazzolo, Pietro Fanton, Ilaria Barison, Marny Fedrigo, Annalisa Angelini, Chiara Castellani, Enrico Grisan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.03387
• Pdf URL: https://arxiv.org/pdf/2505.03387
• Copy Paste: [[2505.03387]] Improving Omics-Based Classification: The Role of Feature Selection and Synthetic Data Generation(https://arxiv.org/abs/2505.03387)
• Keywords: generation
• Abstract: Given the increasing complexity of omics datasets, a key challenge is not only improving classification performance but also enhancing the transparency and reliability of model decisions. Effective model performance and feature selection are fundamental for explainability and reliability. In many cases, high dimensional omics datasets suffer from limited number of samples due to clinical constraints, patient conditions, phenotypes rarity and others conditions. Current omics based classification models often suffer from narrow interpretability, making it difficult to discern meaningful insights where trust and reproducibility are critical. This study presents a machine learning based classification framework that integrates feature selection with data augmentation techniques to achieve high standard classification accuracy while ensuring better interpretability. Using the publicly available dataset (E MTAB 8026), we explore a bootstrap analysis in six binary classification scenarios to evaluate the proposed model's behaviour. We show that the proposed pipeline yields cross validated perfomance on small dataset that is conserved when the trained classifier is applied to a larger test set. Our findings emphasize the fundamental balance between accuracy and feature selection, highlighting the positive effect of introducing synthetic data for better generalization, even in scenarios with very limited samples availability.
• 摘要：鉴于OMICS数据集的复杂性越来越复杂，关键的挑战不仅是提高分类性能，而且还提高了模型决策的透明度和可靠性。有效的模型性能和功能选择是解释性和可靠性的基础。在许多情况下，由于临床限制，患者状况，表型稀有性和其他状况，高维度数据集的样品数量有限有限。当前基于OMICS的分类模型通常会遭受狭narrow的解释性，因此很难辨别有意义的见解，而信任和可重复性至关重要。这项研究提出了一个基于机器学习的分类框架，该框架将功能选择与数据增强技术集成在一起，以实现高标准分类精度，同时确保更好的解释性。使用公开可用的数据集（E MTAB 8026），我们在六个二进制分类方案中探索了引导分析，以评估所提出的模型的行为。我们表明，所提出的管道在将训练有素的分类器应用于较大的测试集时，在小数据集上产生了交叉验证的完整性。我们的发现强调了准确性和特征选择之间的基本平衡，即使在样本可用性非常有限的情况下，也突出了引入合成数据以更好地泛化的积极效果。

Title: EOPose : Exemplar-based object reposing using Generalized Pose Correspondences

• Authors: Sarthak Mehrotra, Rishabh Jain, Mayur Hemani, Balaji Krishnamurthy, Mausoom Sarkar
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.03394
• Pdf URL: https://arxiv.org/pdf/2505.03394
• Copy Paste: [[2505.03394]] EOPose : Exemplar-based object reposing using Generalized Pose Correspondences(https://arxiv.org/abs/2505.03394)
• Keywords: generative
• Abstract: Reposing objects in images has a myriad of applications, especially for e-commerce where several variants of product images need to be produced quickly. In this work, we leverage the recent advances in unsupervised keypoint correspondence detection between different object images of the same class to propose an end-to-end framework for generic object reposing. Our method, EOPose, takes a target pose-guidance image as input and uses its keypoint correspondence with the source object image to warp and re-render the latter into the target pose using a novel three-step approach. Unlike generative approaches, our method also preserves the fine-grained details of the object such as its exact colors, textures, and brand marks. We also prepare a new dataset of paired objects based on the Objaverse dataset to train and test our network. EOPose produces high-quality reposing output as evidenced by different image quality metrics (PSNR, SSIM and FID). Besides a description of the method and the dataset, the paper also includes detailed ablation and user studies to indicate the efficacy of the proposed method
• 摘要：在图像中重新放置对象具有无数的应用程序，尤其是对于电子商务，需要快速生产产品图像的几种变体。在这项工作中，我们利用了同一类的不同对象图像之间的无监督关键点对应检测的最新进展，以提出一个通用对象的端到端框架。我们的方法Eopose将目标姿势施加图像作为输入，并将其与源对象图像的关键点对应进行扭曲并将后者重新渲染为目标姿势，并使用新颖的三步方法将其重新渲染到目标姿势中。与生成方法不同，我们的方法还保留了对象的精细细节，例如其确切的颜色，纹理和品牌标记。我们还根据OBJAVERSE数据集准备了一个新的配对对象数据集，以训练和测试我们的网络。 Eopose产生高质量的重新稳定输出，这是由不同的图像质量指标（PSNR，SSIM和FID）所证明的。除了对方法和数据集的描述外，本文还包括详细的消融和用户研究，以指示该方法的功效

Title: DDaTR: Dynamic Difference-aware Temporal Residual Network for Longitudinal Radiology Report Generation

• Authors: Shanshan Song, Hui Tang, Honglong Yang, Xiaomeng Li
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.03401
• Pdf URL: https://arxiv.org/pdf/2505.03401
• Copy Paste: [[2505.03401]] DDaTR: Dynamic Difference-aware Temporal Residual Network for Longitudinal Radiology Report Generation(https://arxiv.org/abs/2505.03401)
• Keywords: generation
• Abstract: Radiology Report Generation (RRG) automates the creation of radiology reports from medical imaging, enhancing the efficiency of the reporting process. Longitudinal Radiology Report Generation (LRRG) extends RRG by incorporating the ability to compare current and prior exams, facilitating the tracking of temporal changes in clinical findings. Existing LRRG approaches only extract features from prior and current images using a visual pre-trained encoder, which are then concatenated to generate the final report. However, these methods struggle to effectively capture both spatial and temporal correlations during the feature extraction process. Consequently, the extracted features inadequately capture the information of difference across exams and thus underrepresent the expected progressions, leading to sub-optimal performance in LRRG. To address this, we develop a novel dynamic difference-aware temporal residual network (DDaTR). In DDaTR, we introduce two modules at each stage of the visual encoder to capture multi-level spatial correlations. The Dynamic Feature Alignment Module (DFAM) is designed to align prior features across modalities for the integrity of prior clinical information. Prompted by the enriched prior features, the dynamic difference-aware module (DDAM) captures favorable difference information by identifying relationships across exams. Furthermore, our DDaTR employs the dynamic residual network to unidirectionally transmit longitudinal information, effectively modelling temporal correlations. Extensive experiments demonstrated superior performance over existing methods on three benchmarks, proving its efficacy in both RRG and LRRG tasks.
• 摘要：放射学报告产生（RRG）自动化了医学成像的放射学报告，从而提高了报告过程的效率。纵向放射学报告生成（LRRG）通过合并比较电流和先前检查的能力，从而扩展了RRG，从而促进了临床发现中时间变化的跟踪。现有的LRRG仅使用视觉预训练的编码器从先前和当前图像中提取特征，然后将其串联以生成最终报告。但是，这些方法难以有效捕获特征提取过程中的空间和时间相关性。因此，提取的特征不足地捕获了整个考试之间的差异信息，因此预期的进展不足，导致LRRG中的次优性能。为了解决这个问题，我们开发了一种新型的动态差异感知时间残留网络（DDATR）。在DDATR中，我们在视觉编码器的每个阶段介绍了两个模块，以捕获多级空间相关性。动态特征比对模块（DFAM）旨在使跨模态的先前特征与先前的临床信息的完整性保持一致。在丰富的先验功能的提示下，动态差异感知模块（DDAM）通过识别跨考试的关系捕获了有利的差异信息。此外，我们的DDATR使用动态残留网络进行单向传输纵向信息，从而有效地对时间相关进行建模。广泛的实验表明，在三个基准上的现有方法表明，在RRG和LRRG任务中都证明了其功效。

Title: Phenotype-Guided Generative Model for High-Fidelity Cardiac MRI Synthesis: Advancing Pretraining and Clinical Applications

• Authors: Ziyu Li, Yujian Hu, Zhengyao Ding, Yiheng Mao, Haitao Li, Fan Yi, Hongkun Zhang, Zhengxing Huang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.03426
• Pdf URL: https://arxiv.org/pdf/2505.03426
• Copy Paste: [[2505.03426]] Phenotype-Guided Generative Model for High-Fidelity Cardiac MRI Synthesis: Advancing Pretraining and Clinical Applications(https://arxiv.org/abs/2505.03426)
• Keywords: generation, generative
• Abstract: Cardiac Magnetic Resonance (CMR) imaging is a vital non-invasive tool for diagnosing heart diseases and evaluating cardiac health. However, the limited availability of large-scale, high-quality CMR datasets poses a major challenge to the effective application of artificial intelligence (AI) in this domain. Even the amount of unlabeled data and the health status it covers are difficult to meet the needs of model pretraining, which hinders the performance of AI models on downstream tasks. In this study, we present Cardiac Phenotype-Guided CMR Generation (CPGG), a novel approach for generating diverse CMR data that covers a wide spectrum of cardiac health status. The CPGG framework consists of two stages: in the first stage, a generative model is trained using cardiac phenotypes derived from CMR data; in the second stage, a masked autoregressive diffusion model, conditioned on these phenotypes, generates high-fidelity CMR cine sequences that capture both structural and functional features of the heart in a fine-grained manner. We synthesized a massive amount of CMR to expand the pretraining data. Experimental results show that CPGG generates high-quality synthetic CMR data, significantly improving performance on various downstream tasks, including diagnosis and cardiac phenotypes prediction. These gains are demonstrated across both public and private datasets, highlighting the effectiveness of our approach. Code is availabel at this https URL.
• 摘要：心脏磁共振（CMR）成像是诊断心脏病和评估心脏健康的重要非侵入性工具。但是，大规模，高质量的CMR数据集的有限可用性对在该域中有效应用（AI）的有效应用构成了重大挑战。即使是未标记的数据及其涵盖的健康状况的数量也很难满足模型预处理的需求，这阻碍了AI模型在下游任务上的性能。在这项研究中，我们介绍了心脏表型引导的CMR生成（CPGG），这是一种新的方法，用于生成各种CMR数据，涵盖了广泛的心脏健康状况。 CPGG框架由两个阶段组成：在第一阶段，使用从CMR数据得出的心脏表型训练生成模型；在第二阶段，以这些表型为条件的掩盖自回归扩散模型产生了高保真的CMR CINE序列，该序列以细粒度的方式捕获心脏的结构和功能特征。我们合成了大量的CMR来扩展预处理数据。实验结果表明，CPGG会产生高质量的合成CMR数据，从而显着提高了各种下游任务的性能，包括诊断和心脏表型预测。这些收益在公共和私人数据集中得到了证明，突出了我们方法的有效性。该https URL的代码可用。

Title: A Fusion-Guided Inception Network for Hyperspectral Image Super-Resolution

• Authors: Usman Muhammad, Jorma Laaksonen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.03431
• Pdf URL: https://arxiv.org/pdf/2505.03431
• Copy Paste: [[2505.03431]] A Fusion-Guided Inception Network for Hyperspectral Image Super-Resolution(https://arxiv.org/abs/2505.03431)
• Keywords: super-resolution
• Abstract: The fusion of low-spatial-resolution hyperspectral images (HSIs) with high-spatial-resolution conventional images (e.g., panchromatic or RGB) has played a significant role in recent advancements in HSI super-resolution. However, this fusion process relies on the availability of precise alignment between image pairs, which is often challenging in real-world scenarios. To mitigate this limitation, we propose a single-image super-resolution model called the Fusion-Guided Inception Network (FGIN). Specifically, we first employ a spectral-spatial fusion module to effectively integrate spectral and spatial information at an early stage. Next, an Inception-like hierarchical feature extraction strategy is used to capture multiscale spatial dependencies, followed by a dedicated multi-scale fusion block. To further enhance reconstruction quality, we incorporate an optimized upsampling module that combines bilinear interpolation with depthwise separable convolutions. Experimental evaluations on two publicly available hyperspectral datasets demonstrate the competitive performance of our method.
• 摘要：低空间分辨率高光谱图像（HSIS）与高空间分辨率常规图像（例如，Panchrostic或RGB）的融合在HSI超级分辨率的最新进步中发挥了重要作用。但是，这种融合过程依赖于图像对之间的精确对齐的可用性，这在现实世界中通常具有挑战性。为了减轻这种限制，我们提出了一个称为Fusion引导的成立网络（FGIN）的单位图像超分辨率模型。具体而言，我们首先采用光谱空间融合模块在早期阶段有效整合光谱和空间信息。接下来，使用类似层次的分层特征提取策略来捕获多尺度空间依赖性，然后是专用的多尺度融合块。为了进一步提高重建质量，我们结合了一个优化的上采样模块，将双线性插值与深度可分离的卷积结合在一起。对两个公开可获得的高光谱数据集的实验评估证明了我们方法的竞争性能。

Title: Wasserstein Convergence of Score-based Generative Models under Semiconvexity and Discontinuous Gradients

• Authors: Stefano Bruno, Sotirios Sabanis
• Subjects: cs.LG, math.OC, math.PR, stat.ML
• Abstract URL: https://arxiv.org/abs/2505.03432
• Pdf URL: https://arxiv.org/pdf/2505.03432
• Copy Paste: [[2505.03432]] Wasserstein Convergence of Score-based Generative Models under Semiconvexity and Discontinuous Gradients(https://arxiv.org/abs/2505.03432)
• Keywords: generation, generative
• Abstract: Score-based Generative Models (SGMs) approximate a data distribution by perturbing it with Gaussian noise and subsequently denoising it via a learned reverse diffusion process. These models excel at modeling complex data distributions and generating diverse samples, achieving state-of-the-art performance across domains such as computer vision, audio generation, reinforcement learning, and computational biology. Despite their empirical success, existing Wasserstein-2 convergence analysis typically assume strong regularity conditions-such as smoothness or strict log-concavity of the data distribution-that are rarely satisfied in practice. In this work, we establish the first non-asymptotic Wasserstein-2 convergence guarantees for SGMs targeting semiconvex distributions with potentially discontinuous gradients. Our upper bounds are explicit and sharp in key parameters, achieving optimal dependence of $O(\sqrt{d})$ on the data dimension $d$ and convergence rate of order one. The framework accommodates a wide class of practically relevant distributions, including symmetric modified half-normal distributions, Gaussian mixtures, double-well potentials, and elastic net potentials. By leveraging semiconvexity without requiring smoothness assumptions on the potential such as differentiability, our results substantially broaden the theoretical foundations of SGMs, bridging the gap between empirical success and rigorous guarantees in non-smooth, complex data regimes.
• 摘要：基于得分的生成模型（SGM）通过用高斯噪声扰动数据分布，然后通过学习的反向扩散过程将其剥开。这些模型在建模复杂的数据分布并生成各种样本，在跨计算机视觉，音频产生，增强学习和计算生物学等领域中实现最先进的性能方面表现出色。尽管经验成功，但现有的Wasserstein-2收敛分析通常假定强大的规律性条件，例如平滑度或严格的数据分布的对数洞穴，而在实践中很少满足。在这项工作中，我们建立了针对具有潜在不连续梯度的半串分布的SGMS的第一个非质合剂wasserstein-2收敛保证。我们的上限在关键参数上是显式且清晰的，可以在数据尺寸$ d $和订单第一的收敛速率上实现$ o（\ sqrt {d}）$的最佳依赖性。该框架可容纳一系列实际相关的分布，包括对称修改的半正常分布，高斯混合物，双孔电位和弹性净电位。通过在不需要对潜力（例如不同的性能）上的平稳性假设的情况下，我们的结果实质上扩大了SGM的理论基础，从而弥合了经验成功与严格的保证在非平滑，复杂的数据制度中的严格保证之间的差距。

Title: Robustness in AI-Generated Detection: Enhancing Resistance to Adversarial Attacks

• Authors: Sun Haoxuan, Hong Yan, Zhan Jiahui, Chen Haoxing, Lan Jun, Zhu Huijia, Wang Weiqiang, Zhang Liqing, Zhang Jianfu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.03435
• Pdf URL: https://arxiv.org/pdf/2505.03435
• Copy Paste: [[2505.03435]] Robustness in AI-Generated Detection: Enhancing Resistance to Adversarial Attacks(https://arxiv.org/abs/2505.03435)
• Keywords: generation, generative
• Abstract: The rapid advancement of generative image technology has introduced significant security concerns, particularly in the domain of face generation detection. This paper investigates the vulnerabilities of current AI-generated face detection systems. Our study reveals that while existing detection methods often achieve high accuracy under standard conditions, they exhibit limited robustness against adversarial attacks. To address these challenges, we propose an approach that integrates adversarial training to mitigate the impact of adversarial examples. Furthermore, we utilize diffusion inversion and reconstruction to further enhance detection robustness. Experimental results demonstrate that minor adversarial perturbations can easily bypass existing detection systems, but our method significantly improves the robustness of these systems. Additionally, we provide an in-depth analysis of adversarial and benign examples, offering insights into the intrinsic characteristics of AI-generated content. All associated code will be made publicly available in a dedicated repository to facilitate further research and verification.
• 摘要：生成图像技术的快速发展引入了重大的安全问题，尤其是在面部生成检测领域。本文研究了当前AI生成的面部检测系统的脆弱性。我们的研究表明，尽管现有的检测方法通常在标准条件下达到高精度，但它们对对抗性攻击的鲁棒性有限。为了应对这些挑战，我们提出了一种整合对抗训练以减轻对抗性例子的影响的方法。此外，我们利用扩散反演和重建来进一步增强检测鲁棒性。实验结果表明，较小的对抗扰动可以轻松绕过现有的检测系统，但是我们的方法显着提高了这些系统的鲁棒性。此外，我们提供了对对抗和良性示例的深入分析，从而提供了对AI生成含量的内在特征的见解。所有相关的代码将在专用存储库中公开提供，以促进进一步的研究和验证。

Title: A new membership inference attack that spots memorization in generative and predictive models: Loss-Based with Reference Model algorithm (LBRM)

• Authors: Faiz Taleb, Ivan Gazeau, Maryline Laurent
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.03490
• Pdf URL: https://arxiv.org/pdf/2505.03490
• Copy Paste: [[2505.03490]] A new membership inference attack that spots memorization in generative and predictive models: Loss-Based with Reference Model algorithm (LBRM)(https://arxiv.org/abs/2505.03490)
• Keywords: generative
• Abstract: Generative models can unintentionally memorize training data, posing significant privacy risks. This paper addresses the memorization phenomenon in time series imputation models, introducing the Loss-Based with Reference Model (LBRM) algorithm. The LBRM method leverages a reference model to enhance the accuracy of membership inference attacks, distinguishing between training and test data. Our contributions are twofold: first, we propose an innovative method to effectively extract and identify memorized training data, significantly improving detection accuracy. On average, without fine-tuning, the AUROC improved by approximately 40\%. With fine-tuning, the AUROC increased by approximately 60\%. Second, we validate our approach through membership inference attacks on two types of architectures designed for time series imputation, demonstrating the robustness and versatility of the LBRM approach in different contexts. These results highlight the significant enhancement in detection accuracy provided by the LBRM approach, addressing privacy risks in time series imputation models.
• 摘要：生成模型可以无意间记住培训数据，从而带来很大的隐私风险。本文介绍了时间序列插补模型中的记忆现象，并通过参考模型（LBRM）算法引入了基于损失的现象。 LBRM方法利用参考模型来增强会员推理攻击的准确性，从而区分培训和测试数据。我们的贡献是双重的：首先，我们提出了一种创新的方法来有效提取和识别记忆的训练数据，从而显着提高了检测准确性。平均而言，没有微调，AUROC提高了约40 \％。通过微调，AUROC增加了约60 \％。其次，我们通过成员推理攻击对时间序列插补设计的两种类型的体系结构来验证我们的方法，从而证明了在不同情况下LBRM方法的鲁棒性和多功能性。这些结果突出了LBRM方法提供的检测准确性的显着增强，从而解决了时间序列归档模型中的隐私风险。

Title: Modality-Guided Dynamic Graph Fusion and Temporal Diffusion for Self-Supervised RGB-T Tracking

• Authors: Shenglan Li, Rui Yao, Yong Zhou, Hancheng Zhu, Kunyang Sun, Bing Liu, Zhiwen Shao, Jiaqi Zhao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.03507
• Pdf URL: https://arxiv.org/pdf/2505.03507
• Copy Paste: [[2505.03507]] Modality-Guided Dynamic Graph Fusion and Temporal Diffusion for Self-Supervised RGB-T Tracking(https://arxiv.org/abs/2505.03507)
• Keywords: generative
• Abstract: To reduce the reliance on large-scale annotations, self-supervised RGB-T tracking approaches have garnered significant attention. However, the omission of the object region by erroneous pseudo-label or the introduction of background noise affects the efficiency of modality fusion, while pseudo-label noise triggered by similar object noise can further affect the tracking performance. In this paper, we propose GDSTrack, a novel approach that introduces dynamic graph fusion and temporal diffusion to address the above challenges in self-supervised RGB-T tracking. GDSTrack dynamically fuses the modalities of neighboring frames, treats them as distractor noise, and leverages the denoising capability of a generative model. Specifically, by constructing an adjacency matrix via an Adjacency Matrix Generator (AMG), the proposed Modality-guided Dynamic Graph Fusion (MDGF) module uses a dynamic adjacency matrix to guide graph attention, focusing on and fusing the object's coherent regions. Temporal Graph-Informed Diffusion (TGID) models MDGF features from neighboring frames as interference, and thus improving robustness against similar-object noise. Extensive experiments conducted on four public RGB-T tracking datasets demonstrate that GDSTrack outperforms the existing state-of-the-art methods. The source code is available at this https URL.
• 摘要：为了减少对大规模注释的依赖，自我监督的RGB-T跟踪方法引起了极大的关注。但是，通过错误的伪标签或引入背景噪声省略对象区域会影响模态融合的效率，而相似对象噪声触发的伪标记噪声会进一步影响跟踪性能。在本文中，我们提出了GDSTRACK，这是一种新颖的方法，它引入了动态图融合和时间扩散，以解决自我监督的RGB-T跟踪中上述挑战。 GDSTRACK动态融合了相邻帧的方式，将它们视为干扰器噪声，并利用生成模型的降解能力。具体而言，通过通过邻接矩阵发生器（AMG）构建邻接矩阵，提出的模态引导的动态图融合（MDGF）模块使用动态邻接矩阵来指导图形注意力，重点关注并融合对象的相干区域。时间图信息扩散（TGID）模型来自相邻帧的MDGF特征作为干扰，从而提高了针对相似对象噪声的鲁棒性。在四个公共RGB-T跟踪数据集上进行的广泛实验表明，GDSTRACK优于现有的最新方法。源代码可在此HTTPS URL上找到。

Title: Optimization of Module Transferability in Single Image Super-Resolution: Universality Assessment and Cycle Residual Blocks

• Authors: Haotong Cheng, Zhiqi Zhang, Hao Li, Xinshang Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.03522
• Pdf URL: https://arxiv.org/pdf/2505.03522
• Copy Paste: [[2505.03522]] Optimization of Module Transferability in Single Image Super-Resolution: Universality Assessment and Cycle Residual Blocks(https://arxiv.org/abs/2505.03522)
• Keywords: super-resolution
• Abstract: Deep learning has substantially advanced the Single Image Super-Resolution (SISR). However, existing researches have predominantly focused on raw performance gains, with little attention paid to quantifying the transferability of architectural components. In this paper, we introduce the concept of "Universality" and its associated definitions which extend the traditional notion of "Generalization" to encompass the modules' ease of transferability, thus revealing the relationships between module universality and model generalizability. Then we propose the Universality Assessment Equation (UAE), a metric for quantifying how readily a given module could be transplanted across models. Guided by the UAE results of standard residual blocks and other plug-and-play modules, we further design two optimized modules, Cycle Residual Block (CRB) and Depth-Wise Cycle Residual Block (DCRB). Through comprehensive experiments on natural-scene benchmarks, remote-sensing datasets, extreme-industrial imagery and on-device deployments, we demonstrate that networks embedded with the proposed plug-and-play modules outperform several state-of-the-arts, reaching a PSNR enhancement of up to 0.83dB or enabling a 71.3% reduction in parameters with negligible loss in reconstruction fidelity.
• 摘要：深度学习已经大大提高了单图超分辨率（SISR）。但是，现有的研究主要集中在原始绩效增长上，而很少关注量化建筑组件的可传递性。在本文中，我们介绍了“普遍性”的概念及其相关的定义，该定义扩展了传统的“概括”概念，以涵盖模块的可传递性易用性，从而揭示了模块通用性与模型通用性之间的关系。然后，我们提出了通用评估方程（UAE），这是一种用于量化在模型中可以移植给定模块的容易移植的度量。在阿联酋的标准残差块和其他插件模块的结果下，我们进一步设计了两个优化的模块，即循环残留块（CRB）和深度循环残留块（DCRB）。通过对天然基础基准，遥感数据集，极端工业图像和设备部署的全面实验，我们证明，与拟议的插件模式嵌入的网络超过了几个最先进的播放模式，超过了几个最先进的网络，可以增强PSNR的范围，从而增强了0.83db或Reffor a Refction a Refction 71.3％REDUCTION in 71.3％。忠诚。

Title: Causal Intervention Framework for Variational Auto Encoder Mechanistic Interpretability

• Authors: Dip Roy
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.03530
• Pdf URL: https://arxiv.org/pdf/2505.03530
• Copy Paste: [[2505.03530]] Causal Intervention Framework for Variational Auto Encoder Mechanistic Interpretability(https://arxiv.org/abs/2505.03530)
• Keywords: generative
• Abstract: Mechanistic interpretability of deep learning models has emerged as a crucial research direction for understanding the functioning of neural networks. While significant progress has been made in interpreting discriminative models like transformers, understanding generative models such as Variational Autoencoders (VAEs) remains challenging. This paper introduces a comprehensive causal intervention framework for mechanistic interpretability of VAEs. We develop techniques to identify and analyze "circuit motifs" in VAEs, examining how semantic factors are encoded, processed, and disentangled through the network layers. Our approach uses targeted interventions at different levels: input manipulations, latent space perturbations, activation patching, and causal mediation analysis. We apply our framework to both synthetic datasets with known causal relationships and standard disentanglement benchmarks. Results show that our interventions can successfully isolate functional circuits, map computational graphs to causal graphs of semantic factors, and distinguish between polysemantic and monosemantic units. Furthermore, we introduce metrics for causal effect strength, intervention specificity, and circuit modularity that quantify the interpretability of VAE components. Experimental results demonstrate clear differences between VAE variants, with FactorVAE achieving higher disentanglement scores (0.084) and effect strengths (mean 4.59) compared to standard VAE (0.064, 3.99) and Beta-VAE (0.051, 3.43). Our framework advances the mechanistic understanding of generative models and provides tools for more transparent and controllable VAE architectures.
• 摘要：深度学习模型的机械解释性已成为理解神经网络功能的关键研究方向。尽管在解释诸如变形金刚之类的歧视模型方面取得了重大进展，但了解生成模型（例如变异自动编码器（VAE））仍然具有挑战性。本文介绍了一个全面的因果干预框架，用于VAE的机械性解释性。我们开发技术来识别和分析VAE中的“电路图案”，研究如何通过网络层编码，处理和分解语义因素。我们的方法在不同级别采用有针对性的干预措施：输入操作，潜在空间扰动，激活修补和因果中介分析。我们将框架应用于具有已知因果关系和标准分离基准测试的综合数据集。结果表明，我们的干预措施可以成功地隔离功能电路，将计算图映射到语义因素的因果图，并区分多义和单义单元。此外，我们介绍了量化VAE组件的可解释性的因果效应强度，干预特异性和电路模块的指标。实验结果表明，与标准VAE（0.064，3.99）和beta-vae（0.051，3.43）相比，Vaee变体之间的VAE变体之间存在明显的差异，而FactorVae获得了更高的分离得分（0.084）和效应强度（平均4.59）。我们的框架提高了对生成模型的机械理解，并为更透明和可控制的VAE架构提供了工具。

Title: Panoramic Out-of-Distribution Segmentation

• Authors: Mengfei Duan, Kailun Yang, Yuheng Zhang, Yihong Cao, Fei Teng, Kai Luo, Jiaming Zhang, Zhiyong Li, Shutao Li
• Subjects: cs.CV, cs.RO, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.03539
• Pdf URL: https://arxiv.org/pdf/2505.03539
• Copy Paste: [[2505.03539]] Panoramic Out-of-Distribution Segmentation(https://arxiv.org/abs/2505.03539)
• Keywords: restoration
• Abstract: Panoramic imaging enables capturing 360° images with an ultra-wide Field-of-View (FoV) for dense omnidirectional perception. However, current panoramic semantic segmentation methods fail to identify outliers, and pinhole Out-of-distribution Segmentation (OoS) models perform unsatisfactorily in the panoramic domain due to background clutter and pixel distortions. To address these issues, we introduce a new task, Panoramic Out-of-distribution Segmentation (PanOoS), achieving OoS for panoramas. Furthermore, we propose the first solution, POS, which adapts to the characteristics of panoramic images through text-guided prompt distribution learning. Specifically, POS integrates a disentanglement strategy designed to materialize the cross-domain generalization capability of CLIP. The proposed Prompt-based Restoration Attention (PRA) optimizes semantic decoding by prompt guidance and self-adaptive correction, while Bilevel Prompt Distribution Learning (BPDL) refines the manifold of per-pixel mask embeddings via semantic prototype supervision. Besides, to compensate for the scarcity of PanOoS datasets, we establish two benchmarks: DenseOoS, which features diverse outliers in complex environments, and QuadOoS, captured by a quadruped robot with a panoramic annular lens system. Extensive experiments demonstrate superior performance of POS, with AuPRC improving by 34.25% and FPR95 decreasing by 21.42% on DenseOoS, outperforming state-of-the-art pinhole-OoS methods. Moreover, POS achieves leading closed-set segmentation capabilities. Code and datasets will be available at this https URL.
• 摘要：全景成像可以使用超宽的视野（FOV）捕获360°图像，以进行致密的全向感知。但是，当前的全景语义分割方法无法识别异常值，并且由于背景混乱和像素扭曲，针孔分布分割（OOS）模型在全景结构域中表现不佳。为了解决这些问题，我们介绍了一项新任务，全景分布细分（Panoos），为Panoramas实现OOS。此外，我们提出了第一个解决方案，即通过文本引导的及时分发学习来适应全景图像的特征。具体而言，POS集成了旨在实现夹子跨域概括能力的分离策略。提出的基于及时的恢复注意力（PRA）通过及时的指导和自适应校正优化语义解码，而二重性及时分发学习（BPDL）通过语义原型监督优化了每个像素掩码嵌入的歧管。此外，为了弥补Panoos数据集的稀缺性，我们建立了两个基准：Denseoos，具有复杂环境中各种异常值，而Quadoos则由具有全景环形透镜系统的四足机器人捕获。广泛的实验表明，POS的表现卓越，AUPRC在密度上提高了34.25％，FPR95降低了21.42％，表现优于最先进的针孔 - OLOOS方法。此外，POS可以实现领先的封闭式分割功能。代码和数据集将在此HTTPS URL上可用。

Title: Generating Synthetic Data via Augmentations for Improved Facial Resemblance in DreamBooth and InstantID

• Authors: Koray Ulusan, Benjamin Kiefer
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.03557
• Pdf URL: https://arxiv.org/pdf/2505.03557
• Copy Paste: [[2505.03557]] Generating Synthetic Data via Augmentations for Improved Facial Resemblance in DreamBooth and InstantID(https://arxiv.org/abs/2505.03557)
• Keywords: generation
• Abstract: The personalization of Stable Diffusion for generating professional portraits from amateur photographs is a burgeoning area, with applications in various downstream contexts. This paper investigates the impact of augmentations on improving facial resemblance when using two prominent personalization techniques: DreamBooth and InstantID. Through a series of experiments with diverse subject datasets, we assessed the effectiveness of various augmentation strategies on the generated headshots' fidelity to the original subject. We introduce FaceDistance, a wrapper around FaceNet, to rank the generations based on facial similarity, which aided in our assessment. Ultimately, this research provides insights into the role of augmentations in enhancing facial resemblance in SDXL-generated portraits, informing strategies for their effective deployment in downstream applications.
• 摘要：从业余照片中产生专业肖像的稳定扩散的个性化是一个新兴的领域，其应用在各种下游环境中。本文使用两种突出的个性化技术：Dreambooth和Instantid时，研究了增强对改善面部相似之处的影响。通过一系列具有不同主题数据集的实验，我们评估了各种增强策略对产生的头像对原始主题的保真度的有效性。我们介绍了面对面的围绕FaceNet的包装，以基于面部相似性对几代人进行排名，这有助于我们的评估。最终，这项研究提供了有关增强在增强SDXL生成的肖像中面部相似之处的作用的见解，为其在下游应用程序中有效部署的策略提供了依据。

Title: Rapid AI-based generation of coverage paths for dispensing applications

• Authors: Simon Baeuerle, Ian F. Mendonca, Kristof Van Laerhoven, Ralf Mikut, Andreas Steimer
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.03560
• Pdf URL: https://arxiv.org/pdf/2505.03560
• Copy Paste: [[2505.03560]] Rapid AI-based generation of coverage paths for dispensing applications(https://arxiv.org/abs/2505.03560)
• Keywords: generation
• Abstract: Coverage Path Planning of Thermal Interface Materials (TIM) plays a crucial role in the design of power electronics and electronic control units. Up to now, this is done manually by experts or by using optimization approaches with a high computational effort. We propose a novel AI-based approach to generate dispense paths for TIM and similar dispensing applications. It is a drop-in replacement for optimization-based approaches. An Artificial Neural Network (ANN) receives the target cooling area as input and directly outputs the dispense path. Our proposed setup does not require labels and we show its feasibility on multiple target areas. The resulting dispense paths can be directly transferred to automated manufacturing equipment and do not exhibit air entrapments. The approach of using an ANN to predict process parameters for a desired target state in real-time could potentially be transferred to other manufacturing processes.
• 摘要：热界面材料（TIM）的覆盖路径计划在电力电子和电子控制单元的设计中起着至关重要的作用。到目前为止，这是由专家或使用高度计算工作的优化方法手动完成的。我们提出了一种基于AI的新型方法，以生成针对TIM和类似分配应用的分配路径。这是用于基于优化的方法的替换。人工神经网络（ANN）接收目标冷却区域作为输入，并直接输出分配路径。我们提出的设置不需要标签，我们在多个目标区域显示了它的可行性。所得的分配路径可以直接转移到自动制造设备上，并且不显示空气夹带。使用ANN实时预测所需目标状态的过程参数的方法可能会转移到其他制造过程中。

Title: Ergodic Generative Flows

• Authors: Leo Maxime Brunswic, Mateo Clemente, Rui Heng Yang, Adam Sigal, Amir Rasouli, Yinchuan Li
• Subjects: cs.LG, cs.AI, math.DG, math.DS
• Abstract URL: https://arxiv.org/abs/2505.03561
• Pdf URL: https://arxiv.org/pdf/2505.03561
• Copy Paste: [[2505.03561]] Ergodic Generative Flows(https://arxiv.org/abs/2505.03561)
• Keywords: generative
• Abstract: Generative Flow Networks (GFNs) were initially introduced on directed acyclic graphs to sample from an unnormalized distribution density. Recent works have extended the theoretical framework for generative methods allowing more flexibility and enhancing application range. However, many challenges remain in training GFNs in continuous settings and for imitation learning (IL), including intractability of flow-matching loss, limited tests of non-acyclic training, and the need for a separate reward model in imitation learning. The present work proposes a family of generative flows called Ergodic Generative Flows (EGFs) which are used to address the aforementioned issues. First, we leverage ergodicity to build simple generative flows with finitely many globally defined transformations (diffeomorphisms) with universality guarantees and tractable flow-matching loss (FM loss). Second, we introduce a new loss involving cross-entropy coupled to weak flow-matching control, coined KL-weakFM loss. It is designed for IL training without a separate reward model. We evaluate IL-EGFs on toy 2D tasks and real-world datasets from NASA on the sphere, using the KL-weakFM loss. Additionally, we conduct toy 2D reinforcement learning experiments with a target reward, using the FM loss.
• 摘要：最初在定向的无环图上引入了生成流动网络（GFN），以从非正均分布密度进行采样。最近的作品扩展了生成方法的理论框架，从而可以更灵活并增强应用程序范围。但是，在连续环境和模仿学习（IL）的培训GFN中仍然存在许多挑战，包括匹配匹配损失的棘手性，非囊泡培训的有限测试以及在模仿学习中需要单独的奖励模型。目前的工作提出了一个称为Ergodic Generative Flow（EGFS）的生成流量，用于解决上述问题。首先，我们利用恐怖性建立简单的生成流，并具有许多全球定义的转换（差异），并保证了普遍性和可拖动的流量匹配损失（FM损失）。其次，我们引入了一种新的损失，涉及跨凝结耦合到弱流量匹配控制，造成的KL-WeakFM损失。它是为IL培训而设计的，而无需单独的奖励模型。我们使用KL-WeakFM损失评估了来自NASA的Toy 2D任务和实际数据集的IL-EGF。此外，我们使用FM损失进行了目标奖励的玩具2D增强学习实验。

Title: Real-Time Person Image Synthesis Using a Flow Matching Model

• Authors: Jiwoo Jeong, Kirok Kim, Wooju Kim, Nam-Joon Kim
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.03562
• Pdf URL: https://arxiv.org/pdf/2505.03562
• Copy Paste: [[2505.03562]] Real-Time Person Image Synthesis Using a Flow Matching Model(https://arxiv.org/abs/2505.03562)
• Keywords: generation, generative
• Abstract: Pose-Guided Person Image Synthesis (PGPIS) generates realistic person images conditioned on a target pose and a source image. This task plays a key role in various real-world applications, such as sign language video generation, AR/VR, gaming, and live streaming. In these scenarios, real-time PGPIS is critical for providing immediate visual feedback and maintaining user this http URL, achieving real-time performance remains a significant challenge due to the complexity of synthesizing high-fidelity images from diverse and dynamic human poses. Recent diffusion-based methods have shown impressive image quality in PGPIS, but their slow sampling speeds hinder deployment in time-sensitive applications. This latency is particularly problematic in tasks like generating sign language videos during live broadcasts, where rapid image updates are required. Therefore, developing a fast and reliable PGPIS model is a crucial step toward enabling real-time interactive systems. To address this challenge, we propose a generative model based on flow matching (FM). Our approach enables faster, more stable, and more efficient training and sampling. Furthermore, the proposed model supports conditional generation and can operate in latent space, making it especially suitable for real-time PGPIS applications where both speed and quality are critical. We evaluate our proposed method, Real-Time Person Image Synthesis Using a Flow Matching Model (RPFM), on the widely used DeepFashion dataset for PGPIS tasks. Our results show that RPFM achieves near-real-time sampling speeds while maintaining performance comparable to the state-of-the-art models. Our methodology trades off a slight, acceptable decrease in generated-image accuracy for over a twofold increase in generation speed, thereby ensuring real-time performance.
• 摘要：姿势指导的人图像合成（PGPI）生成了以目标姿势和源图像为条件的真实人图像。此任务在各种现实世界应用中起着关键作用，例如手语视频生成，AR/VR，游戏和实时流媒体。在这些情况下，实时PGPI对于提供直接的视觉反馈和维护用户至关重要，因此实现实时性能仍然是一个重大挑战，这是由于合成来自多样化和动态人类姿势的高保真图像的复杂性。最近基于扩散的方法在PGPI中显示出令人印象深刻的图像质量，但是它们的缓慢采样速度阻碍了时间敏感的应用中的部署。在实时广播中生成手语视频（在需要快速图像更新）中生成手语视频等任务中，这种延迟尤其有问题。因此，开发快速可靠的PGPIS模型是实现实时交互系统的关键步骤。为了应对这一挑战，我们提出了一个基于流匹配（FM）的生成模型。我们的方法可以更快，更稳定，更有效的培训和抽样。此外，提出的模型支持有条件的生成并可以在潜在空间中运行，这使其特别适合速度和质量都至关重要的实时PGPIS应用。我们使用流量匹配模型（RPFM）在广泛使用的DeepFashion数据集上评估了我们提出的方法，即实时人员图像合成。我们的结果表明，RPFM达到了近实时的抽样速度，同时保持与最先进模型相当的性能。我们的方法可以使生成图像准确性的略有下降，可接受的降低，以超过生成速度的提高，从而确保实时性能。

Title: From Pixels to Polygons: A Survey of Deep Learning Approaches for Medical Image-to-Mesh Reconstruction

• Authors: Fengming Lin, Arezoo Zakeri, Yidan Xue, Michael MacRaild, Haoran Dou, Zherui Zhou, Ziwei Zou, Ali Sarrami-Foroushani, Jinming Duan, Alejandro F. Frangi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.03599
• Pdf URL: https://arxiv.org/pdf/2505.03599
• Copy Paste: [[2505.03599]] From Pixels to Polygons: A Survey of Deep Learning Approaches for Medical Image-to-Mesh Reconstruction(https://arxiv.org/abs/2505.03599)
• Keywords: generative
• Abstract: Deep learning-based medical image-to-mesh reconstruction has rapidly evolved, enabling the transformation of medical imaging data into three-dimensional mesh models that are critical in computational medicine and in silico trials for advancing our understanding of disease mechanisms, and diagnostic and therapeutic techniques in modern medicine. This survey systematically categorizes existing approaches into four main categories: template models, statistical models, generative models, and implicit models. Each category is analysed in detail, examining their methodological foundations, strengths, limitations, and applicability to different anatomical structures and imaging modalities. We provide an extensive evaluation of these methods across various anatomical applications, from cardiac imaging to neurological studies, supported by quantitative comparisons using standard metrics. Additionally, we compile and analyze major public datasets available for medical mesh reconstruction tasks and discuss commonly used evaluation metrics and loss functions. The survey identifies current challenges in the field, including requirements for topological correctness, geometric accuracy, and multi-modality integration. Finally, we present promising future research directions in this domain. This systematic review aims to serve as a comprehensive reference for researchers and practitioners in medical image analysis and computational medicine.
• 摘要：基于深度学习的医学图像到网格的重建已经迅速发展，从而使医学成像数据转换为三维网格模型，这些模型在计算医学和硅硅试验中至关重要，以促进我们对疾病机制的理解以及现代医学中诊断和治疗技术的理解。该调查系统地将现有方法分为四个主要类别：模板模型，统计模型，生成模型和隐式模型。对每个类别进行详细分析，研究其方法论基础，优势，局限性以及对不同解剖结构和成像方式的适用性。我们对这些方法进行了广泛的评估，从心脏成像到神经学研究，通过使用标准指标进行定量比较支持。此外，我们编译和分析可用于医疗网格重建任务的主要公共数据集，并讨论常用的评估指标和损失功能。该调查确定了该领域的当前挑战，包括对拓扑正确性，几何准确性和多模式集成的要求。最后，我们介绍了该领域中有希望的未来研究方向。这项系统评价旨在为医学图像分析和计算医学中的研究人员和从业人员提供全面的参考。

Title: PAHA: Parts-Aware Audio-Driven Human Animation with Diffusion Model

• Authors: Y.B. Wang, S.Z. Zhou, J.F. Wu, T. Hu, J.N. Zhang, Y. Liu
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2505.03603
• Pdf URL: https://arxiv.org/pdf/2505.03603
• Copy Paste: [[2505.03603]] PAHA: Parts-Aware Audio-Driven Human Animation with Diffusion Model(https://arxiv.org/abs/2505.03603)
• Keywords: generation
• Abstract: Audio-driven human animation technology is widely used in human-computer interaction, and the emergence of diffusion models has further advanced its development. Currently, most methods rely on multi-stage generation and intermediate representations, resulting in long inference time and issues with generation quality in specific foreground regions and audio-motion consistency. These shortcomings are primarily due to the lack of localized fine-grained supervised guidance. To address above challenges, we propose PAHA, an end-to-end audio-driven upper-body human animation framework with diffusion model. We introduce two key methods: Parts-Aware Re-weighting (PAR) and Parts Consistency Enhancement (PCE). PAR dynamically adjusts regional training loss weights based on pose confidence scores, effectively improving visual quality. PCE constructs and trains diffusion-based regional audio-visual classifiers to improve the consistency of motion and co-speech audio. Afterwards, we design two novel inference guidance methods for the foregoing classifiers, Sequential Guidance (SG) and Differential Guidance (DG), to balance efficiency and quality respectively. Additionally, we build CNAS, the first public Chinese News Anchor Speech dataset, to advance research and validation in this field. Extensive experimental results and user studies demonstrate that PAHA significantly outperforms existing methods in audio-motion alignment and video-related evaluations. The codes and CNAS dataset will be released upon acceptance.
• 摘要：音频驱动的人类动画技术广泛用于人类计算机的互动中，扩散模型的出现进一步推动了其发展。当前，大多数方法都依赖于多阶段的生成和中间表示，从而导致了很长的推理时间以及在特定前景区域的发电质量和音频运动一致性的发电质量。这些缺点主要是由于缺乏局部细粒度监督指导。为了应对上述挑战，我们提出了PAHA，这是一种具有扩散模型的端到端音频驱动的上身动画框架。我们介绍了两种关键方法：零件感知重新加权（PAR）和零件一致性增强（PCE）。基于姿势置信度得分，可以动态调整区域训练减肥权重，从而有效提高视觉质量。 PCE构建和训练基于扩散的区域视听分类器，以提高运动和语音音频的一致性。之后，我们为上述分类器，顺序指导（SG）和差异指导（DG）设计了两种新颖的推理指导方法，以分别平衡效率和质量。此外，我们构建了CNA，这是中国公共新闻主播语音数据集，以推进该领域的研究和验证。广泛的实验结果和用户研究表明，PAHA在音频 - 运动对准和与视频有关的评估中的现有方法显着优于现有方法。接受代码和CNA数据集将在接受后发布。

Title: Bounding Box-Guided Diffusion for Synthesizing Industrial Images and Segmentation Map

• Authors: Alessandro Simoni, Francesco Pelosin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.03623
• Pdf URL: https://arxiv.org/pdf/2505.03623
• Copy Paste: [[2505.03623]] Bounding Box-Guided Diffusion for Synthesizing Industrial Images and Segmentation Map(https://arxiv.org/abs/2505.03623)
• Keywords: generation, generative
• Abstract: Synthetic dataset generation in Computer Vision, particularly for industrial applications, is still underexplored. Industrial defect segmentation, for instance, requires highly accurate labels, yet acquiring such data is costly and time-consuming. To address this challenge, we propose a novel diffusion-based pipeline for generating high-fidelity industrial datasets with minimal supervision. Our approach conditions the diffusion model on enriched bounding box representations to produce precise segmentation masks, ensuring realistic and accurately localized defect synthesis. Compared to existing layout-conditioned generative methods, our approach improves defect consistency and spatial accuracy. We introduce two quantitative metrics to evaluate the effectiveness of our method and assess its impact on a downstream segmentation task trained on real and synthetic data. Our results demonstrate that diffusion-based synthesis can bridge the gap between artificial and real-world industrial data, fostering more reliable and cost-efficient segmentation models. The code is publicly available at this https URL.
• 摘要：计算机视觉中的合成数据集生成，尤其是对于工业应用，仍然没有被忽视。例如，工业缺陷细分需要高度准确的标签，但是获取此类数据是昂贵且耗时的。为了应对这一挑战，我们提出了一条新型的基于扩散的管道，用于以最少的监督生成高保真工业数据集。我们的方法条件在丰富的边界框表示上的扩散模型以产生精确的分割掩模，从而确保现实且准确地局部缺陷合成。与现有的布局条件生成方法相比，我们的方法提高了缺陷的一致性和空间精度。我们介绍了两个定量指标，以评估我们方法的有效性，并评估其对对实际和合成数据训练的下游分割任务的影响。我们的结果表明，基于扩散的合成可以弥合人工和现实世界中的工业数据之间的差距，从而促进了更可靠和具有成本效益的分割模型。该代码在此HTTPS URL上公开可用。

Title: Breaking Annotation Barriers: Generalized Video Quality Assessment via Ranking-based Self-Supervision

• Authors: Linhan Cao, Wei Sun, Kaiwei Zhang, Yicong Peng, Guangtao Zhai, Xiongkuo Min
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.03631
• Pdf URL: https://arxiv.org/pdf/2505.03631
• Copy Paste: [[2505.03631]] Breaking Annotation Barriers: Generalized Video Quality Assessment via Ranking-based Self-Supervision(https://arxiv.org/abs/2505.03631)
• Keywords: quality assessment
• Abstract: Video quality assessment (VQA) is essential for quantifying perceptual quality in various video processing workflows, spanning from camera capture systems to over-the-top streaming platforms. While recent supervised VQA models have made substantial progress, the reliance on manually annotated datasets -- a process that is labor-intensive, costly, and difficult to scale up -- has hindered further optimization of their generalization to unseen video content and distortions. To bridge this gap, we introduce a self-supervised learning framework for VQA to learn quality assessment capabilities from large-scale, unlabeled web videos. Our approach leverages a \textbf{learning-to-rank} paradigm to train a large multimodal model (LMM) on video pairs automatically labeled via two manners, including quality pseudo-labeling by existing VQA models and relative quality ranking based on synthetic distortion simulations. Furthermore, we introduce a novel \textbf{iterative self-improvement training strategy}, where the trained model acts an improved annotator to iteratively refine the annotation quality of training data. By training on a dataset $10\times$ larger than the existing VQA benchmarks, our model: (1) achieves zero-shot performance on in-domain VQA benchmarks that matches or surpasses supervised models; (2) demonstrates superior out-of-distribution (OOD) generalization across diverse video content and distortions; and (3) sets a new state-of-the-art when fine-tuned on human-labeled datasets. Extensive experimental results validate the effectiveness of our self-supervised approach in training generalized VQA models. The datasets and code will be publicly released to facilitate future research.
• 摘要：视频质量评估（VQA）对于量化各种视频处理工作流程中的感知质量至关重要，从相机捕获系统到顶级流媒体平台。尽管最近有监督的VQA模型取得了重大进展，但对手动注释的数据集的依赖（这一过程是劳动密集型，昂贵且难以扩大的过程）妨碍了他们进一步优化其概括以使视频内容和扭曲的扭曲。为了弥合这一差距，我们为VQA引入了一个自我监督的学习框架，以从大规模的，未标记的网络视频中学习质量评估功能。我们的方法利用A \ textbf {学习到秩}范式在视频对上训练大型多模式模型（LMM），该视频对自动通过两种方式标记，包括现有VQA模型和基于合成畸变模拟的相对质量排名的质量伪标记和相对质量排名。此外，我们介绍了一种小说\ textbf {迭代自我改进培训策略}，其中训练有素的模型可以改进注释者，以迭代地完善培训数据的注释质量。通过在数据集中培训$ 10 \ times $ $ $ $比现有的VQA基准测试，我们的型号：（1）在匹配或超过监督模型的内域VQA基准上实现零射击性能； （2）证明了跨不同视频内容和扭曲的出色分布（OOD）概括； （3）在人体标记的数据集上进行微调时，设置了一个新的最先进。广泛的实验结果证明了我们自我监督方法在训练广义VQA模型中的有效性。数据集和代码将公开发布，以促进未来的研究。

Title: Towards Smart Point-and-Shoot Photography

• Authors: Jiawan Li, Fei Zhou, Zhipeng Zhong, Jiongzhi Lin, Guoping Qiu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.03638
• Pdf URL: https://arxiv.org/pdf/2505.03638
• Copy Paste: [[2505.03638]] Towards Smart Point-and-Shoot Photography(https://arxiv.org/abs/2505.03638)
• Keywords: quality assessment
• Abstract: Hundreds of millions of people routinely take photos using their smartphones as point and shoot (PAS) cameras, yet very few would have the photography skills to compose a good shot of a scene. While traditional PAS cameras have built-in functions to ensure a photo is well focused and has the right brightness, they cannot tell the users how to compose the best shot of a scene. In this paper, we present a first of its kind smart point and shoot (SPAS) system to help users to take good photos. Our SPAS proposes to help users to compose a good shot of a scene by automatically guiding the users to adjust the camera pose live on the scene. We first constructed a large dataset containing 320K images with camera pose information from 4000 scenes. We then developed an innovative CLIP-based Composition Quality Assessment (CCQA) model to assign pseudo labels to these images. The CCQA introduces a unique learnable text embedding technique to learn continuous word embeddings capable of discerning subtle visual quality differences in the range covered by five levels of quality description words {bad, poor, fair, good, perfect}. And finally we have developed a camera pose adjustment model (CPAM) which first determines if the current view can be further improved and if so it outputs the adjust suggestion in the form of two camera pose adjustment angles. The two tasks of CPAM make decisions in a sequential manner and each involves different sets of training samples, we have developed a mixture-of-experts model with a gated loss function to train the CPAM in an end-to-end manner. We will present extensive results to demonstrate the performances of our SPAS system using publicly available image composition datasets.
• 摘要：数以千计的人通常会使用智能手机作为点和拍摄摄像机拍照，但很少有人会拥有摄影技巧来构成场景的良好拍摄。尽管传统的PAS摄像机具有内置功能，以确保照片专注并且具有正确的亮度，但他们不能告诉用户如何构成场景的最佳镜头。在本文中，我们提出了同类智能点和拍摄（SPA）系统的第一个，以帮助用户拍摄好照片。我们的水疗中心建议通过自动指导用户调整现场的相机姿势来帮助用户构成场景的良好拍摄。我们首先构建了一个大型数据集，其中包含320k图像，其中包含来自4000个场景的相机姿势信息。然后，我们开发了一种创新的基于夹的组成质量评估（CCQA）模型，以将伪标签分配给这些图像。 CCQA引入了一种独特的可学习文本嵌入技术，以学习能够辨别五个级别质量描述单词覆盖范围内的微妙的视觉质量差异{不好，穷，穷，公平，好，完美}。最后，我们开发了一个相机姿势调整模型（CPAM），该模型首先确定是否可以进一步改进当前视图，并以两个相机姿势调整角的形式输出调整建议。 CPAM的两个任务以依次的方式做出决策，每个任务都涉及不同的训练样本集，我们开发了具有封闭损耗函数的专家模型，以端到端的方式训练CPAM。我们将提出广泛的结果，以使用公开可用的图像组成数据集证明我们的Spas系统的性能。

Title: Distribution-Conditional Generation: From Class Distribution to Creative Generation

• Authors: Fu Feng, Yucheng Xie, Xu Yang, Jing Wang, Xin Geng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.03667
• Pdf URL: https://arxiv.org/pdf/2505.03667
• Copy Paste: [[2505.03667]] Distribution-Conditional Generation: From Class Distribution to Creative Generation(https://arxiv.org/abs/2505.03667)
• Keywords: generation
• Abstract: Text-to-image (T2I) diffusion models are effective at producing semantically aligned images, but their reliance on training data distributions limits their ability to synthesize truly novel, out-of-distribution concepts. Existing methods typically enhance creativity by combining pairs of known concepts, yielding compositions that, while out-of-distribution, remain linguistically describable and bounded within the existing semantic space. Inspired by the soft probabilistic outputs of classifiers on ambiguous inputs, we propose Distribution-Conditional Generation, a novel formulation that models creativity as image synthesis conditioned on class distributions, enabling semantically unconstrained creative generation. Building on this, we propose DisTok, an encoder-decoder framework that maps class distributions into a latent space and decodes them into tokens of creative concept. DisTok maintains a dynamic concept pool and iteratively sampling and fusing concept pairs, enabling the generation of tokens aligned with increasingly complex class distributions. To enforce distributional consistency, latent vectors sampled from a Gaussian prior are decoded into tokens and rendered into images, whose class distributions-predicted by a vision-language model-supervise the alignment between input distributions and the visual semantics of generated tokens. The resulting tokens are added to the concept pool for subsequent composition. Extensive experiments demonstrate that DisTok, by unifying distribution-conditioned fusion and sampling-based synthesis, enables efficient and flexible token-level generation, achieving state-of-the-art performance with superior text-image alignment and human preference scores.
• 摘要：文本对图像（T2I）扩散模型可有效地产生语义对齐的图像，但是它们对训练数据分布的依赖限制了它们综合真正新颖，分布的概念的能力。现有方法通常通过结合已知概念对，产生构图来增强创造力，尽管该概念在现有的语义空间内仍可以在语言上描述和界定。受分类器在模棱两可的输入中的软概率输出的启发，我们提出了分布条件生成，这是一种新颖的公式，将创造力建模为图像合成，以班级分布为条件，从而实现了语义上不受限制的创意生成。在此基础上，我们提出了Distok，这是一个编码器框架，该框架将类分布映射到潜在空间中，并将它们解码为创意概念的代币。 Distok保持动态概念池以及迭代采样和融合概念对，使代币的产生与日益复杂的类别分布对齐。为了实施分布一致性，从高斯先验中采样的潜在矢量被解码为令牌并渲染到图像中，其类别分布由视觉模型模型避免了输入分布与生成代币的视觉语义之间的比对。将产生的令牌添加到概念库中以进行后续组成。广泛的实验表明，通过统一分布条件的融合和基于采样的合成，Distok可以实现有效，灵活的代币级产生，从而通过出色的文本图像对齐和人类的偏好得分实现最先进的性能。