2025-02-27

Title: FilterRAG: Zero-Shot Informed Retrieval-Augmented Generation to Mitigate Hallucinations in VQA

• Authors: S M Sarwar
• Subjects: cs.CV, cs.CL, cs.IR, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.18536
• Pdf URL: https://arxiv.org/pdf/2502.18536
• Copy Paste: [[2502.18536]] FilterRAG: Zero-Shot Informed Retrieval-Augmented Generation to Mitigate Hallucinations in VQA(https://arxiv.org/abs/2502.18536)
• Keywords: generation
• Abstract: Visual Question Answering requires models to generate accurate answers by integrating visual and textual understanding. However, VQA models still struggle with hallucinations, producing convincing but incorrect answers, particularly in knowledge-driven and Out-of-Distribution scenarios. We introduce FilterRAG, a retrieval-augmented framework that combines BLIP-VQA with Retrieval-Augmented Generation to ground answers in external knowledge sources like Wikipedia and DBpedia. FilterRAG achieves 36.5% accuracy on the OK-VQA dataset, demonstrating its effectiveness in reducing hallucinations and improving robustness in both in-domain and Out-of-Distribution settings. These findings highlight the potential of FilterRAG to improve Visual Question Answering systems for real-world deployment.
• 摘要：视觉问题答案需要模型通过集成视觉和文本理解来产生准确的答案。但是，VQA模型仍然在幻觉上挣扎，产生了令人信服但不正确的答案，尤其是在知识驱动和分布的场景中。我们介绍了FilterRag，这是一种检索式框架，将Blip-VQA与检索效果的生成结合在一起，以在Wikipedia和Dbpedia等外部知识来源中的地面答案。 FilterRag在OK-VQA数据集上达到了36.5％的精度，证明了其在减少幻觉和提高域内和分布设置的鲁棒性方面的有效性。这些发现突出了FilterRag的潜力，可以改善现实世界部署的视觉响应答案系统。

Title: Diffusion Models for conditional MRI generation

• Authors: Miguel Herencia García del Castillo, Ricardo Moya Garcia, Manuel Jesús Cerezo Mazón, Ekaitz Arriola Garcia, Pablo Menéndez Fernández-Miranda
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.18620
• Pdf URL: https://arxiv.org/pdf/2502.18620
• Copy Paste: [[2502.18620]] Diffusion Models for conditional MRI generation(https://arxiv.org/abs/2502.18620)
• Keywords: generation
• Abstract: In this article, we present a Latent Diffusion Model (LDM) for the generation of brain Magnetic Resonance Imaging (MRI), conditioning its generation based on pathology (Healthy, Glioblastoma, Sclerosis, Dementia) and acquisition modality (T1w, T1ce, T2w, Flair, PD). To evaluate the quality of the generated images, the Fréchet Inception Distance (FID) and Multi-Scale Structural Similarity Index (MS-SSIM) metrics were employed. The results indicate that the model generates images with a distribution similar to real ones, maintaining a balance between visual fidelity and diversity. Additionally, the model demonstrates extrapolation capability, enabling the generation of configurations that were not present in the training data. The results validate the potential of the model to increase in the number of samples in clinical datasets, balancing underrepresented classes, and evaluating AI models in medicine, contributing to the development of diagnostic tools in radiology without compromising patient privacy.
• 摘要：在本文中，我们提出了一个潜在的扩散模型（LDM），用于生成脑磁共振成像（MRI），根据病理学（健康，胶质母细胞瘤，硬化，痴呆症）和获取方式（T1W，T1CE，T1CE，T1CE，T2W，FLAIR，FLAIR，PD）来调节其生成。为了评估生成的图像的质量，采用了Fréchet成立距离（FID）和多尺度结构相似性指数（MS-SSSIM）指标。结果表明，该模型生成的图像具有类似于真实图像，从而保持视觉保真度和多样性之间的平衡。此外，该模型展示了外推能力，从而使训练数据中不存在的配置产生。结果证明了该模型在临床数据集中增加样品数量，平衡代表性不足的类别以及评估医学中的AI模型的潜力，这有助于开发放射学诊断工具而不损害患者隐私。

Title: Faster, Cheaper, Better: Multi-Objective Hyperparameter Optimization for LLM and RAG Systems

• Authors: Matthew Barker, Andrew Bell, Evan Thomas, James Carr, Thomas Andrews, Umang Bhatt
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.18635
• Pdf URL: https://arxiv.org/pdf/2502.18635
• Copy Paste: [[2502.18635]] Faster, Cheaper, Better: Multi-Objective Hyperparameter Optimization for LLM and RAG Systems(https://arxiv.org/abs/2502.18635)
• Keywords: generation
• Abstract: While Retrieval Augmented Generation (RAG) has emerged as a popular technique for improving Large Language Model (LLM) systems, it introduces a large number of choices, parameters and hyperparameters that must be made or tuned. This includes the LLM, embedding, and ranker models themselves, as well as hyperparameters governing individual RAG components. Yet, collectively optimizing the entire configuration in a RAG or LLM system remains under-explored - especially in multi-objective settings - due to intractably large solution spaces, noisy objective evaluations, and the high cost of evaluations. In this work, we introduce the first approach for multi-objective parameter optimization of cost, latency, safety and alignment over entire LLM and RAG systems. We find that Bayesian optimization methods significantly outperform baseline approaches, obtaining a superior Pareto front on two new RAG benchmark tasks. We conclude our work with important considerations for practitioners who are designing multi-objective RAG systems, highlighting nuances such as how optimal configurations may not generalize across tasks and objectives.
• 摘要：尽管检索增强发电（RAG）已成为改进大型语言模型（LLM）系统的流行技术，但它引入了必须制定或调整的大量选择，参数和超参数。这包括LLM，嵌入和排名模型本身，以及管理单个抹布组件的超参数。但是，由于较大的解决方案空间，嘈杂的客观评估和高评估成本，因此在抹布或LLM系统中总体优化了抹布或LLM系统中的整个配置。在这项工作中，我们介绍了整个LLM和抹布系统的成本，延迟，安全性和对齐方式的多目标参数优化的第一种方法。我们发现，贝叶斯优化方法明显超过了基线方法，在两个新的抹布基准任务上获得了帕累托前锋。我们对设计多目标抹布系统的从业人员进行了重要的考虑，以重要的考虑来结束我们的工作，突出了诸如最佳配置如何无法跨越任务和目标概括之类的细微差别。

Title: M2-omni: Advancing Omni-MLLM for Comprehensive Modality Support with Competitive Performance

• Authors: Qingpei Guo, Kaiyou Song, Zipeng Feng, Ziping Ma, Qinglong Zhang, Sirui Gao, Xuzheng Yu, Yunxiao Sun, Tai-WeiChang, Jingdong Chen, Ming Yang, Jun Zhou
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.18778
• Pdf URL: https://arxiv.org/pdf/2502.18778
• Copy Paste: [[2502.18778]] M2-omni: Advancing Omni-MLLM for Comprehensive Modality Support with Competitive Performance(https://arxiv.org/abs/2502.18778)
• Keywords: generation
• Abstract: We present M2-omni, a cutting-edge, open-source omni-MLLM that achieves competitive performance to GPT-4o. M2-omni employs a unified multimodal sequence modeling framework, which empowers Large Language Models(LLMs) to acquire comprehensive cross-modal understanding and generation capabilities. Specifically, M2-omni can process arbitrary combinations of audio, video, image, and text modalities as input, generating multimodal sequences interleaving with audio, image, or text outputs, thereby enabling an advanced and interactive real-time experience. The training of such an omni-MLLM is challenged by significant disparities in data quantity and convergence rates across modalities. To address these challenges, we propose a step balance strategy during pre-training to handle the quantity disparities in modality-specific data. Additionally, a dynamically adaptive balance strategy is introduced during the instruction tuning stage to synchronize the modality-wise training progress, ensuring optimal convergence. Notably, we prioritize preserving strong performance on pure text tasks to maintain the robustness of M2-omni's language understanding capability throughout the training process. To our best knowledge, M2-omni is currently a very competitive open-source model to GPT-4o, characterized by its comprehensive modality and task support, as well as its exceptional performance. We expect M2-omni will advance the development of omni-MLLMs, thus facilitating future research in this domain.
• 摘要：我们提出了M2-omni，这是一种尖端的开源Omni-Mllm，可实现GPT-4O的竞争性能。 M2-OMNI采用统一的多模式序列建模框架，该框架授权大型语言模型（LLMS）获得综合的跨模式理解和发电能力。具体而言，M2-OMNI可以处理音频，视频，图像和文本模式的任意组合，作为输入，生成与音频，图像或文本输出相互交织的多模式序列，从而启用高级和交互式的实时体验。这种OMNI-MLLM的培训受到跨模式的数据数量和收敛率的显着差异的挑战。为了应对这些挑战，我们在预培训期间提出了一个步骤平衡策略，以处理特定于模态数据中的数量差异。此外，在教学调整阶段中引入了动态自适应平衡策略，以同步模态训练进度，从而确保最佳收敛。值得注意的是，我们优先考虑在纯文本任务上保持强大的绩效，以保持M2-omni语言理解能力在整个培训过程中的稳健性。据我们所知，M2-OMNI目前是GPT-4O的非常有竞争力的开源模型，其特征是其全面的方式和任务支持及其出色的性能。我们预计M2-OMNI将推进Omni-Mllms的发展，从而促进该领域的未来研究。

Title: Optimal Stochastic Trace Estimation in Generative Modeling

• Authors: Xinyang Liu, Hengrong Du, Wei Deng, Ruqi Zhang
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.18808
• Pdf URL: https://arxiv.org/pdf/2502.18808
• Copy Paste: [[2502.18808]] Optimal Stochastic Trace Estimation in Generative Modeling(https://arxiv.org/abs/2502.18808)
• Keywords: generation, generative
• Abstract: Hutchinson estimators are widely employed in training divergence-based likelihoods for diffusion models to ensure optimal transport (OT) properties. However, this estimator often suffers from high variance and scalability concerns. To address these challenges, we investigate Hutch++, an optimal stochastic trace estimator for generative models, designed to minimize training variance while maintaining transport optimality. Hutch++ is particularly effective for handling ill-conditioned matrices with large condition numbers, which commonly arise when high-dimensional data exhibits a low-dimensional structure. To mitigate the need for frequent and costly QR decompositions, we propose practical schemes that balance frequency and accuracy, backed by theoretical guarantees. Our analysis demonstrates that Hutch++ leads to generations of higher quality. Furthermore, this method exhibits effective variance reduction in various applications, including simulations, conditional time series forecasts, and image generation.
• 摘要：Hutchinson估计量广泛用于基于差异模型的基于差异的可能性，以确保最佳运输（OT）特性。但是，该估计器通常遭受较高的差异和可伸缩性问题。为了应对这些挑战，我们研究了Hutch ++是生成模型的最佳随机痕量估计器，旨在最大程度地减少训练方差，同时保持运输最佳。 Hutch ++对于处理具有较大条件数量的条件矩阵特别有效，当高维数据表现出低维结构时，通常会出现。为了减轻对频繁且昂贵的QR分解的需求，我们提出了实用方案，以平衡频率和准确性，并得到理论保证的支持。我们的分析表明，Hutch ++导致了几代人的质量。此外，该方法在各种应用中表现出有效的方差降低，包括模拟，有条件的时间序列预测和图像产生。

Title: A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loops

• Authors: Shi Fu, Yingjie Wang, Yuzhu Chen, Xinmei Tian, Dacheng Tao
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.18865
• Pdf URL: https://arxiv.org/pdf/2502.18865
• Copy Paste: [[2502.18865]] A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loops(https://arxiv.org/abs/2502.18865)
• Keywords: generative
• Abstract: High-quality data is essential for training large generative models, yet the vast reservoir of real data available online has become nearly depleted. Consequently, models increasingly generate their own data for further training, forming Self-consuming Training Loops (STLs). However, the empirical results have been strikingly inconsistent: some models degrade or even collapse, while others successfully avoid these failures, leaving a significant gap in theoretical understanding to explain this discrepancy. This paper introduces the intriguing notion of recursive stability and presents the first theoretical generalization analysis, revealing how both model architecture and the proportion between real and synthetic data influence the success of STLs. We further extend this analysis to transformers in in-context learning, showing that even a constant-sized proportion of real data ensures convergence, while also providing insights into optimal synthetic data sizing.
• 摘要：高质量的数据对于培训大型生成模型至关重要，但是在线可用的大量真实数据库已几乎耗尽。因此，模型越来越多地生成自己的数据以进行进一步的培训，形成自我消耗的训练循环（STLS）。但是，经验结果显着不一致：有些模型退化甚至崩溃，而另一些模型成功地避免了这些失败，从而在理论上理解中留下了很大的差距，以解释这种差异。本文介绍了递归稳定性的有趣概念，并提出了第一个理论概括分析，揭示了模型架构与真实数据和合成数据之间的比例如何影响STL的成功。我们进一步将此分析扩展到了在文本学习中的变压器，表明即使是恒定的实际数据比例也可以确保收敛，同时还提供了对最佳合成数据大小的见解。

Title: A Dual-Purpose Framework for Backdoor Defense and Backdoor Amplification in Diffusion Models

• Authors: Vu Tuan Truong Long, Bao Le
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19047
• Pdf URL: https://arxiv.org/pdf/2502.19047
• Copy Paste: [[2502.19047]] A Dual-Purpose Framework for Backdoor Defense and Backdoor Amplification in Diffusion Models(https://arxiv.org/abs/2502.19047)
• Keywords: generative
• Abstract: Diffusion models have emerged as state-of-the-art generative frameworks, excelling in producing high-quality multi-modal samples. However, recent studies have revealed their vulnerability to backdoor attacks, where backdoored models generate specific, undesirable outputs called backdoor target (e.g., harmful images) when a pre-defined trigger is embedded to their inputs. In this paper, we propose PureDiffusion, a dual-purpose framework that simultaneously serves two contrasting roles: backdoor defense and backdoor attack amplification. For defense, we introduce two novel loss functions to invert backdoor triggers embedded in diffusion models. The first leverages trigger-induced distribution shifts across multiple timesteps of the diffusion process, while the second exploits the denoising consistency effect when a backdoor is activated. Once an accurate trigger inversion is achieved, we develop a backdoor detection method that analyzes both the inverted trigger and the generated backdoor targets to identify backdoor attacks. In terms of attack amplification with the role of an attacker, we describe how our trigger inversion algorithm can be used to reinforce the original trigger embedded in the backdoored diffusion model. This significantly boosts attack performance while reducing the required backdoor training time. Experimental results demonstrate that PureDiffusion achieves near-perfect detection accuracy, outperforming existing defenses by a large margin, particularly against complex trigger patterns. Additionally, in an attack scenario, our attack amplification approach elevates the attack success rate (ASR) of existing backdoor attacks to nearly 100\% while reducing training time by up to 20x.
• 摘要：扩散模型已成为最先进的生成框架，在生产高质量的多模式样本方面表现出色。但是，最近的研究揭示了它们对后门攻击的脆弱性，当将预定义的触发器嵌入到其输入中时，后门模型会产生特定的，不受欢迎的输出称为后门目标（例如，有害图像）。在本文中，我们提出了纯净的填充，这是一个双重用途框架，同时扮演着两个对比角色：后门防御和后门攻击放大。为了防御，我们引入了两个新型损失功能，以倒入嵌入扩散模型中的后门触发器。第一个利用触发诱导的分布在扩散过程的多个时间步上移动，而第二个则在激活后门时利用了去索的一致性效果。一旦实现了准确的触发反转，我们就开发了一种后门检测方法，该方法分析了倒置触发器和生成的后门目标，以识别后门攻击。就攻击者的角色而言，我们描述了如何使用触发反转算法来加强嵌入在后置扩散模型中的原始触发器。这大大提高了攻击性能，同时减少了所需的后门训练时间。实验结果表明，纯化源达到接近完美的检测准确性，超过了现有的防御能力，尤其是针对复杂的触发模式。此外，在攻击情况下，我们的攻击放大方法将现有后门攻击的攻击成功率（ASR）提高到近100 \％，同时将训练时间降低到20倍。

Title: Dynamic Degradation Decomposition Network for All-in-One Image Restoration

• Authors: Huiqiang Wang, Mingchen Song, Guoqiang Zhong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19068
• Pdf URL: https://arxiv.org/pdf/2502.19068
• Copy Paste: [[2502.19068]] Dynamic Degradation Decomposition Network for All-in-One Image Restoration(https://arxiv.org/abs/2502.19068)
• Keywords: restoration
• Abstract: Currently, restoring clean images from a variety of degradation types using a single model is still a challenging task. Existing all-in-one image restoration approaches struggle with addressing complex and ambiguously defined degradation types. In this paper, we introduce a dynamic degradation decomposition network for all-in-one image restoration, named D$^3$Net. D$^3$Net achieves degradation-adaptive image restoration with guided prompt through cross-domain interaction and dynamic degradation decomposition. Concretely, in D$^3$Net, the proposed Cross-Domain Degradation Analyzer (CDDA) engages in deep interaction between frequency domain degradation characteristics and spatial domain image features to identify and model variations of different degradation types on the image manifold, generating degradation correction prompt and strategy prompt, which guide the following decomposition process. Furthermore, the prompt-based Dynamic Decomposition Mechanism (DDM) for progressive degradation decomposition, that encourages the network to adaptively select restoration strategies utilizing the two-level prompt generated by CDDA. Thanks to the synergistic cooperation between CDDA and DDM, D$^3$Net achieves superior flexibility and scalability in handling unknown degradation, while effectively reducing unnecessary computational overhead. Extensive experiments on multiple image restoration tasks demonstrate that D$^3$Net significantly outperforms the state-of-the-art approaches, especially improving PSNR by 5.47dB and 3.30dB on the SOTS-Outdoor and GoPro datasets, respectively.
• 摘要：当前，使用单个模型从各种降解类型中恢复干净的图像仍然是一项具有挑战性的任务。现有的多合一图像恢复方法努力解决复杂且模棱两可的降解类型。在本文中，我们引入了一个动态降解分解网络，用于多合一图像修复，名为D $^3 $ net。 D $^3 $ NET通过跨域交互和动态降解分解，通过引导提示来实现降解自动图像恢复。具体而言，在d $^3 $ net中，提议的跨域退化分析仪（CDDA）在频域降解特性和空间域图像特征之间进行了深入的相互作用，以识别和模拟图像流形的不同降解类型的变化，从而产生降级校正提示和策略提示和下面的解除分解过程。此外，用于进行性降解分解的基于及时的动态分解机制（DDM），它鼓励网络利用CDDA生成的两级提示来适应选择恢复策略。得益于CDDA和DDM之间的协同合作，D $^3 $ NET在处理未知降解方面具有出色的灵活性和可扩展性，同时有效地减少了不必要的计算开销。关于多个图像恢复任务的广泛实验表明，d $^3 $净的净值明显优于最先进的方法，尤其是在Sots-Outdoor和GoPro数据集上分别将PSNR和3.30dB改进。

Title: A Sample-Level Evaluation and Generative Framework for Model Inversion Attacks

• Authors: Haoyang Li, Li Bai, Qingqing Ye, Haibo Hu, Yaxin Xiao, Huadi Zheng, Jianliang Xu
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2502.19070
• Pdf URL: https://arxiv.org/pdf/2502.19070
• Copy Paste: [[2502.19070]] A Sample-Level Evaluation and Generative Framework for Model Inversion Attacks(https://arxiv.org/abs/2502.19070)
• Keywords: generative
• Abstract: Model Inversion (MI) attacks, which reconstruct the training dataset of neural networks, pose significant privacy concerns in machine learning. Recent MI attacks have managed to reconstruct realistic label-level private data, such as the general appearance of a target person from all training images labeled on him. Beyond label-level privacy, in this paper we show sample-level privacy, the private information of a single target sample, is also important but under-explored in the MI literature due to the limitations of existing evaluation metrics. To address this gap, this study introduces a novel metric tailored for training-sample analysis, namely, the Diversity and Distance Composite Score (DDCS), which evaluates the reconstruction fidelity of each training sample by encompassing various MI attack attributes. This, in turn, enhances the precision of sample-level privacy assessments. Leveraging DDCS as a new evaluative lens, we observe that many training samples remain resilient against even the most advanced MI attack. As such, we further propose a transfer learning framework that augments the generative capabilities of MI attackers through the integration of entropy loss and natural gradient descent. Extensive experiments verify the effectiveness of our framework on improving state-of-the-art MI attacks over various metrics including DDCS, coverage and FID. Finally, we demonstrate that DDCS can also be useful for MI defense, by identifying samples susceptible to MI attacks in an unsupervised manner.
• 摘要：重建神经网络训练数据集的模型反转（MI）攻击在机器学习中构成了严重的隐私问题。最近的MI攻击已设法重建了现实的标签级私人数据，例如，来自所有标记在他身上的所有培训图像中的目标人的一般外观。除了标签级别的隐私之外，在本文中，我们显示了单个目标样本的私人信息样本级隐私，但由于现有评估指标的局限性，在MI文献中也很重要。为了解决这一差距，这项研究介绍了一种针对训练样本分析的新型度量，即多样性和距离综合评分（DDC），该指标通过包含各种MI攻击属性来评估每个训练样本的重建保真度。反过来，这提高了样本级别的隐私评估的精度。利用DDC作为一种新的评估镜头，我们观察到，即使是最先进的MI攻击，许多培训样本仍然具有弹性。因此，我们进一步提出了一个转移学习框架，该框架通过整合熵损失和自然梯度下降来增强MI攻击者的生成能力。广泛的实验验证了我们框架对改善各种指标（包括DDC，覆盖范围和FID）的最新MI攻击的有效性。最后，我们证明，通过以无监督的方式识别容易受到MI攻击的样品，DDC也对MI防御也很有用。

Title: A Survey on Foundation-Model-Based Industrial Defect Detection

• Authors: Tianle Yang, Luyao Chang, Jiadong Yan, Juntao Li, Zhi Wang, Ke Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19106
• Pdf URL: https://arxiv.org/pdf/2502.19106
• Copy Paste: [[2502.19106]] A Survey on Foundation-Model-Based Industrial Defect Detection(https://arxiv.org/abs/2502.19106)
• Keywords: generation
• Abstract: As industrial products become abundant and sophisticated, visual industrial defect detection receives much attention, including two-dimensional and three-dimensional visual feature modeling. Traditional methods use statistical analysis, abnormal data synthesis modeling, and generation-based models to separate product defect features and complete defect detection. Recently, the emergence of foundation models has brought visual and textual semantic prior knowledge. Many methods are based on foundation models (FM) to improve the accuracy of detection, but at the same time, increase model complexity and slow down inference speed. Some FM-based methods have begun to explore lightweight modeling ways, which have gradually attracted attention and deserve to be systematically analyzed. In this paper, we conduct a systematic survey with comparisons and discussions of foundation model methods from different aspects and briefly review non-foundation model (NFM) methods recently published. Furthermore, we discuss the differences between FM and NFM methods from training objectives, model structure and scale, model performance, and potential directions for future exploration. Through comparison, we find FM methods are more suitable for few-shot and zero-shot learning, which are more in line with actual industrial application scenarios and worthy of in-depth research.
• 摘要：随着工业产品变得丰富而精致，视觉工业缺陷检测受到了很多关注，包括二维和三维视觉特征建模。传统方法使用统计分析，异常数据合成模型和基于生成的模型来分开产品缺陷特征和完全缺陷检测。最近，基础模型的出现带来了视觉和文本语义的先验知识。许多方法基于基础模型（FM）来提高检测的准确性，但同时，提高了模型的复杂性并减慢推理速度。一些基于FM的方法已经开始探索轻巧的建模方法，这些方法逐渐吸引了人们的注意力并应进行系统的分析。在本文中，我们对来自不同方面的基础模型方法进行了比较和讨论进行了系统的调查，并简要回顾了最近发布的非基础模型（NFM）方法。此外，我们讨论了FM和NFM方法之间的差异，训练目标，模型结构和规模，模型性能以及未来探索的潜在方向。通过比较，我们发现FM方法更适合于几次射击和零照片学习，这更符合实际的工业应用方案，值得进行深入研究。

Title: SCA3D: Enhancing Cross-modal 3D Retrieval via 3D Shape and Caption Paired Data Augmentation

• Authors: Junlong Ren, Hao Wu, Hui Xiong, Hao Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19128
• Pdf URL: https://arxiv.org/pdf/2502.19128
• Copy Paste: [[2502.19128]] SCA3D: Enhancing Cross-modal 3D Retrieval via 3D Shape and Caption Paired Data Augmentation(https://arxiv.org/abs/2502.19128)
• Keywords: generation
• Abstract: The cross-modal 3D retrieval task aims to achieve mutual matching between text descriptions and 3D shapes. This has the potential to enhance the interaction between natural language and the 3D environment, especially within the realms of robotics and embodied artificial intelligence (AI) applications. However, the scarcity and expensiveness of 3D data constrain the performance of existing cross-modal 3D retrieval methods. These methods heavily rely on features derived from the limited number of 3D shapes, resulting in poor generalization ability across diverse scenarios. To address this challenge, we introduce SCA3D, a novel 3D shape and caption online data augmentation method for cross-modal 3D retrieval. Our approach uses the LLaVA model to create a component library, captioning each segmented part of every 3D shape within the dataset. Notably, it facilitates the generation of extensive new 3D-text pairs containing new semantic features. We employ both inter and intra distances to align various components into a new 3D shape, ensuring that the components do not overlap and are closely fitted. Further, text templates are utilized to process the captions of each component and generate new text descriptions. Besides, we use unimodal encoders to extract embeddings for 3D shapes and texts based on the enriched dataset. We then calculate fine-grained cross-modal similarity using Earth Mover's Distance (EMD) and enhance cross-modal matching with contrastive learning, enabling bidirectional retrieval between texts and 3D shapes. Extensive experiments show our SCA3D outperforms previous works on the Text2Shape dataset, raising the Shape-to-Text RR@1 score from 20.03 to 27.22 and the Text-to-Shape RR@1 score from 13.12 to 16.67. Codes can be found in this https URL.
• 摘要：跨模式3D检索任务旨在实现文本描述和3D形状之间的相互匹配。这有可能增强自然语言与3D环境之间的相互作用，尤其是在机器人技术和体现人工智能（AI）应用的领域。但是，3D数据的稀缺性和耗时限制了现有的跨模式3D检索方法的性能。这些方法在很大程度上依赖于从有限数量的3D形状获得的特征，从而导致各种情况的泛化能力差。为了应对这一挑战，我们介绍了SCA3D，这是一种新颖的3D形状和字幕在线数据增强方法，用于跨模式3D检索。我们的方法使用LLAVA模型创建一个组件库，并为数据集中的每个3D形状的每个分段部分加上字幕。值得注意的是，它促进了包含新的语义特征的广泛的新3D文本对。我们使用间和内部距离将各种组件对齐为新的3D形状，以确保组件不会重叠并紧密拟合。此外，使用文本模板来处理每个组件的标题并生成新的文本描述。此外，我们使用单模式编码器根据富集的数据集提取3D形状和文本的嵌入。然后，我们使用Earth Mover的距离（EMD）来计算细粒度的跨模式相似性，并通过对比度学习增强交叉模式匹配，从而在文本和3D形状之间进行双向检索。广泛的实验表明，我们的SCA3D在Text2Shape数据集上的表现优于先前的作品，从而将形状到文本RR@1分数从20.03提高到27.22，而文本对形状RR@1得分从13.12到16.67。代码可以在此HTTPS URL中找到。

Title: A Model-Centric Review of Deep Learning for Protein Design

• Authors: Gregory W. Kyro, Tianyin Qiu, Victor S. Batista
• Subjects: cs.LG, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2502.19173
• Pdf URL: https://arxiv.org/pdf/2502.19173
• Copy Paste: [[2502.19173]] A Model-Centric Review of Deep Learning for Protein Design(https://arxiv.org/abs/2502.19173)
• Keywords: generation, generative
• Abstract: Deep learning has transformed protein design, enabling accurate structure prediction, sequence optimization, and de novo protein generation. Advances in single-chain protein structure prediction via AlphaFold2, RoseTTAFold, ESMFold, and others have achieved near-experimental accuracy, inspiring successive work extended to biomolecular complexes via AlphaFold Multimer, RoseTTAFold All-Atom, AlphaFold 3, Chai-1, Boltz-1 and others. Generative models such as ProtGPT2, ProteinMPNN, and RFdiffusion have enabled sequence and backbone design beyond natural evolution-based limitations. More recently, joint sequence-structure co-design models, including ESM3, have integrated both modalities into a unified framework, resulting in improved designability. Despite these advances, challenges still exist pertaining to modeling sequence-structure-function relationships and ensuring robust generalization beyond the regions of protein space spanned by the training data. Future advances will likely focus on joint sequence-structure-function co-design frameworks that are able to model the fitness landscape more effectively than models that treat these modalities independently. Current capabilities, coupled with the dizzying rate of progress, suggest that the field will soon enable rapid, rational design of proteins with tailored structures and functions that transcend the limitations imposed by natural evolution. In this review, we discuss the current capabilities of deep learning methods for protein design, focusing on some of the most revolutionary and capable models with respect to their functionality and the applications that they enable, leading up to the current challenges of the field and the optimal path forward.
• 摘要：深度学习改变了蛋白质设计，从而实现了准确的结构预测，序列优化和从头蛋白质的产生。通过Alphafold2，Rosettafold，Esmfold等人的单链蛋白结构的进步已经达到了近乎实验的准确性，激发了连续的工作，通过Alphafold Multimer，Rosettafold All-Altom，Alphafold 3，Alphafold 3，Chai-1，Boltz-1和其他。 Protgpt2，ProteinMPNN和RFDiffusion等生成模型已超越基于自然进化的限制，使序列和骨干设计超出了序列。最近，包括ESM3在内的联合序列结构共同设计模型已将两种模式集成到统一的框架中，从而提高了设计性。尽管有这些进展，但仍存在与建模序列结构函数关系有关的挑战，并确保超出训练数据所跨越的蛋白质空间区域的稳健概括。与独立处理这些模式的模型相比，未来的进步可能会集中在能够更有效地对健身景观建模的联合序列结构共同设计框架上。当前的能力，再加上令人眼花progration乱的进度率，表明该领域将很快实现具有量身定制的结构和功能的蛋白质快速，合理的设计，这些蛋白质超越了自然进化所施加的局限性。在这篇综述中，我们讨论了蛋白质设计深度学习方法的当前功能，重点介绍了其功能性最具革命性和最有能力的模型，以及它们启用的应用，导致了该领域的当前挑战和最佳路径。

Title: INFO-SEDD: Continuous Time Markov Chains as Scalable Information Metrics Estimators

• Authors: Alberto Foresti, Giulio Franzese, Pietro Michiardi
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19183
• Pdf URL: https://arxiv.org/pdf/2502.19183
• Copy Paste: [[2502.19183]] INFO-SEDD: Continuous Time Markov Chains as Scalable Information Metrics Estimators(https://arxiv.org/abs/2502.19183)
• Keywords: generative
• Abstract: Information-theoretic quantities play a crucial role in understanding non-linear relationships between random variables and are widely used across scientific disciplines. However, estimating these quantities remains an open problem, particularly in the case of high-dimensional discrete distributions. Current approaches typically rely on embedding discrete data into a continuous space and applying neural estimators originally designed for continuous distributions, a process that may not fully capture the discrete nature of the underlying data. We consider Continuous-Time Markov Chains (CTMCs), stochastic processes on discrete state-spaces which have gained popularity due to their generative modeling applications. In this work, we introduce INFO-SEDD, a novel method for estimating information-theoretic quantities of discrete data, including mutual information and entropy. Our approach requires the training of a single parametric model, offering significant computational and memory advantages. Additionally, it seamlessly integrates with pretrained networks, allowing for efficient reuse of pretrained generative models. To evaluate our approach, we construct a challenging synthetic benchmark. Our experiments demonstrate that INFO-SEDD is robust and outperforms neural competitors that rely on embedding techniques. Moreover, we validate our method on a real-world task: estimating the entropy of an Ising model. Overall, INFO-SEDD outperforms competing methods and shows scalability to high-dimensional scenarios, paving the way for new applications where estimating MI between discrete distribution is the focus. The promising results in this complex, high-dimensional scenario highlight INFO-SEDD as a powerful new estimator in the toolkit for information-theoretical analysis.
• 摘要：信息理论在理解随机变量之间的非线性关系中起着至关重要的作用，并且在科学学科中广泛使用。但是，估计这些数量仍然是一个空旷的问题，尤其是在高维离散分布的情况下。当前的方法通常依靠将离散数据嵌入到连续空间中，并应用最初为连续分布设计的神经估计器，该过程可能无法完全捕获基础数据的离散性质。我们考虑了连续的马尔可夫连锁链（CTMC），即离散状态空间上的随机过程，这些过程由于其生成建模应用而获得了流行。在这项工作中，我们介绍了Info-Sedd，这是一种估计信息理论数量离散数据的新方法，包括共同信息和熵。我们的方法需要培训单个参数模型，并提供显着的计算和内存优势。此外，它与预处理的网络无缝集成，从而有效地重复使用了预定的生成模型。为了评估我们的方法，我们构建了一个具有挑战性的合成基准。我们的实验表明，Info-Sedd是强大的，并且优于依赖嵌入技术的神经竞争者。此外，我们在现实世界任务上验证了我们的方法：估计ISING模型的熵。总体而言，Info-Sedd优于竞争方法，并显示出对高维场景的可扩展性，为新应用程序铺平了道路，在新应用程序中估算离散分布之间的MI是重点。在这个复杂，高维的场景中，有希望的结果将信息塞德（Info-Sedd）作为工具包中有力的新估计器，用于信息理论分析。

Title: Self-supervised conformal prediction for uncertainty quantification in Poisson imaging problems

• Authors: Bernardin Tamo Amougou, Marcelo Pereyra, Barbara Pascal
• Subjects: cs.CV, eess.IV, eess.SP
• Abstract URL: https://arxiv.org/abs/2502.19194
• Pdf URL: https://arxiv.org/pdf/2502.19194
• Copy Paste: [[2502.19194]] Self-supervised conformal prediction for uncertainty quantification in Poisson imaging problems(https://arxiv.org/abs/2502.19194)
• Keywords: restoration
• Abstract: Image restoration problems are often ill-posed, leading to significant uncertainty in reconstructed images. Accurately quantifying this uncertainty is essential for the reliable interpretation of reconstructed images. However, image restoration methods often lack uncertainty quantification capabilities. Conformal prediction offers a rigorous framework to augment image restoration methods with accurate uncertainty quantification estimates, but it typically requires abundant ground truth data for calibration. This paper presents a self-supervised conformal prediction method for Poisson imaging problems which leverages Poisson Unbiased Risk Estimator to eliminate the need for ground truth data. The resulting self-calibrating conformal prediction approach is applicable to any Poisson linear imaging problem that is ill-conditioned, and is particularly effective when combined with modern self-supervised image restoration techniques trained directly on measurement data. The proposed method is demonstrated through numerical experiments on image denoising and deblurring; its performance are comparable to supervised conformal prediction methods relying on ground truth data.
• 摘要：图像恢复问题通常是错误的，导致重建图像的明显不确定性。准确地量化这种不确定性对于对重建图像的可靠解释至关重要。但是，图像恢复方法通常缺乏不确定性定量功能。共形预测提供了一个严格的框架，可通过准确的不确定性量化估计来增强图像恢复方法，但通常需要大量的地面真相数据进行校准。本文提出了一种用于泊松成像问题的自我监督的保形预测方法，该方法利用泊松无偏见的风险估计器来消除对地面真相数据的需求。由此产生的自我校准的保形预测方法适用于任何条件不足的泊松线性成像问题，并且与直接在测量数据上训练的现代自我监督图像恢复技术结合使用时尤其有效。通过对图像denoising和Deblurring进行的数值实验证明了所提出的方法。它的性能与依靠地面真相数据的监督共形预测方法相媲美。

Title: HDM: Hybrid Diffusion Model for Unified Image Anomaly Detection

• Authors: Zekang Weng, Jinjin Shi, Jinwei Wang, Zeming Han
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19200
• Pdf URL: https://arxiv.org/pdf/2502.19200
• Copy Paste: [[2502.19200]] HDM: Hybrid Diffusion Model for Unified Image Anomaly Detection(https://arxiv.org/abs/2502.19200)
• Keywords: generation
• Abstract: Image anomaly detection plays a vital role in applications such as industrial quality inspection and medical imaging, where it directly contributes to improving product quality and system reliability. However, existing methods often struggle with complex and diverse anomaly patterns. In particular, the separation between generation and discrimination tasks limits the effective coordination between anomaly sample generation and anomaly region detection. To address these challenges, we propose a novel hybrid diffusion model (HDM) that integrates generation and discrimination into a unified framework. The model consists of three key modules: the Diffusion Anomaly Generation Module (DAGM), the Diffusion Discriminative Module (DDM), and the Probability Optimization Module (POM). DAGM generates realistic and diverse anomaly samples, improving their representativeness. DDM then applies a reverse diffusion process to capture the differences between generated and normal samples, enabling precise anomaly region detection and localization based on probability distributions. POM refines the probability distributions during both the generation and discrimination phases, ensuring high-quality samples are used for training. Extensive experiments on multiple industrial image datasets demonstrate that our method outperforms state-of-the-art approaches, significantly improving both image-level and pixel-level anomaly detection performance, as measured by AUROC.
• 摘要：图像异常检测在诸如工业质量检查和医学成像等应用中起着至关重要的作用，在该应用中，它直接有助于提高产品质量和系统可靠性。但是，现有的方法通常在复杂而多样的异常模式中遇到困难。特别是，产生和歧视任务之间的分离限制了样品产生和异常区域检测之间的有效协调。为了应对这些挑战，我们提出了一种新型的混合扩散模型（HDM），该模型将产生和歧视整合到统一的框架中。该模型由三个关键模块组成：扩散异常生成模块（DAGM），扩散区分模块（DDM）和概率优化模块（POM）。 DAGM生成了现实和多样化的异常样品，从而提高了它们的代表性。然后，DDM应用一个反向扩散过程来捕获生成的样品和正常样品之间的差异，从而根据概率分布实现精确的异常区域检测和定位。 POM在发电阶段和歧视阶段都优化了概率分布，以确保将高质量的样本用于训练。在多个工业图像数据集上进行的广泛实验表明，我们的方法的表现要优于最先进的方法，从而显着改善了图像级和像素级异常检测性能，如AUROC所测量。

Title: ProxyTransformation: Preshaping Point Cloud Manifold With Proxy Attention For 3D Visual Grounding

• Authors: Qihang Peng, Henry Zheng, Gao Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19247
• Pdf URL: https://arxiv.org/pdf/2502.19247
• Copy Paste: [[2502.19247]] ProxyTransformation: Preshaping Point Cloud Manifold With Proxy Attention For 3D Visual Grounding(https://arxiv.org/abs/2502.19247)
• Keywords: generation
• Abstract: Embodied intelligence requires agents to interact with 3D environments in real time based on language instructions. A foundational task in this domain is ego-centric 3D visual grounding. However, the point clouds rendered from RGB-D images retain a large amount of redundant background data and inherent noise, both of which can interfere with the manifold structure of the target regions. Existing point cloud enhancement methods often require a tedious process to improve the manifold, which is not suitable for real-time tasks. We propose Proxy Transformation suitable for multimodal task to efficiently improve the point cloud manifold. Our method first leverages Deformable Point Clustering to identify the point cloud sub-manifolds in target regions. Then, we propose a Proxy Attention module that utilizes multimodal proxies to guide point cloud transformation. Built upon Proxy Attention, we design a submanifold transformation generation module where textual information globally guides translation vectors for different submanifolds, optimizing relative spatial relationships of target regions. Simultaneously, image information guides linear transformations within each submanifold, refining the local point cloud manifold of target regions. Extensive experiments demonstrate that Proxy Transformation significantly outperforms all existing methods, achieving an impressive improvement of 7.49% on easy targets and 4.60% on hard targets, while reducing the computational overhead of attention blocks by 40.6%. These results establish a new SOTA in ego-centric 3D visual grounding, showcasing the effectiveness and robustness of our approach.
• 摘要：体现的智能要求代理根据语言说明实时与3D环境进行交互。该领域的基本任务是以自我为中心的3D视觉接地。但是，从RGB-D图像呈现的点云保留了大量的冗余背景数据和固有的噪声，这两者都可以干扰目标区域的歧管结构。现有的点云增强方法通常需要一个乏味的过程来改善流形，这不适合实时任务。我们提出适用于多模式任务的代理转换，以有效地改善点云流形。我们的方法首先利用可变形点聚类来识别目标区域中的点云子序列。然后，我们提出了一个使用多模式代理来指导点云转换的代理注意模块。基于代理人的关注，我们设计了一个子手机转换生成模块，其中文本信息全球指导了不同子手机的翻译向量，从而优化了目标区域的相对空间关系。同时，图像信息指导每个子手法中的线性变换，从而完善了目标区域的局部点云流形。广泛的实验表明，代理转换显着胜过所有现有方法，在易于目标方面取得了令人印象深刻的提高7.49％，而硬目标的4.60％，同时将注意力阻滞的计算开销降低了40.6％。这些结果在以自我为中心的3D视觉接地中建立了新的SOTA，展示了我们方法的有效性和鲁棒性。

Title: Efficient Federated Search for Retrieval-Augmented Generation

• Authors: Rachid Guerraoui, Anne-Marie Kermarrec, Diana Petrescu, Rafael Pires, Mathis Randl, Martijn de Vos
• Subjects: cs.LG, cs.DC, cs.IR
• Abstract URL: https://arxiv.org/abs/2502.19280
• Pdf URL: https://arxiv.org/pdf/2502.19280
• Copy Paste: [[2502.19280]] Efficient Federated Search for Retrieval-Augmented Generation(https://arxiv.org/abs/2502.19280)
• Keywords: generation
• Abstract: Large language models (LLMs) have demonstrated remarkable capabilities across various domains but remain susceptible to hallucinations and inconsistencies, limiting their reliability. Retrieval-augmented generation (RAG) mitigates these issues by grounding model responses in external knowledge sources. Existing RAG workflows often leverage a single vector database, which is impractical in the common setting where information is distributed across multiple repositories. We introduce RAGRoute, a novel mechanism for federated RAG search. RAGRoute dynamically selects relevant data sources at query time using a lightweight neural network classifier. By not querying every data source, this approach significantly reduces query overhead, improves retrieval efficiency, and minimizes the retrieval of irrelevant information. We evaluate RAGRoute using the MIRAGE and MMLU benchmarks and demonstrate its effectiveness in retrieving relevant documents while reducing the number of queries. RAGRoute reduces the total number of queries up to 77.5% and communication volume up to 76.2%.
• 摘要：大型语言模型（LLM）表现出各个领域的出色功能，但仍容易受到幻觉和矛盾的影响，从而限制了它们的可靠性。检索授权的生成（RAG）通过基于外部知识来源中的模型响应来减轻这些问题。现有的抹布工作流通常利用单个矢量数据库，这在信息分布在多个存储库的公共环境中是不切实际的。我们介绍了Ragroute，这是一种用于联合抹布搜索的新型机制。 RAGROUTE使用轻型神经网络分类器在查询时间动态选择相关的数据源。通过不查询每个数据源，这种方法可大大降低查询开销，提高检索效率，并最大程度地减少无关信息的检索。我们使用MIRAGE和MMLU基准评估了Ragroute，并在减少查询数量的同时，证明了其在检索相关文档方面的有效性。 ragroute将查询总数减少到77.5％，通信量最高为76.2％。

Title: On the Importance of Text Preprocessing for Multimodal Representation Learning and Pathology Report Generation

• Authors: Ruben T. Lucassen, Tijn van de Luijtgaarden, Sander P.J. Moonemans, Gerben E. Breimer, Willeke A.M. Blokx, Mitko Veta
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19285
• Pdf URL: https://arxiv.org/pdf/2502.19285
• Copy Paste: [[2502.19285]] On the Importance of Text Preprocessing for Multimodal Representation Learning and Pathology Report Generation(https://arxiv.org/abs/2502.19285)
• Keywords: generation
• Abstract: Vision-language models in pathology enable multimodal case retrieval and automated report generation. Many of the models developed so far, however, have been trained on pathology reports that include information which cannot be inferred from paired whole slide images (e.g., patient history), potentially leading to hallucinated sentences in generated reports. To this end, we investigate how the selection of information from pathology reports for vision-language modeling affects the quality of the multimodal representations and generated reports. More concretely, we compare a model trained on full reports against a model trained on preprocessed reports that only include sentences describing the cell and tissue appearances based on the H&E-stained slides. For the experiments, we built upon the BLIP-2 framework and used a cutaneous melanocytic lesion dataset of 42,433 H&E-stained whole slide images and 19,636 corresponding pathology reports. Model performance was assessed using image-to-text and text-to-image retrieval, as well as qualitative evaluation of the generated reports by an expert pathologist. Our results demonstrate that text preprocessing prevents hallucination in report generation. Despite the improvement in the quality of the generated reports, training the vision-language model on full reports showed better cross-modal retrieval performance.
• 摘要：病理学中的视觉模型可实现多模式病例检索和自动报告的生成。然而，到目前为止，许多模型都已在病理报告上进行了培训，这些报告包括无法从配对的整个幻灯片图像（例如患者历史记录）中推断出的信息，这可能会导致生成报告中的幻觉句子。为此，我们研究了视觉模型的病理报告中信息的选择如何影响多模式表示和生成报告的质量。更具体地说，我们比较了一个在完整报告中训练的模型与经过预处理报告训练的模型，该模型仅包括根据H＆E染色幻灯片描述细胞和组织外观的句子。在实验中，我们建立在BLIP-2框架上，并使用了42,433 H＆e染色的整个幻灯片图像和19,636个相应病理学报告的皮肤黑素细胞病变数据集。使用图像到文本和文本对图像检索评估模型性能，并对专家病理学家对生成的报告进行定性评估。我们的结果表明，文本预处理可阻止报告生成中的幻觉。尽管生成的报告的质量有所提高，但在完整报告中培训视觉模型表现出更好的跨模式检索性能。

Title: Pathology Report Generation and Multimodal Representation Learning for Cutaneous Melanocytic Lesions

• Authors: Ruben T. Lucassen, Sander P.J. Moonemans, Tijn van de Luijtgaarden, Gerben E. Breimer, Willeke A.M. Blokx, Mitko Veta
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19293
• Pdf URL: https://arxiv.org/pdf/2502.19293
• Copy Paste: [[2502.19293]] Pathology Report Generation and Multimodal Representation Learning for Cutaneous Melanocytic Lesions(https://arxiv.org/abs/2502.19293)
• Keywords: generation
• Abstract: Millions of melanocytic skin lesions are examined by pathologists each year, the majority of which concern common nevi (i.e., ordinary moles). While most of these lesions can be diagnosed in seconds, writing the corresponding pathology report is much more time-consuming. Automating part of the report writing could, therefore, alleviate the increasing workload of pathologists. In this work, we develop a vision-language model specifically for the pathology domain of cutaneous melanocytic lesions. The model follows the Contrastive Captioner framework and was trained and evaluated using a melanocytic lesion dataset of 42,512 H&E-stained whole slide images and 19,645 corresponding pathology reports. Our results show that the quality scores of model-generated reports were on par with pathologist-written reports for common nevi, assessed by an expert pathologist in a reader study. While report generation revealed to be more difficult for rare melanocytic lesion subtypes, the cross-modal retrieval performance for these cases was considerably better.
• 摘要：每年病理学家检查数百万个黑素细胞皮肤病变，其中大多数涉及常见的NEVI（即普通痣）。尽管这些病变中的大多数可以在几秒钟内诊断，但编写相应的病理报告耗时要耗时。因此，报告写作的一部分可以减轻病理学家的工作量不断增加。在这项工作中，我们开发了专门针对皮肤黑素细胞病变病理结构域的视觉模型。该模型遵循对比字幕框架，并使用42,512 H＆e染色的整个幻灯片图像和19,645个相应病理学报告的黑素细胞病变数据集进行了训练和评估。我们的结果表明，模型生成的报告的质量得分与一名专家病理学家在读者研究中评估的常见NEVI的病理学家所写的报告相当。虽然报告的产生显示稀有黑素细胞病变亚型更困难，但这些病例的跨模式检索性能要好得多。

Title: FSPO: Few-Shot Preference Optimization of Synthetic Preference Data in LLMs Elicits Effective Personalization to Real Users

• Authors: Anikait Singh, Sheryl Hsu, Kyle Hsu, Eric Mitchell, Stefano Ermon, Tatsunori Hashimoto, Archit Sharma, Chelsea Finn
• Subjects: cs.LG, cs.AI, cs.CL, cs.HC, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.19312
• Pdf URL: https://arxiv.org/pdf/2502.19312
• Copy Paste: [[2502.19312]] FSPO: Few-Shot Preference Optimization of Synthetic Preference Data in LLMs Elicits Effective Personalization to Real Users(https://arxiv.org/abs/2502.19312)
• Keywords: generation
• Abstract: Effective personalization of LLMs is critical for a broad range of user-interfacing applications such as virtual assistants and content curation. Inspired by the strong in-context learning capabilities of LLMs, we propose Few-Shot Preference Optimization (FSPO), which reframes reward modeling as a meta-learning problem. Under this framework, an LLM learns to quickly adapt to a user via a few labeled preferences from that user, constructing a personalized reward function for them. Additionally, since real-world preference data is scarce and challenging to collect at scale, we propose careful design choices to construct synthetic preference datasets for personalization, generating over 1M synthetic personalized preferences using publicly available LLMs. In particular, to successfully transfer from synthetic data to real users, we find it crucial for the data to exhibit both high diversity and coherent, self-consistent structure. We evaluate FSPO on personalized open-ended generation for up to 1,500 synthetic users across across three domains: movie reviews, pedagogical adaptation based on educational background, and general question answering, along with a controlled human study. Overall, FSPO achieves an 87% Alpaca Eval winrate on average in generating responses that are personalized to synthetic users and a 72% winrate with real human users in open-ended question answering.
• 摘要：LLM的有效个性化对于多种用户交流应用程序（例如虚拟助手和内容策划）至关重要。受LLM的强大内在学习能力的启发，我们提出了很少的偏好优化（FSPO），将奖励建模重新构建为元学习问题。在此框架下，LLM学会了通过该用户的一些标记的首选项快速适应用户，并为其构建个性化的奖励功能。此外，由于现实世界中的偏好数据稀缺，并且在大规模收集方面挑战，因此我们建议仔细的设计选择来构建用于个性化的合成偏好数据集，并使用公开可用的LLM产生超过1M的合成个性化偏好。特别是，要成功地从合成数据转移到真实用户，我们发现数据表现出高度多样性和相干，自洽的结构至关重要。我们评估了FSPO的个性化开放式一代，可用于跨三个领域的多达1,500个合成用户：电影评论，基于教育背景的教学适应以及一般的问题回答以及对照人的人类研究。总体而言，FSPO在产生对合成用户的个性化响应方面平均达到了87％的羊驼毛评估，并在开放式的问题答案中与真正的人类用户获得了72％的获胜。

Title: CoopDETR: A Unified Cooperative Perception Framework for 3D Detection via Object Query

• Authors: Zhe Wang, Shaocong Xu, Xucai Zhuang, Tongda Xu, Yan Wang, Jingjing Liu, Yilun Chen, Ya-Qin Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19313
• Pdf URL: https://arxiv.org/pdf/2502.19313
• Copy Paste: [[2502.19313]] CoopDETR: A Unified Cooperative Perception Framework for 3D Detection via Object Query(https://arxiv.org/abs/2502.19313)
• Keywords: generation
• Abstract: Cooperative perception enhances the individual perception capabilities of autonomous vehicles (AVs) by providing a comprehensive view of the environment. However, balancing perception performance and transmission costs remains a significant challenge. Current approaches that transmit region-level features across agents are limited in interpretability and demand substantial bandwidth, making them unsuitable for practical applications. In this work, we propose CoopDETR, a novel cooperative perception framework that introduces object-level feature cooperation via object query. Our framework consists of two key modules: single-agent query generation, which efficiently encodes raw sensor data into object queries, reducing transmission cost while preserving essential information for detection; and cross-agent query fusion, which includes Spatial Query Matching (SQM) and Object Query Aggregation (OQA) to enable effective interaction between queries. Our experiments on the OPV2V and V2XSet datasets demonstrate that CoopDETR achieves state-of-the-art performance and significantly reduces transmission costs to 1/782 of previous methods.
• 摘要：合作感知通过提供对环境的全面看法，增强了自动驾驶汽车（AV）的个人看法能力。但是，平衡感知性能和传播成本仍然是一个重大挑战。当前在跨代理传输区域级特征的方法限制了可解释性和需求大量的带宽，这使得它们不适合实际应用。在这项工作中，我们提出了一个新颖的合作感知框架Copdetr，该框架通过对象查询介绍了对象级特征合作。我们的框架由两个关键模块组成：单格查询生成，它们有效地将原始传感器数据编码到对象查询中，从而降低了传输成本，同时保留了检测基本信息；和跨代理查询融合，其中包括空间查询匹配（SQM）和对象查询聚合（OQA），以实现查询之间的有效相互作用。我们在OPV2V和V2XSET数据集上进行的实验表明，Coopdetr可以实现最先进的性能，并将传输成本显着降低到先前的1/782。

Title: Model Adaptation: Unsupervised Domain Adaptation without Source Data

• Authors: Rui Li, Qianfen Jiao, Wenming Cao, Hau-San Wong, Si Wu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19316
• Pdf URL: https://arxiv.org/pdf/2502.19316
• Copy Paste: [[2502.19316]] Model Adaptation: Unsupervised Domain Adaptation without Source Data(https://arxiv.org/abs/2502.19316)
• Keywords: generative
• Abstract: In this paper, we investigate a challenging unsupervised domain adaptation setting -- unsupervised model adaptation. We aim to explore how to rely only on unlabeled target data to improve performance of an existing source prediction model on the target domain, since labeled source data may not be available in some real-world scenarios due to data privacy issues. For this purpose, we propose a new framework, which is referred to as collaborative class conditional generative adversarial net to bypass the dependence on the source data. Specifically, the prediction model is to be improved through generated target-style data, which provides more accurate guidance for the generator. As a result, the generator and the prediction model can collaborate with each other without source data. Furthermore, due to the lack of supervision from source data, we propose a weight constraint that encourages similarity to the source model. A clustering-based regularization is also introduced to produce more discriminative features in the target domain. Compared to conventional domain adaptation methods, our model achieves superior performance on multiple adaptation tasks with only unlabeled target data, which verifies its effectiveness in this challenging setting.
• 摘要：在本文中，我们研究了一个具有挑战性的无监督域适应设置 - 无监督的模型适应。我们旨在探索如何仅依靠未标记的目标数据来提高目标域上现有源预测模型的性能，因为由于数据隐私问题，在某些真实世界中可能无法在某些真实世界中使用标记的源数据。为此，我们提出了一个新框架，该框架被称为协作类别的有条件生成对抗网，以绕过对源数据的依赖性。具体而言，预测模型将通过生成的目标式数据进行改进，该数据为生成器提供了更准确的指导。结果，生成器和预测模型可以在没有源数据的情况下相互协作。此外，由于缺乏源数据的监督，我们提出了一个鼓励与源模型相似的权重限制。还引入了基于聚类的正则化，以在目标域中产生更多的判别特征。与传统的域适应方法相比，我们的模型在仅使用未标记的目标数据的多个适应任务上实现了卓越的性能，这在这种挑战性的环境中验证了其有效性。

Title: Partition Tree Weighting for Non-Stationary Stochastic Bandits

• Authors: Joel Veness, Marcus Hutter, Andras Gyorgy, Jordi Grau-Moya
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19325
• Pdf URL: https://arxiv.org/pdf/2502.19325
• Copy Paste: [[2502.19325]] Partition Tree Weighting for Non-Stationary Stochastic Bandits(https://arxiv.org/abs/2502.19325)
• Keywords: generation
• Abstract: This paper considers a generalisation of universal source coding for interaction data, namely data streams that have actions interleaved with observations. Our goal will be to construct a coding distribution that is both universal \emph{and} can be used as a control policy. Allowing for action generation needs careful treatment, as naive approaches which do not distinguish between actions and observations run into the self-delusion problem in universal settings. We showcase our perspective in the context of the challenging non-stationary stochastic Bernoulli bandit problem. Our main contribution is an efficient and high performing algorithm for this problem that generalises the Partition Tree Weighting universal source coding technique for passive prediction to the control setting.
• 摘要：本文考虑了对交互数据的通用源编码的概括，即具有与观测值相交的动作的数据流。我们的目标是构建一个通用\ emph {and}的编码分布，可以用作控制策略。允许产生行动需要仔细的治疗，因为在普遍环境中没有区分动作和观察结果的天真方法。我们在具有挑战性的非平稳随机Bernoulli强盗问题的背景下展示了我们的观点。我们的主要贡献是针对此问题的有效且高性能的算法，它概括了分区树加权通用源编码技术，用于被动预测控制设置。

Title: Consistent Amortized Clustering via Generative Flow Networks

• Authors: Irit Chelly, Roy Uziel, Oren Freifeld, Ari Pakman
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19337
• Pdf URL: https://arxiv.org/pdf/2502.19337
• Copy Paste: [[2502.19337]] Consistent Amortized Clustering via Generative Flow Networks(https://arxiv.org/abs/2502.19337)
• Keywords: generative
• Abstract: Neural models for amortized probabilistic clustering yield samples of cluster labels given a set-structured input, while avoiding lengthy Markov chain runs and the need for explicit data likelihoods. Existing methods which label each data point sequentially, like the Neural Clustering Process, often lead to cluster assignments highly dependent on the data order. Alternatively, methods that sequentially create full clusters, do not provide assignment probabilities. In this paper, we introduce GFNCP, a novel framework for amortized clustering. GFNCP is formulated as a Generative Flow Network with a shared energy-based parametrization of policy and reward. We show that the flow matching conditions are equivalent to consistency of the clustering posterior under marginalization, which in turn implies order invariance. GFNCP also outperforms existing methods in clustering performance on both synthetic and real-world data.
• 摘要：摊销概率聚类的神经模型产生了集群标签的样本，给定设置结构的输入，同时避免了冗长的马尔可夫链运行，并且需要明确的数据可能性。依次像神经聚类过程一样依次标记每个数据点的现有方法通常会导致聚类分配高度依赖于数据顺序。或者，依次创建完整簇，不提供分配概率的方法。在本文中，我们介绍了GFNCP，这是一种用于摊销聚类的新型框架。 GFNCP被配制为具有共享的策略和奖励参数化的生成流网络。我们表明，流匹配条件等于边缘化下聚类后验的一致性，这又意味着顺序不变性。 GFNCP还优于合成和现实世界数据的聚类性能方面的现有方法。

Title: Deep Learning For Time Series Analysis With Application On Human Motion

• Authors: Ali Ismail-Fawaz
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19364
• Pdf URL: https://arxiv.org/pdf/2502.19364
• Copy Paste: [[2502.19364]] Deep Learning For Time Series Analysis With Application On Human Motion(https://arxiv.org/abs/2502.19364)
• Keywords: generation, generative
• Abstract: Time series data, defined by equally spaced points over time, is essential in fields like medicine, telecommunications, and energy. Analyzing it involves tasks such as classification, clustering, prototyping, and regression. Classification identifies normal vs. abnormal movements in skeleton-based motion sequences, clustering detects stock market behavior patterns, prototyping expands physical therapy datasets, and regression predicts patient recovery. Deep learning has recently gained traction in time series analysis due to its success in other domains. This thesis leverages deep learning to enhance classification with feature engineering, introduce foundation models, and develop a compact yet state-of-the-art architecture. We also address limited labeled data with self-supervised learning. Our contributions apply to real-world tasks, including human motion analysis for action recognition and rehabilitation. We introduce a generative model for human motion data, valuable for cinematic production and gaming. For prototyping, we propose a shape-based synthetic sample generation method to support regression models when data is scarce. Lastly, we critically evaluate discriminative and generative models, identifying limitations in current methodologies and advocating for a robust, standardized evaluation framework. Our experiments on public datasets provide novel insights and methodologies, advancing time series analysis with practical applications.
• 摘要：时间序列数据由随着时间的流逝而定义，在药物，电信和能量等领域至关重要。分析它涉及分类，聚类，原型和回归等任务。分类确定基于骨架的运动序列中的正常运动与异常运动，聚类检测股票市场行为模式，原型型扩展物理治疗数据集，而回归预测了患者的康复。由于其在其他领域的成功，深度学习最近在时间序列分析中获得了吸引力。该论文利用深度学习来通过功能工程，介绍基础模型并开发紧凑而最先进的体系结构来增强分类。我们还通过自我监督学习来解决有限的标记数据。我们的贡献适用于现实世界中的任务，包括针对行动识别和康复的人类运动分析。我们引入了人类运动数据的生成模型，对电影制作和游戏很有价值。对于原型制作，我们提出了一种基于形状的合成样品生成方法，以支持数据稀缺时支持回归模型。最后，我们批判性地评估了歧视性和生成性模型，确定当前方法中的局限性，并倡导一个可靠的，标准化的评估框架。我们在公共数据集上的实验提供了新颖的见解和方法，并通过实用应用来推进时间序列分析。