2025-06-19

Title: Integrating Dynamical Systems Learning with Foundational Models: A Meta-Evolutionary AI Framework for Clinical Trials

• Authors: Joseph Geraci, Bessi Qorri, Christian Cumbaa, Mike Tsay, Paul Leonczyk, Luca Pani
• Subjects: cs.LG, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2506.14782
• Pdf URL: https://arxiv.org/pdf/2506.14782
• Copy Paste: [[2506.14782]] Integrating Dynamical Systems Learning with Foundational Models: A Meta-Evolutionary AI Framework for Clinical Trials(https://arxiv.org/abs/2506.14782)
• Keywords: generation
• Abstract: Artificial intelligence (AI) has evolved into an ecosystem of specialized "species," each with unique strengths. We analyze two: DeepSeek-V3, a 671-billion-parameter Mixture of Experts large language model (LLM) exemplifying scale-driven generality, and NetraAI, a dynamical system-based framework engineered for stability and interpretability on small clinical trial datasets. We formalize NetraAI's foundations, combining contraction mappings, information geometry, and evolutionary algorithms to identify predictive patient cohorts. Features are embedded in a metric space and iteratively contracted toward stable attractors that define latent subgroups. A pseudo-temporal embedding and long-range memory enable exploration of higher-order feature interactions, while an internal evolutionary loop selects compact, explainable 2-4-variable bundles ("Personas"). To guide discovery, we introduce an LLM Strategist as a meta-evolutionary layer that observes Persona outputs, prioritizes promising variables, injects domain knowledge, and assesses robustness. This two-tier architecture mirrors the human scientific process: NetraAI as experimentalist, the LLM as theorist, forming a self-improving loop. In case studies (schizophrenia, depression, pancreatic cancer), NetraAI uncovered small, high-effect-size subpopulations that transformed weak baseline models (AUC ~0.50-0.68) into near-perfect classifiers using only a few features. We position NetraAI at the intersection of dynamical systems, information geometry, and evolutionary learning, aligned with emerging concept-level reasoning paradigms such as LeCun's Joint Embedding Predictive Architecture (JEPA). By prioritizing reliable, explainable knowledge, NetraAI offers a new generation of adaptive, self-reflective AI to accelerate clinical discovery.
• 摘要：人工智能（AI）已演变为专业“物种”的生态系统，每个生态系统都具有独特的优势。我们分析了两个：DeepSeek-V3，这是专家大型语言模型（LLM）的6710亿参数混合物，例如规模驱动的一般性，而Netraai是一个基于动态系统的基于动力学的框架，该框架在小型临床试验数据集上设计为稳定性和可解释性。我们将Netraai的基础正式化，结合了收缩映射，信息几何形状和进化算法，以识别预测性患者同伙。功能嵌入度量空间中，并迭代地收缩朝定义潜在亚组的稳定吸引子。伪周期性嵌入和远程记忆可以探索高阶功能相互作用，而内部进化环选择紧凑，可解释的2-4变量捆绑包（“角色”）。为了指导发现，我们将LLM策略师作为一个元进化层，观察角色输出，优先考虑有希望的变量，注入域知识并评估鲁棒性。这种两层架构反映了人类的科学过程：Netraai作为实验主义者，LLM为理论家，形成了自我改善的循环。在案例研究（精神分裂症，抑郁症，胰腺癌）中，Netraai发现了仅使用几个特征将弱基线模型（AUC 〜0.50-0.68）转化为近乎完美的分类器的小型，高效应的亚群。我们将Netraai定位在动态系统，信息几何学和进化学习的交集中，并与新兴概念级推理范式（如Lecun的联合嵌入预测性结构（JEPA））保持一致。通过优先考虑可靠，可解释的知识，Netraai提供了新一代的自适应，自我反思的AI来加速临床发现。

Title: ETS: Open Vocabulary Electroencephalography-To-Text Decoding and Sentiment Classification

• Authors: Mohamed Masry, Mohamed Amen, Mohamed Elzyat, Mohamed Hamed, Norhan Magdy, Maram Khaled
• Subjects: cs.LG, cs.CL, cs.HC
• Abstract URL: https://arxiv.org/abs/2506.14783
• Pdf URL: https://arxiv.org/pdf/2506.14783
• Copy Paste: [[2506.14783]] ETS: Open Vocabulary Electroencephalography-To-Text Decoding and Sentiment Classification(https://arxiv.org/abs/2506.14783)
• Keywords: generation
• Abstract: Decoding natural language from brain activity using non-invasive electroencephalography (EEG) remains a significant challenge in neuroscience and machine learning, particularly for open-vocabulary scenarios where traditional methods struggle with noise and variability. Previous studies have achieved high accuracy on small-closed vocabularies, but it still struggles on open vocabularies. In this study, we propose ETS, a framework that integrates EEG with synchronized eye-tracking data to address two critical tasks: (1) open-vocabulary text generation and (2) sentiment classification of perceived language. Our model achieves a superior performance on BLEU and Rouge score for EEG-To-Text decoding and up to 10% F1 score on EEG-based ternary sentiment classification, which significantly outperforms supervised baselines. Furthermore, we show that our proposed model can handle data from various subjects and sources, showing great potential for high performance open vocabulary eeg-to-text system.
• 摘要：使用非侵入性脑电图（EEG）从大脑活动中解码自然语言仍然是神经科学和机器学习中的重大挑战，尤其是对于开放式摄影的场景，传统方法与噪声和可变性很难。先前的研究在小关闭的词汇上取得了很高的准确性，但它仍然在开放式词汇上挣扎。在这项研究中，我们提出了ETS，该框架将脑电图与同步的眼睛跟踪数据集成在一起，以解决两个关键任务：（1）开放式唱歌的文本生成和（2）感知语言的情感分类。我们的模型在BLEU和Rouge得分方面取得了卓越的表现，对脑电图解码，基于脑电图的三元性情感分类最高为10％的F1分数，这极大地超过了监督的基线。此外，我们表明我们提出的模型可以处理来自各种主题和来源的数据，从而显示出高性能开放词汇量表到文本系统的巨大潜力。

Title: DAVID-XR1: Detecting AI-Generated Videos with Explainable Reasoning

• Authors: Yifeng Gao, Yifan Ding, Hongyu Su, Juncheng Li, Yunhan Zhao, Lin Luo, Zixing Chen, Li Wang, Xin Wang, Yixu Wang, Xingjun Ma, Yu-Gang Jiang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.14827
• Pdf URL: https://arxiv.org/pdf/2506.14827
• Copy Paste: [[2506.14827]] DAVID-XR1: Detecting AI-Generated Videos with Explainable Reasoning(https://arxiv.org/abs/2506.14827)
• Keywords: generation
• Abstract: As AI-generated video becomes increasingly pervasive across media platforms, the ability to reliably distinguish synthetic content from authentic footage has become both urgent and essential. Existing approaches have primarily treated this challenge as a binary classification task, offering limited insight into where or why a model identifies a video as AI-generated. However, the core challenge extends beyond simply detecting subtle artifacts; it requires providing fine-grained, persuasive evidence that can convince auditors and end-users alike. To address this critical gap, we introduce DAVID-X, the first dataset to pair AI-generated videos with detailed defect-level, temporal-spatial annotations and written rationales. Leveraging these rich annotations, we present DAVID-XR1, a video-language model designed to deliver an interpretable chain of visual reasoning-including defect categorization, temporal-spatial localization, and natural language explanations. This approach fundamentally transforms AI-generated video detection from an opaque black-box decision into a transparent and verifiable diagnostic process. We demonstrate that a general-purpose backbone, fine-tuned on our compact dataset and enhanced with chain-of-thought distillation, achieves strong generalization across a variety of generators and generation modes. Our results highlight the promise of explainable detection methods for trustworthy identification of AI-generated video content.
• 摘要：随着AI生成的视频在整个媒体平台之间变得越来越普遍，可以可靠地区分合成内容和真实镜头的能力变得既紧急又至关重要。现有方法主要将这一挑战视为二进制分类任务，从而有限地了解模型将视频识别为AI生成的何处或为什么。但是，核心挑战不仅仅是发现微妙的文物。它需要提供细粒度，有说服力的证据，以说服审计师和最终用户。为了解决这个关键的差距，我们介绍了David-X，这是第一个将AI生成视频与详细的缺陷级，时间空间注释和书面理由配对的数据集。利用这些丰富的注释，我们提出了David-XR1，这是一个视频语言模型，旨在提供一系列可解释的视觉推理链，包括缺陷分类，时间空间定位和自然语言解释。这种方法从根本上将AI生成的视频检测从不透明的黑盒决策转变为透明且可验证的诊断过程。我们证明，在紧凑型数据集上进行了微调的通用骨干，并通过经过经过经过经过经过经过经验的蒸馏链增强，在各种发电机和发电模式中实现了强烈的概括。我们的结果突出了可解释的检测方法的承诺，以鉴定AI生成的视频内容。

Title: ArchShapeNet:An Interpretable 3D-CNN Framework for Evaluating Architectural Shapes

• Authors: Jun Yin, Jing Zhong, Pengyu Zeng, Peilin Li, Zixuan Dai, Miao Zhang, Shuai Lu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.14832
• Pdf URL: https://arxiv.org/pdf/2506.14832
• Copy Paste: [[2506.14832]] ArchShapeNet:An Interpretable 3D-CNN Framework for Evaluating Architectural Shapes(https://arxiv.org/abs/2506.14832)
• Keywords: generative
• Abstract: In contemporary architectural design, the growing complexity and diversity of design demands have made generative plugin tools essential for quickly producing initial concepts and exploring novel 3D forms. However, objectively analyzing the differences between human-designed and machine-generated 3D forms remains a challenge, limiting our understanding of their respective strengths and hindering the advancement of generative tools. To address this, we built ArchForms-4000, a dataset containing 2,000 architect-designed and 2,000 Evomass-generated 3D forms; Proposed ArchShapeNet, a 3D convolutional neural network tailored for classifying and analyzing architectural forms, incorporating a saliency module to highlight key spatial features aligned with architectural reasoning; And conducted comparative experiments showing our model outperforms human experts in distinguishing form origins, achieving 94.29% accuracy, 96.2% precision, and 98.51% recall. This study not only highlights the distinctive advantages of human-designed forms in spatial organization, proportional harmony, and detail refinement but also provides valuable insights for enhancing generative design tools in the future.
• 摘要：在当代建筑设计中，设计需求的日益增长的复杂性和多样性使生成插件工具对于快速产生初始概念和探索新颖的3D形式至关重要。但是，客观地分析人工设计和机器生成的3D形式之间的差异仍然是一个挑战，限制了我们对它们各自优势的理解并阻碍了生成工具的进步。为了解决这个问题，我们构建了Archforms-4000，该数据集包含2,000个建筑师设计和2,000个ivomass生成的3D表格；拟议的ArchShapenet是一个针对分类和分析建筑形式进行分类和分析的3D卷积神经网络，并结合了显着模块，以突出与建筑推理一致的关键空间特征；并进行了比较实验，该实验表明我们的模型的表现优于人类专家，以区分形式起源，达到94.29％的精度，96.2％的精度和98.51％的召回率。这项研究不仅强调了空间组织，比例和谐和细节改进的人类设计形式的独特优势，而且还为将来增强生成设计工具提供了宝贵的见解。

Title: Improved Iterative Refinement for Chart-to-Code Generation via Structured Instruction

• Authors: Chengzhi Xu, Yuyang Wang, Lai Wei, Lichao Sun, Weiran Huang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.14837
• Pdf URL: https://arxiv.org/pdf/2506.14837
• Copy Paste: [[2506.14837]] Improved Iterative Refinement for Chart-to-Code Generation via Structured Instruction(https://arxiv.org/abs/2506.14837)
• Keywords: generation
• Abstract: Recently, multimodal large language models (MLLMs) have attracted increasing research attention due to their powerful visual understanding capabilities. While they have achieved impressive results on various vision tasks, their performance on chart-to-code generation remains suboptimal. This task requires MLLMs to generate executable code that can reproduce a given chart, demanding not only precise visual understanding but also accurate translation of visual elements into structured code. Directly prompting MLLMs to perform this complex task often yields unsatisfactory results. To address this challenge, we propose {ChartIR}, an iterative refinement method based on structured instruction. First, we distinguish two tasks: visual understanding and code translation. To accomplish the visual understanding component, we design two types of structured instructions: description and difference. The description instruction captures the visual elements of the reference chart, while the difference instruction characterizes the discrepancies between the reference chart and the generated chart. These instructions effectively transform visual features into language representations, thereby facilitating the subsequent code translation process. Second, we decompose the overall chart generation pipeline into two stages: initial code generation and iterative refinement, enabling progressive enhancement of the final output. Experimental results show that, compared to other method, our method achieves superior performance on both the open-source model Qwen2-VL and the closed-source model GPT-4o.
• 摘要：最近，由于其强大的视觉理解能力，多模式的大型语言模型（MLLM）吸引了越来越多的研究注意力。尽管他们在各种视觉任务上取得了令人印象深刻的结果，但它们在图表到代码的生成上的表现仍然不佳。此任务要求MLLM生成可以重现给定图表的可执行代码，不仅需要精确的视觉理解，而且要求将视觉元素准确地转换为结构化代码。直接提示MLLM执行此复杂任务通常会产生不令人满意的结果。为了应对这一挑战，我们提出{Chartir}，这是一种基于结构化指令的迭代改进方法。首先，我们区分了两个任务：视觉理解和代码翻译。为了完成视觉理解组件，我们设计了两种类型的结构化指令：描述和差异。描述指令捕获了参考图的视觉元素，而差异指令则表征了参考图和生成图表之间的差异。这些指令有效地将视觉特征转换为语言表示，从而促进了后续的代码翻译过程。其次，我们将总图表生成管道分解为两个阶段：初始代码生成和迭代改进，从而逐步增强最终输出。实验结果表明，与其他方法相比，我们的方法在开源模型QWEN2-VL和封闭源模型GPT-4O上都能达到卓越的性能。

Title: CACTUS as a Reliable Tool for Early Classification of Age-related Macular Degeneration

• Authors: Luca Gherardini, Imre Lengyel, Tunde Peto, Caroline C.W. Klaverd, Magda A. Meester-Smoord, Johanna Maria Colijnd, EYE-RISK Consortium, E3 Consortium, Jose Sousa
• Subjects: cs.LG, cs.CV, stat.AP
• Abstract URL: https://arxiv.org/abs/2506.14843
• Pdf URL: https://arxiv.org/pdf/2506.14843
• Copy Paste: [[2506.14843]] CACTUS as a Reliable Tool for Early Classification of Age-related Macular Degeneration(https://arxiv.org/abs/2506.14843)
• Keywords: generation
• Abstract: Machine Learning (ML) is used to tackle various tasks, such as disease classification and prediction. The effectiveness of ML models relies heavily on having large amounts of complete data. However, healthcare data is often limited or incomplete, which can hinder model performance. Additionally, issues like the trustworthiness of solutions vary with the datasets used. The lack of transparency in some ML models further complicates their understanding and use. In healthcare, particularly in the case of Age-related Macular Degeneration (AMD), which affects millions of older adults, early diagnosis is crucial due to the absence of effective treatments for reversing progression. Diagnosing AMD involves assessing retinal images along with patients' symptom reports. There is a need for classification approaches that consider genetic, dietary, clinical, and demographic factors. Recently, we introduced the -Comprehensive Abstraction and Classification Tool for Uncovering Structures-(CACTUS), aimed at improving AMD stage classification. CACTUS offers explainability and flexibility, outperforming standard ML models. It enhances decision-making by identifying key factors and providing confidence in its results. The important features identified by CACTUS allow us to compare with existing medical knowledge. By eliminating less relevant or biased data, we created a clinical scenario for clinicians to offer feedback and address biases.
• 摘要：机器学习（ML）用于解决各种任务，例如疾病分类和预测。 ML模型的有效性在很大程度上取决于拥有大量完整数据。但是，医疗保健数据通常受到限制或不完整，这可能会阻碍模型性能。此外，诸如解决方案的可信度之类的问题随所使用的数据集各不相同。在某些ML模型中缺乏透明度使他们的理解和使用更加复杂。在医疗保健中，特别是在与年龄相关的黄斑变性（AMD）的情况下，影响了数百万老年人，由于缺乏有效的逆转进展治疗，早期诊断至关重要。诊断AMD涉及评估视网膜图像以及患者的症状报告。需要考虑遗传，饮食，临床和人口统计学因素的分类方法。最近，我们引入了旨在改善AMD阶段分类的结构（仙人掌）的 - 概述的抽象和分类工具。仙人掌提供了解释性和灵活性，优于标准ML模型。它通过确定关键因素并提供对结果的信心来增强决策。仙人掌确定的重要特征使我们能够与现有的医学知识进行比较。通过消除相关或有偏见的数据，我们为临床医生创建了一个临床方案，以提供反馈和解决偏见。

Title: Efficient Retail Video Annotation: A Robust Key Frame Generation Approach for Product and Customer Interaction Analysis

• Authors: Varun Mannam, Zhenyu Shi
• Subjects: cs.CV, cs.AI, cs.HC, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.14854
• Pdf URL: https://arxiv.org/pdf/2506.14854
• Copy Paste: [[2506.14854]] Efficient Retail Video Annotation: A Robust Key Frame Generation Approach for Product and Customer Interaction Analysis(https://arxiv.org/abs/2506.14854)
• Keywords: generation
• Abstract: Accurate video annotation plays a vital role in modern retail applications, including customer behavior analysis, product interaction detection, and in-store activity recognition. However, conventional annotation methods heavily rely on time-consuming manual labeling by human annotators, introducing non-robust frame selection and increasing operational costs. To address these challenges in the retail domain, we propose a deep learning-based approach that automates key-frame identification in retail videos and provides automatic annotations of products and customers. Our method leverages deep neural networks to learn discriminative features by embedding video frames and incorporating object detection-based techniques tailored for retail environments. Experimental results showcase the superiority of our approach over traditional methods, achieving accuracy comparable to human annotator labeling while enhancing the overall efficiency of retail video annotation. Remarkably, our approach leads to an average of 2 times cost savings in video annotation. By allowing human annotators to verify/adjust less than 5% of detected frames in the video dataset, while automating the annotation process for the remaining frames without reducing annotation quality, retailers can significantly reduce operational costs. The automation of key-frame detection enables substantial time and effort savings in retail video labeling tasks, proving highly valuable for diverse retail applications such as shopper journey analysis, product interaction detection, and in-store security monitoring.
• 摘要：准确的视频注释在现代零售应用中起着至关重要的作用，包括客户行为分析，产品互动检测和店内活动识别。但是，传统的注释方法在很大程度上依赖于人类注释者的耗时手动标记，引入了非稳定框架选择并增加了运营成本。为了解决零售领域中的这些挑战，我们提出了一种基于深度学习的方法，该方法可以在零售视频中自动化密钥框架标识，并提供对产品和客户的自动注释。我们的方法通过嵌入视频帧并结合针对零售环境量身定制的基于对象检测的技术来利用深层神经网络来学习判别特征。实验结果展示了我们的方法比传统方法的优越性，在提高零售视频注释的整体效率的同时，达到了与人类注释者标签相当的准确性。值得注意的是，我们的方法平均节省了视频注释成本的2倍。通过允许人类注释者在视频数据集中验证/调整少于5％的检测帧的5％，同时自动化剩余框架的注释过程而不会降低注释质量，零售商可以大大降低运营成本。密钥框架检测的自动化可以在零售视频标签任务中节省大量时间和精力，证明对诸如购物者旅程分析，产品互动检测和店内安全监控等多样化的零售应用程序非常有价值。

Title: Frequency-Calibrated Membership Inference Attacks on Medical Image Diffusion Models

• Authors: Xinkai Zhao, Yuta Tokuoka, Junichiro Iwasawa, Keita Oda
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.14919
• Pdf URL: https://arxiv.org/pdf/2506.14919
• Copy Paste: [[2506.14919]] Frequency-Calibrated Membership Inference Attacks on Medical Image Diffusion Models(https://arxiv.org/abs/2506.14919)
• Keywords: generation
• Abstract: The increasing use of diffusion models for image generation, especially in sensitive areas like medical imaging, has raised significant privacy concerns. Membership Inference Attack (MIA) has emerged as a potential approach to determine if a specific image was used to train a diffusion model, thus quantifying privacy risks. Existing MIA methods often rely on diffusion reconstruction errors, where member images are expected to have lower reconstruction errors than non-member images. However, applying these methods directly to medical images faces challenges. Reconstruction error is influenced by inherent image difficulty, and diffusion models struggle with high-frequency detail reconstruction. To address these issues, we propose a Frequency-Calibrated Reconstruction Error (FCRE) method for MIAs on medical image diffusion models. By focusing on reconstruction errors within a specific mid-frequency range and excluding both high-frequency (difficult to reconstruct) and low-frequency (less informative) regions, our frequency-selective approach mitigates the confounding factor of inherent image difficulty. Specifically, we analyze the reverse diffusion process, obtain the mid-frequency reconstruction error, and compute the structural similarity index score between the reconstructed and original images. Membership is determined by comparing this score to a threshold. Experiments on several medical image datasets demonstrate that our FCRE method outperforms existing MIA methods.
• 摘要：越来越多地使用扩散模型来产生图像，尤其是在诸如医学成像之类的敏感领域，引起了严重的隐私问题。会员推理攻击（MIA）已成为一种潜在的方法，可以确定是否使用特定图像来训练扩散模型，从而量化隐私风险。现有的MIA方法通常依赖于扩散重建误差，在该错误中，成员图像的重建错误比非会员图像较低。但是，将这些方法直接应用于医学图像面临挑战。重建误差受固有图像难度的影响，而扩散模型则在高频细节重建方面遇到了困难。为了解决这些问题，我们提出了一种在医学图像扩散模型上使用频率校准的重建误差（FCRE）方法。通过关注特定的中频范围内的重建错误，并排除了高频（难以重建）和低频（信息较少）区域，我们的频率选择方法可以减轻固有图像难度的混杂因素。具体而言，我们分析了反向扩散过程，获得中频重建误差，并计算重建和原始图像之间的结构相似性指数分数。会员资格是通过将此分数与阈值进行比较来确定的。几个医疗图像数据集的实验表明，我们的FCRE方法的表现优于现有的MIA方法。

Title: Break Stylistic Sophon: Are We Really Meant to Confine the Imagination in Style Transfer?

• Authors: Gary Song Yan, Yusen Zhang, Jinyu Zhao, Hao Zhang, Zhangping Yang, Guanye Xiong, Yanfei Liu, Tao Zhang, Yujie He, Siyuan Tian, Yao Gou, Min Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.15033
• Pdf URL: https://arxiv.org/pdf/2506.15033
• Copy Paste: [[2506.15033]] Break Stylistic Sophon: Are We Really Meant to Confine the Imagination in Style Transfer?(https://arxiv.org/abs/2506.15033)
• Keywords: generation
• Abstract: In this pioneering study, we introduce StyleWallfacer, a groundbreaking unified training and inference framework, which not only addresses various issues encountered in the style transfer process of traditional methods but also unifies the framework for different tasks. This framework is designed to revolutionize the field by enabling artist level style transfer and text driven stylization. First, we propose a semantic-based style injection method that uses BLIP to generate text descriptions strictly aligned with the semantics of the style image in CLIP space. By leveraging a large language model to remove style-related descriptions from these descriptions, we create a semantic gap. This gap is then used to fine-tune the model, enabling efficient and drift-free injection of style knowledge. Second, we propose a data augmentation strategy based on human feedback, incorporating high-quality samples generated early in the fine-tuning process into the training set to facilitate progressive learning and significantly reduce its overfitting. Finally, we design a training-free triple diffusion process using the fine-tuned model, which manipulates the features of self-attention layers in a manner similar to the cross-attention mechanism. Specifically, in the generation process, the key and value of the content-related process are replaced with those of the style-related process to inject style while maintaining text control over the model. We also introduce query preservation to mitigate disruptions to the original content. Under such a design, we have achieved high-quality image-driven style transfer and text-driven stylization, delivering artist-level style transfer results while preserving the original image content. Moreover, we achieve image color editing during the style transfer process for the first time.
• 摘要：在这项开创性的研究中，我们介绍了StyleWallFacer，这是一个开创性的统一培训和推理框架，它不仅解决了传统方法的样式转移过程中遇到的各种问题，而且还统一了不同任务的框架。该框架旨在通过实现艺术家级别的风格转移和文本驱动的风格化来彻底改变该领域。首先，我们提出了一种基于语义的样式注入方法，该方法使用BLIP生成与剪辑空间中样式图像的语义严格一致的文本说明。通过利用大型语言模型从这些描述中删除与样式相关的描述，我们创建了语义差距。然后，该差距用于微调模型，从而实现了风格知识的有效且无漂移的注入。其次，我们提出了一种基于人类反馈的数据增强策略，将微调过程早期生成的高质量样本纳入培训集中，以促进渐进性学习并大大减少其过度拟合。最后，我们使用微型模型设计了无训练的三重扩散过程，该模型以类似于交叉注意机制的方式来操纵自我注意力层的特征。具体而言，在生成过程中，与内容相关的过程的关键和价值被与样式相关的过程所取代的过程，同时维持对模型的文本控制。我们还引入查询保存，以减轻对原始内容的干扰。在这样的设计下，我们实现了高质量的图像驱动样式传输和文本驱动的风格，从而在保留原始图像内容的同时，提供了艺术家级风格的转换结果。此外，我们首次在样式传输过程中实现图像颜色编辑。

Title: Sequential Policy Gradient for Adaptive Hyperparameter Optimization

• Authors: Zheng Li, Jerry Cheng, Huanying Helen Gu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.15051
• Pdf URL: https://arxiv.org/pdf/2506.15051
• Copy Paste: [[2506.15051]] Sequential Policy Gradient for Adaptive Hyperparameter Optimization(https://arxiv.org/abs/2506.15051)
• Keywords: generation
• Abstract: Reinforcement learning is essential for neural architecture search and hyperparameter optimization, but the conventional approaches impede widespread use due to prohibitive time and computational costs. Inspired by DeepSeek-V3 multi-token prediction architecture, we propose Sequential Policy Gradient modeling (SPG), a novel trajectory generation paradigm for lightweight online hyperparameter optimization. In contrast to conventional policy gradient methods, SPG extends the base model with temporary modules, enabling it to generate state-action (padded) trajectories in a single forward pass. Our experiments demonstrate that models gain performance when retrained with SPG on their original datasets and also outperform standard transfer fine-tuning. We evaluate on five datasets spanning computer vision (ImageNet, COCO), natural language processing (GLUE, SQuAD), and audio (SUPERB) to assess the industrial applicability of SPG. The proposed method demonstrates consistent improvements across widely adopted models, achieving performance gains of $+0.2\sim7\%$, with significantly low computational costs. Fully reproducible code and pre-trained models: this https URL.
• 摘要：强化学习对于神经体系结构搜索和超参数优化至关重要，但是由于时间和计算成本，常规方法阻碍了广泛使用。受DeepSeek-V3多token预测体系结构的启发，我们提出了顺序策略梯度建模（SPG），这是一种新型的轨迹生成范式，用于轻量化在线在线超参数优化。与常规的策略梯度方法相反，SPG使用临时模块扩展了基本模型，从而使其能够在单个正向通行证中生成状态行动（填充）轨迹。我们的实验表明，当用SPG在其原始数据集上进行重新培训时，模型可以提高性能，并且表现优于标准转移微调。我们在跨越计算机视觉的五个数据集（ImageNet，Coco），自然语言处理（胶水，小队）和音频（SUPERB）上评估，以评估SPG的工业适用性。提出的方法表明，在广泛采用的模型中，取得了一致的改进，实现了$+0.2 \ sim7 \％$的性能增长，计算成本显着降低。完全可重现的代码和预训练的模型：此HTTPS URL。

Title: Neural Canonical Polyadic Factorization for Traffic Analysis

• Authors: Yikai Hou, Peng Tang
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2506.15079
• Pdf URL: https://arxiv.org/pdf/2506.15079
• Copy Paste: [[2506.15079]] Neural Canonical Polyadic Factorization for Traffic Analysis(https://arxiv.org/abs/2506.15079)
• Keywords: generation
• Abstract: Modern intelligent transportation systems rely on accurate spatiotemporal traffic analysis to optimize urban mobility and infrastructure resilience. However, pervasive missing data caused by sensor failures and heterogeneous sensing gaps fundamentally hinders reliable traffic modeling. This paper proposes a Neural Canonical Polyadic Factorization (NCPF) model that synergizes low-rank tensor algebra with deep representation learning for robust traffic data imputation. The model innovatively embeds CP decomposition into neural architecture through learnable embedding projections, where sparse traffic tensors are encoded into dense latent factors across road segments, time intervals, and mobility metrics. A hierarchical feature fusion mechanism employs Hadamard products to explicitly model multilinear interactions, while stacked multilayer perceptron layers nonlinearly refine these representations to capture complex spatiotemporal couplings. Extensive evaluations on six urban traffic datasets demonstrate NCPF's superiority over six state-of-the-art baselines. By unifying CP decomposition's interpretable factor analysis with neural network's nonlinear expressive power, NCPF provides a principled yet flexible approaches for high-dimensional traffic data imputation, offering critical support for next-generation transportation digital twins and adaptive traffic control systems.
• 摘要：现代的智能运输系统依靠准确的时空交通分析来优化城市流动性和基础设施的弹性。但是，传感器故障和异质感应差距从根本上阻碍了可靠的交通建模引起的普遍缺失的数据。本文提出了一个神经规范的多核分解（NCPF）模型，该模型协同稳定的交通数据插值以深层表示，使低级张量代数协同。该模型通过可学习的预测将CP分解嵌入神经架构中，在该预测中，稀疏的交通张量被编码为道路细分市场，时间间隔和移动性指标的密集潜在因素。分层特征融合机制采用Hadamard产品明确对多线性相互作用进行建模，而堆叠的多层感知层则非线性地完善这些表示形式以捕获复杂的时空耦合。对六个城市交通数据集的广泛评估表明，NCPF优于六个最先进的基线。 NCPF通过神经网络的非线性表达能力统一CP分解的可解释因素分析，为高维交通数据插补提供了一种原则但灵活的方法，为下一代运输数字双胞胎和自适应交通控制系统提供了重要的支持。

Title: ReSeDis: A Dataset for Referring-based Object Search across Large-Scale Image Collections

• Authors: Ziling Huang, Yidan Zhang, Shin'ichi Satoh
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.15180
• Pdf URL: https://arxiv.org/pdf/2506.15180
• Copy Paste: [[2506.15180]] ReSeDis: A Dataset for Referring-based Object Search across Large-Scale Image Collections(https://arxiv.org/abs/2506.15180)
• Keywords: generation
• Abstract: Large-scale visual search engines are expected to solve a dual problem at once: (i) locate every image that truly contains the object described by a sentence and (ii) identify the object's bounding box or exact pixels within each hit. Existing techniques address only one side of this challenge. Visual grounding yields tight boxes and masks but rests on the unrealistic assumption that the object is present in every test image, producing a flood of false alarms when applied to web-scale collections. Text-to-image retrieval excels at sifting through massive databases to rank relevant images, yet it stops at whole-image matches and offers no fine-grained localization. We introduce Referring Search and Discovery (ReSeDis), the first task that unifies corpus-level retrieval with pixel-level grounding. Given a free-form description, a ReSeDis model must decide whether the queried object appears in each image and, if so, where it is, returning bounding boxes or segmentation masks. To enable rigorous study, we curate a benchmark in which every description maps uniquely to object instances scattered across a large, diverse corpus, eliminating unintended matches. We further design a task-specific metric that jointly scores retrieval recall and localization precision. Finally, we provide a straightforward zero-shot baseline using a frozen vision-language model, revealing significant headroom for future study. ReSeDis offers a realistic, end-to-end testbed for building the next generation of robust and scalable multimodal search systems.
• 摘要：大规模的视觉搜索引擎有望立即解决双重问题：（i）找到真正包含句子描述的对象的每个图像，以及（ii）标识每个命中中对象的边界框或精确像素。现有技术仅解决此挑战的一方面。视觉接地会产生紧密的盒子和口罩，但基于一个不切实际的假设，即每个测试图像中都存在对象，当应用于网络规模的集合时会产生大量错误警报。文本对图像检索在筛选大量数据库方面表现出色，以对相关图像进行排名，但它在整个图像匹配中停止，并且没有提供细粒度的本地化。我们介绍推荐搜索和发现（resedis），这是将语料库级检索与像素级接地统一的第一个任务。给定自由形式的描述，resedis模型必须确定查询对象是否出现在每个图像中，如果是的，则是返回边界框或分段掩码的位置。为了实现严格的研究，我们策划了一个基准测试，在该基准中，每个描述都独特地映射到散布在大型，多样的语料库中的对象实例，消除了意外的匹配。我们进一步设计了一个特定于任务的指标，该指标共同评分检索召回和本地化精度。最后，我们使用冷冻视觉模型提供直接的零射基线，揭示了未来研究的重要余地。 Resedis提供了一个现实的，端到端的测试床，用于构建下一代强大而可扩展的多模式搜索系统。

Title: Learning Task-Agnostic Skill Bases to Uncover Motor Primitives in Animal Behaviors

• Authors: Jiyi Wang, Jingyang Ke, Bo Dai, Anqi Wu
• Subjects: cs.LG, q-bio.NC
• Abstract URL: https://arxiv.org/abs/2506.15190
• Pdf URL: https://arxiv.org/pdf/2506.15190
• Copy Paste: [[2506.15190]] Learning Task-Agnostic Skill Bases to Uncover Motor Primitives in Animal Behaviors(https://arxiv.org/abs/2506.15190)
• Keywords: generation, generative
• Abstract: Animals flexibly recombine a finite set of core motor primitives to meet diverse task demands, but existing behavior-segmentation methods oversimplify this process by imposing discrete syllables under restrictive generative assumptions. To reflect the animal behavior generation procedure, we introduce skill-based imitation learning (SKIL) for behavior understanding, a reinforcement learning-based imitation framework that (1) infers interpretable skill sets, i.e., latent basis functions of behavior, by leveraging representation learning on transition probabilities, and (2) parameterizes policies as dynamic mixtures of these skills. We validate our approach on a simple grid world, a discrete labyrinth, and unconstrained videos of freely moving animals. Across tasks, it identifies reusable skill components, learns continuously evolving compositional policies, and generates realistic trajectories beyond the capabilities of traditional discrete models. By exploiting generative behavior modeling with compositional representations, our method offers a concise, principled account of how complex animal behaviors emerge from dynamic combinations of fundamental motor primitives.
• 摘要：动物可以灵活地重组一组有限的核心运动原语，以满足各种任务需求，但是现有的行为分割方法通过在限制性生成假设下施加离散的音节来过度简化此过程。为了反映动物行为的产生程序，我们引入了基于技能的模仿学习（SKIL），以了解行为理解，这是一个基于强化学习的模仿框架，（1）通过利用代表性学习在过渡概率上，将可解释的技能集，即行为的潜在基础功能，以及（2）将政策作为这些技能的动态混合而将政策作为动态混合。我们在简单的网格世界中验证了我们的方法，一个离散的迷宫以及​​自由移动动物的无约束视频。在整个任务中，它可以确定可重复使用的技能组件，学习不断发展的组成政策，并产生超出传统离散模型功能的现实轨迹。通过用组成表示来利用生成行为建模，我们的方法提供了一个简洁的，原则上说明复杂的动物行为如何从基本运动原始基原始原始原始动态组合中产生的复杂行为。

Title: Privacy-Shielded Image Compression: Defending Against Exploitation from Vision-Language Pretrained Models

• Authors: Xuelin Shen, Jiayin Xu, Kangsheng Yin, Wenhan Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.15201
• Pdf URL: https://arxiv.org/pdf/2506.15201
• Copy Paste: [[2506.15201]] Privacy-Shielded Image Compression: Defending Against Exploitation from Vision-Language Pretrained Models(https://arxiv.org/abs/2506.15201)
• Keywords: generation
• Abstract: The improved semantic understanding of vision-language pretrained (VLP) models has made it increasingly difficult to protect publicly posted images from being exploited by search engines and other similar tools. In this context, this paper seeks to protect users' privacy by implementing defenses at the image compression stage to prevent exploitation. Specifically, we propose a flexible coding method, termed Privacy-Shielded Image Compression (PSIC), that can produce bitstreams with multiple decoding options. By default, the bitstream is decoded to preserve satisfactory perceptual quality while preventing interpretation by VLP models. Our method also retains the original image compression functionality. With a customizable input condition, the proposed scheme can reconstruct the image that preserves its full semantic information. A Conditional Latent Trigger Generation (CLTG) module is proposed to produce bias information based on customizable conditions to guide the decoding process into different reconstructed versions, and an Uncertainty-Aware Encryption-Oriented (UAEO) optimization function is designed to leverage the soft labels inferred from the target VLP model's uncertainty on the training data. This paper further incorporates an adaptive multi-objective optimization strategy to obtain improved encrypting performance and perceptual quality simultaneously within a unified training process. The proposed scheme is plug-and-play and can be seamlessly integrated into most existing Learned Image Compression (LIC) models. Extensive experiments across multiple downstream tasks have demonstrated the effectiveness of our design.
• 摘要：对视觉识别（VLP）模型的语义理解的改进，使得保护公开张贴的图像免受搜索引擎和其他类似工具的利用变得越来越困难。在这种情况下，本文试图通过在图像压缩阶段实施防御来保护用户的隐私以防止剥削。具体而言，我们提出了一种灵活的编码方法，该方法称为隐私屏蔽图像压缩（PSIC），该方法可以产生带有多个解码选项的Bitstreams。默认情况下，对Bitstream的解码是为了保留令人满意的感知质量，同时可以通过VLP模型进行解释。我们的方法还保留原始图像压缩功能。使用可自定义的输入条件，建议的方案可以重建保留其完整语义信息的图像。提出了条件潜在的触发器产生（CLTG）模块，以根据可自定义条件产生偏见信息，以将解码过程引导到不同的重建版本中，并且不确定性的加密为导向的（UAEA）优化功能旨在利用目标VLP模型对培训数据的不确定性的软标签，以利用培训数据中推断出的软标签。本文进一步结合了一种自适应多目标优化策略，以在统一的培训过程中同时获得改进的加密性能和感知质量。所提出的方案是插件，可以无缝集成到大多数现有的学到的图像压缩（LIC）模型中。跨多个下游任务进行的广泛实验证明了我们的设计有效性。

Title: Singular Value Decomposition on Kronecker Adaptation for Large Language Model

• Authors: Yee Hin Chong, Peng Qu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.15251
• Pdf URL: https://arxiv.org/pdf/2506.15251
• Copy Paste: [[2506.15251]] Singular Value Decomposition on Kronecker Adaptation for Large Language Model(https://arxiv.org/abs/2506.15251)
• Keywords: generation
• Abstract: Large pre-trained Transformer models achieve state-of-the-art results across diverse language and reasoning tasks, but full fine-tuning incurs substantial storage, memory, and computational overhead. Parameter-efficient fine-tuning (PEFT) methods mitigate these costs by learning only a small subset of task-specific parameters, yet existing approaches either introduce inference-time latency (adapter modules), suffer from suboptimal convergence (randomly initialized low-rank updates), or rely on fixed rank choices that may not match task complexity (Kronecker-based decompositions). We propose SoKA (SVD on Kronecker Adaptation), a novel PEFT strategy that combines Kronecker-product tensor factorization with SVD-driven initialization and spectrum-aware dynamic rank selection. Our Kronecker-Product SVD (KPSVD) procedure extracts principal components of the full weight update into compact Kronecker factors, while an adaptive rank selection algorithm uses energy-threshold and elbow-point criteria to prune negligible components. Empirical evaluation on LLaMA2-7B across arithmetic reasoning (GSM8K), formal mathematics (MATH), and code generation (MBPP) demonstrates that SoKA requires only 0.99M trainable parameters, 25% fewer than LoRA/PiSSA, while matching or exceeding baseline performance. Moreover, SoKA exhibits faster convergence and more stable gradients, highlighting its robustness and efficiency for large-scale model adaptation.
• 摘要：大型预训练的变压器模型在各种语言和推理任务中实现了最新的结果，但是完整的微调会带来大量的存储，内存和计算开销。 Parameter-efficient fine-tuning (PEFT) methods mitigate these costs by learning only a small subset of task-specific parameters, yet existing approaches either introduce inference-time latency (adapter modules), suffer from suboptimal convergence (randomly initialized low-rank updates), or rely on fixed rank choices that may not match task complexity (Kronecker-based decompositions).我们提出了SOKA（Kronecker Adaptation的SVD），这是一种新型的PEFT策略，将Kronecker-Prododuct张量分解与SVD驱动的初始化和Spectrum-Aware-Aware Aware Dynamic Rank选择相结合。我们的Kronecker-Product SVD（KPSVD）程序提取了全权重更新的主要成分，即紧凑型Kronecker因子，而自适应等级选择算法则使用能量阈值和肘部标准来修剪可忽视的组件。跨算术推理（GSM8K），正式数学（数学）和代码生成（MBPP）对LLAMA2-7B的经验评估表明，SOKA仅需要99万个可训练的参数，比Lora/PISSA少25％，而匹配或超过基线表现。此外，索卡表现出更快的收敛性和更稳定的梯度，突出了其稳健性和效率，以适应大规模模型。

Title: MEGC2025: Micro-Expression Grand Challenge on Spot Then Recognize and Visual Question Answering

• Authors: Xinqi Fan, Jingting Li, John See, Moi Hoon Yap, Wen-Huang Cheng, Xiaobai Li, Xiaopeng Hong, Su-Jing Wang, Adrian K. Davision
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2506.15298
• Pdf URL: https://arxiv.org/pdf/2506.15298
• Copy Paste: [[2506.15298]] MEGC2025: Micro-Expression Grand Challenge on Spot Then Recognize and Visual Question Answering(https://arxiv.org/abs/2506.15298)
• Keywords: generation
• Abstract: Facial micro-expressions (MEs) are involuntary movements of the face that occur spontaneously when a person experiences an emotion but attempts to suppress or repress the facial expression, typically found in a high-stakes environment. In recent years, substantial advancements have been made in the areas of ME recognition, spotting, and generation. However, conventional approaches that treat spotting and recognition as separate tasks are suboptimal, particularly for analyzing long-duration videos in realistic settings. Concurrently, the emergence of multimodal large language models (MLLMs) and large vision-language models (LVLMs) offers promising new avenues for enhancing ME analysis through their powerful multimodal reasoning capabilities. The ME grand challenge (MEGC) 2025 introduces two tasks that reflect these evolving research directions: (1) ME spot-then-recognize (ME-STR), which integrates ME spotting and subsequent recognition in a unified sequential pipeline; and (2) ME visual question answering (ME-VQA), which explores ME understanding through visual question answering, leveraging MLLMs or LVLMs to address diverse question types related to MEs. All participating algorithms are required to run on this test set and submit their results on a leaderboard. More details are available at this https URL.
• 摘要：面部微表达（MES）是面部的非自愿运动，当一个人经历情绪但试图抑制或压抑面部表情时，它通常是在高风险环境中发现的。近年来，在我的认可，斑点和一代方面取得了重大进步。但是，将斑点和识别视为单独任务的传统方法是次优的，特别是用于分析现实环境中的长期视频。同时，多模式大语言模型（MLLM）和大型视觉模型（LVLMS）的出现提供了有希望的新途径，以通过其强大的多模式推理能力来增强我的分析。 ME Grand Challenge（MEGC）2025介绍了两项反映了这些不断发展的研究方向的任务：（1）我点点 - 然后认可（ME-STR），这将我集成了我的发现和随后的识别，并在统一的顺序管道中； （2）我的视觉问题回答（me-vqa），它通过视觉问题回答，利用mllms或lvlms来探讨我的理解，以解决与MES相关的各种问题类型。所有参与的算法都必须在此测试集上运行，并将其结果提交在排行榜上。此HTTPS URL提供了更多详细信息。

Title: Conditional Generative Modeling for Enhanced Credit Risk Management in Supply Chain Finance

• Authors: Qingkai Zhang, L. Jeff Hong, Houmin Yan
• Subjects: cs.LG, q-fin.RM
• Abstract URL: https://arxiv.org/abs/2506.15305
• Pdf URL: https://arxiv.org/pdf/2506.15305
• Copy Paste: [[2506.15305]] Conditional Generative Modeling for Enhanced Credit Risk Management in Supply Chain Finance(https://arxiv.org/abs/2506.15305)
• Keywords: generative
• Abstract: The rapid expansion of cross-border e-commerce (CBEC) has created significant opportunities for small and medium-sized enterprises (SMEs), yet financing remains a critical challenge due to SMEs' limited credit histories. Third-party logistics (3PL)-led supply chain finance (SCF) has emerged as a promising solution, leveraging in-transit inventory as collateral. We propose an advanced credit risk management framework tailored for 3PL-led SCF, addressing the dual challenges of credit risk assessment and loan size determination. Specifically, we leverage conditional generative modeling of sales distributions through Quantile-Regression-based Generative Metamodeling (QRGMM) as the foundation for risk estimation. We propose a unified framework that enables flexible estimation of multiple risk measures while introducing a functional risk measure formulation that systematically captures the relationship between these risk measures and varying loan levels, supported by theoretical guarantees. To capture complex covariate interactions in e-commerce sales data, we integrate QRGMM with Deep Factorization Machines (DeepFM). Extensive experiments on synthetic and real-world data validate the efficacy of our model for credit risk assessment and loan size determination. This study represents a pioneering application of generative AI in CBEC SCF risk management, offering a solid foundation for enhanced credit practices and improved SME access to capital.
• 摘要：跨境电子商务（CBEC）的迅速扩张为中小型企业（中小企业）创造了巨大的机会，但是由于中小企业的信用历史有限，融资仍然是一项关键挑战。第三方物流（3PL）领导的供应链融资（SCF）已成为有前途的解决方案，利用透视库存为抵押品。我们提出了一个针对3PL领导的SCF量身定制的高级信用风险管理框架，以解决信用风险评估和贷款规模确定的双重挑战。具体而言，我们通过基于分位数的生成元模型（QRGMM）来利用销售分布的条件生成建模作为风险估计的基础。我们提出了一个统一的框架，该框架可以灵活地估算多种风险措施，同时引入功能性风险措施公式，该公式系统地捕获了这些风险措施与不同的贷款水平之间的关系，并得到理论保证的支持。为了捕获电子商务销售数据中复杂的协变量相互作用，我们将QRGMM与深层分解机（DEEPFM）集成在一起。关于合成和现实数据数据的广泛实验验证了我们模型对信用风险评估和贷款规模确定的功效。这项研究代表了生成AI在CBEC SCF风险管理中的开创性应用，为增强信贷实践和改善中小企业获得资本的稳固基础。

Title: Active Learning-Guided Seq2Seq Variational Autoencoder for Multi-target Inhibitor Generation

• Authors: Júlia Vilalta-Mor, Alexis Molina, Laura Ortega Varga, Isaac Filella-Merce, Victor Guallar
• Subjects: cs.LG, cs.AI, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2506.15309
• Pdf URL: https://arxiv.org/pdf/2506.15309
• Copy Paste: [[2506.15309]] Active Learning-Guided Seq2Seq Variational Autoencoder for Multi-target Inhibitor Generation(https://arxiv.org/abs/2506.15309)
• Keywords: generation
• Abstract: Simultaneously optimizing molecules against multiple therapeutic targets remains a profound challenge in drug discovery, particularly due to sparse rewards and conflicting design constraints. We propose a structured active learning (AL) paradigm integrating a sequence-to-sequence (Seq2Seq) variational autoencoder (VAE) into iterative loops designed to balance chemical diversity, molecular quality, and multi-target affinity. Our method alternates between expanding chemically feasible regions of latent space and progressively constraining molecules based on increasingly stringent multi-target docking thresholds. In a proof-of-concept study targeting three related coronavirus main proteases (SARS-CoV-2, SARS-CoV, MERS-CoV), our approach efficiently generated a structurally diverse set of pan-inhibitor candidates. We demonstrate that careful timing and strategic placement of chemical filters within this active learning pipeline markedly enhance exploration of beneficial chemical space, transforming the sparse-reward, multi-objective drug design problem into an accessible computational task. Our framework thus provides a generalizable roadmap for efficiently navigating complex polypharmacological landscapes.
• 摘要：同时针对多种治疗靶标优化分子仍然是药物发现中的深刻挑战，尤其是由于稀疏的奖励和矛盾的设计限制。我们提出了一个结构化的主动学习（AL）范式，该范式整合了序列到序列（SEQ2SEQ）变化自动编码器（VAE）中的迭代回路，旨在平衡化学多样性，分子质量和多目标亲和力。我们的方法基于越来越严格的多目标对接阈值，在扩展潜在空间的化学可行区域和逐步约束分子之间进行了交替。在针对三个相关冠状病毒主要蛋白酶（SARS-COV-2，SARS-COV，MERS-COV）的概念验证研究中，我们的方法有效地产生了一组结构上多样的泛抑制剂候选物。我们证明，在此主动学习管道中仔细的时机和化学过滤器的策略放置明显增强了对有益的化学空间的探索，将稀疏的奖励，多目标的药物设计问题转化为可访问的计算任务。因此，我们的框架提供了可有效浏览复杂多药理学景观的可推广路线图。

Title: MapFM: Foundation Model-Driven HD Mapping with Multi-Task Contextual Learning

• Authors: Leonid Ivanov, Vasily Yuryev, Dmitry Yudin
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.15313
• Pdf URL: https://arxiv.org/pdf/2506.15313
• Copy Paste: [[2506.15313]] MapFM: Foundation Model-Driven HD Mapping with Multi-Task Contextual Learning(https://arxiv.org/abs/2506.15313)
• Keywords: generation
• Abstract: In autonomous driving, high-definition (HD) maps and semantic maps in bird's-eye view (BEV) are essential for accurate localization, planning, and decision-making. This paper introduces an enhanced End-to-End model named MapFM for online vectorized HD map generation. We show significantly boost feature representation quality by incorporating powerful foundation model for encoding camera images. To further enrich the model's understanding of the environment and improve prediction quality, we integrate auxiliary prediction heads for semantic segmentation in the BEV representation. This multi-task learning approach provides richer contextual supervision, leading to a more comprehensive scene representation and ultimately resulting in higher accuracy and improved quality of the predicted vectorized HD maps. The source code is available at this https URL.
• 摘要：在自主驾驶中，在鸟眼视图（BEV）中，高清（HD）地图和语义图对于准确的定位，计划和决策至关重要。本文介绍了一个名为MAPFM的增强端到端模型，用于在线矢量化高清地图生成。通过合并用于编码相机图像的强大基础模型，我们可以显着提高功能表示质量。为了进一步丰富模型对环境的理解并提高预测质量，我们整合了辅助预测头，以在BEV表示中进行语义分割。这种多任务学习方法提供了更丰富的上下文监督，从而导致了更全面的场景表示形式，并最终导致了更高的准确性和提高预测的矢量化HD地图的质量。源代码可在此HTTPS URL上找到。

Title: Knowledge Distillation Framework for Accelerating High-Accuracy Neural Network-Based Molecular Dynamics Simulations

• Authors: Naoki Matsumura, Yuta Yoshimoto, Yuto Iwasaki, Meguru Yamazaki, Yasufumi Sakai
• Subjects: cs.LG, cond-mat.mtrl-sci, physics.comp-ph
• Abstract URL: https://arxiv.org/abs/2506.15337
• Pdf URL: https://arxiv.org/pdf/2506.15337
• Copy Paste: [[2506.15337]] Knowledge Distillation Framework for Accelerating High-Accuracy Neural Network-Based Molecular Dynamics Simulations(https://arxiv.org/abs/2506.15337)
• Keywords: generation
• Abstract: Neural network potentials (NNPs) offer a powerful alternative to traditional force fields for molecular dynamics (MD) simulations. Accurate and stable MD simulations, crucial for evaluating material properties, require training data encompassing both low-energy stable structures and high-energy structures. Conventional knowledge distillation (KD) methods fine-tune a pre-trained NNP as a teacher model to generate training data for a student model. However, in material-specific models, this fine-tuning process increases energy barriers, making it difficult to create training data containing high-energy structures. To address this, we propose a novel KD framework that leverages a non-fine-tuned, off-the-shelf pre-trained NNP as a teacher. Its gentler energy landscape facilitates the exploration of a wider range of structures, including the high-energy structures crucial for stable MD simulations. Our framework employs a two-stage training process: first, the student NNP is trained with a dataset generated by the off-the-shelf teacher; then, it is fine-tuned with a smaller, high-accuracy density functional theory (DFT) dataset. We demonstrate the effectiveness of our framework by applying it to both organic (polyethylene glycol) and inorganic (L$_{10}$GeP$_{2}$S$_{12}$) materials, achieving comparable or superior accuracy in reproducing physical properties compared to existing methods. Importantly, our method reduces the number of expensive DFT calculations by 10x compared to existing NNP generation methods, without sacrificing accuracy.
• 摘要：神经网络电位（NNP）为分子动力学（MD）模拟提供了有力的替代方案。准确稳定的MD模拟，对于评估材料特性至关重要，需要训练数据，包括低能稳定结构和高能量结构。常规知识蒸馏（KD）方法将预先训练的NNP微调作为教师模型，以生成学生模型的培训数据。但是，在特定于材料的模型中，这种微调过程增加了能源障碍，因此难以创建含有高能量结构的训练数据。为了解决这个问题，我们提出了一个新颖的KD框架，该框架利用了一个非精致的，现成的预先培训的NNP作为老师。它更柔和的能源景观有助于探索更广泛的结构，包括对稳定的MD模拟至关重要的高能量结构。我们的框架采用了两个阶段的培训过程：首先，学生NNP接受了由现成的老师生成的数据集培训；然后，它通过较小的高临界度密度理论（DFT）数据集进行了微调。我们通过将框架应用于有机（聚乙烯乙二醇）和无机（l $ _ {10} $ gep $ _ {2} $ s $ _ {12} $）材料中，证明了框架的有效性，与现有方法相比，可以实现可比较或卓越的准确性。重要的是，与现有的NNP生成方法相比，我们的方法将昂贵的DFT计算的数量减少了10倍，而无需牺牲准确性。

Title: Sampling 3D Molecular Conformers with Diffusion Transformers

• Authors: J. Thorben Frank, Winfried Ripken, Gregor Lied, Klaus-Robert Müller, Oliver T. Unke, Stefan Chmiela
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.15378
• Pdf URL: https://arxiv.org/pdf/2506.15378
• Copy Paste: [[2506.15378]] Sampling 3D Molecular Conformers with Diffusion Transformers(https://arxiv.org/abs/2506.15378)
• Keywords: generation, generative
• Abstract: Diffusion Transformers (DiTs) have demonstrated strong performance in generative modeling, particularly in image synthesis, making them a compelling choice for molecular conformer generation. However, applying DiTs to molecules introduces novel challenges, such as integrating discrete molecular graph information with continuous 3D geometry, handling Euclidean symmetries, and designing conditioning mechanisms that generalize across molecules of varying sizes and structures. We propose DiTMC, a framework that adapts DiTs to address these challenges through a modular architecture that separates the processing of 3D coordinates from conditioning on atomic connectivity. To this end, we introduce two complementary graph-based conditioning strategies that integrate seamlessly with the DiT architecture. These are combined with different attention mechanisms, including both standard non-equivariant and SO(3)-equivariant formulations, enabling flexible control over the trade-off between between accuracy and computational efficiency. Experiments on standard conformer generation benchmarks (GEOM-QM9, -DRUGS, -XL) demonstrate that DiTMC achieves state-of-the-art precision and physical validity. Our results highlight how architectural choices and symmetry priors affect sample quality and efficiency, suggesting promising directions for large-scale generative modeling of molecular structures. Code available at this https URL.
• 摘要：扩散变压器（DIT）在生成建模中表现出强烈的性能，尤其是在图像合成中，使其成为分子构象异构体产生的令人信服的选择。但是，将DIT应用于分子会引入新的挑战，例如将离散的分子图信息与连续的3D几何形状相结合，处理欧几里得对称性以及设计条件机制，这些机制跨越了各种大小和结构的分子。我们提出了DITMC，该框架通过模块化体系结构进行调整以应对这些挑战，该模块化体系结构将3D坐标的处理与原子连接的条件分开。为此，我们介绍了两种基于图形的调节策略，它们与DIT体系结构无缝集成。这些结合了不同的注意机制，包括标准的非等价式和（3）均衡配方，从而可以灵活控制准确性和计算效率之间的权衡。对标准构象产生基准（GEOM-QM9，-DRUGS，-XL）的实验表明，DITMC实现了最先进的精度和物理有效性。我们的结果强调了建筑选择和对称先验如何影响样品质量和效率，这表明了分子结构大规模生成建模的有希望的方向。可在此HTTPS URL上找到代码。

Title: When Model Knowledge meets Diffusion Model: Diffusion-assisted Data-free Image Synthesis with Alignment of Domain and Class

• Authors: Yujin Kim, Hyunsoo Kim, Hyunwoo J.Kim, Suhyun Kim
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.15381
• Pdf URL: https://arxiv.org/pdf/2506.15381
• Copy Paste: [[2506.15381]] When Model Knowledge meets Diffusion Model: Diffusion-assisted Data-free Image Synthesis with Alignment of Domain and Class(https://arxiv.org/abs/2506.15381)
• Keywords: generation
• Abstract: Open-source pre-trained models hold great potential for diverse applications, but their utility declines when their training data is unavailable. Data-Free Image Synthesis (DFIS) aims to generate images that approximate the learned data distribution of a pre-trained model without accessing the original data. However, existing DFIS meth ods produce samples that deviate from the training data distribution due to the lack of prior knowl edge about natural images. To overcome this limitation, we propose DDIS, the first Diffusion-assisted Data-free Image Synthesis method that leverages a text-to-image diffusion model as a powerful image prior, improving synthetic image quality. DDIS extracts knowledge about the learned distribution from the given model and uses it to guide the diffusion model, enabling the generation of images that accurately align with the training data distribution. To achieve this, we introduce Domain Alignment Guidance (DAG) that aligns the synthetic data domain with the training data domain during the diffusion sampling process. Furthermore, we optimize a single Class Alignment Token (CAT) embedding to effectively capture class-specific attributes in the training dataset. Experiments on PACS and Ima geNet demonstrate that DDIS outperforms prior DFIS methods by generating samples that better reflect the training data distribution, achieving SOTA performance in data-free applications.
• 摘要：开源预培训的模型可用于不同的应用程序，但是当他们的培训数据不可用时，其实用性会下降。无数据图像合成（DFIS）旨在生成近似预训练模型的学习数据分布的图像，而无需访问原始数据。但是，由于缺乏关于自然图像的先验知识边缘，现有的DFIS甲基OD会产生偏离训练数据分布的样品。为了克服这一限制，我们提出了DDIS，这是第一个扩散辅助的无数据图像合成方法，该方法利用文本对图像扩散模型作为强大的图像先验，从而提高了合成图像质量。 DDIS提取了从给定模型中学到的分布的知识，并使用它来指导扩散模型，从而使图像的产生能够准确地与训练数据分布保持一致。为了实现这一目标，我们引入了在扩散采样过程中将合成数据域与训练数据域保持一致的域对准指南（DAG）。此外，我们优化了嵌入单个类对齐令牌（CAT），以有效捕获训练数据集中的特定类别属性。 PACS和IMA遗传的实验表明，DDI通过生成更好地反映训练数据分布的样本，在无数据应用中实现SOTA性能来优于先前的DFI方法。

Title: Provable Maximum Entropy Manifold Exploration via Diffusion Models

• Authors: Riccardo De Santi, Marin Vlastelica, Ya-Ping Hsieh, Zebang Shen, Niao He, Andreas Krause
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.15385
• Pdf URL: https://arxiv.org/pdf/2506.15385
• Copy Paste: [[2506.15385]] Provable Maximum Entropy Manifold Exploration via Diffusion Models(https://arxiv.org/abs/2506.15385)
• Keywords: generative
• Abstract: Exploration is critical for solving real-world decision-making problems such as scientific discovery, where the objective is to generate truly novel designs rather than mimic existing data distributions. In this work, we address the challenge of leveraging the representational power of generative models for exploration without relying on explicit uncertainty quantification. We introduce a novel framework that casts exploration as entropy maximization over the approximate data manifold implicitly defined by a pre-trained diffusion model. Then, we present a novel principle for exploration based on density estimation, a problem well-known to be challenging in practice. To overcome this issue and render this method truly scalable, we leverage a fundamental connection between the entropy of the density induced by a diffusion model and its score function. Building on this, we develop an algorithm based on mirror descent that solves the exploration problem as sequential fine-tuning of a pre-trained diffusion model. We prove its convergence to the optimal exploratory diffusion model under realistic assumptions by leveraging recent understanding of mirror flows. Finally, we empirically evaluate our approach on both synthetic and high-dimensional text-to-image diffusion, demonstrating promising results.
• 摘要：探索对于解决现实世界的决策问题（例如科学发现）至关重要，该问题的目的是生成真正的新颖设计而不是模仿现有的数据分布。在这项工作中，我们解决了利用生成模型的代表力进行探索的挑战，而无需依赖明确的不确定性量化。我们介绍了一个新颖的框架，该框架将探索作为熵最大化，这是通过预先训练的扩散模型隐含定义的近似数据歧管。然后，我们提出了一个基于密度估计的探索的新颖原则，这个问题在实践中众所周知。为了克服这个问题并渲染这种方法真正可扩展，我们利用了扩散模型引起的密度的熵与其得分函数之间的基本联系。在此基础上，我们开发了一种基于镜下降的算法，该算法解决了探索问题，作为预训练扩散模型的顺序微调。我们通过利用镜像流的最新理解来证明其在现实假设下与最佳探索性扩散模型的收敛。最后，我们对合成和高维文本对图扩散的凭经验评估了我们的方法，这表明了有希望的结果。

Title: Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Material

• Authors: Team Hunyuan3D, Shuhui Yang, Mingxin Yang, Yifei Feng, Xin Huang, Sheng Zhang, Zebin He, Di Luo, Haolin Liu, Yunfei Zhao, Qingxiang Lin, Zeqiang Lai, Xianghui Yang, Huiwen Shi, Zibo Zhao, Bowen Zhang, Hongyu Yan, Lifu Wang, Sicong Liu, Jihong Zhang, Meng Chen, Liang Dong, Yiwen Jia, Yulin Cai, Jiaao Yu, Yixuan Tang, Dongyuan Guo, Junlin Yu, Hao Zhang, Zheng Ye, Peng He, Runzhou Wu, Shida Wei, Chao Zhang, Yonghao Tan, Yifu Sun, Lin Niu, Shirui Huang, Bojian Zheng, Shu Liu, Shilin Chen, Xiang Yuan, Xiaofeng Yang, Kai Liu, Jianchen Zhu, Peng Chen, Tian Liu, Di Wang, Yuhong Liu, Linus, Jie Jiang, Jingwei Huang, Chunchao Guo
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.15442
• Pdf URL: https://arxiv.org/pdf/2506.15442
• Copy Paste: [[2506.15442]] Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Material(https://arxiv.org/abs/2506.15442)
• Keywords: generation, generative
• Abstract: 3D AI-generated content (AIGC) is a passionate field that has significantly accelerated the creation of 3D models in gaming, film, and design. Despite the development of several groundbreaking models that have revolutionized 3D generation, the field remains largely accessible only to researchers, developers, and designers due to the complexities involved in collecting, processing, and training 3D models. To address these challenges, we introduce Hunyuan3D 2.1 as a case study in this tutorial. This tutorial offers a comprehensive, step-by-step guide on processing 3D data, training a 3D generative model, and evaluating its performance using Hunyuan3D 2.1, an advanced system for producing high-resolution, textured 3D assets. The system comprises two core components: the Hunyuan3D-DiT for shape generation and the Hunyuan3D-Paint for texture synthesis. We will explore the entire workflow, including data preparation, model architecture, training strategies, evaluation metrics, and deployment. By the conclusion of this tutorial, you will have the knowledge to finetune or develop a robust 3D generative model suitable for applications in gaming, virtual reality, and industrial design.
• 摘要：3D AI生成的内容（AIGC）是一个充满激情的领域，它显着加速了游戏，电影和设计中的3D模型。尽管开发了几种彻底改变3D代的开创性模型，但由于收集，处理和培训3D模型所涉及的复杂性，该领域仍然可以在研究人员，开发人员和设计师方面取得很大的访问。为了应对这些挑战，我们将Hunyuan3d 2.1作为本教程中的案例研究。本教程提供了一项有关处理3D数据，培训3D生成模型的全面，分步的指南，并使用Hunyuan3d 2.1评估其性能，Hunyuan3d 2.1是一种用于生产高分辨率，质感3D资产的高级系统。该系统包括两个核心组成部分：用于形状生成的Hunyuan3D-DIT和用于纹理合成的Hunyuan3D-Paint。我们将探索整个工作流程，包括数据准备，模型架构，培训策略，评估指标和部署。通过本教程的结论，您将有知识来列出或开发适合游戏，虚拟现实和工业设计应用的强大3D生成模型。

Title: Multimodal Large Language Models for Medical Report Generation via Customized Prompt Tuning

• Authors: Chunlei Li, Jingyang Hou, Yilei Shi, Jingliang Hu, Xiao Xiang Zhu, Lichao Mou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.15477
• Pdf URL: https://arxiv.org/pdf/2506.15477
• Copy Paste: [[2506.15477]] Multimodal Large Language Models for Medical Report Generation via Customized Prompt Tuning(https://arxiv.org/abs/2506.15477)
• Keywords: generation
• Abstract: Medical report generation from imaging data remains a challenging task in clinical practice. While large language models (LLMs) show great promise in addressing this challenge, their effective integration with medical imaging data still deserves in-depth exploration. In this paper, we present MRG-LLM, a novel multimodal large language model (MLLM) that combines a frozen LLM with a learnable visual encoder and introduces a dynamic prompt customization mechanism. Our key innovation lies in generating instance-specific prompts tailored to individual medical images through conditional affine transformations derived from visual features. We propose two implementations: prompt-wise and promptbook-wise customization, enabling precise and targeted report generation. Extensive experiments on IU X-ray and MIMIC-CXR datasets demonstrate that MRG-LLM achieves state-of-the-art performance in medical report generation. Our code will be made publicly available.
• 摘要：在临床实践中，从成像数据中生成医疗报告仍然是一项具有挑战性的任务。尽管大型语言模型（LLM）在应对这一挑战方面表现出了巨大的希望，但它们与医学成像数据的有效整合仍然值得深入探索。在本文中，我们介绍了一种新型的多模式大语言模型（MLLM）MRG-LLM，该模型（MLLM）将冷冻LLM与可学习的视觉编码器结合在一起，并引入了动态及时的自定义机制。我们的关键创新在于通过通过视觉特征得出的条件仿射转换来生成针对单个医学图像的实例特定提示。我们提出了两个实现：及时和及时的定制，实现精确和有针对性的报告生成。关于IU X射线和MIMIC-CXR数据集的广泛实验表明，MRG-LLM在医疗报告生成中实现了最先进的表现。我们的代码将公开可用。

Title: GenHOI: Generalizing Text-driven 4D Human-Object Interaction Synthesis for Unseen Objects

• Authors: Shujia Li, Haiyu Zhang, Xinyuan Chen, Yaohui Wang, Yutong Ban
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.15483
• Pdf URL: https://arxiv.org/pdf/2506.15483
• Copy Paste: [[2506.15483]] GenHOI: Generalizing Text-driven 4D Human-Object Interaction Synthesis for Unseen Objects(https://arxiv.org/abs/2506.15483)
• Keywords: generation
• Abstract: While diffusion models and large-scale motion datasets have advanced text-driven human motion synthesis, extending these advances to 4D human-object interaction (HOI) remains challenging, mainly due to the limited availability of large-scale 4D HOI datasets. In our study, we introduce GenHOI, a novel two-stage framework aimed at achieving two key objectives: 1) generalization to unseen objects and 2) the synthesis of high-fidelity 4D HOI sequences. In the initial stage of our framework, we employ an Object-AnchorNet to reconstruct sparse 3D HOI keyframes for unseen objects, learning solely from 3D HOI datasets, thereby mitigating the dependence on large-scale 4D HOI datasets. Subsequently, we introduce a Contact-Aware Diffusion Model (ContactDM) in the second stage to seamlessly interpolate sparse 3D HOI keyframes into densely temporally coherent 4D HOI sequences. To enhance the quality of generated 4D HOI sequences, we propose a novel Contact-Aware Encoder within ContactDM to extract human-object contact patterns and a novel Contact-Aware HOI Attention to effectively integrate the contact signals into diffusion models. Experimental results show that we achieve state-of-the-art results on the publicly available OMOMO and 3D-FUTURE datasets, demonstrating strong generalization abilities to unseen objects, while enabling high-fidelity 4D HOI generation.
• 摘要：尽管扩散模型和大规模运动数据集具有先进的文本驱动人类运动合成，但将这些进步扩展到4D人类对象相互作用（HOI）仍然具有挑战性，这主要是由于大型4D HOI数据集的可用性有限。在我们的研究中，我们介绍了Genhoi，这是一个新型的两阶段框架，旨在实现两个关键目标：1）概括是看不见的对象和2）高保真4D HOI序列的合成。在框架的初始阶段，我们采用对象 - 兰板来重建稀疏的3D HOI钥匙帧，以使对象仅从3D HOI数据集中学习，从而减轻对大规模4D HOI数据集的依赖性。随后，我们在第二阶段引入了一个接触感知的扩散模型（ContactDM），以无缝插入稀疏的3D HOI键帧，以密集的时间连贯的4D HOI序列。为了提高生成的4D HOI序列的质量，我们提出了一个新颖的接触感知编码器，以提取人类对象的接触模式和新型的接触式HOI注意，以有效地将接触信号整合到扩散模型中。实验结果表明，我们在公开可用的OMOMO和3D-FUTURE数据集上实现了最先进的结果，这表明了强大的概括能力，可以看不见对象，同时启用了高保真4D HOI的生成。

Title: RePCS: Diagnosing Data Memorization in LLM-Powered Retrieval-Augmented Generation

• Authors: Le Vu Anh, Nguyen Viet Anh, Mehmet Dik, Luong Van Nghia
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.15513
• Pdf URL: https://arxiv.org/pdf/2506.15513
• Copy Paste: [[2506.15513]] RePCS: Diagnosing Data Memorization in LLM-Powered Retrieval-Augmented Generation(https://arxiv.org/abs/2506.15513)
• Keywords: generation
• Abstract: Retrieval-augmented generation (RAG) has become a common strategy for updating large language model (LLM) responses with current, external information. However, models may still rely on memorized training data, bypass the retrieved evidence, and produce contaminated outputs. We introduce Retrieval-Path Contamination Scoring (RePCS), a diagnostic method that detects such behavior without requiring model access or retraining. RePCS compares two inference paths: (i) a parametric path using only the query, and (ii) a retrieval-augmented path using both the query and retrieved context by computing the Kullback-Leibler (KL) divergence between their output distributions. A low divergence suggests that the retrieved context had minimal impact, indicating potential memorization. This procedure is model-agnostic, requires no gradient or internal state access, and adds only a single additional forward pass. We further derive PAC-style guarantees that link the KL threshold to user-defined false positive and false negative rates. On the Prompt-WNQA benchmark, RePCS achieves a ROC-AUC of 0.918. This result outperforms the strongest prior method by 6.5 percentage points while keeping latency overhead below 4.7% on an NVIDIA T4 GPU. RePCS offers a lightweight, black-box safeguard to verify whether a RAG system meaningfully leverages retrieval, making it especially valuable in safety-critical applications.
• 摘要：检索演示的生成（RAG）已成为使用当前外部信息更新大语言模型（LLM）响应的常见策略。但是，模型仍然可能依靠记忆的训练数据，绕过检索到的证据并产生受污染的产出。我们引入了检索路径污染评分（REPC），这是一种诊断方法，可检测此类行为而无需模型访问或重新训练。 RETCS比较了两个推理路径：（i）仅使用查询的参数路径，（ii）通过查询和检索上下文的检索仪，通过计算其输出分布之间的Kullback-Leibler（KL）差异。较低的差异表明检索到的上下文的影响很小，表明潜在的记忆。此过程是模型 - 不合时宜的，不需要梯度或内部状态访问，并且仅添加一个额外的前向通过。我们进一步得出PAC风格可以确保将KL阈值与用户定义的假阳性和假阴性率联系起来。在及时WNQA基准测试中，RETC的ROC-AUC为0.918。该结果的表现优于最强的先前方法，而在NVIDIA T4 GPU上，将延迟开销低于4.7％。 REPCS提供了轻巧的黑盒保护措施，以验证抹布系统是否有意义地利用了检索，使其在安全至关重要的应用中特别有价值。

Title: CLAIM: Clinically-Guided LGE Augmentation for Realistic and Diverse Myocardial Scar Synthesis and Segmentation

• Authors: Farheen Ramzan, Yusuf Kiberu, Nikesh Jathanna, Shahnaz Jamil-Copley, Richard H. Clayton, Chen (Cherise)Chen
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.15549
• Pdf URL: https://arxiv.org/pdf/2506.15549
• Copy Paste: [[2506.15549]] CLAIM: Clinically-Guided LGE Augmentation for Realistic and Diverse Myocardial Scar Synthesis and Segmentation(https://arxiv.org/abs/2506.15549)
• Keywords: generation
• Abstract: Deep learning-based myocardial scar segmentation from late gadolinium enhancement (LGE) cardiac MRI has shown great potential for accurate and timely diagnosis and treatment planning for structural cardiac diseases. However, the limited availability and variability of LGE images with high-quality scar labels restrict the development of robust segmentation models. To address this, we introduce CLAIM: \textbf{C}linically-Guided \textbf{L}GE \textbf{A}ugmentation for Real\textbf{i}stic and Diverse \textbf{M}yocardial Scar Synthesis and Segmentation framework, a framework for anatomically grounded scar generation and segmentation. At its core is the SMILE module (Scar Mask generation guided by cLinical knowledgE), which conditions a diffusion-based generator on the clinically adopted AHA 17-segment model to synthesize images with anatomically consistent and spatially diverse scar patterns. In addition, CLAIM employs a joint training strategy in which the scar segmentation network is optimized alongside the generator, aiming to enhance both the realism of synthesized scars and the accuracy of the scar segmentation performance. Experimental results show that CLAIM produces anatomically coherent scar patterns and achieves higher Dice similarity with real scar distributions compared to baseline models. Our approach enables controllable and realistic myocardial scar synthesis and has demonstrated utility for downstream medical imaging task.
• 摘要：深度学习的心肌疤痕分割从晚期增强（LGE）心脏MRI中，对结构性心脏病的准确及时诊断和治疗计划显示出很大的潜力。但是，具有高质量疤痕标签的LGE图像的有限可用性和可变性限制了强大的分割模型的发展。 To address this, we introduce CLAIM: \textbf{C}linically-Guided \textbf{L}GE \textbf{A}ugmentation for Real\textbf{i}stic and Diverse \textbf{M}yocardial Scar Synthesis and Segmentation framework, a framework for anatomically grounded scar generation and segmentation.核心是微笑模块（以临床知识为指导的疤痕掩模生成），该模块在临床上采用的AHA 17段模型上探讨了基于扩散的发电机，以综合具有解剖一致且空间上不同的疤痕模式的图像。此外，索赔还采用了联合培训策略，在该策略中，疤痕分割网络与发电机一起进行了优化，旨在增强合成疤痕的现实主义和疤痕细分性能的准确性。实验结果表明，与基线模型相比，主张会产生解剖上连贯的疤痕模式，并与实际疤痕分布相似。我们的方法可以使可控和逼真的心肌疤痕合成，并证明了下游医学成像任务的实用性。

Title: Control and Realism: Best of Both Worlds in Layout-to-Image without Training

• Authors: Bonan Li, Yinhan Hu, Songhua Liu, Xinchao Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.15563
• Pdf URL: https://arxiv.org/pdf/2506.15563
• Copy Paste: [[2506.15563]] Control and Realism: Best of Both Worlds in Layout-to-Image without Training(https://arxiv.org/abs/2506.15563)
• Keywords: generation
• Abstract: Layout-to-Image generation aims to create complex scenes with precise control over the placement and arrangement of subjects. Existing works have demonstrated that pre-trained Text-to-Image diffusion models can achieve this goal without training on any specific data; however, they often face challenges with imprecise localization and unrealistic artifacts. Focusing on these drawbacks, we propose a novel training-free method, WinWinLay. At its core, WinWinLay presents two key strategies, Non-local Attention Energy Function and Adaptive Update, that collaboratively enhance control precision and realism. On one hand, we theoretically demonstrate that the commonly used attention energy function introduces inherent spatial distribution biases, hindering objects from being uniformly aligned with layout instructions. To overcome this issue, non-local attention prior is explored to redistribute attention scores, facilitating objects to better conform to the specified spatial conditions. On the other hand, we identify that the vanilla backpropagation update rule can cause deviations from the pre-trained domain, leading to out-of-distribution artifacts. We accordingly introduce a Langevin dynamics-based adaptive update scheme as a remedy that promotes in-domain updating while respecting layout constraints. Extensive experiments demonstrate that WinWinLay excels in controlling element placement and achieving photorealistic visual fidelity, outperforming the current state-of-the-art methods.
• 摘要：布局到图像生成旨在创建复杂的场景，以精确控制对象的位置和安排。现有的作品表明，预先训练的文本对图像扩散模型可以实现此目标，而无需对任何特定数据培训。但是，他们经常以不精确的本地化和不切实际的人工制品面临挑战。为了关注这些弊端，我们提出了一种新颖的无培训方法Winwinlay。 Winwinlay的核心提出了两种关键策略，即非本地注意力能量功能和自适应更新，从而协作增强了控制精度和现实主义。一方面，我们从理论上证明了常用的注意力函数引入了固有的空间分布偏见，阻碍对象与布局指令一致一致。为了克服这个问题，探索了非本地关注以重新分布注意力评分，从而促进对象以更好地符合指定的空间条件。另一方面，我们确定香草反向传播更新规则可能会导致与预训练的域偏离，从而导致分布外部伪像。因此，我们引入了基于兰格文的自适应更新方案，作为一种补救措施，在尊重布局约束时促进内域更新。广泛的实验表明，温温莱（Winwinlay）在控制元素的放置和实现逼真的视觉保真度方面表现出色，表现优于当前的最新方法。

Title: Show-o2: Improved Native Unified Multimodal Models

• Authors: Jinheng Xie, Zhenheng Yang, Mike Zheng Shou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.15564
• Pdf URL: https://arxiv.org/pdf/2506.15564
• Copy Paste: [[2506.15564]] Show-o2: Improved Native Unified Multimodal Models(https://arxiv.org/abs/2506.15564)
• Keywords: generation
• Abstract: This paper presents improved native unified multimodal models, \emph{i.e.,} Show-o2, that leverage autoregressive modeling and flow matching. Built upon a 3D causal variational autoencoder space, unified visual representations are constructed through a dual-path of spatial (-temporal) fusion, enabling scalability across image and video modalities while ensuring effective multimodal understanding and generation. Based on a language model, autoregressive modeling and flow matching are natively applied to the language head and flow head, respectively, to facilitate text token prediction and image/video generation. A two-stage training recipe is designed to effectively learn and scale to larger models. The resulting Show-o2 models demonstrate versatility in handling a wide range of multimodal understanding and generation tasks across diverse modalities, including text, images, and videos. Code and models are released at this https URL.
• 摘要：本文提出了改进的本机统一多模型模型\ emph {i.e。，} show-o2，该模型利用自回旋建模和流量匹配。建立在3D因果变异自动编码器空间的基础上，统一的视觉表示是通过空间（ - 静态）融合的双路径构建的，在图像和视频模态上启用可扩展性，同时确保有效的多模式理解和产生。基于语言模型，分别将自回归建模和流程匹配分别用于语言头和流动头，以促进文本令牌预测和图像/视频生成。两阶段的培训配方旨在有效地学习和扩展到大型模型。由此产生的Show-O2模型在处理各种方式（包括文本，图像和视频）的多种多模式理解和生成任务方面表明了多功能性。代码和模型在此HTTPS URL上发布。

Title: One-Step Diffusion for Detail-Rich and Temporally Consistent Video Super-Resolution

• Authors: Yujing Sun, Lingchen Sun, Shuaizheng Liu, Rongyuan Wu, Zhengqiang Zhang, Lei Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.15591
• Pdf URL: https://arxiv.org/pdf/2506.15591
• Copy Paste: [[2506.15591]] One-Step Diffusion for Detail-Rich and Temporally Consistent Video Super-Resolution(https://arxiv.org/abs/2506.15591)
• Keywords: restoration, super-resolution, generative
• Abstract: It is a challenging problem to reproduce rich spatial details while maintaining temporal consistency in real-world video super-resolution (Real-VSR), especially when we leverage pre-trained generative models such as stable diffusion (SD) for realistic details synthesis. Existing SD-based Real-VSR methods often compromise spatial details for temporal coherence, resulting in suboptimal visual quality. We argue that the key lies in how to effectively extract the degradation-robust temporal consistency priors from the low-quality (LQ) input video and enhance the video details while maintaining the extracted consistency priors. To achieve this, we propose a Dual LoRA Learning (DLoRAL) paradigm to train an effective SD-based one-step diffusion model, achieving realistic frame details and temporal consistency simultaneously. Specifically, we introduce a Cross-Frame Retrieval (CFR) module to aggregate complementary information across frames, and train a Consistency-LoRA (C-LoRA) to learn robust temporal representations from degraded inputs. After consistency learning, we fix the CFR and C-LoRA modules and train a Detail-LoRA (D-LoRA) to enhance spatial details while aligning with the temporal space defined by C-LoRA to keep temporal coherence. The two phases alternate iteratively for optimization, collaboratively delivering consistent and detail-rich outputs. During inference, the two LoRA branches are merged into the SD model, allowing efficient and high-quality video restoration in a single diffusion step. Experiments show that DLoRAL achieves strong performance in both accuracy and speed. Code and models are available at this https URL.
• 摘要：重现丰富的空间细节是一个具有挑战性的问题，同时在现实世界视频超分辨率（Real-VSR）中保持时间一致性，尤其是当我们利用诸如稳定扩散（SD）之类的预训练的生成模型（用于现实细节）时。现有的基于SD的Real-VSR方法通常会损害时间连贯性的空间细节，从而产生次优的视觉质量。我们认为，关键在于如何从低质量（LQ）输入视频中有效地提取降解的时间一致性先验并增强视频详细信息，同时保持提取的一致性先验。为了实现这一目标，我们提出了双重LORA学习（Dloral）范式来培训有效的基于SD的一步扩散模型，同时实现了现实的框架细节和时间一致性。具体而言，我们引入了一个跨框架检索（CFR）模块，以跨帧汇总互补信息，并训练一致性 - 洛拉（C-Lora）（C-lora）以从退化的输入中学习强大的时间表示。经过一致性学习，我们修复了CFR和C-Lora模块，并训练细节Lora（D-Lora）以增强空间细节，同时与C-Lora定义的时间空间保持一致以保持时间连贯性。这两个阶段在迭代中交替进行优化，协作提供一致且细节丰富的输出。在推断期间，将两个Lora分支合并为SD模型，从而可以在单个扩散步骤中有效且高质量的视频恢复。实验表明，Dloral在准确性和速度方面都达到了强劲的性能。代码和型号可在此HTTPS URL上找到。

Title: HOIDiNi: Human-Object Interaction through Diffusion Noise Optimization

• Authors: Roey Ron, Guy Tevet, Haim Sawdayee, Amit H. Bermano
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.15625
• Pdf URL: https://arxiv.org/pdf/2506.15625
• Copy Paste: [[2506.15625]] HOIDiNi: Human-Object Interaction through Diffusion Noise Optimization(https://arxiv.org/abs/2506.15625)
• Keywords: generation
• Abstract: We present HOIDiNi, a text-driven diffusion framework for synthesizing realistic and plausible human-object interaction (HOI). HOI generation is extremely challenging since it induces strict contact accuracies alongside a diverse motion manifold. While current literature trades off between realism and physical correctness, HOIDiNi optimizes directly in the noise space of a pretrained diffusion model using Diffusion Noise Optimization (DNO), achieving both. This is made feasible thanks to our observation that the problem can be separated into two phases: an object-centric phase, primarily making discrete choices of hand-object contact locations, and a human-centric phase that refines the full-body motion to realize this blueprint. This structured approach allows for precise hand-object contact without compromising motion naturalness. Quantitative, qualitative, and subjective evaluations on the GRAB dataset alone clearly indicate HOIDiNi outperforms prior works and baselines in contact accuracy, physical validity, and overall quality. Our results demonstrate the ability to generate complex, controllable interactions, including grasping, placing, and full-body coordination, driven solely by textual prompts. this https URL.
• 摘要：我们提出了Hoidini，这是一种通过文本驱动的扩散框架，用于综合现实和合理的人类对象相互作用（HOI）。 HOI生成极具挑战性，因为它与多样化的运动歧管一起引起了严格的接触精度。尽管当前的文献在现实主义和身体上的正确性之间进行了交流，但Hoidini使用扩散噪声优化（DNO）直接在预审计扩散模型的噪声空间中进行了优化，从而实现了这两者。这是可行的，这要归功于我们的观察到可以将问题分为两个阶段：一个以对象为中心的阶段，主要是对手动对象接触位置进行离散选择，以及以人为中心的阶段，可以完善全身运动以实现这种蓝图。这种结构化方法允许精确的手动接触，而不会损害自然性。仅在抓取数据集上进行定量，定性和主观评估清楚地表明，Hoidini的表现优于先前的作品和接触精度，身体有效性和整体质量的基准。我们的结果表明，仅由文本提示驱动，可以产生复杂，可控的相互作用，包括抓地力，放置和全身协调。此HTTPS URL。

Title: Demystifying the Visual Quality Paradox in Multimodal Large Language Models

• Authors: Shuo Xing, Lanqing Guo, Hongyuan Hua, Seoyoung Lee, Peiran Li, Yufei Wang, Zhangyang Wang, Zhengzhong Tu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.15645
• Pdf URL: https://arxiv.org/pdf/2506.15645
• Copy Paste: [[2506.15645]] Demystifying the Visual Quality Paradox in Multimodal Large Language Models(https://arxiv.org/abs/2506.15645)
• Keywords: restoration
• Abstract: Recent Multimodal Large Language Models (MLLMs) excel on benchmark vision-language tasks, yet little is known about how input visual quality shapes their responses. Does higher perceptual quality of images already translate to better MLLM understanding? We conduct the first systematic study spanning leading MLLMs and a suite of vision-language benchmarks, applying controlled degradations and stylistic shifts to each image. Surprisingly, we uncover a visual-quality paradox: model, task, and even individual-instance performance can improve when images deviate from human-perceived fidelity. Off-the-shelf restoration pipelines fail to reconcile these idiosyncratic preferences. To close the gap, we introduce Visual-Quality Test-Time Tuning (VQ-TTT)-a lightweight adaptation module that: (1) inserts a learnable, low-rank kernel before the frozen vision encoder to modulate frequency content; and (2) fine-tunes only shallow vision-encoder layers via LoRA. VQ-TTT dynamically adjusts each input image in a single forward pass, aligning it with task-specific model preferences. Across the evaluated MLLMs and all datasets, VQ-TTT lifts significant average accuracy, with no external models, cached features, or extra training data. These findings redefine ``better'' visual inputs for MLLMs and highlight the need for adaptive, rather than universally ``clean'', imagery, in the new era of AI being the main data customer.
• 摘要：最近在基准视觉语言任务上出色的多模式大型语言模型（MLLMS）脱颖而出，但对输入视觉质量如何塑造其响应知之甚少。较高的图像感知质量是否已经转化为更好的MLLM理解？我们进行了第一个系统的研究，该研究涵盖了领先的MLLM和一系列视觉基准，将受控的降解和风格转移应用于每个图像。令人惊讶的是，当图像偏离人类感知的忠诚度时，我们发现了一个视觉质量悖论：模型，任务甚至个人实例性能。现成的修复管道无法调和这些特质偏好。为了缩小差距，我们引入了视觉质量测试时间调整（VQ-TTT）-轻巧的适应模块，该模块：（1）在冷冻视觉编码器之前插入可学习的，低级别的内核以调节频率内容； （2）仅通过洛拉（Lora）的微调浅视力编码层。 VQ-TTT在单个正向通过中动态调整每个输入图像，将其与特定于任务的模型偏好对齐。在评估的MLLM和所有数据集中，VQ-TTT提高了显着的平均准确性，没有外部模型，缓存功能或额外的培训数据。这些发现重新定义了MLLMS的视觉输入，并强调了对AI的新时代的自适应，而不是普遍的``清洁''的需求。

Title: Dual-Stage Value-Guided Inference with Margin-Based Reward Adjustment for Fast and Faithful VLM Captioning

• Authors: Ankan Deria, Adinath Madhavrao Dukre, Feilong Tang, Sara Atito, Sudipta Roy, Muhammad Awais, Muhammad Haris Khan, Imran Razzak
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.15649
• Pdf URL: https://arxiv.org/pdf/2506.15649
• Copy Paste: [[2506.15649]] Dual-Stage Value-Guided Inference with Margin-Based Reward Adjustment for Fast and Faithful VLM Captioning(https://arxiv.org/abs/2506.15649)
• Keywords: generation
• Abstract: Despite significant advances in inference-time search for vision-language models (VLMs), existing approaches remain both computationally expensive and prone to unpenalized, low-confidence generations which often lead to persistent hallucinations. We introduce \textbf{Value-guided Inference with Margin-based Reward (ViMaR)}, a two-stage inference framework that improves both efficiency and output fidelity by combining a temporal-difference value model with a margin-aware reward adjustment. In the first stage, we perform a single pass to identify the highest-value caption among diverse candidates. In the second stage, we selectively refine only those segments that were overlooked or exhibit weak visual grounding, thereby eliminating frequently rewarded evaluations. A calibrated margin-based penalty discourages low-confidence continuations while preserving descriptive richness. Extensive experiments across multiple VLM architectures demonstrate that ViMaR generates captions that are significantly more reliable, factually accurate, detailed, and explanatory, while achieving over 4$\times$ speedup compared to existing value-guided methods. Specifically, we show that ViMaR trained solely on LLaVA Mistral-7B, \textit{generalizes effectively to guide decoding in a stronger unseen model}. To further validate this, we adapt the ViMaR to steer generation in LLaVA-OneVision-Qwen2-7B, leading to consistent improvements in caption quality and demonstrating robust cross-model guidance. This cross-model generalization highlights ViMaR's flexibility and modularity, positioning it as a scalable and transferable inference-time decoding strategy. Furthermore, when ViMaR-generated captions are used for self-training, the underlying models achieve substantial gains across a broad suite of visual comprehension benchmarks, underscoring the potential of fast, accurate, and self-improving VLM pipelines.
• 摘要：尽管在推理时间搜索视觉模型（VLM）方面取得了重大进展，但现有的方法在计算上仍然是昂贵的，而且容易发生不受欢迎的低信任世代，这通常会导致持续的幻觉。我们介绍了\ textbf {值引导的推理与基于边距的奖励（VIMAR）}，这是一个两阶段的推理框架，通过将时间差异值模型与保证金感知的奖励调整相结合，从而提高了效率和输出保真度。在第一阶段，我们执行单个通行证，以确定不同候选人中最高值的标题。在第二阶段，我们仅有选择地完善那些被忽视或表现出弱视觉接地的细分市场，从而消除了经常受到奖励的评估。校准的基于保证金的罚款会阻止低信心的连续性，同时保持描述性丰富。跨多个VLM体系结构进行的广泛实验表明，Vimar生成的字幕明显更可靠，实际上准确，详细和解释，而与现有的价值指导方法相比，实现了超过4 $ \ times $ speedup。具体来说，我们表明，Vimar仅在Llava Mistral-7b上进行训练，\ textit {有效地指导在更强的看不见模型中进行解码}。为了进一步验证这一点，我们将Vimar适应了Llava-onevision-Qwen2-7b的转向产生，从而导致标题质量的一致改进并证明了强大的跨模型指导。这种跨模型的概括强调了Vimar的灵活性和模块化，将其定位为可扩展且可转移的推理时间解码策略。此外，当使用Vimar生成的标题进行自我训练时，基础模型在广泛的视觉理解基准中实现了可观的增长，突显了快速，准确和自我支持的VLM管道的潜力。

Title: UniRelight: Learning Joint Decomposition and Synthesis for Video Relighting

• Authors: Kai He, Ruofan Liang, Jacob Munkberg, Jon Hasselgren, Nandita Vijaykumar, Alexander Keller, Sanja Fidler, Igor Gilitschenski, Zan Gojcic, Zian Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.15673
• Pdf URL: https://arxiv.org/pdf/2506.15673
• Copy Paste: [[2506.15673]] UniRelight: Learning Joint Decomposition and Synthesis for Video Relighting(https://arxiv.org/abs/2506.15673)
• Keywords: generative
• Abstract: We address the challenge of relighting a single image or video, a task that demands precise scene intrinsic understanding and high-quality light transport synthesis. Existing end-to-end relighting models are often limited by the scarcity of paired multi-illumination data, restricting their ability to generalize across diverse scenes. Conversely, two-stage pipelines that combine inverse and forward rendering can mitigate data requirements but are susceptible to error accumulation and often fail to produce realistic outputs under complex lighting conditions or with sophisticated materials. In this work, we introduce a general-purpose approach that jointly estimates albedo and synthesizes relit outputs in a single pass, harnessing the generative capabilities of video diffusion models. This joint formulation enhances implicit scene comprehension and facilitates the creation of realistic lighting effects and intricate material interactions, such as shadows, reflections, and transparency. Trained on synthetic multi-illumination data and extensive automatically labeled real-world videos, our model demonstrates strong generalization across diverse domains and surpasses previous methods in both visual fidelity and temporal consistency.
• 摘要：我们应对重新保留单个图像或视频的挑战，该任务需要精确的场景内在的理解和高质量的光传输合成。现有的端到端重新确定模型通常受到配对多刷数据的稀缺性的限制，从而限制了它们在各种场景中概括的能力。相反，将逆渲染和前向渲染相结合的两阶段管道可以减轻数据要求，但容易受到错误积累的影响，并且通常在复杂的照明条件下或使用复杂的材料下无法产生现实的输出。在这项工作中，我们引入了一种通用方法，该方法共同估计反照率并在单个通行证中综合了重新输出，从而利用了视频扩散模型的生成能力。该联合配方增强了隐式场景的理解，并促进了逼真的照明效应和复杂的材料相互作用（例如阴影，反射和透明度）的创造。经过合成多刷数据和广泛标记为现实世界的视频的培训，我们的模型表明了跨不同领域的强烈概括，并且超过了视觉保真度和时间一致性的先前方法。

Title: Sekai: A Video Dataset towards World Exploration

• Authors: Zhen Li, Chuanhao Li, Xiaofeng Mao, Shaoheng Lin, Ming Li, Shitian Zhao, Zhaopan Xu, Xinyue Li, Yukang Feng, Jianwen Sun, Zizhen Li, Fanrui Zhang, Jiaxin Ai, Zhixiang Wang, Yuwei Wu, Tong He, Jiangmiao Pang, Yu Qiao, Yunde Jia, Kaipeng Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.15675
• Pdf URL: https://arxiv.org/pdf/2506.15675
• Copy Paste: [[2506.15675]] Sekai: A Video Dataset towards World Exploration(https://arxiv.org/abs/2506.15675)
• Keywords: generation
• Abstract: Video generation techniques have made remarkable progress, promising to be the foundation of interactive world exploration. However, existing video generation datasets are not well-suited for world exploration training as they suffer from some limitations: limited locations, short duration, static scenes, and a lack of annotations about exploration and the world. In this paper, we introduce Sekai (meaning ``world'' in Japanese), a high-quality first-person view worldwide video dataset with rich annotations for world exploration. It consists of over 5,000 hours of walking or drone view (FPV and UVA) videos from over 100 countries and regions across 750 cities. We develop an efficient and effective toolbox to collect, pre-process and annotate videos with location, scene, weather, crowd density, captions, and camera trajectories. Experiments demonstrate the quality of the dataset. And, we use a subset to train an interactive video world exploration model, named YUME (meaning ``dream'' in Japanese). We believe Sekai will benefit the area of video generation and world exploration, and motivate valuable applications.
• 摘要：视频生成技术取得了显着的进步，有望成为互动世界探索的基础。但是，现有的视频生成数据集并不适合世界探索培训，因为它们受到了某些局限性：限制的位置，持续时间短，静态场景以及对探索和世界的缺乏注释。在本文中，我们介绍了Sekai（在日语中的意思是``世界''，这是一个高质量的第一人称视图世界视频数据集，并具有丰富的世界探索注释。它包括超过5,000个小时的步行或无人机景观（FPV和UVA），来自100个国家和750个城市的地区。我们开发了一个有效的工具箱，以收集，预处理和注释视频，其中包括位置，场景，天气，人群密度，字幕和摄像头轨迹。实验证明了数据集的质量。而且，我们使用一个子集来训练一个互动视频世界探索模型，名为Yume（在日语中为``梦想''）。我们认为，Sekai将受益于视频生成和世界探索领域，并激励有价值的应用。

Title: Evolutionary Caching to Accelerate Your Off-the-Shelf Diffusion Model

• Authors: Anirud Aggarwal, Abhinav Shrivastava, Matthew Gwilliam
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.15682
• Pdf URL: https://arxiv.org/pdf/2506.15682
• Copy Paste: [[2506.15682]] Evolutionary Caching to Accelerate Your Off-the-Shelf Diffusion Model(https://arxiv.org/abs/2506.15682)
• Keywords: generation
• Abstract: Diffusion-based image generation models excel at producing high-quality synthetic content, but suffer from slow and computationally expensive inference. Prior work has attempted to mitigate this by caching and reusing features within diffusion transformers across inference steps. These methods, however, often rely on rigid heuristics that result in limited acceleration or poor generalization across architectures. We propose Evolutionary Caching to Accelerate Diffusion models (ECAD), a genetic algorithm that learns efficient, per-model, caching schedules forming a Pareto frontier, using only a small set of calibration prompts. ECAD requires no modifications to network parameters or reference images. It offers significant inference speedups, enables fine-grained control over the quality-latency trade-off, and adapts seamlessly to different diffusion models. Notably, ECAD's learned schedules can generalize effectively to resolutions and model variants not seen during calibration. We evaluate ECAD on PixArt-alpha, PixArt-Sigma, and this http URL using multiple metrics (FID, CLIP, Image Reward) across diverse benchmarks (COCO, MJHQ-30k, PartiPrompts), demonstrating consistent improvements over previous approaches. On PixArt-alpha, ECAD identifies a schedule that outperforms the previous state-of-the-art method by 4.47 COCO FID while increasing inference speedup from 2.35x to 2.58x. Our results establish ECAD as a scalable and generalizable approach for accelerating diffusion inference. Our project website is available at this https URL and our code is available at this https URL.
• 摘要：基于扩散的图像生成模型在产生高质量合成含量方面表现出色，但遭受缓慢且计算昂贵的推断。先前的工作试图通过跨推理步骤中的扩散变压器中的缓存和重复使用功能来减轻这种情况。但是，这些方法通常依赖于严格的启发式方法，从而导致架构的加速度有限或泛化不良。我们提出进化缓存以加速扩散模型（ECAD），这是一种遗传算法，仅使用一组少量的校准提示，可以学习有效的，人均的缓存时间表，形成帕累托前沿。 ECAD不需要对网络参数或参考图像进行修改。它提供了明显的推理加速度，可以对质量延迟权衡进行细粒度的控制，并无缝地适应不同的扩散模型。值得注意的是，ECAD学到的时间表可以有效地概括为在校准过程中看不见的分辨率和模型变体。我们使用多个指标（COCO，MJHQ-30K，partiprompts）对Pixart-Alpha，Pixart-Sigma和此HTTP URL进行了ECAD，并使用多个指标（FID，CLIP，图像奖励）评估了ECAD。在Pixart-Alpha上，ECAD确定了一个时间表，该时间表以4.47 Coco FID优于先前的最新方法，同时将推理速度从2.35倍增加到2.58倍。我们的结果将ECAD确定为一种可扩展的加速扩散推断方法。我们的项目网站可在此HTTPS URL上找到，我们的代码可在此HTTPS URL上找到。