2025-02-12

Title: Prompt-Aware Scheduling for Efficient Text-to-Image Inferencing System

• Authors: Shubham Agarwal, Saud Iqbal, Subrata Mitra
• Subjects: cs.LG, cs.DC, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.06798
• Pdf URL: https://arxiv.org/pdf/2502.06798
• Copy Paste: [[2502.06798]] Prompt-Aware Scheduling for Efficient Text-to-Image Inferencing System(https://arxiv.org/abs/2502.06798)
• Keywords: generative
• Abstract: Traditional ML models utilize controlled approximations during high loads, employing faster, but less accurate models in a process called accuracy scaling. However, this method is less effective for generative text-to-image models due to their sensitivity to input prompts and performance degradation caused by large model loading overheads. This work introduces a novel text-to-image inference system that optimally matches prompts across multiple instances of the same model operating at various approximation levels to deliver high-quality images under high loads and fixed budgets.
• 摘要：传统 ML 模型在高负载期间使用受控近似值，在称为准确度扩展的过程中采用速度更快但准确度较低的模型。但是，这种方法对于生成式文本到图像模型不太有效，因为它们对输入提示很敏感，并且模型加载开销较大会导致性能下降。这项工作引入了一种新颖的文本到图像推理系统，该系统可以在不同近似级别运行的同一模型的多个实例之间最佳地匹配提示，以在高负载和固定预算下提供高质量的图像。

Title: Emotion Recognition and Generation: A Comprehensive Review of Face, Speech, and Text Modalities

• Authors: Rebecca Mobbs, Dimitrios Makris, Vasileios Argyriou
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.06803
• Pdf URL: https://arxiv.org/pdf/2502.06803
• Copy Paste: [[2502.06803]] Emotion Recognition and Generation: A Comprehensive Review of Face, Speech, and Text Modalities(https://arxiv.org/abs/2502.06803)
• Keywords: generation
• Abstract: Emotion recognition and generation have emerged as crucial topics in Artificial Intelligence research, playing a significant role in enhancing human-computer interaction within healthcare, customer service, and other fields. Although several reviews have been conducted on emotion recognition and generation as separate entities, many of these works are either fragmented or limited to specific methodologies, lacking a comprehensive overview of recent developments and trends across different modalities. In this survey, we provide a holistic review aimed at researchers beginning their exploration in emotion recognition and generation. We introduce the fundamental principles underlying emotion recognition and generation across facial, vocal, and textual modalities. This work categorises recent state-of-the-art research into distinct technical approaches and explains the theoretical foundations and motivations behind these methodologies, offering a clearer understanding of their application. Moreover, we discuss evaluation metrics, comparative analyses, and current limitations, shedding light on the challenges faced by researchers in the field. Finally, we propose future research directions to address these challenges and encourage further exploration into developing robust, effective, and ethically responsible emotion recognition and generation systems.
• 摘要：情绪识别和生成已成为人工智能研究的重要课题，在增强医疗保健、客户服务和其他领域的人机交互方面发挥着重要作用。尽管已经对情绪识别和生成进行了多次审查，但其中许多工作要么支离破碎，要么局限于特定方法，缺乏对不同模式的最新发展和趋势的全面概述。在这篇综述中，我们为开始探索情绪识别和生成的研究人员提供了全面的评论。我们介绍了面部、声音和文本模式下情绪识别和生成的基本原理。这项工作将最近的最先进研究分为不同的技术方法，并解释了这些方法背后的理论基础和动机，从而更清楚地了解它们的应用。此外，我们讨论了评估指标、比较分析和当前的局限性，揭示了该领域研究人员面临的挑战。最后，我们提出了应对这些挑战的未来研究方向，并鼓励进一步探索开发强大、有效且合乎道德的情绪识别和生成系统。

Title: Efficient Diffusion Models: A Survey

• Authors: Hui Shen, Jingxuan Zhang, Boning Xiong, Rui Hu, Shoufa Chen, Zhongwei Wan, Xin Wang, Yu Zhang, Zixuan Gong, Guangyin Bao, Chaofan Tao, Yongfeng Huang, Ye Yuan, Mi Zhang
• Subjects: cs.LG, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.06805
• Pdf URL: https://arxiv.org/pdf/2502.06805
• Copy Paste: [[2502.06805]] Efficient Diffusion Models: A Survey(https://arxiv.org/abs/2502.06805)
• Keywords: generation, generative
• Abstract: Diffusion models have emerged as powerful generative models capable of producing high-quality contents such as images, videos, and audio, demonstrating their potential to revolutionize digital content creation. However, these capabilities come at the cost of their significant computational resources and lengthy generation time, underscoring the critical need to develop efficient techniques for practical deployment. In this survey, we provide a systematic and comprehensive review of research on efficient diffusion models. We organize the literature in a taxonomy consisting of three main categories, covering distinct yet interconnected efficient diffusion model topics from algorithm-level, system-level, and framework perspective, respectively. We have also created a GitHub repository where we organize the papers featured in this survey at this https URL. We hope our survey can serve as a valuable resource to help researchers and practitioners gain a systematic understanding of efficient diffusion model research and inspire them to contribute to this important and exciting field.
• 摘要：传播模型已经成为强大的生成模型，能够生成高质量的内容，如图像、视频和音频，展示了它们彻底改变数字内容创作的潜力。然而，这些能力是以大量的计算资源和漫长的生成时间为代价的，这凸显了开发有效技术以供实际部署的迫切需要。在这篇综述中，我们对高效传播模型的研究进行了系统而全面的回顾。我们将文献组织成一个由三个主要类别组成的分类法，分别从算法级、系统级和框架级的角度涵盖了不同但相互关联的高效传播模型主题。我们还创建了一个 GitHub 存储库，我们将这篇综述中的论文组织在这个 https URL 上。我们希望我们的综述可以成为一种宝贵的资源，帮助研究人员和从业者系统地了解高效传播模型研究，并激励他们为这个重要而令人兴奋的领域做出贡献。

Title: Logits are All We Need to Adapt Closed Models

• Authors: Gaurush Hiranandani, Haolun Wu, Subhojyoti Mukherjee, Sanmi Koyejo
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.06806
• Pdf URL: https://arxiv.org/pdf/2502.06806
• Copy Paste: [[2502.06806]] Logits are All We Need to Adapt Closed Models(https://arxiv.org/abs/2502.06806)
• Keywords: generation
• Abstract: Many commercial Large Language Models (LLMs) are often closed-source, limiting developers to prompt tuning for aligning content generation with specific applications. While these models currently do not provide access to token logits, we argue that if such access were available, it would enable more powerful adaptation techniques beyond prompt engineering. In this paper, we propose a token-level probability reweighting framework that, given access to logits and a small amount of task-specific data, can effectively steer black-box LLMs toward application-specific content generation. Our approach views next-token prediction through the lens of supervised classification. We show that aligning black-box LLMs with task-specific data can be formulated as a label noise correction problem, leading to \emph{Plugin} model -- an autoregressive probability reweighting model that operates solely on logits. We provide theoretical justification for why reweighting logits alone is sufficient for task adaptation. Extensive experiments with multiple datasets, LLMs, and reweighting models demonstrate the effectiveness of our method, advocating for broader access to token logits in closed-source models.
• 摘要：许多商用大型语言模型 (LLM) 通常是闭源的，这限制了开发人员进行快速调整以使内容生成与特定应用程序保持一致。虽然这些模型目前不提供对 token logit 的访问，但我们认为，如果可以访问，它将支持除了快速工程之外更强大的适应技术。在本文中，我们提出了一个 token 级概率重新加权框架，只要能够访问 logit 和少量特定于任务的数据，就可以有效地引导黑盒 LLM 生成特定于应用程序的内容。我们的方法通过监督分类的视角来看待下一个 token 预测。我们表明，将黑盒 LLM 与特定于任务的数据对齐可以表述为标签噪声校正问题，从而产生 \emph{Plugin} 模型——一种仅对 logit 进行操作的自回归概率重新加权模型。我们提供了理论依据，说明为什么仅重新加权 logit 就足以进行任务适应。使用多个数据集、LLM 和重加权模型进行的大量实验证明了我们方法的有效性，并主张更广泛地访问闭源模型中的标记日志。

Title: Harness Local Rewards for Global Benefits: Effective Text-to-Video Generation Alignment with Patch-level Reward Models

• Authors: Shuting Wang, Haihong Tang, Zhicheng Dou, Chenyan Xiong
• Subjects: cs.LG, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.06812
• Pdf URL: https://arxiv.org/pdf/2502.06812
• Copy Paste: [[2502.06812]] Harness Local Rewards for Global Benefits: Effective Text-to-Video Generation Alignment with Patch-level Reward Models(https://arxiv.org/abs/2502.06812)
• Keywords: generation
• Abstract: The emergence of diffusion models (DMs) has significantly improved the quality of text-to-video generation models (VGMs). However, current VGM optimization primarily emphasizes the global quality of videos, overlooking localized errors, which leads to suboptimal generation capabilities. To address this issue, we propose a post-training strategy for VGMs, HALO, which explicitly incorporates local feedback from a patch reward model, providing detailed and comprehensive training signals with the video reward model for advanced VGM optimization. To develop an effective patch reward model, we distill GPT-4o to continuously train our video reward model, which enhances training efficiency and ensures consistency between video and patch reward distributions. Furthermore, to harmoniously integrate patch rewards into VGM optimization, we introduce a granular DPO (Gran-DPO) algorithm for DMs, allowing collaborative use of both patch and video rewards during the optimization process. Experimental results indicate that our patch reward model aligns well with human annotations and HALO substantially outperforms the baselines across two evaluation methods. Further experiments quantitatively prove the existence of patch defects, and our proposed method could effectively alleviate this issue.
• 摘要：扩散模型 (DM) 的出现显著提高了文本到视频生成模型 (VGM) 的质量。然而，目前的 VGM 优化主要强调视频的整体质量，而忽略了局部错误，导致生成能力欠佳。为了解决这个问题，我们提出了一种 VGM 的后训练策略 HALO，它明确地结合了来自补丁奖励模型的局部反馈，为高级 VGM 优化提供了视频奖励模型的详细而全面的训练信号。为了开发一个有效的补丁奖励模型，我们提取了 GPT-4o 来持续训练我们的视频奖励模型，从而提高了训练效率并确保了视频和补丁奖励分布之间的一致性。此外，为了将补丁奖励和谐地整合到 VGM 优化中，我们为 DM 引入了一种细粒度的 DPO (Gran-DPO) 算法，允许在优化过程中协同使用补丁和视频奖励。实验结果表明，我们的补丁奖励模型与人工注释非常吻合，并且 HALO 在两种评估方法中的表现都大大优于基线。进一步的实验定量证明了斑块缺陷的存在，我们提出的方法可以有效地缓解这一问题。

Title: Diffusion Instruction Tuning

• Authors: Chen Jin, Ryutaro Tanno, Amrutha Saseendran, Tom Diethe, Philip Teare
• Subjects: cs.LG, cs.AI, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.06814
• Pdf URL: https://arxiv.org/pdf/2502.06814
• Copy Paste: [[2502.06814]] Diffusion Instruction Tuning(https://arxiv.org/abs/2502.06814)
• Keywords: generation
• Abstract: We introduce Lavender, a simple supervised fine-tuning (SFT) method that boosts the performance of advanced vision-language models (VLMs) by leveraging state-of-the-art image generation models such as Stable Diffusion. Specifically, Lavender aligns the text-vision attention in the VLM transformer with the equivalent used by Stable Diffusion during SFT, instead of adapting separate encoders. This alignment enriches the model's visual understanding and significantly boosts performance across in- and out-of-distribution tasks. Lavender requires just 0.13 million training examples, 2.5% of typical large-scale SFT datasets, and fine-tunes on standard hardware (8 GPUs) in a single day. It consistently improves state-of-the-art open-source multimodal LLMs (e.g., Llama-3.2-11B, MiniCPM-Llama3-v2.5), achieving up to 30% gains and a 68% boost on challenging out-of-distribution medical QA tasks. By efficiently transferring the visual expertise of image generators with minimal supervision, Lavender offers a scalable solution for more accurate vision-language systems. All code, training data, and models will be shared at this https URL.
• 摘要：我们引入了 Lavender，这是一种简单的监督微调 (SFT) 方法，它利用 Stable Diffusion 等最先进的图像生成模型来提升高级视觉语言模型 (VLM) 的性能。具体来说，Lavender 将 VLM 转换器中的文本视觉注意力与 SFT 期间 Stable Diffusion 使用的等效注意力对齐，而不是调整单独的编码器。这种对齐丰富了模型的视觉理解，并显著提升了分布内和分布外任务的性能。Lavender 只需要 13 万个训练示例、典型大型 SFT 数据集的 2.5%，并在一天内在标准硬件（8 个 GPU）上进行微调。它不断改进最先进的开源多模态 LLM（例如 Llama-3.2-11B、MiniCPM-Llama3-v2.5），在具有挑战性的分布式医疗 QA 任务上实现了高达 30% 的增益和 68% 的提升。通过在最低限度的监督下高效地转移图像生成器的视觉专业知识，Lavender 为更准确的视觉语言系统提供了可扩展的解决方案。所有代码、训练数据和模型都将在此 https URL 上共享。

Title: DeepCell: Multiview Representation Learning for Post-Mapping Netlists

• Authors: Zhengyuan Shi, Chengyu Ma, Ziyang Zheng, Lingfeng Zhou, Hongyang Pan, Wentao Jiang, Fan Yang, Xiaoyan Yang, Zhufei Chu, Qiang Xu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.06816
• Pdf URL: https://arxiv.org/pdf/2502.06816
• Copy Paste: [[2502.06816]] DeepCell: Multiview Representation Learning for Post-Mapping Netlists(https://arxiv.org/abs/2502.06816)
• Keywords: generation
• Abstract: Representation learning for post-mapping (PM) netlists is a critical challenge in Electronic Design Automation (EDA), driven by the diverse and complex nature of modern circuit designs. Existing approaches focus on intermediate representations like And-Inverter Graphs (AIGs), limiting their applicability to post-synthesis stages. We introduce DeepCell, a multiview representation learning framework that integrates structural and functional insights from both PM netlists and AIGs to learn rich, generalizable embeddings. At its core, DeepCell employs the novel Mask Circuit Modeling (MCM) mechanism, which refines PM netlist representations in a self-supervised manner using pretrained AIG encoders. DeepCell sets a new benchmark in PM netlist representation, outperforming existing methods in predictive accuracy and reconstruction fidelity. To validate its efficacy, we apply DeepCell to functional Engineering Change Orders (ECO), achieving significant reductions in patch generation costs and runtime while improving patch quality.
• 摘要：后映射 (PM) 网表的表示学习是电子设计自动化 (EDA) 中的一项关键挑战，其驱动力来自现代电路设计的多样性和复杂性。现有方法侧重于中间表示，如与-反相图 (AIG)，这限制了它们在后综合阶段的适用性。我们引入了 DeepCell，这是一个多视图表示学习框架，它集成了来自 PM 网表和 AIG 的结构和功能见解，以学习丰富、可泛化的嵌入。DeepCell 的核心是采用新颖的掩模电路建模 (MCM) 机制，该机制使用预训练的 AIG 编码器以自监督的方式细化 PM 网表表示。DeepCell 在 PM 网表表示方面树立了新的标杆，在预测准确性和重建保真度方面优于现有方法。为了验证其有效性，我们将 DeepCell 应用于功能工程变更单 (ECO)，在提高补丁质量的同时，显著降低了补丁生成成本和运行时间。

Title: Functional 3D Scene Synthesis through Human-Scene Optimization

• Authors: Yao Wei, Matteo Toso, Pietro Morerio, Michael Ying Yang, Alessio Del Bue
• Subjects: cs.LG, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.06819
• Pdf URL: https://arxiv.org/pdf/2502.06819
• Copy Paste: [[2502.06819]] Functional 3D Scene Synthesis through Human-Scene Optimization(https://arxiv.org/abs/2502.06819)
• Keywords: generation, generative
• Abstract: This paper presents a novel generative approach that outputs 3D indoor environments solely from a textual description of the scene. Current methods often treat scene synthesis as a mere layout prediction task, leading to rooms with overlapping objects or overly structured scenes, with limited consideration of the practical usability of the generated environment. Instead, our approach is based on a simple, but effective principle: we condition scene synthesis to generate rooms that are usable by humans. This principle is implemented by synthesizing 3D humans that interact with the objects composing the scene. If this human-centric scene generation is viable, the room layout is functional and it leads to a more coherent 3D structure. To this end, we propose a novel method for functional 3D scene synthesis, which consists of reasoning, 3D assembling and optimization. We regard text guided 3D synthesis as a reasoning process by generating a scene graph via a graph diffusion network. Considering object functional co-occurrence, a new strategy is designed to better accommodate human-object interaction and avoidance, achieving human-aware 3D scene optimization. We conduct both qualitative and quantitative experiments to validate the effectiveness of our method in generating coherent 3D scene synthesis results.
• 摘要：本文提出了一种新颖的生成方法，该方法仅根据场景的文本描述输出 3D 室内环境。当前的方法通常将场景合成视为单纯的布局预测任务，导致房间中物体重叠或场景结构过于复杂，而对生成环境的实际可用性考虑有限。相反，我们的方法基于一个简单但有效的原则：我们调节场景合成以生成人类可用的房间。该原则通过合成与构成场景的物体交互的 3D 人类来实现。如果这种以人为中心的场景生成可行，则房间布局是实用的，并会产生更连贯的 3D 结构。为此，我们提出了一种新颖的功能性 3D 场景合成方法，该方法包括推理、3D 组装和优化。我们将文本引导的 3D 合成视为一个推理过程，通过图形扩散网络生成场景图。考虑到物体功能共现，设计了一种新策略来更好地适应人与物体的交互和避免，实现人感知的 3D 场景优化。我们进行了定性和定量实验来验证我们的方法在生成连贯的 3D 场景合成结果方面的有效性。

Title: DiffListener: Discrete Diffusion Model for Listener Generation

• Authors: Siyeol Jung, Taehwan Kim
• Subjects: cs.LG, cs.CL, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.06822
• Pdf URL: https://arxiv.org/pdf/2502.06822
• Copy Paste: [[2502.06822]] DiffListener: Discrete Diffusion Model for Listener Generation(https://arxiv.org/abs/2502.06822)
• Keywords: generation
• Abstract: The listener head generation (LHG) task aims to generate natural nonverbal listener responses based on the speaker's multimodal cues. While prior work either rely on limited modalities (e.g. audio and facial information) or employ autoregressive approaches which have limitations such as accumulating prediction errors. To address these limitations, we propose DiffListener, a discrete diffusion based approach for non-autoregressive listener head generation. Our model takes the speaker's facial information, audio, and text as inputs, additionally incorporating facial differential information to represent the temporal dynamics of expressions and movements. With this explicit modeling of facial dynamics, DiffListener can generate coherent reaction sequences in a non-autoregressive manner. Through comprehensive experiments, DiffListener demonstrates state-of-the-art performance in both quantitative and qualitative evaluations. The user study shows that DiffListener generates natural context-aware listener reactions that are well synchronized with the speaker. The code and demo videos are available in this https URL
• 摘要：听众头部生成 (LHG) 任务旨在根据说话者的多模态提示生成自然的非语言听众响应。而先前的工作要么依赖于有限的模态（例如音频和面部信息），要么采用自回归方法，但这种方法存在诸如累积预测误差等局限性。为了解决这些局限性，我们提出了 DiffListener，这是一种基于离散扩散的非自回归听众头部生成方法。我们的模型将说话者的面部信息、音频和文本作为输入，另外还结合面部差异信息来表示表情和动作的时间动态。通过这种明确的面部动态建模，DiffListener 可以以非自回归的方式生成连贯的反应序列。通过全面的实验，DiffListener 在定量和定性评估中都表现出了最先进的性能。用户研究表明，DiffListener 可以生成自然的情境感知听众反应，这些反应与说话者同步良好。代码和演示视频可在此 https URL 中找到

Title: CTR-Driven Advertising Image Generation with Multimodal Large Language Models

• Authors: Xingye Chen, Wei Feng, Zhenbang Du, Weizhen Wang, Yanyin Chen, Haohan Wang, Linkai Liu, Yaoyu Li, Jinyuan Zhao, Yu Li, Zheng Zhang, Jingjing Lv, Junjie Shen, Zhangang Lin, Jingping Shao, Yuanjie Shao, Xinge You, Changxin Gao, Nong Sang
• Subjects: cs.LG, cs.GR, cs.IR
• Abstract URL: https://arxiv.org/abs/2502.06823
• Pdf URL: https://arxiv.org/pdf/2502.06823
• Copy Paste: [[2502.06823]] CTR-Driven Advertising Image Generation with Multimodal Large Language Models(https://arxiv.org/abs/2502.06823)
• Keywords: generation
• Abstract: In web data, advertising images are crucial for capturing user attention and improving advertising effectiveness. Most existing methods generate background for products primarily focus on the aesthetic quality, which may fail to achieve satisfactory online performance. To address this limitation, we explore the use of Multimodal Large Language Models (MLLMs) for generating advertising images by optimizing for Click-Through Rate (CTR) as the primary objective. Firstly, we build targeted pre-training tasks, and leverage a large-scale e-commerce multimodal dataset to equip MLLMs with initial capabilities for advertising image generation tasks. To further improve the CTR of generated images, we propose a novel reward model to fine-tune pre-trained MLLMs through Reinforcement Learning (RL), which can jointly utilize multimodal features and accurately reflect user click preferences. Meanwhile, a product-centric preference optimization strategy is developed to ensure that the generated background content aligns with the product characteristics after fine-tuning, enhancing the overall relevance and effectiveness of the advertising images. Extensive experiments have demonstrated that our method achieves state-of-the-art performance in both online and offline metrics. Our code and pre-trained models are publicly available at: this https URL.
• 摘要：在网络数据中，广告图片对于吸引用户注意力、提升广告效果至关重要。现有的产品背景生成方法大多侧重于美观度，可能无法达到令人满意的线上效果。针对这一限制，我们探索使用以点击率 (CTR) 为主要目标的多模态大型语言模型 (MLLM) 生成广告图片。首先，我们构建有针对性的预训练任务，并利用大规模电商多模态数据集为 MLLM 配备初始的广告图片生成任务能力。为了进一步提高生成图片的点击率，我们提出了一种新颖的奖励模型，通过强化学习 (RL) 对预训练的 MLLM 进行微调，使其能够联合利用多模态特征并准确反映用户的点击偏好。同时，制定了以产品为中心的偏好优化策略，确保生成的背景内容在微调后与产品特征相符，从而提升广告图片的整体相关性和有效性。大量实验表明，我们的方法在线上和离线指标上都达到了最佳性能。我们的代码和预训练模型可在此 https URL 上公开获取。

Title: Learning to Synthesize Compatible Fashion Items Using Semantic Alignment and Collocation Classification: An Outfit Generation Framework

• Authors: Dongliang Zhou, Haijun Zhang, Kai Yang, Linlin Liu, Han Yan, Xiaofei Xu, Zhao Zhang, Shuicheng Yan
• Subjects: cs.LG, cs.AI, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.06827
• Pdf URL: https://arxiv.org/pdf/2502.06827
• Copy Paste: [[2502.06827]] Learning to Synthesize Compatible Fashion Items Using Semantic Alignment and Collocation Classification: An Outfit Generation Framework(https://arxiv.org/abs/2502.06827)
• Keywords: generation, generative
• Abstract: The field of fashion compatibility learning has attracted great attention from both the academic and industrial communities in recent years. Many studies have been carried out for fashion compatibility prediction, collocated outfit recommendation, artificial intelligence (AI)-enabled compatible fashion design, and related topics. In particular, AI-enabled compatible fashion design can be used to synthesize compatible fashion items or outfits in order to improve the design experience for designers or the efficacy of recommendations for customers. However, previous generative models for collocated fashion synthesis have generally focused on the image-to-image translation between fashion items of upper and lower clothing. In this paper, we propose a novel outfit generation framework, i.e., OutfitGAN, with the aim of synthesizing a set of complementary items to compose an entire outfit, given one extant fashion item and reference masks of target synthesized items. OutfitGAN includes a semantic alignment module, which is responsible for characterizing the mapping correspondence between the existing fashion items and the synthesized ones, to improve the quality of the synthesized images, and a collocation classification module, which is used to improve the compatibility of a synthesized outfit. In order to evaluate the performance of our proposed models, we built a large-scale dataset consisting of 20,000 fashion outfits. Extensive experimental results on this dataset show that our OutfitGAN can synthesize photo-realistic outfits and outperform state-of-the-art methods in terms of similarity, authenticity and compatibility measurements.
• 摘要：近年来，时尚兼容性学习领域引起了学术界和工业界的极大关注。许多研究针对时尚兼容性预测、搭配服装推荐、人工智能 (AI) 支持的兼容时装设计以及相关主题。特别是，人工智能支持的兼容时装设计可用于合成兼容的时尚单品或服装，以改善设计师的设计体验或向顾客推荐的有效性。然而，以前的搭配时尚合成生成模型通常侧重于上装和下装时尚单品之间的图像到图像转换。在本文中，我们提出了一种新颖的服装生成框架，即 OutfitGAN，旨在在给定一件现有时尚单品和目标合成单品的参考掩码的情况下，合成一组互补单品以组成一整套服装。OutfitGAN 包括语义对齐模块，负责表征现有时尚单品和合成单品之间的映射对应关系，以提高合成图像的质量，以及搭配分类模块，用于提高合成服装的兼容性。为了评估我们提出的模型的性能，我们构建了一个包含 20,000 套时尚服装的大规模数据集。在该数据集上进行的大量实验结果表明，我们的 OutfitGAN 可以合成照片般逼真的服装，并且在相似性、真实性和兼容性测量方面优于最先进的方法。

Title: Entropy Adaptive Decoding: Dynamic Model Switching for Efficient Inference

• Authors: Toby Simonds
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.06833
• Pdf URL: https://arxiv.org/pdf/2502.06833
• Copy Paste: [[2502.06833]] Entropy Adaptive Decoding: Dynamic Model Switching for Efficient Inference(https://arxiv.org/abs/2502.06833)
• Keywords: generation
• Abstract: We present Entropy Adaptive Decoding (EAD), a novel approach for efficient language model inference that dynamically switches between different-sized models based on prediction uncertainty. By monitoring rolling entropy in model logit distributions, our method identifies text regions where a smaller model suffices and switches to a larger model only when prediction uncertainty exceeds a threshold. Unlike speculative decoding approaches that maintain perfect output fidelity through verification, EAD accepts controlled output divergence in exchange for computational efficiency. Our experiments on the MATH benchmark demonstrate remarkable efficiency gains across different model families. Using the LLaMA family, we maintain 96.7\% of the 11B model's performance (50.4\% vs 52.1\%) while using it for only 43\% of tokens, decreasing computational cost by 41.5\%. These gains become more pronounced with larger size differentials in the Qwen family, where we achieve 92.9\% of the 14B model's performance (74.3\% vs 80.0\%) while using it for just 25\% of tokens, decreasing computational cost by 67\%. The consistency of these results across model pairs suggests that language model computation can be significantly optimized by selectively deploying model capacity based on local generation complexity. Our findings indicate that current approaches to model inference may be unnecessarily conservative in their pursuit of perfect output fidelity, and that accepting minor performance trade-offs can enable dramatic reductions in computational costs.
• 摘要：我们提出了熵自适应解码 (EAD)，这是一种新颖的高效语言模型推理方法，可根据预测不确定性在不同大小的模型之间动态切换。通过监控模型逻辑分布中的滚动熵，我们的方法可以识别出较小模型就足够的文本区域，并且只有当预测不确定性超过阈值时才切换到较大的模型。与通过验证保持完美输出保真度的推测解码方法不同，EAD 接受受控的输出发散以换取计算效率。我们在 MATH 基准上的实验表明，不同模型系列的效率显著提高。使用 LLaMA 系列，我们保持了 11B 模型性能的 96.7%（50.4% vs 52.1%），同时仅将其用于 43% 的标记，从而将计算成本降低了 41.5%。在 Qwen 系列中，随着规模差异的增大，这些收益变得更加明显，我们仅使用 25% 的 token 就实现了 14B 模型性能的 92.9%（74.3% vs 80.0%），从而将计算成本降低了 67%。这些结果在模型对之间的一致性表明，通过根据局部生成复杂性选择性地部署模型容量，可以显著优化语言模型计算。我们的研究结果表明，当前的模型推理方法在追求完美的输出保真度时可能不必要地保守，而接受轻微的性能权衡可以大幅降低计算成本。

Title: TorchResist: Open-Source Differentiable Resist Simulator

• Authors: Zixiao Wang, Jieya Zhou, Su Zheng, Shuo Yin, Kaichao Liang, Shoubo Hu, Xiao Chen, Bei Yu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.06838
• Pdf URL: https://arxiv.org/pdf/2502.06838
• Copy Paste: [[2502.06838]] TorchResist: Open-Source Differentiable Resist Simulator(https://arxiv.org/abs/2502.06838)
• Keywords: generative
• Abstract: Recent decades have witnessed remarkable advancements in artificial intelligence (AI), including large language models (LLMs), image and video generative models, and embodied AI systems. These advancements have led to an explosive increase in the demand for computational power, challenging the limits of Moore's Law. Optical lithography, a critical technology in semiconductor manufacturing, faces significant challenges due to its high costs. To address this, various lithography simulators have been developed. However, many of these simulators are limited by their inadequate photoresist modeling capabilities. This paper presents TorchResist, an open-source, differentiable photoresist this http URL employs an analytical approach to model the photoresist process, functioning as a white-box system with at most twenty interpretable parameters. Leveraging modern differentiable programming techniques and parallel computing on GPUs, TorchResist enables seamless co-optimization with other tools across multiple related tasks. Our experimental results demonstrate that TorchResist achieves superior accuracy and efficiency compared to existing solutions. The source code is publicly available.
• 摘要：近几十年来，人工智能 (AI) 取得了显著进步，包括大型语言模型 (LLM)、图像和视频生成模型以及具身 AI 系统。这些进步导致对计算能力的需求爆炸式增长，挑战了摩尔定律的极限。光学光刻是半导体制造中的一项关键技术，由于成本高昂而面临重大挑战。为了解决这个问题，已经开发了各种光刻模拟器。然而，许多这些模拟器都受到光刻胶建模能力不足的限制。本文介绍了 TorchResist，一种开源的可微分光刻胶，该 http URL 采用分析方法来模拟光刻胶工艺，作为一个白盒系统运行，最多有 20 个可解释的参数。利用现代可微分编程技术和 GPU 上的并行计算，TorchResist 能够与其他工具在多个相关任务中进行无缝协同优化。我们的实验结果表明，与现有解决方案相比，TorchResist 实现了更高的准确性和效率。源代码是公开的。

Title: Vision-Integrated LLMs for Autonomous Driving Assistance : Human Performance Comparison and Trust Evaluation

• Authors: Namhee Kim, Woojin Park
• Subjects: cs.CV, cs.AI, cs.HC
• Abstract URL: https://arxiv.org/abs/2502.06843
• Pdf URL: https://arxiv.org/pdf/2502.06843
• Copy Paste: [[2502.06843]] Vision-Integrated LLMs for Autonomous Driving Assistance : Human Performance Comparison and Trust Evaluation(https://arxiv.org/abs/2502.06843)
• Keywords: generation
• Abstract: Traditional autonomous driving systems often struggle with reasoning in complex, unexpected scenarios due to limited comprehension of spatial relationships. In response, this study introduces a Large Language Model (LLM)-based Autonomous Driving (AD) assistance system that integrates a vision adapter and an LLM reasoning module to enhance visual understanding and decision-making. The vision adapter, combining YOLOv4 and Vision Transformer (ViT), extracts comprehensive visual features, while GPT-4 enables human-like spatial reasoning and response generation. Experimental evaluations with 45 experienced drivers revealed that the system closely mirrors human performance in describing situations and moderately aligns with human decisions in generating appropriate responses.
• 摘要：传统的自动驾驶系统由于对空间关系的理解有限，在复杂、意外的场景中推理往往很困难。为此，本研究引入了一种基于大型语言模型 (LLM) 的自动驾驶 (AD) 辅助系统，该系统集成了视觉适配器和 LLM 推理模块，以增强视觉理解和决策能力。视觉适配器结合了 YOLOv4 和 Vision Transformer (ViT)，提取了全面的视觉特征，而 GPT-4 则实现了类似人类的空间推理和响应生成。对 45 名经验丰富的驾驶员进行的实验评估表明，该系统在描述情况方面与人类的表现非常相似，并且在生成适当的响应方面与人类的决策适度一致。

Title: Prot2Chat: Protein LLM with Early Fusion of Sequence and Structure

• Authors: Zhicong Wang, Zicheng Ma, Ziqiang Cao, Changlong Zhou, Jun Zhang, Yiqin Gao
• Subjects: cs.LG, cs.AI, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2502.06846
• Pdf URL: https://arxiv.org/pdf/2502.06846
• Copy Paste: [[2502.06846]] Prot2Chat: Protein LLM with Early Fusion of Sequence and Structure(https://arxiv.org/abs/2502.06846)
• Keywords: generation
• Abstract: Proteins play a pivotal role in living organisms, yet understanding their functions presents significant challenges, including the limited flexibility of classification-based methods, the inability to effectively leverage spatial structural information, and the lack of systematic evaluation metrics for protein Q&A systems. To address these limitations, we propose Prot2Chat, a novel framework that integrates multimodal protein representations with natural language through a unified module, enabling large language model (LLM)-driven answer generation. Our model incorporates a modified ProteinMPNN encoder, which encodes protein sequence and structural information in a unified manner, a protein-text adapter with cross-attention mechanisms, and a LLaMA3 decoder. To optimize training efficiency, we freeze the encoder and employ LoRA techniques for the decoder. We conducted experiments on two datasets, both automated metrics and expert evaluations demonstrate the superior performance of our model. Furthermore, zero-shot prediction results highlight its strong generalization capabilities. This framework offers a promising solution for bridging protein domain knowledge with natural language understanding, paving the way for transformative advancements in protein-related research.
• 摘要：蛋白质在生物体中起着关键作用，但了解其功能面临着重大挑战，包括基于分类的方法灵活性有限、无法有效利用空间结构信息以及蛋白质问答系统缺乏系统评估指标。为了解决这些限制，我们提出了 Prot2Chat，这是一个新颖的框架，它通过统一的模块将多模态蛋白质表征与自然语言相结合，从而实现大型语言模型 (LLM) 驱动的答案生成。我们的模型结合了改进的 ProteinMPNN 编码器（以统一的方式编码蛋白质序列和结构信息）、具有交叉注意机制的蛋白质文本适配器和 LLaMA3 解码器。为了优化训练效率，我们冻结了编码器并对解码器采用了 LoRA 技术。我们在两个数据集上进行了实验，自动指标和专家评估都证明了我们模型的卓越性能。此外，零样本预测结果凸显了其强大的泛化能力。该框架为将蛋白质领域知识与自然语言理解相结合提供了一种有前途的解决方案，为蛋白质相关研究的变革性进步铺平了道路。

Title: Native Fortran Implementation of TensorFlow-Trained Deep and Bayesian Neural Networks

• Authors: Aidan Furlong, Xingang Zhao, Bob Salko, Xu Wu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.06853
• Pdf URL: https://arxiv.org/pdf/2502.06853
• Copy Paste: [[2502.06853]] Native Fortran Implementation of TensorFlow-Trained Deep and Bayesian Neural Networks(https://arxiv.org/abs/2502.06853)
• Keywords: generation
• Abstract: Over the past decade, the investigation of machine learning (ML) within the field of nuclear engineering has grown significantly. With many approaches reaching maturity, the next phase of investigation will determine the feasibility and usefulness of ML model implementation in a production setting. Several of the codes used for reactor design and assessment are primarily written in the Fortran language, which is not immediately compatible with TensorFlow-trained ML models. This study presents a framework for implementing deep neural networks (DNNs) and Bayesian neural networks (BNNs) in Fortran, allowing for native execution without TensorFlow's C API, Python runtime, or ONNX conversion. Designed for ease of use and computational efficiency, the framework can be implemented in any Fortran code, supporting iterative solvers and UQ via ensembles or BNNs. Verification was performed using a two-input, one-output test case composed of a noisy sinusoid to compare Fortran-based predictions to those from TensorFlow. The DNN predictions showed negligible differences and achieved a 19.6x speedup, whereas the BNN predictions exhibited minor disagreement, plausibly due to differences in random number generation. An 8.0x speedup was noted for BNN inference. The approach was then further verified on a nuclear-relevant problem predicting critical heat flux (CHF), which demonstrated similar behavior along with significant computational gains. Discussion regarding the framework's successful integration into the CTF thermal-hydraulics code is also included, outlining its practical usefulness. Overall, this framework was shown to be effective at implementing both DNN and BNN model inference within Fortran, allowing for the continued study of ML-based methods in real-world nuclear applications.
• 摘要：在过去十年中，核工程领域对机器学习 (ML) 的研究取得了长足的发展。随着许多方法日趋成熟，下一阶段的研究将确定 ML 模型在生产环境中实施的可行性和实用性。用于反应堆设计和评估的多种代码主要用 Fortran 语言编写，该语言与 TensorFlow 训练的 ML 模型并不直接兼容。本研究提出了一个在 Fortran 中实现深度神经网络 (DNN) 和贝叶斯神经网络 (BNN) 的框架，允许在无需 TensorFlow 的 C API、Python 运行时或 ONNX 转换的情况下进行本机执行。该框架易于使用且计算效率高，可以用任何 Fortran 代码实现，通过集成或 BNN 支持迭代求解器和 UQ。使用由噪声正弦波组成的双输入单输出测试用例进行验证，以将基于 Fortran 的预测与来自 TensorFlow 的预测进行比较。 DNN 预测显示出可忽略不计的差异，并实现了 19.6 倍的加速，而 BNN 预测表现出轻微的分歧，这可能是由于随机数生成的差异造成的。BNN 推理的速度提高了 8.0 倍。该方法随后在预测临界热通量 (CHF) 的核相关问题上得到了进一步验证，结果显示其行为相似，同时计算量显著增加。本文还讨论了该框架如何成功集成到 CTF 热工水力学代码中，概述了其实际用途。总体而言，该框架被证明能够有效地在 Fortran 中实现 DNN 和 BNN 模型推理，从而可以继续研究基于 ML 的方法在现实世界核应用中的应用。

Title: AutoSketch: VLM-assisted Style-Aware Vector Sketch Completion

• Authors: Hsiao-Yuan Chin, I-Chao Shen, Yi-Ting Chiu, Bing-Yu Chen
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.06860
• Pdf URL: https://arxiv.org/pdf/2502.06860
• Copy Paste: [[2502.06860]] AutoSketch: VLM-assisted Style-Aware Vector Sketch Completion(https://arxiv.org/abs/2502.06860)
• Keywords: generation
• Abstract: The ability to automatically complete a partial sketch that depicts a complex scene, e.g., "a woman chatting with a man in the park", is very useful. However, existing sketch generation methods create sketches from scratch; they do not complete a partial sketch in the style of the original. To address this challenge, we introduce AutoSketch, a styleaware vector sketch completion method that accommodates diverse sketch styles. Our key observation is that the style descriptions of a sketch in natural language preserve the style during automatic sketch completion. Thus, we use a pretrained vision-language model (VLM) to describe the styles of the partial sketches in natural language and replicate these styles using newly generated strokes. We initially optimize the strokes to match an input prompt augmented by style descriptions extracted from the VLM. Such descriptions allow the method to establish a diffusion prior in close alignment with that of the partial sketch. Next, we utilize the VLM to generate an executable style adjustment code that adjusts the strokes to conform to the desired style. We compare our method with existing methods across various sketch styles and prompts, performed extensive ablation studies and qualitative and quantitative evaluations, and demonstrate that AutoSketch can support various sketch scenarios.
• 摘要：自动完成描绘复杂场景的部分草图（例如“公园里的女人和男人聊天”）的能力非常有用。然而，现有的草图生成方法都是从头开始创建草图；它们无法以原始风格完成部分草图。为了应对这一挑战，我们引入了 AutoSketch，这是一种可感知风格的矢量草图完成方法，可以适应多种草图风格。我们的主要观察结果是，自然语言中草图的风格描述在自动草图完成过程中保留了风格。因此，我们使用预训练的视觉语言模型 (VLM) 以自然语言描述部分草图的风格，并使用新生成的笔画复制这些风格。我们最初优化笔画以匹配输入提示，并通过从 VLM 中提取的风格描述进行增强。这样的描述允许该方法建立与部分草图紧密一致的扩散先验。接下来，我们利用 VLM 生成可执行的风格调整代码，调整笔画以符合所需的风格。我们将我们的方法与各种草图风格和提示中的现有方法进行了比较，进行了广泛的消融研究和定性和定量评估，并证明 AutoSketch 可以支持各种草图场景。

Title: BF-GAN: Development of an AI-driven Bubbly Flow Image Generation Model Using Generative Adversarial Networks

• Authors: Wen Zhou, Shuichiro Miwa, Yang Liu, Koji Okamoto
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.06863
• Pdf URL: https://arxiv.org/pdf/2502.06863
• Copy Paste: [[2502.06863]] BF-GAN: Development of an AI-driven Bubbly Flow Image Generation Model Using Generative Adversarial Networks(https://arxiv.org/abs/2502.06863)
• Keywords: generation, generative
• Abstract: A generative AI architecture called bubbly flow generative adversarial networks (BF-GAN) is developed, designed to generate realistic and high-quality bubbly flow images through physically conditioned inputs, jg and jf. Initially, 52 sets of bubbly flow experiments under varying conditions are conducted to collect 140,000 bubbly flow images with physical labels of jg and jf for training data. A multi-scale loss function is then developed, incorporating mismatch loss and pixel loss to enhance the generative performance of BF-GAN further. Regarding evaluative metrics of generative AI, the BF-GAN has surpassed conventional GAN. Physically, key parameters of bubbly flow generated by BF-GAN are extracted and compared with measurement values and empirical correlations, validating BF-GAN's generative performance. The comparative analysis demonstrate that the BF-GAN can generate realistic and high-quality bubbly flow images with any given jg and jf within the research scope. BF-GAN offers a generative AI solution for two-phase flow research, substantially lowering the time and cost required to obtain high-quality data. In addition, it can function as a benchmark dataset generator for bubbly flow detection and segmentation algorithms, enhancing overall productivity in this research domain. The BF-GAN model is available online (this https URL).
• 摘要：开发了一种称为气泡流生成对抗网络 (BF-GAN) 的生成式 AI 架构，旨在通过物理条件输入 jg 和 jf 生成逼真的高质量气泡流图像。首先，进行了 52 组不同条件下的气泡流实验，以收集 140,000 个具有物理标签 jg 和 jf 的气泡流图像作为训练数据。然后开发了一种多尺度损失函数，结合失配损失和像素损失，以进一步增强 BF-GAN 的生成性能。在生成式 AI 的评估指标方面，BF-GAN 已经超越了传统 GAN。在物理上，提取了 BF-GAN 生成的气泡流的关键参数，并将其与测量值和经验相关性进行比较，验证了 BF-GAN 的生成性能。比较分析表明，BF-GAN 可以在研究范围内使用任何给定的 jg 和 jf 生成逼真的高质量气泡流图像。 BF-GAN 为两相流研究提供了生成式 AI 解决方案，大大降低了获取高质量数据所需的时间和成本。此外，它还可以作为气泡流检测和分割算法的基准数据集生成器，从而提高该研究领域的整体生产力。BF-GAN 模型可在线获取（此 https URL）。

Title: WirelessGPT: A Generative Pre-trained Multi-task Learning Framework for Wireless Communication

• Authors: Tingting Yang, Ping Zhang, Mengfan Zheng, Yuxuan Shi, Liwen Jing, Jianbo Huang, Nan Li
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.06877
• Pdf URL: https://arxiv.org/pdf/2502.06877
• Copy Paste: [[2502.06877]] WirelessGPT: A Generative Pre-trained Multi-task Learning Framework for Wireless Communication(https://arxiv.org/abs/2502.06877)
• Keywords: generative
• Abstract: This paper introduces WirelessGPT, a pioneering foundation model specifically designed for multi-task learning in wireless communication and sensing. Specifically, WirelessGPT leverages large-scale wireless channel datasets for unsupervised pretraining and extracting universal channel representations, which captures complex spatiotemporal dependencies. In fact,this task-agnostic design adapts WirelessGPT seamlessly to a wide range of downstream tasks, using a unified representation with minimal fine-tuning. By unifying communication and sensing functionalities, WirelessGPT addresses the limitations of task-specific models, offering a scalable and efficient solution for integrated sensing and communication (ISAC). With an initial parameter size of around 80 million, WirelessGPT demonstrates significant improvements over conventional methods and smaller AI models, reducing reliance on large-scale labeled data. As the first foundation model capable of supporting diverse tasks across different domains, WirelessGPT establishes a new benchmark, paving the way for future advancements in multi-task wireless systems.
• 摘要：本文介绍了 WirelessGPT，这是一种专为无线通信和传感中的多任务学习而设计的开创性基础模型。具体来说，WirelessGPT 利用大规模无线信道数据集进行无监督预训练并提取通用信道表示，从而捕获复杂的时空依赖关系。事实上，这种与任务无关的设计使 WirelessGPT 能够无缝适应各种下游任务，使用统一的表示并进行最少的微调。通过统一通信和传感功能，WirelessGPT 解决了任务特定模型的局限性，为集成传感和通信 (ISAC) 提供了可扩展且高效的解决方案。WirelessGPT 的初始参数大小约为 8000 万，与传统方法和较小的 AI 模型相比有显著改进，减少了对大规模标记数据的依赖。作为第一个能够支持不同领域不同任务的基础模型，WirelessGPT 建立了新的基准，为多任务无线系统的未来发展铺平了道路。

Title: Learning Conformal Abstention Policies for Adaptive Risk Management in Large Language and Vision-Language Models

• Authors: Sina Tayebati, Divake Kumar, Nastaran Darabi, Dinithi Jayasuriya, Ranganath Krishnan, Amit Ranjan Trivedi
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.06884
• Pdf URL: https://arxiv.org/pdf/2502.06884
• Copy Paste: [[2502.06884]] Learning Conformal Abstention Policies for Adaptive Risk Management in Large Language and Vision-Language Models(https://arxiv.org/abs/2502.06884)
• Keywords: generation
• Abstract: Large Language and Vision-Language Models (LLMs/VLMs) are increasingly used in safety-critical applications, yet their opaque decision-making complicates risk assessment and reliability. Uncertainty quantification (UQ) helps assess prediction confidence and enables abstention when uncertainty is high. Conformal prediction (CP), a leading UQ method, provides statistical guarantees but relies on static thresholds, which fail to adapt to task complexity and evolving data distributions, leading to suboptimal trade-offs in accuracy, coverage, and informativeness. To address this, we propose learnable conformal abstention, integrating reinforcement learning (RL) with CP to optimize abstention thresholds dynamically. By treating CP thresholds as adaptive actions, our approach balances multiple objectives, minimizing prediction set size while maintaining reliable coverage. Extensive evaluations across diverse LLM/VLM benchmarks show our method outperforms Least Ambiguous Classifiers (LAC) and Adaptive Prediction Sets (APS), improving accuracy by up to 3.2%, boosting AUROC for hallucination detection by 22.19%, enhancing uncertainty-guided selective generation (AUARC) by 21.17%, and reducing calibration error by 70%-85%. These improvements hold across multiple models and datasets while consistently meeting the 90% coverage target, establishing our approach as a more effective and flexible solution for reliable decision-making in safety-critical applications. The code is available at: {this https URL}.
• 摘要：大型语言和视觉语言模型 (LLM/VLM) 在安全关键应用中的使用越来越多，但其不透明的决策使风险评估和可靠性变得复杂。不确定性量化 (UQ) 有助于评估预测信心，并在不确定性较高时启用弃权。共形预测 (CP) 是一种领先的 UQ 方法，它提供了统计保证，但依赖于静态阈值，无法适应任务复杂性和不断变化的数据分布，导致准确性、覆盖率和信息量方面的权衡不理想。为了解决这个问题，我们提出了可学习的共形弃权，将强化学习 (RL) 与 CP 相结合，以动态优化弃权阈值。通过将 CP 阈值视为自适应动作，我们的方法平衡了多个目标，最小化预测集大小，同时保持可靠的覆盖范围。在各种 LLM/VLM 基准上进行的广泛评估表明，我们的方法优于最小模糊分类器 (LAC) 和自适应预测集 (APS)，准确率提高了 3.2%，幻觉检测的 AUROC 提高了 22.19%，不确定性引导的选择性生成 (AUARC) 提高了 21.17%，校准误差降低了 70%-85%。这些改进适用于多个模型和数据集，同时始终满足 90% 的覆盖率目标，使我们的方法成为一种更有效、更灵活的解决方案，可在安全关键型应用中进行可靠的决策。代码可在以下位置获取：{此 https URL}。

Title: Can ChatGPT Diagnose Alzheimer's Disease?

• Authors: Quoc-Toan Nguyen, Linh Le, Xuan-The Tran, Thomas Do, Chin-Teng Lin
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.06907
• Pdf URL: https://arxiv.org/pdf/2502.06907
• Copy Paste: [[2502.06907]] Can ChatGPT Diagnose Alzheimer's Disease?(https://arxiv.org/abs/2502.06907)
• Keywords: generative
• Abstract: Can ChatGPT diagnose Alzheimer's Disease (AD)? AD is a devastating neurodegenerative condition that affects approximately 1 in 9 individuals aged 65 and older, profoundly impairing memory and cognitive function. This paper utilises 9300 electronic health records (EHRs) with data from Magnetic Resonance Imaging (MRI) and cognitive tests to address an intriguing question: As a general-purpose task solver, can ChatGPT accurately detect AD using EHRs? We present an in-depth evaluation of ChatGPT using a black-box approach with zero-shot and multi-shot methods. This study unlocks ChatGPT's capability to analyse MRI and cognitive test results, as well as its potential as a diagnostic tool for AD. By automating aspects of the diagnostic process, this research opens a transformative approach for the healthcare system, particularly in addressing disparities in resource-limited regions where AD specialists are scarce. Hence, it offers a foundation for a promising method for early detection, supporting individuals with timely interventions, which is paramount for Quality of Life (QoL).
• 摘要：ChatGPT 能否诊断阿尔茨海默病 (AD)？AD 是一种毁灭性的神经退行性疾病，影响大约 1/9 的 65 岁以上人群，严重损害记忆力和认知功能。本文利用 9300 份电子健康记录 (EHR) 以及来自磁共振成像 (MRI) 和认知测试的数据来解决一个有趣的问题：作为通用任务解决器，ChatGPT 能否使用 EHR 准确检测 AD？我们使用黑盒方法、零样本和多样本方法对 ChatGPT 进行了深入评估。这项研究释放了 ChatGPT 分析 MRI 和认知测试结果的能力，以及它作为 AD 诊断工具的潜力。通过自动化诊断过程的各个方面，这项研究为医疗保健系统开辟了一种变革性方法，特别是在解决资源有限、AD 专家稀缺的地区存在的差距方面。因此，它为一种有前景的早期检测方法奠定了基础，支持个人及时干预，这对生活质量（QoL）至关重要。

Title: GraNNite: Enabling High-Performance Execution of Graph Neural Networks on Resource-Constrained Neural Processing Units

• Authors: Arghadip Das, Shamik Kundu, Arnab Raha, Soumendu Ghosh, Deepak Mathaikutty, Vijay Raghunathan
• Subjects: cs.LG, cs.AI, cs.AR
• Abstract URL: https://arxiv.org/abs/2502.06921
• Pdf URL: https://arxiv.org/pdf/2502.06921
• Copy Paste: [[2502.06921]] GraNNite: Enabling High-Performance Execution of Graph Neural Networks on Resource-Constrained Neural Processing Units(https://arxiv.org/abs/2502.06921)
• Keywords: generation
• Abstract: Graph Neural Networks (GNNs) are vital for learning from graph-structured data, enabling applications in network analysis, recommendation systems, and speech analytics. Deploying them on edge devices like client PCs and laptops enhances real-time processing, privacy, and cloud independence. GNNs aid Retrieval-Augmented Generation (RAG) for Large Language Models (LLMs) and enable event-based vision tasks. However, irregular memory access, sparsity, and dynamic structures cause high latency and energy overhead on resource-constrained devices. While modern edge processors integrate CPUs, GPUs, and NPUs, NPUs designed for data-parallel tasks struggle with irregular GNN computations. We introduce GraNNite, the first hardware-aware framework optimizing GNN execution on commercial-off-the-shelf (COTS) SOTA DNN accelerators via a structured three-step methodology: (1) enabling NPU execution, (2) optimizing performance, and (3) trading accuracy for efficiency gains. Step 1 employs GraphSplit for workload distribution and StaGr for static aggregation, while GrAd and NodePad handle dynamic graphs. Step 2 boosts performance using EffOp for control-heavy tasks and GraSp for sparsity exploitation. Graph Convolution optimizations PreG, SymG, and CacheG reduce redundancy and memory transfers. Step 3 balances quality versus efficiency, where QuantGr applies INT8 quantization, and GrAx1, GrAx2, and GrAx3 accelerate attention, broadcast-add, and SAGE-max aggregation. On Intel Core Ultra AI PCs, GraNNite achieves 2.6X to 7.6X speedups over default NPU mappings and up to 8.6X energy gains over CPUs and GPUs, delivering 10.8X and 6.7X higher performance than CPUs and GPUs, respectively, across GNN models.
• 摘要：图神经网络 (GNN) 对于从图结构数据中学习至关重要，可实现网络分析、推荐系统和语音分析中的应用。将它们部署在客户端 PC 和笔记本电脑等边缘设备上可增强实时处理、隐私和云独立性。GNN 有助于大型语言模型 (LLM) 的检索增强生成 (RAG)，并支持基于事件的视觉任务。然而，不规则的内存访问、稀疏性和动态结构会导致资源受限的设备上出现高延迟和能源开销。虽然现代边缘处理器集成了 CPU、GPU 和 NPU，但为数据并行任务设计的 NPU 难以应对不规则的 GNN 计算。我们推出了 GraNNite，这是第一个硬件感知框架，它通过结构化的三步方法优化商用现货 (COTS) SOTA DNN 加速器上的 GNN 执行：(1) 启用 NPU 执行，(2) 优化性能，以及 (3) 以准确性换取效率提升。步骤 1 使用 GraphSplit 进行工作负载分配，使用 StaGr 进行静态聚合，而 GrAd 和 NodePad 处理动态图。步骤 2 使用 EffOp 执行控制繁重的任务，使用 GraSp 进行稀疏性利用，从而提高性能。图卷积优化 PreG、SymG 和 CacheG 减少了冗余和内存传输。步骤 3 平衡质量与效率，其中 QuantGr 应用 INT8 量化，GrAx1、GrAx2 和 GrAx3 加速注意力、广播添加和 SAGE-max 聚合。在英特尔酷睿超 AI PC 上，GraNNite 比默认 NPU 映射实现了 2.6 倍到 7.6 倍的加速，比 CPU 和 GPU 实现了高达 8.6 倍的能量增益，在 GNN 模型中分别比 CPU 和 GPU 提供 10.8 倍和 6.7 倍的性能。

Title: GAS: Generative Avatar Synthesis from a Single Image

• Authors: Yixing Lu, Junting Dong, Youngjoong Kwon, Qin Zhao, Bo Dai, Fernando De la Torre
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.06957
• Pdf URL: https://arxiv.org/pdf/2502.06957
• Copy Paste: [[2502.06957]] GAS: Generative Avatar Synthesis from a Single Image(https://arxiv.org/abs/2502.06957)
• Keywords: generation, generative
• Abstract: We introduce a generalizable and unified framework to synthesize view-consistent and temporally coherent avatars from a single image, addressing the challenging problem of single-image avatar generation. While recent methods employ diffusion models conditioned on human templates like depth or normal maps, they often struggle to preserve appearance information due to the discrepancy between sparse driving signals and the actual human subject, resulting in multi-view and temporal inconsistencies. Our approach bridges this gap by combining the reconstruction power of regression-based 3D human reconstruction with the generative capabilities of a diffusion model. The dense driving signal from the initial reconstructed human provides comprehensive conditioning, ensuring high-quality synthesis faithful to the reference appearance and structure. Additionally, we propose a unified framework that enables the generalization learned from novel pose synthesis on in-the-wild videos to naturally transfer to novel view synthesis. Our video-based diffusion model enhances disentangled synthesis with high-quality view-consistent renderings for novel views and realistic non-rigid deformations in novel pose animation. Results demonstrate the superior generalization ability of our method across in-domain and out-of-domain in-the-wild datasets. Project page: this https URL
• 摘要：我们引入了一个可通用的统一框架，用于从单个图像合成视图一致且时间连贯的头像，解决了单图像头像生成的难题。虽然最近的方法采用了以深度或法线图等人体模板为条件的扩散模型，但由于稀疏驱动信号与实际人体主体之间的差异，它们往往难以保留外观信息，从而导致多视图和时间不一致。我们的方法通过将基于回归的 3D 人体重建的重建能力与扩散模型的生成能力相结合来弥补这一差距。来自初始重建人体的密集驱动信号提供了全面的调节，确保了忠实于参考外观和结构的高质量合成。此外，我们提出了一个统一的框架，使从野外视频的新姿势合成中学习到的泛化能够自然地转移到新视图合成中。我们的基于视频的扩散模型通过高质量视图一致渲染增强了解缠结合成，以实现新姿势动画中逼真的非刚性变形。结果表明，我们的方法在域内和域外的野外数据集中具有出色的泛化能力。项目页面：此 https URL

Title: Outsourced diffusion sampling: Efficient posterior inference in latent spaces of generative models

• Authors: Siddarth Venkatraman, Mohsin Hasan, Minsu Kim, Luca Scimeca, Marcin Sendera, Yoshua Bengio, Glen Berseth, Nikolay Malkin
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.06999
• Pdf URL: https://arxiv.org/pdf/2502.06999
• Copy Paste: [[2502.06999]] Outsourced diffusion sampling: Efficient posterior inference in latent spaces of generative models(https://arxiv.org/abs/2502.06999)
• Keywords: generation, generative
• Abstract: Any well-behaved generative model over a variable $\mathbf{x}$ can be expressed as a deterministic transformation of an exogenous ('outsourced') Gaussian noise variable $\mathbf{z}$: $\mathbf{x}=f_\theta(\mathbf{z})$. In such a model (e.g., a VAE, GAN, or continuous-time flow-based model), sampling of the target variable $\mathbf{x} \sim p_\theta(\mathbf{x})$ is straightforward, but sampling from a posterior distribution of the form $p(\mathbf{x}\mid\mathbf{y}) \propto p_\theta(\mathbf{x})r(\mathbf{x},\mathbf{y})$, where $r$ is a constraint function depending on an auxiliary variable $\mathbf{y}$, is generally intractable. We propose to amortize the cost of sampling from such posterior distributions with diffusion models that sample a distribution in the noise space ($\mathbf{z}$). These diffusion samplers are trained by reinforcement learning algorithms to enforce that the transformed samples $f_\theta(\mathbf{z})$ are distributed according to the posterior in the data space ($\mathbf{x}$). For many models and constraints of interest, the posterior in the noise space is smoother than the posterior in the data space, making it more amenable to such amortized inference. Our method enables conditional sampling under unconditional GAN, (H)VAE, and flow-based priors, comparing favorably both with current amortized and non-amortized inference methods. We demonstrate the proposed outsourced diffusion sampling in several experiments with large pretrained prior models: conditional image generation, reinforcement learning with human feedback, and protein structure generation.
• 摘要：任何针对变量 $\mathbf{x}$ 的行为良好的生成模型都可以表示为外生（“外包”）高斯噪声变量 $\mathbf{z}$ 的确定性变换：$\mathbf{x}=f_\theta(\mathbf{z})$。在这样的模型中（例如，VAE、GAN 或基于连续时间流的模型），目标变量 $\mathbf{x} \sim p_\theta(\mathbf{x})$ 的采样很简单，但从形式为 $p(\mathbf{x}\mid\mathbf{y}) \propto p_\theta(\mathbf{x})r(\mathbf{x},\mathbf{y})$ 的后验分布中采样通常是难以处理的，其中 $r$ 是依赖于辅助变量 $\mathbf{y}$ 的约束函数。我们建议使用在噪声空间 ($\mathbf{z}$ 中采样分布的扩散模型来摊销从此类后验分布中采样的成本。这些扩散采样器由强化学习算法训练，以强制转换后的样本 $f_\theta(\mathbf{z})$ 根据数据空间 ($\mathbf{x}$) 中的后验分布。对于许多感兴趣的模型和约束，噪声空间中的后验比数据空间中的后验更平滑，使其更适合这种摊销推理。我们的方法能够在无条件 GAN、(H)VAE 和基于流的先验下进行条件采样，与当前的摊销和非摊销推理方法相比均具有优势。我们在几个具有大型预训练先验模型的实验中展示了所提出的外包扩散采样：条件图像生成、带有人工反馈的强化学习和蛋白质结构生成。

Title: From Image to Video: An Empirical Study of Diffusion Representations

• Authors: Pedro Vélez, Luisa F. Polanía, Yi Yang, Chuhan Zhang, Rishab Kabra, Anurag Arnab, Mehdi S. M. Sajjadi
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07001
• Pdf URL: https://arxiv.org/pdf/2502.07001
• Copy Paste: [[2502.07001]] From Image to Video: An Empirical Study of Diffusion Representations(https://arxiv.org/abs/2502.07001)
• Keywords: generation, generative
• Abstract: Diffusion models have revolutionized generative modeling, enabling unprecedented realism in image and video synthesis. This success has sparked interest in leveraging their representations for visual understanding tasks. While recent works have explored this potential for image generation, the visual understanding capabilities of video diffusion models remain largely uncharted. To address this gap, we systematically compare the same model architecture trained for video versus image generation, analyzing the performance of their latent representations on various downstream tasks including image classification, action recognition, depth estimation, and tracking. Results show that video diffusion models consistently outperform their image counterparts, though we find a striking range in the extent of this superiority. We further analyze features extracted from different layers and with varying noise levels, as well as the effect of model size and training budget on representation and generation quality. This work marks the first direct comparison of video and image diffusion objectives for visual understanding, offering insights into the role of temporal information in representation learning.
• 摘要：扩散模型彻底改变了生成模型，使图像和视频合成具有前所未有的真实感。这一成功激发了人们对利用其表示进行视觉理解任务的兴趣。虽然最近的研究已经探索了这种图像生成的潜力，但视频扩散模型的视觉理解能力仍然很大程度上未知。为了解决这一差距，我们系统地比较了为视频和图像生成训练的相同模型架构，分析了它们在各种下游任务中的潜在表示的性能，包括图像分类、动作识别、深度估计和跟踪。结果表明，视频扩散模型的表现始终优于图像扩散模型，尽管我们发现这种优越性的程度范围惊人。我们进一步分析了从不同层和不同噪声水平提取的特征，以及模型大小和训练预算对表示和生成质量的影响。这项工作标志着首次直接比较视频和图像扩散目标以实现视觉理解，深入了解了时间信息在表示学习中的作用。

Title: Grounding Creativity in Physics: A Brief Survey of Physical Priors in AIGC

• Authors: Siwei Meng, Yawei Luo, Ping Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07007
• Pdf URL: https://arxiv.org/pdf/2502.07007
• Copy Paste: [[2502.07007]] Grounding Creativity in Physics: A Brief Survey of Physical Priors in AIGC(https://arxiv.org/abs/2502.07007)
• Keywords: generation, generative
• Abstract: Recent advancements in AI-generated content have significantly improved the realism of 3D and 4D generation. However, most existing methods prioritize appearance consistency while neglecting underlying physical principles, leading to artifacts such as unrealistic deformations, unstable dynamics, and implausible objects interactions. Incorporating physics priors into generative models has become a crucial research direction to enhance structural integrity and motion realism. This survey provides a review of physics-aware generative methods, systematically analyzing how physical constraints are integrated into 3D and 4D generation. First, we examine recent works in incorporating physical priors into static and dynamic 3D generation, categorizing methods based on representation types, including vision-based, NeRF-based, and Gaussian Splatting-based approaches. Second, we explore emerging techniques in 4D generation, focusing on methods that model temporal dynamics with physical simulations. Finally, we conduct a comparative analysis of major methods, highlighting their strengths, limitations, and suitability for different materials and motion dynamics. By presenting an in-depth analysis of physics-grounded AIGC, this survey aims to bridge the gap between generative models and physical realism, providing insights that inspire future research in physically consistent content generation.
• 摘要：人工智能生成内容的最新进展显著提高了 3D 和 4D 生成的真实感。然而，大多数现有方法优先考虑外观一致性，而忽略了底层物理原理，导致不切实际的变形、不稳定的动态和不合理的物体相互作用等伪影。将物理先验融入生成模型已成为增强结构完整性和运动真实感的重要研究方向。本综述回顾了物理感知生成方法，系统地分析了物理约束如何融入 3D 和 4D 生成。首先，我们研究了最近将物理先验融入静态和动态 3D 生成中的研究，根据表示类型对方法进行分类，包括基于视觉、基于 NeRF 和基于高斯 Splatting 的方法。其次，我们探索 4D 生成的新兴技术，重点关注使用物理模拟对时间动态进行建模的方法。最后，我们对主要方法进行了比较分析，强调了它们的优势、局限性以及对不同材料和运动动力学的适用性。通过对基于物理的 AIGC 进行深入分析，本调查旨在弥合生成模型与物理现实之间的差距，提供启发未来物理一致内容生成研究的见解。

Title: Detecting Neurodegenerative Diseases using Frame-Level Handwriting Embeddings

• Authors: Sarah Laouedj, Yuzhe Wang, Jesus Villalba, Thomas Thebaud, Laureano Moro-Velazquez, Najim Dehak
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07025
• Pdf URL: https://arxiv.org/pdf/2502.07025
• Copy Paste: [[2502.07025]] Detecting Neurodegenerative Diseases using Frame-Level Handwriting Embeddings(https://arxiv.org/abs/2502.07025)
• Keywords: generative
• Abstract: In this study, we explored the use of spectrograms to represent handwriting signals for assessing neurodegenerative diseases, including 42 healthy controls (CTL), 35 subjects with Parkinson's Disease (PD), 21 with Alzheimer's Disease (AD), and 15 with Parkinson's Disease Mimics (PDM). We applied CNN and CNN-BLSTM models for binary classification using both multi-channel fixed-size and frame-based spectrograms. Our results showed that handwriting tasks and spectrogram channel combinations significantly impacted classification performance. The highest F1-score (89.8%) was achieved for AD vs. CTL, while PD vs. CTL reached 74.5%, and PD vs. PDM scored 77.97%. CNN consistently outperformed CNN-BLSTM. Different sliding window lengths were tested for constructing frame-based spectrograms. A 1-second window worked best for AD, longer windows improved PD classification, and window length had little effect on PD vs. PDM.
• 摘要：在本研究中，我们探索了使用声谱图表示手写信号以评估神经退行性疾病的方法，其中包括 42 名健康对照者 (CTL)、35 名帕金森病 (PD) 患者、21 名阿尔茨海默病 (AD) 患者和 15 名帕金森病模仿者 (PDM)。我们应用 CNN 和 CNN-BLSTM 模型进行二元分类，使用多通道固定大小和基于帧的声谱图。我们的结果表明，手写任务和声谱图通道组合显著影响分类性能。AD vs. CTL 的 F1 分数最高 (89.8%)，而 PD vs. CTL 达到 74.5%，PD vs. PDM 得分为 77.97%。CNN 的表现始终优于 CNN-BLSTM。测试了不同的滑动窗口长度来构建基于帧的声谱图。 1 秒的窗口最适合 AD，较长的窗口可改善 PD 分类，而窗口长度对 PD 与 PDM 影响不大。

Title: Contextual Thompson Sampling via Generation of Missing Data

• Authors: Kelly W. Zhang, Tiffany Tianhui Cai, Hongseok Namkoong, Daniel Russo
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.07064
• Pdf URL: https://arxiv.org/pdf/2502.07064
• Copy Paste: [[2502.07064]] Contextual Thompson Sampling via Generation of Missing Data(https://arxiv.org/abs/2502.07064)
• Keywords: generation, generative
• Abstract: We introduce a framework for Thompson sampling contextual bandit algorithms, in which the algorithm's ability to quantify uncertainty and make decisions depends on the quality of a generative model that is learned offline. Instead of viewing uncertainty in the environment as arising from unobservable latent parameters, our algorithm treats uncertainty as stemming from missing, but potentially observable, future outcomes. If these future outcomes were all observed, one could simply make decisions using an "oracle" policy fit on the complete dataset. Inspired by this conceptualization, at each decision-time, our algorithm uses a generative model to probabilistically impute missing future outcomes, fits a policy using the imputed complete dataset, and uses that policy to select the next action. We formally show that this algorithm is a generative formulation of Thompson Sampling and prove a state-of-the-art regret bound for it. Notably, our regret bound i) depends on the probabilistic generative model only through the quality of its offline prediction loss, and ii) applies to any method of fitting the "oracle" policy, which easily allows one to adapt Thompson sampling to decision-making settings with fairness and/or resource constraints.
• 摘要：我们引入了汤普森抽样上下文老虎机算法的框架，其中算法量化不确定性和做出决策的能力取决于离线学习的生成模型的质量。我们的算法不是将环境中的不确定性视为来自不可观察的潜在参数，而是将不确定性视为来自缺失但可能可观察的未来结果。如果这些未来结果都被观察到，那么人们可以简单地使用适合完整数据集的“预言机”策略做出决策。受此概念的启发，在每个决策时间，我们的算法使用生成模型以概率方式估算缺失的未来结果，使用估算的完整数据集拟合策略，并使用该策略选择下一个动作。我们正式表明该算法是汤普森抽样的生成公式，并证明了其最先进的遗憾界限。值得注意的是，我们的遗憾界限 i）仅通过其离线预测损失的质量取决于概率生成模型，以及 ii）适用于任何适合“oracle”策略的方法，这可以轻松地让人们将汤普森抽样适应具有公平性和/或资源约束的决策设置。

Title: Evaluating the Systematic Reasoning Abilities of Large Language Models through Graph Coloring

• Authors: Alex Heyman, Joel Zylberberg
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07087
• Pdf URL: https://arxiv.org/pdf/2502.07087
• Copy Paste: [[2502.07087]] Evaluating the Systematic Reasoning Abilities of Large Language Models through Graph Coloring(https://arxiv.org/abs/2502.07087)
• Keywords: generation
• Abstract: Contemporary large language models are powerful problem-solving tools, but they exhibit weaknesses in their reasoning abilities which ongoing research seeks to mitigate. We investigate graph coloring as a means of evaluating an LLM's capacities for systematic step-by-step reasoning and possibility space exploration, as well as effects of semantic problem framing. We test Claude 3.5 Sonnet, Llama 3.1 405B, Gemini 1.5 Pro, GPT-4o, o1-mini, and DeepSeek-R1 on a dataset of $k$-coloring problems with $2 \leq k \leq 4$ and vertex count $4 \leq n \leq 8$, using partial algorithmic solvers to further categorize problems by difficulty. In addition to substantial but varying framing effects, we find that all models except o1-mini and R1 exhibit $>60\%$ error rates on difficult problem types in all frames ($>15\%$ for o1-mini and $>10\%$ for R1), and no model achieves perfect accuracy even in the simple domain of 2-coloring 4-vertex graphs. Our results highlight both the considerable recent progress in LLM systematic reasoning and the limits of its reliability, especially in relation to increasing computational costs. We expect that more complex graph coloring problems, and procedural generation of arbitrary-complexity reasoning problems more broadly, offer further untapped potential for LLM benchmarking.
• 摘要：当代大型语言模型是强大的问题解决工具，但它们在推理能力方面存在弱点，正在进行的研究旨在缓解这一弱点。我们研究了图形着色作为评估 LLM 系统逐步推理和可能性空间探索能力以及语义问题框架效果的一种手段。我们在 $k$ 着色问题数据集上测试了 Claude 3.5 Sonnet、Llama 3.1 405B、Gemini 1.5 Pro、GPT-4o、o1-mini 和 DeepSeek-R1，其中 $2 \leq k \leq 4$ 和顶点数 $4 \leq n \leq 8$，使用部分算法求解器进一步按难度对问题进行分类。除了显著但不同的框架效应外，我们发现除 o1-mini 和 R1 之外的所有模型在所有框架中对困难问题类型的错误率均超过 60%（o1-mini 为 15% 以上，R1 为 10% 以上），并且即使在简单的 2 着色 4 顶点图域中，也没有模型能够达到完美的准确率。我们的研究结果既突出了 LLM 系统推理方面近期取得的重大进展，也突出了其可靠性的局限性，尤其是在计算成本不断增加的情况下。我们预计，更复杂的图着色问题和更广泛地任意复杂度推理问题的程序生成将为 LLM 基准测试提供更多尚未开发的潜力。

Title: Likelihood-Free Estimation for Spatiotemporal Hawkes processes with missing data and application to predictive policing

• Authors: Pramit Das, Moulinath Banerjee, Yuekai Sun
• Subjects: cs.LG, stat.AP, stat.ME
• Abstract URL: https://arxiv.org/abs/2502.07111
• Pdf URL: https://arxiv.org/pdf/2502.07111
• Copy Paste: [[2502.07111]] Likelihood-Free Estimation for Spatiotemporal Hawkes processes with missing data and application to predictive policing(https://arxiv.org/abs/2502.07111)
• Keywords: generative
• Abstract: With the growing use of AI technology, many police departments use forecasting software to predict probable crime hotspots and allocate patrolling resources effectively for crime prevention. The clustered nature of crime data makes self-exciting Hawkes processes a popular modeling choice. However, one significant challenge in fitting such models is the inherent missingness in crime data due to non-reporting, which can bias the estimated parameters of the predictive model, leading to inaccurate downstream hotspot forecasts, often resulting in over or under-policing in various communities, especially the vulnerable ones. Our work introduces a Wasserstein Generative Adversarial Networks (WGAN) driven likelihood-free approach to account for unreported crimes in Spatiotemporal Hawkes models. We demonstrate through empirical analysis how this methodology improves the accuracy of parametric estimation in the presence of data missingness, leading to more reliable and efficient policing strategies.
• 摘要：随着人工智能技术的广泛应用，许多警察部门使用预测软件来预测可能的犯罪热点，并有效分配巡逻资源以预防犯罪。犯罪数据的聚类性质使自激霍克斯过程成为一种流行的建模选择。然而，拟合此类模型的一个重大挑战是由于未报告而导致的犯罪数据固有缺失，这可能会使预测模型的估计参数产生偏差，导致下游热点预测不准确，通常导致各个社区（尤其是脆弱社区）的警务过度或不足。我们的工作引入了一种由 Wasserstein 生成对抗网络 (WGAN) 驱动的无似然方法来解释时空霍克斯模型中的未报告犯罪。我们通过实证分析证明了这种方法如何在数据缺失的情况下提高参数估计的准确性，从而带来更可靠、更有效的警务策略。

Title: Conditional Distribution Quantization in Machine Learning

• Authors: Blaise Delattre, Sylvain Delattre, Alexandre Vérine, Alexandre Allauzen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07151
• Pdf URL: https://arxiv.org/pdf/2502.07151
• Copy Paste: [[2502.07151]] Conditional Distribution Quantization in Machine Learning(https://arxiv.org/abs/2502.07151)
• Keywords: generation
• Abstract: Conditional expectation \mathbb{E}(Y \mid X) often fails to capture the complexity of multimodal conditional distributions \mathcal{L}(Y \mid X). To address this, we propose using n-point conditional quantizations--functional mappings of X that are learnable via gradient descent--to approximate \mathcal{L}(Y \mid X). This approach adapts Competitive Learning Vector Quantization (CLVQ), tailored for conditional distributions. It goes beyond single-valued predictions by providing multiple representative points that better reflect multimodal structures. It enables the approximation of the true conditional law in the Wasserstein distance. The resulting framework is theoretically grounded and useful for uncertainty quantification and multimodal data generation tasks. For example, in computer vision inpainting tasks, multiple plausible reconstructions may exist for the same partially observed input image X. We demonstrate the effectiveness of our approach through experiments on synthetic and real-world datasets.
• 摘要：条件期望 \mathbb{E}(​​Y \mid X) 通常无法捕捉多模态条件分布 \mathcal{L}(Y \mid X) 的复杂性。为了解决这个问题，我们建议使用 n 点条件量化（可通过梯度下降学习的 X 的功能映射）来近似 \mathcal{L}(Y \mid X)。此方法采用了针对条件分布量身定制的竞争学习矢量量化 (CLVQ)。它超越了单值预测，提供了多个可以更好地反映多模态结构的代表点。它能够在 Wasserstein 距离中近似真实条件定律。由此产生的框架具有理论基础，可用于不确定性量化和多模态数据生成任务。例如，在计算机视觉修复任务中，对于同一张部分观察到的输入图像 X，可能存在多个合理的重建。我们通过对合成和真实世界数据集的实验证明了我们方法的有效性。

Title: Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning

• Authors: Feng Chen, Allan Raventos, Nan Cheng, Surya Ganguli, Shaul Druckmann
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.07154
• Pdf URL: https://arxiv.org/pdf/2502.07154
• Copy Paste: [[2502.07154]] Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning(https://arxiv.org/abs/2502.07154)
• Keywords: generation
• Abstract: Recent progress in large language models (LLMs) highlights the power of scaling test-time compute to achieve strong performance on complex tasks, such as mathematical reasoning and code generation. This raises a critical question: how should model training be modified to optimize performance under a subsequent test-time compute strategy and budget? To explore this, we focus on pass@N, a simple test-time strategy that searches for a correct answer in $N$ independent samples. We show, surprisingly, that training with cross-entropy (CE) loss can be ${\it misaligned}$ with pass@N in that pass@N accuracy ${\it decreases}$ with longer training. We explain the origins of this misalignment in terms of model overconfidence induced by CE, and experimentally verify our prediction of overconfidence as an impediment to scaling test-time compute via pass@N. Furthermore we suggest a principled, modified training loss that is better aligned to pass@N by limiting model confidence and rescuing pass@N test performance. Our algorithm demonstrates improved mathematical reasoning on MATH and MiniF2F benchmarks under several scenarios: (1) providing answers to math questions; and (2) proving theorems by searching over proof trees of varying shapes. Overall our work underscores the importance of co-designing two traditionally separate phases of LLM development: training-time protocols and test-time search and reasoning strategies.
• 摘要：大型语言模型 (LLM) 的最新进展凸显了扩展测试时间计算以在复杂任务（例如数学推理和代码生成）上实现强大性能的强大功能。这提出了一个关键问题：应如何修改模型训练以在后续的测试时间计算策略和预算下优化性能？为了探索这个问题，我们专注于 pass@N，这是一种简单的测试时间策略，可在 $N$ 个独立样本中搜索正确答案。令人惊讶的是，我们表明，使用交叉熵 (CE) 损失的训练可能与 pass@N 不一致，因为 pass@N 准确率会随着训练时间的延长而降低。我们从 CE 引起的模型过度自信的角度解释了这种不一致的起源，并通过实验验证了我们对过度自信作为通过 pass@N 扩展测试时间计算的障碍的预测。此外，我们提出了一种原则性的、经过修改的训练损失，通过限制模型置信度和挽救 pass@N 测试性能，使其更好地与 pass@N 保持一致。我们的算法在以下几种情况下展示了 MATH 和 MiniF2F 基准测试中数学推理能力的提升：(1) 提供数学问题的答案；(2) 通过搜索不同形状的证明树来证明定理。总的来说，我们的工作强调了共同设计 LLM 开发两个传统上独立的阶段的重要性：训练时协议和测试时搜索和推理策略。

Title: Explaining 3D Computed Tomography Classifiers with Counterfactuals

• Authors: Joseph Paul Cohen, Louis Blankemeier, Akshay Chaudhari
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.07156
• Pdf URL: https://arxiv.org/pdf/2502.07156
• Copy Paste: [[2502.07156]] Explaining 3D Computed Tomography Classifiers with Counterfactuals(https://arxiv.org/abs/2502.07156)
• Keywords: generation
• Abstract: Counterfactual explanations in medical imaging are critical for understanding the predictions made by deep learning models. We extend the Latent Shift counterfactual generation method from 2D applications to 3D computed tomography (CT) scans. We address the challenges associated with 3D data, such as limited training samples and high memory demands, by implementing a slice-based approach. This method leverages a 2D encoder trained on CT slices, which are subsequently combined to maintain 3D context. We demonstrate this technique on two models for clinical phenotype prediction and lung segmentation. Our approach is both memory-efficient and effective for generating interpretable counterfactuals in high-resolution 3D medical imaging.
• 摘要：医学成像中的反事实解释对于理解深度学习模型的预测至关重要。我们将 Latent Shift 反事实生成方法从 2D 应用扩展到 3D 计算机断层扫描 (CT)。我们通过实施基于切片的方法解决了与 3D 数据相关的挑战，例如有限的训练样本和高内存需求。该方法利用在 CT 切片上训练的 2D 编码器，随后将其组合以保持 3D 上下文。我们在两个用于临床表型预测和肺分割的模型上展示了这种技术。我们的方法既节省内存，又能有效地在高分辨率 3D 医学成像中生成可解释的反事实。

Title: HDCompression: Hybrid-Diffusion Image Compression for Ultra-Low Bitrates

• Authors: Lei Lu, Yize Li, Yanzhi Wang, Wei Wang, Wei Jiang
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2502.07160
• Pdf URL: https://arxiv.org/pdf/2502.07160
• Copy Paste: [[2502.07160]] HDCompression: Hybrid-Diffusion Image Compression for Ultra-Low Bitrates(https://arxiv.org/abs/2502.07160)
• Keywords: generative
• Abstract: Image compression under ultra-low bitrates remains challenging for both conventional learned image compression (LIC) and generative vector-quantized (VQ) modeling. Conventional LIC suffers from severe artifacts due to heavy quantization, while generative VQ modeling gives poor fidelity due to the mismatch between learned generative priors and specific inputs. In this work, we propose Hybrid-Diffusion Image Compression (HDCompression), a dual-stream framework that utilizes both generative VQ-modeling and diffusion models, as well as conventional LIC, to achieve both high fidelity and high perceptual quality. Different from previous hybrid methods that directly use pre-trained LIC models to generate low-quality fidelity-preserving information from heavily quantized latent, we use diffusion models to extract high-quality complimentary fidelity information from the ground-truth input, which can enhance the system performance in several aspects: improving indices map prediction, enhancing the fidelity-preserving output of the LIC stream, and refining conditioned image reconstruction with VQ-latent correction. In addition, our diffusion model is based on a dense representative vector (DRV), which is lightweight with very simple sampling schedulers. Extensive experiments demonstrate that our HDCompression outperforms the previous conventional LIC, generative VQ-modeling, and hybrid frameworks in both quantitative metrics and qualitative visualization, providing balanced robust compression performance at ultra-low bitrates.
• 摘要：超低比特率下的图像压缩对于传统的学习图像压缩 (LIC) 和生成矢量量化 (VQ) 建模来说仍然具有挑战性。传统的 LIC 由于重度量化而出现严重的伪影，而生成 VQ 建模由于学习到的生成先验和特定输入不匹配而导致保真度较差。在这项工作中，我们提出了混合扩散图像压缩 (HDCompression)，这是一个双流框架，它利用生成 VQ 建模和扩散模型以及传统的 LIC 来实现高保真度和高感知质量。与以前的混合方法直接使用预训练的 LIC 模型从重度量化的潜在中生成低质量的保真度保持信息不同，我们使用扩散模型从地面实况输入中提取高质量的互补保真度信息，这可以在几个方面增强系统性能：改进索引图预测，增强 LIC 流的保真度保持输出，以及使用 VQ 潜在校正细化条件图像重建。此外，我们的扩散模型基于密集代表向量 (DRV)，该模型非常轻量，采样调度程序非常简单。大量实验表明，我们的 HDCompression 在定量指标和定性可视化方面均优于之前的传统 LIC、生成式 VQ 建模和混合框架，在超低比特率下提供均衡的稳健压缩性能。

Title: Playmate: Flexible Control of Portrait Animation via 3D-Implicit Space Guided Diffusion

• Authors: Xingpei Ma, Jiaran Cai, Yuansheng Guan, Shenneng Huang, Qiang Zhang, Shunsi Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07203
• Pdf URL: https://arxiv.org/pdf/2502.07203
• Copy Paste: [[2502.07203]] Playmate: Flexible Control of Portrait Animation via 3D-Implicit Space Guided Diffusion(https://arxiv.org/abs/2502.07203)
• Keywords: generation
• Abstract: Recent diffusion-based talking face generation models have demonstrated impressive potential in synthesizing videos that accurately match a speech audio clip with a given reference identity. However, existing approaches still encounter significant challenges due to uncontrollable factors, such as inaccurate lip-sync, inappropriate head posture and the lack of fine-grained control over facial expressions. In order to introduce more face-guided conditions beyond speech audio clips, a novel two-stage training framework Playmate is proposed to generate more lifelike facial expressions and talking faces. In the first stage, we introduce a decoupled implicit 3D representation along with a meticulously designed motion-decoupled module to facilitate more accurate attribute disentanglement and generate expressive talking videos directly from audio cues. Then, in the second stage, we introduce an emotion-control module to encode emotion control information into the latent space, enabling fine-grained control over emotions and thereby achieving the ability to generate talking videos with desired emotion. Extensive experiments demonstrate that Playmate outperforms existing state-of-the-art methods in terms of video quality and lip-synchronization, and improves flexibility in controlling emotion and head pose. The code will be available at this https URL.
• 摘要：最近基于扩散的说话人脸生成模型在合成视频方面表现出了令人印象深刻的潜力，这些视频可以将语音音频片段与给定的参考身份精确匹配。然而，现有的方法仍然面临着重大挑战，原因是不可控的因素，例如不准确的口型同步、不适当的头部姿势以及缺乏对面部表情的细粒度控制。为了在语音音频片段之外引入更多的面部引导条件，提出了一种新颖的两阶段训练框架 Playmate，以生成更逼真的面部表情和说话人脸。在第一阶段，我们引入了一个解耦的隐式 3D 表示以及精心设计的运动解耦模块，以促进更准确的属性解缠，并直接从音频提示生成富有表现力的说话视频。然后，在第二阶段，我们引入一个情绪控制模块，将情绪控制信息编码到潜在空间中，实现对情绪的细粒度控制，从而实现生成具有所需情绪的说话视频的能力。大量实验表明，Playmate 在视频质量和口型同步方面的表现优于现有的最先进方法，并且提高了控制情绪和头部姿势的灵活性。代码将在此 https URL 上提供。

Title: Improve the Training Efficiency of DRL for Wireless Communication Resource Allocation: The Role of Generative Diffusion Models

• Authors: Xinren Zhang, Jiadong Yu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07211
• Pdf URL: https://arxiv.org/pdf/2502.07211
• Copy Paste: [[2502.07211]] Improve the Training Efficiency of DRL for Wireless Communication Resource Allocation: The Role of Generative Diffusion Models(https://arxiv.org/abs/2502.07211)
• Keywords: generation, generative
• Abstract: Dynamic resource allocation in mobile wireless networks involves complex, time-varying optimization problems, motivating the adoption of deep reinforcement learning (DRL). However, most existing works rely on pre-trained policies, overlooking dynamic environmental changes that rapidly invalidate the policies. Periodic retraining becomes inevitable but incurs prohibitive computational costs and energy consumption-critical concerns for resource-constrained wireless systems. We identify three root causes of inefficient retraining: high-dimensional state spaces, suboptimal action spaces exploration-exploitation trade-offs, and reward design limitations. To overcome these limitations, we propose Diffusion-based Deep Reinforcement Learning (D2RL), which leverages generative diffusion models (GDMs) to holistically enhance all three DRL components. Iterative refinement process and distribution modelling of GDMs enable (1) the generation of diverse state samples to improve environmental understanding, (2) balanced action space exploration to escape local optima, and (3) the design of discriminative reward functions that better evaluate action quality. Our framework operates in two modes: Mode I leverages GDMs to explore reward spaces and design discriminative reward functions that rigorously evaluate action quality, while Mode II synthesizes diverse state samples to enhance environmental understanding and generalization. Extensive experiments demonstrate that D2RL achieves faster convergence and reduced computational costs over conventional DRL methods for resource allocation in wireless communications while maintaining competitive policy performance. This work underscores the transformative potential of GDMs in overcoming fundamental DRL training bottlenecks for wireless networks, paving the way for practical, real-time deployments.
• 摘要：移动无线网络中的动态资源分配涉及复杂的、随时间变化的优化问题，这促使人们采用深度强化学习 (DRL)。然而，大多数现有工作都依赖于预先训练的策略，忽略了迅速使策略失效的动态环境变化。定期重新训练是不可避免的，但会产生高昂的计算成本和能源消耗——这对于资源受限的无线系统来说是至关重要的问题。我们确定了再训练效率低下的三个根本原因：高维状态空间、次优动作空间探索-利用权衡和奖励设计限制。为了克服这些限制，我们提出了基于扩散的深度强化学习 (D2RL)，它利用生成扩散模型 (GDM) 来整体增强所有三个 DRL 组件。GDM 的迭代细化过程和分布建模可以 (1) 生成不同的状态样本以改善对环境的理解，(2) 平衡动作空间探索以摆脱局部最优，以及 (3) 设计判别奖励函数以更好地评估动作质量。我们的框架以两种模式运行：模式 I 利用 GDM 探索奖励空间并设计严格评估行动质量的判别奖励函数，而模式 II 则综合不同的状态样本以增强对环境的理解和泛化。大量实验表明，与传统的 DRL 方法相比，D2RL 在无线通信资源分配方面实现了更快的收敛速度和更低的计算成本，同时保持了具有竞争力的策略性能。这项工作强调了 GDM 在克服无线网络基本 DRL 训练瓶颈方面的变革潜力，为实际的实时部署铺平了道路。

Title: DrugImproverGPT: A Large Language Model for Drug Optimization with Fine-Tuning via Structured Policy Optimization

• Authors: Xuefeng Liu, Songhao Jiang, Siyu Chen, Zhuoran Yang, Yuxin Chen, Ian Foster, Rick Stevens
• Subjects: cs.LG, cs.CL, q-bio.BM, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.07237
• Pdf URL: https://arxiv.org/pdf/2502.07237
• Copy Paste: [[2502.07237]] DrugImproverGPT: A Large Language Model for Drug Optimization with Fine-Tuning via Structured Policy Optimization(https://arxiv.org/abs/2502.07237)
• Keywords: generative
• Abstract: Finetuning a Large Language Model (LLM) is crucial for generating results towards specific objectives. This research delves into the realm of drug optimization and introduce a novel reinforcement learning algorithm to finetune a drug optimization LLM-based generative model, enhancing the original drug across target objectives, while retains the beneficial chemical properties of the original drug. This work is comprised of two primary components: (1) DrugImprover: A framework tailored for improving robustness and efficiency in drug optimization. It includes a LLM designed for drug optimization and a novel Structured Policy Optimization (SPO) algorithm, which is theoretically grounded. This algorithm offers a unique perspective for fine-tuning the LLM-based generative model by aligning the improvement of the generated molecule with the input molecule under desired objectives. (2) A dataset of 1 million compounds, each with OEDOCK docking scores on 5 human proteins associated with cancer cells and 24 binding sites from SARS-CoV-2 virus. We conduct a comprehensive evaluation of SPO and demonstrate its effectiveness in improving the original drug across target properties. Our code and dataset will be publicly available at: this https URL.
• 摘要：微调大型语言模型 (LLM) 对于生成针对特定目标的结果至关重要。本研究深入研究了药物优化领域，并引入了一种新颖的强化学习算法来微调基于药物优化 LLM 的生成模型，从而增强原始药物在目标目标上的性能，同时保留原始药物的有益化学特性。这项工作由两个主要部分组成：(1) DrugImprover：为提高药物优化的稳健性和效率而量身定制的框架。它包括一个专为药物优化而设计的 LLM 和一个新颖的结构化策略优化 (SPO) 算法，该算法具有理论基础。该算法通过将生成的分子的改进与期望目标下的输入分子对齐，为微调基于 LLM 的生成模型提供了独特的视角。(2) 一个包含 100 万种化合物的数据集，每种化合物在与癌细胞相关的 5 种人类蛋白质和来自 SARS-CoV-2 病毒的 24 个结合位点上都有 OEDOCK 对接分数。我们对 SPO 进行了全面评估，并证明了其在改善原始药物的目标属性方面的有效性。我们的代码和数据集将在以下网址公开提供：此 https URL。

Title: Diffusion Suction Grasping with Large-Scale Parcel Dataset

• Authors: Ding-Tao Huang, Xinyi He, Debei Hua, Dongfang Yu, En-Te Lin, Long Zeng
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.07238
• Pdf URL: https://arxiv.org/pdf/2502.07238
• Copy Paste: [[2502.07238]] Diffusion Suction Grasping with Large-Scale Parcel Dataset(https://arxiv.org/abs/2502.07238)
• Keywords: generation
• Abstract: While recent advances in object suction grasping have shown remarkable progress, significant challenges persist particularly in cluttered and complex parcel handling scenarios. Two fundamental limitations hinder current approaches: (1) the lack of a comprehensive suction grasp dataset tailored for parcel manipulation tasks, and (2) insufficient adaptability to diverse object characteristics including size variations, geometric complexity, and textural diversity. To address these challenges, we present Parcel-Suction-Dataset, a large-scale synthetic dataset containing 25 thousand cluttered scenes with 410 million precision-annotated suction grasp poses. This dataset is generated through our novel geometric sampling algorithm that enables efficient generation of optimal suction grasps incorporating both physical constraints and material properties. We further propose Diffusion-Suction, an innovative framework that reformulates suction grasp prediction as a conditional generation task through denoising diffusion probabilistic models. Our method iteratively refines random noise into suction grasp score maps through visual-conditioned guidance from point cloud observations, effectively learning spatial point-wise affordances from our synthetic dataset. Extensive experiments demonstrate that the simple yet efficient Diffusion-Suction achieves new state-of-the-art performance compared to previous models on both Parcel-Suction-Dataset and the public SuctionNet-1Billion benchmark.
• 摘要：虽然物体吸力抓取方面的最新进展已显示出显著的进步，但重大挑战仍然存在，特别是在杂乱和复杂的包裹处理场景中。两个基本限制阻碍了当前的方法：（1）缺乏针对包裹处理任务量身定制的全面吸力抓取数据集，以及（2）对各种物体特征（包括尺寸变化、几何复杂性和纹理多样性）的适应性不足。为了应对这些挑战，我们提出了 Parcel-Suction-Dataset，这是一个大规模合成数据集，包含 25,000 个杂乱场景和 4.1 亿个精确注释的吸力抓取姿势。该数据集是通过我们新颖的几何采样算法生成的，该算法能够高效生成结合物理约束和材料特性的最佳吸力抓取。我们进一步提出了 Diffusion-Suction，这是一个创新框架，它通过去噪扩散概率模型将吸力抓取预测重新表述为条件生成任务。我们的方法通过点云观测的视觉条件指导，将随机噪声迭代细化为吸力抓取分数图，从而有效地从我们的合成数据集中学习空间逐点可供性。大量实验表明，简单而有效的 Diffusion-Suction 与之前的模型相比，在 Parcel-Suction-Dataset 和公共 SuctionNet-1Billion 基准上都实现了新的最先进的性能。

Title: Contextual Gesture: Co-Speech Gesture Video Generation through Context-aware Gesture Representation

• Authors: Pinxin Liu, Pengfei Zhang, Hyeongwoo Kim, Pablo Garrido, Ari Sharpio, Kyle Olszewski
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.07239
• Pdf URL: https://arxiv.org/pdf/2502.07239
• Copy Paste: [[2502.07239]] Contextual Gesture: Co-Speech Gesture Video Generation through Context-aware Gesture Representation(https://arxiv.org/abs/2502.07239)
• Keywords: generation
• Abstract: Co-speech gesture generation is crucial for creating lifelike avatars and enhancing human-computer interactions by synchronizing gestures with speech. Despite recent advancements, existing methods struggle with accurately identifying the rhythmic or semantic triggers from audio for generating contextualized gesture patterns and achieving pixel-level realism. To address these challenges, we introduce Contextual Gesture, a framework that improves co-speech gesture video generation through three innovative components: (1) a chronological speech-gesture alignment that temporally connects two modalities, (2) a contextualized gesture tokenization that incorporate speech context into motion pattern representation through distillation, and (3) a structure-aware refinement module that employs edge connection to link gesture keypoints to improve video generation. Our extensive experiments demonstrate that Contextual Gesture not only produces realistic and speech-aligned gesture videos but also supports long-sequence generation and video gesture editing applications, shown in Fig.1 Project Page: this https URL.
• 摘要：通过将手势与语音同步，同步语音手势生成对于创建逼真的虚拟形象和增强人机交互至关重要。尽管现有方法最近取得了进展，但它们仍难以从音频中准确识别节奏或语义触发器，从而生成情境化手势模式并实现像素级真实感。为了应对这些挑战，我们引入了 Contextual Gesture，这是一个通过三个创新组件改进同步语音手势视频生成的框架：(1) 按时间顺序连接两种模态的语音-手势对齐，(2) 通过提炼将语音上下文纳入运动模式表示的情境化手势标记，以及 (3) 结构感知细化模块，它使用边连接来链接手势关键点以改进视频生成。我们大量的实验表明，Contextual Gesture 不仅可以制作逼真且与语音对齐的手势视频，而且还支持长序列生成和视频手势编辑应用程序，如图 1 所示。项目页面：此 https URL。

Title: Linear Transformers as VAR Models: Aligning Autoregressive Attention Mechanisms with Autoregressive Forecasting

• Authors: Jiecheng Lu, Shihao Yang
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.07244
• Pdf URL: https://arxiv.org/pdf/2502.07244
• Copy Paste: [[2502.07244]] Linear Transformers as VAR Models: Aligning Autoregressive Attention Mechanisms with Autoregressive Forecasting(https://arxiv.org/abs/2502.07244)
• Keywords: generative
• Abstract: Autoregressive attention-based time series forecasting (TSF) has drawn increasing interest, with mechanisms like linear attention sometimes outperforming vanilla attention. However, deeper Transformer architectures frequently misalign with autoregressive objectives, obscuring the underlying VAR structure embedded within linear attention and hindering their ability to capture the data generative processes in TSF. In this work, we first show that a single linear attention layer can be interpreted as a dynamic vector autoregressive (VAR) structure. We then explain that existing multi-layer Transformers have structural mismatches with the autoregressive forecasting objective, which impair interpretability and generalization ability. To address this, we show that by rearranging the MLP, attention, and input-output flow, multi-layer linear attention can also be aligned as a VAR model. Then, we propose Structural Aligned Mixture of VAR (SAMoVAR), a linear Transformer variant that integrates interpretable dynamic VAR weights for multivariate TSF. By aligning the Transformer architecture with autoregressive objectives, SAMoVAR delivers improved performance, interpretability, and computational efficiency, comparing to SOTA TSF models.
• 摘要：基于自回归注意机制的时间序列预测 (TSF) 引起了越来越多的关注，线性注意等机制有时甚至优于普通注意机制。然而，更深层的 Transformer 架构经常与自回归目标不一致，从而掩盖了线性注意机制中嵌入的底层 VAR 结构，并阻碍了它们捕获 TSF 中数据生成过程的能力。在这项工作中，我们首先表明单个线性注意层可以解释为动态向量自回归 (VAR) 结构。然后，我们解释了现有的多层 Transformer 与自回归预测目标存在结构不匹配，从而损害了可解释性和泛化能力。为了解决这个问题，我们表明，通过重新排列 MLP、注意机制和输入输出流，多层线性注意机制也可以作为 VAR 模型对齐。然后，我们提出了结构对齐的 VAR 混合 (SAMoVAR)，这是一种线性 Transformer 变体，它集成了可解释的动态 VAR 权重用于多变量 TSF。通过将 Transformer 架构与自回归目标相结合，与 SOTA TSF 模型相比，SAMoVAR 可提供更高的性能、可解释性和计算效率。

Title: Articulate That Object Part (ATOP): 3D Part Articulation from Text and Motion Personalization

• Authors: Aditya Vora, Sauradip Nag, Hao Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07278
• Pdf URL: https://arxiv.org/pdf/2502.07278
• Copy Paste: [[2502.07278]] Articulate That Object Part (ATOP): 3D Part Articulation from Text and Motion Personalization(https://arxiv.org/abs/2502.07278)
• Keywords: generation
• Abstract: We present ATOP (Articulate That Object Part), a novel method based on motion personalization to articulate a 3D object with respect to a part and its motion as prescribed in a text prompt. Specifically, the text input allows us to tap into the power of modern-day video diffusion to generate plausible motion samples for the right object category and part. In turn, the input 3D object provides image prompting to personalize the generated video to that very object we wish to articulate. Our method starts with a few-shot finetuning for category-specific motion generation, a key first step to compensate for the lack of articulation awareness by current video diffusion models. For this, we finetune a pre-trained multi-view image generation model for controllable multi-view video generation, using a small collection of video samples obtained for the target object category. This is followed by motion video personalization that is realized by multi-view rendered images of the target 3D object. At last, we transfer the personalized video motion to the target 3D object via differentiable rendering to optimize part motion parameters by a score distillation sampling loss. We show that our method is capable of generating realistic motion videos and predict 3D motion parameters in a more accurate and generalizable way, compared to prior works.
• 摘要：我们提出了 ATOP（Articulate That Object Part，表达物体部分），这是一种基于运动个性化的新方法，用于根据文本提示规定的部分及其运动表达 3D 物体。具体来说，文本输入使我们能够利用现代视频扩散的强大功能，为正确的物体类别和部分生成合理的运动样本。反过来，输入的 3D 物体提供图像提示，将生成的视频个性化为我们希望表达的物体。我们的方法从针对特定类别的运动生成的几次微调开始，这是弥补当前视频扩散模型缺乏表达意识的关键第一步。为此，我们使用为目标物体类别获得的一小组视频样本，对预先训练的多视图图像生成模型进行微调，以实现可控的多视图视频生成。接下来是通过目标 3D 物体的多视图渲染图像实现的运动视频个性化。最后，我们通过可微分渲染将个性化视频运动转移到目标 3D 对象，通过分数蒸馏采样损失优化部分运动参数。与之前的研究相比，我们的方法能够生成逼真的运动视频，并以更准确、更通用的方式预测 3D 运动参数。

Title: Generation of Drug-Induced Cardiac Reactions towards Virtual Clinical Trials

• Authors: Qian Shao, Bang Du, Zepeng Li, Qiyuan Chen, Hongxia Xu, Jimeng Sun, Jian Wu, Jintai Chen
• Subjects: cs.LG, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2502.07297
• Pdf URL: https://arxiv.org/pdf/2502.07297
• Copy Paste: [[2502.07297]] Generation of Drug-Induced Cardiac Reactions towards Virtual Clinical Trials(https://arxiv.org/abs/2502.07297)
• Keywords: generation, generative
• Abstract: Clinical trials are pivotal in cardiac drug development, yet they often fail due to inadequate efficacy and unexpected safety issues, leading to significant financial losses. Using in-silico trials to replace a part of physical clinical trials, e.g., leveraging advanced generative models to generate drug-influenced electrocardiograms (ECGs), seems an effective method to reduce financial risk and potential harm to trial participants. While existing generative models have demonstrated progress in ECG generation, they fall short in modeling drug reactions due to limited fidelity and inability to capture individualized drug response patterns. In this paper, we propose a Drug-Aware Diffusion Model (DADM), which could simulate individualized drug reactions while ensuring fidelity. To ensure fidelity, we construct a set of ordinary differential equations to provide external physical knowledge (EPK) of the realistic ECG morphology. The EPK is used to adaptively constrain the morphology of the generated ECGs through a dynamic cross-attention (DCA) mechanism. Furthermore, we propose an extension of ControlNet to incorporate demographic and drug data, simulating individual drug reactions. We compare DADM with the other eight state-of-the-art ECG generative models on two real-world databases covering 8 types of drug regimens. The results demonstrate that DADM can more accurately simulate drug-induced changes in ECGs, improving the accuracy by at least 5.79% and recall by 8%.
• 摘要：临床试验是心脏药物研发的关键，但它们常常由于疗效不足和意外的安全问题而失败，导致重大的经济损失。使用计算机模拟试验取代部分实体临床试验，例如利用先进的生成模型生成受药物影响的心电图 (ECG)，似乎是一种降低财务风险和对试验参与者的潜在伤害的有效方法。虽然现有的生成模型在 ECG 生成方面取得了进展，但它们在模拟药物反应方面存在不足，因为保真度有限且无法捕捉个性化的药物反应模式。在本文中，我们提出了一种药物感知扩散模型 (DADM)，它可以模拟个性化的药物反应，同时确保保真度。为了确保保真度，我们构建了一组常微分方程来提供真实 ECG 形态的外部物理知识 (EPK)。EPK 用于通过动态交叉注意 (DCA) 机制自适应地约束生成的 ECG 的形态。此外，我们提出了 ControlNet 的扩展，以整合人口统计学和药物数据，模拟个体药物反应。我们在两个涵盖 8 种药物方案的真实数据库上将 DADM 与其他八个最先进的 ECG 生成模型进行了比较。结果表明，DADM 可以更准确地模拟药物引起的 ECG 变化，准确率至少提高了 5.79%，召回率提高了 8%。

Title: Integrating Physics and Data-Driven Approaches: An Explainable and Uncertainty-Aware Hybrid Model for Wind Turbine Power Prediction

• Authors: Alfonso Gijón, Simone Eiraudo, Antonio Manjavacas, Daniele Salvatore Schiera, Miguel Molina-Solana, Juan Gómez-Romero
• Subjects: cs.LG, cs.AI, cs.CE
• Abstract URL: https://arxiv.org/abs/2502.07344
• Pdf URL: https://arxiv.org/pdf/2502.07344
• Copy Paste: [[2502.07344]] Integrating Physics and Data-Driven Approaches: An Explainable and Uncertainty-Aware Hybrid Model for Wind Turbine Power Prediction(https://arxiv.org/abs/2502.07344)
• Keywords: generation
• Abstract: The rapid growth of the wind energy sector underscores the urgent need to optimize turbine operations and ensure effective maintenance through early fault detection systems. While traditional empirical and physics-based models offer approximate predictions of power generation based on wind speed, they often fail to capture the complex, non-linear relationships between other input variables and the resulting power output. Data-driven machine learning methods present a promising avenue for improving wind turbine modeling by leveraging large datasets, enhancing prediction accuracy but often at the cost of interpretability. In this study, we propose a hybrid semi-parametric model that combines the strengths of both approaches, applied to a dataset from a wind farm with four turbines. The model integrates a physics-inspired submodel, providing a reasonable approximation of power generation, with a non-parametric submodel that predicts the residuals. This non-parametric submodel is trained on a broader range of variables to account for phenomena not captured by the physics-based component. The hybrid model achieves a 37% improvement in prediction accuracy over the physics-based model. To enhance interpretability, SHAP values are used to analyze the influence of input features on the residual submodel's output. Additionally, prediction uncertainties are quantified using a conformalized quantile regression method. The combination of these techniques, alongside the physics grounding of the parametric submodel, provides a flexible, accurate, and reliable framework. Ultimately, this study opens the door for evaluating the impact of unmodeled variables on wind turbine power generation, offering a basis for potential optimization.
• 摘要：风能行业的快速增长凸显了优化涡轮机运行并通过早期故障检测系统确保有效维护的迫切需求。虽然传统的经验和基于物理的模型可以根据风速提供发电量的近似预测，但它们往往无法捕捉到其他输入变量与由此产生的电力输出之间复杂的非线性关系。数据驱动的机器学习方法通​​过利用大型数据集为改进风力涡轮机建模提供了一种有希望的途径，提高了预测准确性，但往往以牺牲可解释性为代价。在本研究中，我们提出了一种混合半参数模型，该模型结合了两种方法的优势，应用于拥有四台涡轮机的风电场的数据集。该模型集成了一个物理启发式子模型，提供发电量的合理近似值，以及一个预测残差的非参数子模型。这个非参数子模型在更广泛的变量上进行训练，以解释基于物理的组件未捕获的现象。混合模型的预测准确度比基于物理的模型提高了 37%。为了提高可解释性，SHAP 值用于分析输入特征对残差子模型输出的影响。此外，使用共形分位数回归方法量化预测不确定性。这些技术的结合，加上参数子模型的物理基础，提供了一个灵活、准确和可靠的框架。最终，这项研究为评估未建模变量对风力涡轮机发电的影响打开了大门，为潜在的优化提供了基础。

Title: USRNet: Unified Scene Recovery Network for Enhancing Traffic Imaging under Multiple Adverse Weather Conditions

• Authors: Yuxu Lu, Ai Chen, Dong Yang, Ryan Wen Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07372
• Pdf URL: https://arxiv.org/pdf/2502.07372
• Copy Paste: [[2502.07372]] USRNet: Unified Scene Recovery Network for Enhancing Traffic Imaging under Multiple Adverse Weather Conditions(https://arxiv.org/abs/2502.07372)
• Keywords: restoration
• Abstract: Advancements in computer vision technology have facilitated the extensive deployment of intelligent transportation systems and visual surveillance systems across various applications, including autonomous driving, public safety, and environmental monitoring. However, adverse weather conditions such as haze, rain, snow, and more complex mixed degradation can significantly degrade image quality. The degradation compromises the accuracy and reliability of these systems across various scenarios. To tackle the challenge of developing adaptable models for scene restoration, we introduce the unified scene recovery network (USRNet), capable of handling multiple types of image degradation. The USRNet features a sophisticated architecture consisting of a scene encoder, an attention-driven node independent learning mechanism (NILM), an edge decoder, and a scene restoration module. The scene encoder, powered by advanced residual blocks, extracts deep features from degraded images in a progressive manner, ensuring thorough encoding of degradation information. To enhance the USRNet's adaptability in diverse weather conditions, we introduce NILM, which enables the network to learn and respond to different scenarios with precision, thereby increasing its robustness. The edge decoder is designed to extract edge features with precision, which is essential for maintaining image sharpness. Experimental results demonstrate that USRNet surpasses existing methods in handling complex imaging degradations, thereby improving the accuracy and reliability of visual systems across diverse scenarios. The code resources for this work can be accessed in this https URL.
• 摘要：计算机视觉技术的进步促进了智能交通系统和视觉监控系统在各种应用中的广泛部署，包括自动驾驶、公共安全和环境监测。然而，恶劣的天气条件，如雾霾、雨、雪和更复杂的混合退化，会显著降低图像质量。退化会损害这些系统在各种情况下的准确性和可靠性。为了应对开发适应性场景恢复模型的挑战，我们引入了统一场景恢复网络 (USRNet)，它能够处理多种类型的图像退化。USRNet 具有复杂的架构，包括场景编码器、注意力驱动的节点独立学习机制 (NILM)、边缘解码器和场景恢复模块。场景编码器由先进的残差块驱动，以渐进的方式从退化图像中提取深度特征，确保对退化信息进行彻底编码。为了增强 USRNet 在不同天气条件下的适应性，我们引入了 NILM，它使网络能够精确地学习和响应不同的场景，从而提高其鲁棒性。边缘解码器旨在精确提取边缘特征，这对于保持图像清晰度至关重要。实验结果表明，USRNet 在处理复杂图像退化方面超越了现有方法，从而提高了视觉系统在不同场景中的准确性和可靠性。这项工作的代码资源可以通过此 https URL 访问。

Title: Spatial Degradation-Aware and Temporal Consistent Diffusion Model for Compressed Video Super-Resolution

• Authors: Hongyu An, Xinfeng Zhang, Shijie Zhao, Li Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07381
• Pdf URL: https://arxiv.org/pdf/2502.07381
• Copy Paste: [[2502.07381]] Spatial Degradation-Aware and Temporal Consistent Diffusion Model for Compressed Video Super-Resolution(https://arxiv.org/abs/2502.07381)
• Keywords: super-resolution, generation
• Abstract: Due to limitations of storage and bandwidth, videos stored and transmitted on the Internet are usually low-quality with low-resolution and compression noise. Although video super-resolution (VSR) is an efficient technique to enhance video resolution, relatively VSR methods focus on compressed videos. Directly applying general VSR approaches leads to the failure of improving practical videos, especially when frames are highly compressed at a low bit rate. Recently, diffusion models have achieved superior performance in low-level visual tasks, and their high-realism generation capability enables them to be applied in VSR. To synthesize more compression-lost details and refine temporal consistency, we propose a novel Spatial Degradation-Aware and Temporal Consistent (SDATC) diffusion model for compressed VSR. Specifically, we introduce a distortion Control module (DCM) to modulate diffusion model inputs and guide the generation. Next, the diffusion model executes the denoising process for texture generation with fine-tuned spatial prompt-based compression-aware module (PCAM) and spatio-temporal attention module (STAM). PCAM extracts features to encode specific compression information dynamically. STAM extends the spatial attention mechanism to a spatio-temporal dimension for capturing temporal correlation. Extensive experimental results on benchmark datasets demonstrate the effectiveness of the proposed modules in enhancing compressed videos.
• 摘要：由于存储和带宽的限制，在互联网上存储和传输的视频通常质量较低、分辨率低且有压缩噪声。虽然视频超分辨率 (VSR) 是一种提高视频分辨率的有效技术，但相对而言，VSR 方法侧重于压缩视频。直接应用一般的 VSR 方法无法改善实际视频，尤其是当帧以低比特率高度压缩时。最近，扩散模型在低级视觉任务中取得了优异的表现，其高真实感生成能力使它们能够应用于 VSR。为了合成更多压缩丢失的细节并改进时间一致性，我们提出了一种用于压缩 VSR 的新型空间退化感知和时间一致性 (SDATC) 扩散模型。具体而言，我们引入了一个失真控制模块 (DCM) 来调节扩散模型输入并指导生成。接下来，扩散模型使用微调的空间提示压缩感知模块 (PCAM) 和时空注意模块 (STAM) 执行纹理生成的去噪过程。PCAM 提取特征以动态编码特定的压缩信息。 STAM 将空间注意机制扩展到时空维度以捕捉时间相关性。基准数据集上的大量实验结果证明了所提出的模块在增强压缩视频方面的有效性。

Title: Optimizing Knowledge Distillation in Transformers: Enabling Multi-Head Attention without Alignment Barriers

• Authors: Zhaodong Bing, Linze Li, Jiajun Liang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07436
• Pdf URL: https://arxiv.org/pdf/2502.07436
• Copy Paste: [[2502.07436]] Optimizing Knowledge Distillation in Transformers: Enabling Multi-Head Attention without Alignment Barriers(https://arxiv.org/abs/2502.07436)
• Keywords: generation, generative
• Abstract: Knowledge distillation (KD) in transformers often faces challenges due to misalignment in the number of attention heads between teacher and student models. Existing methods either require identical head counts or introduce projectors to bridge dimensional gaps, limiting flexibility and efficiency. We propose Squeezing-Heads Distillation (SHD), a novel approach that enables seamless knowledge transfer between models with varying head counts by compressing multi-head attention maps via efficient linear approximation. Unlike prior work, SHD eliminates alignment barriers without additional parameters or architectural modifications. Our method dynamically approximates the combined effect of multiple teacher heads into fewer student heads, preserving fine-grained attention patterns while reducing redundancy. Experiments across language (LLaMA, GPT) and vision (DiT, MDT) generative and vision (DeiT) discriminative tasks demonstrate SHD's effectiveness: it outperforms logit-based and feature-alignment KD baselines, achieving state-of-the-art results in image classification, image generation language fine-tuning, and language pre-training. The key innovations of flexible head compression, projector-free design, and linear-time complexity make SHD a versatile and scalable solution for distilling modern transformers. This work bridges a critical gap in KD, enabling efficient deployment of compact models without compromising performance.
• 摘要：Transformer 中的知识蒸馏 (KD) 经常面临挑战，因为教师和学生模型之间的注意力头数量不一致。现有方法要么需要相同的注意力头数量，要么引入投影仪来弥补维度差距，从而限制了灵活性和效率。我们提出了挤压注意力头蒸馏 (SHD)，这是一种新颖的方法，通过有效的线性近似压缩多头注意力图，实现具有不同注意力头数量的模型之间的无缝知识转移。与之前的工作不同，SHD 消除了对齐障碍，无需额外的参数或架构修改。我们的方法将多个教师注意力头的组合效果动态地近似为更少的学生注意力头，在减少冗余的同时保留了细粒度的注意力模式。在语言（LLaMA、GPT）和视觉（DiT、MDT）生成和视觉（DeiT）判别任务中进行的实验证明了 SHD 的有效性：它的表现优于基于逻辑和特征对齐的 KD 基线，在图像分类、图像生成语言微调和语言预训练方面取得了最先进的成果。灵活的头部压缩、无投影仪设计和线性时间复杂度等关键创新使 SHD 成为提取现代 Transformer 的多功能且可扩展的解决方案。这项工作弥补了 KD 中的一个关键空白，可以在不影响性能的情况下高效部署紧凑模型。

Title: RusCode: Russian Cultural Code Benchmark for Text-to-Image Generation

• Authors: Viacheslav Vasilev, Julia Agafonova, Nikolai Gerasimenko, Alexander Kapitanov, Polina Mikhailova, Evelina Mironova, Denis Dimitrov
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.07455
• Pdf URL: https://arxiv.org/pdf/2502.07455
• Copy Paste: [[2502.07455]] RusCode: Russian Cultural Code Benchmark for Text-to-Image Generation(https://arxiv.org/abs/2502.07455)
• Keywords: generation, generative
• Abstract: Text-to-image generation models have gained popularity among users around the world. However, many of these models exhibit a strong bias toward English-speaking cultures, ignoring or misrepresenting the unique characteristics of other language groups, countries, and nationalities. The lack of cultural awareness can reduce the generation quality and lead to undesirable consequences such as unintentional insult, and the spread of prejudice. In contrast to the field of natural language processing, cultural awareness in computer vision has not been explored as extensively. In this paper, we strive to reduce this gap. We propose a RusCode benchmark for evaluating the quality of text-to-image generation containing elements of the Russian cultural code. To do this, we form a list of 19 categories that best represent the features of Russian visual culture. Our final dataset consists of 1250 text prompts in Russian and their translations into English. The prompts cover a wide range of topics, including complex concepts from art, popular culture, folk traditions, famous people's names, natural objects, scientific achievements, etc. We present the results of a human evaluation of the side-by-side comparison of Russian visual concepts representations using popular generative models.
• 摘要：文本到图像生成模型在世界各地的用户中越来越受欢迎。然而，许多此类模型都表现出对英语文化的强烈偏见，忽略或歪曲了其他语言群体、国家和民族的独特特征。缺乏文化意识会降低生成质量并导致不良后果，例如无意的侮辱和偏见的传播。与自然语言处理领域相比，计算机视觉中的文化意识尚未得到广泛探索。在本文中，我们努力缩小这一差距。我们提出了一个 RusCode 基准，用于评估包含俄罗斯文化代码元素的文本到图像生成质量。为此，我们列出了 19 个最能代表俄罗斯视觉文化特征的类别。我们的最终数据集包含 1250 个俄语文本提示及其英文翻译。提示涵盖了广泛的主题，包括艺术、流行文化、民间传统、名人姓名、自然物体、科学成就等的复杂概念。我们展示了使用流行生成模型对俄罗斯视觉概念表征进行并排比较的人工评估结果。

Title: Enhance-A-Video: Better Generated Video for Free

• Authors: Yang Luo, Xuanlei Zhao, Mengzhao Chen, Kaipeng Zhang, Wenqi Shao, Kai Wang, Zhangyang Wang, Yang You
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07508
• Pdf URL: https://arxiv.org/pdf/2502.07508
• Copy Paste: [[2502.07508]] Enhance-A-Video: Better Generated Video for Free(https://arxiv.org/abs/2502.07508)
• Keywords: generation
• Abstract: DiT-based video generation has achieved remarkable results, but research into enhancing existing models remains relatively unexplored. In this work, we introduce a training-free approach to enhance the coherence and quality of DiT-based generated videos, named Enhance-A-Video. The core idea is enhancing the cross-frame correlations based on non-diagonal temporal attention distributions. Thanks to its simple design, our approach can be easily applied to most DiT-based video generation frameworks without any retraining or fine-tuning. Across various DiT-based video generation models, our approach demonstrates promising improvements in both temporal consistency and visual quality. We hope this research can inspire future explorations in video generation enhancement.
• 摘要：基于 DiT 的视频生成已取得显著成果，但增强现有模型的研究仍相对较少。在这项工作中，我们引入了一种无需训练的方法来增强基于 DiT 的生成视频的连贯性和质量，称为 Enhance-A-Video。其核心思想是基于非对角时间注意力分布增强跨帧相关性。由于其设计简单，我们的方法可以轻松应用于大多数基于 DiT 的视频生成框架，而无需任何再训练或微调。在各种基于 DiT 的视频生成模型中，我们的方法在时间一致性和视觉质量方面都表现出了令人鼓舞的改进。我们希望这项研究能够激发未来对视频生成增强的探索。

Title: VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generation

• Authors: Sixiao Zheng, Zimian Peng, Yanpeng Zhou, Yi Zhu, Hang Xu, Xiangru Huang, Yanwei Fu
• Subjects: cs.CV, cs.AI, cs.LG, cs.MM
• Abstract URL: https://arxiv.org/abs/2502.07531
• Pdf URL: https://arxiv.org/pdf/2502.07531
• Copy Paste: [[2502.07531]] VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generation(https://arxiv.org/abs/2502.07531)
• Keywords: generation
• Abstract: Recent image-to-video generation methods have demonstrated success in enabling control over one or two visual elements, such as camera trajectory or object motion. However, these methods are unable to offer control over multiple visual elements due to limitations in data and network efficacy. In this paper, we introduce VidCRAFT3, a novel framework for precise image-to-video generation that enables control over camera motion, object motion, and lighting direction simultaneously. To better decouple control over each visual element, we propose the Spatial Triple-Attention Transformer, which integrates lighting direction, text, and image in a symmetric way. Since most real-world video datasets lack lighting annotations, we construct a high-quality synthetic video dataset, the VideoLightingDirection (VLD) dataset. This dataset includes lighting direction annotations and objects of diverse appearance, enabling VidCRAFT3 to effectively handle strong light transmission and reflection effects. Additionally, we propose a three-stage training strategy that eliminates the need for training data annotated with multiple visual elements (camera motion, object motion, and lighting direction) simultaneously. Extensive experiments on benchmark datasets demonstrate the efficacy of VidCRAFT3 in producing high-quality video content, surpassing existing state-of-the-art methods in terms of control granularity and visual coherence. All code and data will be publicly available. Project page: this https URL.
• 摘要：最近的图像到视频生成方法已证明能够成功控制一两个视觉元素，例如相机轨迹或物体运动。然而，由于数据和网络效率的限制，这些方法无法提供对多个视觉元素的控制。在本文中，我们介绍了 VidCRAFT3，这是一种用于精确图像到视频生成的新型框架，可以同时控制相机运动、物体运动和照明方向。为了更好地解耦对每个视觉元素的控制，我们提出了空间三重注意变换器，它以对称的方式集成了照明方向、文本和图像。由于大多数现实世界的视频数据集缺乏照明注释，我们构建了一个高质量的合成视频数据集，即 VideoLightingDirection (VLD) 数据集。该数据集包括照明方向注释和外观多样的物体，使 VidCRAFT3 能够有效处理强光透射和反射效果。此外，我们提出了一种三阶段训练策略，从而无需同时使用多个视觉元素（相机运动、物体运动和照明方向）注释训练数据。在基准数据集上进行的大量实验证明了 VidCRAFT3 在制作高质量视频内容方面的有效性，在控制粒度和视觉连贯性方面超越了现有的最先进方法。所有代码和数据都将公开。项目页面：此 https URL。

Title: Generative Modeling with Bayesian Sample Inference

• Authors: Marten Lienen, Marcel Kollovieh, Stephan Günnemann
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.07580
• Pdf URL: https://arxiv.org/pdf/2502.07580
• Copy Paste: [[2502.07580]] Generative Modeling with Bayesian Sample Inference(https://arxiv.org/abs/2502.07580)
• Keywords: generative
• Abstract: We derive a novel generative model from the simple act of Gaussian posterior inference. Treating the generated sample as an unknown variable to infer lets us formulate the sampling process in the language of Bayesian probability. Our model uses a sequence of prediction and posterior update steps to narrow down the unknown sample from a broad initial belief. In addition to a rigorous theoretical analysis, we establish a connection between our model and diffusion models and show that it includes Bayesian Flow Networks (BFNs) as a special case. In our experiments, we demonstrate improved performance over both BFNs and Variational Diffusion Models, achieving competitive likelihood scores on CIFAR10 and ImageNet.
• 摘要：我们从简单的高斯后验推理中推导出一种新颖的生成模型。将生成的样本视为要推断的未知变量，让我们能够用贝叶斯概率的语言来制定采样过程。我们的模型使用一系列预测和后验更新步骤，从广泛的初始信念中缩小未知样本的范围。除了严格的理论分析外，我们还建立了模型与扩散模型之间的联系，并表明它包括贝叶斯流网络 (BFN) 作为特例。在我们的实验中，我们展示了比 BFN 和变分扩散模型更好的性能，在 CIFAR10 和 ImageNet 上获得了具有竞争力的似然分数。

Title: SEMU: Singular Value Decomposition for Efficient Machine Unlearning

• Authors: Marcin Sendera, Łukasz Struski, Kamil Książek, Kryspin Musiol, Jacek Tabor, Dawid Rymarczyk
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07587
• Pdf URL: https://arxiv.org/pdf/2502.07587
• Copy Paste: [[2502.07587]] SEMU: Singular Value Decomposition for Efficient Machine Unlearning(https://arxiv.org/abs/2502.07587)
• Keywords: generative
• Abstract: While the capabilities of generative foundational models have advanced rapidly in recent years, methods to prevent harmful and unsafe behaviors remain underdeveloped. Among the pressing challenges in AI safety, machine unlearning (MU) has become increasingly critical to meet upcoming safety regulations. Most existing MU approaches focus on altering the most significant parameters of the model. However, these methods often require fine-tuning substantial portions of the model, resulting in high computational costs and training instabilities, which are typically mitigated by access to the original training dataset. In this work, we address these limitations by leveraging Singular Value Decomposition (SVD) to create a compact, low-dimensional projection that enables the selective forgetting of specific data points. We propose Singular Value Decomposition for Efficient Machine Unlearning (SEMU), a novel approach designed to optimize MU in two key aspects. First, SEMU minimizes the number of model parameters that need to be modified, effectively removing unwanted knowledge while making only minimal changes to the model's weights. Second, SEMU eliminates the dependency on the original training dataset, preserving the model's previously acquired knowledge without additional data requirements. Extensive experiments demonstrate that SEMU achieves competitive performance while significantly improving efficiency in terms of both data usage and the number of modified parameters.
• 摘要：虽然近年来生成基础模型的能力发展迅速，但防止有害和不安全行为的方法仍然不够完善。在人工智能安全面临的紧迫挑战中，机器学习 (MU) 已成为满足即将出台的安全法规日益重要的因素。大多数现有的 MU 方法都侧重于改变模型中最重要的参数。然而，这些方法通常需要对模型的大部分进行微调，导致计算成本高昂和训练不稳定性，而这些通常可以通过访问原始训练数据集来缓解。在这项工作中，我们通过利用奇异值分解 (SVD) 来创建一个紧凑的低维投影来解决这些限制，从而实现对特定数据点的选择性遗忘。我们提出了用于高效机器学习的奇异值分解 (SEMU)，这是一种旨在从两个关键方面优化 MU 的新方法。首先，SEMU 最大限度地减少了需要修改的模型参数数量，有效地删除了不需要的知识，同时只对模型的权重进行了最小的更改。其次，SEMU 消除了对原始训练数据集的依赖，保留了模型先前获得的知识，而无需额外的数据要求。大量实验表明，SEMU 实现了具有竞争力的性能，同时在数据使用和修改参数数量方面显著提高了效率。

Title: Consistency Training with Physical Constraints

• Authors: Che-Chia Chang, Chen-Yang Dai, Te-Sheng Lin, Ming-Chih Lai, Chieh-Hsin Lai
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07636
• Pdf URL: https://arxiv.org/pdf/2502.07636
• Copy Paste: [[2502.07636]] Consistency Training with Physical Constraints(https://arxiv.org/abs/2502.07636)
• Keywords: generative
• Abstract: We propose a physics-aware Consistency Training (CT) method that accelerates sampling in Diffusion Models with physical constraints. Our approach leverages a two-stage strategy: (1) learning the noise-to-data mapping via CT, and (2) incorporating physics constraints as a regularizer. Experiments on toy examples show that our method generates samples in a single step while adhering to the imposed constraints. This approach has the potential to efficiently solve partial differential equations (PDEs) using deep generative modeling.
• 摘要：我们提出了一种物理感知一致性训练 (CT) 方法，该方法可加速具有物理约束的扩散模型中的采样。我们的方法利用了两阶段策略：(1) 通过 CT 学习噪声到数据的映射，以及 (2) 将物理约束作为正则化器。对玩具示例的实验表明，我们的方法可以在遵守施加的约束的同时一步生成样本。这种方法有可能使用深度生成模型有效地解决偏微分方程 (PDE)。

Title: Goedel-Prover: A Frontier Model for Open-Source Automated Theorem Proving

• Authors: Yong Lin, Shange Tang, Bohan Lyu, Jiayun Wu, Hongzhou Lin, Kaiyu Yang, Jia Li, Mengzhou Xia, Danqi Chen, Sanjeev Arora, Chi Jin
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.07640
• Pdf URL: https://arxiv.org/pdf/2502.07640
• Copy Paste: [[2502.07640]] Goedel-Prover: A Frontier Model for Open-Source Automated Theorem Proving(https://arxiv.org/abs/2502.07640)
• Keywords: generation
• Abstract: We introduce Goedel-Prover, an open-source large language model (LLM) that achieves the state-of-the-art (SOTA) performance in automated formal proof generation for mathematical problems. The key challenge in this field is the scarcity of formalized math statements and proofs, which we tackle in the following ways. We train statement formalizers to translate the natural language math problems from Numina into formal language (Lean 4), creating a dataset of 1.64 million formal statements. LLMs are used to check that the formal statements accurately preserve the content of the original natural language problems. We then iteratively build a large dataset of formal proofs by training a series of provers. Each prover succeeds in proving many statements that the previous ones could not, and these new proofs are added to the training set for the next prover. The final prover outperforms all existing open-source models in whole-proof generation. On the miniF2F benchmark, it achieves a 57.6% success rate (Pass@32), exceeding the previous best open-source model by 7.6%. On PutnamBench, Goedel-Prover successfully solves 7 problems (Pass@512), ranking first on the leaderboard. Furthermore, it generates 29.7K formal proofs for Lean Workbook problems, nearly doubling the 15.7K produced by earlier works.
• 摘要：我们引入了 Goedel-Prover，这是一个开源大型语言模型 (LLM)，它在数学问题的自动形式化证明生成方面实现了最先进 (SOTA) 的性能。该领域的关键挑战是形式化数学语句和证明的稀缺性，我们通过以下方式解决这一问题。我们训练语句形式化器将 Numina 中的自然语言数学问题翻译成形式语言 (Lean 4)，从而创建一个包含 164 万个形式语句的数据集。LLM 用于检查形式语句是否准确保留了原始自然语言问题的内容。然后，我们通过训练一系列证明器迭代构建一个大型形式证明数据集。每个证明器都成功证明了许多之前无法证明的语句，这些新证明被添加到下一个证明器的训练集中。最终的证明器在整个证明生成方面优于所有现有的开源模型。在 miniF2F 基准测试中，它取得了 57.6% 的成功率 (Pass@32)，比之前最好的开源模型高出 7.6%。在 PutnamBench 上，Goedel-Prover 成功解决了 7 个问题 (Pass@512)，位居排行榜第一。此外，它还为 Lean Workbook 问题生成了 29.7K 个形式化证明，几乎是之前作品 15.7K 的两倍。

Title: Magic 1-For-1: Generating One Minute Video Clips within One Minute

• Authors: Hongwei Yi, Shitong Shao, Tian Ye, Jiantong Zhao, Qingyu Yin, Michael Lingelbach, Li Yuan, Yonghong Tian, Enze Xie, Daquan Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07701
• Pdf URL: https://arxiv.org/pdf/2502.07701
• Copy Paste: [[2502.07701]] Magic 1-For-1: Generating One Minute Video Clips within One Minute(https://arxiv.org/abs/2502.07701)
• Keywords: generation
• Abstract: In this technical report, we present Magic 1-For-1 (Magic141), an efficient video generation model with optimized memory consumption and inference latency. The key idea is simple: factorize the text-to-video generation task into two separate easier tasks for diffusion step distillation, namely text-to-image generation and image-to-video generation. We verify that with the same optimization algorithm, the image-to-video task is indeed easier to converge over the text-to-video task. We also explore a bag of optimization tricks to reduce the computational cost of training the image-to-video (I2V) models from three aspects: 1) model convergence speedup by using a multi-modal prior condition injection; 2) inference latency speed up by applying an adversarial step distillation, and 3) inference memory cost optimization with parameter sparsification. With those techniques, we are able to generate 5-second video clips within 3 seconds. By applying a test time sliding window, we are able to generate a minute-long video within one minute with significantly improved visual quality and motion dynamics, spending less than 1 second for generating 1 second video clips on average. We conduct a series of preliminary explorations to find out the optimal tradeoff between computational cost and video quality during diffusion step distillation and hope this could be a good foundation model for open-source explorations. The code and the model weights are available at this https URL.
• 摘要：在本技术报告中，我们介绍了 Magic 1-For-1 (Magic141)，这是一种高效的视频生成模型，具有优化的内存消耗和推理延迟。其关键思想很简单：将文本到视频生成任务分解为两个单独的、更简单的扩散步骤蒸馏任务，即文本到图像生成和图像到视频生成。我们验证了使用相同的优化算法，图像到视频任务确实比文本到视频任务更容易收敛。我们还从三个方面探索了一系列优化技巧，以降低训练图像到视频 (I2V) 模型的计算成本：1) 通过使用多模态先验条件注入来加速模型收敛；2) 通过应用对抗性步骤蒸馏来加速推理延迟，3) 通过参数稀疏化优化推理内存成本。利用这些技术，我们能够在 3 秒内生成 5 秒的视频片段。通过应用测试时间滑动窗口，我们能够在一分钟内生成一分钟的视频，并且显著提高视觉质量和运动动态，平均花费不到 1 秒的时间生成 1 秒的视频片段。我们进行了一系列初步探索，以找出扩散步骤蒸馏过程中计算成本和视频质量之间的最佳权衡，并希望这可以成为开源探索的良好基础模型。代码和模型权重可在此 https URL 上找到。

Title: Near-Optimal Sample Complexity in Reward-Free Kernel-Based Reinforcement Learning

• Authors: Aya Kayal, Sattar Vakili, Laura Toni, Alberto Bernacchia
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07715
• Pdf URL: https://arxiv.org/pdf/2502.07715
• Copy Paste: [[2502.07715]] Near-Optimal Sample Complexity in Reward-Free Kernel-Based Reinforcement Learning(https://arxiv.org/abs/2502.07715)
• Keywords: generative
• Abstract: Reinforcement Learning (RL) problems are being considered under increasingly more complex structures. While tabular and linear models have been thoroughly explored, the analytical study of RL under nonlinear function approximation, especially kernel-based models, has recently gained traction for their strong representational capacity and theoretical tractability. In this context, we examine the question of statistical efficiency in kernel-based RL within the reward-free RL framework, specifically asking: how many samples are required to design a near-optimal policy? Existing work addresses this question under restrictive assumptions about the class of kernel functions. We first explore this question by assuming a generative model, then relax this assumption at the cost of increasing the sample complexity by a factor of H, the length of the episode. We tackle this fundamental problem using a broad class of kernels and a simpler algorithm compared to prior work. Our approach derives new confidence intervals for kernel ridge regression, specific to our RL setting, which may be of broader applicability. We further validate our theoretical findings through simulations.
• 摘要：人们正在越来越复杂的结构下考虑强化学习 (RL) 问题。虽然表格和线性模型已经得到彻底探索，但非线性函数逼近下的 RL 分析研究，尤其是基于核的模型，最近因其强大的表示能力和理论可处理性而受到关注。在此背景下，我们在无奖励 RL 框架内研究了基于核的 RL 的统计效率问题，具体问：设计接近最优策略需要多少样本？现有工作在对核函数类的限制性假设下解决了这个问题。我们首先通过假设生成模型来探索这个问题，然后放宽这一假设，代价是将样本复杂度增加 H 倍，即情节的长度。我们使用广泛的核类和与之前的工作相比更简单的算法来解决这个基本问题。我们的方法为核岭回归推导出新的置信区间，特定于我们的 RL 设置，这可能具有更广泛的适用性。我们通过模拟进一步验证了我们的理论发现。

Title: Revisiting Non-Acyclic GFlowNets in Discrete Environments

• Authors: Nikita Morozov, Ian Maksimov, Daniil Tiapkin, Sergey Samsonov
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.07735
• Pdf URL: https://arxiv.org/pdf/2502.07735
• Copy Paste: [[2502.07735]] Revisiting Non-Acyclic GFlowNets in Discrete Environments(https://arxiv.org/abs/2502.07735)
• Keywords: generative
• Abstract: Generative Flow Networks (GFlowNets) are a family of generative models that learn to sample objects from a given probability distribution, potentially known up to a normalizing constant. Instead of working in the object space, GFlowNets proceed by sampling trajectories in an appropriately constructed directed acyclic graph environment, greatly relying on the acyclicity of the graph. In our paper, we revisit the theory that relaxes the acyclicity assumption and present a simpler theoretical framework for non-acyclic GFlowNets in discrete environments. Moreover, we provide various novel theoretical insights related to training with fixed backward policies, the nature of flow functions, and connections between entropy-regularized RL and non-acyclic GFlowNets, which naturally generalize the respective concepts and theoretical results from the acyclic setting. In addition, we experimentally re-examine the concept of loss stability in non-acyclic GFlowNet training, as well as validate our own theoretical findings.
• 摘要：生成流网络 (GFlowNets) 是一类生成模型，它们学习从给定的概率分布中采样对象，可能已知一个归一化常数。GFlowNets 不是在对象空间中工作，而是通过在适当构建的有向无环图环境中采样轨迹来进行，这在很大程度上依赖于图的无环性。在我们的论文中，我们重新审视了放宽无环性假设的理论，并为离散环境中的非无环 GFlowNets 提出了一个更简单的理论框架。此外，我们提供了各种新颖的理论见解，涉及使用固定后向策略进行训练、流函数的性质以及熵正则化 RL 和非无环 GFlowNets 之间的联系，这些见解自然地概括了无环设置中的各自概念和理论结果。此外，我们通过实验重新审视了非无环 GFlowNet 训练中的损失稳定性概念，并验证了我们自己的理论发现。

Title: Next Block Prediction: Video Generation via Semi-Auto-Regressive Modeling

• Authors: Shuhuai Ren, Shuming Ma, Xu Sun, Furu Wei
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.07737
• Pdf URL: https://arxiv.org/pdf/2502.07737
• Copy Paste: [[2502.07737]] Next Block Prediction: Video Generation via Semi-Auto-Regressive Modeling(https://arxiv.org/abs/2502.07737)
• Keywords: generation
• Abstract: Next-Token Prediction (NTP) is a de facto approach for autoregressive (AR) video generation, but it suffers from suboptimal unidirectional dependencies and slow inference speed. In this work, we propose a semi-autoregressive (semi-AR) framework, called Next-Block Prediction (NBP), for video generation. By uniformly decomposing video content into equal-sized blocks (e.g., rows or frames), we shift the generation unit from individual tokens to blocks, allowing each token in the current block to simultaneously predict the corresponding token in the next block. Unlike traditional AR modeling, our framework employs bidirectional attention within each block, enabling tokens to capture more robust spatial dependencies. By predicting multiple tokens in parallel, NBP models significantly reduce the number of generation steps, leading to faster and more efficient inference. Our model achieves FVD scores of 103.3 on UCF101 and 25.5 on K600, outperforming the vanilla NTP model by an average of 4.4. Furthermore, thanks to the reduced number of inference steps, the NBP model generates 8.89 frames (128x128 resolution) per second, achieving an 11x speedup. We also explored model scales ranging from 700M to 3B parameters, observing significant improvements in generation quality, with FVD scores dropping from 103.3 to 55.3 on UCF101 and from 25.5 to 19.5 on K600, demonstrating the scalability of our approach.
• 摘要：下一个标记预测 (NTP) 是自回归 (AR) 视频生成的一种事实上的方法，但它存在单向依赖性不理想和推理速度慢的问题。在这项工作中，我们提出了一种用于视频生成的半自回归 (半 AR) 框架，称为下一个块预测 (NBP)。通过将视频内容均匀分解为大小相等的块（例如，行或帧），我们将生成单元从单个标记转移到块，允许当前块中的每个标记同时预测下一个块中的相应标记。与传统的 AR 建模不同，我们的框架在每个块内采用双向注意力，使标记能够捕获更强大的空间依赖性。通过并行预测多个标记，NBP 模型显著减少了生成步骤的数量，从而实现更快、更高效的推理。我们的模型在 UCF101 上获得了 103.3 的 FVD 分数，在 K600 上获得了 25.5 的 FVD 分数，平均比普通 NTP 模型高出 4.4。此外，由于推理步骤减少，NBP 模型每秒生成 8.89 帧（128x128 分辨率），速度提高了 11 倍。我们还探索了从 700M 到 3B 参数的模型规模，观察到生成质量显著提高，UCF101 上的 FVD 分数从 103.3 降至 55.3，K600 上的 FVD 分数从 25.5 降至 19.5，证明了我们方法的可扩展性。

Title: CausalGeD: Blending Causality and Diffusion for Spatial Gene Expression Generation

• Authors: Rabeya Tus Sadia, Md Atik Ahamed, Qiang Cheng
• Subjects: cs.CV, q-bio.GN
• Abstract URL: https://arxiv.org/abs/2502.07751
• Pdf URL: https://arxiv.org/pdf/2502.07751
• Copy Paste: [[2502.07751]] CausalGeD: Blending Causality and Diffusion for Spatial Gene Expression Generation(https://arxiv.org/abs/2502.07751)
• Keywords: generation
• Abstract: The integration of single-cell RNA sequencing (scRNA-seq) and spatial transcriptomics (ST) data is crucial for understanding gene expression in spatial context. Existing methods for such integration have limited performance, with structural similarity often below 60\%, We attribute this limitation to the failure to consider causal relationships between genes. We present CausalGeD, which combines diffusion and autoregressive processes to leverage these relationships. By generalizing the Causal Attention Transformer from image generation to gene expression data, our model captures regulatory mechanisms without predefined relationships. Across 10 tissue datasets, CausalGeD outperformed state-of-the-art baselines by 5- 32\% in key metrics, including Pearson's correlation and structural similarity, advancing both technical and biological insights.
• 摘要：单细胞 RNA 测序 (scRNA-seq) 和空间转录组学 (ST) 数据的整合对于理解空间背景下的基因表达至关重要。现有的此类整合方法性能有限，结构相似性通常低于 60\%，我们将这种限制归因于未能考虑基因之间的因果关系。我们提出了 CausalGeD，它结合了扩散和自回归过程来利用这些关系。通过将因果注意力转换器从图像生成推广到基因表达数据，我们的模型可以捕获没有预定义关系的调控机制。在 10 个组织数据集中，CausalGeD 在关键指标（包括皮尔逊相关性和结构相似性）方面的表现比最先进的基线高出 5-32\%，从而提高了技术和生物学洞察力。

Title: Direct Ascent Synthesis: Revealing Hidden Generative Capabilities in Discriminative Models

• Authors: Stanislav Fort, Jonathan Whitaker
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07753
• Pdf URL: https://arxiv.org/pdf/2502.07753
• Copy Paste: [[2502.07753]] Direct Ascent Synthesis: Revealing Hidden Generative Capabilities in Discriminative Models(https://arxiv.org/abs/2502.07753)
• Keywords: generation, generative
• Abstract: We demonstrate that discriminative models inherently contain powerful generative capabilities, challenging the fundamental distinction between discriminative and generative architectures. Our method, Direct Ascent Synthesis (DAS), reveals these latent capabilities through multi-resolution optimization of CLIP model representations. While traditional inversion attempts produce adversarial patterns, DAS achieves high-quality image synthesis by decomposing optimization across multiple spatial scales (1x1 to 224x224), requiring no additional training. This approach not only enables diverse applications -- from text-to-image generation to style transfer -- but maintains natural image statistics ($1/f^2$ spectrum) and guides the generation away from non-robust adversarial patterns. Our results demonstrate that standard discriminative models encode substantially richer generative knowledge than previously recognized, providing new perspectives on model interpretability and the relationship between adversarial examples and natural image synthesis.
• 摘要：我们证明判别模型本身就具有强大的生成能力，挑战了判别和生成架构之间的根本区别。我们的方法直接上升合成 (DAS) 通过多分辨率优化 CLIP 模型表示来揭示这些潜在能力。虽然传统的反演尝试会产生对抗模式，但 DAS 通过在多个空间尺度（1x1 到 224x224）上分解优化来实现高质量的图像合成，无需额外训练。这种方法不仅支持从文本到图像生成到风格转换的各种应用，而且还保持了自然图像统计数据（$1/f^2$ 频谱）并引导生成远离非鲁棒对抗模式。我们的结果表明，标准判别模型编码的生成知识比以前认识到的要丰富得多，为模型的可解释性以及对抗示例与自然图像合成之间的关系提供了新的视角。

Title: Stay-Positive: A Case for Ignoring Real Image Features in Fake Image Detection

• Authors: Anirudh Sundara Rajan, Yong Jae Lee
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07778
• Pdf URL: https://arxiv.org/pdf/2502.07778
• Copy Paste: [[2502.07778]] Stay-Positive: A Case for Ignoring Real Image Features in Fake Image Detection(https://arxiv.org/abs/2502.07778)
• Keywords: generative
• Abstract: Detecting AI generated images is a challenging yet essential task. A primary difficulty arises from the detectors tendency to rely on spurious patterns, such as compression artifacts, which can influence its decisions. These issues often stem from specific patterns that the detector associates with the real data distribution, making it difficult to isolate the actual generative traces. We argue that an image should be classified as fake if and only if it contains artifacts introduced by the generative model. Based on this premise, we propose Stay Positive, an algorithm designed to constrain the detectors focus to generative artifacts while disregarding those associated with real data. Experimental results demonstrate that detectors trained with Stay Positive exhibit reduced susceptibility to spurious correlations, leading to improved generalization and robustness to post processing. Additionally, unlike detectors that associate artifacts with real images, those that focus purely on fake artifacts are better at detecting inpainted real images.
• 摘要：检测 AI 生成的图像是一项具有挑战性但必不可少的任务。主要困难在于检测器倾向于依赖虚假模式，例如压缩伪影，这会影响其决策。这些问题通常源于检测器与真实数据分布相关联的特定模式，因此很难分离实际的生成轨迹。我们认为，当且仅当图像包含生成模型引入的伪影时，才应将其归类为假图像。基于这个前提，我们提出了 Stay Positive，这是一种旨在将检测器的焦点限制在生成伪影上而忽略与真实数据相关的伪影的算法。实验结果表明，使用 Stay Positive 训练的检测器对虚假相关性的敏感性降低，从而提高了泛化能力和对后处理的鲁棒性。此外，与将伪影与真实图像相关联的检测器不同，那些只关注假伪影的检测器更擅长检测修复的真实图像。

Title: DarwinLM: Evolutionary Structured Pruning of Large Language Models

• Authors: Shengkun Tang, Oliver Sieberling, Eldar Kurtic, Zhiqiang Shen, Dan Alistarh
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.07780
• Pdf URL: https://arxiv.org/pdf/2502.07780
• Copy Paste: [[2502.07780]] DarwinLM: Evolutionary Structured Pruning of Large Language Models(https://arxiv.org/abs/2502.07780)
• Keywords: generation
• Abstract: Large Language Models (LLMs) have achieved significant success across various NLP tasks. However, their massive computational costs limit their widespread use, particularly in real-time applications. Structured pruning offers an effective solution by compressing models and directly providing end-to-end speed improvements, regardless of the hardware environment. Meanwhile, different components of the model exhibit varying sensitivities towards pruning, calling for \emph{non-uniform} model compression. However, a pruning method should not only identify a capable substructure, but also account for post-compression training. To this end, we propose \sysname, a method for \emph{training-aware} structured pruning. \sysname builds upon an evolutionary search process, generating multiple offspring models in each generation through mutation, and selecting the fittest for survival. To assess the effect of post-training, we incorporate a lightweight, multistep training process within the offspring population, progressively increasing the number of tokens and eliminating poorly performing models in each selection stage. We validate our method through extensive experiments on Llama-2-7B, Llama-3.1-8B and Qwen-2.5-14B-Instruct, achieving state-of-the-art performance for structured pruning. For instance, \sysname surpasses ShearedLlama while requiring $5\times$ less training data during post-compression training.
• 摘要：大型语言模型 (LLM) 在各种 NLP 任务中取得了显著的成功。然而，它们巨大的计算成本限制了它们的广泛使用，特别是在实时应用中。结构化剪枝提供了一种有效的解决方案，它通过压缩模型并直接提供端到端的速度改进，而不管硬件环境如何。同时，模型的不同组件对剪枝表现出不同的敏感性，需要 \emph{非均匀} 模型压缩。然而，剪枝方法不仅应该识别有能力的子结构，还应该考虑压缩后的训练。为此，我们提出了 \sysname，一种 \emph{训练感知} 结构化剪枝方法。\sysname 建立在进化搜索过程的基础上，通过突变在每一代生成多个后代模型，并选择最适合生存的模型。为了评估后训练的效果，我们在后代群体中加入了轻量级、多步骤的训练过程，逐步增加标记数量并在每个选择阶段消除表现不佳的模型。我们通过对 Llama-2-7B、Llama-3.1-8B 和 Qwen-2.5-14B-Instruct 进行大量实验来验证我们的方法，在结构化剪枝方面取得了最先进的性能。例如，\sysname 超越了 ShearedLlama，同时在压缩后训练期间所需的训练数据减少了 $5\times$。

Title: Pippo: High-Resolution Multi-View Humans from a Single Image

• Authors: Yash Kant, Ethan Weber, Jin Kyu Kim, Rawal Khirodkar, Su Zhaoen, Julieta Martinez, Igor Gilitschenski, Shunsuke Saito, Timur Bagautdinov
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.07785
• Pdf URL: https://arxiv.org/pdf/2502.07785
• Copy Paste: [[2502.07785]] Pippo: High-Resolution Multi-View Humans from a Single Image(https://arxiv.org/abs/2502.07785)
• Keywords: generation, generative
• Abstract: We present Pippo, a generative model capable of producing 1K resolution dense turnaround videos of a person from a single casually clicked photo. Pippo is a multi-view diffusion transformer and does not require any additional inputs - e.g., a fitted parametric model or camera parameters of the input image. We pre-train Pippo on 3B human images without captions, and conduct multi-view mid-training and post-training on studio captured humans. During mid-training, to quickly absorb the studio dataset, we denoise several (up to 48) views at low-resolution, and encode target cameras coarsely using a shallow MLP. During post-training, we denoise fewer views at high-resolution and use pixel-aligned controls (e.g., Spatial anchor and Plucker rays) to enable 3D consistent generations. At inference, we propose an attention biasing technique that allows Pippo to simultaneously generate greater than 5 times as many views as seen during training. Finally, we also introduce an improved metric to evaluate 3D consistency of multi-view generations, and show that Pippo outperforms existing works on multi-view human generation from a single image.
• 摘要：我们提出了 Pippo，这是一个生成模型，它能够根据一张随意点击的照片生成 1K 分辨率的人物密集转身视频。Pippo 是一个多视图扩散变换器，不需要任何其他输入 - 例如，输入图像的拟合参数模型或相机参数。我们在 3B 张无字幕的人体图像上对 Pippo 进行预训练，并对工作室拍摄的人体进行多视图中期训练和后期训​​练。在中期训练期间，为了快速吸收工作室数据集，我们对几个（最多 48 个）低分辨率视图进行去噪，并使用浅层 MLP 对目标相机进行粗略编码。在后训练期间，我们对高分辨率下的较少视图进行去噪，并使用像素对齐的控件（例如空间锚点和 Plucker 射线）实现 3D 一致生成。在推理方面，我们提出了一种注意力偏差技术，使 Pippo 能够同时生成比训练期间多 5 倍的视图。最后，我们还引入了一个改进的指标来评估多视图生成的 3D 一致性，并表明 Pippo 在从单个图像进行多视图人体生成方面优于现有作品。