2025-08-06

Title: ECGTwin: Personalized ECG Generation Using Controllable Diffusion Model

• Authors: Yongfan Lai, Bo Liu, Xinyan Guan, Qinghao Zhao, Hongyan Li, Shenda Hong
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.02720
• Pdf URL: https://arxiv.org/pdf/2508.02720
• Copy Paste: [[2508.02720]] ECGTwin: Personalized ECG Generation Using Controllable Diffusion Model(https://arxiv.org/abs/2508.02720)
• Keywords: generation, generative
• Abstract: Personalized electrocardiogram (ECG) generation is to simulate a patient's ECG digital twins tailored to specific conditions. It has the potential to transform traditional healthcare into a more accurate individualized paradigm, while preserving the key benefits of conventional population-level ECG synthesis. However, this promising task presents two fundamental challenges: extracting individual features without ground truth and injecting various types of conditions without confusing generative model. In this paper, we present ECGTwin, a two-stage framework designed to address these challenges. In the first stage, an Individual Base Extractor trained via contrastive learning robustly captures personal features from a reference ECG. In the second stage, the extracted individual features, along with a target cardiac condition, are integrated into the diffusion-based generation process through our novel AdaX Condition Injector, which injects these signals via two dedicated and specialized pathways. Both qualitative and quantitative experiments have demonstrated that our model can not only generate ECG signals of high fidelity and diversity by offering a fine-grained generation controllability, but also preserving individual-specific features. Furthermore, ECGTwin shows the potential to enhance ECG auto-diagnosis in downstream application, confirming the possibility of precise personalized healthcare solutions.
• 摘要：个性化心电图（ECG）的生成是为了模拟患者的ECG数字双胞胎，适合特定条件。它有可能将传统的医疗保健转变为更准确的个性化范式，同时保留传统人口级的心电图合成的关键好处。但是，这项有前途的任务提出了两个基本挑战：提取个人特征而没有地面真理，并注入各种类型的条件而不会混淆生成模型。在本文中，我们提出了ECGTWIN，这是一个旨在应对这些挑战的两阶段框架。在第一阶段，通过对比度学习训练的单个基础提取器可靠地捕获参考ECG的个人功能。在第二阶段，提取的单个特征以及目标心脏条件通过我们的新型ADAX条件喷油器整合到基于扩散的生成过程中，该过程通过两种专用和专业的途径向这些信号注入这些信号。定性和定量实验都表明，我们的模型不仅可以通过提供精细颗粒的生成可控性，而且可以保留个人特定的特征来产生高保真和多样性的ECG信号。此外，ECGTWIN显示了在下游应用中增强ECG自动诊断的潜力，从而确认了精确的个性化医疗保健解决方案的可能性。

Title: Embedding-Enhanced Probabilistic Modeling of Ferroelectric Field Effect Transistors (FeFETs)

• Authors: Tasnia Nobi Afee, Jack Hutchins, Md Mazharul Islam, Thomas Kampfe, Ahmedullah Aziz
• Subjects: cs.LG, cs.ET
• Abstract URL: https://arxiv.org/abs/2508.02737
• Pdf URL: https://arxiv.org/pdf/2508.02737
• Copy Paste: [[2508.02737]] Embedding-Enhanced Probabilistic Modeling of Ferroelectric Field Effect Transistors (FeFETs)(https://arxiv.org/abs/2508.02737)
• Keywords: generation
• Abstract: FeFETs hold strong potential for advancing memory and logic technologies, but their inherent randomness arising from both operational cycling and fabrication variability poses significant challenges for accurate and reliable modeling. Capturing this variability is critical, as it enables designers to predict behavior, optimize performance, and ensure reliability and robustness against variations in manufacturing and operating conditions. Existing deterministic and machine learning-based compact models often fail to capture the full extent of this variability or lack the mathematical smoothness required for stable circuit-level integration. In this work, we present an enhanced probabilistic modeling framework for FeFETs that addresses these limitations. Building upon a Mixture Density Network (MDN) foundation, our approach integrates C-infinity continuous activation functions for smooth, stable learning and a device-specific embedding layer to capture intrinsic physical variability across devices. Sampling from the learned embedding distribution enables the generation of synthetic device instances for variability-aware simulation. With an R2 of 0.92, the model demonstrates high accuracy in capturing the variability of FeFET current behavior. Altogether, this framework provides a scalable, data-driven solution for modeling the full stochastic behavior of FeFETs and offers a strong foundation for future compact model development and circuit simulation integration.
• 摘要：FEFET具有推进记忆和逻辑技术的强大潜力，但是它们由操作循环和制造变异性引起的固有随机性对准确和可靠的建模构成了重大挑战。捕获这种可变性至关重要，因为它使设计人员能够预测行为，优化性能，并确保可靠性和鲁棒性，以应对制造和操作条件的变化。现有的确定性和基于机器学习的紧凑型模型通常无法捕获这种可变性的全部程度，或者缺乏稳定电路级集成所需的数学平滑度。在这项工作中，我们为FEFET提供了一个增强的概率建模框架，以解决这些限制。在混合密度网络（MDN）基础的基础上，我们的方法集成了C-内度连续激活功能，以进行平滑，稳定的学习和特定于设备的嵌入层，以捕获设备之间的内在物理变异性。从学到的嵌入分布中进行采样，可以生成合成设备实例，以实现可变性的模拟。该模型以R2为0.92，在捕获FEFET电流行为的可变性方面表现出很高的精度。总的来说，该框架提供了一个可扩展的，数据驱动的解决方案，用于建模FEFET的完整随机行为，并为未来的紧凑型模型开发和电路模拟集成提供了坚实的基础。

Title: Synthetic medical data generation: state of the art and application to trauma mechanism classification

• Authors: Océane Doremus, Ariel Guerra-Adames, Marta Avalos-Fernandez, Vianney Jouhet, Cédric Gil-Jardiné, Emmanuel Lagarde
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2508.02771
• Pdf URL: https://arxiv.org/pdf/2508.02771
• Copy Paste: [[2508.02771]] Synthetic medical data generation: state of the art and application to trauma mechanism classification(https://arxiv.org/abs/2508.02771)
• Keywords: generation
• Abstract: Faced with the challenges of patient confidentiality and scientific reproducibility, research on machine learning for health is turning towards the conception of synthetic medical databases. This article presents a brief overview of state-of-the-art machine learning methods for generating synthetic tabular and textual data, focusing their application to the automatic classification of trauma mechanisms, followed by our proposed methodology for generating high-quality, synthetic medical records combining tabular and unstructured text data.
• 摘要：面对患者机密性和科学可重复性的挑战，对健康的机器学习研究正在转向合成医疗数据库的概念。本文简要概述了最新的机器学习方法，该方法用于生成综合表格和文本数据，将其应用于创伤机制的自动分类，然后是我们提出的方法，用于生成高质量的，合成的医疗记录，结合了表格和非结构化的文本数据。

Title: DreamVVT: Mastering Realistic Video Virtual Try-On in the Wild via a Stage-Wise Diffusion Transformer Framework

• Authors: Tongchun Zuo, Zaiyu Huang, Shuliang Ning, Ente Lin, Chao Liang, Zerong Zheng, Jianwen Jiang, Yuan Zhang, Mingyuan Gao, Xin Dong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.02807
• Pdf URL: https://arxiv.org/pdf/2508.02807
• Copy Paste: [[2508.02807]] DreamVVT: Mastering Realistic Video Virtual Try-On in the Wild via a Stage-Wise Diffusion Transformer Framework(https://arxiv.org/abs/2508.02807)
• Keywords: generation
• Abstract: Video virtual try-on (VVT) technology has garnered considerable academic interest owing to its promising applications in e-commerce advertising and entertainment. However, most existing end-to-end methods rely heavily on scarce paired garment-centric datasets and fail to effectively leverage priors of advanced visual models and test-time inputs, making it challenging to accurately preserve fine-grained garment details and maintain temporal consistency in unconstrained scenarios. To address these challenges, we propose DreamVVT, a carefully designed two-stage framework built upon Diffusion Transformers (DiTs), which is inherently capable of leveraging diverse unpaired human-centric data to enhance adaptability in real-world scenarios. To further leverage prior knowledge from pretrained models and test-time inputs, in the first stage, we sample representative frames from the input video and utilize a multi-frame try-on model integrated with a vision-language model (VLM), to synthesize high-fidelity and semantically consistent keyframe try-on images. These images serve as complementary appearance guidance for subsequent video generation. \textbf{In the second stage}, skeleton maps together with fine-grained motion and appearance descriptions are extracted from the input content, and these along with the keyframe try-on images are then fed into a pretrained video generation model enhanced with LoRA adapters. This ensures long-term temporal coherence for unseen regions and enables highly plausible dynamic motions. Extensive quantitative and qualitative experiments demonstrate that DreamVVT surpasses existing methods in preserving detailed garment content and temporal stability in real-world scenarios. Our project page this https URL
• 摘要：Video Virtual Try-On（VVT）技术由于其在电子商务广告和娱乐中的有希望的应用而引起了极大的学术兴趣。但是，大多数现有的端到端方法都在很大程度上依赖于以稀缺为中心的服装数据集，并且无法有效利用高级视觉模型和测试时间输入的先验，这使得准确地保留精细颗粒的服装细节并保持不受约束的场景中的时间一致性。为了应对这些挑战，我们提出了DreamVvt，这是一个基于扩散变压器（DITS）构建的精心设计的两阶段框架，该框架本质上能够利用多元化的不成对人为中心的数据来增强现实世界中的适应性。为了进一步利用验证的模型和测试时间输入的先验知识，在第一阶段，我们从输入视频中采样了代表性帧，并利用与视觉语言模型（VLM）集成的多帧try-On模型，以合成高效率和语义上一致的一致性的键frame try-frame try-on hon-on try-on try-on home图像。这些图像是随后的视频生成的互补外观指导。 \ textbf {在第二阶段}，从输入内容中提取了骨架图，并从输入内容中提取出细粒度的运动和外观描述，然后将这些图像与关键框架的图像一起被馈送到预告片的视频生成模型中，并通过Lora适配器增强。这确保了看不见的区域的长期时间连贯性，并实现了高度合理的动态运动。广泛的定量和定性实验表明，DreamVVT超过了在实际情况下保存详细的服装内容和时间稳定性方面的现有方法。我们的项目页面此HTTPS URL

Title: Learning from B Cell Evolution: Adaptive Multi-Expert Diffusion for Antibody Design via Online Optimization

• Authors: Hanqi Feng, Peng Qiu, Mengchun Zhang, Yiran Tao, You Fan, Jingtao Xu, Barnabas Poczos
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.02834
• Pdf URL: https://arxiv.org/pdf/2508.02834
• Copy Paste: [[2508.02834]] Learning from B Cell Evolution: Adaptive Multi-Expert Diffusion for Antibody Design via Online Optimization(https://arxiv.org/abs/2508.02834)
• Keywords: generation
• Abstract: Recent advances in diffusion models have shown remarkable potential for antibody design, yet existing approaches apply uniform generation strategies that cannot adapt to each antigen's unique requirements. Inspired by B cell affinity maturation, where antibodies evolve through multi-objective optimization balancing affinity, stability, and self-avoidance, we propose the first biologically-motivated framework that leverages physics-based domain knowledge within an online meta-learning system. Our method employs multiple specialized experts (van der Waals, molecular recognition, energy balance, and interface geometry) whose parameters evolve during generation based on iterative feedback, mimicking natural antibody refinement cycles. Instead of fixed protocols, this adaptive guidance discovers personalized optimization strategies for each target. Our experiments demonstrate that this approach: (1) discovers optimal SE(3)-equivariant guidance strategies for different antigen classes without pre-training, preserving molecular symmetries throughout optimization; (2) significantly enhances hotspot coverage and interface quality through target-specific adaptation, achieving balanced multi-objective optimization characteristic of therapeutic antibodies; (3) establishes a paradigm for iterative refinement where each antibody-antigen system learns its unique optimization profile through online evaluation; (4) generalizes effectively across diverse design challenges, from small epitopes to large protein interfaces, enabling precision-focused campaigns for individual targets.
• 摘要：扩散模型的最新进展显示出抗体设计的巨大潜力，但是现有的方法采用了统一的发电策略，无法适应每个抗原的独特要求。受B细胞亲和力成熟的启发，该抗体通过多目标优化的平衡亲和力，稳定性和自我避免而演变，我们提出了第一个在线元学习系统中利用基于物理的域知识的生物学动机框架。我们的方法采用了多个专业专家（范德华，分子识别，能量平衡和界面几何形状），它们的参数基于迭代反馈在发电过程中演变，模仿了天然抗体细化周期。该自适应指导不是固定的协议，而是发现了每个目标的个性化优化策略。我们的实验表明，这种方法：（1）发现了不同抗原类别的最佳SE（3） - 等级指导策略，而无需预训练，在整个优化过程中保留分子对称性； （2）通过针对特定目标的适应来显着增强热点覆盖范围和界面质量，从而实现治疗性抗体的平衡多目标优化特征； （3）建立了迭代精炼的范式，每个抗体 - 抗原系统通过在线评估学习其独特的优化概况； （4）从小型表现到大蛋白质界面，有效地跨越各种设计挑战，从而为个人目标提供精确的运动。

Title: Defending Against Knowledge Poisoning Attacks During Retrieval-Augmented Generation

• Authors: Kennedy Edemacu, Vinay M. Shashidhar, Micheal Tuape, Dan Abudu, Beakcheol Jang, Jong Wook Kim
• Subjects: cs.LG, cs.IR
• Abstract URL: https://arxiv.org/abs/2508.02835
• Pdf URL: https://arxiv.org/pdf/2508.02835
• Copy Paste: [[2508.02835]] Defending Against Knowledge Poisoning Attacks During Retrieval-Augmented Generation(https://arxiv.org/abs/2508.02835)
• Keywords: generation
• Abstract: Retrieval-Augmented Generation (RAG) has emerged as a powerful approach to boost the capabilities of large language models (LLMs) by incorporating external, up-to-date knowledge sources. However, this introduces a potential vulnerability to knowledge poisoning attacks, where attackers can compromise the knowledge source to mislead the generation model. One such attack is the PoisonedRAG in which the injected adversarial texts steer the model to generate an attacker-chosen response to a target question. In this work, we propose novel defense methods, FilterRAG and ML-FilterRAG, to mitigate the PoisonedRAG attack. First, we propose a new property to uncover distinct properties to differentiate between adversarial and clean texts in the knowledge data source. Next, we employ this property to filter out adversarial texts from clean ones in the design of our proposed approaches. Evaluation of these methods using benchmark datasets demonstrate their effectiveness, with performances close to those of the original RAG systems.
• 摘要：通过合并外部的，最新的知识源来提高大语言模型（LLMS）的功能，以提高大型语言模型（LLM）的能力的强大方法出现。但是，这引入了对知识中毒攻击的潜在脆弱性，在这种攻击者中，攻击者可能会损害知识源以误导生成模型。一种这样的攻击是毒药，其中注入的对抗文本引导该模型对目标问题产生攻击者选择的回答。在这项工作中，我们提出了新颖的防御方法，FilterRag和ML-Filterrag，以减轻毒药的攻击。首先，我们提出了一个新属性，以发现不同的属性，以区分知识数据源中的对抗和干净文本。接下来，我们采用此属性来滤除我们提出的方法的设计中，从干净的文本中滤除对抗文本。使用基准数据集对这些方法的评估证明了它们的有效性，并且表现靠近原始的抹布系统。

Title: VisuCraft: Enhancing Large Vision-Language Models for Complex Visual-Guided Creative Content Generation via Structured Information Extraction

• Authors: Rongxin Jiang, Robert Long, Chenghao Gu, Mingrui Yan
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2508.02890
• Pdf URL: https://arxiv.org/pdf/2508.02890
• Copy Paste: [[2508.02890]] VisuCraft: Enhancing Large Vision-Language Models for Complex Visual-Guided Creative Content Generation via Structured Information Extraction(https://arxiv.org/abs/2508.02890)
• Keywords: generation
• Abstract: This paper introduces VisuCraft, a novel framework designed to significantly enhance the capabilities of Large Vision-Language Models (LVLMs) in complex visual-guided creative content generation. Existing LVLMs often exhibit limitations in maintaining high visual fidelity, genuine creativity, and precise adherence to nuanced user instructions when generating long-form texts. VisuCraft addresses these challenges by integrating a multimodal structured information extractor (E) and a dynamic prompt generation module (G). The extractor distills fine-grained visual attributes from input images into a rich, structured representation, which the dynamic prompt module then combines with user instructions to create highly optimized prompts for underlying LVLMs (e.g., LLaVA, InstructBLIP). Evaluated on the self-constructed ImageStoryGen-500K dataset using VisuGen Metrics (Visual Grounding, Creativity, and Instruction Adherence), VisuCraft consistently outperforms baseline LVLMs across tasks like story generation and poetry composition. Our results demonstrate remarkable improvements, particularly in creativity and instruction adherence, validating VisuCraft's effectiveness in producing imaginative, visually grounded, and user-aligned long-form creative text. This work unlocks new potential for LVLMs in sophisticated creative AI applications.
• 摘要：本文介绍了Visucraft，这是一个新颖的框架，旨在显着增强复杂视觉引导的创意成分生成中大型视觉模型（LVLM）的功能。现有的LVLM经常在保持高视觉保真度，真正的创造力以及对生成长形式文本时的细微差别用户指令方面表现出局限性。 Visucraft通过整合多模式结构信息提取器（E）和动态及时生成模块（G）来解决这些挑战。提取器将细粒度的视觉属性从输入图像提炼成丰富的结构化表示形式，然后动态提示模块与用户指令结合使用，以创建高度优化的提示，用于基础LVLMS（例如Llava，TenderchBlip）。使用Visugen指标（视觉接地，创造力和教学依从性）对自我建构的ImagestoryGen-500K数据集进行评估，Visucraft始终在故事产生和诗歌组成之类的任务上胜过基线LVLM。我们的结果表明了显着的改进，尤其是在创造力和指导依从性方面，验证了Visucraft在产生想象力，视觉扎根和用户一致的长期创意文本方面的有效性。这项工作将在复杂的创意AI应用程序中解锁了LVLM的新潜力。

Title: How Would It Sound? Material-Controlled Multimodal Acoustic Profile Generation for Indoor Scenes

• Authors: Mahnoor Fatima Saad, Ziad Al-Halah
• Subjects: cs.CV, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2508.02905
• Pdf URL: https://arxiv.org/pdf/2508.02905
• Copy Paste: [[2508.02905]] How Would It Sound? Material-Controlled Multimodal Acoustic Profile Generation for Indoor Scenes(https://arxiv.org/abs/2508.02905)
• Keywords: generation
• Abstract: How would the sound in a studio change with a carpeted floor and acoustic tiles on the walls? We introduce the task of material-controlled acoustic profile generation, where, given an indoor scene with specific audio-visual characteristics, the goal is to generate a target acoustic profile based on a user-defined material configuration at inference time. We address this task with a novel encoder-decoder approach that encodes the scene's key properties from an audio-visual observation and generates the target Room Impulse Response (RIR) conditioned on the material specifications provided by the user. Our model enables the generation of diverse RIRs based on various material configurations defined dynamically at inference time. To support this task, we create a new benchmark, the Acoustic Wonderland Dataset, designed for developing and evaluating material-aware RIR prediction methods under diverse and challenging settings. Our results demonstrate that the proposed model effectively encodes material information and generates high-fidelity RIRs, outperforming several baselines and state-of-the-art methods.
• 摘要：工作室中的声音将如何用毯子上的地板和墙壁上的声学瓷砖改变？我们介绍了材料控制的声学轮廓生成的任务，在室内场景中具有特定的视听特征，目标是根据推理时在推理时基于用户定义的材料配置生成目标声学概况。我们使用一种新颖的编码器方法来解决此任务，该方法从音频观察中编码场景的关键属性，并生成基于用户提供的材料规范条件的目标房间脉冲响应（RIR）。我们的模型可以基于在推理时动态定义的各种材料配置来生成不同的RIR。为了支持这项任务，我们创建了一个新的基准测试，即声学仙境数据集，该数据集旨在开发和评估不同和挑战性的环境下的材料感知的RIR预测方法。我们的结果表明，所提出的模型有效地编码了物质信息并生成高保真的RIR，表现优于几个基线和最先进的方法。

Title: GrandJury: A Collaborative Machine Learning Model Evaluation Protocol for Dynamic Quality Rubrics

• Authors: Arthur Cho
• Subjects: cs.LG, cs.AI, cs.HC
• Abstract URL: https://arxiv.org/abs/2508.02926
• Pdf URL: https://arxiv.org/pdf/2508.02926
• Copy Paste: [[2508.02926]] GrandJury: A Collaborative Machine Learning Model Evaluation Protocol for Dynamic Quality Rubrics(https://arxiv.org/abs/2508.02926)
• Keywords: generative
• Abstract: Generative Machine Learning models have become central to modern systems, powering applications in creative writing, summarization, multi-hop reasoning, and context-aware dialogue. These models underpin large-scale AI assistants, workflow automation, and autonomous decision-making. In such domains, acceptable response is rarely absolute or static, but plural and highly context-dependent. Yet standard evaluation regimes still rely on static, benchmark-style tests, incentivizing optimization toward leaderboard scores rather than alignment with dynamic user needs or evolving realities. GrandJury introduces a formal evaluation protocol combining time-decayed aggregation, complete traceability, with the support of dynamic, transparent task rubric attribution, and multi-rater human judgment. Together, these elements enable pluralistic, accountable evaluation that captures evolving consensus and surfaces disagreement. We provide an open-source implementation (grandjury PyPI package) and a public collection of Large Language Model (LLM) inference outputs to illustrate the need and method. GrandJury provides a new paradigm for AI practitioners when evaluating machine learning outputs without absolute ground truth.
• 摘要：生成机器学习模型已成为现代系统的核心，在创意写作，摘要，多跳上推理和上下文感知对话中为应用程序提供了动力。这些模型是大规模AI助手，工作流程自动化和自动决策的基础。在这样的域中，可接受的响应很少是绝对或静态的，但复数且高度依赖于上下文。然而，标准评估制度仍然依赖于静态，基准风格的测试，激励优化对排行榜得分，而不是与动态用户需求或不断发展的现实保持一致。 Grandjury引入了一项正式的评估协议，该协议结合了时间份额的聚合，完全可追溯性，并支持动态，透明的任务标题归因和多评价者人类的判断。这些要素共同实现了多元化的，负责任的评估，从而捕获了不断发展的共识和表面分歧。我们提供开源实施（Grandjury PYPI软件包）和大型语言模型（LLM）推理输出的公共集合，以说明需求和方法。 Grandjury在无需绝对地面真相评估机器学习输出时为AI从业者提供了新的范式。

Title: X-Actor: Emotional and Expressive Long-Range Portrait Acting from Audio

• Authors: Chenxu Zhang, Zenan Li, Hongyi Xu, You Xie, Xiaochen Zhao, Tianpei Gu, Guoxian Song, Xin Chen, Chao Liang, Jianwen Jiang, Linjie Luo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.02944
• Pdf URL: https://arxiv.org/pdf/2508.02944
• Copy Paste: [[2508.02944]] X-Actor: Emotional and Expressive Long-Range Portrait Acting from Audio(https://arxiv.org/abs/2508.02944)
• Keywords: generation
• Abstract: We present X-Actor, a novel audio-driven portrait animation framework that generates lifelike, emotionally expressive talking head videos from a single reference image and an input audio clip. Unlike prior methods that emphasize lip synchronization and short-range visual fidelity in constrained speaking scenarios, X-Actor enables actor-quality, long-form portrait performance capturing nuanced, dynamically evolving emotions that flow coherently with the rhythm and content of speech. Central to our approach is a two-stage decoupled generation pipeline: an audio-conditioned autoregressive diffusion model that predicts expressive yet identity-agnostic facial motion latent tokens within a long temporal context window, followed by a diffusion-based video synthesis module that translates these motions into high-fidelity video animations. By operating in a compact facial motion latent space decoupled from visual and identity cues, our autoregressive diffusion model effectively captures long-range correlations between audio and facial dynamics through a diffusion-forcing training paradigm, enabling infinite-length emotionally-rich motion prediction without error accumulation. Extensive experiments demonstrate that X-Actor produces compelling, cinematic-style performances that go beyond standard talking head animations and achieves state-of-the-art results in long-range, audio-driven emotional portrait acting.
• 摘要：我们提出了X-Actor，这是一种新颖的音频驱动肖像画框架，从单个参考图像和输入音频剪辑中产生栩栩如生，情感表达的会说话的头视频。与以前强调唇部同步和在受约束的语言场景中的短程视觉保真度的方法不同，X演员可以使演员质量，长形式的肖像性能捕捉到细微的细致，动态发展的情绪，这些情绪与节奏和语音的内容相干地流动。我们方法的核心是一个两阶段脱钩的生成管道：一个音频条件的自回归扩散模型，该模型可预测在长时间的时间上下文窗口中，可预测具有扩散的视频综合模块，将这些动作转化为高素质视频动画。通过在紧凑的面部运动潜在空间与视觉和身份提示解耦，我们的自回归扩散模型有效地捕获了音频和面部动力学之间的远距离相关性，通过扩散式训练训练范式，实现无限长度的情绪上的情感运动，而无需误差积累。广泛的实验表明，X演员会产生引人注目的电影风格表演，超越了标准的会说话的头部动画，并实现了最新的最先进的表演，从而导致了远程，音频驱动的情感肖像画。

Title: Seeing It Before It Happens: In-Generation NSFW Detection for Diffusion-Based Text-to-Image Models

• Authors: Fan Yang, Yihao Huang, Jiayi Zhu, Ling Shi, Geguang Pu, Jin Song Dong, Kailong Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03006
• Pdf URL: https://arxiv.org/pdf/2508.03006
• Copy Paste: [[2508.03006]] Seeing It Before It Happens: In-Generation NSFW Detection for Diffusion-Based Text-to-Image Models(https://arxiv.org/abs/2508.03006)
• Keywords: generation
• Abstract: Diffusion-based text-to-image (T2I) models enable high-quality image generation but also pose significant risks of misuse, particularly in producing not-safe-for-work (NSFW) content. While prior detection methods have focused on filtering prompts before generation or moderating images afterward, the in-generation phase of diffusion models remains largely unexplored for NSFW detection. In this paper, we introduce In-Generation Detection (IGD), a simple yet effective approach that leverages the predicted noise during the diffusion process as an internal signal to identify NSFW content. This approach is motivated by preliminary findings suggesting that the predicted noise may capture semantic cues that differentiate NSFW from benign prompts, even when the prompts are adversarially crafted. Experiments conducted on seven NSFW categories show that IGD achieves an average detection accuracy of 91.32% over naive and adversarial NSFW prompts, outperforming seven baseline methods.
• 摘要：基于扩散的文本形象（T2I）模型可实现高质量的图像生成，但也构成了滥用的重大风险，尤其是在生产不安全的工作（NSFW）内容时。虽然先前的检测方法已重点是在生成或随后对图像进行调节之前的过滤提示，但扩散模型的生成阶段对于NSFW检测仍未探索。在本文中，我们引入了生成检测（IGD），这是一种简单而有效的方法，它利用扩散过程中预测的噪声作为内部信号来识别NSFW含量。这种方法是由初步发现激发的，表明预测的噪声可能会捕获NSFW与良性提示区分开的语义提示，即使提示是对对抗性制作的。在七个NSFW类别上进行的实验表明，IGD的平均检测准确性比NAIVE和对抗性NSFW提示达到91.32％，表现优于七个基线方法。

Title: MoCA: Identity-Preserving Text-to-Video Generation via Mixture of Cross Attention

• Authors: Qi Xie (1), Yongjia Ma (2), Donglin Di (2), Xuehao Gao (3), Xun Yang (1) ((1) University of Science and Technology of China, (2) Li Auto, (3) Northwestern Polytechnical University)
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03034
• Pdf URL: https://arxiv.org/pdf/2508.03034
• Copy Paste: [[2508.03034]] MoCA: Identity-Preserving Text-to-Video Generation via Mixture of Cross Attention(https://arxiv.org/abs/2508.03034)
• Keywords: generation
• Abstract: Achieving ID-preserving text-to-video (T2V) generation remains challenging despite recent advances in diffusion-based models. Existing approaches often fail to capture fine-grained facial dynamics or maintain temporal identity coherence. To address these limitations, we propose MoCA, a novel Video Diffusion Model built on a Diffusion Transformer (DiT) backbone, incorporating a Mixture of Cross-Attention mechanism inspired by the Mixture-of-Experts paradigm. Our framework improves inter-frame identity consistency by embedding MoCA layers into each DiT block, where Hierarchical Temporal Pooling captures identity features over varying timescales, and Temporal-Aware Cross-Attention Experts dynamically model spatiotemporal relationships. We further incorporate a Latent Video Perceptual Loss to enhance identity coherence and fine-grained details across video frames. To train this model, we collect CelebIPVid, a dataset of 10,000 high-resolution videos from 1,000 diverse individuals, promoting cross-ethnicity generalization. Extensive experiments on CelebIPVid show that MoCA outperforms existing T2V methods by over 5% across Face similarity.
• 摘要：尽管基于扩散的模型取得了进步，但实现具有ID ID的文本对视频（T2V）的生成仍然具有挑战性。现有方法通常无法捕获细粒度的面部动力或保持时间身份连贯性。为了解决这些局限性，我们提出了MOCA，这是建立在扩散变压器（DIT）骨架上的新型视频扩散模型，并结合了受到专家范式混合物启发的跨注意机制的混合物。我们的框架通过将MOCA层嵌入每个DIT块来提高框架间的身份一致性，在该块中，层次的时间池在不同的时间标准上捕获了身份特征，而时间感知的跨意识专家则动态地模拟了时空关系。我们进一步结合了一个潜在的视频感知损失，以增强视频框架的身份连贯性和细粒度的细节。为了训练该模型，我们收集了Celebipvid，这是一个来自1,000个不同个人的10,000个高分辨率视频的数据集，从而促进了跨种族的概括。在Celebipvid上进行的广泛实验表明，MOCA在面部相似性的情况下优于现有的T2V方法超过5％。

Title: A Novel Multimodal Framework for Early Detection of Alzheimers Disease Using Deep Learning

• Authors: Tatwadarshi P Nagarhalli, Sanket Patil, Vishal Pande, Uday Aswalekar, Prafulla Patil
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2508.03046
• Pdf URL: https://arxiv.org/pdf/2508.03046
• Copy Paste: [[2508.03046]] A Novel Multimodal Framework for Early Detection of Alzheimers Disease Using Deep Learning(https://arxiv.org/abs/2508.03046)
• Keywords: generative
• Abstract: Alzheimers Disease (AD) is a progressive neurodegenerative disorder that poses significant challenges in its early diagnosis, often leading to delayed treatment and poorer outcomes for patients. Traditional diagnostic methods, typically reliant on single data modalities, fall short of capturing the multifaceted nature of the disease. In this paper, we propose a novel multimodal framework for the early detection of AD that integrates data from three primary sources: MRI imaging, cognitive assessments, and biomarkers. This framework employs Convolutional Neural Networks (CNN) for analyzing MRI images and Long Short-Term Memory (LSTM) networks for processing cognitive and biomarker data. The system enhances diagnostic accuracy and reliability by aggregating results from these distinct modalities using advanced techniques like weighted averaging, even in incomplete data. The multimodal approach not only improves the robustness of the detection process but also enables the identification of AD at its earliest stages, offering a significant advantage over conventional methods. The integration of biomarkers and cognitive tests is particularly crucial, as these can detect Alzheimer's long before the onset of clinical symptoms, thereby facilitating earlier intervention and potentially altering the course of the disease. This research demonstrates that the proposed framework has the potential to revolutionize the early detection of AD, paving the way for more timely and effective treatments
• 摘要：阿尔茨海默氏病（AD）是一种进行性神经退行性疾病，在早期诊断中构成了重大挑战，通常会导致治疗延迟，患者的结局较差。传统的诊断方法通常依赖于单个数据方式，而没有捕获该疾病的多方面性质。在本文中，我们提出了一个新型的多模式框架，用于早期检测AD，该框架整合了来自三个主要来源的数据：MRI成像，认知评估和生物标志物。该框架采用卷积神经网络（CNN）来分析MRI图像和长期记忆（LSTM）网络来处理认知和生物标志物数据。该系统通过使用加权平均等高级技术从这些不同的方式中汇总这些不同方式的结果来提高诊断准确性和可靠性，即使在不完整的数据中也是如此。多模式方法不仅改善了检测过程的鲁棒性，而且还可以在其最早的阶段鉴定AD，从而比常规方法具有显着优势。生物标志物和认知测试的整合尤为重要，因为这些可以在临床症状发作之前很长时间检测阿尔茨海默氏症，从而促进早期的干预措施并可能改变疾病的病程。这项研究表明，所提出的框架有可能彻底改变AD的早期发现，为更及时有效的治疗铺平道路

Title: Multi-human Interactive Talking Dataset

• Authors: Zeyu Zhu, Weijia Wu, Mike Zheng Shou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03050
• Pdf URL: https://arxiv.org/pdf/2508.03050
• Copy Paste: [[2508.03050]] Multi-human Interactive Talking Dataset(https://arxiv.org/abs/2508.03050)
• Keywords: generation
• Abstract: Existing studies on talking video generation have predominantly focused on single-person monologues or isolated facial animations, limiting their applicability to realistic multi-human interactions. To bridge this gap, we introduce MIT, a large-scale dataset specifically designed for multi-human talking video generation. To this end, we develop an automatic pipeline that collects and annotates multi-person conversational videos. The resulting dataset comprises 12 hours of high-resolution footage, each featuring two to four speakers, with fine-grained annotations of body poses and speech interactions. It captures natural conversational dynamics in multi-speaker scenario, offering a rich resource for studying interactive visual behaviors. To demonstrate the potential of MIT, we furthur propose CovOG, a baseline model for this novel task. It integrates a Multi-Human Pose Encoder (MPE) to handle varying numbers of speakers by aggregating individual pose embeddings, and an Interactive Audio Driver (IAD) to modulate head dynamics based on speaker-specific audio features. Together, these components showcase the feasibility and challenges of generating realistic multi-human talking videos, establishing MIT as a valuable benchmark for future research. The code is avalibale at: this https URL.
• 摘要：现有关于谈话视频生成的研究主要集中在单人的独白或孤立的面部动画上，从而限制了它们对现实的多人类互动的适用性。为了弥合这一差距，我们介绍了MIT，MIT是一个专门为多人说话的视频生成而设计的大规模数据集。为此，我们开发了一种自动管道，该管道收集和注释多人对话视频。由此产生的数据集包含12个小时的高分辨率录像，每个录像带具有两到四个扬声器，并具有对身体姿势和语音相互作用的细粒度注释。它捕获了多演讲者场景中的自然对话动态，为研究交互式视觉行为提供了丰富的资源。为了证明MIT的潜力，我们Furthur提出了Covog，这是这项新任务的基线模型。它集成了多人类姿势编码器（MPE），通过汇总单个姿势嵌入来处理不同数量的扬声器，以及一个交互式音频驱动程序（IAD），以根据扬声器特定音频功能来调节头部动力学。这些组件共同展示了产生现实的多人说话视频的可行性和挑战，并确立了MIT作为未来研究的宝贵基准。该代码为Avalibale，网址为：此HTTPS URL。

Title: HiTeC: Hierarchical Contrastive Learning on Text-Attributed Hypergraph with Semantic-Aware Augmentation

• Authors: Mengting Pan, Fan Li, Xiaoyang Wang, Wenjie Zhang, Xuemin Lin
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.03104
• Pdf URL: https://arxiv.org/pdf/2508.03104
• Copy Paste: [[2508.03104]] HiTeC: Hierarchical Contrastive Learning on Text-Attributed Hypergraph with Semantic-Aware Augmentation(https://arxiv.org/abs/2508.03104)
• Keywords: generation
• Abstract: Contrastive learning (CL) has become a dominant paradigm for self-supervised hypergraph learning, enabling effective training without costly labels. However, node entities in real-world hypergraphs are often associated with rich textual information, which is overlooked in prior works. Directly applying existing CL-based methods to such text-attributed hypergraphs (TAHGs) leads to three key limitations: (1) The common use of graph-agnostic text encoders overlooks the correlations between textual content and hypergraph topology, resulting in suboptimal representations. (2) Their reliance on random data augmentations introduces noise and weakens the contrastive objective. (3) The primary focus on node- and hyperedge-level contrastive signals limits the ability to capture long-range dependencies, which is essential for expressive representation learning. Although HyperBERT pioneers CL on TAHGs, its co-training paradigm suffers from poor scalability. To fill the research gap, we introduce HiTeC, a two-stage hierarchical contrastive learning framework with semantic-aware augmentation for scalable and effective self-supervised learning on TAHGs. In the first stage, we pre-train the text encoder with a structure-aware contrastive objective to overcome the graph-agnostic nature of conventional methods. In the second stage, we introduce two semantic-aware augmentation strategies, including prompt-enhanced text augmentation and semantic-aware hyperedge drop, to facilitate informative view generation. Furthermore, we propose a multi-scale contrastive loss that extends existing objectives with an $s$-walk-based subgraph-level contrast to better capture long-range dependencies. By decoupling text encoder pretraining from hypergraph contrastive learning, this two-stage design enhances scalability without compromising representation quality. Extensive experiments confirm the effectiveness of HiTeC.
• 摘要：对比度学习（CL）已成为自我监视的超图学习的主导范式，从而实现有效的培训而无需昂贵的标签。但是，现实世界中超图中的节点实体通常与丰富的文本信息相关联，这在先前的工作中被忽略了。将现有基于CL的方法直接应用于此类文本属性的超图（TAHGS）导致三个关键局限性：（1）图形无形文本编码器的共同用途可忽略文本内容与超级刻度拓扑之间的相关性，从而导致次优表示。 （2）他们对随机数据的依赖引入了噪声并削弱了对比度目标。 （3）对节点和超边级对比信号的主要重点限制了捕获长期依赖性的能力，这对于表达性表示学习至关重要。尽管Hyperbert Pioneers Cl在Tahgs上，但其共同训练范式的可扩展性较差。为了填补研究差距，我们介绍了HITEC，这是一个两阶段的分层对比学习框架，具有语义感知的增强，以实现可扩展有效的自我监督学习。在第一阶段，我们将对文本编码器进行培训，具有结构感知的对比目标，以克服传统方法的图形性质。在第二阶段，我们介绍了两种语义感知的增强策略，包括迅速增强的文本增强和语义感知的HyperEdge Drop，以促进信息的信息。此外，我们提出了一个多尺度的对比损失，该损失以基于$ s $ walk的子图级对比度扩展了现有目标，以更好地捕获远距离依赖性。通过将文本编码器与超图形对比学习进行预绘，这种两阶段的设计可增强可扩展性，而不会损害表示质量。广泛的实验证实了HITEC的有效性。

Title: UniEdit-I: Training-free Image Editing for Unified VLM via Iterative Understanding, Editing and Verifying

• Authors: Chengyu Bai, Jintao Chen, Xiang Bai, Yilong Chen, Qi She, Ming Lu, Shanghang Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03142
• Pdf URL: https://arxiv.org/pdf/2508.03142
• Copy Paste: [[2508.03142]] UniEdit-I: Training-free Image Editing for Unified VLM via Iterative Understanding, Editing and Verifying(https://arxiv.org/abs/2508.03142)
• Keywords: generation
• Abstract: In recent years, unified vision-language models (VLMs) have rapidly advanced, effectively tackling both visual understanding and generation tasks within a single design. While many unified VLMs have explored various design choices, the recent hypothesis from OpenAI's GPT-4o suggests a promising generation pipeline: Understanding VLM->Visual Feature->Projector->Diffusion Model->Image. The understanding VLM is frozen, and only the generation-related modules are trained. This pipeline maintains the strong capability of understanding VLM while enabling the image generation ability of the unified VLM. Although this pipeline has shown very promising potential for the future development of unified VLM, how to easily enable image editing capability is still unexplored. In this paper, we introduce a novel training-free framework named UniEdit-I to enable the unified VLM with image editing capability via three iterative steps: understanding, editing, and verifying. 1. The understanding step analyzes the source image to create a source prompt through structured semantic analysis and makes minimal word replacements to form the target prompt based on the editing instruction. 2. The editing step introduces a time-adaptive offset, allowing for coherent editing from coarse to fine throughout the denoising process. 3. The verification step checks the alignment between the target prompt and the intermediate edited image, provides automatic consistency scores and corrective feedback, and determines whether to stop early or continue the editing loop. This understanding, editing, and verifying loop iterates until convergence, delivering high-fidelity editing in a training-free manner. We implemented our method based on the latest BLIP3-o and achieved state-of-the-art (SOTA) performance on the GEdit-Bench benchmark.
• 摘要：近年来，统一的视觉模型（VLMS）迅速高级，有效地解决了单个设计中的视觉理解和发电任务。尽管许多统一的VLM都探索了各种设计选择，但OpenAI的GPT-4O的最新假设提出了有希望的生成管道：理解VLM--> Visual Trauty-> projector->投影仪 - >扩散模型 - >图像。理解VLM被冷冻，并且只有与一代相关的模块经过训练。该管道保持了强大的理解VLM的能力，同时可以实现统一VLM的图像产生能力。尽管该管道对统一VLM的未来开发表现出非常有希望的潜力，但如何轻松启用图像编辑功能仍未探索。在本文中，我们介绍了一个名为uniedit-I的新颖的无培训框架，以通过三个迭代步骤启用具有图像编辑能力的统一VLM：理解，编辑和验证。 1。理解步骤分析源图像，通过结构化的语义分析创建源提示，并根据编辑说明使最小的单词替换以形成目标提示。 2。编辑步骤引入了时间自适应的偏移，可以在整个denoising过程中从粗到细的连贯编辑。 3。验证步骤检查目标提示和中间编辑图像之间的对齐，提供自动一致性得分和纠正反馈，并确定是提早停止还是继续编辑循环。这种理解，编辑和验证循环会迭代，直到收敛为止，以无训练的方式提供高保真编辑。我们根据最新的Blip3-O实施了方法，并在GEDIT基础基准测试中实现了最先进的（SOTA）性能。

Title: SARD: Segmentation-Aware Anomaly Synthesis via Region-Constrained Diffusion with Discriminative Mask Guidance

• Authors: Yanshu Wang, Xichen Xu, Xiaoning Lei, Guoyang Xie
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03143
• Pdf URL: https://arxiv.org/pdf/2508.03143
• Copy Paste: [[2508.03143]] SARD: Segmentation-Aware Anomaly Synthesis via Region-Constrained Diffusion with Discriminative Mask Guidance(https://arxiv.org/abs/2508.03143)
• Keywords: generation
• Abstract: Synthesizing realistic and spatially precise anomalies is essential for enhancing the robustness of industrial anomaly detection systems. While recent diffusion-based methods have demonstrated strong capabilities in modeling complex defect patterns, they often struggle with spatial controllability and fail to maintain fine-grained regional fidelity. To overcome these limitations, we propose SARD (Segmentation-Aware anomaly synthesis via Region-constrained Diffusion with discriminative mask Guidance), a novel diffusion-based framework specifically designed for anomaly generation. Our approach introduces a Region-Constrained Diffusion (RCD) process that preserves the background by freezing it and selectively updating only the foreground anomaly regions during the reverse denoising phase, thereby effectively reducing background artifacts. Additionally, we incorporate a Discriminative Mask Guidance (DMG) module into the discriminator, enabling joint evaluation of both global realism and local anomaly fidelity, guided by pixel-level masks. Extensive experiments on the MVTec-AD and BTAD datasets show that SARD surpasses existing methods in segmentation accuracy and visual quality, setting a new state-of-the-art for pixel-level anomaly synthesis.
• 摘要：综合现实和空间精确的异常对于增强工业异常检测系统的鲁棒性至关重要。尽管最近的基于扩散的方法表明在建模复杂的缺陷模式中表现出很强的能力，但它们通常在空间可控性方面挣扎，并且无法维持细粒度的区域忠诚度。为了克服这些局限性，我们提出了SAD（通过区域限制性掩模指导通过区域约束扩散进行分割的异常合成），这是一个专门为异常生成设计的新型基于扩散的框架。我们的方法引入了一个受区域约束的扩散（RCD）过程，该过程通过冷冻并选择性地仅在反向denoising阶段进行前景异常区域来保存背景，从而有效地减少了背景文物。此外，我们将歧视性掩盖指南（DMG）模块纳入歧视器中，从而在以像素级掩码为指导下对全球现实主义和局部异常忠诚度进行了联合评估。在MVTEC-AD和BTAD数据集上进行的广泛实验表明，SARD超过了分割精度和视觉质量的现有方法，为像素级异常合成设定了新的最新技术。

Title: Frontier: Simulating the Next Generation of LLM Inference Systems

• Authors: Yicheng Feng, Xin Tan, Kin Hang Sew, Yimin Jiang, Yibo Zhu, Hong Xu
• Subjects: cs.LG, cs.AI, cs.DC
• Abstract URL: https://arxiv.org/abs/2508.03148
• Pdf URL: https://arxiv.org/pdf/2508.03148
• Copy Paste: [[2508.03148]] Frontier: Simulating the Next Generation of LLM Inference Systems(https://arxiv.org/abs/2508.03148)
• Keywords: generation
• Abstract: Large Language Model (LLM) inference is growing increasingly complex with the rise of Mixture-of-Experts (MoE) models and disaggregated architectures that decouple components like prefill/decode (PD) or attention/FFN (AF) for heterogeneous scaling. Existing simulators, architected for co-located, dense models, are unable to capture the intricate system dynamics of these emerging paradigms. We present Frontier, a high-fidelity simulator designed from the ground up for this new landscape. Frontier introduces a unified framework to model both co-located and disaggregated systems, providing native support for MoE inference with expert parallelism (EP). It enables the simulation of complex workflows like cross-cluster expert routing and advanced pipelining strategies for latency hiding. To ensure fidelity and usability, Frontier incorporates refined operator models for improved accuracy. Frontier empowers the community to design and optimize the future of LLM inference at scale.
• 摘要：大型语言模型（LLM）的推论随着专家的混合物（MOE）模型的兴起和分解的结构的增长而变得越来越复杂，这些结构将诸如Prefl/Decode（PD）或注意/FFN（AF）之类的组件进行异质缩放。现有的模拟器（用于共同存在的密集模型）无法捕获这些新兴范式的复杂系统动力学。我们介绍了Frontier，这是一种从头开始设计的高保真模拟器，为这一新景观设计。 Frontier引入了一个统一的框架，以建模共同定位和分解的系统，从而提供了与专家并行性（EP）（EP）的MOE推断的本机支持。它可以模拟复杂的工作流程，例如跨群集专家路由和延迟隐藏的高级管道策略。为了确保保真度和可用性，Frontier合并了精制的操作员模型，以提高准确性。 Frontier授权社区设计和优化LLM推论的规模推论。

Title: SAVER: Mitigating Hallucinations in Large Vision-Language Models via Style-Aware Visual Early Revision

• Authors: Zhaoxu Li, Chenqi Kong, Yi Yu, Qiangqiang Wu, Xinghao Jiang, Ngai-Man Cheung, Bihan Wen, Alex Kot, Xudong Jiang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03177
• Pdf URL: https://arxiv.org/pdf/2508.03177
• Copy Paste: [[2508.03177]] SAVER: Mitigating Hallucinations in Large Vision-Language Models via Style-Aware Visual Early Revision(https://arxiv.org/abs/2508.03177)
• Keywords: generative
• Abstract: Large Vision-Language Models (LVLMs) recently achieve significant breakthroughs in understanding complex visual-textual contexts. However, hallucination issues still limit their real-world applicability. Although previous mitigation methods effectively reduce hallucinations in photographic images, they largely overlook the potential risks posed by stylized images, which play crucial roles in critical scenarios such as game scene understanding, art education, and medical analysis. In this work, we first construct a dataset comprising photographic images and their corresponding stylized versions with carefully annotated caption labels. We then conduct head-to-head comparisons on both discriminative and generative tasks by benchmarking 13 advanced LVLMs on the collected datasets. Our findings reveal that stylized images tend to induce significantly more hallucinations than their photographic counterparts. To address this issue, we propose Style-Aware Visual Early Revision SAVER, a novel mechanism that dynamically adjusts LVLMs' final outputs based on the token-level visual attention patterns, leveraging early-layer feedback to mitigate hallucinations caused by stylized images. Extensive experiments demonstrate that SAVER achieves state-of-the-art performance in hallucination mitigation across various models, datasets, and tasks.
• 摘要：大型视觉模型（LVLM）最近在理解复杂的视觉文本上下文时取得了重大突破。但是，幻觉问题仍然限制了其现实世界的适用性。尽管以前的缓解方法有效地减少了摄影图像中的幻觉，但它们在很大程度上忽略了风格化图像带来的潜在风险，这些图像在关键场景中扮演着重要的角色，例如游戏场景理解，艺术教育和医学分析。在这项工作中，我们首先构建一个包括摄影图像的数据集及其相应的风格化版本，并带有精心注释的字幕标签。然后，我们通过基准收集的数据集上的13个高级LVLM来对歧视性和生成任务进行正面比较。我们的发现表明，程式化的图像往往会引起比其摄影对应物更大的幻觉。为了解决这个问题，我们提出了一种风格感知的视觉早期修订节省，这是一种新型机制，该机制基于令牌级别的视觉注意力模式，动态调整LVLM的最终输出，利用早期层的反馈来减轻由风格化图像引起的幻觉。广泛的实验表明，Saver在各种型号，数据集和任务之间缓解幻觉的最新性能。

Title: Scaling DRL for Decision Making: A Survey on Data, Network, and Training Budget Strategies

• Authors: Yi Ma, Hongyao Tang, Chenjun Xiao, Yaodong Yang, Wei Wei, Jianye Hao, Jiye Liang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2508.03194
• Pdf URL: https://arxiv.org/pdf/2508.03194
• Copy Paste: [[2508.03194]] Scaling DRL for Decision Making: A Survey on Data, Network, and Training Budget Strategies(https://arxiv.org/abs/2508.03194)
• Keywords: generation
• Abstract: In recent years, the expansion of neural network models and training data has driven remarkable progress in deep learning, particularly in computer vision and natural language processing. This advancement is underpinned by the concept of Scaling Laws, which demonstrates that scaling model parameters and training data enhances learning performance. While these fields have witnessed breakthroughs, such as the development of large language models like GPT-4 and advanced vision models like Midjourney, the application of scaling laws in deep reinforcement learning (DRL) remains relatively unexplored. Despite its potential to improve performance, the integration of scaling laws into DRL for decision making has not been fully realized. This review addresses this gap by systematically analyzing scaling strategies in three dimensions: data, network, and training budget. In data scaling, we explore methods to optimize data efficiency through parallel sampling and data generation, examining the relationship between data volume and learning outcomes. For network scaling, we investigate architectural enhancements, including monolithic expansions, ensemble and MoE methods, and agent number scaling techniques, which collectively enhance model expressivity while posing unique computational challenges. Lastly, in training budget scaling, we evaluate the impact of distributed training, high replay ratios, large batch sizes, and auxiliary training on training efficiency and convergence. By synthesizing these strategies, this review not only highlights their synergistic roles in advancing DRL for decision making but also provides a roadmap for future research. We emphasize the importance of balancing scalability with computational efficiency and outline promising directions for leveraging scaling to unlock the full potential of DRL in various tasks such as robot control, autonomous driving and LLM training.
• 摘要：近年来，神经网络模型和培训数据的扩展促进了深度学习方面的显着进步，尤其是在计算机视觉和自然语言处理中。这种进步的基础是缩放定律的概念，这表明缩放模型参数和培训数据可增强学习绩效。尽管这些领域已经看到了突破，但例如开发GPT-4和Midjourney等高级视觉模型的开发，但扩展定律在深度强化学习（DRL）中的应用仍然相对尚未探索。尽管具有提高绩效的潜力，但尚未完全实现将缩放定律整合到DRL中进行决策。这篇评论通过系统地分析三个维度的缩放策略来解决这一差距：数据，网络和培训预算。在数据缩放中，我们探讨了通过并行采样和数据生成来优化数据效率的方法，从而研究了数据量和学习成果之间的关系。对于网络缩放，我们研究了建筑的增强功能，包括整体扩展，集合和MOE方法以及代理编号缩放技术，这些技术共同增强了模型表现力，同时构成了独特的计算挑战。最后，在培训预算缩放中，我们评估了分布式培训，高重型比率，大批量大小以及辅助培训对培训效率和收敛的影响。通过综合这些策略，本综述不仅强调了它们在推进DRL进行决策中的协同作用，而且还为将来的研究提供了路线图。我们强调了平衡可扩展性与计算效率和概述有希望的方向的重要性，以利用缩放来解锁DRL在机器人控制，自动驾驶和LLM培训等各种任务中的全部潜力。

Title: Neovascularization Segmentation via a Multilateral Interaction-Enhanced Graph Convolutional Network

• Authors: Tao Chen, Dan Zhang, Da Chen, Huazhu Fu, Kai Jin, Shanshan Wang, Laurent D. Cohen, Yitian Zhao, Quanyong Yi, Jiong Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03197
• Pdf URL: https://arxiv.org/pdf/2508.03197
• Copy Paste: [[2508.03197]] Neovascularization Segmentation via a Multilateral Interaction-Enhanced Graph Convolutional Network(https://arxiv.org/abs/2508.03197)
• Keywords: generation
• Abstract: Choroidal neovascularization (CNV), a primary characteristic of wet age-related macular degeneration (wet AMD), represents a leading cause of blindness worldwide. In clinical practice, optical coherence tomography angiography (OCTA) is commonly used for studying CNV-related pathological changes, due to its micron-level resolution and non-invasive nature. Thus, accurate segmentation of CNV regions and vessels in OCTA images is crucial for clinical assessment of wet AMD. However, challenges existed due to irregular CNV shapes and imaging limitations like projection artifacts, noises and boundary blurring. Moreover, the lack of publicly available datasets constraints the CNV analysis. To address these challenges, this paper constructs the first publicly accessible CNV dataset (CNVSeg), and proposes a novel multilateral graph convolutional interaction-enhanced CNV segmentation network (MTG-Net). This network integrates both region and vessel morphological information, exploring semantic and geometric duality constraints within the graph domain. Specifically, MTG-Net consists of a multi-task framework and two graph-based cross-task modules: Multilateral Interaction Graph Reasoning (MIGR) and Multilateral Reinforcement Graph Reasoning (MRGR). The multi-task framework encodes rich geometric features of lesion shapes and surfaces, decoupling the image into three task-specific feature maps. MIGR and MRGR iteratively reason about higher-order relationships across tasks through a graph mechanism, enabling complementary optimization for task-specific objectives. Additionally, an uncertainty-weighted loss is proposed to mitigate the impact of artifacts and noise on segmentation accuracy. Experimental results demonstrate that MTG-Net outperforms existing methods, achieving a Dice socre of 87.21\% for region segmentation and 88.12\% for vessel segmentation.
• 摘要：脉络膜新血管形成（CNV）是湿年龄相关的黄斑变性（湿AMD）的主要特征，代表了全球失明的主要原因。在临床实践中，光学相干断层扫描（OCTA）通常用于研究与CNV相关的病理变化，这是由于其微分级别的分辨率和非侵入性性质。因此，八八颗图像中CNV区域和血管的准确分割对于湿AMD的临床评估至关重要。然而，由于不规则的CNV形状和成像局限性（例如投影伪像，噪音和边界模糊）而存在挑战。此外，缺乏公开可用的数据集限制了CNV分析。为了应对这些挑战，本文构建了第一个可公开访问的CNV数据集（CNVSEG），并提出了一种新型的多边图卷积卷积增强的CNV分割网络（MTG-NET）。该网络同时集成了区域和血管形态信息，探索了图域内的语义和几何二元性约束。具体而言，MTG-NET由一个多任务框架和两个基于图的跨任务模块组成：多边交互图形推理（迁移）和多边增强图形推理（MRGR）。多任务框架编码病变形状和表面的丰富几何特征，将图像分解为三个特定于任务的特征图。迁移和MRGR迭代的原因是通过图形机制跨任务之间的高阶关系，从而可以针对特定于任务的目标进行互补优化。此外，提出了不确定性加权的损失，以减轻伪影和噪声对分割精度的影响。实验结果表明，MTG-NET的表现优于现有方法，对于区域分割的骰子SOCRE的骰子为87.21 \％，容器分割达到88.12 \％。

Title: Convergence of Deterministic and Stochastic Diffusion-Model Samplers: A Simple Analysis in Wasserstein Distance

• Authors: Eliot Beyler (SIERRA), Francis Bach (SIERRA)
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2508.03210
• Pdf URL: https://arxiv.org/pdf/2508.03210
• Copy Paste: [[2508.03210]] Convergence of Deterministic and Stochastic Diffusion-Model Samplers: A Simple Analysis in Wasserstein Distance(https://arxiv.org/abs/2508.03210)
• Keywords: generative
• Abstract: We provide new convergence guarantees in Wasserstein distance for diffusion-based generative models, covering both stochastic (DDPM-like) and deterministic (DDIM-like) sampling methods. We introduce a simple framework to analyze discretization, initialization, and score estimation errors. Notably, we derive the first Wasserstein convergence bound for the Heun sampler and improve existing results for the Euler sampler of the probability flow ODE. Our analysis emphasizes the importance of spatial regularity of the learned score function and argues for controlling the score error with respect to the true reverse process, in line with denoising score matching. We also incorporate recent results on smoothed Wasserstein distances to sharpen initialization error bounds.
• 摘要：我们为基于扩散的生成模型的Wasserstein距离提供了新的收敛保证，涵盖了随机（DDPM样）和确定性（DDIM样）采样方法。我们引入了一个简单的框架来分析离散，初始化和得分估计错误。值得注意的是，我们得出了针对Heun采样器的第一个Wasserstein收敛性，并改善了概率流动的Euler采样器的现有结果。我们的分析强调了学习分数函数的空间规律性的重要性，并认为相对于真实的反向过程控制得分误差，这与DeNoing的分数匹配相一致。我们还将最新的结果纳入了平滑的Wasserstein距离，以增强初始化误差边界。

Title: The Power of Many: Synergistic Unification of Diverse Augmentations for Efficient Adversarial Robustness

• Authors: Wang Yu-Hang, Shiwei Li, Jianxiang Liao, Li Bohan, Jian Liu, Wenfei Yin
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.03213
• Pdf URL: https://arxiv.org/pdf/2508.03213
• Copy Paste: [[2508.03213]] The Power of Many: Synergistic Unification of Diverse Augmentations for Efficient Adversarial Robustness(https://arxiv.org/abs/2508.03213)
• Keywords: generation
• Abstract: Adversarial perturbations pose a significant threat to deep learning models. Adversarial Training (AT), the predominant defense method, faces challenges of high computational costs and a degradation in standard performance. While data augmentation offers an alternative path, existing techniques either yield limited robustness gains or incur substantial training overhead. Therefore, developing a defense mechanism that is both highly efficient and strongly robust is of paramount this http URL this work, we first conduct a systematic analysis of existing augmentation techniques, revealing that the synergy among diverse strategies -- rather than any single method -- is crucial for enhancing robustness. Based on this insight, we propose the Universal Adversarial Augmenter (UAA) framework, which is characterized by its plug-and-play nature and training efficiency. UAA decouples the expensive perturbation generation process from model training by pre-computing a universal transformation offline, which is then used to efficiently generate unique adversarial perturbations for each sample during this http URL experiments conducted on multiple benchmarks validate the effectiveness of UAA. The results demonstrate that UAA establishes a new state-of-the-art (SOTA) for data-augmentation-based adversarial defense strategies , without requiring the online generation of adversarial examples during training. This framework provides a practical and efficient pathway for building robust models,Our code is available in the supplementary materials.
• 摘要：对抗性扰动对深度学习模型构成了重大威胁。主要的防御方法对抗训练（AT）面临高计算成本的挑战和标准绩效的退化。尽管数据增强提供了替代路径，但现有技术要么产生有限的鲁棒性增长，要么会导致大量培训开销。因此，开发一种既高效又强大的防御机制，这是最重要的这项工作，我们首先对现有增强技术进行系统分析，表明各种策略之间的协同作用 - 而不是任何单一方法 - 对于增强鲁棒性至关重要。基于这种见识，我们提出了通用对抗增强器（UAA）框架，该框架的特征是其插件的性质和训练效率。 UAA通过预先计算脱机的通用转换来将昂贵的扰动生成过程与模型培训相融合，然后在此HTTP URL实验上对每个样本进行了有效产生独特的对抗扰动，以验证多个基准测试的效果。结果表明，UAA建立了一个新的最先进的（SOTA），用于基于数据启发的对抗防御策略，而无需在培训期间在线生成对抗性示例。该框架为构建强大模型提供了一种实用，有效的途径，我们的代码可在补充材料中找到。

Title: FFHQ-Makeup: Paired Synthetic Makeup Dataset with Facial Consistency Across Multiple Styles

• Authors: Xingchao Yang, Shiori Ueda, Yuantian Huang, Tomoya Akiyama, Takafumi Taketomi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03241
• Pdf URL: https://arxiv.org/pdf/2508.03241
• Copy Paste: [[2508.03241]] FFHQ-Makeup: Paired Synthetic Makeup Dataset with Facial Consistency Across Multiple Styles(https://arxiv.org/abs/2508.03241)
• Keywords: generation
• Abstract: Paired bare-makeup facial images are essential for a wide range of beauty-related tasks, such as virtual try-on, facial privacy protection, and facial aesthetics analysis. However, collecting high-quality paired makeup datasets remains a significant challenge. Real-world data acquisition is constrained by the difficulty of collecting large-scale paired images, while existing synthetic approaches often suffer from limited realism or inconsistencies between bare and makeup images. Current synthetic methods typically fall into two categories: warping-based transformations, which often distort facial geometry and compromise the precision of makeup; and text-to-image generation, which tends to alter facial identity and expression, undermining consistency. In this work, we present FFHQ-Makeup, a high-quality synthetic makeup dataset that pairs each identity with multiple makeup styles while preserving facial consistency in both identity and expression. Built upon the diverse FFHQ dataset, our pipeline transfers real-world makeup styles from existing datasets onto 18K identities by introducing an improved makeup transfer method that disentangles identity and makeup. Each identity is paired with 5 different makeup styles, resulting in a total of 90K high-quality bare-makeup image pairs. To the best of our knowledge, this is the first work that focuses specifically on constructing a makeup dataset. We hope that FFHQ-Makeup fills the gap of lacking high-quality bare-makeup paired datasets and serves as a valuable resource for future research in beauty-related tasks.
• 摘要：配对的裸露面部图像对于多种与美容有关的任务（例如虚拟试验，面部隐私保护和面部美学分析）至关重要。但是，收集高质量的配对化妆数据集仍然是一个重大挑战。现实世界的数据采集受到收集大规模配对图像的困难的限制，而现有的合成方法通常会遭受裸露和化妆图像之间的现实主义或不一致之处。当前的合成方法通常分为两类：基于翘曲的转换，通常会扭曲面部几何形状并妥协化妆的精度；和文本对图像生成，倾向于改变面部身份和表达，破坏一致性。在这项工作中，我们提出了FFHQ Makeup，这是一种高质量的合成化妆数据集，该数据集将每个身份与多种化妆样式配对，同时保持身份和表达的面部一致性。我们的管道建立在不同的FFHQ数据集的基础上，通过引入一种改进的化妆传输方法，将现有数据集从现有数据集中转移到18K身份。每个身份都与5种不同的化妆样式配对，从而导致90k高质量的裸露图像对。据我们所知，这是第一项专门用于构建化妆数据集的工作。我们希望FFHQ制作能够填补缺乏高质量的裸露制作配对数据集的空白，并为与美容相关的任务进行未来研究提供了宝贵的资源。

Title: Ultralight Polarity-Split Neuromorphic SNN for Event-Stream Super-Resolution

• Authors: Chuanzhi Xu, Haoxian Zhou, Langyi Chen, Yuk Ying Chung, Qiang Qu
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.03244
• Pdf URL: https://arxiv.org/pdf/2508.03244
• Copy Paste: [[2508.03244]] Ultralight Polarity-Split Neuromorphic SNN for Event-Stream Super-Resolution(https://arxiv.org/abs/2508.03244)
• Keywords: super-resolution
• Abstract: Event cameras offer unparalleled advantages such as high temporal resolution, low latency, and high dynamic range. However, their limited spatial resolution poses challenges for fine-grained perception tasks. In this work, we propose an ultra-lightweight, stream-based event-to-event super-resolution method based on Spiking Neural Networks (SNNs), designed for real-time deployment on resource-constrained devices. To further reduce model size, we introduce a novel Dual-Forward Polarity-Split Event Encoding strategy that decouples positive and negative events into separate forward paths through a shared SNN. Furthermore, we propose a Learnable Spatio-temporal Polarity-aware Loss (LearnSTPLoss) that adaptively balances temporal, spatial, and polarity consistency using learnable uncertainty-based weights. Experimental results demonstrate that our method achieves competitive super-resolution performance on multiple datasets while significantly reducing model size and inference time. The lightweight design enables embedding the module into event cameras or using it as an efficient front-end preprocessing for downstream vision tasks.
• 摘要：事件摄像机具有无与伦比的优势，例如高时间分辨率，低潜伏期和高动态范围。但是，他们有限的空间分辨率为细粒度的感知任务带来了挑战。在这项工作中，我们提出了一种基于尖峰神经网络（SNNS）的超轻质，基于流的事件到事件超分辨率方法，该方法旨在在资源受限设备上实时部署。为了进一步降低模型大小，我们引入了一种新型的双向极性分解事件编码策略，该事件编码策略，将正极事件和负面事件通过共享的SNN分离为单独的前向路径。此外，我们提出了一种可学习的时空极性感知损失（Learnstploss），该损失（Learnstploss）使用可学习的基于不确定性的权重适应时间，空间和极性一致性。实验结果表明，我们的方法在多个数据集上实现了具有竞争力的超分辨率性能，同时显着降低了模型大小和推理时间。轻巧的设计使该模块可以将模块嵌入事件摄像机中，或将其用作有效的前端预处理，以实现下游视觉任务。

Title: V.I.P. : Iterative Online Preference Distillation for Efficient Video Diffusion Models

• Authors: Jisoo Kim, Wooseok Seo, Junwan Kim, Seungho Park, Sooyeon Park, Youngjae Yu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.03254
• Pdf URL: https://arxiv.org/pdf/2508.03254
• Copy Paste: [[2508.03254]] V.I.P. : Iterative Online Preference Distillation for Efficient Video Diffusion Models(https://arxiv.org/abs/2508.03254)
• Keywords: generation
• Abstract: With growing interest in deploying text-to-video (T2V) models in resource-constrained environments, reducing their high computational cost has become crucial, leading to extensive research on pruning and knowledge distillation methods while maintaining performance. However, existing distillation methods primarily rely on supervised fine-tuning (SFT), which often leads to mode collapse as pruned models with reduced capacity fail to directly match the teacher's outputs, ultimately resulting in degraded quality. To address this challenge, we propose an effective distillation method, ReDPO, that integrates DPO and SFT. Our approach leverages DPO to guide the student model to focus on recovering only the targeted properties, rather than passively imitating the teacher, while also utilizing SFT to enhance overall performance. We additionally propose V.I.P., a novel framework for filtering and curating high-quality pair datasets, along with a step-by-step online approach for calibrated training. We validate our method on two leading T2V models, VideoCrafter2 and AnimateDiff, achieving parameter reduction of 36.2% and 67.5% each, while maintaining or even surpassing the performance of full models. Further experiments demonstrate the effectiveness of both ReDPO and V.I.P. framework in enabling efficient and high-quality video generation. Our code and videos are available at this https URL.
• 摘要：随着对在资源受限环境中部署文本到视频（T2V）模型的兴趣越来越多，降低其高计算成本已经变得至关重要，从而在维持性能的同时，对修剪和知识蒸馏方法进行了广泛的研究。但是，现有的蒸馏方法主要依赖于监督的微调（SFT），这通常会导致模式崩溃，因为修剪的模型具有降低的容量，无法直接匹配教师的产量，最终导致质量退化。为了应对这一挑战，我们提出了一种有效的蒸馏方法RedPo，该方法将DPO和SFT整合在一起。我们的方法利用DPO指导学生模型专注于仅恢复目标特性，而不是被动地模仿教师，同时还利用SFT来提高整体表现。我们另外提出了V.I.P.，这是一个用于过滤和策划高质量对数据集的新型框架，以及用于校准培训的逐步在线方法。我们在两种领先的T2V型号（VideoCrafter2和Animatediff）上验证了我们的方法，在维持甚至超过完整型号的性能的同时，各种参数减少了36.2％和67.5％。进一步的实验证明了Redpo和V.I.P.的有效性。实现高效和高质量视频的框架。我们的代码和视频可在此HTTPS URL上找到。

Title: Beyond Isolated Words: Diffusion Brush for Handwritten Text-Line Generation

• Authors: Gang Dai, Yifan Zhang, Yutao Qin, Qiangya Guo, Shuangping Huang, Shuicheng Yan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03256
• Pdf URL: https://arxiv.org/pdf/2508.03256
• Copy Paste: [[2508.03256]] Beyond Isolated Words: Diffusion Brush for Handwritten Text-Line Generation(https://arxiv.org/abs/2508.03256)
• Keywords: generation
• Abstract: Existing handwritten text generation methods primarily focus on isolated words. However, realistic handwritten text demands attention not only to individual words but also to the relationships between them, such as vertical alignment and horizontal spacing. Therefore, generating entire text lines emerges as a more promising and comprehensive task. However, this task poses significant challenges, including the accurate modeling of complex style patterns encompassing both intra- and inter-word relationships, and maintaining content accuracy across numerous characters. To address these challenges, we propose DiffBrush, a novel diffusion-based model for handwritten text-line generation. Unlike existing methods, DiffBrush excels in both style imitation and content accuracy through two key strategies: (1) content-decoupled style learning, which disentangles style from content to better capture intra-word and inter-word style patterns by using column- and row-wise masking; and (2) multi-scale content learning, which employs line and word discriminators to ensure global coherence and local accuracy of textual content. Extensive experiments show that DiffBrush excels in generating high-quality text lines, particularly in style reproduction and content preservation. Code is available at this https URL.
• 摘要：现有的手写文本生成方法主要集中于孤立的单词。但是，现实的手写文字不仅需要注意单词，还需要对它们之间的关系，例如垂直对齐和水平间距。因此，生成整个文本行是一项更有希望和全面的任务。但是，此任务提出了重大挑战，包括对涵盖内部和词间关系的复杂样式模式的准确建模，以及在众多字符上保持内容准确性。为了应对这些挑战，我们提出了Diffrush，这是一种基于手写文本线生成的新型基于扩散的模型。与现有方法不同，Diffrush通过两种关键策略都以样式模仿和内容精度出色：（1）使用列和行遮盖式的内容，从内容中解散了内容的样式，从而将风格从内容中删除，从而更好地捕获词内和界面样式的模式； （2）多尺度内容学习，它采用线条和单词歧视器来确保文本内容的全局连贯性和局部准确性。广泛的实验表明，Diffrush在生成高质量的文本线条方面表现出色，尤其是在样式复制和内容保存方面。代码可在此HTTPS URL上找到。

Title: Zero Shot Domain Adaptive Semantic Segmentation by Synthetic Data Generation and Progressive Adaptation

• Authors: Jun Luo, Zijing Zhao, Yang Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03300
• Pdf URL: https://arxiv.org/pdf/2508.03300
• Copy Paste: [[2508.03300]] Zero Shot Domain Adaptive Semantic Segmentation by Synthetic Data Generation and Progressive Adaptation(https://arxiv.org/abs/2508.03300)
• Keywords: generation
• Abstract: Deep learning-based semantic segmentation models achieve impressive results yet remain limited in handling distribution shifts between training and test data. In this paper, we present SDGPA (Synthetic Data Generation and Progressive Adaptation), a novel method that tackles zero-shot domain adaptive semantic segmentation, in which no target images are available, but only a text description of the target domain's style is provided. To compensate for the lack of target domain training data, we utilize a pretrained off-the-shelf text-to-image diffusion model, which generates training images by transferring source domain images to target style. Directly editing source domain images introduces noise that harms segmentation because the layout of source images cannot be precisely maintained. To address inaccurate layouts in synthetic data, we propose a method that crops the source image, edits small patches individually, and then merges them back together, which helps improve spatial precision. Recognizing the large domain gap, SDGPA constructs an augmented intermediate domain, leveraging easier adaptation subtasks to enable more stable model adaptation to the target domain. Additionally, to mitigate the impact of noise in synthetic data, we design a progressive adaptation strategy, ensuring robust learning throughout the training process. Extensive experiments demonstrate that our method achieves state-of-the-art performance in zero-shot semantic segmentation. The code is available at this https URL
• 摘要：基于深度学习的语义细分模型取得了令人印象深刻的结果，但在处理训练和测试数据之间的分配变化方面仍然有限。在本文中，我们介绍了SDGPA（合成数据生成和渐进式适应性），这是一种可解决零射击域自适应语义分割的新方法，其中无目标图像可用，但只提供了目标域样式的文本描述。为了弥补缺乏目标域训练数据，我们使用了预式现成的文本到图像扩散模型，该模型通过将源域图像传输到目标样式来生成训练图像。直接编辑源域图像引入了危害分割的噪声，因为无法精确维护源图像的布局。为了解决合成数据中的不准确布局，我们提出了一种方法来撰写源图像，单独编辑小斑块，然后将它们合并为重新合并，这有助于提高空间精度。识别较大的域间隙，SDGPA构建了增强的中间域，利用更轻松的适应子任务来使更稳定的模型适应目标域。此外，为了减轻噪声在综合数据中的影响，我们设计了一种渐进式适应策略，确保在整个训练过程中进行健全的学习。广泛的实验表明，我们的方法在零射击语义分段中实现了最先进的性能。该代码可在此HTTPS URL上找到

Title: Skywork UniPic: Unified Autoregressive Modeling for Visual Understanding and Generation

• Authors: Peiyu Wang, Yi Peng, Yimeng Gan, Liang Hu, Tianyidan Xie, Xiaokun Wang, Yichen Wei, Chuanxin Tang, Bo Zhu, Changshi Li, Hongyang Wei, Eric Li, Xuchen Song, Yang Liu, Yahui Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03320
• Pdf URL: https://arxiv.org/pdf/2508.03320
• Copy Paste: [[2508.03320]] Skywork UniPic: Unified Autoregressive Modeling for Visual Understanding and Generation(https://arxiv.org/abs/2508.03320)
• Keywords: generation
• Abstract: We introduce Skywork UniPic, a 1.5 billion-parameter autoregressive model that unifies image understanding, text-to-image generation, and image editing within a single architecture-eliminating the need for task-specific adapters or inter-module connectors-and demonstrate that compact multimodal systems can achieve state-of-the-art performance on commodity hardware. Skywork UniPic achieves a GenEval score of 0.86, surpassing most existing unified models; sets a new DPG-Bench complex-generation record of 85.5; attains 5.83 on GEditBench-EN and 3.49 on ImgEdit-Bench for image editing; and generates 1024 x 1024 images with under 15 GB of GPU memory (e.g., RTX 4090). (1) a decoupled encoding strategy that leverages a masked autoregressive encoder for synthesis and a SigLIP2 encoder for understanding, all feeding a shared autoregressive decoder; (2) a progressive, resolution-aware training schedule scaling from 256 x 256 to 1024 x 1024 while dynamically unfreezing parameters to balance capacity and stability; and (3) meticulously curated, 100 million-scale datasets augmented with task-specific reward models to refine generation and editing objectives. By demonstrating that high-fidelity multimodal integration need not incur prohibitive resource demands, Skywork UniPic establishes a practical paradigm for deployable, high-fidelity multimodal AI. Code and weights are publicly available at this https URL.
• 摘要：我们介绍了Skywork Unipic，这是一种15亿参数自回归模型，在单个体系结构中统一图像理解，文本到图像的生成以及图像编辑，从而在单个体系结构中进行了编辑，从而阐明了对任务特异性适配器或模块间连接器的需求，并证明了紧凑的多模态系统可以实现在商品硬件上的确切性能。 Skywork Unipic达到了0.86的遗传评分，超过了大多数现有的统一模型。设置了新的DPG基础复合生成记录85.5；在Geditbench-en上达到5.83，在Imgedit板凳上获得图像编辑的3.49；并生成1024 x 1024图像，具有15 GB的GPU存储器（例如RTX 4090）。 （1）一种脱钩的编码策略，该策略利用掩盖的自回归编码器进行合成和siglip2编码器来理解，所有这些都可以喂养共享的自动回归解码器； （2）渐进的，分辨率感知的培训时间表从256 x 256到1024 x 1024，而动态释放参数以平衡容量和稳定性； （3）精心策划的，1亿个规模的数据集增强了特定于任务的奖励模型，以完善生成和编辑目标。通过证明高保真多模式整合不需要产生过度的资源需求，Skywork Unipic为可部署的高保真多模式AI建立了实用的范式。该HTTPS URL公开可用代码和权重。

Title: Macro-from-Micro Planning for High-Quality and Parallelized Autoregressive Long Video Generation

• Authors: Xunzhi Xiang, Yabo Chen, Guiyu Zhang, Zhongyu Wang, Zhe Gao, Quanming Xiang, Gonghu Shang, Junqi Liu, Haibin Huang, Yang Gao, Chi Zhang, Qi Fan, Xuelong Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03334
• Pdf URL: https://arxiv.org/pdf/2508.03334
• Copy Paste: [[2508.03334]] Macro-from-Micro Planning for High-Quality and Parallelized Autoregressive Long Video Generation(https://arxiv.org/abs/2508.03334)
• Keywords: generation
• Abstract: Current autoregressive diffusion models excel at video generation but are generally limited to short temporal durations. Our theoretical analysis indicates that the autoregressive modeling typically suffers from temporal drift caused by error accumulation and hinders parallelization in long video synthesis. To address these limitations, we propose a novel planning-then-populating framework centered on Macro-from-Micro Planning (MMPL) for long video generation. MMPL sketches a global storyline for the entire video through two hierarchical stages: Micro Planning and Macro Planning. Specifically, Micro Planning predicts a sparse set of future keyframes within each short video segment, offering motion and appearance priors to guide high-quality video segment generation. Macro Planning extends the in-segment keyframes planning across the entire video through an autoregressive chain of micro plans, ensuring long-term consistency across video segments. Subsequently, MMPL-based Content Populating generates all intermediate frames in parallel across segments, enabling efficient parallelization of autoregressive generation. The parallelization is further optimized by Adaptive Workload Scheduling for balanced GPU execution and accelerated autoregressive video generation. Extensive experiments confirm that our method outperforms existing long video generation models in quality and stability. Generated videos and comparison results are in our project page.
• 摘要：当前的自回归扩散模型在视频生成时表现出色，但通常仅限于短时持续时间。我们的理论分析表明，自回归建模通常受到误差积累引起的时间漂移，并阻碍长时间视频合成中的并行化。为了解决这些局限性，我们提出了一个新颖的计划，然后以长期视频生成为中心以宏观 - 薄膜计划（MMPL）为中心。 MMPL通过两个层次结构阶段为整个视频绘制一个全球故事情节：微型计划和宏观计划。具体而言，微型计划预测每个简短视频段中的一组未来的关键帧集，提供运动和外观先验，以指导高质量的视频段生成。宏规划通过自回归的微型计划链扩展了整个视频的段内框架计划，从而确保了整个视频段的长期一致性。随后，基于MMPL的内容填充物在整个细分市场中并行生成所有中间帧，从而有效地平行自回归产生。通过自适应工作负载调度进行平衡的GPU执行和加速自动回归视频的生成，可以进一步优化并行化。广泛的实验证实，我们的方法在质量和稳定性方面的表现优于现有的长视频生成模型。生成的视频和比较结果在我们的项目页面中。

Title: Beyond Illumination: Fine-Grained Detail Preservation in Extreme Dark Image Restoration

• Authors: Tongshun Zhang, Pingping Liu, Zixuan Zhong, Zijian Zhang, Qiuzhan Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03336
• Pdf URL: https://arxiv.org/pdf/2508.03336
• Copy Paste: [[2508.03336]] Beyond Illumination: Fine-Grained Detail Preservation in Extreme Dark Image Restoration(https://arxiv.org/abs/2508.03336)
• Keywords: restoration
• Abstract: Recovering fine-grained details in extremely dark images remains challenging due to severe structural information loss and noise corruption. Existing enhancement methods often fail to preserve intricate details and sharp edges, limiting their effectiveness in downstream applications like text and edge detection. To address these deficiencies, we propose an efficient dual-stage approach centered on detail recovery for dark images. In the first stage, we introduce a Residual Fourier-Guided Module (RFGM) that effectively restores global illumination in the frequency domain. RFGM captures inter-stage and inter-channel dependencies through residual connections, providing robust priors for high-fidelity frequency processing while mitigating error accumulation risks from unreliable priors. The second stage employs complementary Mamba modules specifically designed for textural structure refinement: (1) Patch Mamba operates on channel-concatenated non-downsampled patches, meticulously modeling pixel-level correlations to enhance fine-grained details without resolution loss. (2) Grad Mamba explicitly focuses on high-gradient regions, alleviating state decay in state space models and prioritizing reconstruction of sharp edges and boundaries. Extensive experiments on multiple benchmark datasets and downstream applications demonstrate that our method significantly improves detail recovery performance while maintaining efficiency. Crucially, the proposed modules are lightweight and can be seamlessly integrated into existing Fourier-based frameworks with minimal computational overhead. Code is available at this https URL.
• 摘要：由于严重的结构信息丢失和噪音腐败，在极度黑暗的图像中恢复细粒度的细节仍然具有挑战性。现有的增强方法通常无法保留复杂的细节和尖锐的边缘，从而限制了它们在文本和边缘检测等下游应用中的有效性。为了解决这些缺陷，我们提出了一种有效的双阶段方法，该方法以深色图像的细节恢复为中心。在第一阶段，我们引入了一个残留的傅立叶引导模块（RFGM），该模块有效地恢复了频域中的全局照明。 RFGM通过剩余连接捕获阶段间和通道间的依赖性，为高保真频率处理提供了强大的先验，同时减轻了不可靠的先验的错误积累风险。第二阶段采用专门设计用于纹理结构改进的互补MAMBA模块：（1）贴片Mamba在通道焦糖的非下降贴片上运行，精心建模像素级相关性，以增强细粒度的细节而不分辨率损失。 （2）Grad Mamba明确专注于高梯度区域，减轻国家空间模型中的状态衰变，并优先重建锋利的边缘和边界。在多个基准数据集和下游应用程序上进行的大量实验表明，我们的方法在维持效率的同时显着提高了详细的恢复性能。至关重要的是，所提出的模块是轻巧的，可以无缝地集成到具有最小计算开销的现有基于傅立叶的框架中。代码可在此HTTPS URL上找到。

Title: Diffusion Once and Done: Degradation-Aware LoRA for Efficient All-in-One Image Restoration

• Authors: Ni Tang, Xiaotong Luo, Zihan Cheng, Liangtai Zhou, Dongxiao Zhang, Yanyun Qu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03373
• Pdf URL: https://arxiv.org/pdf/2508.03373
• Copy Paste: [[2508.03373]] Diffusion Once and Done: Degradation-Aware LoRA for Efficient All-in-One Image Restoration(https://arxiv.org/abs/2508.03373)
• Keywords: restoration
• Abstract: Diffusion models have revealed powerful potential in all-in-one image restoration (AiOIR), which is talented in generating abundant texture details. The existing AiOIR methods either retrain a diffusion model or fine-tune the pretrained diffusion model with extra conditional guidance. However, they often suffer from high inference costs and limited adaptability to diverse degradation types. In this paper, we propose an efficient AiOIR method, Diffusion Once and Done (DOD), which aims to achieve superior restoration performance with only one-step sampling of Stable Diffusion (SD) models. Specifically, multi-degradation feature modulation is first introduced to capture different degradation prompts with a pretrained diffusion model. Then, parameter-efficient conditional low-rank adaptation integrates the prompts to enable the fine-tuning of the SD model for adapting to different degradation types. Besides, a high-fidelity detail enhancement module is integrated into the decoder of SD to improve structural and textural details. Experiments demonstrate that our method outperforms existing diffusion-based restoration approaches in both visual quality and inference efficiency.
• 摘要：扩散模型已经揭示了多合一图像修复（Aioir）的强大潜力，该图像恢复（Aioir）在产生丰富的纹理细节方面很有才华。现有的AIOIR方法可以通过额外的条件指导来重新训练扩散模型或微调预验扩散模型。但是，他们通常会遭受高推理成本和对各种降解类型的适应性的有限。在本文中，我们提出了一种有效的AIOIR方法，即一旦完成（DOD），旨在仅使用稳定扩散（SD）模型的一步采样来实现出色的恢复性能。具体而言，首先引入多降解功能调制，以通过预审预定的扩散模型捕获不同的降解提示。然后，参数效率高的条件低级适应性集成了提示，以使SD模型的微调适应不同的降解类型。此外，将高保真细节增强模块集成到SD的解码器中，以改善结构和纹理细节。实验表明，我们的方法在视觉质量和推理效率方面均优于现有基于扩散的恢复方法。

Title: SCFlow: Implicitly Learning Style and Content Disentanglement with Flow Models

• Authors: Pingchuan Ma, Xiaopei Yang, Yusong Li, Ming Gui, Felix Krause, Johannes Schusterbauer, Björn Ommer
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.03402
• Pdf URL: https://arxiv.org/pdf/2508.03402
• Copy Paste: [[2508.03402]] SCFlow: Implicitly Learning Style and Content Disentanglement with Flow Models(https://arxiv.org/abs/2508.03402)
• Keywords: generation, generative
• Abstract: Explicitly disentangling style and content in vision models remains challenging due to their semantic overlap and the subjectivity of human perception. Existing methods propose separation through generative or discriminative objectives, but they still face the inherent ambiguity of disentangling intertwined concepts. Instead, we ask: Can we bypass explicit disentanglement by learning to merge style and content invertibly, allowing separation to emerge naturally? We propose SCFlow, a flow-matching framework that learns bidirectional mappings between entangled and disentangled representations. Our approach is built upon three key insights: 1) Training solely to merge style and content, a well-defined task, enables invertible disentanglement without explicit supervision; 2) flow matching bridges on arbitrary distributions, avoiding the restrictive Gaussian priors of diffusion models and normalizing flows; and 3) a synthetic dataset of 510,000 samples (51 styles $\times$ 10,000 content samples) was curated to simulate disentanglement through systematic style-content pairing. Beyond controllable generation tasks, we demonstrate that SCFlow generalizes to ImageNet-1k and WikiArt in zero-shot settings and achieves competitive performance, highlighting that disentanglement naturally emerges from the invertible merging process.
• 摘要：由于其语义重叠和人类感知的主观性，视觉模型中明确散布的风格和内容仍然具有挑战性。现有方法通过生成或歧视目标提出分离，但它们仍然面临着解开交织在一起的概念的固有歧义。取而代之的是，我们问：我们能否通过学习使风格和内容不可逆转地合并，使分离自然而然地绕开明确的分离？我们提出了SCFLOW，这是一个流动匹配框架，可以学习纠缠和删除表示形式之间的双向映射。我们的方法是基于三个关键见解的：1）培训仅融合样式和内容，定义明确的任务，可以在不明确监督的情况下进行可逆的解开； 2）在任意分布上的流动匹配桥，避免了扩散模型的限制性高斯先验和归一化流； 3）策划了510,000个样品（51次$ \ times $ 10,000内容样本）的合成数据集，以通过系统的风格配对来模拟分离。除了可控的生成任务之外，我们还证明了SCFLOF在零拍设置中概括为ImageNet-1K和Wikiart，并实现了竞争性能，强调了脱节是自然而然地从可逆的合并过程中出现的。

Title: Learning Latent Representations for Image Translation using Frequency Distributed CycleGAN

• Authors: Shivangi Nigam, Adarsh Prasad Behera, Shekhar Verma, P. Nagabhushan
• Subjects: cs.CV, cs.AI, cs.GR
• Abstract URL: https://arxiv.org/abs/2508.03415
• Pdf URL: https://arxiv.org/pdf/2508.03415
• Copy Paste: [[2508.03415]] Learning Latent Representations for Image Translation using Frequency Distributed CycleGAN(https://arxiv.org/abs/2508.03415)
• Keywords: restoration, generative
• Abstract: This paper presents Fd-CycleGAN, an image-to-image (I2I) translation framework that enhances latent representation learning to approximate real data distributions. Building upon the foundation of CycleGAN, our approach integrates Local Neighborhood Encoding (LNE) and frequency-aware supervision to capture fine-grained local pixel semantics while preserving structural coherence from the source domain. We employ distribution-based loss metrics, including KL/JS divergence and log-based similarity measures, to explicitly quantify the alignment between real and generated image distributions in both spatial and frequency domains. To validate the efficacy of Fd-CycleGAN, we conduct experiments on diverse datasets -- Horse2Zebra, Monet2Photo, and a synthetically augmented Strike-off dataset. Compared to baseline CycleGAN and other state-of-the-art methods, our approach demonstrates superior perceptual quality, faster convergence, and improved mode diversity, particularly in low-data regimes. By effectively capturing local and global distribution characteristics, Fd-CycleGAN achieves more visually coherent and semantically consistent translations. Our results suggest that frequency-guided latent learning significantly improves generalization in image translation tasks, with promising applications in document restoration, artistic style transfer, and medical image synthesis. We also provide comparative insights with diffusion-based generative models, highlighting the advantages of our lightweight adversarial approach in terms of training efficiency and qualitative output.
• 摘要：本文介绍了FD-Cyclegan，这是一个图像到图像图像（I2i）翻译框架，该框架增强了潜在表示学习以近似实际数据分布。我们的方法以Cyclegan的基础为基础，整合了本地邻域编码（LNE）和频率了解的监督，以捕获细粒度的本地像素语义，同时从源域中保留结构连贯性。我们采用基于分布的损失指标，包括KL/JS差异和基于日志的相似性度量，以明确量化空间和频域中的真实图像分布和生成的图像分布之间的比对。为了验证FD-Cyclegan的功效，我们在不同数据集上进行实验-Horse2zebra，Monet2Photo和合成增强的罢工数据集。与基线自行车和其他最先进的方法相比，我们的方法表明了卓越的感知质量，更快的收敛性和改进的模式多样性，尤其是在低数据级方案中。通过有效捕获局部和全球分布特征，FD-Cyclegan实现了更具视觉连贯和语义一致的翻译。我们的结果表明，频率引导的潜在学习可显着改善图像翻译任务的概括，并在文档恢复，艺术风格转移和医学图像综合方面具有有希望的应用。我们还提供了具有基于扩散的生成模型的比较见解，从而在训练效率和定性产出方面强调了我们轻量级对抗方法的优势。

Title: R2GenKG: Hierarchical Multi-modal Knowledge Graph for LLM-based Radiology Report Generation

• Authors: Futian Wang, Yuhan Qiao, Xiao Wang, Fuling Wang, Yuxiang Zhang, Dengdi Sun
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.03426
• Pdf URL: https://arxiv.org/pdf/2508.03426
• Copy Paste: [[2508.03426]] R2GenKG: Hierarchical Multi-modal Knowledge Graph for LLM-based Radiology Report Generation(https://arxiv.org/abs/2508.03426)
• Keywords: generation
• Abstract: X-ray medical report generation is one of the important applications of artificial intelligence in healthcare. With the support of large foundation models, the quality of medical report generation has significantly improved. However, challenges such as hallucination and weak disease diagnostic capability still persist. In this paper, we first construct a large-scale multi-modal medical knowledge graph (termed M3KG) based on the ground truth medical report using the GPT-4o. It contains 2477 entities, 3 kinds of relations, 37424 triples, and 6943 disease-aware vision tokens for the CheXpert Plus dataset. Then, we sample it to obtain multi-granularity semantic graphs and use an R-GCN encoder for feature extraction. For the input X-ray image, we adopt the Swin-Transformer to extract the vision features and interact with the knowledge using cross-attention. The vision tokens are fed into a Q-former and retrieved the disease-aware vision tokens using another cross-attention. Finally, we adopt the large language model to map the semantic knowledge graph, input X-ray image, and disease-aware vision tokens into language descriptions. Extensive experiments on multiple datasets fully validated the effectiveness of our proposed knowledge graph and X-ray report generation framework. The source code of this paper will be released on this https URL.
• 摘要：X射线医疗报告的一代是人工智能在医疗保健中的重要应用之一。在大型基础模型的支持下，医疗报告的质量已大大提高。但是，诸如幻觉和疾病诊断能力弱的挑战仍然存在。在本文中，我们首先根据使用GPT-4O的地面真相医学报告构建了一个大规模的多模式医学知识图（称为M3kg）。它包含2477个实体，3种关系，37424个三元组和6943个疾病感知的视力令牌，用于Chexpert Plus数据集。然后，我们对其进行采样以获得多粒性语义图，并使用R-GCN编码器进行特征提取。对于输入X射线图像，我们采用Swin-Transformer来提取视觉特征并使用交叉注意与知识相互作用。视力令牌被馈入Q形成剂，并使用另一种跨注意事项检索了感染视力令牌。最后，我们采用大型语言模型来绘制语义知识图，输入X射线图像以及疾病感知的视觉令牌，以绘制语言描述。多个数据集上的广泛实验完全验证了我们提出的知识图和X射线报告生成框架的有效性。本文的源代码将在此HTTPS URL上发布。

Title: RAAG: Ratio Aware Adaptive Guidance

• Authors: Shangwen Zhu, Qianyu Peng, Yuting Hu, Zhantao Yang, Han Zhang, Zhao Pu, Ruili Feng, Fan Cheng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03442
• Pdf URL: https://arxiv.org/pdf/2508.03442
• Copy Paste: [[2508.03442]] RAAG: Ratio Aware Adaptive Guidance(https://arxiv.org/abs/2508.03442)
• Keywords: generation, generative
• Abstract: Flow-based generative models have recently achieved remarkable progress in image and video synthesis, with classifier-free guidance (CFG) becoming the standard tool for high-fidelity, controllable generation. However, despite their practical success, little is known about how guidance interacts with different stages of the sampling process-especially in the fast, low-step regimes typical of modern flow-based pipelines. In this work, we uncover and analyze a fundamental instability: the earliest reverse steps are acutely sensitive to the guidance scale, owing to a pronounced spike in the relative strength (RATIO) of conditional to unconditional predictions. Through rigorous theoretical analysis and empirical validation, we show that this RATIO spike is intrinsic to the data distribution, independent of the model architecture, and causes exponential error amplification when paired with strong guidance. To address this, we propose a simple, theoretically grounded, RATIO-aware adaptive guidance schedule that automatically dampens the guidance scale at early steps based on the evolving RATIO, using a closed-form exponential decay. Our method is lightweight, requires no additional inference overhead, and is compatible with standard flow frameworks. Experiments across state-of-the-art image (SD3.5, Lumina) and video (WAN2.1) models demonstrate that our approach enables up to 3x faster sampling while maintaining or improving generation quality, robustness, and semantic alignment. Extensive ablation studies further confirm the generality and stability of our schedule across models, datasets, and hyperparameters. Our findings highlight the critical role of stepwise guidance adaptation in unlocking the full potential of fast flow-based generative models.
• 摘要：基于流量的生成模型最近在图像和视频综合方面取得了显着的进展，无分类器指导（CFG）成为高保真，可控生成的标准工具。但是，尽管他们的实践成功，但对指导如何与采样过程的不同阶段相互作用，尤其是在典型的基于现代流动的管道的快速，低步骤方案中相互作用。在这项工作中，我们发现和分析了基本不稳定性：最早的反向步骤对引导量表非常敏感，这是由于条件与无条件预测的相对强度（比率）的明显尖峰。通过严格的理论分析和经验验证，我们表明该比率尖峰是数据分布的固有的，与模型架构无关，并在与强制指导配对时会导致指数误差放大。为了解决这个问题，我们提出了一个简单的，理论上的，比率感知的自适应指导时间表，该计划使用封闭形式的指数衰减自动根据不断发展的比率自动抑制引导量表。我们的方法轻巧，不需要额外的推理开销，并且与标准流框架兼容。跨最新图像（SD3.5，Lumina）和视频（WAN2.1）模型进行的实验表明，我们的方法可实现最多3倍的采样速度，同时保持或提高发电质量，稳健性和语义一致性。广泛的消融研究进一步证实了我们跨模型，数据集和超参数的日程安排的一般性和稳定性。我们的发现突出了逐步指导适应在解锁基于快速流量的生成模型的全部潜力方面的关键作用。

Title: READ: Real-time and Efficient Asynchronous Diffusion for Audio-driven Talking Head Generation

• Authors: Haotian Wang, Yuzhe Weng, Jun Du, Haoran Xu, Xiaoyan Wu, Shan He, Bing Yin, Cong Liu, Jianqing Gao, Qingfeng Liu
• Subjects: cs.CV, cs.SD
• Abstract URL: https://arxiv.org/abs/2508.03457
• Pdf URL: https://arxiv.org/pdf/2508.03457
• Copy Paste: [[2508.03457]] READ: Real-time and Efficient Asynchronous Diffusion for Audio-driven Talking Head Generation(https://arxiv.org/abs/2508.03457)
• Keywords: generation
• Abstract: The introduction of diffusion models has brought significant advances to the field of audio-driven talking head generation. However, the extremely slow inference speed severely limits the practical implementation of diffusion-based talking head generation models. In this study, we propose READ, the first real-time diffusion-transformer-based talking head generation framework. Our approach first learns a spatiotemporal highly compressed video latent space via a temporal VAE, significantly reducing the token count to accelerate generation. To achieve better audio-visual alignment within this compressed latent space, a pre-trained Speech Autoencoder (SpeechAE) is proposed to generate temporally compressed speech latent codes corresponding to the video latent space. These latent representations are then modeled by a carefully designed Audio-to-Video Diffusion Transformer (A2V-DiT) backbone for efficient talking head synthesis. Furthermore, to ensure temporal consistency and accelerated inference in extended generation, we propose a novel asynchronous noise scheduler (ANS) for both the training and inference process of our framework. The ANS leverages asynchronous add-noise and asynchronous motion-guided generation in the latent space, ensuring consistency in generated video clips. Experimental results demonstrate that READ outperforms state-of-the-art methods by generating competitive talking head videos with significantly reduced runtime, achieving an optimal balance between quality and speed while maintaining robust metric stability in long-time generation.
• 摘要：扩散模型的引入为音频驱动的会说话的头部生成带来了重大进步。但是，极慢的推理速度严重限制了基于扩散的说话头部生成模型的实际实现。在这项研究中，我们建议阅读，这是第一个基于实时扩散转化器的说话头生成框架。我们的方法首先通过颞vae学习了时空高度压缩的视频潜在空间，从而大大降低了令牌数量以加速发电。为了在此压缩潜在空间内实现更好的视听对齐，提出了预训练的语音自动编码器（Speechae）来生成与视频潜在空间相对应的时间压缩的语音潜在代码。然后，这些潜在表示由精心设计的音频到视频扩散变压器（A2V-DIT）主链建模，以进行有效的说话头合成。此外，为了确保时间一致性并加速了扩展生成的推断，我们为我们的框架的训练和推理过程提出了一种新型异步噪声调度程序（ANS）。 ANS在潜在空间中利用异步的附加噪声和异步运动引导产生，从而确保生成的视频剪辑的一致性。实验结果表明，通过生成具有显着降低的运行时的竞争性说话的头部视频，读取质量和速度之间的最佳平衡，同时保持长期发电的稳健度量稳定性，从而超过了最先进的方法。

Title: VideoGuard: Protecting Video Content from Unauthorized Editing

• Authors: Junjie Cao, Kaizhou Li, Xinchun Yu, Hongxiang Li, Xiaoping Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.03480
• Pdf URL: https://arxiv.org/pdf/2508.03480
• Copy Paste: [[2508.03480]] VideoGuard: Protecting Video Content from Unauthorized Editing(https://arxiv.org/abs/2508.03480)
• Keywords: generative
• Abstract: With the rapid development of generative technology, current generative models can generate high-fidelity digital content and edit it in a controlled manner. However, there is a risk that malicious individuals might misuse these capabilities for misleading activities. Although existing research has attempted to shield photographic images from being manipulated by generative models, there remains a significant disparity in the protection offered to video content editing. To bridge the gap, we propose a protection method named VideoGuard, which can effectively protect videos from unauthorized malicious editing. This protection is achieved through the subtle introduction of nearly unnoticeable perturbations that interfere with the functioning of the intended generative diffusion models. Due to the redundancy between video frames, and inter-frame attention mechanism in video diffusion models, simply applying image-based protection methods separately to every video frame can not shield video from unauthorized editing. To tackle the above challenge, we adopt joint frame optimization, treating all video frames as an optimization entity. Furthermore, we extract video motion information and fuse it into optimization objectives. Thus, these alterations can effectively force the models to produce outputs that are implausible and inconsistent. We provide a pipeline to optimize this perturbation. Finally, we use both objective metrics and subjective metrics to demonstrate the efficacy of our method, and the results show that the protection performance of VideoGuard is superior to all the baseline methods.
• 摘要：随着生成技术的快速发展，当前的生成模型可以生成高保真的数字内容并以受控方式进行编辑。但是，恶意个人可能会滥用这些能力来误导活动。尽管现有的研究试图屏蔽摄影图像不受生成模型操纵，但在视频内容编辑提供的保护方面仍然存在显着差异。为了弥合差距，我们提出了一种名为Vietoguard的保护方法，该方法可以有效地保护视频免受未经授权的恶意编辑。通过微妙的引入几乎不明显的扰动来实现这种保护，这些扰动干扰了预期的生成扩散模型的功能。由于视频帧之间的冗余以及视频扩散模型中的框架相关机制，只需将基于图像的保护方法分别应用于每个视频框架，就无法屏蔽未经授权的编辑视频。为了应对上述挑战，我们采用关节框架优化，将所有视频框架视为优化实体。此外，我们提取视频运动信息并将其融合到优化目标中。因此，这些改变可以有效地迫使模型产生令人难以置信且不一致的输出。我们提供一条管道来优化这种扰动。最后，我们使用客观指标和主观指标来证明我们方法的疗效，结果表明，视频节的保护性能优于所有基线方法。

Title: Draw Your Mind: Personalized Generation via Condition-Level Modeling in Text-to-Image Diffusion Models

• Authors: Hyungjin Kim, Seokho Ahn, Young-Duk Seo
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2508.03481
• Pdf URL: https://arxiv.org/pdf/2508.03481
• Copy Paste: [[2508.03481]] Draw Your Mind: Personalized Generation via Condition-Level Modeling in Text-to-Image Diffusion Models(https://arxiv.org/abs/2508.03481)
• Keywords: generation
• Abstract: Personalized generation in T2I diffusion models aims to naturally incorporate individual user preferences into the generation process with minimal user intervention. However, existing studies primarily rely on prompt-level modeling with large-scale models, often leading to inaccurate personalization due to the limited input token capacity of T2I diffusion models. To address these limitations, we propose DrUM, a novel method that integrates user profiling with a transformer-based adapter to enable personalized generation through condition-level modeling in the latent space. DrUM demonstrates strong performance on large-scale datasets and seamlessly integrates with open-source text encoders, making it compatible with widely used foundation T2I models without requiring additional fine-tuning.
• 摘要：T2I扩散模型中的个性化生成旨在自然地将单个用户的偏好纳入生成过程中，并以最少的用户干预。但是，现有的研究主要依赖于具有大型模型的迅速建模，这通常导致由于T2I扩散模型的输入令牌能力有限，因此由于个性化不准确。为了解决这些限制，我们提出了DRUM，这是一种新颖的方法，将用户分析与基于变压器的适配器集成在一起，以通过潜在空间中的条件级建模来实现个性化生成。鼓在大规模数据集上表现出强烈的性能，并与开源文本编码器无缝集成，使其与广泛使用的基础T2I型号兼容，而无需进行其他微调。

Title: When Cars Have Stereotypes: Auditing Demographic Bias in Objects from Text-to-Image Models

• Authors: Dasol Choi Jihwan Lee, Minjae Lee, Minsuk Kahng
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.03483
• Pdf URL: https://arxiv.org/pdf/2508.03483
• Copy Paste: [[2508.03483]] When Cars Have Stereotypes: Auditing Demographic Bias in Objects from Text-to-Image Models(https://arxiv.org/abs/2508.03483)
• Keywords: generation, generative
• Abstract: While prior research on text-to-image generation has predominantly focused on biases in human depictions, we investigate a more subtle yet pervasive phenomenon: demographic bias in generated objects (e.g., cars). We introduce SODA (Stereotyped Object Diagnostic Audit), a novel framework for systematically measuring such biases. Our approach compares visual attributes of objects generated with demographic cues (e.g., "for young people'') to those from neutral prompts, across 2,700 images produced by three state-of-the-art models (GPT Image-1, Imagen 4, and Stable Diffusion) in five object categories. Through a comprehensive analysis, we uncover strong associations between specific demographic groups and visual attributes, such as recurring color patterns prompted by gender or ethnicity cues. These patterns reflect and reinforce not only well-known stereotypes but also more subtle and unintuitive biases. We also observe that some models generate less diverse outputs, which in turn amplifies the visual disparities compared to neutral prompts. Our proposed auditing framework offers a practical approach for testing, revealing how stereotypes still remain embedded in today's generative models. We see this as an essential step toward more systematic and responsible AI development.
• 摘要：虽然先前对文本形象生成的研究主要集中在人类描述中的偏见上，但我们研究了一种更微妙而普遍的现象：生成的物体中的人口偏见（例如，汽车）。我们介绍了苏打水（刻板印象的对象诊断审核），这是一个系统地测量此类偏见的新型框架。我们的方法将通过人口统计学提示（例如，对于年轻人'）与中性提示产生的物体的视觉属性进行比较，在三个最先进模型（GPT Image-1，Imagen 4和稳定的扩散）中产生的2,700张图像（在五个对象类别中产生的2,700张图像）。提示。这些模式不仅可以增强著名的刻板印象，而且更加微妙和非直觉的偏见。人工智能发展。

Title: LRQ-DiT: Log-Rotation Post-Training Quantization of Diffusion Transformers for Text-to-Image Generation

• Authors: Lianwei Yang, Haokun Lin, Tianchen Zhao, Yichen Wu, Hongyu Zhu, Ruiqi Xie, Zhenan Sun, Yu Wang, Qingyi Gu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03485
• Pdf URL: https://arxiv.org/pdf/2508.03485
• Copy Paste: [[2508.03485]] LRQ-DiT: Log-Rotation Post-Training Quantization of Diffusion Transformers for Text-to-Image Generation(https://arxiv.org/abs/2508.03485)
• Keywords: generation
• Abstract: Diffusion Transformers (DiTs) have achieved impressive performance in text-to-image generation. However, their high computational cost and large parameter sizes pose significant challenges for usage in resource-constrained scenarios. Post-training quantization (PTQ) is a promising solution to reduce memory usage and accelerate inference, but existing PTQ methods suffer from severe performance degradation under extreme low-bit settings. We identify two key obstacles to low-bit post-training quantization for DiT models: (1) model weights follow a Gaussian-like distribution with long tails, causing uniform quantization to poorly allocate intervals and leading to significant errors; (2) two types of activation outliers: (i) Mild Outliers with slightly elevated values, and (ii) Salient Outliers with large magnitudes concentrated in specific channels, which disrupt activation quantization. To address these issues, we propose LRQ-DiT, an efficient and accurate PTQ framework. We introduce Twin-Log Quantization (TLQ), a log-based method that aligns well with the weight distribution and reduces quantization errors. We also propose an Adaptive Rotation Scheme (ARS) that dynamically applies Hadamard or outlier-aware rotations based on activation fluctuation, effectively mitigating the impact of both types of outliers. We evaluate LRQ-DiT on PixArt and FLUX under various bit-width settings, and validate the performance on COCO, MJHQ, and sDCI datasets. LRQ-DiT achieves low-bit quantization of DiT models while preserving image quality, outperforming existing PTQ baselines.
• 摘要：扩散变压器（DIT）在文本到图像的生成中取得了令人印象深刻的性能。但是，在资源约束的情况下，它们的高计算成本和较大的参数大小对使用构成了重大挑战。训练后量化（PTQ）是减少记忆使用和加速推断的有前途的解决方案，但是现有的PTQ方法在极端低位设置下遭受了严重的性能降解。我们确定了DIT模型的低位训练后量化量化的两个关键障碍：（1）模型权重遵循长尾巴的高斯样分布，导致统一的量化到分配较差的间隔和导致重大误差； （2）两种类型的激活异常值：（i）具有略有升高值的轻度异常值，以及（ii）较大幅度集中在特定通道中的显着异常值，这破坏了激活量化。为了解决这些问题，我们建议LRQ-DIT是一个有效而准确的PTQ框架。我们引入了Twin-Log量化（TLQ），这是一种基于日志的方法，与重量分布很好地对齐并减少量化误差。我们还提出了一种自适应旋转方案（ARS），该方案基于激活波动，动态应用Hadamard或异常值旋转，有效地减轻了两种异常值的影响。我们在各种位宽度设置下评估了Pixart和Flux上的LRQ-DIT，并验证COCO，MJHQ和SDCI数据集的性能。 LRQ-DIT在保持图像质量的同时，可以实现DIT模型的低位量化，表现优于现有的PTQ基准。

Title: ParticleSAM: Small Particle Segmentation for Material Quality Monitoring in Recycling Processes

• Authors: Yu Zhou, Pelle Thielmann, Ayush Chamoli, Bruno Mirbach, Didier Stricker, Jason Rambach
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03490
• Pdf URL: https://arxiv.org/pdf/2508.03490
• Copy Paste: [[2508.03490]] ParticleSAM: Small Particle Segmentation for Material Quality Monitoring in Recycling Processes(https://arxiv.org/abs/2508.03490)
• Keywords: generation
• Abstract: The construction industry represents a major sector in terms of resource consumption. Recycled construction material has high reuse potential, but quality monitoring of the aggregates is typically still performed with manual methods. Vision-based machine learning methods could offer a faster and more efficient solution to this problem, but existing segmentation methods are by design not directly applicable to images with hundreds of small particles. In this paper, we propose ParticleSAM, an adaptation of the segmentation foundation model to images with small and dense objects such as the ones often encountered in construction material particles. Moreover, we create a new dense multi-particle dataset simulated from isolated particle images with the assistance of an automated data generation and labeling pipeline. This dataset serves as a benchmark for visual material quality control automation while our segmentation approach has the potential to be valuable in application areas beyond construction where small-particle segmentation is needed. Our experimental results validate the advantages of our method by comparing to the original SAM method both in quantitative and qualitative experiments.
• 摘要：就资源消耗而言，建筑业代表了一个主要部门。回收的建筑材料具有很高的重用潜力，但是对聚集体的质量监测通常仍使用手动方法进行。基于视觉的机器学习方法可以为此问题提供更快，更有效的解决方案，但是现有的分割方法不直接适用于具有数百个小颗粒的图像。在本文中，我们提出了PracelesAM，将分割基础模型的适应性适应为具有小而密集的物体的图像，例如在建筑材料颗粒中经常遇到的物体。此外，我们在自动数据生成和标记管道的帮助下创建了一个新的密集的多粒子数据集，该数据集从孤立的粒子图像中模拟。该数据集是视觉材料质量控制自动化的基准，而我们的分割方法有可能在需要天粒子细分的施工之外的应用领域中具有价值。我们的实验结果通过与定量和定性实验中的原始SAM方法进行比较来验证我们方法的优势。

Title: EditGarment: An Instruction-Based Garment Editing Dataset Constructed with Automated MLLM Synthesis and Semantic-Aware Evaluation

• Authors: Deqiang Yin, Junyi Guo, Huanda Lu, Fangyu Wu, Dongming Lu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03497
• Pdf URL: https://arxiv.org/pdf/2508.03497
• Copy Paste: [[2508.03497]] EditGarment: An Instruction-Based Garment Editing Dataset Constructed with Automated MLLM Synthesis and Semantic-Aware Evaluation(https://arxiv.org/abs/2508.03497)
• Keywords: generation
• Abstract: Instruction-based garment editing enables precise image modifications via natural language, with broad applications in fashion design and customization. Unlike general editing tasks, it requires understanding garment-specific semantics and attribute dependencies. However, progress is limited by the scarcity of high-quality instruction-image pairs, as manual annotation is costly and hard to scale. While MLLMs have shown promise in automated data synthesis, their application to garment editing is constrained by imprecise instruction modeling and a lack of fashion-specific supervisory signals. To address these challenges, we present an automated pipeline for constructing a garment editing dataset. We first define six editing instruction categories aligned with real-world fashion workflows to guide the generation of balanced and diverse instruction-image triplets. Second, we introduce Fashion Edit Score, a semantic-aware evaluation metric that captures semantic dependencies between garment attributes and provides reliable supervision during construction. Using this pipeline, we construct a total of 52,257 candidate triplets and retain 20,596 high-quality triplets to build EditGarment, the first instruction-based dataset tailored to standalone garment editing. The project page is this https URL.
• 摘要：基于教学的服装编辑可以通过自然语言进行精确的图像修改，并在时装设计和定制中进行广泛的应用。与一般编辑任务不同，它需要了解特定于服装的语义和属性依赖性。但是，由于手动注释昂贵且难以扩展，因此进度受到高质量教学图像对的稀缺限制。尽管MLLM在自动数据综合中表现出了希望，但它们在服装编辑中的应用受到不精确的指导建模和缺乏特定于时尚的监督信号的约束。为了应对这些挑战，我们提出了一条自动化管道，用于构建服装编辑数据集。我们首先定义了六个与现实世界时尚工作流相符的编辑指令类别，以指导平衡和多样化的教学图像三胞胎。其次，我们介绍了时尚编辑得分，这是一种语义感知评估指标，可捕获服装属性之间的语义依赖性，并在施工过程中提供可靠的监督。使用此管道，我们总共构建了52,257个候选三胞胎，并保留20,596个高质量的三胞胎来构建编辑，这是第一个基于指令的数据集，该数据集量身定制，该数据集量身定制为独立服装编辑。项目页面是此HTTPS URL。

Title: Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learning

• Authors: Alexander Golubev, Maria Trofimova, Sergei Polezhaev, Ibragim Badertdinov, Maksim Nekrashevich, Anton Shevtsov, Simon Karasik, Sergey Abramov, Andrei Andriushchenko, Filipp Fisin, Sergei Skvortsov, Boris Yangel
• Subjects: cs.LG, cs.CL, cs.SE
• Abstract URL: https://arxiv.org/abs/2508.03501
• Pdf URL: https://arxiv.org/pdf/2508.03501
• Copy Paste: [[2508.03501]] Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learning(https://arxiv.org/abs/2508.03501)
• Keywords: generation
• Abstract: Research on applications of Reinforcement Learning (RL) to Large Language Models (LLMs) has mostly been focused on single-turn problems, such as mathematical reasoning or single-shot code generation. While these problems can be viewed as token-level multi-turn MDPs, this view corresponds to a degenerate case of multi-turn interaction where the environment provides no feedback. This contrasts with many real-world domains, such as software engineering (SWE), which require rich multi-turn interactions with a stateful environment that responds to each action with a non-trivial observation. To bridge this gap, we demonstrate the successful application of RL to this general regime. Using a modified Decoupled Advantage Policy Optimization (DAPO) algorithm, we train an agent based on Qwen2.5-72B-Instruct to solve real-world software engineering tasks. Our approach increases the agent's success rate on the SWE-bench Verified benchmark from a 20% rejection fine-tuned baseline to 39%, without relying on any teacher models. On SWE-rebench, our agent matches or outperforms leading open-weight models such as DeepSeek-V3-0324 and Qwen3-235B-A22B using an identical scaffolding, offering a viable path toward building more capable autonomous agents for complex real-world problems based on open models.
• 摘要：关于强化学习（RL）在大型语言模型（LLMS）中的应用的研究主要集中在单转弯问题上，例如数学推理或单次代码生成。尽管可以将这些问题视为令牌级的多转移MDP，但此视图对应于多转交互的堕落情况，而环境没有提供反馈。这与许多现实世界中的域（例如软件工程（SWE））形成鲜明对比，这些领域需要与状态环境进行丰富的多转交流，该环境对每个动作都以非平凡的观察做出了反应。为了弥合这一差距，我们证明了RL在该一般制度中的成功应用。使用修改后的分离优势优化（DAPO）算法，我们培训基于QWEN2.5-72B教学的代理，以求解现实世界中的软件工程任务。我们的方法将代理商在SWE-Bench验证的基准测试中的成功率从20％的拒绝微调基线增加到39％，而无需依赖任何教师模型。在SWE-Rebench上，我们的代理商使用相同的脚手架进行匹配或优于领先的开放式型号，例如DeepSeek-V3-0324和Qwen3-235b-A22b，为基于开放型号的复杂现实世界问题提供了可行的自主剂。

Title: MAUP: Training-free Multi-center Adaptive Uncertainty-aware Prompting for Cross-domain Few-shot Medical Image Segmentation

• Authors: Yazhou Zhu, Haofeng Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03511
• Pdf URL: https://arxiv.org/pdf/2508.03511
• Copy Paste: [[2508.03511]] MAUP: Training-free Multi-center Adaptive Uncertainty-aware Prompting for Cross-domain Few-shot Medical Image Segmentation(https://arxiv.org/abs/2508.03511)
• Keywords: generation
• Abstract: Cross-domain Few-shot Medical Image Segmentation (CD-FSMIS) is a potential solution for segmenting medical images with limited annotation using knowledge from other domains. The significant performance of current CD-FSMIS models relies on the heavily training procedure over other source medical domains, which degrades the universality and ease of model deployment. With the development of large visual models of natural images, we propose a training-free CD-FSMIS model that introduces the Multi-center Adaptive Uncertainty-aware Prompting (MAUP) strategy for adapting the foundation model Segment Anything Model (SAM), which is trained with natural images, into the CD-FSMIS task. To be specific, MAUP consists of three key innovations: (1) K-means clustering based multi-center prompts generation for comprehensive spatial coverage, (2) uncertainty-aware prompts selection that focuses on the challenging regions, and (3) adaptive prompt optimization that can dynamically adjust according to the target region complexity. With the pre-trained DINOv2 feature encoder, MAUP achieves precise segmentation results across three medical datasets without any additional training compared with several conventional CD-FSMIS models and training-free FSMIS model. The source code is available at: this https URL.
• 摘要：跨域几乎没有医学图像分割（CD-FSMIS）是使用其他领域的知识来分割有限注释的医学图像的潜在解决方案。当前的CD-FSMIS模型的显着性能取决于其他源医疗领域的严重培训程序，这使模型部署的普遍性和易度性降低了。随着自然图像的大型视觉模型的开发，我们提出了一个无训练的CD-FSMIS模型，该模型介绍了多中心自适应不确定性感知的提示（MAUP）策略，以适应CD-FSMIS任务，以适应基础模型段的任何模型（SAM），该模型（SAM）经过自然图像训练。具体而言，MAUP由三个关键创新组成：（1）基于K-Means聚类的多中心提示，提示生成全面的空间覆盖范围，（2）不确定性感知的提示提示选择着重于挑战性地区，以及（3）自适应及时的优化，可以根据目标区域的复杂性进行动态调整。通过预先训练的Dinov2功能编码器，MAUP与几种常规的CD-FSMIS模型和无培训的FSMIS模型相比，在三个医疗数据集中取得了精确的分割结果，而无需进行任何额外的培训。源代码可在以下网址提供：此HTTPS URL。

Title: MoKA: Mixture of Kronecker Adapters

• Authors: Mohammadreza Sadeghi, Mahsa Ghazvini Nejad, MirHamed Jafarzadeh Asl, Yu Gu, Yuanhao Yu, Masoud Asgharian, Vahid Partovi Nia
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2508.03527
• Pdf URL: https://arxiv.org/pdf/2508.03527
• Copy Paste: [[2508.03527]] MoKA: Mixture of Kronecker Adapters(https://arxiv.org/abs/2508.03527)
• Keywords: generation, generative
• Abstract: Parameter-efficient fine-tuning (PEFT) is essential for reducing the computational overhead of large language models (LLMs). Low-rank family adapters are commonly used to control the parameter size efficiently while maintaining the generative power of LLMs. However, their limited expressiveness due to the rank constraint often restricts their performance on complex tasks. We propose Mixture of Kronecker Adapters (MoKA), a new generation of Kronecker adapters that addresses this limitation by modeling weight updates as a mixture of Kronecker products. Our proposed adapter leverages a gating mechanism that measures the importance of each Kronecker factor, enabling more expressive adaptation. Moreover, MoKA enables a rank flexibility that provides a better trade-off between parameter efficiency and accuracy. To ensure hardware efficiency, we reformulate Kronecker computations using standard matrix operations, allowing seamless deployment on GPU-optimized hardware. We conduct extensive experiments on instruction-tuning and commonsense reasoning tasks using low-bit quantized versions of LLaMA2-7B and LLaMA3-8B models. MoKA not only outperforms PEFT baselines, but also reduces the number of trainable parameters up to 27x, achieving state-of-the-art trade-offs between performance and parameter efficiency.
• 摘要：参数有效的微调（PEFT）对于减少大语言模型（LLMS）的计算开销至关重要。低级家庭适配器通常用于有效地控制参数大小，同时保持LLM的生成能力。但是，由于等级约束，它们的表现力有限通常会限制其在复杂任务上的表现。我们提出了Kronecker适配器（MOKA）的混合物，这是一种新一代的Kronecker适配器，通过将重量更新建模为Kronecker产品的混合物来解决此限制。我们提出的适配器利用了一种衡量每个kronecker因子的重要性的门控机制，从而实现了更具表现力的适应性。此外，Moka可以实现等级灵活性，从而在参数效率和准确性之间提供更好的权衡。为了确保硬件效率，我们使用标准矩阵操作对Kronecker计算进行了重新计算，从而可以在GPU优化的硬件上进行无缝部署。我们使用低位量化版本的Llama2-7B和Llama3-8B模型进行了有关指导调用和常识性推理任务的广泛实验。 Moka不仅胜过PEFT基线，而且还将可训练参数的数量减少到27倍，从而实现了性能和参数效率之间的最新权衡。

Title: CoEmoGen: Towards Semantically-Coherent and Scalable Emotional Image Content Generation

• Authors: Kaishen Yuan, Yuting Zhang, Shang Gao, Yijie Zhu, Wenshuo Chen, Yutao Yue
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03535
• Pdf URL: https://arxiv.org/pdf/2508.03535
• Copy Paste: [[2508.03535]] CoEmoGen: Towards Semantically-Coherent and Scalable Emotional Image Content Generation(https://arxiv.org/abs/2508.03535)
• Keywords: generation
• Abstract: Emotional Image Content Generation (EICG) aims to generate semantically clear and emotionally faithful images based on given emotion categories, with broad application prospects. While recent text-to-image diffusion models excel at generating concrete concepts, they struggle with the complexity of abstract emotions. There have also emerged methods specifically designed for EICG, but they excessively rely on word-level attribute labels for guidance, which suffer from semantic incoherence, ambiguity, and limited scalability. To address these challenges, we propose CoEmoGen, a novel pipeline notable for its semantic coherence and high scalability. Specifically, leveraging multimodal large language models (MLLMs), we construct high-quality captions focused on emotion-triggering content for context-rich semantic guidance. Furthermore, inspired by psychological insights, we design a Hierarchical Low-Rank Adaptation (HiLoRA) module to cohesively model both polarity-shared low-level features and emotion-specific high-level semantics. Extensive experiments demonstrate CoEmoGen's superiority in emotional faithfulness and semantic coherence from quantitative, qualitative, and user study perspectives. To intuitively showcase scalability, we curate EmoArt, a large-scale dataset of emotionally evocative artistic images, providing endless inspiration for emotion-driven artistic creation. The dataset and code are available at this https URL.
• 摘要：情感图像内容产生（EICG）旨在根据给定的情感类别产生语义清晰和情感上的忠实图像，并具有广泛的应用前景。尽管最近的文本到图像扩散模型在产生具体概念方面表现出色，但它们在抽象情绪的复杂性上挣扎。还出现了专门为EICG设计的方法，但是它们过度依赖于单词级属性标签来进行指导，这些标签遭受了语义不一致，模棱两可和有限的可扩展性。为了应对这些挑战，我们提出了Coemogen，这是一种新的管道，以其语义连贯性和高可扩展性而闻名。具体而言，我们利用多模式大语言模型（MLLM）构建了高质量的字幕，重点是情感触发内容，以进行上下文丰富的语义指导。此外，受到心理洞察力的启发，我们设计了一个分层的低级别适应（Hilora）模块，以凝聚为极性共享的低水平特征和特定于情感的高级语义。广泛的实验表明，从定量，定性和用户学习的角度来看，伴生的情感忠诚和语义连贯性的优势。为了直观地展示可扩展性，我们策划了Emoart，这是一个大规模的情感艺术图像数据集，为情感驱动的艺术创作提供了无尽的灵感。该数据集和代码可在此HTTPS URL上找到。

Title: Quality-Aware Language-Conditioned Local Auto-Regressive Anomaly Synthesis and Detection

• Authors: Long Qian, Bingke Zhu, Yingying Chen, Ming Tang, Jinqiao Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03539
• Pdf URL: https://arxiv.org/pdf/2508.03539
• Copy Paste: [[2508.03539]] Quality-Aware Language-Conditioned Local Auto-Regressive Anomaly Synthesis and Detection(https://arxiv.org/abs/2508.03539)
• Keywords: generation
• Abstract: Despite substantial progress in anomaly synthesis methods, existing diffusion-based and coarse inpainting pipelines commonly suffer from structural deficiencies such as micro-structural discontinuities, limited semantic controllability, and inefficient generation. To overcome these limitations, we introduce ARAS, a language-conditioned, auto-regressive anomaly synthesis approach that precisely injects local, text-specified defects into normal images via token-anchored latent editing. Leveraging a hard-gated auto-regressive operator and a training-free, context-preserving masked sampling kernel, ARAS significantly enhances defect realism, preserves fine-grained material textures, and provides continuous semantic control over synthesized anomalies. Integrated within our Quality-Aware Re-weighted Anomaly Detection (QARAD) framework, we further propose a dynamic weighting strategy that emphasizes high-quality synthetic samples by computing an image-text similarity score with a dual-encoder model. Extensive experiments across three benchmark datasets-MVTec AD, VisA, and BTAD, demonstrate that our QARAD outperforms SOTA methods in both image- and pixel-level anomaly detection tasks, achieving improved accuracy, robustness, and a 5 times synthesis speedup compared to diffusion-based alternatives. Our complete code and synthesized dataset will be publicly available.
• 摘要：尽管在异常合成方法方面取得了长足的进展，但现有的基于扩散和粗糙的填充管道通常会遭受结构性缺陷，例如微结构不连续性，有限的语义可控性和效率低下的产生。为了克服这些局限性，我们介绍了ARA，这是一种语言条件，自动回归异常合成方法，该方法精确地将本地的，文本指定的缺陷通过令牌锚定的潜在编辑注入了正常图像。 Aras利用硬门控自动回归运算符和无训练的，具有上下文的掩盖采样内核，可显着增强缺陷现实主义，保留细粒的材料纹理，并对合成的异常提供连续的语义控制。在我们的质量意识重新加权异常检测（QARAD）框架中，我们进一步提出了一种动态加权策略，该策略通过使用双编码器模型计算图像文本相似性得分来强调高质量的合成样本。在三个基准数据集AD，VISA和BTAD上进行的广泛实验表明，我们的Qarad在图像和像素级异常检测任务中的表现都超过了SOTA方法，从而达到了提高的精度，鲁棒性，稳健性，并且与扩散基于扩散基于基于扩散的替代方案相比，合成了5倍。我们的完整代码和合成的数据集将公开可用。

Title: Zero-Variance Gradients for Variational Autoencoders

• Authors: Zilei Shao, Anji Liu, Guy Van den Broeck
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2508.03587
• Pdf URL: https://arxiv.org/pdf/2508.03587
• Copy Paste: [[2508.03587]] Zero-Variance Gradients for Variational Autoencoders(https://arxiv.org/abs/2508.03587)
• Keywords: generative
• Abstract: Training deep generative models like Variational Autoencoders (VAEs) is often hindered by the need to backpropagate gradients through the stochastic sampling of their latent variables, a process that inherently introduces estimation variance, which can slow convergence and degrade performance. In this paper, we propose a new perspective that sidesteps this problem, which we call Silent Gradients. Instead of improving stochastic estimators, we leverage specific decoder architectures to analytically compute the expected ELBO, yielding a gradient with zero variance. We first provide a theoretical foundation for this method and demonstrate its superiority over existing estimators in a controlled setting with a linear decoder. To generalize our approach for practical use with complex, expressive decoders, we introduce a novel training dynamic that uses the exact, zero-variance gradient to guide the early stages of encoder training before annealing to a standard stochastic estimator. Our experiments show that this technique consistently improves the performance of established baselines, including reparameterization, Gumbel-Softmax, and REINFORCE, across multiple datasets. This work opens a new direction for training generative models by combining the stability of analytical computation with the expressiveness of deep, nonlinear architecture.
• 摘要：训练深层生成模型（如变异自动编码器（VAE））通常受到通过其潜在变量的随机抽样来反向流动梯度的阻碍，该过程固有地引入了估计方差，从而减慢收敛性和降低性能。在本文中，我们提出了一种避开此问题的新观点，我们称之为无声梯度。我们利用特定的解码器体系结构来分析预期的ELBO，而不是改进随机估计器，而是产生了零方差的梯度。我们首先为该方法提供了理论基础，并通过线性解码器在受控设置中证明了其优于现有估计器的优势。为了概括我们与复杂的，表达的解码器进行实际使用的方法，我们引入了一种新颖的训练动态，该动态使用确切的零变化梯度来指导编码器训练的早期阶段，然后再退火到标准的随机估计器。我们的实验表明，该技术始终提高已建立的基准的性能，包括重新聚集，gumbel-softmax和跨多个数据集的增强。这项工作通过将分析计算的稳定性与深度非线性结构的表现力相结合，为训练生成模型打开了一个新的方向。

Title: MetaScope: Optics-Driven Neural Network for Ultra-Micro Metalens Endoscopy

• Authors: Wuyang Li, Wentao Pan, Xiaoyuan Liu, Zhendong Luo, Chenxin Li, Hengyu Liu, Din Ping Tsai, Mu Ku Chen, Yixuan Yuan
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.03596
• Pdf URL: https://arxiv.org/pdf/2508.03596
• Copy Paste: [[2508.03596]] MetaScope: Optics-Driven Neural Network for Ultra-Micro Metalens Endoscopy(https://arxiv.org/abs/2508.03596)
• Keywords: restoration
• Abstract: Miniaturized endoscopy has advanced accurate visual perception within the human body. Prevailing research remains limited to conventional cameras employing convex lenses, where the physical constraints with millimetre-scale thickness impose serious impediments on the micro-level clinical. Recently, with the emergence of meta-optics, ultra-micro imaging based on metalenses (micron-scale) has garnered great attention, serving as a promising solution. However, due to the physical difference of metalens, there is a large gap in data acquisition and algorithm research. In light of this, we aim to bridge this unexplored gap, advancing the novel metalens endoscopy. First, we establish datasets for metalens endoscopy and conduct preliminary optical simulation, identifying two derived optical issues that physically adhere to strong optical priors. Second, we propose MetaScope, a novel optics-driven neural network tailored for metalens endoscopy driven by physical optics. MetaScope comprises two novel designs: Optics-informed Intensity Adjustment (OIA), rectifying intensity decay by learning optical embeddings, and Optics-informed Chromatic Correction (OCC), mitigating chromatic aberration by learning spatial deformations informed by learned Point Spread Function (PSF) distributions. To enhance joint learning, we further deploy a gradient-guided distillation to transfer knowledge from the foundational model adaptively. Extensive experiments demonstrate that MetaScope not only outperforms state-of-the-art methods in both metalens segmentation and restoration but also achieves impressive generalized ability in real biomedical scenes.
• 摘要：微型内窥镜检查具有人体内部的精确视觉感知。当前的研究仍然限于采用凸透镜的常规摄像机，在毫米尺寸的物理限制下，厚度的物理约束对微观临床临床造成了严重的障碍。最近，随着元视元的出现，基于金属（Micron cale）的超微小成像引起了人们的极大关注，成为了有希望的解决方案。但是，由于金属的物理差异，数据采集和算法研究存在很大差距。鉴于此，我们旨在弥合这个未开发的缝隙，推动新型金属内窥镜检查。首先，我们为金属内窥镜检查建立数据集并进行初步光学模拟，从而确定了两个派生的光学问题，它们物理上遵循强态较强的光学先验。其次，我们提出了MetaScope，这是一种针对由物理光学驱动的金属内窥镜量身定制的新型光学驱动的神经网络。 Metascope包括两个新型设计：光学信息强度调节（OIA），通过学习光学嵌入来纠正强度衰减以及光学信息校正（OCC），通过学习通过学习点扩散功能（PSF）分布来告知的空间变形来减轻色差。为了增强联合学习，我们进一步部署了梯度引导的蒸馏，以适应从基础模型转移知识。广泛的实验表明，MetaScope不仅要优于金属分割和恢复的最先进方法，而且还可以在实际生物医学场景中实现令人印象深刻的广义能力。

Title: evTransFER: A Transfer Learning Framework for Event-based Facial Expression Recognition

• Authors: Rodrigo Verschae, Ignacio Bugueno-Cordova
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03609
• Pdf URL: https://arxiv.org/pdf/2508.03609
• Copy Paste: [[2508.03609]] evTransFER: A Transfer Learning Framework for Event-based Facial Expression Recognition(https://arxiv.org/abs/2508.03609)
• Keywords: generative
• Abstract: Event-based cameras are bio-inspired vision sensors that asynchronously capture per-pixel intensity changes with microsecond latency, high temporal resolution, and high dynamic range, providing valuable information about the spatio-temporal dynamics of the scene. In the present work, we propose evTransFER, a transfer learning-based framework and architecture for face expression recognition using event-based cameras. The main contribution is a feature extractor designed to encode the spatio-temporal dynamics of faces, built by training an adversarial generative method on a different problem (facial reconstruction) and then transferring the trained encoder weights to the face expression recognition system. We show that this proposed transfer learning method greatly improves the ability to recognize facial expressions compared to training a network from scratch. In addition, we propose an architecture that incorporates an LSTM to capture longer-term facial expression dynamics, and we introduce a new event-based representation, referred to as TIE, both of which further improve the results. We evaluate the proposed framework on the event-based facial expression database e-CK+ and compare it to state-of-the-art methods. The results show that the proposed framework evTransFER achieves a 93.6\% recognition rate on the e-CK+ database, significantly improving the accuracy (25.9\% points or more) when compared to state-of-the-art performance for similar problems.
• 摘要：基于事件的相机是受生物启发的视觉传感器，它们异步捕获每像素强度随着微秒延迟，高时间分辨率和高动态范围而变化，从而提供了有关场景时空动力学的宝贵信息。在目前的工作中，我们建议使用基于事件的摄像机进行面部表达识别的基于转移学习的框架和架构EVTransfer。主要贡献是一种特征提取器，旨在编码面部时空动力学，该动力学是通过在不同问题（面部重建）上训练对抗性生成方法构建的，然后将训练有素的编码器重量转移到面部表达识别系统中。我们表明，与从头开始训练网络相比，这种提出的转移学习方法大大提高了识别面部表情的能力。此外，我们提出了一种结合LSTM来捕获长期面部表达动态的体系结构，我们引入了一种新的基于事件的表示形式，称为TIE，两者都进一步改善了结果。我们在基于事件的面部表达数据库E-CK+上评估了提出的框架，并将其与最新方法进行比较。结果表明，与类似问题相比，提议的框架EVTRANSFER在E-CK+数据库上达到了93.6 \％的识别率，显着提高了准确性（25.9 \％\％点或更多）。

Title: A DbC Inspired Neurosymbolic Layer for Trustworthy Agent Design

• Authors: Claudiu Leoveanu-Condrei
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.03665
• Pdf URL: https://arxiv.org/pdf/2508.03665
• Copy Paste: [[2508.03665]] A DbC Inspired Neurosymbolic Layer for Trustworthy Agent Design(https://arxiv.org/abs/2508.03665)
• Keywords: generation, generative
• Abstract: Generative models, particularly Large Language Models (LLMs), produce fluent outputs yet lack verifiable guarantees. We adapt Design by Contract (DbC) and type-theoretic principles to introduce a contract layer that mediates every LLM call. Contracts stipulate semantic and type requirements on inputs and outputs, coupled with probabilistic remediation to steer generation toward compliance. The layer exposes the dual view of LLMs as semantic parsers and probabilistic black-box components. Contract satisfaction is probabilistic and semantic validation is operationally defined through programmer-specified conditions on well-typed data structures. More broadly, this work postulates that any two agents satisfying the same contracts are \emph{functionally equivalent} with respect to those contracts.
• 摘要：生成模型，尤其是大型语言模型（LLM），产生流利的产出，但缺乏可验证的保证。我们根据合同（DBC）和类型理论原则调整设计，以引入一个合同层，以调节每个LLM调用。合同对输入和输出的语义和类型要求，再加上概率补救措施，从而使产生对遵守。该层将LLM的双重视图视为语义解析器和概率黑盒组件。合同满意度是概率的，语义验证是通过针对良好的数据结构的程序员指定条件来定义的。从更广泛的角度来看，这项工作假定满足相同合同的任何两个代理都是相对于这些合同的\ emph {功能等效}。

Title: Veila: Panoramic LiDAR Generation from a Monocular RGB Image

• Authors: Youquan Liu, Lingdong Kong, Weidong Yang, Ao Liang, Jianxiong Gao, Yang Wu, Xiang Xu, Xin Li, Linfeng Li, Runnan Chen, Ben Fei
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2508.03690
• Pdf URL: https://arxiv.org/pdf/2508.03690
• Copy Paste: [[2508.03690]] Veila: Panoramic LiDAR Generation from a Monocular RGB Image(https://arxiv.org/abs/2508.03690)
• Keywords: generation, generative
• Abstract: Realistic and controllable panoramic LiDAR data generation is critical for scalable 3D perception in autonomous driving and robotics. Existing methods either perform unconditional generation with poor controllability or adopt text-guided synthesis, which lacks fine-grained spatial control. Leveraging a monocular RGB image as a spatial control signal offers a scalable and low-cost alternative, which remains an open problem. However, it faces three core challenges: (i) semantic and depth cues from RGB are vary spatially, complicating reliable conditioning generation; (ii) modality gaps between RGB appearance and LiDAR geometry amplify alignment errors under noisy diffusion; and (iii) maintaining structural coherence between monocular RGB and panoramic LiDAR is challenging, particularly in non-overlap regions between images and LiDAR. To address these challenges, we propose Veila, a novel conditional diffusion framework that integrates: a Confidence-Aware Conditioning Mechanism (CACM) that strengthens RGB conditioning by adaptively balancing semantic and depth cues according to their local reliability; a Geometric Cross-Modal Alignment (GCMA) for robust RGB-LiDAR alignment under noisy diffusion; and a Panoramic Feature Coherence (PFC) for enforcing global structural consistency across monocular RGB and panoramic LiDAR. Additionally, we introduce two metrics, Cross-Modal Semantic Consistency and Cross-Modal Depth Consistency, to evaluate alignment quality across modalities. Experiments on nuScenes, SemanticKITTI, and our proposed KITTI-Weather benchmark demonstrate that Veila achieves state-of-the-art generation fidelity and cross-modal consistency, while enabling generative data augmentation that improves downstream LiDAR semantic segmentation.
• 摘要：现实且可控的全景LIDAR数据生成对于自动驾驶和机器人技术中的可扩展3D感知至关重要。现有方法要么执行无条件的可控性，要么采用文本引导的综合，这缺乏细粒度的空间控制。利用单眼RGB图像作为空间控制信号提供了可扩展且低成本的替代方案，这仍然是一个开放的问题。但是，它面临三个核心挑战：（i）RGB的语义和深度线索在空间上有所不同，使可靠的调理产生复杂化； （ii）RGB外观与LiDAR几何形状之间的模态差距在嘈杂的扩散下放大对齐误差； （iii）保持单眼RGB和全景激光雷达之间的结构相干性是具有挑战性的，尤其是在图像和激光雷达之间的非重叠区域。为了应对这些挑战，我们提出了一个整合的新型有条件扩散框架：一种新颖的条件扩散框架：一种置信度感知的调理机制（CACM），该机制通过根据其局部可靠性自适应地平衡语义和深度来加强RGB调理；在噪声扩散下进行稳健的RGB-LIDAR比对的几何跨模式比对（GCMA）；以及一个全景特征相干性（PFC），用于在单眼RGB和全景激光雷达上执行全局结构一致性。此外，我们介绍了两个指标，即跨模式语义一致性和跨模式深度一致性，以评估跨模式的一致性质量。关于Nuscenes，Semantickitti和我们提出的Kitti-Weather基准的实验表明，Veila实现了最新的产生忠诚度和跨模式的一致性，同时可以增强生成数据，从而改善了下游底线的底激光态分段。

Title: La La LiDAR: Large-Scale Layout Generation from LiDAR Data

• Authors: Youquan Liu, Lingdong Kong, Weidong Yang, Xin Li, Ao Liang, Runnan Chen, Ben Fei, Tongliang Liu
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2508.03691
• Pdf URL: https://arxiv.org/pdf/2508.03691
• Copy Paste: [[2508.03691]] La La LiDAR: Large-Scale Layout Generation from LiDAR Data(https://arxiv.org/abs/2508.03691)
• Keywords: generation, generative
• Abstract: Controllable generation of realistic LiDAR scenes is crucial for applications such as autonomous driving and robotics. While recent diffusion-based models achieve high-fidelity LiDAR generation, they lack explicit control over foreground objects and spatial relationships, limiting their usefulness for scenario simulation and safety validation. To address these limitations, we propose Large-scale Layout-guided LiDAR generation model ("La La LiDAR"), a novel layout-guided generative framework that introduces semantic-enhanced scene graph diffusion with relation-aware contextual conditioning for structured LiDAR layout generation, followed by foreground-aware control injection for complete scene generation. This enables customizable control over object placement while ensuring spatial and semantic consistency. To support our structured LiDAR generation, we introduce Waymo-SG and nuScenes-SG, two large-scale LiDAR scene graph datasets, along with new evaluation metrics for layout synthesis. Extensive experiments demonstrate that La La LiDAR achieves state-of-the-art performance in both LiDAR generation and downstream perception tasks, establishing a new benchmark for controllable 3D scene generation.
• 摘要：可控制的生成型激光雷达场景对于诸如自动驾驶和机器人技术等应用至关重要。尽管最近基于扩散的模型获得了高保真的激光雷达的产生，但它们缺乏对前景对象和空间关系的明确控制，从而限制了它们对场景模拟和安全验证的有用性。为了解决这些局限性，我们提出了大规模布局引导的激光雷达生成模型（“ La la lidar”），这是一种新型布局引导的生成框架，它引入了语义增强的场景图形扩散，并为实现的lidar lidar布局带来了相关性lidar布局，其次是前景衡量控制的生成，然后是完整场景生成的注射。这可以在确保空间和语义一致性的同时对对象放置进行自定义的控制。为了支持我们的结构化激光雷达生成，我们介绍了Waymo-SG和Nuscenes-SG，这是两个大规模的LIDAR场景图数据集，以及用于布局合成的新评估指标。广泛的实验表明，La La Lidar在Lidar Generation和Powstream Insception任务中都达到了最先进的表现，为可控的3D场景生成建立了新的基准。

Title: LiDARCrafter: Dynamic 4D World Modeling from LiDAR Sequences

• Authors: Ao Liang, Youquan Liu, Yu Yang, Dongyue Lu, Linfeng Li, Lingdong Kong, Huaici Zhao, Wei Tsang Ooi
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2508.03692
• Pdf URL: https://arxiv.org/pdf/2508.03692
• Copy Paste: [[2508.03692]] LiDARCrafter: Dynamic 4D World Modeling from LiDAR Sequences(https://arxiv.org/abs/2508.03692)
• Keywords: generation, generative
• Abstract: Generative world models have become essential data engines for autonomous driving, yet most existing efforts focus on videos or occupancy grids, overlooking the unique LiDAR properties. Extending LiDAR generation to dynamic 4D world modeling presents challenges in controllability, temporal coherence, and evaluation standardization. To this end, we present LiDARCrafter, a unified framework for 4D LiDAR generation and editing. Given free-form natural language inputs, we parse instructions into ego-centric scene graphs, which condition a tri-branch diffusion network to generate object structures, motion trajectories, and geometry. These structured conditions enable diverse and fine-grained scene editing. Additionally, an autoregressive module generates temporally coherent 4D LiDAR sequences with smooth transitions. To support standardized evaluation, we establish a comprehensive benchmark with diverse metrics spanning scene-, object-, and sequence-level aspects. Experiments on the nuScenes dataset using this benchmark demonstrate that LiDARCrafter achieves state-of-the-art performance in fidelity, controllability, and temporal consistency across all levels, paving the way for data augmentation and simulation. The code and benchmark are released to the community.
• 摘要：生成世界模型已成为自动驾驶的重要数据引擎，但是大多数现有的努力都集中在视频或占用网格上，俯瞰着独特的LiDAR特性。将激光雷达的生成扩展到动态4D世界建模带来了可控性，时间连贯性和评估标准化方面的挑战。为此，我们提出了Lidarcrafter，这是4D激光雷达生成和编辑的统一框架。给定自由形式的自然语言输入，我们将指令解析到以自我为中心的场景图中，该图形条件是三大分支扩散网络，以生成对象结构，运动轨迹和几何形状。这些结构化条件可实现多样化和细粒度的现场编辑。此外，自回旋模块生成具有光滑跃迁的时间相干的4D激光镜序列。为了支持标准化的评估，我们建立了一个综合基准，该基准具有不同的指标，涵盖了场景，对象和序列级别的方面。使用此基准测试的Nuscenes数据集进行的实验表明，Lidarcrafter在所有级别的忠诚度，可控性和时间一致性方面实现了最先进的性能，为数据增强和仿真铺平了道路。该代码和基准已将其发布给社区。

Title: LongVie: Multimodal-Guided Controllable Ultra-Long Video Generation

• Authors: Jianxiong Gao, Zhaoxi Chen, Xian Liu, Jianfeng Feng, Chenyang Si, Yanwei Fu, Yu Qiao, Ziwei Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.03694
• Pdf URL: https://arxiv.org/pdf/2508.03694
• Copy Paste: [[2508.03694]] LongVie: Multimodal-Guided Controllable Ultra-Long Video Generation(https://arxiv.org/abs/2508.03694)
• Keywords: generation
• Abstract: Controllable ultra-long video generation is a fundamental yet challenging task. Although existing methods are effective for short clips, they struggle to scale due to issues such as temporal inconsistency and visual degradation. In this paper, we initially investigate and identify three key factors: separate noise initialization, independent control signal normalization, and the limitations of single-modality guidance. To address these issues, we propose LongVie, an end-to-end autoregressive framework for controllable long video generation. LongVie introduces two core designs to ensure temporal consistency: 1) a unified noise initialization strategy that maintains consistent generation across clips, and 2) global control signal normalization that enforces alignment in the control space throughout the entire video. To mitigate visual degradation, LongVie employs 3) a multi-modal control framework that integrates both dense (e.g., depth maps) and sparse (e.g., keypoints) control signals, complemented by 4) a degradation-aware training strategy that adaptively balances modality contributions over time to preserve visual quality. We also introduce LongVGenBench, a comprehensive benchmark consisting of 100 high-resolution videos spanning diverse real-world and synthetic environments, each lasting over one minute. Extensive experiments show that LongVie achieves state-of-the-art performance in long-range controllability, consistency, and quality.
• 摘要：可控的超长视频生成是一项基本但具有挑战性的任务。尽管现有方法对于短剪辑有效，但由于时间不一致和视觉降解等问题，它们很难进行扩展。在本文中，我们最初研究并确定了三个关键因素：独立的噪声初始化，独立的控制信号归一化以及单模式指导的局限性。为了解决这些问题，我们提出了Longvie，这是一个端到端的自动回归框架，可控制长期视频。 Longvie介绍了两种核心设计，以确保时间一致性：1）统一的噪声初始化策略，该策略在整个剪辑中保持一致的生成，以及2）全局控制信号归一化，该信号正常化在整个视频中强制执行控制空间的一致性。为了减轻视觉降低，Longvie使用3）一个多模式控制框架，该框架可以整合密集（例如，深度图）和稀疏（例如，关键点）控制信号，并补充4）一种降级意识到的训练策略，可适应均衡模态的均衡，以保持均衡的时间越来越多地促进视觉质量。我们还介绍了Longvgenbench，这是一个全面的基准测试，该基准由100个高分辨率视频组成，涵盖了多样化的现实世界和合成环境，每个都持续了一分钟。广泛的实验表明，Longvie在远程可控性，一致性和质量方面取得了最新的性能。