2025-08-15

Title: Empowering Morphing Attack Detection using Interpretable Image-Text Foundation Model

• Authors: Sushrut Patwardhan, Raghavendra Ramachandra, Sushma Venkatesh
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.10110
• Pdf URL: https://arxiv.org/pdf/2508.10110
• Copy Paste: [[2508.10110]] Empowering Morphing Attack Detection using Interpretable Image-Text Foundation Model(https://arxiv.org/abs/2508.10110)
• Keywords: generation
• Abstract: Morphing attack detection has become an essential component of face recognition systems for ensuring a reliable verification scenario. In this paper, we present a multimodal learning approach that can provide a textual description of morphing attack detection. We first show that zero-shot evaluation of the proposed framework using Contrastive Language-Image Pretraining (CLIP) can yield not only generalizable morphing attack detection, but also predict the most relevant text snippet. We present an extensive analysis of ten different textual prompts that include both short and long textual prompts. These prompts are engineered by considering the human understandable textual snippet. Extensive experiments were performed on a face morphing dataset that was developed using a publicly available face biometric dataset. We present an evaluation of SOTA pre-trained neural networks together with the proposed framework in the zero-shot evaluation of five different morphing generation techniques that are captured in three different mediums.
• 摘要：变形攻击检测已成为确保可靠验证方案的面部识别系统的重要组成部分。在本文中，我们提出了一种多模式学习方法，可以提供有关变形攻击检测的文本描述。我们首先表明，使用对比性语言图像预处理（剪辑）对所提出的框架进行零射击评估不仅可以产生可概括的变形攻击检测，而且还可以预测最相关的文本片段。我们对十个不同的文本提示进行了广泛的分析，其中包括短文字提示。这些提示是通过考虑人类可以理解的文本片段来设计的。大量实验是在使用公开可用面部生物识别数据集开发的面部变形数据集上进行的。我们提出了对SOTA预训练的神经网络的评估，以及对五种不同介质捕获的五种不同变形生成技术的零射门评估中提出的框架。

Title: Constrained Decoding of Diffusion LLMs with Context-Free Grammars

• Authors: Niels Mündler, Jasper Dekoninck, Martin Vechev
• Subjects: cs.LG, cs.FL, cs.SE
• Abstract URL: https://arxiv.org/abs/2508.10111
• Pdf URL: https://arxiv.org/pdf/2508.10111
• Copy Paste: [[2508.10111]] Constrained Decoding of Diffusion LLMs with Context-Free Grammars(https://arxiv.org/abs/2508.10111)
• Keywords: generation
• Abstract: Large language models (LLMs) have shown promising performance across diverse domains. Many practical applications of LLMs, such as code completion and structured data extraction, require adherence to syntactic constraints specified by a formal language. Yet, due to their probabilistic nature, LLM output is not guaranteed to adhere to such formal languages. Prior work has proposed constrained decoding as a means to restrict LLM generation to particular formal languages. However, existing works are not applicable to the emerging paradigm of diffusion LLMs, when used in practical scenarios such as the generation of formally correct C++ or JSON output. In this paper we address this challenge and present the first constrained decoding method for diffusion models, one that can handle formal languages captured by context-free grammars. We begin by reducing constrained decoding to the more general additive infilling problem, which asks whether a partial output can be completed to a valid word in the target language. This problem also naturally subsumes the previously unaddressed multi-region infilling constrained decoding. We then reduce this problem to the task of deciding whether the intersection of the target language and a regular language is empty and present an efficient algorithm to solve it for context-free languages. Empirical results on various applications, such as C++ code infilling and structured data extraction in JSON, demonstrate that our method achieves near-perfect syntactic correctness while consistently preserving or improving functional correctness. Importantly, our efficiency optimizations ensure that the computational overhead remains practical.
• 摘要：大型语言模型（LLM）显示了各种领域的有希望的表现。 LLM的许多实际应用，例如代码完成和结构化数据提取，都需要遵守正式语言指定的句法约束。然而，由于其概率性质，LLM输出不能保证遵守这种形式语言。先前的工作提出了限制解码，以此作为将LLM生成限制为特定正式语言的一种手段。但是，现有作品不适用于在实用场景（例如形式上正确正确的C ++或JSON输出）中使用的扩散LLM的新兴范式。在本文中，我们解决了这一挑战，并提出了扩散模型的第一个约束解码方法，该方法可以处理由无上下文的语法捕获的正式语言。我们首先将约束的解码减少到更通用的添加填充问题，该问题询问是否可以将部分输出完成到目标语言中的有效单词。这个问题也自然归因于先前未经解决的多区域填充约束解码。然后，我们将这个问题减少到确定目标语言和普通语言的交集是否为空的任务，并提出有效的算法来解决该算法以解决无上下文的语言。在JSON中的C ++代码填充和结构化数据提取等各种应用的经验结果表明，我们的方法实现了接近完美的句法正确性，同时始终保持或改善功能正确性。重要的是，我们的效率优化确保了计算开销仍然是实用的。

Title: From Intent to Execution: Multimodal Chain-of-Thought Reinforcement Learning for Precise CAD Code Generation

• Authors: Ke Niu, Haiyang Yu, Zhuofan Chen, Mengyang Zhao, Teng Fu, Bin Li, Xiangyang Xue
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10118
• Pdf URL: https://arxiv.org/pdf/2508.10118
• Copy Paste: [[2508.10118]] From Intent to Execution: Multimodal Chain-of-Thought Reinforcement Learning for Precise CAD Code Generation(https://arxiv.org/abs/2508.10118)
• Keywords: generation
• Abstract: Computer-Aided Design (CAD) plays a vital role in engineering and manufacturing, yet current CAD workflows require extensive domain expertise and manual modeling effort. Recent advances in large language models (LLMs) have made it possible to generate code from natural language, opening new opportunities for automating parametric 3D modeling. However, directly translating human design intent into executable CAD code remains highly challenging, due to the need for logical reasoning, syntactic correctness, and numerical precision. In this work, we propose CAD-RL, a multimodal Chain-of-Thought (CoT) guided reinforcement learning post training framework for CAD modeling code generation. Our method combines CoT-based Cold Start with goal-driven reinforcement learning post training using three task-specific rewards: executability reward, geometric accuracy reward, and external evaluation reward. To ensure stable policy learning under sparse and high-variance reward conditions, we introduce three targeted optimization strategies: Trust Region Stretch for improved exploration, Precision Token Loss for enhanced dimensions parameter accuracy, and Overlong Filtering to reduce noisy supervision. To support training and benchmarking, we release ExeCAD, a noval dataset comprising 16,540 real-world CAD examples with paired natural language and structured design language descriptions, executable CADQuery scripts, and rendered 3D models. Experiments demonstrate that CAD-RL achieves significant improvements in reasoning quality, output precision, and code executability over existing VLMs.
• 摘要：计算机辅助设计（CAD）在工程和制造中起着至关重要的作用，但是当前的CAD工作流程需要广泛的域专业知识和手动建模工作。大型语言模型（LLMS）的最新进展使得从自然语言生成代码，为自动化参数3D建模开放了新的机会。但是，由于需要逻辑推理，句法正确性和数值精度，直接将意图转化为可执行的CAD代码的人类设计仍然具有高度挑战性。在这项工作中，我们提出了CAD-RL，这是一种多模式链（COT）指导的加固学习，用于CAD建模代码生成后的培训框架。我们的方法结合了基于COT的冷启动与目标驱动的强化学习后培训后使用特定于任务的奖励：可执行性奖励，几何准确性奖励和外部评估奖励。为了确保在稀疏且高空奖励条件下进行稳定的政策学习，我们介绍了三种有针对性的优化策略：信任区域延伸，以改善探索，精确令牌损失，以增强维度参数的准确性和较长的过滤，以减少嘈杂的监督。为了支持培训和基准测试，我们发布了Execad，这是一个Noval数据集，其中包括16,540个现实世界中的CAD示例，其中包含配对的自然语言和结构化设计语言描述，可执行的CADQUERY脚本以及渲染3D模型。实验表明，CAD-RL在现有VLM上的推理质量，输出精度和代码可执行性方面取得了重大改进。

Title: Deep Learning Enables Large-Scale Shape and Appearance Modeling in Total-Body DXA Imaging

• Authors: Arianna Bunnell, Devon Cataldi, Yannik Glaser, Thomas K. Wolfgruber, Steven Heymsfield, Alan B. Zonderman, Thomas L. Kelly, Peter Sadowski, John A. Shepherd
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10132
• Pdf URL: https://arxiv.org/pdf/2508.10132
• Copy Paste: [[2508.10132]] Deep Learning Enables Large-Scale Shape and Appearance Modeling in Total-Body DXA Imaging(https://arxiv.org/abs/2508.10132)
• Keywords: generation
• Abstract: Total-body dual X-ray absorptiometry (TBDXA) imaging is a relatively low-cost whole-body imaging modality, widely used for body composition assessment. We develop and validate a deep learning method for automatic fiducial point placement on TBDXA scans using 1,683 manually-annotated TBDXA scans. The method achieves 99.5% percentage correct keypoints in an external testing dataset. To demonstrate the value for shape and appearance modeling (SAM), our method is used to place keypoints on 35,928 scans for five different TBDXA imaging modes, then associations with health markers are tested in two cohorts not used for SAM model generation using two-sample Kolmogorov-Smirnov tests. SAM feature distributions associated with health biomarkers are shown to corroborate existing evidence and generate new hypotheses on body composition and shape's relationship to various frailty, metabolic, inflammation, and cardiometabolic health markers. Evaluation scripts, model weights, automatic point file generation code, and triangulation files are available at this https URL.
• 摘要：总体型X射线吸收仪（TBDXA）成像是一种相对低成本的全身成像方式，广泛用于人体组成评估。我们使用1,683次手动注销的TBDXA扫描来开发并验证一种对TBDXA扫描自动基准放置的深度学习方法。该方法在外部测试数据集中达到了99.5％的正确关键点。为了证明形状和外观建模的值（SAM），我们的方法用于在35,928张扫描中对五种不同的TBDXA成像模式放置关键点，然后使用两样本的Kolmogorov-kolmogorov-smirnov测试来对与健康标记的相关性进行测试。与健康生物标志物相关的SAM特征分布被证实可以证实现有的证据，并对身体成分和形状与各种脆弱，代谢，炎症和心脏代谢健康标记物的关系产生新的假设。评估脚本，模型权重，自动点文件生成代码和三角剖分文件可在此HTTPS URL上找到。

Title: rETF-semiSL: Semi-Supervised Learning for Neural Collapse in Temporal Data

• Authors: Yuhan Xie, William Cappelletti, Mahsa Shoaran, Pascal Frossard
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.10147
• Pdf URL: https://arxiv.org/pdf/2508.10147
• Copy Paste: [[2508.10147]] rETF-semiSL: Semi-Supervised Learning for Neural Collapse in Temporal Data(https://arxiv.org/abs/2508.10147)
• Keywords: generative
• Abstract: Deep neural networks for time series must capture complex temporal patterns, to effectively represent dynamic data. Self- and semi-supervised learning methods show promising results in pre-training large models, which -- when finetuned for classification -- often outperform their counterparts trained from scratch. Still, the choice of pretext training tasks is often heuristic and their transferability to downstream classification is not granted, thus we propose a novel semi-supervised pre-training strategy to enforce latent representations that satisfy the Neural Collapse phenomenon observed in optimally trained neural classifiers. We use a rotational equiangular tight frame-classifier and pseudo-labeling to pre-train deep encoders with few labeled samples. Furthermore, to effectively capture temporal dynamics while enforcing embedding separability, we integrate generative pretext tasks with our method, and we define a novel sequential augmentation strategy. We show that our method significantly outperforms previous pretext tasks when applied to LSTMs, transformers, and state-space models on three multivariate time series classification datasets. These results highlight the benefit of aligning pre-training objectives with theoretically grounded embedding geometry.
• 摘要：时间序列的深神经网络必须捕获复杂的时间模式，以有效地表示动态数据。自我监督的学习方法在预训练大型模型中显示出令人鼓舞的结果，当对分类进行填充时，通常比从头开始训练的对手。尽管如此，借口训练任务的选择通常是启发式的，并且没有批准其对下游分类的可转移性，因此我们提出了一种新型的半监督预训练的预训练策略，以执行在受过最佳训练的神经分类器中观察到的神经崩溃现象的潜在表述。我们使用旋转的等法紧密框架分类器和伪标记来预先培训深度编码器，这些编码器很少。此外，为了有效地捕获时间动力学，同时可以在嵌入可分离性的同时将生成借个任务与方法整合在一起，并定义了一种新颖的顺序增强策略。我们表明，当在三个多元时间序列分类数据集上应用于LSTM，变形金刚和状态空间模型时，我们的方法显着优于以前的借口任务。这些结果突出了将预训练目标与理论接地的嵌入几何形状保持一致的好处。

Title: Improving watermelon (Citrullus lanatus) disease classification with generative artificial intelligence (GenAI)-based synthetic and real-field images via a custom EfficientNetV2-L model

• Authors: Nitin Rai, Nathan S. Boyd, Gary E. Vallad, Arnold W. Schumann
• Subjects: cs.CV, cs.AI, cs.ET
• Abstract URL: https://arxiv.org/abs/2508.10156
• Pdf URL: https://arxiv.org/pdf/2508.10156
• Copy Paste: [[2508.10156]] Improving watermelon (Citrullus lanatus) disease classification with generative artificial intelligence (GenAI)-based synthetic and real-field images via a custom EfficientNetV2-L model(https://arxiv.org/abs/2508.10156)
• Keywords: generative
• Abstract: The current advancements in generative artificial intelligence (GenAI) models have paved the way for new possibilities for generating high-resolution synthetic images, thereby offering a promising alternative to traditional image acquisition for training computer vision models in agriculture. In the context of crop disease diagnosis, GenAI models are being used to create synthetic images of various diseases, potentially facilitating model creation and reducing the dependency on resource-intensive in-field data collection. However, limited research has been conducted on evaluating the effectiveness of integrating real with synthetic images to improve disease classification performance. Therefore, this study aims to investigate whether combining a limited number of real images with synthetic images can enhance the prediction accuracy of an EfficientNetV2-L model for classifying watermelon \textit{(Citrullus lanatus)} diseases. The training dataset was divided into five treatments: H0 (only real images), H1 (only synthetic images), H2 (1:1 real-to-synthetic), H3 (1:10 real-to-synthetic), and H4 (H3 + random images to improve variability and model generalization). All treatments were trained using a custom EfficientNetV2-L architecture with enhanced fine-tuning and transfer learning techniques. Models trained on H2, H3, and H4 treatments demonstrated high precision, recall, and F1-score metrics. Additionally, the weighted F1-score increased from 0.65 (on H0) to 1.00 (on H3-H4) signifying that the addition of a small number of real images with a considerable volume of synthetic images improved model performance and generalizability. Overall, this validates the findings that synthetic images alone cannot adequately substitute for real images; instead, both must be used in a hybrid manner to maximize model performance for crop disease classification.
• 摘要：当前的生成人工智能（Genai）模型的进步为生成高分辨率合成图像的新可能性铺平了道路，从而为农业中的培训计算机视觉模型提供了一种有希望的替代方法。在作物疾病诊断的背景下，Genai模型被用于创建各种疾病的合成图像，有可能促进模型的创建并降低对资源密集型现场数据收集的依赖。但是，已经进行了有限的研究，以评估将真实图像与合成图像整合以改善疾病分类性能的有效性。因此，本研究旨在研究将有限数量的真实图像与合成图像结合起来是否可以提高用于对西瓜\ textit {（（citrullus lanatus）}分类的有效NETV2-L模型的预测准确性。训练数据集分为五种治疗方法：H0（仅真实图像），H1（仅合成图像），H2（1：1真实到合成），H3（1:10真实到合成）和H4（H3 +随机图像以改善变异性和模型通用化）。所有处理都使用具有增强的微调和转移学习技术的自定义效率NETV2-L体系结构培训。接受H2，H3和H4处理训练的模型表现出很高的精度，召回和F1得分指标。此外，加权F1分数从0.65（在H0上）增加到1.00（在H3-H4上），这表明添加了少量具有相当大量合成图像的真实图像提高了模型性能和可推广性。总体而言，这证明了单独合成图像不能充分代替真实图像的发现。取而代之的是，两者都必须以杂种方式使用，以最大程度地提高模型性能进行农作物疾病分类。

Title: SynSpill: Improved Industrial Spill Detection With Synthetic Data

• Authors: Aaditya Baranwal, Abdul Mueez, Jason Voelker, Guneet Bhatia, Shruti Vyas
• Subjects: cs.CV, cs.ET
• Abstract URL: https://arxiv.org/abs/2508.10171
• Pdf URL: https://arxiv.org/pdf/2508.10171
• Copy Paste: [[2508.10171]] SynSpill: Improved Industrial Spill Detection With Synthetic Data(https://arxiv.org/abs/2508.10171)
• Keywords: generation
• Abstract: Large-scale Vision-Language Models (VLMs) have transformed general-purpose visual recognition through strong zero-shot capabilities. However, their performance degrades significantly in niche, safety-critical domains such as industrial spill detection, where hazardous events are rare, sensitive, and difficult to annotate. This scarcity -- driven by privacy concerns, data sensitivity, and the infrequency of real incidents -- renders conventional fine-tuning of detectors infeasible for most industrial settings. We address this challenge by introducing a scalable framework centered on a high-quality synthetic data generation pipeline. We demonstrate that this synthetic corpus enables effective Parameter-Efficient Fine-Tuning (PEFT) of VLMs and substantially boosts the performance of state-of-the-art object detectors such as YOLO and DETR. Notably, in the absence of synthetic data (SynSpill dataset), VLMs still generalize better to unseen spill scenarios than these detectors. When SynSpill is used, both VLMs and detectors achieve marked improvements, with their performance becoming comparable. Our results underscore that high-fidelity synthetic data is a powerful means to bridge the domain gap in safety-critical applications. The combination of synthetic generation and lightweight adaptation offers a cost-effective, scalable pathway for deploying vision systems in industrial environments where real data is scarce/impractical to obtain. Project Page: this https URL
• 摘要：大型视觉模型（VLM）通过强零射击能力转化了通用视觉识别。但是，它们的性能在利基市场，安全至关重要的领域（例如工业泄漏检测）中大大降低，因为危险事件罕见，敏感且难以注释。这种稀缺性 - 由隐私问题，数据敏感性和实际事件的不频率驱动 - 导致大多数工业环境不可行的探测器进行常规微调。我们通过引入以高质量合成数据生成管道为中心的可扩展框架来应对这一挑战。我们证明该合成语料库可以实现VLM的有效参数有效的微调（PEFT），并显着提高了最新的对象检测器（例如Yolo和Detr）的性能。值得注意的是，在没有合成数据（Synspill数据集）的情况下，VLMS仍然比这些检测器更好地推广到看不见的溢出场景。当使用合成螺旋时，VLM和检测器都可以实现明显的改进，它们的性能变得可比性。我们的结果表明，高保真合成数据是弥合安全至关重要应用中域间隙的有力手段。合成生成和轻巧适应性的结合为在实际数据稀缺/不切实际的工业环境中部署视觉系统提供了一种具有成本效益的可扩展途径。项目页面：此HTTPS URL

Title: An Explainable AI based approach for Monitoring Animal Health

• Authors: Rahul Janaa, Shubham Dixit, Mrityunjay Sharma, Ritesh Kumar
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.10210
• Pdf URL: https://arxiv.org/pdf/2508.10210
• Copy Paste: [[2508.10210]] An Explainable AI based approach for Monitoring Animal Health(https://arxiv.org/abs/2508.10210)
• Keywords: generation
• Abstract: Monitoring cattle health and optimizing yield are key challenges faced by dairy farmers due to difficulties in tracking all animals on the farm. This work aims to showcase modern data-driven farming practices based on explainable machine learning(ML) methods that explain the activity and behaviour of dairy cattle (cows). Continuous data collection of 3-axis accelerometer sensors and usage of robust ML methodologies and algorithms, provide farmers and researchers with actionable information on cattle activity, allowing farmers to make informed decisions and incorporate sustainable practices. This study utilizes Bluetooth-based Internet of Things (IoT) devices and 4G networks for seamless data transmission, immediate analysis, inference generation, and explains the models performance with explainability frameworks. Special emphasis is put on the pre-processing of the accelerometers time series data, including the extraction of statistical characteristics, signal processing techniques, and lag-based features using the sliding window technique. Various hyperparameter-optimized ML models are evaluated across varying window lengths for activity classification. The k-nearest neighbour Classifier achieved the best performance, with AUC of mean 0.98 and standard deviation of 0.0026 on the training set and 0.99 on testing set). In order to ensure transparency, Explainable AI based frameworks such as SHAP is used to interpret feature importance that can be understood and used by practitioners. A detailed comparison of the important features, along with the stability analysis of selected features, supports development of explainable and practical ML models for sustainable livestock management.
• 摘要：监测牛健康和优化产量是乳业农民面临的主要挑战，因为很难追踪农场上的所有动物。这项工作旨在根据可解释的机器学习（ML）方法来展示现代数据驱动的农业实践，这些方法解释了奶牛的活动和行为（牛）。连续数据收集3轴加速度计传感器以及强大的ML方法和算法的使用，为农民和研究人员提供有关牛活动的可行信息，使农民能够做出明智的决定并结合可持续的实践。这项研究利用基于蓝牙的物联网（IoT）设备和4G网络来无缝数据传输，即时分析，推理生成，并用可解释性框架解释了模型性能。特别强调加速度计时间序列数据的预处理，包括使用滑动窗口技术提取统计特征，信号处理技术和基于滞后的特征。各种高参数优化的ML模型在不同的窗口长度上进行了评估，以进行活动分类。 K-Neartiment Spracifier取得了最佳性能，在训练组中，AUC为0.98，标准偏差为0.0026，测试集为0.99）。为了确保透明度，可解释的基于AI的框架（例如Shap）用于解释从业者可以理解和使用的特征重要性。对重要特征的详细比较，以及对选定功能的稳定性分析，支持开发可持续牲畜管理的可解释和实用的ML模型。

Title: Comparison of D-Wave Quantum Annealing and Markov Chain Monte Carlo for Sampling from a Probability Distribution of a Restricted Boltzmann Machine

• Authors: Abdelmoula El Yazizi, Samee U. Khan, Yaroslav Koshka
• Subjects: cs.LG, quant-ph, stat.ML
• Abstract URL: https://arxiv.org/abs/2508.10228
• Pdf URL: https://arxiv.org/pdf/2508.10228
• Copy Paste: [[2508.10228]] Comparison of D-Wave Quantum Annealing and Markov Chain Monte Carlo for Sampling from a Probability Distribution of a Restricted Boltzmann Machine(https://arxiv.org/abs/2508.10228)
• Keywords: generation
• Abstract: A local-valley (LV) centered approach to assessing the quality of sampling from Restricted Boltzmann Machines (RBMs) was applied to the latest generation of the D-Wave quantum annealer. D-Wave and Gibbs samples from a classically trained RBM were obtained at conditions relevant to the contrastive-divergence-based RBM learning. The samples were compared for the number of the LVs to which they belonged and the energy of the corresponding local minima. No significant (desirable) increase in the number of the LVs has been achieved by decreasing the D-Wave annealing time. At any training epoch, the states sampled by the D-Wave belonged to a somewhat higher number of LVs than in the Gibbs sampling. However, many of those LVs found by the two techniques differed. For high-probability sampled states, the two techniques were (unfavorably) less complementary and more overlapping. Nevertheless, many potentially "important" local minima, i.e., those having intermediate, even if not high, probability values, were found by only one of the two sampling techniques while missed by the other. The two techniques overlapped less at later than earlier training epochs, which is precisely the stage of the training when modest improvements to the sampling quality could make meaningful differences for the RBM trainability. The results of this work may explain the failure of previous investigations to achieve substantial (or any) improvement when using D-Wave-based sampling. However, the results reveal some potential for improvement, e.g., using a combined classical-quantum approach.
• 摘要：一种以局部谷（LV）为中心的方法，用于评估限制性玻尔兹曼机器（RBMS）采样质量，将其应用于最新一代的D-Wave量子退火器。在与基于对比的RBM学习有关的条件下，获得了经过经典训练的RBM的D-WAVE和GIBBS样品。比较样品的属于其属于的LV的数量和相应的局部最小值的能量。通过减少D波退火时间，无法实现LVS数量的显着增加（理想）。在任何训练时期，由D-WAVE取样的状态属于LV的数量要比Gibbs采样更高。但是，这两种技术发现的许多LV都不同。对于高概率采样状态，这两种技术（不利的）互补性较少，重叠率更高。然而，许多潜在的“重要”局部最小值，即，即使不是很高的概率值，也只有两种抽样技术中的一种，而另一个则被另一个被遗漏。这两种技术在晚期训练时期的晚期重叠较少，这正是训练的阶段，当对采样质量的适度改进可能会给RBM训练性带来有意义的差异。这项工作的结果可能解释了先前研究在使用基于D波的采样时取得重大（或任何）改进的失败。但是，结果揭示了一些改进的潜力，例如使用合并的经典量子方法。

Title: High Fidelity Text to Image Generation with Contrastive Alignment and Structural Guidance

• Authors: Danyi Gao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10280
• Pdf URL: https://arxiv.org/pdf/2508.10280
• Copy Paste: [[2508.10280]] High Fidelity Text to Image Generation with Contrastive Alignment and Structural Guidance(https://arxiv.org/abs/2508.10280)
• Keywords: generation
• Abstract: This paper addresses the performance bottlenecks of existing text-driven image generation methods in terms of semantic alignment accuracy and structural consistency. A high-fidelity image generation method is proposed by integrating text-image contrastive constraints with structural guidance mechanisms. The approach introduces a contrastive learning module that builds strong cross-modal alignment constraints to improve semantic matching between text and image. At the same time, structural priors such as semantic layout maps or edge sketches are used to guide the generator in spatial-level structural modeling. This enhances the layout completeness and detail fidelity of the generated images. Within the overall framework, the model jointly optimizes contrastive loss, structural consistency loss, and semantic preservation loss. A multi-objective supervision mechanism is adopted to improve the semantic consistency and controllability of the generated content. Systematic experiments are conducted on the COCO-2014 dataset. Sensitivity analyses are performed on embedding dimensions, text length, and structural guidance strength. Quantitative metrics confirm the superior performance of the proposed method in terms of CLIP Score, FID, and SSIM. The results show that the method effectively bridges the gap between semantic alignment and structural fidelity without increasing computational complexity. It demonstrates a strong ability to generate semantically clear and structurally complete images, offering a viable technical path for joint text-image modeling and image generation.
• 摘要：本文以语义对准准确性和结构一致性来解决现有文本驱动图像生成方法的性能瓶颈。通过将文本图像对比的约束与结构指导机制相结合，提出了高保真的图像生成方法。该方法引入了一个对比度学习模块，该模块建立了强大的跨模式对齐约束，以改善文本和图像之间的语义匹配。同时，使用诸如语义布局图或边缘草图之类的结构先验来指导空间级结构建模中的发生器。这增强了生成图像的布局完整性和详细信息。在整个框架内，该模型共同优化了对比损失，结构一致性损失和语义保存损失。采用多目标监督机制来提高生成内容的语义一致性和可控性。系统实验是在CoCO-2014数据集上进行的。对嵌入尺寸，文本长度和结构指导强度进行灵敏度分析。定量指标证实了该方法在夹得分，FID和SSIM方面的出色性能。结果表明，该方法有效地弥合了语义比对与结构保真度之间的差距，而不会增加计算复杂性。它表现出强大的能力，可以生成语义清晰和结构完整的图像，为关节文本图像建模和图像生成提供可行的技术路径。

Title: InterSyn: Interleaved Learning for Dynamic Motion Synthesis in the Wild

• Authors: Yiyi Ma, Yuanzhi Liang, Xiu Li, Chi Zhang, Xuelong Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10297
• Pdf URL: https://arxiv.org/pdf/2508.10297
• Copy Paste: [[2508.10297]] InterSyn: Interleaved Learning for Dynamic Motion Synthesis in the Wild(https://arxiv.org/abs/2508.10297)
• Keywords: generation
• Abstract: We present Interleaved Learning for Motion Synthesis (InterSyn), a novel framework that targets the generation of realistic interaction motions by learning from integrated motions that consider both solo and multi-person dynamics. Unlike previous methods that treat these components separately, InterSyn employs an interleaved learning strategy to capture the natural, dynamic interactions and nuanced coordination inherent in real-world scenarios. Our framework comprises two key modules: the Interleaved Interaction Synthesis (INS) module, which jointly models solo and interactive behaviors in a unified paradigm from a first-person perspective to support multiple character interactions, and the Relative Coordination Refinement (REC) module, which refines mutual dynamics and ensures synchronized motions among characters. Experimental results show that the motion sequences generated by InterSyn exhibit higher text-to-motion alignment and improved diversity compared with recent methods, setting a new benchmark for robust and natural motion synthesis. Additionally, our code will be open-sourced in the future to promote further research and development in this area.
• 摘要：我们提出了运动合成的交错学习（Intersyn），这是一个新颖的框架，它通过从考虑独奏和多人动力学的集成运动中学习来靶向逼真的相互作用动作。与以前分别处理这些组件的方法不同，Intersone采用了交织的学习策略来捕获实际情况中固有的自然，动态相互作用和细微的协调。我们的框架包括两个关键模块：交织的相互作用合成（INS）模块，从第一人称角度，在统一范式中共同对单人范式进行了独奏和交互行为，以支持多个角色相互作用，并确定相互配合的改进（REC）模块，该模块可完善互动动力学并确保字符中的同步动机。实验结果表明，与最近的方法相比，Intersyn产生的运动序列表现出更高的文本到动作比对，并改善了多样性，为鲁棒和自然运动合成树立了新的基准。此外，我们的代码将来将在未来开源，以促进该领域的进一步研究和发展。

Title: SynBrain: Enhancing Visual-to-fMRI Synthesis via Probabilistic Representation Learning

• Authors: Weijian Mai, Jiamin Wu, Yu Zhu, Zhouheng Yao, Dongzhan Zhou, Andrew F. Luo, Qihao Zheng, Wanli Ouyang, Chunfeng Song
• Subjects: cs.LG, cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2508.10298
• Pdf URL: https://arxiv.org/pdf/2508.10298
• Copy Paste: [[2508.10298]] SynBrain: Enhancing Visual-to-fMRI Synthesis via Probabilistic Representation Learning(https://arxiv.org/abs/2508.10298)
• Keywords: generative
• Abstract: Deciphering how visual stimuli are transformed into cortical responses is a fundamental challenge in computational neuroscience. This visual-to-neural mapping is inherently a one-to-many relationship, as identical visual inputs reliably evoke variable hemodynamic responses across trials, contexts, and subjects. However, existing deterministic methods struggle to simultaneously model this biological variability while capturing the underlying functional consistency that encodes stimulus information. To address these limitations, we propose SynBrain, a generative framework that simulates the transformation from visual semantics to neural responses in a probabilistic and biologically interpretable manner. SynBrain introduces two key components: (i) BrainVAE models neural representations as continuous probability distributions via probabilistic learning while maintaining functional consistency through visual semantic constraints; (ii) A Semantic-to-Neural Mapper acts as a semantic transmission pathway, projecting visual semantics into the neural response manifold to facilitate high-fidelity fMRI synthesis. Experimental results demonstrate that SynBrain surpasses state-of-the-art methods in subject-specific visual-to-fMRI encoding performance. Furthermore, SynBrain adapts efficiently to new subjects with few-shot data and synthesizes high-quality fMRI signals that are effective in improving data-limited fMRI-to-image decoding performance. Beyond that, SynBrain reveals functional consistency across trials and subjects, with synthesized signals capturing interpretable patterns shaped by biological neural variability. The code will be made publicly available.
• 摘要：在计算神经科学中，将视觉刺激转化为皮质反应是一个基本挑战。这种视觉到神经映射本质上是一种一对多的关系，因为相同的视觉输入可靠地唤起了整个试验，环境和受试者的可变血液动力学反应。但是，现有的确定性方法难以同时对这种生物学变异性进行建模，同时捕获编码刺激信息的潜在功能一致性。为了解决这些局限性，我们提出了Synbrain，这是一个生成框架，以概率和生物学上可解释的方式模拟从视觉语义到神经反应的转变。 Synbrain引入了两个关键组成部分：（I）通过概率学习，Brainvae将神经表示作为连续概率分布，同时通过视觉语义限制来维持功能一致性； （ii）语义到神经映射器充当语义传输途径，将视觉语义投射到神经反应歧管中，以促进高保真fMRI综合。实验结果表明，在特定于特定的视觉到FMRI编码性能中，Synbrain超过了最新方法。此外，Synbrain有效地适应了很少的数据，并综合了有效改善数据限制的fMRI-FMRI到图像解码性能的高质量fMRI信号。除此之外，Synbrain还揭示了跨试验和受试者的功能一致性，并具有合成的信号，捕获了由生物神经变异性塑造的可解释模式。该代码将公开可用。

Title: Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advances

• Authors: Yuanzhi Liang, Yijie Fang, Rui Li, Ziqi Ni, Ruijie Su, Chi Zhang, Xuelong Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10316
• Pdf URL: https://arxiv.org/pdf/2508.10316
• Copy Paste: [[2508.10316]] Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advances(https://arxiv.org/abs/2508.10316)
• Keywords: generation, generative
• Abstract: Generative models have made significant progress in synthesizing visual content, including images, videos, and 3D/4D structures. However, they are typically trained with surrogate objectives such as likelihood or reconstruction loss, which often misalign with perceptual quality, semantic accuracy, or physical realism. Reinforcement learning (RL) offers a principled framework for optimizing non-differentiable, preference-driven, and temporally structured objectives. Recent advances demonstrate its effectiveness in enhancing controllability, consistency, and human alignment across generative tasks. This survey provides a systematic overview of RL-based methods for visual content generation. We review the evolution of RL from classical control to its role as a general-purpose optimization tool, and examine its integration into image, video, and 3D/4D generation. Across these domains, RL serves not only as a fine-tuning mechanism but also as a structural component for aligning generation with complex, high-level goals. We conclude with open challenges and future research directions at the intersection of RL and generative modeling.
• 摘要：生成模型在综合视觉内容（包括图像，视频和3D/4D结构）方面取得了重大进展。但是，它们通常接受替代目标（例如可能性或重建损失）的培训，这些目标通常与知觉质量，语义准确性或物理现实主义息息。增强学习（RL）提供了一个原则上的框架，用于优化非差异，偏好驱动和时间结构化的目标。最近的进步表明，其在增强生成任务之间增强可控性，一致性和人类一致性方面的有效性。该调查提供了用于视觉内容生成的基于RL的方法的系统概述。我们回顾了RL从经典控制到其作为通用优化工具的作用的演变，并检查了其集成到图像，视频和3D/4D世代中。在这些域中，RL不仅可以用作微调机制，而且还可以作为使生成与复杂，高级目标对齐的结构成分。我们在RL和生成建模的交集中以公开挑战和未来的研究方向结束。

Title: Towards Spatially Consistent Image Generation: On Incorporating Intrinsic Scene Properties into Diffusion Models

• Authors: Hyundo Lee, Suhyung Choi, Byoung-Tak Zhang, Inwoo Hwang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10382
• Pdf URL: https://arxiv.org/pdf/2508.10382
• Copy Paste: [[2508.10382]] Towards Spatially Consistent Image Generation: On Incorporating Intrinsic Scene Properties into Diffusion Models(https://arxiv.org/abs/2508.10382)
• Keywords: generation
• Abstract: Image generation models trained on large datasets can synthesize high-quality images but often produce spatially inconsistent and distorted images due to limited information about the underlying structures and spatial layouts. In this work, we leverage intrinsic scene properties (e.g., depth, segmentation maps) that provide rich information about the underlying scene, unlike prior approaches that solely rely on image-text pairs or use intrinsics as conditional inputs. Our approach aims to co-generate both images and their corresponding intrinsics, enabling the model to implicitly capture the underlying scene structure and generate more spatially consistent and realistic images. Specifically, we first extract rich intrinsic scene properties from a large image dataset with pre-trained estimators, eliminating the need for additional scene information or explicit 3D representations. We then aggregate various intrinsic scene properties into a single latent variable using an autoencoder. Building upon pre-trained large-scale Latent Diffusion Models (LDMs), our method simultaneously denoises the image and intrinsic domains by carefully sharing mutual information so that the image and intrinsic reflect each other without degrading image quality. Experimental results demonstrate that our method corrects spatial inconsistencies and produces a more natural layout of scenes while maintaining the fidelity and textual alignment of the base model (e.g., Stable Diffusion).
• 摘要：在大型数据集上训练的图像生成模型可以合成高质量的图像，但由于有关基础结构和空间布局的信息有限，因此通常会产生空间不一致和扭曲的图像。在这项工作中，我们利用固有的场景属性（例如，深度，分割图）提供了有关基础场景的丰富信息，这与仅依赖图像介绍对或使用内在的有条件输入的先验方法不同。我们的方法旨在共同生成图像及其相应的内在物质，使该模型能够隐式捕获基础场景结构并生成更加一致和逼真的图像。具体而言，我们首先从具有预训练的估计器的大图像数据集中提取丰富的固有场景属性，从而消除了对其他场景信息或显式3D表示的需求。然后，我们使用自动编码器将各种固有场景属性汇总到单个潜在变量中。在预训练的大规模潜扩散模型（LDMS）的基础上，我们的方法同时通过仔细共享相互信息来同时降低图像和内在域，从而使图像和固有信息相互反映而不会降低图像质量。实验结果表明，我们的方法纠正了空间上的不一致，并产生了更自然的场景布局，同时保持基本模型的忠诚度和文本比对（例如，稳定的扩散）。

Title: PQ-DAF: Pose-driven Quality-controlled Data Augmentation for Data-scarce Driver Distraction Detection

• Authors: Haibin Sun, Xinghui Song
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.10397
• Pdf URL: https://arxiv.org/pdf/2508.10397
• Copy Paste: [[2508.10397]] PQ-DAF: Pose-driven Quality-controlled Data Augmentation for Data-scarce Driver Distraction Detection(https://arxiv.org/abs/2508.10397)
• Keywords: quality assessment
• Abstract: Driver distraction detection is essential for improving traffic safety and reducing road accidents. However, existing models often suffer from degraded generalization when deployed in real-world scenarios. This limitation primarily arises from the few-shot learning challenge caused by the high cost of data annotation in practical environments, as well as the substantial domain shift between training datasets and target deployment conditions. To address these issues, we propose a Pose-driven Quality-controlled Data Augmentation Framework (PQ-DAF) that leverages a vision-language model for sample filtering to cost-effectively expand training data and enhance cross-domain robustness. Specifically, we employ a Progressive Conditional Diffusion Model (PCDMs) to accurately capture key driver pose features and synthesize diverse training examples. A sample quality assessment module, built upon the CogVLM vision-language model, is then introduced to filter out low-quality synthetic samples based on a confidence threshold, ensuring the reliability of the augmented dataset. Extensive experiments demonstrate that PQ-DAF substantially improves performance in few-shot driver distraction detection, achieving significant gains in model generalization under data-scarce conditions.
• 摘要：驾驶员分心检测对于改善交通安全和减少道路事故至关重要。但是，现有模型在现实世界中部署时通常会遭受降解的概括。这种局限性主要是由于实际环境中数据注释的高成本以及训练数据集和目标部署条件之间的实质域变化引起的几乎没有射击的学习挑战。为了解决这些问题，我们提出了一个姿势驱动的质量控制数据增强框架（PQ-DAF），该框架利用视觉模型来进行样品过滤，以扩大训练数据并增强跨域鲁棒性。具体而言，我们采用渐进式有条件扩散模型（PCDM）来准确捕获关键驱动器姿势特征并合成各种训练示例。然后引入基于COGVLM视觉模型的样本质量评估模块，以根据置信阈值过滤低质量的合成样本，以确保增强数据集的可靠性。广泛的实验表明，PQ-DAF在几乎没有驱动器分心检测中大大提高了性能，从而在数据筛选条件下实现了模型概括的显着增长。

Title: NanoControl: A Lightweight Framework for Precise and Efficient Control in Diffusion Transformer

• Authors: Shanyuan Liu, Jian Zhu, Junda Lu, Yue Gong, Liuzhuozheng Li, Bo Cheng, Yuhang Ma, Liebucha Wu, Xiaoyu Wu, Dawei Leng, Yuhui Yin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10424
• Pdf URL: https://arxiv.org/pdf/2508.10424
• Copy Paste: [[2508.10424]] NanoControl: A Lightweight Framework for Precise and Efficient Control in Diffusion Transformer(https://arxiv.org/abs/2508.10424)
• Keywords: generation
• Abstract: Diffusion Transformers (DiTs) have demonstrated exceptional capabilities in text-to-image synthesis. However, in the domain of controllable text-to-image generation using DiTs, most existing methods still rely on the ControlNet paradigm originally designed for UNet-based diffusion models. This paradigm introduces significant parameter overhead and increased computational costs. To address these challenges, we propose the Nano Control Diffusion Transformer (NanoControl), which employs Flux as the backbone network. Our model achieves state-of-the-art controllable text-to-image generation performance while incurring only a 0.024\% increase in parameter count and a 0.029\% increase in GFLOPs, thus enabling highly efficient controllable generation. Specifically, rather than duplicating the DiT backbone for control, we design a LoRA-style (low-rank adaptation) control module that directly learns control signals from raw conditioning inputs. Furthermore, we introduce a KV-Context Augmentation mechanism that integrates condition-specific key-value information into the backbone in a simple yet highly effective manner, facilitating deep fusion of conditional features. Extensive benchmark experiments demonstrate that NanoControl significantly reduces computational overhead compared to conventional control approaches, while maintaining superior generation quality and achieving improved controllability.
• 摘要：扩散变压器（DIT）在文本对图像合成中表现出了特殊的功能。但是，在使用DIT的可控文本到图像生成的域中，大多数现有方法仍然依赖于最初为基于UNET的扩散模型设计的ControlNet范式。该范式引入了重要的参数开销和增加的计算成本。为了应对这些挑战，我们提出了使用通量作为骨干网络的纳米控制扩散变压器（纳米控制）。我们的模型可实现最新的可控文本生成性能，同时仅产生0.024 \％的参数计数和0.029 \％的GFLOPS增加，从而实现了高效的可控制生成。具体而言，我们设计了一个直接从原始条件输入中学习控制信号的LORA风格（低级适应）控制模块，而不是重复DIT主链进行控制。此外，我们引入了一种KV-Context增强机制，该机制将条件特定的键值信息集成到主链中，以一种简单而高效的方式将其集成到主链中，从而促进了条件特征的深层融合。广泛的基准实验表明，与常规控制方法相比，纳米控制可以显着减少计算间接费用，同时保持上等的发电质量并提高可控性。

Title: Trajectory-aware Shifted State Space Models for Online Video Super-Resolution

• Authors: Qiang Zhu, Xiandong Meng, Yuxian Jiang, Fan Zhang, David Bull, Shuyuan Zhu, Bing Zeng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10453
• Pdf URL: https://arxiv.org/pdf/2508.10453
• Copy Paste: [[2508.10453]] Trajectory-aware Shifted State Space Models for Online Video Super-Resolution(https://arxiv.org/abs/2508.10453)
• Keywords: super-resolution, generation
• Abstract: Online video super-resolution (VSR) is an important technique for many real-world video processing applications, which aims to restore the current high-resolution video frame based on temporally previous frames. Most of the existing online VSR methods solely employ one neighboring previous frame to achieve temporal alignment, which limits long-range temporal modeling of videos. Recently, state space models (SSMs) have been proposed with linear computational complexity and a global receptive field, which significantly improve computational efficiency and performance. In this context, this paper presents a novel online VSR method based on Trajectory-aware Shifted SSMs (TS-Mamba), leveraging both long-term trajectory modeling and low-complexity Mamba to achieve efficient spatio-temporal information aggregation. Specifically, TS-Mamba first constructs the trajectories within a video to select the most similar tokens from the previous frames. Then, a Trajectory-aware Shifted Mamba Aggregation (TSMA) module consisting of proposed shifted SSMs blocks is employed to aggregate the selected tokens. The shifted SSMs blocks are designed based on Hilbert scannings and corresponding shift operations to compensate for scanning losses and strengthen the spatial continuity of Mamba. Additionally, we propose a trajectory-aware loss function to supervise the trajectory generation, ensuring the accuracy of token selection when training our model. Extensive experiments on three widely used VSR test datasets demonstrate that compared with six online VSR benchmark models, our TS-Mamba achieves state-of-the-art performance in most cases and over 22.7\% complexity reduction (in MACs). The source code for TS-Mamba will be available at this https URL.
• 摘要：在线视频超级分辨率（VSR）是许多现实世界视频处理应用程序的重要技术，该应用程序旨在恢复基于暂时的框架的当前高分辨率视频框架。大多数现有的在线VSR方法仅采用一个相邻的先前框架来实现时间对齐，这限制了视频的远程时间建模。最近，已经提出了具有线性计算复杂性和全球接收场的状态空间模型（SSM），从而显着提高了计算效率和性能。在这种情况下，本文介绍了一种基于轨迹感知的SSM（TS-MAMBA）的新型在线VSR方法，利用长期轨迹建模和低复杂MAMBA来实现有效的时空信息汇总。具体而言，TS-Mamba首先在视频中构造轨迹，以从以前的帧中选择最相似的令牌。然后，采用轨迹感知的移动MAMBA聚集（TSMA）模块，该模块由建议的移位SSMS块组成，以汇总所选令牌。移动的SSM块是根据希尔伯特扫描和相应的换档操作设计的，以补偿扫描损失并增强MAMBA的空间连续性。此外，我们提出一个轨迹感知的损失函数来监督轨迹生成，以确保在训练模型时选择令牌选择的准确性。在三个广泛使用的VSR测试数据集上进行的广泛实验表明，与六个在线VSR基准模型相比，我们的TS-Mamba在大多数情况下可以实现最先进的性能，超过22.7 \％的复杂性降低（Macs中）。 TS-Mamba的源代码将在此HTTPS URL上找到。

Title: RealAC: A Domain-Agnostic Framework for Realistic and Actionable Counterfactual Explanations

• Authors: Asiful Arefeen, Shovito Barua Soumma, Hassan Ghasemzadeh
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.10455
• Pdf URL: https://arxiv.org/pdf/2508.10455
• Copy Paste: [[2508.10455]] RealAC: A Domain-Agnostic Framework for Realistic and Actionable Counterfactual Explanations(https://arxiv.org/abs/2508.10455)
• Keywords: generation
• Abstract: Counterfactual explanations provide human-understandable reasoning for AI-made decisions by describing minimal changes to input features that would alter a model's prediction. To be truly useful in practice, such explanations must be realistic and feasible -- they should respect both the underlying data distribution and user-defined feasibility constraints. Existing approaches often enforce inter-feature dependencies through rigid, hand-crafted constraints or domain-specific knowledge, which limits their generalizability and ability to capture complex, nonlinear relations inherent in data. Moreover, they rarely accommodate user-specified preferences and suggest explanations that are causally implausible or infeasible to act upon. We introduce RealAC, a domain-agnostic framework for generating realistic and actionable counterfactuals. RealAC automatically preserves complex inter-feature dependencies without relying on explicit domain knowledge -- by aligning the joint distributions of feature pairs between factual and counterfactual instances. The framework also allows end-users to ``freeze'' attributes they cannot or do not wish to change by suppressing change in frozen features during optimization. Evaluations on three synthetic and two real datasets demonstrate that RealAC balances realism with actionability. Our method outperforms state-of-the-art baselines and Large Language Model-based counterfactual generation techniques in causal edge score, dependency preservation score, and IM1 realism metric and offers a solution for causality-aware and user-centric counterfactual generation.
• 摘要：反事实解释通过描述将改变模型预测的输入特征的最小变化来为AI制定的决策提供人为理解的推理。为了在实践中真正有用，这种解释必须是现实的和可行的 - 它们应尊重潜在的数据分配和用户定义的可行性约束。现有的方法通常通过僵化，手工制作的约束或特定领域的知识来实现场间依赖性，这限制了其概述数据中固有的复杂，非线性关系的能力。此外，它们很少适合用户指定的偏好，并提出了因果关系不可行或不可行的解释。我们介绍了Realac，这是一种域形不足的框架，用于生成现实且可行的反事实。 RealAC会自动保留复杂的功能间依赖性，而无需依赖明确的领域知识 - 通过使事实对和反事实实例之间的特征对的联合分布对齐。该框架还允许最终用户``冻结''属性在优化过程中抑制冷冻功能的变化而无法更改。对三个合成和两个真实数据集的评估表明，Realac将现实主义与可行性平衡。我们的方法在因果边缘得分，依赖关系保存评分和IM1现实主义指标中优于最先进的基线和基于语言模型的大型反事实生成技术，并为因果关系感知和以用户为中心的反事实发电提供了解决方案。

Title: GraphFedMIG: Tackling Class Imbalance in Federated Graph Learning via Mutual Information-Guided Generation

• Authors: Xinrui Li, Qilin Fan, Tianfu Wang, Kaiwen Wei, Ke Yu, Xu Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2508.10471
• Pdf URL: https://arxiv.org/pdf/2508.10471
• Copy Paste: [[2508.10471]] GraphFedMIG: Tackling Class Imbalance in Federated Graph Learning via Mutual Information-Guided Generation(https://arxiv.org/abs/2508.10471)
• Keywords: generation, generative
• Abstract: Federated graph learning (FGL) enables multiple clients to collaboratively train powerful graph neural networks without sharing their private, decentralized graph data. Inherited from generic federated learning, FGL is critically challenged by statistical heterogeneity, where non-IID data distributions across clients can severely impair model performance. A particularly destructive form of this is class imbalance, which causes the global model to become biased towards majority classes and fail at identifying rare but critical events. This issue is exacerbated in FGL, as nodes from a minority class are often surrounded by biased neighborhood information, hindering the learning of expressive embeddings. To grapple with this challenge, we propose GraphFedMIG, a novel FGL framework that reframes the problem as a federated generative data augmentation task. GraphFedMIG employs a hierarchical generative adversarial network where each client trains a local generator to synthesize high-fidelity feature representations. To provide tailored supervision, clients are grouped into clusters, each sharing a dedicated discriminator. Crucially, the framework designs a mutual information-guided mechanism to steer the evolution of these client generators. By calculating each client's unique informational value, this mechanism corrects the local generator parameters, ensuring that subsequent rounds of mutual information-guided generation are focused on producing high-value, minority-class features. We conduct extensive experiments on four real-world datasets, and the results demonstrate the superiority of the proposed GraphFedMIG compared with other baselines.
• 摘要：Federated Graph Learning（FGL）使多个客户能够在不共享其私人，分散的图形数据的情况下协作训练强大的图形神经网络。 FGL从通用联合学习中继承，受到统计异质性的严重挑战，跨客户的非IID数据分布可以严重损害模型性能。阶级的不平衡是一种特别破坏性的形式，这会导致全球模型对多数阶级偏见，并且无法识别罕见但关键的事件。 FGL中这个问题加剧了，因为来自少数群体的节点通常被有偏见的邻里信息所包围，从而阻碍了表达性嵌入的学习。为了应对这一挑战，我们提出了GraphFedmig，这是一种新型的FGL框架，将问题折叠为联合生成数据增强任务。 GraphFedmig采用了分层生成对抗网络，每个客户端都会训练本地发电机来合成高保真特征表示。为了提供量身定制的监督，客户分组为群集，每个集群共享一个专用的歧视者。至关重要的是，该框架设计了一种相互的信息引导的机制来引导这些客户发生器的演变。通过计算每个客户的独特信息值，该机制纠正了本地发电机参数，以确保随后的相互信息引导的生成集中于生产高价值，少数级特征。我们对四个现实世界数据集进行了广泛的实验，结果证明了与其他基线相比，提出的Graphfedmig的优越性。

Title: Learning State-Space Models of Dynamic Systems from Arbitrary Data using Joint Embedding Predictive Architectures

• Authors: Jonas Ulmen, Ganesh Sundaram, Daniel Görges
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2508.10489
• Pdf URL: https://arxiv.org/pdf/2508.10489
• Copy Paste: [[2508.10489]] Learning State-Space Models of Dynamic Systems from Arbitrary Data using Joint Embedding Predictive Architectures(https://arxiv.org/abs/2508.10489)
• Keywords: generation
• Abstract: With the advent of Joint Embedding Predictive Architectures (JEPAs), which appear to be more capable than reconstruction-based methods, this paper introduces a novel technique for creating world models using continuous-time dynamic systems from arbitrary observation data. The proposed method integrates sequence embeddings with neural ordinary differential equations (neural ODEs). It employs loss functions that enforce contractive embeddings and Lipschitz constants in state transitions to construct a well-organized latent state space. The approach's effectiveness is demonstrated through the generation of structured latent state-space models for a simple pendulum system using only image data. This opens up a new technique for developing more general control algorithms and estimation techniques with broad applications in robotics.
• 摘要：随着联合嵌入预测体系结构（JEPA）的出现，似乎比基于重建的方法更有能力，因此本文介绍了一种新型技术，用于使用任意观察数据的连续时间动态系统创建世界模型。所提出的方法将序列嵌入与神经普通微分方程（神经ODE）相结合。它采用损失功能，可以在州过渡中强制执行承包嵌入和Lipschitz常数，以构建组织良好的潜在状态空间。通过仅使用图像数据的简单摆系统的结构化潜在空间模型的生成来证明该方法的有效性。这开了一种新技术，用于开发更多的通用控制算法和具有广泛应用机器人技术的估计技术。

Title: A Unified Multi-Agent Framework for Universal Multimodal Understanding and Generation

• Authors: Jiulin Li, Ping Huang, Yexin Li, Shuo Chen, Juewen Hu, Ye Tian
• Subjects: cs.LG, cs.AI, cs.MA
• Abstract URL: https://arxiv.org/abs/2508.10494
• Pdf URL: https://arxiv.org/pdf/2508.10494
• Copy Paste: [[2508.10494]] A Unified Multi-Agent Framework for Universal Multimodal Understanding and Generation(https://arxiv.org/abs/2508.10494)
• Keywords: generation
• Abstract: Real-world multimodal applications often require any-to-any capabilities, enabling both understanding and generation across modalities including text, image, audio, and video. However, integrating the strengths of autoregressive language models (LLMs) for reasoning and diffusion models for high-fidelity generation remains challenging. Existing approaches rely on rigid pipelines or tightly coupled architectures, limiting flexibility and scalability. We propose MAGUS (Multi-Agent Guided Unified Multimodal System), a modular framework that unifies multimodal understanding and generation via two decoupled phases: Cognition and Deliberation. MAGUS enables symbolic multi-agent collaboration within a shared textual workspace. In the Cognition phase, three role-conditioned multimodal LLM agents - Perceiver, Planner, and Reflector - engage in collaborative dialogue to perform structured understanding and planning. The Deliberation phase incorporates a Growth-Aware Search mechanism that orchestrates LLM-based reasoning and diffusion-based generation in a mutually reinforcing manner. MAGUS supports plug-and-play extensibility, scalable any-to-any modality conversion, and semantic alignment - all without the need for joint training. Experiments across multiple benchmarks, including image, video, and audio generation, as well as cross-modal instruction following, demonstrate that MAGUS outperforms strong baselines and state-of-the-art systems. Notably, on the MME benchmark, MAGUS surpasses the powerful closed-source model GPT-4o.
• 摘要：现实世界中的多模式应用程序通常需要任何对任何功能，从而使跨文本，图像，音频和视频在内的模态既可以理解和生成。但是，将自回归语言模型（LLMS）的优势集成为高保真生成的推理和扩散模型仍然具有挑战性。现有方法依赖于刚性管道或紧密耦合的架构，从而限制了灵活性和可扩展性。我们提出了Magus（多代理指导的统一多模式系统），这是一个模块化框架，通过两个解耦阶段统一多模式的理解和生成：认知和审议。 Magus在共享的文本工作区中启用符号多代理协作。在认知阶段，三个角色调节的多模式LLM代理 - 感知者，计划者和反射器 - 进行协作对话，以执行结构化的理解和计划。审议阶段结合了一种增长感知的搜索机制，该机制以相互加固的方式协调基于LLM的推理和基于扩散的生成。 Magus支持插件的可扩展性，可扩展的任何对任何模态转换和语义对齐方式 - 无需进行联合培训。跨多个基准测试的实验，包括图像，视频和音频产生，以及跨模式的指令，表明Magus的表现优于强大的基线和最新的系统。值得注意的是，在MME基准测试上，Magus超过了强大的封闭源模型GPT-4O。

Title: Reasoning in Computer Vision: Taxonomy, Models, Tasks, and Methodologies

• Authors: Ayushman Sarkar, Mohd Yamani Idna Idris, Zhenyu Yu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10523
• Pdf URL: https://arxiv.org/pdf/2508.10523
• Copy Paste: [[2508.10523]] Reasoning in Computer Vision: Taxonomy, Models, Tasks, and Methodologies(https://arxiv.org/abs/2508.10523)
• Keywords: generation
• Abstract: Visual reasoning is critical for a wide range of computer vision tasks that go beyond surface-level object detection and classification. Despite notable advances in relational, symbolic, temporal, causal, and commonsense reasoning, existing surveys often address these directions in isolation, lacking a unified analysis and comparison across reasoning types, methodologies, and evaluation protocols. This survey aims to address this gap by categorizing visual reasoning into five major types (relational, symbolic, temporal, causal, and commonsense) and systematically examining their implementation through architectures such as graph-based models, memory networks, attention mechanisms, and neuro-symbolic systems. We review evaluation protocols designed to assess functional correctness, structural consistency, and causal validity, and critically analyze their limitations in terms of generalizability, reproducibility, and explanatory power. Beyond evaluation, we identify key open challenges in visual reasoning, including scalability to complex scenes, deeper integration of symbolic and neural paradigms, the lack of comprehensive benchmark datasets, and reasoning under weak supervision. Finally, we outline a forward-looking research agenda for next-generation vision systems, emphasizing that bridging perception and reasoning is essential for building transparent, trustworthy, and cross-domain adaptive AI systems, particularly in critical domains such as autonomous driving and medical diagnostics.
• 摘要：视觉推理对于超越表面级对象检测和分类的各种计算机视觉任务至关重要。尽管在关系，象征，时间，因果和常识性推理方面取得了显着进步，但现有的调查通常会孤立地解决这些方向，缺乏跨推理类型，方法论和评估方案的统一分析和比较。这项调查旨在通过将视觉推理分为五种主要类型（关系，象征性，时间，因果关系和常识）来解决这一差距，并通过基于图形的模型，记忆网络，注意力机制和神经符号系统等体系结构进行系统地检查其实现。我们审查了旨在评估功能正确性，结构一致性和因果有效性的评估方案，并严格分析了其限制，可概括性，可重复性和解释能力。除评估外，我们还确定了视觉推理中的关键开放挑战，包括对复杂场景的可扩展性，符号和神经范式的更深入整合，缺乏全面的基准数据集以及在弱监督下进行推理。最后，我们概述了下一代视觉系统的前瞻性研究议程，强调桥接感知和推理对于构建透明，可信赖和跨域自适应AI系统至关重要，尤其是在自动驾驶和医疗诊断等关键领域。

Title: Med-GLIP: Advancing Medical Language-Image Pre-training with Large-scale Grounded Dataset

• Authors: Ziye Deng, Ruihan He, Jiaxiang Liu, Yuan Wang, Zijie Meng, Songtao Jiang, Yong Xie, Zuozhu Liu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.10528
• Pdf URL: https://arxiv.org/pdf/2508.10528
• Copy Paste: [[2508.10528]] Med-GLIP: Advancing Medical Language-Image Pre-training with Large-scale Grounded Dataset(https://arxiv.org/abs/2508.10528)
• Keywords: generation
• Abstract: Medical image grounding aims to align natural language phrases with specific regions in medical images, serving as a foundational task for intelligent diagnosis, visual question answering (VQA), and automated report generation (MRG). However, existing research is constrained by limited modality coverage, coarse-grained annotations, and the absence of a unified, generalizable grounding framework. To address these challenges, we construct a large-scale medical grounding dataset Med-GLIP-5M comprising over 5.3 million region-level annotations across seven imaging modalities, covering diverse anatomical structures and pathological findings. The dataset supports both segmentation and grounding tasks with hierarchical region labels, ranging from organ-level boundaries to fine-grained lesions. Based on this foundation, we propose Med-GLIP, a modality-aware grounding framework trained on Med-GLIP-5M. Rather than relying on explicitly designed expert modules, Med-GLIP implicitly acquires hierarchical semantic understanding from diverse training data -- enabling it to recognize multi-granularity structures, such as distinguishing lungs from pneumonia lesions. Extensive experiments demonstrate that Med-GLIP consistently outperforms state-of-the-art baselines across multiple grounding benchmarks. Furthermore, integrating its spatial outputs into downstream tasks, including medical VQA and report generation, leads to substantial performance gains. Our dataset will be released soon.
• 摘要：医学图像接地旨在使自然语言短语与医学图像中的特定区域保持一致，这是智能诊断，视觉问答答案（VQA）和自动报告生成（MRG）的基础任务。但是，现有的研究受到有限的模态覆盖，粗粒注释以及缺乏统一的，可推广的基础框架的限制。为了应对这些挑战，我们构建了一个大规模的医学接地数据集Med-5m，其中包括七种成像方式的530万个区域级注释，涵盖了各种解剖结构和病理发现。该数据集用分层区域标签支持分割和接地任务，从器官水平的边界到细粒细胞病变。基于这个基础，我们提出了Med-Glip，这是一种在Med-Glip-5M上训练的态度感知的接地框架。 Med-Glip不依赖于明确设计的专家模块，而是隐式地从不同的培训数据中获得了层次的语义理解，从而使其能够识别多晶状体结构，例如区分肺部和肺炎病变。广泛的实验表明，Med-Glip在多个接地基准中始终优于最先进的基线。此外，将其空间输出集成到包括医疗VQA和报告生成在内的下游任务中，可带来可观的性能提高。我们的数据集将很快发布。

Title: Projected Coupled Diffusion for Test-Time Constrained Joint Generation

• Authors: Hao Luan, Yi Xian Goh, See-Kiong Ng, Chun Kai Ling
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2508.10531
• Pdf URL: https://arxiv.org/pdf/2508.10531
• Copy Paste: [[2508.10531]] Projected Coupled Diffusion for Test-Time Constrained Joint Generation(https://arxiv.org/abs/2508.10531)
• Keywords: generation, generative
• Abstract: Modifications to test-time sampling have emerged as an important extension to diffusion algorithms, with the goal of biasing the generative process to achieve a given objective without having to retrain the entire diffusion model. However, generating jointly correlated samples from multiple pre-trained diffusion models while simultaneously enforcing task-specific constraints without costly retraining has remained challenging. To this end, we propose Projected Coupled Diffusion (PCD), a novel test-time framework for constrained joint generation. PCD introduces a coupled guidance term into the generative dynamics to encourage coordination between diffusion models and incorporates a projection step at each diffusion step to enforce hard constraints. Empirically, we demonstrate the effectiveness of PCD in application scenarios of image-pair generation, object manipulation, and multi-robot motion planning. Our results show improved coupling effects and guaranteed constraint satisfaction without incurring excessive computational costs.
• 摘要：对测试时间采样的修改已成为扩散算法的重要扩展，目的是偏向生成过程以实现给定目标，而无需重新培训整个扩散模型。但是，从多个预训练的扩散模型中生成共同关联的样品，同时执行特定于任务的约束而没有昂贵的重新培训仍然具有挑战性。为此，我们提出了预测的耦合扩散（PCD），这是一个新型的测试时间框架，用于受约束的关节产生。 PCD将一个耦合的指导项引入生成动力学，以鼓励扩散模型之间的协调，并在每个扩散步骤中结合一个投影步骤，以执行硬性约束。从经验上讲，我们证明了PCD在图像对生成，对象操纵和多机器人运动计划的应用方案中的有效性。我们的结果表明，耦合效果的改善和保证的约束满意度，而不会产生过多的计算成本。

Title: AR Surgical Navigation With Surface Tracing: Comparing In-SitVisualization with Tool-Tracking Guidance for Neurosurgical Applications

• Authors: Marc J. Fischer, Jeffrey Potts, Gabriel Urreola, Dax Jones, Paolo Palmisciano, E. Bradley Strong, Branden Cord, Andrew D. Hernandez, Julia D. Sharma, E. Brandon Strong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10554
• Pdf URL: https://arxiv.org/pdf/2508.10554
• Copy Paste: [[2508.10554]] AR Surgical Navigation With Surface Tracing: Comparing In-SitVisualization with Tool-Tracking Guidance for Neurosurgical Applications(https://arxiv.org/abs/2508.10554)
• Keywords: generation
• Abstract: Augmented Reality (AR) surgical navigation systems are emerging as the next generation of intraoperative surgical guidance, promising to overcome limitations of traditional navigation systems. However, known issues with AR depth perception due to vergence-accommodation conflict and occlusion handling limitations of the currently commercially available display technology present acute challenges in surgical settings where precision is paramount. This study presents a novel methodology for utilizing AR guidance to register anatomical targets and provide real-time instrument navigation using placement of simulated external ventricular drain catheters on a phantom model as the clinical scenario. The system registers target positions to the patient through a novel surface tracing method and uses real-time infrared tool tracking to aid in catheter placement, relying only on the onboard sensors of the Microsoft HoloLens 2. A group of intended users performed the procedure of simulated insertions under two AR guidance conditions: static in-situ visualization, where planned trajectories are overlaid directly onto the patient anatomy, and real-time tool-tracking guidance, where live feedback of the catheter's pose is provided relative to the plan. Following the insertion tests, computed tomography scans of the phantom models were acquired, allowing for evaluation of insertion accuracy, target deviation, angular error, and depth precision. System Usability Scale surveys assessed user experience and cognitive workload. Tool-tracking guidance improved performance metrics across all accuracy measures and was preferred by users in subjective evaluations. A free copy of this paper and all supplemental materials are available at this https URL.
• 摘要：增强现实（AR）手术导航系统已成为下一代术中手术指导，有望克服传统导航系统的局限性。然而，由于临时 - 住宿冲突和遮挡处理局限性，当前可用的显示技术的限制引起的已知问题，在精度至关重要的外科手术环境中，急性挑战。这项研究提出了一种新的方法，用于利用AR指导注册解剖目标，并使用在幻影模型上放置模拟外部心室漏极导管作为临床场景，提供实时仪器导航。该系统通过一种新型的表面跟踪方法对患者进行了目标位置，并使用实时红外工具跟踪来帮助导管放置，仅依靠Microsoft Hololens 2的机载传感器2。相对于计划提供了导管姿势的实时反馈。经过插入测试，获取了幻影模型的计算机断层扫描扫描，以评估插入精度，目标偏差，角度误差和深度精度。系统可用性量表调查评估了用户体验和认知工作量。工具跟踪指导改善了所有准确度措施的性能指标，并且在主观评估中受到了用户的优势。该纸张和所有补充材料的免费副本均在此HTTPS URL上获得。

Title: HM-Talker: Hybrid Motion Modeling for High-Fidelity Talking Head Synthesis

• Authors: Shiyu Liu, Kui Jiang, Xianming Liu, Hongxun Yao, Xiaocheng Feng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10566
• Pdf URL: https://arxiv.org/pdf/2508.10566
• Copy Paste: [[2508.10566]] HM-Talker: Hybrid Motion Modeling for High-Fidelity Talking Head Synthesis(https://arxiv.org/abs/2508.10566)
• Keywords: generation
• Abstract: Audio-driven talking head video generation enhances user engagement in human-computer interaction. However, current methods frequently produce videos with motion blur and lip jitter, primarily due to their reliance on implicit modeling of audio-facial motion correlations--an approach lacking explicit articulatory priors (i.e., anatomical guidance for speech-related facial movements). To overcome this limitation, we propose HM-Talker, a novel framework for generating high-fidelity, temporally coherent talking heads. HM-Talker leverages a hybrid motion representation combining both implicit and explicit motion cues. Explicit cues use Action Units (AUs), anatomically defined facial muscle movements, alongside implicit features to minimize phoneme-viseme misalignment. Specifically, our Cross-Modal Disentanglement Module (CMDM) extracts complementary implicit/explicit motion features while predicting AUs directly from audio input aligned to visual cues. To mitigate identity-dependent biases in explicit features and enhance cross-subject generalization, we introduce the Hybrid Motion Modeling Module (HMMM). This module dynamically merges randomly paired implicit/explicit features, enforcing identity-agnostic learning. Together, these components enable robust lip synchronization across diverse identities, advancing personalized talking head synthesis. Extensive experiments demonstrate HM-Talker's superiority over state-of-the-art methods in visual quality and lip-sync accuracy.
• 摘要：音频驱动的会说话的主视频生成增强了用户参与人类计算机交互。但是，当前的方法经常产生带有运动模糊和唇部抖动的视频，这主要是由于它们依赖于音频 - 种族运动相关性的隐式建模 - 一种缺乏明确的关节先验的方法（即对语音相关的面部运动的解剖学指南）。为了克服这一局限性，我们提出了HM-Talker，这是一个新型框架，用于产生高保真，暂时的说话头。 HM-TAKER利用混合运动表示，结合了隐式和显式运动提示。显式提示使用动作单元（AUS），解剖学上定义的面部肌肉运动以及隐式特征，以最大程度地减少音素 - 视觉不对对准。具体而言，我们的跨模式分解模块（CMDM）提取了互补的隐式/显式运动特征，同时直接从与视觉提示相符的音频输入直接预测AUS。为了减轻显式特征的身份依赖性偏差并增强了横向概括，我们引入了混合运动建模模块（HMMM）。该模块动态合并了随机配对的隐式/显式特征，从而实现身份 - 静态学习。这些组件共同使跨不同身份的强大唇部同步，从而推进了个性化的会说话的头部合成。广泛的实验表明，HM-Talker优于视觉质量和唇部同步准确性的最先进方法的优势。

Title: Self-Supervised Temporal Super-Resolution of Energy Data using Generative Adversarial Transformer

• Authors: Xuanhao Mu, Gökhan Demirel, Yuzhe Zhang, Jianlei Liu, Thorsten Schlachter, Veit Hagenmeyer
• Subjects: cs.LG, eess.SP, math.NA
• Abstract URL: https://arxiv.org/abs/2508.10587
• Pdf URL: https://arxiv.org/pdf/2508.10587
• Copy Paste: [[2508.10587]] Self-Supervised Temporal Super-Resolution of Energy Data using Generative Adversarial Transformer(https://arxiv.org/abs/2508.10587)
• Keywords: super-resolution, generation, generative
• Abstract: To bridge the temporal granularity gap in energy network design and operation based on Energy System Models, resampling of time series is required. While conventional upsampling methods are computationally efficient, they often result in significant information loss or increased noise. Advanced models such as time series generation models, Super-Resolution models and imputation models show potential, but also face fundamental challenges. The goal of time series generative models is to learn the distribution of the original data to generate high-resolution series with similar statistical characteristics. This is not entirely consistent with the definition of upsampling. Time series Super-Resolution models or imputation models can degrade the accuracy of upsampling because the input low-resolution time series are sparse and may have insufficient context. Moreover, such models usually rely on supervised learning paradigms. This presents a fundamental application paradox: their training requires the high-resolution time series that is intrinsically absent in upsampling application scenarios. To address the mentioned upsampling issue, this paper introduces a new method utilizing Generative Adversarial Transformers (GATs), which can be trained without access to any ground-truth high-resolution data. Compared with conventional interpolation methods, the introduced method can reduce the root mean square error (RMSE) of upsampling tasks by 9%, and the accuracy of a model predictive control (MPC) application scenario is improved by 13%.
• 摘要：为了弥合基于能量系统模型的能量网络设计和操作中的时间粒度差距，需要重新采样时间序列。尽管常规的上采样方法在计算上是有效的，但它们通常会导致大量信息丢失或增加噪声。高级模型，例如时间序列生成模型，超分辨率模型和插补模型，也会显示出潜在的，但也面临着根本的挑战。时间序列生成模型的目标是学习原始数据的分布，以生成具有相似统计特征的高分辨率系列。这与上采样的定义并不完全一致。时间序列的超分辨率模型或归纳模型可以降低UP采样的准确性，因为输入低分辨率时间序列稀疏并且可能没有足够的上下文。此外，此类模型通常依赖于监督的学习范例。这是一个基本的应用悖论：他们的培训需要高分辨率的时间序列，该时间序列在UPPLIPS采样应用程序场景中本质上不存在。为了解决提到的上采样问题，本文介绍了一种利用生成对抗变压器（GAT）的新方法，该方法可以训练，而无需访问任何地面真相高分辨率数据。与常规的插值方法相比，引入的方法可以将UPS采样任务的根平方误差（RMSE）降低9％，并且模型预测控制（MPC）应用程序方案的准确性提高了13％。

Title: Fourier-Guided Attention Upsampling for Image Super-Resolution

• Authors: Daejune Choi, Youchan No, Jinhyung Lee, Duksu Kim
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.10616
• Pdf URL: https://arxiv.org/pdf/2508.10616
• Copy Paste: [[2508.10616]] Fourier-Guided Attention Upsampling for Image Super-Resolution(https://arxiv.org/abs/2508.10616)
• Keywords: super-resolution
• Abstract: We propose Frequency-Guided Attention (FGA), a lightweight upsampling module for single image super-resolution. Conventional upsamplers, such as Sub-Pixel Convolution, are efficient but frequently fail to reconstruct high-frequency details and introduce aliasing artifacts. FGA addresses these issues by integrating (1) a Fourier feature-based Multi-Layer Perceptron (MLP) for positional frequency encoding, (2) a cross-resolution Correlation Attention Layer for adaptive spatial alignment, and (3) a frequency-domain L1 loss for spectral fidelity supervision. Adding merely 0.3M parameters, FGA consistently enhances performance across five diverse super-resolution backbones in both lightweight and full-capacity scenarios. Experimental results demonstrate average PSNR gains of 0.12~0.14 dB and improved frequency-domain consistency by up to 29%, particularly evident on texture-rich datasets. Visual and spectral evaluations confirm FGA's effectiveness in reducing aliasing and preserving fine details, establishing it as a practical, scalable alternative to traditional upsampling methods.
• 摘要：我们提出了频率引导的注意（FGA），这是一种用于单图像超分辨率的轻量级上采样模块。常规的UPSPLAPLER，例如子像素卷积，是有效的，但经常无法重建高频细节并引入混叠伪像。 FGA通过集成（1）用于位置频率编码的基于傅立叶特征的多层感知器（MLP）来解决这些问题，（2）（2）用于自适应空间比对的跨分辨率相关注意力层，（3）用于频谱fidelity监督的频率域L1 L1 L1 L1损失。 FGA仅添加0.30万参数，在轻质和全容量的场景中始终提高五种不同的超分辨率骨架的性能。实验结果表明，PSNR的平均增长率为0.12〜0.14 dB，并提高了频域的一致性高达29％，尤其是在富含纹理的数据集中显而易见。视觉和光谱评估证实了FGA在降低和混溶和保存细节方面的有效性，将其确定为传统上采样方法的实用，可扩展的替代方案。

Title: Increasing the Utility of Synthetic Images through Chamfer Guidance

• Authors: Nicola Dall'Asen, Xiaofeng Zhang, Reyhane Askari Hemmat, Melissa Hall, Jakob Verbeek, Adriana Romero-Soriano, Michal Drozdzal
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10631
• Pdf URL: https://arxiv.org/pdf/2508.10631
• Copy Paste: [[2508.10631]] Increasing the Utility of Synthetic Images through Chamfer Guidance(https://arxiv.org/abs/2508.10631)
• Keywords: generation, generative
• Abstract: Conditional image generative models hold considerable promise to produce infinite amounts of synthetic training data. Yet, recent progress in generation quality has come at the expense of generation diversity, limiting the utility of these models as a source of synthetic training data. Although guidance-based approaches have been introduced to improve the utility of generated data by focusing on quality or diversity, the (implicit or explicit) utility functions oftentimes disregard the potential distribution shift between synthetic and real data. In this work, we introduce Chamfer Guidance: a training-free guidance approach which leverages a handful of real exemplar images to characterize the quality and diversity of synthetic data. We show that by leveraging the proposed Chamfer Guidance, we can boost the diversity of the generations w.r.t. a dataset of real images while maintaining or improving the generation quality on ImageNet-1k and standard geo-diversity benchmarks. Our approach achieves state-of-the-art few-shot performance with as little as 2 exemplar real images, obtaining 96.4\% in terms of precision, and 86.4\% in terms of distributional coverage, which increase to 97.5\% and 92.7\%, respectively, when using 32 real images. We showcase the benefits of the Chamfer Guidance generation by training downstream image classifiers on synthetic data, achieving accuracy boost of up to 15\% for in-distribution over the baselines, and up to 16\% in out-of-distribution. Furthermore, our approach does not require using the unconditional model, and thus obtains a 31\% reduction in FLOPs w.r.t. classifier-free-guidance-based approaches at sampling time.
• 摘要：有条件的图像生成模型具有可产生无限量的合成训练数据的巨大希望。然而，最新的发电质量进展取决于产生多样性的代价，从而将这些模型的实用性限制为合成训练数据的来源。尽管已经引入了基于指导的方法来通过关注质量或多样性来改善生成数据的实用性，但是（隐性或明确的）实用程序功能通常会忽略合成数据和真实数据之间的潜在分布变化。在这项工作中，我们介绍了倒角指导：一种无训练的指导方法，该方法利用了一些真实的示例图像来表征合成数据的质量和多样性。我们表明，通过利用拟议的倒角指导，我们可以提高W.R.T.的多样性。真实图像的数据集，同时维护或改善Imagenet-1k和标准地理多样性基准的发电质量。我们的方法以2个示例真实的图像获得了最新的几次表现，从精度上获得96.4 \％，而分布覆盖率为86.4 \％，当使用32个真实图像时，分别增加到97.5 \％和92.7 \％。我们通过培训下游图像分类器在合成数据上训练倒角引导生成的好处，从而在基准方面实现了高达15％的准确性提高15 \％，而在分布情况下的精度提高了15 \％。此外，我们的方法不需要使用无条件模型，因此可以在W.R.T.中获得31 \％的减少。在抽样时间的基于无分类器指示的方法。

Title: Geospatial Diffusion for Land Cover Imperviousness Change Forecasting

• Authors: Debvrat Varshney, Vibhas Vats, Bhartendu Pandey, Christa Brelsford, Philipe Dias
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10649
• Pdf URL: https://arxiv.org/pdf/2508.10649
• Copy Paste: [[2508.10649]] Geospatial Diffusion for Land Cover Imperviousness Change Forecasting(https://arxiv.org/abs/2508.10649)
• Keywords: generative
• Abstract: Land cover, both present and future, has a significant effect on several important Earth system processes. For example, impervious surfaces heat up and speed up surface water runoff and reduce groundwater infiltration, with concomitant effects on regional hydrology and flood risk. While regional Earth System models have increasing skill at forecasting hydrologic and atmospheric processes at high resolution in future climate scenarios, our ability to forecast land-use and land-cover change (LULC), a critical input to risk and consequences assessment for these scenarios, has lagged behind. In this paper, we propose a new paradigm exploiting Generative AI (GenAI) for land cover change forecasting by framing LULC forecasting as a data synthesis problem conditioned on historical and auxiliary data-sources. We discuss desirable properties of generative models that fundament our research premise, and demonstrate the feasibility of our methodology through experiments on imperviousness forecasting using historical data covering the entire conterminous United States. Specifically, we train a diffusion model for decadal forecasting of imperviousness and compare its performance to a baseline that assumes no change at all. Evaluation across 12 metropolitan areas for a year held-out during training indicate that for average resolutions $\geq 0.7\times0.7km^2$ our model yields MAE lower than such a baseline. This finding corroborates that such a generative model can capture spatiotemporal patterns from historical data that are significant for projecting future change. Finally, we discuss future research to incorporate auxiliary information on physical properties about the Earth, as well as supporting simulation of different scenarios by means of driver variables.
• 摘要：现在和未来的土地覆盖物对几个重要的地球系统过程都有重大影响。例如，不透水的表面加热并加快地表水径流并减少地下水浸润，并对区域水文学和洪水风险产生影响。尽管区域地球系统模型在未来的气候场景中以高分辨率预测水文和大气过程的技能越来越高，但我们预测土地使用和土地覆盖变化的能力（LULC）落后于这些情况的风险和后果评估的关键意见和后果评估。在本文中，我们提出了一种新的范式，利用生成AI（Genai），以通过将LULC预测作为数据综合问题来预测土地覆盖变化，以历史和辅助数据源为条件。我们讨论了基础我们研究前提的生成模型的理想特性，并通过使用涵盖整个综合美国的历史数据进行实验来证明我们方法的可行性。具体而言，我们训练一个扩散模型，以纪念不渗透性，并将其性能与完全没有改变的基线进行比较。在培训期间，在12个大都市地区进行的一年的评估表明，对于平均决议，$ \ geq 0.7 \ times 0.7km^2 $我们的模型的收益低于此类基线。这一发现证实了这种生成模型可以从历史数据中捕获时空模式，这对于预测未来变化很重要。最后，我们讨论了未来的研究，以结合有关地球物理特性的辅助信息，并通过驱动器变量支持对不同方案的模拟。

Title: AddressVLM: Cross-view Alignment Tuning for Image Address Localization using Large Vision-Language Models

• Authors: Shixiong Xu, Chenghao Zhang, Lubin Fan, Yuan Zhou, Bin Fan, Shiming Xiang, Gaofeng Meng, Jieping Ye
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.10667
• Pdf URL: https://arxiv.org/pdf/2508.10667
• Copy Paste: [[2508.10667]] AddressVLM: Cross-view Alignment Tuning for Image Address Localization using Large Vision-Language Models(https://arxiv.org/abs/2508.10667)
• Keywords: generation
• Abstract: Large visual language models (LVLMs) have demonstrated impressive performance in coarse-grained geo-localization at the country or city level, but they struggle with fine-grained street-level localization within urban areas. In this paper, we explore integrating city-wide address localization capabilities into LVLMs, facilitating flexible address-related question answering using street-view images. A key challenge is that the street-view visual question-and-answer (VQA) data provides only microscopic visual cues, leading to subpar performance in fine-tuned models. To tackle this issue, we incorporate perspective-invariant satellite images as macro cues and propose cross-view alignment tuning including a satellite-view and street-view image grafting mechanism, along with an automatic label generation mechanism. Then LVLM's global understanding of street distribution is enhanced through cross-view matching. Our proposed model, named AddressVLM, consists of two-stage training protocols: cross-view alignment tuning and address localization tuning. Furthermore, we have constructed two street-view VQA datasets based on image address localization datasets from Pittsburgh and San Francisco. Qualitative and quantitative evaluations demonstrate that AddressVLM outperforms counterpart LVLMs by over 9% and 12% in average address localization accuracy on these two datasets, respectively.
• 摘要：大型的视觉语言模型（LVLM）在该国或城市一级的粗粒地理定位中表现出了令人印象深刻的表现，但是它们在城市地区的细粒度街道定位挣扎。在本文中，我们探讨了将城市范围的地址本地化功能整合到LVLM中，从而促进了使用街道视图图像来回答灵活的地址相关的问题。一个关键的挑战是，街道视觉视觉问答（VQA）数据仅提供微观视觉提示，从而在微调模型中导致表现不佳。为了解决此问题，我们将视角不变的卫星图像作为宏提示，并提出跨视图对齐调整，包括卫星视图和街道视图图像移植机制，以及自动标签生成机制。然后，通过跨视图匹配，LVLM对街头分配的全球理解得到了增强。我们所提出的模型，名为addressVLM，由两阶段培训协议组成：跨视图对齐调整和地址定位调整。此外，我们基于匹兹堡和旧金山的图像地址本地化数据集构建了两个街道视图VQA数据集。定性和定量评估表明，在这两个数据集上，地址为VLM的平均地址定位精度分别超过9％和12％。

Title: Hybrid Generative Fusion for Efficient and Privacy-Preserving Face Recognition Dataset Generation

• Authors: Feiran Li, Qianqian Xu, Shilong Bao, Boyu Han, Zhiyong Yang, Qingming Huang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.10672
• Pdf URL: https://arxiv.org/pdf/2508.10672
• Copy Paste: [[2508.10672]] Hybrid Generative Fusion for Efficient and Privacy-Preserving Face Recognition Dataset Generation(https://arxiv.org/abs/2508.10672)
• Keywords: generation, generative
• Abstract: In this paper, we present our approach to the DataCV ICCV Challenge, which centers on building a high-quality face dataset to train a face recognition model. The constructed dataset must not contain identities overlapping with any existing public face datasets. To handle this challenge, we begin with a thorough cleaning of the baseline HSFace dataset, identifying and removing mislabeled or inconsistent identities through a Mixture-of-Experts (MoE) strategy combining face embedding clustering and GPT-4o-assisted verification. We retain the largest consistent identity cluster and apply data augmentation up to a fixed number of images per identity. To further diversify the dataset, we generate synthetic identities using Stable Diffusion with prompt engineering. As diffusion models are computationally intensive, we generate only one reference image per identity and efficiently expand it using Vec2Face, which rapidly produces 49 identity-consistent variants. This hybrid approach fuses GAN-based and diffusion-based samples, enabling efficient construction of a diverse and high-quality dataset. To address the high visual similarity among synthetic identities, we adopt a curriculum learning strategy by placing them early in the training schedule, allowing the model to progress from easier to harder samples. Our final dataset contains 50 images per identity, and all newly generated identities are checked with mainstream face datasets to ensure no identity leakage. Our method achieves \textbf{1st place} in the competition, and experimental results show that our dataset improves model performance across 10K, 20K, and 100K identity scales. Code is available at this https URL.
• 摘要：在本文中，我们介绍了DataCV ICCV挑战的方法，该挑战的目的是构建高质量的面部数据集以训练面部识别模型。构造的数据集不得包含与任何现有公共面部数据集重叠的身份。为了应对这一挑战，我们首先要彻底清洁基线HSFACE数据集，通过结合嵌入群集和GPT-4O辅助验证的面部嵌入式嵌入式嵌入式嵌入式嵌入式的Experts（MOE）策略来识别和删除标记错误或不一致的身份。我们保留最大的一致身份群集，并将数据扩展应用于每个身份的固定数量图像。为了进一步使数据集多样化，我们使用迅速的工程使用稳定的扩散生成合成身份。随着扩散模型在计算密集型上，我们仅在标识下生成一个参考图像，并使用vec2face有效地扩展了它，Vec2face迅速产生了49个身份一致的变体。这种混合方法融合了基于GAN和基于扩散的样品，从而有效地构建了多样化和高质量的数据集。为了解决综合身份之间高的视觉相似性，我们通过在培训时间表的早期将其放置来采用课程学习策略，从而使模型从易于易于样本到更艰难的样本。我们的最终数据集每个身份包含50张图像，并使用主流面部数据集检查所有新生成的身份，以确保没有身份泄漏。我们的方法在竞争中实现了\ textbf {1st plot}，实验结果表明，我们的数据集改善了跨10K，20K和100K身份量表的模型性能。代码可在此HTTPS URL上找到。

Title: Physics-Informed Joint Multi-TE Super-Resolution with Implicit Neural Representation for Robust Fetal T2 Mapping

• Authors: Busra Bulut, Maik Dannecker, Thomas Sanchez, Sara Neves Silva, Vladyslav Zalevskyi, Steven Jia, Jean-Baptiste Ledoux, Guillaume Auzias, François Rousseau, Jana Hutter, Daniel Rueckert, Meritxell Bach Cuadra
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10680
• Pdf URL: https://arxiv.org/pdf/2508.10680
• Copy Paste: [[2508.10680]] Physics-Informed Joint Multi-TE Super-Resolution with Implicit Neural Representation for Robust Fetal T2 Mapping(https://arxiv.org/abs/2508.10680)
• Keywords: super-resolution
• Abstract: T2 mapping in fetal brain MRI has the potential to improve characterization of the developing brain, especially at mid-field (0.55T), where T2 decay is slower. However, this is challenging as fetal MRI acquisition relies on multiple motion-corrupted stacks of thick slices, requiring slice-to-volume reconstruction (SVR) to estimate a high-resolution (HR) 3D volume. Currently, T2 mapping involves repeated acquisitions of these stacks at each echo time (TE), leading to long scan times and high sensitivity to motion. We tackle this challenge with a method that jointly reconstructs data across TEs, addressing severe motion. Our approach combines implicit neural representations with a physics-informed regularization that models T2 decay, enabling information sharing across TEs while preserving anatomical and quantitative T2 fidelity. We demonstrate state-of-the-art performance on simulated fetal brain and in vivo adult datasets with fetal-like motion. We also present the first in vivo fetal T2 mapping results at 0.55T. Our study shows potential for reducing the number of stacks per TE in T2 mapping by leveraging anatomical redundancy.
• 摘要：胎儿脑MRI中的T2映射具有改善发育中大脑的表征，尤其是在中场（0.55T）的表征，其中T2衰减较慢。但是，这是具有挑战性的，因为胎儿MRI获取依赖于多个运动腐败的厚切片，需要切片到体积的重建（SVR）来估计高分辨率（HR）3D体积。当前，T2映射涉及在每个回声时间（TE）的这些堆栈的重复采集，从而导致扫描时间很长，并且对运动的敏感性很高。我们通过一种可以共同重建TE的数据，解决严重运动的方法来应对这一挑战。我们的方法将隐式神经表示与物理信息的正则化结合了T2衰减模型，从而在跨TES上共享信息，同时保留解剖学和定量T2保真度。我们展示了具有类似胎儿运动的模拟胎儿大脑和体内成人数据集的最先进性能。我们还提出了第一个在0.55T的体内胎儿T2映射结果。我们的研究表明，通过利用解剖学冗余，在T2映射中减少每TE堆栈数量的潜力。

Title: Novel View Synthesis using DDIM Inversion

• Authors: Sehajdeep SIngh, A V Subramanyam
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10688
• Pdf URL: https://arxiv.org/pdf/2508.10688
• Copy Paste: [[2508.10688]] Novel View Synthesis using DDIM Inversion(https://arxiv.org/abs/2508.10688)
• Keywords: generative
• Abstract: Synthesizing novel views from a single input image is a challenging task. It requires extrapolating the 3D structure of a scene while inferring details in occluded regions, and maintaining geometric consistency across viewpoints. Many existing methods must fine-tune large diffusion backbones using multiple views or train a diffusion model from scratch, which is extremely expensive. Additionally, they suffer from blurry reconstruction and poor generalization. This gap presents the opportunity to explore an explicit lightweight view translation framework that can directly utilize the high-fidelity generative capabilities of a pretrained diffusion model while reconstructing a scene from a novel view. Given the DDIM-inverted latent of a single input image, we employ a camera pose-conditioned translation U-Net, TUNet, to predict the inverted latent corresponding to the desired target view. However, the image sampled using the predicted latent may result in a blurry reconstruction. To this end, we propose a novel fusion strategy that exploits the inherent noise correlation structure observed in DDIM inversion. The proposed fusion strategy helps preserve the texture and fine-grained details. To synthesize the novel view, we use the fused latent as the initial condition for DDIM sampling, leveraging the generative prior of the pretrained diffusion model. Extensive experiments on MVImgNet demonstrate that our method outperforms existing methods.
• 摘要：从单个输入图像中综合新视图是一项具有挑战性的任务。它需要推断场景的3D结构，同时推断遮挡区域中的细节，并在跨角度保持几何一致性。许多现有的方法必须使用多个视图微调大扩散骨干，或从头开始训练扩散模型，这非常昂贵。此外，他们患有模糊的重建和概括不良。这个差距为探索明确的轻巧视图翻译框架提供了机会，该框架可以直接利用预审预定的扩散模型的高保真生成能力，同时从新颖的视图中重建场景。鉴于单个输入图像的DDIM向内偏向潜在的潜在，我们采用了相机姿势调节的翻译U-NET TUNET，以预测与所需目标视图相对应的倒立潜在。但是，使用预测潜伏的图像可能会导致模糊重建。为此，我们提出了一种新型的融合策略，该策略利用了在DDIM倒置中观察到的固有噪声相关结构。提出的融合策略有助于保留纹理和细粒细节。为了综合新的视图，我们将融合的潜在用作DDIM采样的初始条件，利用了预算扩散模型的生成性先验。对MVIMGNET的广泛实验表明，我们的方法表现优于现有方法。

Title: CountCluster: Training-Free Object Quantity Guidance with Cross-Attention Map Clustering for Text-to-Image Generation

• Authors: Joohyeon Lee, Jin-Seop Lee, Jee-Hyong Lee
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10710
• Pdf URL: https://arxiv.org/pdf/2508.10710
• Copy Paste: [[2508.10710]] CountCluster: Training-Free Object Quantity Guidance with Cross-Attention Map Clustering for Text-to-Image Generation(https://arxiv.org/abs/2508.10710)
• Keywords: generation
• Abstract: Diffusion-based text-to-image generation models have demonstrated strong performance in terms of image quality and diversity. However, they still struggle to generate images that accurately reflect the number of objects specified in the input prompt. Several approaches have been proposed that rely on either external counting modules for iterative refinement or quantity representations derived from learned tokens or latent features. However, they still have limitations in accurately reflecting the specified number of objects and overlook an important structural characteristic--The number of object instances in the generated image is largely determined in the early timesteps of the denoising process. To correctly reflect the object quantity for image generation, the highly activated regions in the object cross-attention map at the early timesteps should match the input object quantity, while each region should be clearly separated. To address this issue, we propose \textit{CountCluster}, a method that guides the object cross-attention map to be clustered according to the specified object count in the input, without relying on any external tools or additional training. The proposed method partitions the object cross-attention map into $k$ clusters at inference time based on attention scores, defines an ideal distribution in which each cluster is spatially well-separated, and optimizes the latent to align with this target distribution. Our method achieves an average improvement of 18.5\%p in object count accuracy compared to existing methods, and demonstrates superior quantity control performance across a variety of prompts. Code will be released at: this https URL .
• 摘要：基于扩散的文本到图像生成模型在图像质量和多样性方面表现出了强劲的性能。但是，他们仍然很难生成准确反映输入提示中指定的对象数量的图像。已经提出了几种方法，这些方法依靠外部计数模块来进行迭代改进或从学识关的代币或潜在特征得出的数量表示。但是，它们仍然有局限性在准确反映指定的对象数量并忽略重要的结构特征时存在局限性 - 生成图像中的对象实例数在很大程度上是在DeNoising过程的早期时间段中确定的。要正确反映图像生成的对象数量，在早期时间步中，对象跨注意图中的高度激活区域应与输入对象数量匹配，而每个区域应明确分开。为了解决此问题，我们提出\ textIt {CountCluster}，该方法可以根据输入中的指定对象计数来指导对象交叉注意地图，而无需依赖任何外部工具或其他培训。所提出的方法根据注意力评分在推理时间将对象跨注意事项映射分为$ k $簇，定义一个理想的分布，其中每个群集在空间上是良好的分离，并优化了与此目标分布保持一致的潜在分布。与现有方法相比，我们的方法在物体计数准确性中的平均改善达到18.5 \％p，并在各种提示中证明了较高的数量控制性能。代码将在以下位置发布：此HTTPS URL。

Title: NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scale

• Authors: NextStep Team: Chunrui Han, Guopeng Li, Jingwei Wu, Quan Sun, Yan Cai, Yuang Peng, Zheng Ge, Deyu Zhou, Haomiao Tang, Hongyu Zhou, Kenkun Liu, Ailin Huang, Bin Wang, Changxin Miao, Deshan Sun, En Yu, Fukun Yin, Gang Yu, Hao Nie, Haoran Lv, Hanpeng Hu, Jia Wang, Jian Zhou, Jianjian Sun, Kaijun Tan, Kang An, Kangheng Lin, Liang Zhao, Mei Chen, Peng Xing, Rui Wang, Shiyu Liu, Shutao Xia, Tianhao You, Wei Ji, Xianfang Zeng, Xin Han, Xuelin Zhang, Yana Wei, Yanming Xu, Yimin Jiang, Yingming Wang, Yu Zhou, Yucheng Han, Ziyang Meng, Binxing Jiao, Daxin Jiang, Xiangyu Zhang, Yibo Zhu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10711
• Pdf URL: https://arxiv.org/pdf/2508.10711
• Copy Paste: [[2508.10711]] NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scale(https://arxiv.org/abs/2508.10711)
• Keywords: generation
• Abstract: Prevailing autoregressive (AR) models for text-to-image generation either rely on heavy, computationally-intensive diffusion models to process continuous image tokens, or employ vector quantization (VQ) to obtain discrete tokens with quantization loss. In this paper, we push the autoregressive paradigm forward with NextStep-1, a 14B autoregressive model paired with a 157M flow matching head, training on discrete text tokens and continuous image tokens with next-token prediction objectives. NextStep-1 achieves state-of-the-art performance for autoregressive models in text-to-image generation tasks, exhibiting strong capabilities in high-fidelity image synthesis. Furthermore, our method shows strong performance in image editing, highlighting the power and versatility of our unified approach. To facilitate open research, we will release our code and models to the community.
• 摘要：盛行文本对图像生成的自动回收（AR）模型依赖于重型，计算密集型扩散模型来处理连续的图像令牌，或采用矢量量化（VQ）来获得具有量化损失的离散令牌。在本文中，我们将自回归范式推向NextStep-1，这是14B自回归型号，配对157m流量匹配的头部，对离散文本令牌进行训练以及连续的图像令牌，并具有下一步的预测目标。 NextStep-1在文本到图像生成任务中实现自回归模型的最新性能，在高保真图像合成中表现出强大的功能。此外，我们的方法在图像编辑中显示出强烈的性能，突出了我们统一方法的功能和多功能性。为了促进开放研究，我们将向社区发布代码和模型。

Title: Exploiting Discriminative Codebook Prior for Autoregressive Image Generation

• Authors: Longxiang Tang, Ruihang Chu, Xiang Wang, Yujin Han, Pingyu Wu, Chunming He, Yingya Zhang, Shiwei Zhang, Jiaya Jia
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10719
• Pdf URL: https://arxiv.org/pdf/2508.10719
• Copy Paste: [[2508.10719]] Exploiting Discriminative Codebook Prior for Autoregressive Image Generation(https://arxiv.org/abs/2508.10719)
• Keywords: generation, generative
• Abstract: Advanced discrete token-based autoregressive image generation systems first tokenize images into sequences of token indices with a codebook, and then model these sequences in an autoregressive paradigm. While autoregressive generative models are trained only on index values, the prior encoded in the codebook, which contains rich token similarity information, is not exploited. Recent studies have attempted to incorporate this prior by performing naive k-means clustering on the tokens, helping to facilitate the training of generative models with a reduced codebook. However, we reveal that k-means clustering performs poorly in the codebook feature space due to inherent issues, including token space disparity and centroid distance inaccuracy. In this work, we propose the Discriminative Codebook Prior Extractor (DCPE) as an alternative to k-means clustering for more effectively mining and utilizing the token similarity information embedded in the codebook. DCPE replaces the commonly used centroid-based distance, which is found to be unsuitable and inaccurate for the token feature space, with a more reasonable instance-based distance. Using an agglomerative merging technique, it further addresses the token space disparity issue by avoiding splitting high-density regions and aggregating low-density ones. Extensive experiments demonstrate that DCPE is plug-and-play and integrates seamlessly with existing codebook prior-based paradigms. With the discriminative prior extracted, DCPE accelerates the training of autoregressive models by 42% on LlamaGen-B and improves final FID and IS performance.
• 摘要：先进的离散代币基于自回旋图像生成系统首先将图像将图像带入代码手册中的令牌索引序列，然后在自动回归范式中对这些序列进行建模。尽管仅对索引值培训自回旋生成模型，但未利用代码手册中的先验编码，其中包含丰富的令牌相似性信息。最近的研究试图通过在代币上进行幼稚的K-均值聚类来纳入此之前，从而有助于通过减少的代码书来促进生成模型的培训。但是，我们透露，由于固有的问题，包括令牌空间差异和质心距离不准确，K均值集群在代码书功能空间中的性能很差。在这项工作中，我们提出了歧视性代码书前提取器（DCPE），以替代K-均值聚类，以更有效地采矿并利用代码书中嵌入的令牌相似性信息。 DCPE取代了常用的基于质心的距离，该距离被认为是不合适的，对于令牌特征空间，基于实例的距离更合理。使用集聚合并技术，它通过避免分裂高密度区域并汇总低密度的空间差异问题进一步解决了令牌空间差异问题。广泛的实验表明，DCPE正在插入插件，并与现有的基于基于代码的范式无缝集成。随着判别性提取的提取，DCPE在Lamagen-B上加速了42％的自回归模型的训练，并改善了最终FID，并且是性能。

Title: Privacy-enhancing Sclera Segmentation Benchmarking Competition: SSBC 2025

• Authors: Matej Vitek, Darian Tomašević, Abhijit Das, Sabari Nathan, Gökhan Özbulak, Gözde Ayşe Tataroğlu Özbulak, Jean-Paul Calbimonte, André Anjos, Hariohm Hemant Bhatt, Dhruv Dhirendra Premani, Jay Chaudhari, Caiyong Wang, Jian Jiang, Chi Zhang, Qi Zhang, Iyyakutti Iyappan Ganapathi, Syed Sadaf Ali, Divya Velayudan, Maregu Assefa, Naoufel Werghi, Zachary A. Daniels, Leeon John, Ritesh Vyas, Jalil Nourmohammadi Khiarak, Taher Akbari Saeed, Mahsa Nasehi, Ali Kianfar, Mobina Pashazadeh Panahi, Geetanjali Sharma, Pushp Raj Panth, Raghavendra Ramachandra, Aditya Nigam, Umapada Pal, Peter Peer, Vitomir Štruc
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10737
• Pdf URL: https://arxiv.org/pdf/2508.10737
• Copy Paste: [[2508.10737]] Privacy-enhancing Sclera Segmentation Benchmarking Competition: SSBC 2025(https://arxiv.org/abs/2508.10737)
• Keywords: generative
• Abstract: This paper presents a summary of the 2025 Sclera Segmentation Benchmarking Competition (SSBC), which focused on the development of privacy-preserving sclera-segmentation models trained using synthetically generated ocular images. The goal of the competition was to evaluate how well models trained on synthetic data perform in comparison to those trained on real-world datasets. The competition featured two tracks: $(i)$ one relying solely on synthetic data for model development, and $(ii)$ one combining/mixing synthetic with (a limited amount of) real-world data. A total of nine research groups submitted diverse segmentation models, employing a variety of architectural designs, including transformer-based solutions, lightweight models, and segmentation networks guided by generative frameworks. Experiments were conducted across three evaluation datasets containing both synthetic and real-world images, collected under diverse conditions. Results show that models trained entirely on synthetic data can achieve competitive performance, particularly when dedicated training strategies are employed, as evidenced by the top performing models that achieved $F_1$ scores of over $0.8$ in the synthetic data track. Moreover, performance gains in the mixed track were often driven more by methodological choices rather than by the inclusion of real data, highlighting the promise of synthetic data for privacy-aware biometric development. The code and data for the competition is available at: this https URL.
• 摘要：本文介绍了2025年巩膜分割基准测试竞赛（SSBC）的摘要，该竞赛的重点是开发使用合成生成的眼部图像训练的培训的隐私巩膜分割模型。竞争的目的是评估与在现实世界数据集中培训的模型相比，对合成数据训练的模型的表现良好。该竞赛有两首曲目：$（i）$一，仅依靠合成数据进行模型开发，以及$（ii）$（II）与（有限数量的）现实世界数据组合/混合合成。总共九个研究小组提交了各种架构设计，包括基于变压器的解决方案，轻质模型和由生成框架引导的细分网络。实验是在包含在不同条件下收集的合成图像和现实世界图像的三个评估数据集进行的。结果表明，完全根据合成数据培训的模型可以实现竞争性能，尤其是在采用专用培训策略时，在合成数据轨道中获得了$ f_1 $ 0.8美元的最高表现模型证明。此外，混合轨道中的性能增长通常更多地由方法论选择驱动，而不是包含真实数据，强调了合成数据对隐私感知的生物识别开发的承诺。竞争的代码和数据可在以下网址提供：此HTTPS URL。

Title: AEGIS: Authenticity Evaluation Benchmark for AI-Generated Video Sequences

• Authors: Jieyu Li, Xin Zhang, Joey Tianyi Zhou
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.10771
• Pdf URL: https://arxiv.org/pdf/2508.10771
• Copy Paste: [[2508.10771]] AEGIS: Authenticity Evaluation Benchmark for AI-Generated Video Sequences(https://arxiv.org/abs/2508.10771)
• Keywords: generative
• Abstract: Recent advances in AI-generated content have fueled the rise of highly realistic synthetic videos, posing severe risks to societal trust and digital integrity. Existing benchmarks for video authenticity detection typically suffer from limited realism, insufficient scale, and inadequate complexity, failing to effectively evaluate modern vision-language models against sophisticated forgeries. To address this critical gap, we introduce AEGIS, a novel large-scale benchmark explicitly targeting the detection of hyper-realistic and semantically nuanced AI-generated videos. AEGIS comprises over 10,000 rigorously curated real and synthetic videos generated by diverse, state-of-the-art generative models, including Stable Video Diffusion, CogVideoX-5B, KLing, and Sora, encompassing open-source and proprietary architectures. In particular, AEGIS features specially constructed challenging subsets enhanced with robustness evaluation. Furthermore, we provide multimodal annotations spanning Semantic-Authenticity Descriptions, Motion Features, and Low-level Visual Features, facilitating authenticity detection and supporting downstream tasks such as multimodal fusion and forgery localization. Extensive experiments using advanced vision-language models demonstrate limited detection capabilities on the most challenging subsets of AEGIS, highlighting the dataset's unique complexity and realism beyond the current generalization capabilities of existing models. In essence, AEGIS establishes an indispensable evaluation benchmark, fundamentally advancing research toward developing genuinely robust, reliable, broadly generalizable video authenticity detection methodologies capable of addressing real-world forgery threats. Our dataset is available on this https URL.
• 摘要：AI生成的内容的最新进展推动了高度现实的合成视频的兴起，对社会信任和数字完整性构成了严重的风险。现有的视频真实性检测基准通常会遭受有限的现实主义，不足的规模和不足的复杂性，因此无法有效评估现代视觉模型，以针对复杂的伪造。为了解决这个关键的差距，我们介绍了一种新颖的大规模基准，明确针对检测超现实和语义上细微的AI生成的视频。宙斯盾包括超过10,000多个由多种最先进的生成模型产生的严格策划的真实和合成视频，包括稳定的视频扩散，Cogvideox-5B，Kling和Sora，包括开源和专有体系结构。特别是，宙斯盾具有特殊构建的挑战子集，并通过鲁棒性评估增强。此外，我们提供了跨越语义真实性描述，运动特征和低级视觉特征的多模式注释，促进了真实性检测并支持下游任务，例如多模式融合和伪造定位。使用高级视觉模型的广泛实验表明，在宙斯盾最具挑战性的子集上，有限的检测能力有限，突出了数据集的独特复杂性和现实主义，超出了现有模型的当前通用能力。本质上，宙斯盾建立了必不可少的评估基准，从根本上推进了研究，以发展能够解决现实世界中伪造威胁的真正坚固，可靠，可广泛的视频真实性检测方法。我们的数据集可在此HTTPS URL上找到。

Title: Video-BLADE: Block-Sparse Attention Meets Step Distillation for Efficient Video Generation

• Authors: Youping Gu, Xiaolong Li, Yuhao Hu, Bohan Zhuang
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.10774
• Pdf URL: https://arxiv.org/pdf/2508.10774
• Copy Paste: [[2508.10774]] Video-BLADE: Block-Sparse Attention Meets Step Distillation for Efficient Video Generation(https://arxiv.org/abs/2508.10774)
• Keywords: generation
• Abstract: Diffusion transformers currently lead the field in high-quality video generation, but their slow iterative denoising process and prohibitive quadratic attention costs for long sequences create significant inference bottlenecks. While both step distillation and sparse attention mechanisms have shown promise as independent acceleration strategies, effectively combining these approaches presents critical challenges -- training-free integration yields suboptimal results, while separately training sparse attention after step distillation requires prohibitively expensive high-quality video data. To overcome these limitations, we propose BLADE, an innovative data-free joint training framework that introduces: (1) an Adaptive Block-Sparse Attention (ASA) mechanism for dynamically generating content-aware sparsity masks to focus computation on salient spatiotemporal features, and (2) a sparsity-aware step distillation paradigm built upon Trajectory Distribution Matching (TDM) that directly incorporates sparsity into the distillation process rather than treating it as a separate compression step, with fast convergence. We validate BLADE on text-to-video models like CogVideoX-5B and Wan2.1-1.3B. Our framework demonstrates remarkable efficiency gains across different scales. On Wan2.1-1.3B, BLADE achieves a 14.10x end-to-end inference acceleration over a 50-step baseline. Moreover, on models such as CogVideoX-5B with short video sequence lengths, our framework delivers a robust 8.89x speedup. Crucially, the acceleration is accompanied by a consistent quality improvement. On the VBench-2.0 benchmark, BLADE boosts the score of CogVideoX-5B to 0.569 (from 0.534) and Wan2.1-1.3B to 0.570 (from 0.563), results that are further corroborated by superior ratings in human evaluations. Our code and model weights are publicly available at: this http URL.
• 摘要：扩散变压器目前以高质量的视频生成领域领域，但是长序列的较慢的迭代降解过程和过度的二次注意力成本会产生明显的推理瓶颈。尽管步骤蒸馏和稀疏的注意机制都表现出有望作为独立加速策略的希望，但有效地结合了这些方法，提出了关键的挑战 - 无训练的整合会产生次优的结果，而单独培训稀疏的逐步蒸馏需要较昂贵的昂贵的高质量高质量视频数据。为了克服这些局限性，我们提出了刀片，这是一个无数据的联合培训框架，介绍了：（1）一种适应性的块 - 宽带注意（ASA）机制，用于动态生成内容感知的稀疏性掩盖，以将计算重点放在显着的时空特征上，以及（2）sparsity-aware aware sparsity sparsitige sparsitige sparsitige atige toctory（2）蒸馏过程而不是将其视为具有快速收敛的单独压缩步骤。我们在文本到视频模型（如Cogvideox-5b和WAN2.1-1.3B）上验证刀片。我们的框架表明了不同尺度上的效率取得的显着提高。在WAN2.1-1.3B上，刀片在50步基线上达到了14.10倍的端到端推理加速度。此外，在诸如具有短视频序列长度的cogvideox-5b之类的模型上，我们的框架可提供强大的8.89倍加速。至关重要的是，加速度伴随着一致的质量改进。在VBENCH-2.0基准上，刀片将Cogvideox-5b的得分提高到0.569（从0.534）和WAN2.1-1.3B到0.570（从0.563），结果在人类评估中进一步证实了卓越等级的结果。我们的代码和模型权重可公开可用：此HTTP URL。

Title: IBEX: Information-Bottleneck-EXplored Coarse-to-Fine Molecular Generation under Limited Data

• Authors: Dong Xu, Zhangfan Yang, Jenna Xinyi Yao, Shuangbao Song, Zexuan Zhu, Junkai Ji
• Subjects: cs.LG, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2508.10775
• Pdf URL: https://arxiv.org/pdf/2508.10775
• Copy Paste: [[2508.10775]] IBEX: Information-Bottleneck-EXplored Coarse-to-Fine Molecular Generation under Limited Data(https://arxiv.org/abs/2508.10775)
• Keywords: generation, generative
• Abstract: Three-dimensional generative models increasingly drive structure-based drug discovery, yet it remains constrained by the scarce publicly available protein-ligand complexes. Under such data scarcity, almost all existing pipelines struggle to learn transferable geometric priors and consequently overfit to training-set biases. As such, we present IBEX, an Information-Bottleneck-EXplored coarse-to-fine pipeline to tackle the chronic shortage of protein-ligand complex data in structure-based drug design. Specifically, we use PAC-Bayesian information-bottleneck theory to quantify the information density of each sample. This analysis reveals how different masking strategies affect generalization and indicates that, compared with conventional de novo generation, the constrained Scaffold Hopping task endows the model with greater effective capacity and improved transfer performance. IBEX retains the original TargetDiff architecture and hyperparameters for training to generate molecules compatible with the binding pocket; it then applies an L-BFGS optimization step to finely refine each conformation by optimizing five physics-based terms and adjusting six translational and rotational degrees of freedom in under one second. With only these modifications, IBEX raises the zero-shot docking success rate on CBGBench CrossDocked2020-based from 53% to 64%, improves the mean Vina score from $-7.41 kcal mol^{-1}$ to $-8.07 kcal mol^{-1}$, and achieves the best median Vina energy in 57 of 100 pockets versus 3 for the original TargetDiff. IBEX also increases the QED by 25%, achieves state-of-the-art validity and diversity, and markedly reduces extrapolation error.
• 摘要：三维生成模型越来越多地驱动基于结构的药物发现，但它仍然受到公开可用的蛋白质配体复合物的稀缺限制。在此类数据稀缺下，几乎所有现有的管道都难以学习可转移的几何先验，从而使训练集的偏见过高。因此，我们提出了IBEX，这是一种信息 - 底层探索的粗到精细管道，可解决基于结构的药物设计中蛋白质配体复杂数据的长期短缺。具体而言，我们使用pac-bayesian信息 - 底层理论来量化每个样本的信息密度。该分析揭示了不同的掩蔽策略如何影响泛化，并表明与传统的DE生成相比，受约束的脚手架跳跃任务赋予该模型具有更大的有效能力和提高的转移性能。 IBEX保留了原始的TargetDiff架构和超参数训练，以生成与结合口袋兼容的分子；然后，它应用了L-BFG的优化步骤，通过优化五个基于物理的术语并在一秒钟内调整六个转化和旋转自由度，从而精细地完善每个构象。只有这些修改，IBEX将基于CBGBENCH CROSSDOCKED2020的零射门成功率从53％提高到64％，将平均Vina得分从$ -7.41 kcal mol^{ - 1} $提高到$ -8.07.07.07 kcal mol^{ - 1} $，以及最佳的Median Vina in Frinestian vina in Fertirans in Dernies vina in Frinestian in Ferthers vinaian Vina in 57 Targetdiff。 IBEX还将QED增加了25％，达到了最先进的有效性和多样性，并显着减少了外推误差。

Title: Ultra-High-Definition Reference-Based Landmark Image Super-Resolution with Generative Diffusion Prior

• Authors: Zhenning Shi, Zizheng Yan, Yuhang Yu, Clara Xue, Jingyu Zhuang, Qi Zhang, Jinwei Chen, Tao Li, Qingnan Fan
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.10779
• Pdf URL: https://arxiv.org/pdf/2508.10779
• Copy Paste: [[2508.10779]] Ultra-High-Definition Reference-Based Landmark Image Super-Resolution with Generative Diffusion Prior(https://arxiv.org/abs/2508.10779)
• Keywords: restoration, super-resolution, generative
• Abstract: Reference-based Image Super-Resolution (RefSR) aims to restore a low-resolution (LR) image by utilizing the semantic and texture information from an additional reference high-resolution (reference HR) image. Existing diffusion-based RefSR methods are typically built upon ControlNet, which struggles to effectively align the information between the LR image and the reference HR image. Moreover, current RefSR datasets suffer from limited resolution and poor image quality, resulting in the reference images lacking sufficient fine-grained details to support high-quality restoration. To overcome the limitations above, we propose TriFlowSR, a novel framework that explicitly achieves pattern matching between the LR image and the reference HR image. Meanwhile, we introduce Landmark-4K, the first RefSR dataset for Ultra-High-Definition (UHD) landmark scenarios. Considering the UHD scenarios with real-world degradation, in TriFlowSR, we design a Reference Matching Strategy to effectively match the LR image with the reference HR image. Experimental results show that our approach can better utilize the semantic and texture information of the reference HR image compared to previous methods. To the best of our knowledge, we propose the first diffusion-based RefSR pipeline for ultra-high definition landmark scenarios under real-world degradation. Our code and model will be available at this https URL.
• 摘要：基于参考的图像超分辨率（REFSR）旨在通过从附加参考高分辨率（参考HR）图像中利用语义和纹理信息来恢复低分辨率（LR）图像。现有的基于扩散的REFSR方法通常是基于ControlNet的，该方法难以有效地对齐LR图像和参考HR图像之间的信息。此外，当前的REFSR数据集的分辨率有限和图像质量差，导致参考图像缺乏足够的细粒细节来支持高质量的恢复。为了克服上述局限性，我们提出了Triflowsr，这是一个新型框架，可以在LR图像和参考HR图像之间明确实现模式匹配。同时，我们介绍了Landmark-4K，这是第一个用于超高定义（UHD）地标情景的REFSR数据集。考虑到带有现实世界降级的UHD场景，在Triflowsr中，我们设计了一个参考匹配策略，以有效地将LR图像与参考HR图像匹配。实验结果表明，与以前的方法相比，我们的方法可以更好地利用参考HR图像的语义和纹理信息。据我们所知，我们提出了第一个基于扩散的REFSR管道，以实现现实世界中的超高定义具有里程碑意义的情况。我们的代码和模型将在此HTTPS URL上可用。

Title: Object Fidelity Diffusion for Remote Sensing Image Generation

• Authors: Ziqi Ye, Shuran Ma, Jie Yang, Xiaoyi Yang, Ziyang Gong, Xue Yang, Haipeng Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10801
• Pdf URL: https://arxiv.org/pdf/2508.10801
• Copy Paste: [[2508.10801]] Object Fidelity Diffusion for Remote Sensing Image Generation(https://arxiv.org/abs/2508.10801)
• Keywords: generation
• Abstract: High-precision controllable remote sensing image generation is both meaningful and challenging. Existing diffusion models often produce low-fidelity images due to their inability to adequately capture morphological details, which may affect the robustness and reliability of object detection models. To enhance the accuracy and fidelity of generated objects in remote sensing, this paper proposes Object Fidelity Diffusion (OF-Diff), which effectively improves the fidelity of generated objects. Specifically, we are the first to extract the prior shapes of objects based on the layout for diffusion models in remote sensing. Then, we introduce a dual-branch diffusion model with diffusion consistency loss, which can generate high-fidelity remote sensing images without providing real images during the sampling phase. Furthermore, we introduce DDPO to fine-tune the diffusion process, making the generated remote sensing images more diverse and semantically consistent. Comprehensive experiments demonstrate that OF-Diff outperforms state-of-the-art methods in the remote sensing across key quality metrics. Notably, the performance of several polymorphic and small object classes shows significant improvement. For instance, the mAP increases by 8.3%, 7.7%, and 4.0% for airplanes, ships, and vehicles, respectively.
• 摘要：高精度可控的遥感图像产生既有意义又具有挑战性。现有的扩散模型通常由于无法充分捕获形态学细节而产生低保真图像，这可能会影响对象检测模型的鲁棒性和可靠性。为了提高遥感中生成的对象的准确性和保真度，本文提出了对象的保真度扩散（DIFF），从而有效地改善了生成的对象的保真度。具体而言，我们是第一个基于遥感中扩散模型的布局提取对象的先前形状的人。然后，我们引入了具有扩散一致性损失的双分支扩散模型，该模型可以生成高保真遥感图像，而无需在采样阶段提供真实图像。此外，我们将DDPO介绍以微调扩散过程，从而使生成的遥感图像更加多样化和语义一致。全面的实验表明，野外的遥控质量指标的遥控方法优于最先进的方法。值得注意的是，几种多态性和小物体类别的性能显示出显着改善。例如，飞机，船只和车辆的地图分别增加了8.3％，7.7％和4.0％。

Title: Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generation

• Authors: Harold Haodong Chen, Haojian Huang, Qifeng Chen, Harry Yang, Ser-Nam Lim
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10858
• Pdf URL: https://arxiv.org/pdf/2508.10858
• Copy Paste: [[2508.10858]] Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generation(https://arxiv.org/abs/2508.10858)
• Keywords: generation
• Abstract: Recent advancements in video generation have enabled the creation of high-quality, visually compelling videos. However, generating videos that adhere to the laws of physics remains a critical challenge for applications requiring realism and accuracy. In this work, we propose PhysHPO, a novel framework for Hierarchical Cross-Modal Direct Preference Optimization, to tackle this challenge by enabling fine-grained preference alignment for physically plausible video generation. PhysHPO optimizes video alignment across four hierarchical granularities: a) Instance Level, aligning the overall video content with the input prompt; b) State Level, ensuring temporal consistency using boundary frames as anchors; c) Motion Level, modeling motion trajectories for realistic dynamics; and d) Semantic Level, maintaining logical consistency between narrative and visuals. Recognizing that real-world videos are the best reflections of physical phenomena, we further introduce an automated data selection pipeline to efficiently identify and utilize "good data" from existing large-scale text-video datasets, thereby eliminating the need for costly and time-intensive dataset construction. Extensive experiments on both physics-focused and general capability benchmarks demonstrate that PhysHPO significantly improves physical plausibility and overall video generation quality of advanced models. To the best of our knowledge, this is the first work to explore fine-grained preference alignment and data selection for video generation, paving the way for more realistic and human-preferred video generation paradigms.
• 摘要：视频生成的最新进展使创建高质量，视觉引人入胜的视频。但是，生成遵守物理定律的视频对于需要现实主义和准确性的应用程序仍然是一个关键的挑战。在这项工作中，我们提出了PhysHPO，这是一个用于分层跨模式直接偏好优化的新型框架，以通过实现精细粒度的偏好对齐方式来应对这一挑战，以实现物理上合理的视频生成。 PhysHPO优化了四个层次粒度的视频对齐：a）实例级别，将整个视频内容与输入提示保持一致； b）状态级，确保使用边界框架作为锚点确保时间一致性； c）运动水平，为现实动力学建模运动轨迹；和d）语义水平，保持叙事和视觉效果之间的逻辑一致性。认识到现实世界的视频是物理现象的最佳反思，我们进一步引入了自动数据选择管道，以有效地识别和利用现有大型文本视频数据集中的“良好数据”，从而消除了对昂贵且耗时的数据集构建的需求。针对物理和一般能力基准的广泛实验表明，PhysHPO显着提高了高级模型的物理合理性和整体视频生成质量。据我们所知，这是探索视频生成的细粒偏好对齐和数据选择的第一项工作，为更现实和人类偏爱的视频生成范式铺平了道路。

Title: TexVerse: A Universe of 3D Objects with High-Resolution Textures

• Authors: Yibo Zhang, Li Zhang, Rui Ma, Nan Cao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2508.10868
• Pdf URL: https://arxiv.org/pdf/2508.10868
• Copy Paste: [[2508.10868]] TexVerse: A Universe of 3D Objects with High-Resolution Textures(https://arxiv.org/abs/2508.10868)
• Keywords: generation
• Abstract: We introduce TexVerse, a large-scale 3D dataset featuring high-resolution textures. While recent advances in large-scale 3D datasets have enhanced high-resolution geometry generation, creating high-resolution textures end-to-end remains underexplored due to the lack of suitable datasets. TexVerse fills this gap with a curated collection of over 858K unique high-resolution 3D models sourced from Sketchfab, including more than 158K models with physically based rendering (PBR) materials. Each model encompasses all of its high-resolution variants, bringing the total to 1.6M 3D instances. TexVerse also includes specialized subsets: TexVerse-Skeleton, with 69K rigged models, and TexVerse-Animation, with 54K animated models, both preserving original skeleton and animation data uploaded by the user. We also provide detailed model annotations describing overall characteristics, structural components, and intricate features. TexVerse offers a high-quality data resource with wide-ranging potential applications in texture synthesis, PBR material development, animation, and various 3D vision and graphics tasks.
• 摘要：我们介绍了Texverse，这是一个具有高分辨率纹理的大型3D数据集。尽管大规模3D数据集的最新进展增强了高分辨率几何的生成，但由于缺乏合适的数据集，创建端到端的高分辨率纹理仍然没有被驱动。 Texverse通过策划的858K独特的高分辨率3D模型的策划收藏来填补这一空白，其中包括超过158K的渲染（PBR）材料的158K型号。每个模型都包含其所有高分辨率变体，使总计达到16m 3D实例。 Texverse还包括专门的子集：带有69K操纵模型的Texverse-Skeleton和Texverse-Animation，具有54K动画型号，均保留了用户上传的原始骨架和动画数据。我们还提供了描述整体特征，结构组件和复杂特征的详细模型注释。 Texverse提供了高质量的数据资源，并具有广泛的潜在应用，在纹理综合，PBR材料开发，动画以及各种3D视觉和图形任务中。

Title: ToonComposer: Streamlining Cartoon Production with Generative Post-Keyframing

• Authors: Lingen Li, Guangzhi Wang, Zhaoyang Zhang, Yaowei Li, Xiaoyu Li, Qi Dou, Jinwei Gu, Tianfan Xue, Ying Shan
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.10881
• Pdf URL: https://arxiv.org/pdf/2508.10881
• Copy Paste: [[2508.10881]] ToonComposer: Streamlining Cartoon Production with Generative Post-Keyframing(https://arxiv.org/abs/2508.10881)
• Keywords: generative
• Abstract: Traditional cartoon and anime production involves keyframing, inbetweening, and colorization stages, which require intensive manual effort. Despite recent advances in AI, existing methods often handle these stages separately, leading to error accumulation and artifacts. For instance, inbetweening approaches struggle with large motions, while colorization methods require dense per-frame sketches. To address this, we introduce ToonComposer, a generative model that unifies inbetweening and colorization into a single post-keyframing stage. ToonComposer employs a sparse sketch injection mechanism to provide precise control using keyframe sketches. Additionally, it uses a cartoon adaptation method with the spatial low-rank adapter to tailor a modern video foundation model to the cartoon domain while keeping its temporal prior intact. Requiring as few as a single sketch and a colored reference frame, ToonComposer excels with sparse inputs, while also supporting multiple sketches at any temporal location for more precise motion control. This dual capability reduces manual workload and improves flexibility, empowering artists in real-world scenarios. To evaluate our model, we further created PKBench, a benchmark featuring human-drawn sketches that simulate real-world use cases. Our evaluation demonstrates that ToonComposer outperforms existing methods in visual quality, motion consistency, and production efficiency, offering a superior and more flexible solution for AI-assisted cartoon production.
• 摘要：传统的卡通和动漫生产涉及需要密集的手动努力的关键帧，渐进式和着色阶段。尽管AI的最新进展，现有方法通常会分别分别处理这些阶段，从而导致错误积累和人工制品。例如，内部方法在大型动作中遇到困难，而着色方法需要密集的人均草图。为了解决这个问题，我们介绍了ToonComposer，这是一个生成模型，将嵌入式和着色统一为单个后的关键阶段。 Tooncomposer采用稀疏的草图注入机制，使用KeyFrame草图提供精确的控制。此外，它使用带有空间低级适配器的卡通改编方法来定制现代视频基础模型，同时保持其暂时的先验完整。 ToonComposer需要少于单个草图和彩色参考框架，稀疏输入也很出色，同时还支持任何时间位置的多个草图，以进行更精确的运动控制。这种双重能力可减少手动工作量并提高灵活性，从而在现实世界中赋予艺术家能力。为了评估我们的模型，我们进一步创建了PKBench，这是一种基准，具有模拟现实世界中用例的人绘制草图。我们的评估表明，ToonComposer在视觉质量，运动一致性和生产效率方面的表现优于现有方法，为AI辅助卡通生产提供了优越，更灵活的解决方案。

Title: Puppeteer: Rig and Animate Your 3D Models

• Authors: Chaoyue Song, Xiu Li, Fan Yang, Zhongcong Xu, Jiacheng Wei, Fayao Liu, Jiashi Feng, Guosheng Lin, Jianfeng Zhang
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2508.10898
• Pdf URL: https://arxiv.org/pdf/2508.10898
• Copy Paste: [[2508.10898]] Puppeteer: Rig and Animate Your 3D Models(https://arxiv.org/abs/2508.10898)
• Keywords: generative
• Abstract: Modern interactive applications increasingly demand dynamic 3D content, yet the transformation of static 3D models into animated assets constitutes a significant bottleneck in content creation pipelines. While recent advances in generative AI have revolutionized static 3D model creation, rigging and animation continue to depend heavily on expert intervention. We present Puppeteer, a comprehensive framework that addresses both automatic rigging and animation for diverse 3D objects. Our system first predicts plausible skeletal structures via an auto-regressive transformer that introduces a joint-based tokenization strategy for compact representation and a hierarchical ordering methodology with stochastic perturbation that enhances bidirectional learning capabilities. It then infers skinning weights via an attention-based architecture incorporating topology-aware joint attention that explicitly encodes inter-joint relationships based on skeletal graph distances. Finally, we complement these rigging advances with a differentiable optimization-based animation pipeline that generates stable, high-fidelity animations while being computationally more efficient than existing approaches. Extensive evaluations across multiple benchmarks demonstrate that our method significantly outperforms state-of-the-art techniques in both skeletal prediction accuracy and skinning quality. The system robustly processes diverse 3D content, ranging from professionally designed game assets to AI-generated shapes, producing temporally coherent animations that eliminate the jittering issues common in existing methods.
• 摘要：现代交互应用程序越来越多地需求动态3D内容，但是将静态3D模型转换为动画资产构成了内容创建管道中的重要瓶颈。尽管生成AI的最新进展彻底改变了静态3D模型的创建，但索具和动画仍在很大程度上取决于专家干预。我们提出了Puppeteer，这是一个综合框架，可解决不同3D对象的自动操作和动画。我们的系统首先通过自动回归变压器预测合理的骨骼结构，该变压器引入了基于联合的令牌化策略，用于紧凑型表示，并具有随机扰动的分层订购方法，从而增强了双向学习能力。然后，它通过基于注意力的体系结构融合了拓扑感知的关节注意力，该体系会明确编码基于骨骼图距离的关节间关系。最后，我们通过基于优化的动画管道对这些操纵进步进行补充，该动画管道会生成稳定，高保真的动画，同时比现有方法更有效地计算效率。跨多个基准测试的广泛评估表明，我们的方法在骨骼预测的准确性和皮肤质量方面都显着优于最先进的技术。该系统可靠地处理不同的3D内容，从专业设计的游戏资产到AI生成的形状，产生了时间连贯的动画，从而消除了现有方法中常见的抖动问题。