2025-06-23

Title: Ignition Phase : Standard Training for Fast Adversarial Robustness

• Authors: Wang Yu-Hang, Liu ying, Fang liang, Wang Xuelin, Junkang Guo, Shiwei Li, Lei Gao, Jian Liu, Wenfei Yin
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.15685
• Pdf URL: https://arxiv.org/pdf/2506.15685
• Copy Paste: [[2506.15685]] Ignition Phase : Standard Training for Fast Adversarial Robustness(https://arxiv.org/abs/2506.15685)
• Keywords: generation
• Abstract: Adversarial Training (AT) is a cornerstone defense, but many variants overlook foundational feature representations by primarily focusing on stronger attack generation. We introduce Adversarial Evolution Training (AET), a simple yet powerful framework that strategically prepends an Empirical Risk Minimization (ERM) phase to conventional AT. We hypothesize this initial ERM phase cultivates a favorable feature manifold, enabling more efficient and effective robustness acquisition. Empirically, AET achieves comparable or superior robustness more rapidly, improves clean accuracy, and cuts training costs by 8-25\%. Its effectiveness is shown across multiple datasets, architectures, and when augmenting established AT methods. Our findings underscore the impact of feature pre-conditioning via standard training for developing more efficient, principled robust defenses. Code is available in the supplementary material.
• 摘要：对抗训练（AT）是一个基石防御，但是许多变体主要通过专注于更强大的攻击产生来忽略基础特征表示。我们介绍了对抗性进化训练（AET），这是一个简单而强大的框架，从策略上讲，经验风险最小化（ERM）阶段是常规的。我们假设这个初始的ERM相培养了一个有利的特征歧管，从而实现了更有效和有效的稳健性获取。从经验上讲，AET可以更快地达到可比或优越的鲁棒性，提高清洁准确性，并将培训成本降低8-25 \％。在多个数据集，体系结构以及在方法上建立时，显示了其有效性。我们的发现强调了通过标准培训预处理特征的影响，以开发更高效，有原则的强大防御能力。代码在补充材料中可用。

Title: LLM Web Dynamics: Tracing Model Collapse in a Network of LLMs

• Authors: Tianyu Wang, Lingyou Pang, Akira Horiguchi, Carey E. Priebe
• Subjects: cs.LG, cs.AI, cs.SI, stat.ME
• Abstract URL: https://arxiv.org/abs/2506.15690
• Pdf URL: https://arxiv.org/pdf/2506.15690
• Copy Paste: [[2506.15690]] LLM Web Dynamics: Tracing Model Collapse in a Network of LLMs(https://arxiv.org/abs/2506.15690)
• Keywords: generation
• Abstract: The increasing use of synthetic data from the public Internet has enhanced data usage efficiency in large language model (LLM) training. However, the potential threat of model collapse remains insufficiently explored. Existing studies primarily examine model collapse in a single model setting or rely solely on statistical surrogates. In this work, we introduce LLM Web Dynamics (LWD), an efficient framework for investigating model collapse at the network level. By simulating the Internet with a retrieval-augmented generation (RAG) database, we analyze the convergence pattern of model outputs. Furthermore, we provide theoretical guarantees for this convergence by drawing an analogy to interacting Gaussian Mixture Models.
• 摘要：公共互联网中综合数据的越来越多的使用增强了大型语言模型（LLM）培训的数据使用效率。但是，模型崩溃的潜在威胁仍然不足。现有研究主要检查单个模型设置中的模型崩溃或仅依赖统计替代物。在这项工作中，我们介绍了LLM Web Dynamics（LWD），这是一个有效的框架，用于研究网络级别的模型崩溃。通过使用检索功能生成（RAG）数据库模拟Internet，我们分析了模型输出的收敛模式。此外，我们通过与相互作用的高斯混合模型进行类比来为这种融合提供理论保证。

Title: MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinement

• Authors: Jaehyun Nam, Jinsung Yoon, Jiefeng Chen, Jinwoo Shin, Sercan Ö. Arık, Tomas Pfister
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.15692
• Pdf URL: https://arxiv.org/pdf/2506.15692
• Copy Paste: [[2506.15692]] MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinement(https://arxiv.org/abs/2506.15692)
• Keywords: generation
• Abstract: Agents based on large language models (LLMs) for machine learning engineering (MLE) can automatically implement ML models via code generation. However, existing approaches to build such agents often rely heavily on inherent LLM knowledge and employ coarse exploration strategies that modify the entire code structure at once. This limits their ability to select effective task-specific models and perform deep exploration within specific components, such as experimenting extensively with feature engineering options. To overcome these, we propose MLE-STAR, a novel approach to build MLE agents. MLE-STAR first leverages external knowledge by using a search engine to retrieve effective models from the web, forming an initial solution, then iteratively refines it by exploring various strategies targeting specific ML components. This exploration is guided by ablation studies analyzing the impact of individual code blocks. Furthermore, we introduce a novel ensembling method using an effective strategy suggested by MLE-STAR. Our experimental results show that MLE-STAR achieves medals in 44% of the Kaggle competitions on the MLE-bench, significantly outperforming the best alternative.
• 摘要：基于机器学习工程（MLE）的大型语言模型（LLM）的代理可以自动通过代码生成实现ML模型。但是，现有的构建代理的方法通常很大程度上依赖于固有的LLM知识并采用粗略的勘探策略来一次修改整个代码结构。这限制了他们选择有效的特定任务模型并在特定组件中进行深入探索的能力，例如对功能工程选项进行广泛的实验。为了克服这些，我们提出了MLE-Star，这是一种新型的MLE代理的方法。 MLE-Star首先通过使用搜索引擎从Web中检索有效模型，形成初始解决方案，然后通过探索针对特定ML组件的各种策略来完善它来利用外部知识。这种探索是通过消融研究来指导的，分析了各个代码块的影响。此外，我们使用MLE-Star建议的有效策略介绍了一种新颖的结合方法。我们的实验结果表明，MLE-Star在MLE板凳上的Kaggle竞赛中获得奖牌，这显着超过了最佳选择。

Title: Development of a Multiprocessing Interface Genetic Algorithm for Optimising a Multilayer Perceptron for Disease Prediction

• Authors: Iliyas Ibrahim Iliyas, Souley Boukari, Abdulsalam Yau Gital
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.15694
• Pdf URL: https://arxiv.org/pdf/2506.15694
• Copy Paste: [[2506.15694]] Development of a Multiprocessing Interface Genetic Algorithm for Optimising a Multilayer Perceptron for Disease Prediction(https://arxiv.org/abs/2506.15694)
• Keywords: generation
• Abstract: This study introduces a framework that integrates nonlinear feature extraction, classification, and efficient optimization. First, kernel principal component analysis with a radial basis function kernel reduces dimensionality while preserving 95% of the variance. Second, a multilayer perceptron (MLP) learns to predict disease status. Finally, a modified multiprocessing genetic algorithm (MIGA) optimizes MLP hyperparameters in parallel over ten generations. We evaluated this approach on three datasets: the Wisconsin Diagnostic Breast Cancer dataset, the Parkinson's Telemonitoring dataset, and the chronic kidney disease dataset. The MLP tuned by the MIGA achieved the best accuracy of 99.12% for breast cancer, 94.87% for Parkinson's disease, and 100% for chronic kidney disease. These results outperform those of other methods, such as grid search, random search, and Bayesian optimization. Compared with a standard genetic algorithm, kernel PCA revealed nonlinear relationships that improved classification, and the MIGA's parallel fitness evaluations reduced the tuning time by approximately 60%. The genetic algorithm incurs high computational cost from sequential fitness evaluations, but our multiprocessing interface GA (MIGA) parallelizes this step, slashing the tuning time and steering the MLP toward the best accuracy score of 99.12%, 94.87%, and 100% for breast cancer, Parkinson's disease, and CKD, respectively.
• 摘要：这项研究介绍了一个整合非线性特征提取，分类和有效优化的框架。首先，内核主成分分析具有径向基函数核核会降低维数，同时保留95％的方差。其次，多层感知器（MLP）学会了预测疾病状况。最后，修改的多处理遗传算法（MIGA）在超过十代的超过十代时优化了MLP超参数。我们在三个数据集上评估了这种方法：威斯康星州的诊断乳腺癌数据集，帕金森氏症的远程监控数据集和慢性肾脏病数据集。 MIGA调整的MLP获得了乳腺癌的最佳准确度，帕金森氏病的94.87％，慢性肾脏疾病的最佳准确性为100％。这些结果优于其他方法的结果，例如网格搜索，随机搜索和贝叶斯优化。与标准遗传算法相比，内核PCA揭示了改善分类的非线性关系，而MIGA的平行适应性评估将调谐时间降低了约60％。遗传算法从顺序健身评估中产生了高计算成本，但是我们的多处理界面GA（MIGA）平行于这一步骤，削减了调谐时间，并将MLP转向99.12％，94.87％，94.87％和100％的最佳准确度，分别为乳腺癌，帕克森氏病和100％。

Title: SimuGen: Multi-modal Agentic Framework for Constructing Block Diagram-Based Simulation Models

• Authors: Xinxing Ren, Qianbo Zang, Zekun Guo
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.15695
• Pdf URL: https://arxiv.org/pdf/2506.15695
• Copy Paste: [[2506.15695]] SimuGen: Multi-modal Agentic Framework for Constructing Block Diagram-Based Simulation Models(https://arxiv.org/abs/2506.15695)
• Keywords: generation
• Abstract: Recent advances in large language models (LLMs) have shown impressive performance in mathematical reasoning and code generation. However, LLMs still struggle in the simulation domain, particularly in generating Simulink models, which are essential tools in engineering and scientific research. Our preliminary experiments indicate that LLM agents often fail to produce reliable and complete Simulink simulation code from text-only inputs, likely due to the lack of Simulink-specific data in their pretraining. To address this challenge, we propose SimuGen, a multimodal agent-based framework that automatically generates accurate Simulink simulation code by leveraging both the visual Simulink diagram and domain knowledge. SimuGen coordinates several specialized agents, including an investigator, unit test reviewer, code generator, executor, debug locator, and report writer, supported by a domain-specific knowledge base. This collaborative and modular design enables interpretable, robust, and reproducible Simulink simulation generation. Our source code is publicly available at this https URL.
• 摘要：大型语言模型（LLM）的最新进展在数学推理和代码生成中表现出了令人印象深刻的表现。但是，LLM仍在模拟领域中挣扎，特别是在生成Simulink模型时，这是工程和科学研究中必不可少的工具。我们的初步实验表明，LLM代理通常无法从纯文本输入中产生可靠且完整的模拟代码，这可能是由于其预读预处理中缺乏Simulink特定数据所致。为了应对这一挑战，我们提出了Simugen，这是一种基于多模式代理的框架，通过利用Visual Simulink图和域知识来自动生成准确的Simulink模拟代码。 Simugen协调了几个专业代理，包括调查员，单位测试审稿人，代码生成器，执行器，调试定位器和报告作者，并得到了特定于领域的知识库的支持。这种协作和模块化设计可以使可解释，鲁棒和可重复的模拟仿真生成。我们的源代码可在此HTTPS URL上公开获得。

Title: DeepRTL2: A Versatile Model for RTL-Related Tasks

• Authors: Yi Liu, Hongji Zhang, Yunhao Zhou, Zhengyuan Shi, Changran Xu, Qiang Xu
• Subjects: cs.LG, cs.AR, cs.CL
• Abstract URL: https://arxiv.org/abs/2506.15697
• Pdf URL: https://arxiv.org/pdf/2506.15697
• Copy Paste: [[2506.15697]] DeepRTL2: A Versatile Model for RTL-Related Tasks(https://arxiv.org/abs/2506.15697)
• Keywords: generation
• Abstract: The integration of large language models (LLMs) into electronic design automation (EDA) has significantly advanced the field, offering transformative benefits, particularly in register transfer level (RTL) code generation and understanding. While previous studies have demonstrated the efficacy of fine-tuning LLMs for these generation-based tasks, embedding-based tasks, which are equally critical to EDA workflows, have been largely overlooked. These tasks, including natural language code search, RTL code functionality equivalence checking, and performance prediction, are essential for accelerating and optimizing the hardware design process. To address this gap, we present DeepRTL2, a family of versatile LLMs that unifies both generation- and embedding-based tasks related to RTL. By simultaneously tackling a broad range of tasks, DeepRTL2 represents the first model to provide a comprehensive solution to the diverse challenges in EDA. Through extensive experiments, we show that DeepRTL2 achieves state-of-the-art performance across all evaluated tasks.
• 摘要：大型语言模型（LLM）集成到电子设计自动化（EDA）中已显着提高了该领域，尤其是在寄存器转移级别（RTL）代码生成和理解中。虽然先前的研究表明，微调LLMS对这些基于一代的任务的功效，但基于嵌入的任务对EDA工作流程同样至关重要。这些任务，包括自然语言代码搜索，RTL代码功能等效检查和性能预测，对于加速和优化硬件设计过程至关重要。为了解决这一差距，我们提出了DeepRTL2，这是一个多功能LLM家族，统一了与RTL相关的一代和嵌入式任务。通过同时解决广泛的任务，DEEPRTL2代表了第一个为EDA中各种挑战提供全面解决方案的模型。通过广泛的实验，我们表明DEEPRTL2在所有评估的任务中都达到了最新的性能。

Title: Minifinetuning: Low-Data Generation Domain Adaptation through Corrective Self-Distillation

• Authors: Peter Belcak, Greg Heinrich, Jan Kautz, Pavlo Molchanov
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.15702
• Pdf URL: https://arxiv.org/pdf/2506.15702
• Copy Paste: [[2506.15702]] Minifinetuning: Low-Data Generation Domain Adaptation through Corrective Self-Distillation(https://arxiv.org/abs/2506.15702)
• Keywords: generation
• Abstract: Finetuning language models for a new domain inevitably leads to the deterioration of their general performance. This becomes more pronounced the more limited the finetuning data resource. We introduce minifinetuning (MFT), a method for language model domain adaptation that considerably reduces the effects of overfitting-induced degeneralization in low-data settings and which does so in the absence of any pre-training data for replay. MFT demonstrates 2-10x more favourable specialization-to-degeneralization ratios than standard finetuning across a wide range of models and domains and exhibits an intrinsic robustness to overfitting when data in the new domain is scarce and down to as little as 500 samples. Employing corrective self-distillation that is individualized on the sample level, MFT outperforms parameter-efficient finetuning methods, demonstrates replay-like degeneralization mitigation properties, and is composable with either for a combined effect.
• 摘要：新领域的填充语言模型不可避免地会导致其一般表现的恶化。这变得越来越明显，即填充数据资源越有限。我们介绍了MinifinTuning（MFT），这是一种语言模型域适应的方法，可大大降低过度拟合诱导的脱生物化的影响，而在低数据设置中，它在没有任何重播的预训练数据的情况下确实如此。 MFT比在广泛的模型和域中的标准填充比标准填充比标准填充表现出2-10倍，并且当新域中的数据稀缺且低至500个样本时，表现出固有的鲁棒性。 MFT采用在样本级别上个性化的纠正式自我鉴定，优于参数有效的芬太尼方法，证明了重播样退化缓解属性，并且可以与任何一种合并效应相结合。

Title: Learn from the Past: Fast Sparse Indexing for Large Language Model Decoding

• Authors: Feiyu Yao, Qian Wang
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2506.15704
• Pdf URL: https://arxiv.org/pdf/2506.15704
• Copy Paste: [[2506.15704]] Learn from the Past: Fast Sparse Indexing for Large Language Model Decoding(https://arxiv.org/abs/2506.15704)
• Keywords: generation
• Abstract: As large language models (LLMs) continue to support increasingly longer contexts, the memory demand for key-value (KV) caches during decoding grows rapidly, becoming a critical bottleneck in both GPU memory capacity and PCIe bandwidth. Sparse attention mechanisms alleviate this issue by computing attention weights only for selected key-value pairs. However, their indexing computation typically requires traversing all key vectors, resulting in significant computational and data transfer overhead. To reduce the cost of index retrieval, existing methods often treat each decoding step as an independent process, failing to exploit the temporal correlations embedded in historical decoding information. To this end, we propose LFPS(Learn From the Past for Sparse Indexing), an acceleration method that dynamically constructs sparse indexing candidates based on historical attention patterns. LFPS captures two prevalent trends in decoder attention -vertical patterns (attending to fixed positions) and slash patterns (attending to relative positions) -and incorporates a positional expansion strategy to effectively predict the Top-k indices for the current step. We validate LFPS on challenging long-context benchmarks such as LongBench-RULER, using Llama-3.1-8B-Instruct as the base model. Experimental results show that LFPS achieves up to 22.8$\times$ speedup over full attention and 9.6$\times$ speedup over exact Top-k retrieval on an RTX 4090 GPU and a single CPU core of a Xeon Gold 6430, respectively, while preserving generation accuracy. These results demonstrate that LFPS offers a practical and efficient solution for decoding optimization in long-context LLM inference.
• 摘要：随着大型语言模型（LLMS）继续支持越来越长的上下文，在解码过程中对键值（KV）缓存的内存需求迅速增长，成为GPU内存能力和PCIE带宽的关键瓶颈。稀疏的注意机制通过仅计算精选的键值对来减轻此问题。但是，它们的索引计算通常需要穿越所有关键向量，从而导致大量的计算和数据传输开销。为了降低索引检索的成本，现有方法通常将每个解码步骤视为一个独立的过程，无法利用历史解码信息中嵌入的时间相关性。为此，我们提出了LFP（从过去学习稀疏索引），这是一种加速方法，该方法基于历史注意力模式，动态构建稀疏的索引候选者。 LFP捕获了解码器注意 - 垂直模式（固定位置）和斜线模式（参与相对位置）的两个普遍趋势 - 并结合了位置扩展策略，以有效地预测当前步骤的顶级top -k指数。我们使用Llama-3.1-8B-Instruct作为基础模型来验证LFP的挑战性长篇小写基准（例如Longbench-Ruler）。实验结果表明，LFPS在RTX 4090 GPU上的精确TOP-K检索中获得了高达22.8 $ \ times $加速，并分别在Xeon Gold 6430的单个CPU上获得9.6 $ \ times $速度，同时确保了准确的生成准确。这些结果表明，LFP提供了一种实用有效的解决方案，用于在长篇文化LLM推理中解码优化。

Title: MDPO: Multi-Granularity Direct Preference Optimization for Mathematical Reasoning

• Authors: Yunze Lin
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.15706
• Pdf URL: https://arxiv.org/pdf/2506.15706
• Copy Paste: [[2506.15706]] MDPO: Multi-Granularity Direct Preference Optimization for Mathematical Reasoning(https://arxiv.org/abs/2506.15706)
• Keywords: generation
• Abstract: Mathematical reasoning presents a significant challenge for Large Language Models (LLMs) as it requires ensuring the correctness of each reasoning step. Researchers have been strengthening the mathematical reasoning abilities of LLMs through supervised fine-tuning, but due to the inability to suppress incorrect outputs, illusions can easily arise. Recently, Direct Preference Optimization (DPO) has been widely adopted for aligning human intent by using preference data to prevent LLMs from generating incorrect outputs. However, it has shown limited benefits in long-chain mathematical reasoning, mainly because DPO struggles to effectively capture the differences between accepted and rejected answers from preferences in long-chain data. The inconsistency between DPO training and LLMs' generation metrics also affects the effectiveness of suppressing incorrect outputs. We propose the Multi-Granularity Direct Preference Optimization (MDPO) method, optimizing the mathematical reasoning of LLMs at three granularities: Solution2Solution, Inference2Inference, and Step2Step. Solution2Solution focuses on the correctness of entire long-chain reasoning; Inference2Inference concentrates on logical reasoning between steps; Step2Step corrects computational errors in steps, enhancing the computational capabilities of LLMs. Additionally, we unify the training objectives of the three granularities to align with the generation metrics. We conducted experiments on the open-source models Qwen2 and Llama3, achieving improvements of 1.7% and 0.9% on the GSM8K dataset, and 2.3% and 1.2% on the MATH dataset, outperforming DPO and other DPO variant methods. Furthermore, we also provide a pipeline for constructing MDPO training data that is simple and does not require manual annotation costs.
• 摘要：数学推理给大型语言模型（LLMS）带来了重大挑战，因为它需要确保每个推理步骤的正确性。研究人员一直在通过监督的微调来增强LLM的数学推理能力，但是由于无法抑制不正确的产出，很容易出现幻觉。最近，通过使用偏好数据来防止LLM产生错误的输出，直接偏好优化（DPO）已被广泛采用，以使人类意图对齐。但是，它在长链数学推理中显示出有限的好处，主要是因为DPO努力有效地捕获了长链数据中的偏好中接受答案和被拒绝答案之间的差异。 DPO培训与LLMS发电指标之间的不一致也会影响抑制不正确产出的有效性。我们提出了多粒性直接偏好优化（MDPO）方法，以三个粒度优化LLM的数学推理：solution2solution2solution，inperive2inference和step2Step。 Solution2Slouth的重点是整个长链推理的正确性；推理2inference集中于步骤之间的逻辑推理； STEP2STEP纠正步骤中的计算错误，从而增强了LLMS的计算功能。此外，我们统一了三个粒度的训练目标，以与一代指标保持一致。我们在开源模型QWEN2和LLAMA3上进行了实验，在GSM8K数据集上提高了1.7％和0.9％，而数学数据集的2.3％和1.2％，超过了DPO和其他DPO变体方法。此外，我们还提供了一条管道，用于构建简单且不需要手动注释成本的MDPO培训数据。

Title: Studying and Improving Graph Neural Network-based Motif Estimation

• Authors: Pedro C. Vieira, Miguel E. P. Silva, Pedro Manuel Pinto Ribeiro
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.15709
• Pdf URL: https://arxiv.org/pdf/2506.15709
• Copy Paste: [[2506.15709]] Studying and Improving Graph Neural Network-based Motif Estimation(https://arxiv.org/abs/2506.15709)
• Keywords: generation
• Abstract: Graph Neural Networks (GNNs) are a predominant method for graph representation learning. However, beyond subgraph frequency estimation, their application to network motif significance-profile (SP) prediction remains under-explored, with no established benchmarks in the literature. We propose to address this problem, framing SP estimation as a task independent of subgraph frequency estimation. Our approach shifts from frequency counting to direct SP estimation and modulates the problem as multitarget regression. The reformulation is optimised for interpretability, stability and scalability on large graphs. We validate our method using a large synthetic dataset and further test it on real-world graphs. Our experiments reveal that 1-WL limited models struggle to make precise estimations of SPs. However, they can generalise to approximate the graph generation processes of networks by comparing their predicted SP with the ones originating from synthetic generators. This first study on GNN-based motif estimation also hints at how using direct SP estimation can help go past the theoretical limitations that motif estimation faces when performed through subgraph counting.
• 摘要：图神经网络（GNN）是用于图表学习的主要方法。但是，除了子图频率估计之外，它们在网络基序显着性profile（SP）预测中的应用仍未探索，文献中没有建立的基准。我们建议解决这个问题，将SP估计作为任务构架，而与子图频率估计无关。我们的方法从频率计数转变为直接SP估计，并将问题调节为多坐Multitarget回归。重新制定了大图上的可解释性，稳定性和可伸缩性。我们使用大型合成数据集验证我们的方法，并在现实图表上进一步对其进行测试。我们的实验表明，1-WL有限模型难以对SP进行精确估计。但是，他们可以通过将其预测的SP与源自合成发生器的SP进行比较来概括以近似网络的图生成过程。这项关于基于GNN的基序估计的第一项研究还暗示了使用直接SP估计如何帮助超过通过子图计数进行的基础估计所面对的理论局限性。

Title: BuildingBRep-11K: Precise Multi-Storey B-Rep Building Solids with Rich Layout Metadata

• Authors: Yu Guo, Hongji Fang, Tianyu Fang, Zhe Cui
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.15718
• Pdf URL: https://arxiv.org/pdf/2506.15718
• Copy Paste: [[2506.15718]] BuildingBRep-11K: Precise Multi-Storey B-Rep Building Solids with Rich Layout Metadata(https://arxiv.org/abs/2506.15718)
• Keywords: generation, quality assessment
• Abstract: With the rise of artificial intelligence, the automatic generation of building-scale 3-D objects has become an active research topic, yet training such models still demands large, clean and richly annotated datasets. We introduce BuildingBRep-11K, a collection of 11 978 multi-storey (2-10 floors) buildings (about 10 GB) produced by a shape-grammar-driven pipeline that encodes established building-design principles. Every sample consists of a geometrically exact B-rep solid-covering floors, walls, slabs and rule-based openings-together with a fast-loading .npy metadata file that records detailed per-floor parameters. The generator incorporates constraints on spatial scale, daylight optimisation and interior layout, and the resulting objects pass multi-stage filters that remove Boolean failures, undersized rooms and extreme aspect ratios, ensuring compliance with architectural standards. To verify the dataset's learnability we trained two lightweight PointNet baselines. (i) Multi-attribute regression. A single encoder predicts storey count, total rooms, per-storey vector and mean room area from a 4 000-point cloud. On 100 unseen buildings it attains 0.37-storey MAE (87 \% within $\pm1$), 5.7-room MAE, and 3.2 m$^2$ MAE on mean area. (ii) Defect detection. With the same backbone we classify GOOD versus DEFECT; on a balanced 100-model set the network reaches 54 \% accuracy, recalling 82 \% of true defects at 53 \% precision (41 TP, 9 FN, 37 FP, 13 TN). These pilots show that BuildingBRep-11K is learnable yet non-trivial for both geometric regression and topological quality assessment
• 摘要：随着人工智能的兴起，自动生成建筑规模的3-D对象已成为一个积极的研究主题，但是培训此类模型仍然需要大型，干净且丰富的注释数据集。我们介绍了BuildingBrep-11k，该集合由11 978个多层（2-10层）建筑物（约10 GB）组成，该建筑物由形状 - 格拉玛驱动的管道生产，该管道编码已建立的建筑设计原理。每个样本都由一个几何确切的B-Rep固体覆盖地板，墙壁，平板和基于规则的开口，并带有快速载荷的.npy元数据文件，该文件记录了详细的人均参数。发电机在空间尺度，日光优化和内部布局上进行了约束，并且所得的对象通过多阶段过滤器，以消除布尔故障，尺寸不足的房间和极端长宽比，从而确保遵守建筑标准。为了验证数据集的可学习性，我们训练了两个轻巧的点网基线。 （i）多属性回归。一个编码器可预测4000点云的楼层计数，总房间，每层矢量和平均房间区域。在100个看不见的建筑物上，它达到了0.37层的MAE（$ \ pm1 $中的87 \％），5.7个房间MAE和3.2 m $ $^2 $ MAE在平均区域上。 （ii）缺陷检测。使用相同的主链，我们将良好与缺陷分类；在平衡的100模型设置上，网络达到54 \％的精度，以53 \％的精度召回82 \％的真实缺陷（41 tp，9 fn，37 fp，13 tn）。这些飞行员​​表明，对于几何回归和拓扑质量评估，BuildingBrep-11k是可学习的，却不繁琐

Title: Bohdi: Heterogeneous LLM Fusion with Automatic Data Exploration

• Authors: Junqi Gao, Zhichang Guo, Dazhi Zhang, Dong Li, Runze Liu, Pengfei Li, Kai Tian, Biqing Qi
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.15721
• Pdf URL: https://arxiv.org/pdf/2506.15721
• Copy Paste: [[2506.15721]] Bohdi: Heterogeneous LLM Fusion with Automatic Data Exploration(https://arxiv.org/abs/2506.15721)
• Keywords: generation
• Abstract: Heterogeneous Large Language Model (LLM) fusion integrates the strengths of multiple source LLMs with different architectures into a target LLM with low computational overhead. While promising, existing methods suffer from two major limitations: 1) reliance on real data from limited domain for knowledge fusion, preventing the target LLM from fully acquiring knowledge across diverse domains, and 2) fixed data allocation proportions across domains, failing to dynamically adjust according to the target LLM's varying capabilities across domains, leading to a capability imbalance. To overcome these limitations, we propose Bohdi, a synthetic-data-only heterogeneous LLM fusion framework. Through the organization of knowledge domains into a hierarchical tree structure, Bohdi enables automatic domain exploration and multi-domain data generation through multi-model collaboration, thereby comprehensively extracting knowledge from source LLMs. By formalizing domain expansion and data sampling proportion allocation on the knowledge tree as a Hierarchical Multi-Armed Bandit problem, Bohdi leverages the designed DynaBranches mechanism to adaptively adjust sampling proportions based on the target LLM's performance feedback across domains. Integrated with our proposed Introspection-Rebirth (IR) mechanism, DynaBranches dynamically tracks capability shifts during target LLM's updates via Sliding Window Binomial Likelihood Ratio Testing (SWBLRT), further enhancing its online adaptation capability. Comparative experimental results on a comprehensive suite of benchmarks demonstrate that Bohdi significantly outperforms existing baselines on multiple target LLMs, exhibits higher data efficiency, and virtually eliminates the imbalance in the target LLM's capabilities. Our code is available at this https URL.
• 摘要：异质的大语言模型（LLM）融合将具有不同体系结构的多个源LLM的优势整合到具有低计算开销的目标LLM中。在有希望的同时，现有方法遭受了两个主要局限性：1）依赖有限域中的实际数据以进行知识融合，以防止目标LLM完全获取跨不同领域的知识，以及2）跨域的固定数据分配比例，未能根据目标LLM跨域的不同域而动态调整，从而使能力IMPALINAME IMPALATION。为了克服这些局限性，我们提出了Bohdi，Bohdi是一种仅合成数据的非均质LLM融合框架。通过将知识领域的组织组织成层次的树结构，Bohdi可以通过多模型协作来实现自动域探索和多域数据生成，从而从源LLMS中全面提取知识。通过将知识树上的域扩展和数据采样比例分配为分层多臂匪徒问题，Bohdi利用设计的Dynabranch机制根据目标LLM跨域的性能反馈适应性地调整了采样比例。 Dynabranches与我们提出的内省rebirth（IR）机制集成在一起，通过滑动窗口二项式可能性比率测试（SWBLRT）在目标LLM更新过程中动态跟踪功能变化，从而进一步增强其在线适应能力。综合基准​​套件的比较实验结果表明，Bohdi在多个目标LLM上的现有基准显着优于现有的基准，显示出更高的数据效率，并且实际上消除了目标LLM功能的不平衡。我们的代码可在此HTTPS URL上找到。

Title: UniMate: A Unified Model for Mechanical Metamaterial Generation, Property Prediction, and Condition Confirmation

• Authors: Wangzhi Zhan, Jianpeng Chen, Dongqi Fu, Dawei Zhou
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.15722
• Pdf URL: https://arxiv.org/pdf/2506.15722
• Copy Paste: [[2506.15722]] UniMate: A Unified Model for Mechanical Metamaterial Generation, Property Prediction, and Condition Confirmation(https://arxiv.org/abs/2506.15722)
• Keywords: generation
• Abstract: Metamaterials are artificial materials that are designed to meet unseen properties in nature, such as ultra-stiffness and negative materials indices. In mechanical metamaterial design, three key modalities are typically involved, i.e., 3D topology, density condition, and mechanical property. Real-world complex application scenarios place the demanding requirements on machine learning models to consider all three modalities together. However, a comprehensive literature review indicates that most existing works only consider two modalities, e.g., predicting mechanical properties given the 3D topology or generating 3D topology given the required properties. Therefore, there is still a significant gap for the state-of-the-art machine learning models capturing the whole. Hence, we propose a unified model named UNIMATE, which consists of a modality alignment module and a synergetic diffusion generation module. Experiments indicate that UNIMATE outperforms the other baseline models in topology generation task, property prediction task, and condition confirmation task by up to 80.2%, 5.1%, and 50.2%, respectively. We opensource our proposed UNIMATE model and corresponding results at this https URL.
• 摘要：超材料是人工材料，旨在符合自然界中看不见的特性，例如超厚度和负面材料指数。在机械超材料设计中，通常涉及三种关键方式，即3D拓扑，密度条件和机械性能。现实世界中复杂的应用程序方案将苛刻的要求对机器学习模型进行了考虑，以将所有三种方式一起考虑。但是，全面的文献综述表明，大多数现有作品仅考虑两种模式，例如，考虑到所需属性，给定3D拓扑或生成3D拓扑的机械性能。因此，对于捕获整体的最先进的机器学习模型仍然存在很大的差距。因此，我们提出了一个统一的统一模型，该模型由模态比对模块和协同扩散生成模块组成。实验表明，在拓扑生成任务，属性预测任务和条件确认任务中，不超过其他基线模型的表现分别高达80.2％，5.1％和50.2％。我们在此HTTPS URL处开设了我们提出的联合模型和相应的结果。

Title: Graph Diffusion that can Insert and Delete

• Authors: Matteo Ninniri, Marco Podda, Davide Bacciu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.15725
• Pdf URL: https://arxiv.org/pdf/2506.15725
• Copy Paste: [[2506.15725]] Graph Diffusion that can Insert and Delete(https://arxiv.org/abs/2506.15725)
• Keywords: generation, generative
• Abstract: Generative models of graphs based on discrete Denoising Diffusion Probabilistic Models (DDPMs) offer a principled approach to molecular generation by systematically removing structural noise through iterative atom and bond adjustments. However, existing formulations are fundamentally limited by their inability to adapt the graph size (that is, the number of atoms) during the diffusion process, severely restricting their effectiveness in conditional generation scenarios such as property-driven molecular design, where the targeted property often correlates with the molecular size. In this paper, we reformulate the noising and denoising processes to support monotonic insertion and deletion of nodes. The resulting model, which we call GrIDDD, dynamically grows or shrinks the chemical graph during generation. GrIDDD matches or exceeds the performance of existing graph diffusion models on molecular property targeting despite being trained on a more difficult problem. Furthermore, when applied to molecular optimization, GrIDDD exhibits competitive performance compared to specialized optimization models. This work paves the way for size-adaptive molecular generation with graph diffusion.
• 摘要：基于离散的扩散概率模型（DDPM）的图形生成模型通过系统地消除结构噪声，通过迭代原子和键调节来实现分子产生的原则方法。但是，现有制剂从根本上受到了它们在扩散过程中无法适应图的大小（即原子的数量）的限制，从而严重限制了它们在条件生成场景（例如属性驱动的分子设计）中的有效性，而目标属性通常与分子大小相关。在本文中，我们重新制定了支持单调插入和删除节点的人们处和降解过程。我们称之为GridDD的结果模型会在发电过程中动态增长或收缩化学图。 GRIDDD匹配或超过分子属性靶向的现有图扩散模型的性能，尽管受到了更困难的问题的训练。此外，当应用于分子优化时，与专门优化模型相比，GridDD表现出竞争性能。这项工作为图形扩散的尺寸自适应分子产生铺平了道路。

Title: ADAM-Dehaze: Adaptive Density-Aware Multi-Stage Dehazing for Improved Object Detection in Foggy Conditions

• Authors: Fatmah AlHindaassi, Mohammed Talha Alam, Fakhri Karray
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.15837
• Pdf URL: https://arxiv.org/pdf/2506.15837
• Copy Paste: [[2506.15837]] ADAM-Dehaze: Adaptive Density-Aware Multi-Stage Dehazing for Improved Object Detection in Foggy Conditions(https://arxiv.org/abs/2506.15837)
• Keywords: restoration
• Abstract: Adverse weather conditions, particularly fog, pose a significant challenge to autonomous vehicles, surveillance systems, and other safety-critical applications by severely degrading visual information. We introduce ADAM-Dehaze, an adaptive, density-aware dehazing framework that jointly optimizes image restoration and object detection under varying fog intensities. A lightweight Haze Density Estimation Network (HDEN) classifies each input as light, medium, or heavy fog. Based on this score, the system dynamically routes the image through one of three CORUN branches: Light, Medium, or Complex, each tailored to its haze regime. A novel adaptive loss balances physical-model coherence and perceptual fidelity, ensuring both accurate defogging and preservation of fine details. On Cityscapes and the real-world RTTS benchmark, ADAM-Dehaze improves PSNR by up to 2.1 dB, reduces FADE by 30 percent, and increases object detection mAP by up to 13 points, while cutting inference time by 20 percent. These results highlight the importance of intensity-specific processing and seamless integration with downstream vision tasks. Code available at: this https URL.
• 摘要：不利的天气条件，尤其是雾气，对自动驾驶汽车，监视系统和其他关键安全性应用构成了重大挑战。我们介绍了Adam-Dehaze，这是一个自适应，密度吸引的飞机框架，在不同的雾强度下共同优化了图像恢复和对象检测。轻巧的雾密度估计网络（HDEN）将每个输入归类为光，中或重雾。基于此分数，系统通过三个Corun分支之一，将图像动态路由图像：光，中或复杂，每个分支都根据其雾霾制度量身定制。一种新颖的自适应损失可以平衡物理模型的连贯性和感知忠诚度，从而确保精确的融合和保存精细的细节。在CityScapes和现实世界中的RTTS基准上，Adam-Dehaze将PSNR提高了2.1 dB，将淡出淡出30％，并将对象检测图提高13分，同时将推理时间降低20％。这些结果突出了特定于强度的处理和与下游视觉任务的无缝集成的重要性。代码可用：此HTTPS URL。

Title: EchoShot: Multi-Shot Portrait Video Generation

• Authors: Jiahao Wang, Hualian Sheng, Sijia Cai, Weizhan Zhang, Caixia Yan, Yachuang Feng, Bing Deng, Jieping Ye
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.15838
• Pdf URL: https://arxiv.org/pdf/2506.15838
• Copy Paste: [[2506.15838]] EchoShot: Multi-Shot Portrait Video Generation(https://arxiv.org/abs/2506.15838)
• Keywords: generation, generative
• Abstract: Video diffusion models substantially boost the productivity of artistic workflows with high-quality portrait video generative capacity. However, prevailing pipelines are primarily constrained to single-shot creation, while real-world applications urge for multiple shots with identity consistency and flexible content controllability. In this work, we propose EchoShot, a native and scalable multi-shot framework for portrait customization built upon a foundation video diffusion model. To start with, we propose shot-aware position embedding mechanisms within video diffusion transformer architecture to model inter-shot variations and establish intricate correspondence between multi-shot visual content and their textual descriptions. This simple yet effective design enables direct training on multi-shot video data without introducing additional computational overhead. To facilitate model training within multi-shot scenario, we construct PortraitGala, a large-scale and high-fidelity human-centric video dataset featuring cross-shot identity consistency and fine-grained captions such as facial attributes, outfits, and dynamic motions. To further enhance applicability, we extend EchoShot to perform reference image-based personalized multi-shot generation and long video synthesis with infinite shot counts. Extensive evaluations demonstrate that EchoShot achieves superior identity consistency as well as attribute-level controllability in multi-shot portrait video generation. Notably, the proposed framework demonstrates potential as a foundational paradigm for general multi-shot video modeling.
• 摘要：视频扩散模型大大提高了具有高质量肖像视频生成能力的艺术工作流的生产率。但是，盛行的管道主要限制在单次创建中，而现实世界的应用程序则敦促具有身份一致性和灵活内容可控性的多个镜头。在这项工作中，我们提出了Echoshot，这是一个原始且可扩展的多拍框架，用于构建基础视频扩散模型的肖像自定义。首先，我们建议在视频扩散变压器体系结构中嵌入射击位置嵌入机制，以模拟射击之间的变化，并在多摄像视觉内容及其文本描述之间建立复杂的对应关系。这种简单而有效的设计可以直接培训多拍视频数据，而无需引入其他计算开销。为了促进多拍情景中的模型培训，我们构建了PortaraitGala，这是一个大规模且高保真的人以人为中心的视频数据集，具有交叉镜头身份一致性和细粒度的字幕，例如面部属性，服装，服装和动态运动。为了进一步提高适用性，我们扩展了Echoshot，以执行基于参考图像的个性化多弹射生成和长时间的视频合成，并具有无限的射击计数。广泛的评估表明，Echoshot在多拍肖像视频生成中实现了卓越的身份一致性以及属性级别的可控性。值得注意的是，所提出的框架表明了作为一般多拍视频建模的基础范式的潜力。

Title: Improving Rectified Flow with Boundary Conditions

• Authors: Xixi Hu, Runlong Liao, Keyang Xu, Bo Liu, Yeqing Li, Eugene Ie, Hongliang Fei, Qiang Liu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.15864
• Pdf URL: https://arxiv.org/pdf/2506.15864
• Copy Paste: [[2506.15864]] Improving Rectified Flow with Boundary Conditions(https://arxiv.org/abs/2506.15864)
• Keywords: generative
• Abstract: Rectified Flow offers a simple and effective approach to high-quality generative modeling by learning a velocity field. However, we identify a limitation in directly modeling the velocity with an unconstrained neural network: the learned velocity often fails to satisfy certain boundary conditions, leading to inaccurate velocity field estimations that deviate from the desired ODE. This issue is particularly critical during stochastic sampling at inference, as the score function's errors are amplified near the boundary. To mitigate this, we propose a Boundary-enforced Rectified Flow Model (Boundary RF Model), in which we enforce boundary conditions with a minimal code modification. Boundary RF Model improves performance over vanilla RF model, demonstrating 8.01% improvement in FID score on ImageNet using ODE sampling and 8.98% improvement using SDE sampling.
• 摘要：整流流通过学习速度字段，为高质量生成建模提供了一种简单有效的方法。但是，我们确定了直接用不受限制的神经网络对速度进行建模的限制：学习的速度通常无法满足某些边界条件，从而导致不准确的速度场估计偏离所需的ode。在推断时随机采样期间，由于得分函数的误差在边界附近放大，因此此问题尤其重要。为了减轻这种情况，我们提出了一个边界增强的整流流模型（边界RF模型），在该模型中，我们以最小的代码修改来执行边界条件。边界RF模型改善了Vanilla RF模型的性能，使用ODE采样表明ImageNet上的FID得分提高了8.01％，使用SDE采样提高了8.98％。

Title: VectorEdits: A Dataset and Benchmark for Instruction-Based Editing of Vector Graphics

• Authors: Josef Kuchař, Marek Kadlčík, Michal Spiegel, Michal Štefánik
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.15903
• Pdf URL: https://arxiv.org/pdf/2506.15903
• Copy Paste: [[2506.15903]] VectorEdits: A Dataset and Benchmark for Instruction-Based Editing of Vector Graphics(https://arxiv.org/abs/2506.15903)
• Keywords: generation
• Abstract: We introduce a large-scale dataset for instruction-guided vector image editing, consisting of over 270,000 pairs of SVG images paired with natural language edit instructions. Our dataset enables training and evaluation of models that modify vector graphics based on textual commands. We describe the data collection process, including image pairing via CLIP similarity and instruction generation with vision-language models. Initial experiments with state-of-the-art large language models reveal that current methods struggle to produce accurate and valid edits, underscoring the challenge of this task. To foster research in natural language-driven vector graphic generation and editing, we make our resources created within this work publicly available.
• 摘要：我们引入了一个大规模数据集，用于指导引导的矢量图像编辑，其中包括270,000对SVG图像与自然语言编辑说明配对。我们的数据集可以根据文本命令对矢量图形进行修改的模型进行培训和评估。我们描述了数据收集过程，包括通过剪辑相似性和通过视觉模型生成的指导生成图像配对。对最先进的大语言模型的初步实验表明，当前的方法难以生成准确有效的编辑，强调了这项任务的挑战。为了促进自然语言驱动的矢量图形生成和编辑的研究，我们在这项工作中公开创建了资源。

Title: MoiréXNet: Adaptive Multi-Scale Demoiréing with Linear Attention Test-Time Training and Truncated Flow Matching Prior

• Authors: Liangyan Li, Yimo Ning, Kevin Le, Wei Dong, Yunzhe Li, Jun Chen, Xiaohong Liu
• Subjects: cs.CV, cs.AI, eess.IV
• Abstract URL: https://arxiv.org/abs/2506.15929
• Pdf URL: https://arxiv.org/pdf/2506.15929
• Copy Paste: [[2506.15929]] MoiréXNet: Adaptive Multi-Scale Demoiréing with Linear Attention Test-Time Training and Truncated Flow Matching Prior(https://arxiv.org/abs/2506.15929)
• Keywords: restoration, generative
• Abstract: This paper introduces a novel framework for image and video demoiréing by integrating Maximum A Posteriori (MAP) estimation with advanced deep learning techniques. Demoiréing addresses inherently nonlinear degradation processes, which pose significant challenges for existing methods. Traditional supervised learning approaches either fail to remove moiré patterns completely or produce overly smooth results. This stems from constrained model capacity and scarce training data, which inadequately represent the clean image distribution and hinder accurate reconstruction of ground-truth images. While generative models excel in image restoration for linear degradations, they struggle with nonlinear cases such as demoiréing and often introduce artifacts. To address these limitations, we propose a hybrid MAP-based framework that integrates two complementary components. The first is a supervised learning model enhanced with efficient linear attention Test-Time Training (TTT) modules, which directly learn nonlinear mappings for RAW-to-sRGB demoiréing. The second is a Truncated Flow Matching Prior (TFMP) that further refines the outputs by aligning them with the clean image distribution, effectively restoring high-frequency details and suppressing artifacts. These two components combine the computational efficiency of linear attention with the refinement abilities of generative models, resulting in improved restoration performance.
• 摘要：本文通过将最大后验（MAP）估计与先进的深度学习技术整合在一起，引入了图像和视频演示的新颖框架。 Demoiréing解决了固有的非线性退化过程，这对现有方法构成了重大挑战。传统的监督学习方法要么无法完全删除Moiré模式，要么产生过度平滑的结果。这源于限制的模型容量和稀缺的训练数据，这代表了干净的图像分布和阻碍地面图像的准确重建。尽管生成模型在图像恢复方面出色地降低了线性降解，但它们在诸如Demoiréing之类的非线性案例中挣扎，并且经常引入人工制品。为了解决这些限制，我们提出了一个基于混合地图的框架，该框架集成了两个互补组件。第一个是通过有效的线性注意测试时间培训（TTT）模块增强的监督学习模型，该模块直接学习了用于原始启用demoiréing的非线性映射。第二个是截断的流量匹配先验（TFMP），它通过使输出与干净的图像分布对齐，有效地恢复高频细节并抑制伪像，从而进一步完善了输出。这两个组件将线性注意力的计算效率与生成模型的完善能力相结合，从而改善了恢复性能。

Title: CORAL: Disentangling Latent Representations in Long-Tailed Diffusion

• Authors: Esther Rodriguez, Monica Welfert, Samuel McDowell, Nathan Stromberg, Julian Antolin Camarena, Lalitha Sankar
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.15933
• Pdf URL: https://arxiv.org/pdf/2506.15933
• Copy Paste: [[2506.15933]] CORAL: Disentangling Latent Representations in Long-Tailed Diffusion(https://arxiv.org/abs/2506.15933)
• Keywords: generation
• Abstract: Diffusion models have achieved impressive performance in generating high-quality and diverse synthetic data. However, their success typically assumes a class-balanced training distribution. In real-world settings, multi-class data often follow a long-tailed distribution, where standard diffusion models struggle -- producing low-diversity and lower-quality samples for tail classes. While this degradation is well-documented, its underlying cause remains poorly understood. In this work, we investigate the behavior of diffusion models trained on long-tailed datasets and identify a key issue: the latent representations (from the bottleneck layer of the U-Net) for tail class subspaces exhibit significant overlap with those of head classes, leading to feature borrowing and poor generation quality. Importantly, we show that this is not merely due to limited data per class, but that the relative class imbalance significantly contributes to this phenomenon. To address this, we propose COntrastive Regularization for Aligning Latents (CORAL), a contrastive latent alignment framework that leverages supervised contrastive losses to encourage well-separated latent class representations. Experiments demonstrate that CORAL significantly improves both the diversity and visual quality of samples generated for tail classes relative to state-of-the-art methods.
• 摘要：扩散模型在产生高质量和多样化的合成数据方面取得了令人印象深刻的性能。但是，他们的成功通常假定级别平衡的培训分布。在现实世界中，多级数据经常遵循长尾巴分布，标准扩散模型在其中挣扎 - 为尾部类别产生低多样性和低质量的样本。尽管这种退化有充分的文献记录，但其根本原因仍然很众所周知。在这项工作中，我们研究了在长尾数据集上训练的扩散模型的行为，并确定了一个关键问题：尾部类子空间的潜在表示（从U-NET的瓶颈层）表现出与头等阶级的质量重叠，从而导致特征借贷和发电质量差。重要的是，我们表明，这不仅是由于每个类别的数据有限，而且相对类别的失衡显着促进了这一现象。为了解决这个问题，我们提出了对齐潜伏期（珊瑚）的对比度正规化，这是一个对比的潜在对准框架，利用监督的对比损失来鼓励分离良好的潜在阶级表示。实验表明，相对于最新方法，珊瑚显着提高了尾部类别产生的样品的多样性和视觉质量。

Title: Advanced Sign Language Video Generation with Compressed and Quantized Multi-Condition Tokenization

• Authors: Cong Wang, Zexuan Deng, Zhiwei Jiang, Fei Shen, Yafeng Yin, Shiwei Gan, Zifeng Cheng, Shiping Ge, Qing Gu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.15980
• Pdf URL: https://arxiv.org/pdf/2506.15980
• Copy Paste: [[2506.15980]] Advanced Sign Language Video Generation with Compressed and Quantized Multi-Condition Tokenization(https://arxiv.org/abs/2506.15980)
• Keywords: generation
• Abstract: Sign Language Video Generation (SLVG) seeks to generate identity-preserving sign language videos from spoken language texts. Existing methods primarily rely on the single coarse condition (\eg, skeleton sequences) as the intermediary to bridge the translation model and the video generation model, which limits both the naturalness and expressiveness of the generated videos. To overcome these limitations, we propose SignViP, a novel SLVG framework that incorporates multiple fine-grained conditions for improved generation fidelity. Rather than directly translating error-prone high-dimensional conditions, SignViP adopts a discrete tokenization paradigm to integrate and represent fine-grained conditions (\ie, fine-grained poses and 3D hands). SignViP contains three core components. (1) Sign Video Diffusion Model is jointly trained with a multi-condition encoder to learn continuous embeddings that encapsulate fine-grained motion and appearance. (2) Finite Scalar Quantization (FSQ) Autoencoder is further trained to compress and quantize these embeddings into discrete tokens for compact representation of the conditions. (3) Multi-Condition Token Translator is trained to translate spoken language text to discrete multi-condition tokens. During inference, Multi-Condition Token Translator first translates the spoken language text into discrete multi-condition tokens. These tokens are then decoded to continuous embeddings by FSQ Autoencoder, which are subsequently injected into Sign Video Diffusion Model to guide video generation. Experimental results show that SignViP achieves state-of-the-art performance across metrics, including video quality, temporal coherence, and semantic fidelity. The code is available at this https URL.
• 摘要：手语视频生成（SLVG）旨在从口语文本中生成具有身份的手语视频。现有方法主要依赖于单个粗糙条件（\ eg，骨架序列）作为桥接翻译模型和视频生成模型的中介，这限制了生成的视频的自然性和表现力。为了克服这些局限性，我们提出了SignVip，这是一个新型的SLVG框架，结合了多个细粒条件，以改善产生忠诚度。 Signvip并没有直接翻译易于误差的高维条件，而是采用离散的令牌化范式来整合和表示细粒度的条件（\ ie，细粒度的姿势和3D手）。 Signvip包含三个核心组件。 （1）符号视频扩散模型由多条件编码器共同训练，以学习封装细粒运动和外观的连续嵌入。 （2）进一步训练有限标量量化（FSQ）自动编码器，以压缩和量化这些嵌入到离散令牌中以进行条件的紧凑表示。 （3）对多条件令牌翻译器进行了训练，可以将口语文本翻译成离散的多条件令牌。在推断期间，多条件令牌翻译器首先将口语文本转换为离散的多条件令牌。然后将这些令牌解码为FSQ自动编码器连续嵌入，随后将其注入符号视频扩散模型以指导视频生成。实验结果表明，Signvip在指标上实现了最先进的性能，包括视频质量，时间连贯性和语义保真度。该代码可在此HTTPS URL上找到。

Title: DIGMAPPER: A Modular System for Automated Geologic Map Digitization

• Authors: Weiwei Duan, Michael P. Gerlek, Steven N. Minton, Craig A. Knoblock, Fandel Lin, Theresa Chen, Leeje Jang, Sofia Kirsanova, Zekun Li, Yijun Lin, Yao-Yi Chiang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.16006
• Pdf URL: https://arxiv.org/pdf/2506.16006
• Copy Paste: [[2506.16006]] DIGMAPPER: A Modular System for Automated Geologic Map Digitization(https://arxiv.org/abs/2506.16006)
• Keywords: generation
• Abstract: Historical geologic maps contain rich geospatial information, such as rock units, faults, folds, and bedding planes, that is critical for assessing mineral resources essential to renewable energy, electric vehicles, and national security. However, digitizing maps remains a labor-intensive and time-consuming task. We present DIGMAPPER, a modular, scalable system developed in collaboration with the United States Geological Survey (USGS) to automate the digitization of geologic maps. DIGMAPPER features a fully dockerized, workflow-orchestrated architecture that integrates state-of-the-art deep learning models for map layout analysis, feature extraction, and georeferencing. To overcome challenges such as limited training data and complex visual content, our system employs innovative techniques, including in-context learning with large language models, synthetic data generation, and transformer-based models. Evaluations on over 100 annotated maps from the DARPA-USGS dataset demonstrate high accuracy across polygon, line, and point feature extraction, and reliable georeferencing performance. Deployed at USGS, DIGMAPPER significantly accelerates the creation of analysis-ready geospatial datasets, supporting national-scale critical mineral assessments and broader geoscientific applications.
• 摘要：历史地质地图包含丰富的地理空间信息，例如岩石单元，断层，褶皱和床上用品飞机，这对于评估可再生能源，电动汽车和国家安全至关重要的矿产资源至关重要。但是，数字化地图仍然是一项劳动密集型且耗时的任务。我们提出了Digmapper，这是一种与美国地质调查局（USGS）合作开发的模块化，可扩展的系统，以自动化地质图的数字化。 DigMapper具有完全停靠的，工作流程的架构，该体系结构集成了最新的深度学习模型，以进行地图布局分析，功能提取和地理处理。为了克服诸如有限的培训数据和复杂的视觉内容之类的挑战，我们的系统采用创新技术，包括使用大语言模型，合成数据生成和基于变压器的模型的文化学习。对来自DARPA-USGS数据集的100多个注释图的评估表明，多边形，线和点特征提取以及可靠的地理发作性能的高精度。 DigMapper部署在USGS，大大加快了可以分析的地理空间数据集的创建，支持国家规模的关键矿物质评估和更广泛的地球科学应用。

Title: Vision-Guided Chunking Is All You Need: Enhancing RAG with Multimodal Document Understanding

• Authors: Vishesh Tripathi, Tanmay Odapally, Indraneel Das, Uday Allu, Biddwan Ahmed
• Subjects: cs.LG, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2506.16035
• Pdf URL: https://arxiv.org/pdf/2506.16035
• Copy Paste: [[2506.16035]] Vision-Guided Chunking Is All You Need: Enhancing RAG with Multimodal Document Understanding(https://arxiv.org/abs/2506.16035)
• Keywords: generation
• Abstract: Retrieval-Augmented Generation (RAG) systems have revolutionized information retrieval and question answering, but traditional text-based chunking methods struggle with complex document structures, multi-page tables, embedded figures, and contextual dependencies across page boundaries. We present a novel multimodal document chunking approach that leverages Large Multimodal Models (LMMs) to process PDF documents in batches while maintaining semantic coherence and structural integrity. Our method processes documents in configurable page batches with cross-batch context preservation, enabling accurate handling of tables spanning multiple pages, embedded visual elements, and procedural content. We evaluate our approach on a curated dataset of PDF documents with manually crafted queries, demonstrating improvements in chunk quality and downstream RAG performance. Our vision-guided approach achieves better accuracy compared to traditional vanilla RAG systems, with qualitative analysis showing superior preservation of document structure and semantic coherence.
• 摘要：检索增强的生成（RAG）系统已彻底改变了信息检索和问答，但是传统的基于文本的分解方法与复杂的文档结构，多页表，嵌入式图形以及跨页面边界的上下文依赖关系而努力。我们提出了一种新型的多模式文档块方法，该方法利用大型多模型模型（LMM）分批处理PDF文档，同时保持语义相干性和结构完整性。我们的方法处理可配置的页面批处理中的文档，并具有跨批处理上下文保存，从而可以准确处理跨越多个页面，嵌入式视觉元素和过程内容的表。我们在使用手动制作的查询的PDF文档数据集上评估了我们的方法，这表明了块质量和下游抹布性能的改善。与传统的香草抹布系统相比，我们的视觉指导方法的准确性更高，定性分析表明文档结构和语义连贯性的卓越保存。

Title: PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and Quantized Attention in Visual Generation Models

• Authors: Tianchen Zhao, Ke Hong, Xinhao Yang, Xuefeng Xiao, Huixia Li, Feng Ling, Ruiqi Xie, Siqi Chen, Hongyu Zhu, Yichong Zhang, Yu Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16054
• Pdf URL: https://arxiv.org/pdf/2506.16054
• Copy Paste: [[2506.16054]] PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and Quantized Attention in Visual Generation Models(https://arxiv.org/abs/2506.16054)
• Keywords: generation
• Abstract: In visual generation, the quadratic complexity of attention mechanisms results in high memory and computational costs, especially for longer token sequences required in high-resolution image or multi-frame video generation. To address this, prior research has explored techniques such as sparsification and quantization. However, these techniques face significant challenges under low density and reduced bitwidths. Through systematic analysis, we identify that the core difficulty stems from the dispersed and irregular characteristics of visual attention patterns. Therefore, instead of introducing specialized sparsification and quantization design to accommodate such patterns, we propose an alternative strategy: *reorganizing* the attention pattern to alleviate the challenges. Inspired by the local aggregation nature of visual feature extraction, we design a novel **Pattern-Aware token ReOrdering (PARO)** technique, which unifies the diverse attention patterns into a hardware-friendly block-wise pattern. This unification substantially simplifies and enhances both sparsification and quantization. We evaluate the performance-efficiency trade-offs of various design choices and finalize a methodology tailored for the unified pattern. Our approach, **PAROAttention**, achieves video and image generation with lossless metrics, and nearly identical results from full-precision (FP) baselines, while operating at notably lower density (~20%-30%) and bitwidth (**INT8/INT4**), achieving a **1.9x** to **2.7x** end-to-end latency speedup.
• 摘要：在视觉生成中，注意机制的二次复杂性会导致高记忆和计算成本，尤其是对于高分辨率图像或多帧视频生成中所需的更长的令牌序列。为了解决这个问题，先前的研究探索了诸如稀疏和量化之类的技术。但是，这些技术在低密度和降低的位宽度下面临重大挑战。通过系统的分析，我们确定核心难度来自视觉注意模式的分散和不规则特征。因此，我们提出了一种替代策略： *重组 *注意力模式以减轻挑战。受视觉特征提取的局部聚合性质的启发，我们设计了一种新颖的**模式感知令牌重新排序（Paro）**技术，该技术将各种注意力的注意力模式统一为适合硬件友好的块状模式。这种统一大大简化并增强了稀疏和量化。我们评估了各种设计选择的性能效率权衡，并最终确定了针对统一模式的方法。 Our approach, **PAROAttention**, achieves video and image generation with lossless metrics, and nearly identical results from full-precision (FP) baselines, while operating at notably lower density (~20%-30%) and bitwidth (**INT8/INT4**), achieving a **1.9x** to **2.7x** end-to-end latency speedup.

Title: STAR-Pose: Efficient Low-Resolution Video Human Pose Estimation via Spatial-Temporal Adaptive Super-Resolution

• Authors: Yucheng Jin, Jinyan Chen, Ziyue He, Baojun Han, Furan An
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16061
• Pdf URL: https://arxiv.org/pdf/2506.16061
• Copy Paste: [[2506.16061]] STAR-Pose: Efficient Low-Resolution Video Human Pose Estimation via Spatial-Temporal Adaptive Super-Resolution(https://arxiv.org/abs/2506.16061)
• Keywords: super-resolution
• Abstract: Human pose estimation in low-resolution videos presents a fundamental challenge in computer vision. Conventional methods either assume high-quality inputs or employ computationally expensive cascaded processing, which limits their deployment in resource-constrained environments. We propose STAR-Pose, a spatial-temporal adaptive super-resolution framework specifically designed for video-based human pose estimation. Our method features a novel spatial-temporal Transformer with LeakyReLU-modified linear attention, which efficiently captures long-range temporal dependencies. Moreover, it is complemented by an adaptive fusion module that integrates parallel CNN branch for local texture enhancement. We also design a pose-aware compound loss to achieve task-oriented super-resolution. This loss guides the network to reconstruct structural features that are most beneficial for keypoint localization, rather than optimizing purely for visual quality. Extensive experiments on several mainstream video HPE datasets demonstrate that STAR-Pose outperforms existing approaches. It achieves up to 5.2% mAP improvement under extremely low-resolution (64x48) conditions while delivering 2.8x to 4.4x faster inference than cascaded approaches.
• 摘要：低分辨率视频中的人类姿势估计提出了计算机视觉中的基本挑战。常规方法要么假设高质量输入，要么采用计算昂贵的级联处理，从而限制了其在资源约束环境中的部署。我们提出了Star-Pose，这是一种空间自适应的超级分辨率框架，专为基于视频的人姿势估计而设计。我们的方法具有一种新型的时空变压器，并具有泄漏的线性注意力，从而有效地捕获了远程时间依赖性。此外，它与自适应融合模块相辅相成，该模块集成了并行CNN分支以增强局部纹理。我们还设计了姿势感知的复合损失，以实现面向任务的超分辨率。这种损失指导网络重建对关键点本地化最有益的结构特征，而不是仅对视觉质量进行优化。在几个主流视频HPE数据集上进行了广泛的实验表明，星形姿势的表现优于现有方法。在极低的分辨率（64x48）条件下，它的地图提高高达5.2％，同时提供2.8倍至4.4倍的推理，其推理比级联方法快。

Title: PR-DETR: Injecting Position and Relation Prior for Dense Video Captioning

• Authors: Yizhe Li, Sanping Zhou, Zheng Qin, Le Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16082
• Pdf URL: https://arxiv.org/pdf/2506.16082
• Copy Paste: [[2506.16082]] PR-DETR: Injecting Position and Relation Prior for Dense Video Captioning(https://arxiv.org/abs/2506.16082)
• Keywords: generation
• Abstract: Dense video captioning is a challenging task that aims to localize and caption multiple events in an untrimmed video. Recent studies mainly follow the transformer-based architecture to jointly perform the two sub-tasks, i.e., event localization and caption generation, in an end-to-end manner. Based on the general philosophy of detection transformer, these methods implicitly learn the event locations and event semantics, which requires a large amount of training data and limits the model's performance in practice. In this paper, we propose a novel dense video captioning framework, named PR-DETR, which injects the explicit position and relation prior into the detection transformer to improve the localization accuracy and caption quality, simultaneously. On the one hand, we first generate a set of position-anchored queries to provide the scene-specific position and semantic information about potential events as position prior, which serves as the initial event search regions to eliminate the implausible event proposals. On the other hand, we further design an event relation encoder to explicitly calculate the relationship between event boundaries as relation prior to guide the event interaction to improve the semantic coherence of the captions. Extensive ablation studies are conducted to verify the effectiveness of the position and relation prior. Experimental results also show the competitive performance of our method on ActivityNet Captions and YouCook2 datasets.
• 摘要：密集的视频字幕是一项具有挑战性的任务，旨在在未修饰的视频中本地化和标题多个事件。最近的研究主要遵循基于变压器的体系结构以端到端的方式共同执行两个子任务，即事件定位和标题生成。这些方法基于检测变压器的一般理念，隐含地了解事件位置和事件语义，这需要大量的培训数据并限制模型在实践中的性能。在本文中，我们提出了一个名为PR-Detr的新型致密视频字幕框架，该框架将显式位置和关系注入检测变压器中，以同时提高本地化精度和标题质量。一方面，我们首先生成了一组位置锚定的查询，以提供针对现场的位置和语义信息，以作为先验位置的潜在事件，作为先验的位置，该位置是初步的事件搜索区域，以消除不可信的事件建议。另一方面，我们进一步设计了事件关系编码器，以在指导事件交互之前明确计算事件边界之间的关系，以提高字幕的语义连贯性。进行了广泛的消融研究，以验证职位和关系的有效性。实验结果还显示了我们在活动网字幕和YouCook2数据集上的方法的竞争性能。

Title: FastInit: Fast Noise Initialization for Temporally Consistent Video Generation

• Authors: Chengyu Bai, Yuming Li, Zhongyu Zhao, Jintao Chen, Peidong Jia, Qi She, Ming Lu, Shanghang Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16119
• Pdf URL: https://arxiv.org/pdf/2506.16119
• Copy Paste: [[2506.16119]] FastInit: Fast Noise Initialization for Temporally Consistent Video Generation(https://arxiv.org/abs/2506.16119)
• Keywords: generation
• Abstract: Video generation has made significant strides with the development of diffusion models; however, achieving high temporal consistency remains a challenging task. Recently, FreeInit identified a training-inference gap and introduced a method to iteratively refine the initial noise during inference. However, iterative refinement significantly increases the computational cost associated with video generation. In this paper, we introduce FastInit, a fast noise initialization method that eliminates the need for iterative refinement. FastInit learns a Video Noise Prediction Network (VNPNet) that takes random noise and a text prompt as input, generating refined noise in a single forward pass. Therefore, FastInit greatly enhances the efficiency of video generation while achieving high temporal consistency across frames. To train the VNPNet, we create a large-scale dataset consisting of pairs of text prompts, random noise, and refined noise. Extensive experiments with various text-to-video models show that our method consistently improves the quality and temporal consistency of the generated videos. FastInit not only provides a substantial improvement in video generation but also offers a practical solution that can be applied directly during inference. The code and dataset will be released.
• 摘要：视频产生在扩散模型的发展中取得了长足的进步。但是，达到高度的时间一致性仍然是一项具有挑战性的任务。最近，FreeInit确定了训练缝隙，并引入了一种迭代在推断过程中的初始噪声的方法。但是，迭代精致可显着增加与视频生成相关的计算成本。在本文中，我们介绍了FastInit，这是一种快速的噪声初始化方法，可以消除对迭代精致的需求。 FastInit学习了一个视频噪声预测网络（VNPNET），该网络将随机噪声和文本提示作为输入，从而在单个正向传球中产生精致的噪声。因此，FastInit大大提高了视频生成的效率，同时在整个框架之间实现了高度的时间一致性。为了训练VNPNET，我们创建了一个大规模数据集，该数据集由成对的文本提示，随机噪声和精制噪声组成。通过各种文本对视频模型进行的广泛实验表明，我们的方法一致地提高了生成的视频的质量和时间一致性。 FastInit不仅提供了视频生成的实质性改进，而且还提供了可以在推理期间直接应用的实用解决方案。代码和数据集将发布。

Title: MBA: Multimodal Bidirectional Attack for Referring Expression Segmentation Models

• Authors: Xingbai Chen, Tingchao Fu, Renyang Liu, Wei Zhou, Chao Yi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16157
• Pdf URL: https://arxiv.org/pdf/2506.16157
• Copy Paste: [[2506.16157]] MBA: Multimodal Bidirectional Attack for Referring Expression Segmentation Models(https://arxiv.org/abs/2506.16157)
• Keywords: generation
• Abstract: Referring Expression Segmentation (RES) enables precise object segmentation in images based on natural language descriptions, offering high flexibility and broad applicability in real-world vision tasks. Despite its impressive performance, the robustness of RES models against adversarial examples remains largely unexplored. While prior adversarial attack methods have explored adversarial robustness on conventional segmentation models, they perform poorly when directly applied to RES, failing to expose vulnerabilities in its multimodal structure. Moreover, in practical open-world scenarios, users typically issue multiple, diverse referring expressions to interact with the same image, highlighting the need for adversarial examples that generalize across varied textual inputs. To address these multimodal challenges, we propose a novel adversarial attack strategy termed \textbf{Multimodal Bidirectional Attack}, tailored for RES models. Our method introduces learnable proxy textual embedding perturbation and jointly performs visual-aligned optimization on the image modality and textual-adversarial optimization on the textual modality during attack generation. This dual optimization framework encourages adversarial images to actively adapt to more challenging text embedding during optimization, thereby enhancing their cross-text transferability, which refers to the ability of adversarial examples to remain effective under a variety of unseen or semantically diverse textual inputs. Extensive experiments conducted on multiple RES models and benchmark datasets demonstrate the superior effectiveness of our method compared to existing methods.
• 摘要：参考表达细分（RES）可以根据自然语言描述在图像中进行精确的对象分割，从而在现实世界视觉任务中具有很高的灵活性和广泛的适用性。尽管表现令人印象深刻，但RES模型针对对抗性例子的鲁棒性仍然在很大程度上没有探索。虽然先前的对抗攻击方法探索了传统分割模型上的对抗性鲁棒性，但直接应用于RES时，它们的性能很差，但未能在其多模式结构中暴露漏洞。此外，在实际的开放世界情景下，用户通常会发出多种，不同的参考表达式以与同一图像进行交互，从而强调了对对抗性示例的需求，这些示例的需求跨越了各种文本输入。为了应对这些多模式挑战，我们提出了一种新颖的对抗攻击策略，称为\ textbf {多模式双向攻击}，该攻击是针对RES模型量身定制的。我们的方法介绍了可学习的代理文本嵌入扰动，并在攻击生成过程中对图像模态和文本对话的优化共同执行了视觉对齐的优化。这个双重优化框架鼓励对抗性图像在优化过程中积极适应更具挑战性的文本嵌入，从而增强其跨文本传递性，这是指对抗性示例在各种看不见或语义上多样的文本投入下保持有效的能力。与现有方法相比，在多个RES模型和基准数据集上进行的广泛实验证明了我们方法的较高有效性。

Title: Co-Speech Gesture and Facial Expression Generation for Non-Photorealistic 3D Characters

• Authors: Taisei Omine (1), Naoyuki Kawabata (1), Fuminori Homma (1) ((1) Sony Group Corporation)
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16159
• Pdf URL: https://arxiv.org/pdf/2506.16159
• Copy Paste: [[2506.16159]] Co-Speech Gesture and Facial Expression Generation for Non-Photorealistic 3D Characters(https://arxiv.org/abs/2506.16159)
• Keywords: generation
• Abstract: With the advancement of conversational AI, research on bodily expressions, including gestures and facial expressions, has also progressed. However, many existing studies focus on photorealistic avatars, making them unsuitable for non-photorealistic characters, such as those found in anime. This study proposes methods for expressing emotions, including exaggerated expressions unique to non-photorealistic characters, by utilizing expression data extracted from comics and dialogue-specific semantic gestures. A user study demonstrated significant improvements across multiple aspects when compared to existing research.
• 摘要：随着对话AI的发展，关于身体表达的研究，包括手势和面部表情，也取得了进步。但是，许多现有的研究集中于情感化的化身，使其不适合非遗嘱认证特征，例如在动漫中发现的字符。这项研究提出了表达情绪的方法，包括利用从漫画和特定于对话的语义手势中提取的表达数据，包括非遗迹特征独有的夸张表达式。与现有研究相比，一项用户研究表明，各个方面的改善都有显着改善。

Title: Integrating Generative Adversarial Networks and Convolutional Neural Networks for Enhanced Traffic Accidents Detection and Analysis

• Authors: Zhenghao Xi, Xiang Liu, Yaqi Liu, Yitong Cai, Yangyu Zheng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16186
• Pdf URL: https://arxiv.org/pdf/2506.16186
• Copy Paste: [[2506.16186]] Integrating Generative Adversarial Networks and Convolutional Neural Networks for Enhanced Traffic Accidents Detection and Analysis(https://arxiv.org/abs/2506.16186)
• Keywords: generative
• Abstract: Accident detection using Closed Circuit Television (CCTV) footage is one of the most imperative features for enhancing transport safety and efficient traffic control. To this end, this research addresses the issues of supervised monitoring and data deficiency in accident detection systems by adapting excellent deep learning technologies. The motivation arises from rising statistics in the number of car accidents worldwide; this calls for innovation and the establishment of a smart, efficient and automated way of identifying accidents and calling for help to save lives. Addressing the problem of the scarcity of data, the presented framework joins Generative Adversarial Networks (GANs) for synthesizing data and Convolutional Neural Networks (CNN) for model training. Video frames for accidents and non-accidents are collected from YouTube videos, and we perform resizing, image enhancement and image normalisation pixel range adjustments. Three models are used: CNN, Fine-tuned Convolutional Neural Network (FTCNN) and Vision Transformer (VIT) worked best for detecting accidents from CCTV, obtaining an accuracy rate of 94% and 95%, while the CNN model obtained 88%. Such results show that the proposed framework suits traffic safety applications due to its high real-time accident detection capabilities and broad-scale applicability. This work lays the foundation for intelligent surveillance systems in the future for real-time traffic monitoring, smart city framework, and integration of intelligent surveillance systems into emergency management systems.
• 摘要：使用闭路电视（CCTV）录像的事故检测是提高运输安全和有效交通控制的最必要的特征之一。为此，本研究通过调整出色的深度学习技术来解决事故检测系统中监督监测和数据缺陷的问题。动机是由于全球汽车事故数量的统计数据不断上升。这需要创新，并建立一种聪明，高效和自动化的方式来确定事故并呼吁挽救生命。在解决数据稀缺问题的问题上，提出的框架与生成的对抗网络（GAN）一起用于合成数据和卷积神经网络（CNN）进行模型培训。从YouTube视频中收集了事故和非现场的视频帧，我们进行调整，图像增强和图像标准化像素范围调整。使用了三种模型：CNN，微调的卷积神经网络（FTCNN）和视觉变压器（VIT）最适合检测CCTV的事故，获得94％和95％的准确率，而CNN模型获得了88％。这样的结果表明，该拟议的框架适合交通安全应用，因为其高实时事故检测功能和广泛的适用性。这项工作为未来的智能监视系统奠定了基础，用于实时交通监控，智能城市框架以及将智能监视系统集成到紧急管理系统中。

Title: VideoGAN-based Trajectory Proposal for Automated Vehicles

• Authors: Annajoyce Mariani, Kira Maag, Hanno Gottschalk
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.16209
• Pdf URL: https://arxiv.org/pdf/2506.16209
• Copy Paste: [[2506.16209]] VideoGAN-based Trajectory Proposal for Automated Vehicles(https://arxiv.org/abs/2506.16209)
• Keywords: generative
• Abstract: Being able to generate realistic trajectory options is at the core of increasing the degree of automation of road vehicles. While model-driven, rule-based, and classical learning-based methods are widely used to tackle these tasks at present, they can struggle to effectively capture the complex, multimodal distributions of future trajectories. In this paper we investigate whether a generative adversarial network (GAN) trained on videos of bird's-eye view (BEV) traffic scenarios can generate statistically accurate trajectories that correctly capture spatial relationships between the agents. To this end, we propose a pipeline that uses low-resolution BEV occupancy grid videos as training data for a video generative model. From the generated videos of traffic scenarios we extract abstract trajectory data using single-frame object detection and frame-to-frame object matching. We particularly choose a GAN architecture for the fast training and inference times with respect to diffusion models. We obtain our best results within 100 GPU hours of training, with inference times under 20\,ms. We demonstrate the physical realism of the proposed trajectories in terms of distribution alignment of spatial and dynamic parameters with respect to the ground truth videos from the Waymo Open Motion Dataset.
• 摘要：能够产生逼真的轨迹选择是提高道路车辆自动化程度的核心。尽管以模型为导向的，基于规则和基于经典的学习方法被广泛用于解决这些任务，但他们可能难以有效地捕获未来轨迹的复杂，多模式分布。在本文中，我们研究了对鸟类视图（BEV）视频培训的生成对抗网络（GAN）是否可以生成统计准确的轨迹，从而正确捕获代理之间的空间关系。为此，我们提出了一条使用低分辨率BEV占用网格视频作为视频生成模型的培训数据的管道。从流量方案的生成视频中，我们使用单帧对象检测和框架对象匹配提取抽象轨迹数据。我们特别选择用于扩散模型的快速训练和推理时间的GAN体系结构。我们在100 GPU小时的培训中获得最佳结果，推理时间低于20 \，MS。我们以空间和动态参数相对于Waymo Open Motion DataSet的地面真相视频的分布对齐方式来证明所提出的轨迹的物理现实主义。

Title: Active MRI Acquisition with Diffusion Guided Bayesian Experimental Design

• Authors: Jacopo Iollo, Geoffroy Oudoumanessah, Carole Lartizien, Michel Dojat, Florence Forbes
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.16237
• Pdf URL: https://arxiv.org/pdf/2506.16237
• Copy Paste: [[2506.16237]] Active MRI Acquisition with Diffusion Guided Bayesian Experimental Design(https://arxiv.org/abs/2506.16237)
• Keywords: generative
• Abstract: A key challenge in maximizing the benefits of Magnetic Resonance Imaging (MRI) in clinical settings is to accelerate acquisition times without significantly degrading image quality. This objective requires a balance between under-sampling the raw k-space measurements for faster acquisitions and gathering sufficient raw information for high-fidelity image reconstruction and analysis tasks. To achieve this balance, we propose to use sequential Bayesian experimental design (BED) to provide an adaptive and task-dependent selection of the most informative measurements. Measurements are sequentially augmented with new samples selected to maximize information gain on a posterior distribution over target images. Selection is performed via a gradient-based optimization of a design parameter that defines a subsampling pattern. In this work, we introduce a new active BED procedure that leverages diffusion-based generative models to handle the high dimensionality of the images and employs stochastic optimization to select among a variety of patterns while meeting the acquisition process constraints and budget. So doing, we show how our setting can optimize, not only standard image reconstruction, but also any associated image analysis task. The versatility and performance of our approach are demonstrated on several MRI acquisitions.
• 摘要：在临床环境中最大程度地提高磁共振成像（MRI）的好处的关键挑战是加速采集时间而不显着降低图像质量。该目标需要在采样不足的原始K空间测量速度之间取得平衡，以进行更快的获取，并收集足够的原始信息，以进行高保真图像重建和分析任务。为了达到这种平衡，我们建议使用顺序的贝叶斯实验设计（BED），以提供最有用的测量值的自适应和任务依赖性选择。依次使用选择的新样本来依次增强测量，以最大程度地提高目标图像后验分布的信息增益。选择是通过基于梯度的设计参数的优化来执行的，该设计参数定义了子采样模式。在这项工作中，我们引入了一种新的主​​动床过程，该程序利用基于扩散的生成模型来处理图像的高维度，并采用随机优化来在各种模式中选择，同时满足采集过程约束和预算。因此，我们展示了我们的设置如何优化标准图像重建，还可以优化任何相关的图像分析任务。我们的方法的多功能性和性能在几次MRI获取中得到了证明。

Title: Synthetic ALS-EEG Data Augmentation for ALS Diagnosis Using Conditional WGAN with Weight Clipping

• Authors: Abdulvahap Mutlu, Şengül Doğan, Türker Tuncer
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.16243
• Pdf URL: https://arxiv.org/pdf/2506.16243
• Copy Paste: [[2506.16243]] Synthetic ALS-EEG Data Augmentation for ALS Diagnosis Using Conditional WGAN with Weight Clipping(https://arxiv.org/abs/2506.16243)
• Keywords: generative
• Abstract: Amyotrophic Lateral Sclerosis (ALS) is a rare neurodegenerative disease, and high-quality EEG data from ALS patients are scarce. This data scarcity, coupled with severe class imbalance between ALS and healthy control recordings, poses a challenge for training reliable machine learning classifiers. In this work, we address these issues by generating synthetic EEG signals for ALS patients using a Conditional Wasserstein Generative Adversarial Network (CWGAN). We train CWGAN on a private EEG dataset (ALS vs. non-ALS) to learn the distribution of ALS EEG signals and produce realistic synthetic samples. We preprocess and normalize EEG recordings, and train a CWGAN model to generate synthetic ALS signals. The CWGAN architecture and training routine are detailed, with key hyperparameters chosen for stable training. Qualitative evaluation of generated signals shows that they closely mimic real ALS EEG patterns. The CWGAN training converged with generator and discriminator loss curves stabilizing, indicating successful learning. The synthetic EEG signals appear realistic and have potential use as augmented data for training classifiers, helping to mitigate class imbalance and improve ALS detection accuracy. We discuss how this approach can facilitate data sharing and enhance diagnostic models.
• 摘要：肌萎缩性外侧硬化症（ALS）是一种罕见的神经退行性疾病，ALS患者的高质量脑电图数据很少。这些数据稀缺性，再加上ALS和健康控制记录之间的严重阶级​​失衡，对培训可靠的机器学习分类器构成了挑战。在这项工作中，我们通过使用有条件的Wasserstein生成对抗网络（CWGAN）为ALS患者生成合成EEG信号来解决这些问题。我们在私人脑电图数据集（ALS与非ALS）上训练CWGAN，以了解ALS EEG信号的分布并产生现实的合成样本。我们预处理和标准化脑电图记录，并训练CWGAN模型生成合成的ALS信号。详细介绍了CWGAN的建筑和培训程序，并选择了关键的超参数进行稳定培训。对生成的信号的定性评估表明，它们紧密模仿了真实的脑电图模式。 CWGAN培训与发电机和鉴别器损耗曲线融合了稳定，表明成功学习。合成的脑电图信号看起来很现实，并有可能用作训练分类器的增强数据，有助于减轻类失衡并提高ALS检测准确性。我们讨论这种方法如何促进数据共享并增强诊断模型。

Title: R3eVision: A Survey on Robust Rendering, Restoration, and Enhancement for 3D Low-Level Vision

• Authors: Weeyoung Kwon, Jeahun Sung, Minkyu Jeon, Chanho Eom, Jihyong Oh
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16262
• Pdf URL: https://arxiv.org/pdf/2506.16262
• Copy Paste: [[2506.16262]] R3eVision: A Survey on Robust Rendering, Restoration, and Enhancement for 3D Low-Level Vision(https://arxiv.org/abs/2506.16262)
• Keywords: restoration, super-resolution, generation
• Abstract: Neural rendering methods such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have achieved significant progress in photorealistic 3D scene reconstruction and novel view synthesis. However, most existing models assume clean and high-resolution (HR) multi-view inputs, which limits their robustness under real-world degradations such as noise, blur, low-resolution (LR), and weather-induced artifacts. To address these limitations, the emerging field of 3D Low-Level Vision (3D LLV) extends classical 2D Low-Level Vision tasks including super-resolution (SR), deblurring, weather degradation removal, restoration, and enhancement into the 3D spatial domain. This survey, referred to as R\textsuperscript{3}eVision, provides a comprehensive overview of robust rendering, restoration, and enhancement for 3D LLV by formalizing the degradation-aware rendering problem and identifying key challenges related to spatio-temporal consistency and ill-posed optimization. Recent methods that integrate LLV into neural rendering frameworks are categorized to illustrate how they enable high-fidelity 3D reconstruction under adverse conditions. Application domains such as autonomous driving, AR/VR, and robotics are also discussed, where reliable 3D perception from degraded inputs is critical. By reviewing representative methods, datasets, and evaluation protocols, this work positions 3D LLV as a fundamental direction for robust 3D content generation and scene-level reconstruction in real-world environments.
• 摘要：神经渲染方法，例如神经辐射场（NERF）和3D高斯脱落（3DGS），在光逼真的3D场景重建和新型视图合成方面取得了重大进展。但是，大多数现有模型都采用清洁和高分辨率（HR）多视图输入，这限制了它们在现实世界中的稳健性，例如噪声，模糊，低分辨率（LR）和天气诱导的人工制品。为了解决这些局限性，3D低级视觉（3D LLV）的新兴领域扩展了经典的2D低级视力任务，包括超分辨率（SR），脱张，降解，天气退化去除，恢复和增强，并增强到3D空间领域。这项调查被称为R \ TextSuperScript {3} evision，通过形式化了与时空的一致性和未符号优化的时空一致性和未符号的优化相关的降级呈现呈现的渲染问题，从而对3D LLV进行了全面概述3D LLV的强大渲染，恢复和增强。将LLV整合到神经渲染框架中的最新方法被归类为说明它们如何在不利条件下启用高保真3D重建。还讨论了诸如自主驾驶，AR/VR和机器人技术之类的应用领域，在下降的输入中可靠的3D感知至关重要。通过审查代表性方法，数据集和评估协议，此工作将3D LLV定位为在现实环境中强大的3D内容生成和场景级重建的基本方向。

Title: SycnMapV2: Robust and Adaptive Unsupervised Segmentation

• Authors: Heng Zhang, Zikang Wan, Danilo Vasconcellos Vargas
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.16297
• Pdf URL: https://arxiv.org/pdf/2506.16297
• Copy Paste: [[2506.16297]] SycnMapV2: Robust and Adaptive Unsupervised Segmentation(https://arxiv.org/abs/2506.16297)
• Keywords: generation
• Abstract: Human vision excels at segmenting visual cues without the need for explicit training, and it remains remarkably robust even as noise severity increases. In contrast, existing AI algorithms struggle to maintain accuracy under similar conditions. Here, we present SyncMapV2, the first to solve unsupervised segmentation with state-of-the-art robustness. SyncMapV2 exhibits a minimal drop in mIoU, only 0.01%, under digital corruption, compared to a 23.8% drop observed in SOTA this http URL superior performance extends across various types of corruption: noise (7.3% vs. 37.7%), weather (7.5% vs. 33.8%), and blur (7.0% vs. 29.5%). Notably, SyncMapV2 accomplishes this without any robust training, supervision, or loss functions. It is based on a learning paradigm that uses self-organizing dynamical equations combined with concepts from random networks. Moreover,unlike conventional methods that require re-initialization for each new input, SyncMapV2 adapts online, mimicking the continuous adaptability of human vision. Thus, we go beyond the accurate and robust results, and present the first algorithm that can do all the above online, adapting to input rather than re-initializing. In adaptability tests, SyncMapV2 demonstrates near-zero performance degradation, which motivates and fosters a new generation of robust and adaptive intelligence in the near future.
• 摘要：人类的视觉在不需要明确训练的情况下分割视觉线索方面表现出色，即使噪声严重程度增加，它仍然非常强大。相反，现有的AI算法在相似条件下难以保持准确性。在这里，我们提出SyncMAPV2，这是第一个以最新的鲁棒性解决无监督分割的。 SyncMAPV2在数字腐败下仅显示MIOU的下降最小，仅0.01％，而SOTA在SOTA中观察到的HTTTP URL出色的表现越来越多，跨越各种类型的腐败：噪声（7.3％vs. 37.7％），天气（7.5％vs. 33.8％vs. 33.8％），以及Blur（7.0％），以及Blur（7.0％）（7.0％VS. 29.5％）。值得注意的是，SynCMAPV2在没有任何强大的培训，监督或损失功能的情况下实现了这一目标。它基于一种学习范式，该学习范式使用了自组织的动力学方程，并结合了随机网络的概念。此外，与需要重新定位的常规方法不同，SyncMAPV2在线适应，模仿人类视力的持续适应性。因此，我们超越了准确，可靠的结果，并提出了可以在线完成所有上述所有内容的第一个算法，可以适应输入而不是重新定位。在适应性测试中，SyncMAPV2显示了接近零的性能降低，这激励并促进了新一代的新一代强大和适应性智能。

Title: Improved Exploration in GFlownets via Enhanced Epistemic Neural Networks

• Authors: Sajan Muhammad, Salem Lahlou
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.16313
• Pdf URL: https://arxiv.org/pdf/2506.16313
• Copy Paste: [[2506.16313]] Improved Exploration in GFlownets via Enhanced Epistemic Neural Networks(https://arxiv.org/abs/2506.16313)
• Keywords: generation
• Abstract: Efficiently identifying the right trajectories for training remains an open problem in GFlowNets. To address this, it is essential to prioritize exploration in regions of the state space where the reward distribution has not been sufficiently learned. This calls for uncertainty-driven exploration, in other words, the agent should be aware of what it does not know. This attribute can be measured by joint predictions, which are particularly important for combinatorial and sequential decision problems. In this research, we integrate epistemic neural networks (ENN) with the conventional architecture of GFlowNets to enable more efficient joint predictions and better uncertainty quantification, thereby improving exploration and the identification of optimal trajectories. Our proposed algorithm, ENN-GFN-Enhanced, is compared to the baseline method in GFlownets and evaluated in grid environments and structured sequence generation in various settings, demonstrating both its efficacy and efficiency.
• 摘要：有效地识别适当的训练轨迹仍然是Gflownets中的一个空缺问题。为了解决这个问题，必须优先考虑在奖励分布尚未充分学习的国家空间地区的探索。换句话说，这要求进行不确定性驱动的探索，代理应该意识到它不知道的。该属性可以通过联合预测来衡量，这对于组合和顺序决策问题尤为重要。在这项研究中，我们将认知神经网络（ENN）与Gflownets的常规结构相结合，以实现更有效的关节预测和更好的不确定性量化，从而改善了探索和确定最佳轨迹。我们提出的算法（ENN-GFN增强）与Gflownets中的基线方法进行了比较，并在网格环境中进行了评估，并在各种环境中进行了结构化序列的产生，既表明了其功效和效率。

Title: Watermarking Autoregressive Image Generation

• Authors: Nikola Jovanović, Ismail Labiad, Tomáš Souček, Martin Vechev, Pierre Fernandez
• Subjects: cs.LG, cs.AI, cs.CR, cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16349
• Pdf URL: https://arxiv.org/pdf/2506.16349
• Copy Paste: [[2506.16349]] Watermarking Autoregressive Image Generation(https://arxiv.org/abs/2506.16349)
• Keywords: generation, generative
• Abstract: Watermarking the outputs of generative models has emerged as a promising approach for tracking their provenance. Despite significant interest in autoregressive image generation models and their potential for misuse, no prior work has attempted to watermark their outputs at the token level. In this work, we present the first such approach by adapting language model watermarking techniques to this setting. We identify a key challenge: the lack of reverse cycle-consistency (RCC), wherein re-tokenizing generated image tokens significantly alters the token sequence, effectively erasing the watermark. To address this and to make our method robust to common image transformations, neural compression, and removal attacks, we introduce (i) a custom tokenizer-detokenizer finetuning procedure that improves RCC, and (ii) a complementary watermark synchronization layer. As our experiments demonstrate, our approach enables reliable and robust watermark detection with theoretically grounded p-values.
• 摘要：生成模型的产出的水印已成为追踪其出处的有前途的方法。尽管对自回归图像产生模型及其滥用潜力的兴趣很大，但先前的工作仍未尝试在令牌级别上加水印。在这项工作中，我们通过将语言模型水印技术调整到此环境中提出了第一种这种方法。我们确定了一个关键挑战：缺乏反向周期矛盾（RCC），其中重新施加生成的图像令牌会显着改变令牌序列，从而有效地消除了水印。为了解决这个问题，并使我们的方法鲁棒化到常见的图像转换，神经压缩和删除攻击，我们介绍了（i）一种自定义的令牌范围遗传器的命名程序，可改善RCC，以及（ii）互补的水印同步层。正如我们的实验证明的那样，我们的方法可以通过理论上接地的P值可靠，可靠的水印检测。

Title: CLIP-MG: Guiding Semantic Attention with Skeletal Pose Features and RGB Data for Micro-Gesture Recognition on the iMiGUE Dataset

• Authors: Santosh Patapati, Trisanth Srinivasan, Amith Adiraju
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.16385
• Pdf URL: https://arxiv.org/pdf/2506.16385
• Copy Paste: [[2506.16385]] CLIP-MG: Guiding Semantic Attention with Skeletal Pose Features and RGB Data for Micro-Gesture Recognition on the iMiGUE Dataset(https://arxiv.org/abs/2506.16385)
• Keywords: generation
• Abstract: Micro-gesture recognition is a challenging task in affective computing due to the subtle, involuntary nature of the gestures and their low movement amplitude. In this paper, we introduce a Pose-Guided Semantics-Aware CLIP-based architecture, or CLIP for Micro-Gesture recognition (CLIP-MG), a modified CLIP model tailored for micro-gesture classification on the iMiGUE dataset. CLIP-MG integrates human pose (skeleton) information into the CLIP-based recognition pipeline through pose-guided semantic query generation and a gated multi-modal fusion mechanism. The proposed model achieves a Top-1 accuracy of 61.82%. These results demonstrate both the potential of our approach and the remaining difficulty in fully adapting vision-language models like CLIP for micro-gesture recognition.
• 摘要：由于手势的微妙，非自愿的性质及其低运动幅度，微观识别是情感计算中的一项具有挑战性的任务。在本文中，我们介绍了姿势指导的语义 - 意识到的基于剪辑的体系结构，或用于微观识别的剪辑（剪辑MG），这是一种针对Imigue DataSet上微观式分类的修改剪辑模型。剪辑MG通过姿势引导的语义查询产生和封闭的多模式融合机制将人姿势（骨骼）信息整合到基于夹的识别管道中。所提出的模型的前1位准确性为61.82％。这些结果既表明了我们的方法的潜力，又表明了完全适应视觉模型（如剪辑）的微观识别的难度。

Title: Generating Directed Graphs with Dual Attention and Asymmetric Encoding

• Authors: Alba Carballo-Castro, Manuel Madeira, Yiming Qin, Dorina Thanou, Pascal Frossard
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.16404
• Pdf URL: https://arxiv.org/pdf/2506.16404
• Copy Paste: [[2506.16404]] Generating Directed Graphs with Dual Attention and Asymmetric Encoding(https://arxiv.org/abs/2506.16404)
• Keywords: generation, generative
• Abstract: Directed graphs naturally model systems with asymmetric, ordered relationships, essential to applications in biology, transportation, social networks, and visual understanding. Generating such graphs enables tasks such as simulation, data augmentation and novel instance discovery; however, directed graph generation remains underexplored. We identify two key factors limiting progress in this direction: first, modeling edge directionality introduces a substantially larger dependency space, making the underlying distribution harder to learn; second, the absence of standardized benchmarks hinders rigorous evaluation. Addressing the former requires more expressive models that are sensitive to directional topologies. We propose Directo, the first generative model for directed graphs built upon the discrete flow matching framework. Our approach combines: (i) principled positional encodings tailored to asymmetric pairwise relations, (ii) a dual-attention mechanism capturing both incoming and outgoing dependencies, and (iii) a robust, discrete generative framework. To support evaluation, we introduce a benchmark suite covering synthetic and real-world datasets. It shows that our method performs strongly across diverse settings and even competes with specialized models for particular classes, such as directed acyclic graphs. Our results highlight the effectiveness and generality of our approach, establishing a solid foundation for future research in directed graph generation.
• 摘要：定向图自然建模具有不对称，有序关系的系统，对生物学，运输，社交网络和视觉理解的应用至关重要。生成此类图可以实现诸如模拟，数据增强和新颖实例发现之类的任务；但是，定向图生成仍然没有被逐渐倍增。我们确定了限制在这个方向上进展的两个关键因素：首先，建模边缘方向性引入了更大的依赖性空间，从而使基础分布更难学习；其次，缺乏标准化的基准测试会阻碍严格的评估。解决前者需要更敏感的方向拓扑模型。我们提出了Directo，这是基于离散流匹配框架构建的有向图的第一个生成模型。我们的方法结合了：（i）针对非对称成对关系量身定制的原则位置编码，（ii）一种捕获传入和外向依赖性的双重注意机制，以及（iii）强大的离散生成框架。为了支持评估，我们引入了一个涵盖合成和现实世界数据集的基准套件。它表明，我们的方法在各种环境中的性能强大，甚至与特定类别（例如定向的无环图）的专门模型竞争。我们的结果突出了我们方法的有效性和一般性，为有名图生成的未来研究奠定了坚实的基础。

Title: Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights

• Authors: Zhiyuan Liang, Dongwen Tang, Yuhao Zhou, Xuanlei Zhao, Mingjia Shi, Wangbo Zhao, Zekai Li, Peihao Wang, Konstantin Schürholt, Damian Borth, Michael M. Bronstein, Yang You, Zhangyang Wang, Kai Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.16406
• Pdf URL: https://arxiv.org/pdf/2506.16406
• Copy Paste: [[2506.16406]] Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights(https://arxiv.org/abs/2506.16406)
• Keywords: generation
• Abstract: Modern Parameter-Efficient Fine-Tuning (PEFT) methods such as low-rank adaptation (LoRA) reduce the cost of customizing large language models (LLMs), yet still require a separate optimization run for every downstream dataset. We introduce \textbf{Drag-and-Drop LLMs (\textit{DnD})}, a prompt-conditioned parameter generator that eliminates per-task training by mapping a handful of unlabeled task prompts directly to LoRA weight updates. A lightweight text encoder distills each prompt batch into condition embeddings, which are then transformed by a cascaded hyper-convolutional decoder into the full set of LoRA matrices. Once trained in a diverse collection of prompt-checkpoint pairs, DnD produces task-specific parameters in seconds, yielding i) up to \textbf{12,000$\times$} lower overhead than full fine-tuning, ii) average gains up to \textbf{30\%} in performance over the strongest training LoRAs on unseen common-sense reasoning, math, coding, and multimodal benchmarks, and iii) robust cross-domain generalization despite never seeing the target data or labels. Our results demonstrate that prompt-conditioned parameter generation is a viable alternative to gradient-based adaptation for rapidly specializing LLMs. Our project is available at \href{this https URL}{this https URL}.
• 摘要：现代参数有效的微调（PEFT）方法（例如低级别适应（LORA））降低了自定义大语模型（LLMS）的成本，但仍需要为每个下游数据集进行单独的优化运行。我们介绍了\ textbf {drag-drop llms（\ textit {dnd}）}，这是一种及时条件的参数生成器，通过映射少数未标记的任务提示来消除每个任务训练，直接直接将其提示到lora stast to Lora权重更新。轻巧的文本编码器将每个提示批量蒸馏成条件嵌入，然后由级联的超斜率解码器转换为完整的Lora矩阵。 Once trained in a diverse collection of prompt-checkpoint pairs, DnD produces task-specific parameters in seconds, yielding i) up to \textbf{12,000$\times$} lower overhead than full fine-tuning, ii) average gains up to \textbf{30\%} in performance over the strongest training LoRAs on unseen common-sense reasoning, math, coding, and multimodal基准和iii）尽管从未看到目标数据或标签，但强大的跨域泛化。我们的结果表明，及时条件的参数生成是快速专业化LLM的基于梯度适应的可行替代品。我们的项目可在\ href {this https url} {此https url}上获得。

Title: Progressive Inference-Time Annealing of Diffusion Models for Sampling from Boltzmann Densities

• Authors: Tara Akhound-Sadegh, Jungyoon Lee, Avishek Joey Bose, Valentin De Bortoli, Arnaud Doucet, Michael M. Bronstein, Dominique Beaini, Siamak Ravanbakhsh, Kirill Neklyudov, Alexander Tong
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.16471
• Pdf URL: https://arxiv.org/pdf/2506.16471
• Copy Paste: [[2506.16471]] Progressive Inference-Time Annealing of Diffusion Models for Sampling from Boltzmann Densities(https://arxiv.org/abs/2506.16471)
• Keywords: generative
• Abstract: Sampling efficiently from a target unnormalized probability density remains a core challenge, with relevance across countless high-impact scientific applications. A promising approach towards this challenge is the design of amortized samplers that borrow key ideas, such as probability path design, from state-of-the-art generative diffusion models. However, all existing diffusion-based samplers remain unable to draw samples from distributions at the scale of even simple molecular systems. In this paper, we propose Progressive Inference-Time Annealing (PITA), a novel framework to learn diffusion-based samplers that combines two complementary interpolation techniques: I.) Annealing of the Boltzmann distribution and II.) Diffusion smoothing. PITA trains a sequence of diffusion models from high to low temperatures by sequentially training each model at progressively higher temperatures, leveraging engineered easy access to samples of the temperature-annealed target density. In the subsequent step, PITA enables simulating the trained diffusion model to procure training samples at a lower temperature for the next diffusion model through inference-time annealing using a novel Feynman-Kac PDE combined with Sequential Monte Carlo. Empirically, PITA enables, for the first time, equilibrium sampling of N-body particle systems, Alanine Dipeptide, and tripeptides in Cartesian coordinates with dramatically lower energy function evaluations. Code available at: this https URL
• 摘要：从目标未归一化的概率密度进行有效采样仍然是一个核心挑战，并且在无数的高影响力科学应用中相关。面对这一挑战的一种有希望的方法是借用最先进的生成扩散模型的摊销采样器的设计，例如借用关键思想，例如概率路径设计。但是，所有现有的基于扩散的采样器仍无法从简单分子系统的规模上从分布中绘制样品。在本文中，我们提出了进行性推理时间退火（PITA），这是一种学习基于扩散的采样器的新型框架，结合了两种互补的插值技术：I。）玻尔兹曼分布和II的退火。）扩散平滑。 PITA通过在逐渐更高的温度下依次训练每个模型，从而训练从高温到低温的扩散模型，从而利用工程设计的轻松访问温度解散的目标密度的样品。在随后的步骤中，PITA使模拟受过训练的扩散模型可以通过推理时间退火在较低温度下在较低的温度下采购训练样品，并使用新型的Feynman-KAC PDE与顺序的蒙特卡洛结合使用。从经验上讲，PITA首次使N体粒子系统，丙氨酸二肽和三肽的平衡采样具有较低的能量功能评估的笛卡尔坐标。代码可用：此HTTPS URL

Title: SparseLoRA: Accelerating LLM Fine-Tuning with Contextual Sparsity

• Authors: Samir Khaki, Xiuyu Li, Junxian Guo, Ligeng Zhu, Chenfeng Xu, Konstantinos N. Plataniotis, Amir Yazdanbakhsh, Kurt Keutzer, Song Han, Zhijian Liu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.16500
• Pdf URL: https://arxiv.org/pdf/2506.16500
• Copy Paste: [[2506.16500]] SparseLoRA: Accelerating LLM Fine-Tuning with Contextual Sparsity(https://arxiv.org/abs/2506.16500)
• Keywords: generation
• Abstract: Fine-tuning LLMs is both computationally and memory-intensive. While parameter-efficient fine-tuning methods, such as QLoRA and DoRA, reduce the number of trainable parameters and lower memory usage, they do not decrease computational cost. In some cases, they may even slow down fine-tuning. In this paper, we introduce SparseLoRA, a method that accelerates LLM fine-tuning through contextual sparsity. We propose a lightweight, training-free SVD sparsity estimator that dynamically selects a sparse subset of weights for loss and gradient computation. Also, we systematically analyze and address sensitivity across layers, tokens, and training steps. Our experimental results show that SparseLoRA reduces computational cost by up to 2.2 times and a measured speedup of up to 1.6 times while maintaining accuracy across various downstream tasks, including commonsense and arithmetic reasoning, code generation, and instruction following.
• 摘要：微调LLM既是计算和内存密集的。虽然参数有效的微调方法（例如Qlora和dora）减少了可训练的参数的数量和较低的内存使用情况，但它们不会降低计算成本。在某些情况下，它们甚至可能会减慢微调。在本文中，我们介绍了Sparselora，这种方法通过上下文稀疏来加速LLM微调。我们提出了一个轻巧的，无训练的SVD稀疏估计器，该估计值动态选择稀疏的重量子集以进行损失和梯度计算。此外，我们系统地分析和解决跨层，代币和培训步骤的灵敏度。我们的实验结果表明，Sparselora最多将计算成本降低2.2次，并且测量的速度最高为1.6倍，同时保持各种下游任务的准确性，包括常识性和算术推理，代码生成和以下说明。

Title: Hunyuan3D 2.5: Towards High-Fidelity 3D Assets Generation with Ultimate Details

• Authors: Zeqiang Lai, Yunfei Zhao, Haolin Liu, Zibo Zhao, Qingxiang Lin, Huiwen Shi, Xianghui Yang, Mingxin Yang, Shuhui Yang, Yifei Feng, Sheng Zhang, Xin Huang, Di Luo, Fan Yang, Fang Yang, Lifu Wang, Sicong Liu, Yixuan Tang, Yulin Cai, Zebin He, Tian Liu, Yuhong Liu, Jie Jiang, Linus, Jingwei Huang, Chunchao Guo
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.16504
• Pdf URL: https://arxiv.org/pdf/2506.16504
• Copy Paste: [[2506.16504]] Hunyuan3D 2.5: Towards High-Fidelity 3D Assets Generation with Ultimate Details(https://arxiv.org/abs/2506.16504)
• Keywords: generation
• Abstract: In this report, we present Hunyuan3D 2.5, a robust suite of 3D diffusion models aimed at generating high-fidelity and detailed textured 3D assets. Hunyuan3D 2.5 follows two-stages pipeline of its previous version Hunyuan3D 2.0, while demonstrating substantial advancements in both shape and texture generation. In terms of shape generation, we introduce a new shape foundation model -- LATTICE, which is trained with scaled high-quality datasets, model-size, and compute. Our largest model reaches 10B parameters and generates sharp and detailed 3D shape with precise image-3D following while keeping mesh surface clean and smooth, significantly closing the gap between generated and handcrafted 3D shapes. In terms of texture generation, it is upgraded with phyiscal-based rendering (PBR) via a novel multi-view architecture extended from Hunyuan3D 2.0 Paint model. Our extensive evaluation shows that Hunyuan3D 2.5 significantly outperforms previous methods in both shape and end-to-end texture generation.
• 摘要：在本报告中，我们提出了Hunyuan3d 2.5，这是一个旨在产生高保真性和详细质感3D资产的3D扩散模型的稳健套件。 Hunyuan3d 2.5遵循其先前版本Hunyuan3d 2.0的两阶段管道，同时展示了形状和纹理产生的重大进步。在形状生成方面，我们引入了一种新的形状基础模型-Lattice，该模型通过缩放的高质量数据集，模型尺寸和计算训练。我们最大的型号达到10b参数，并以精确的Image-3D生成尖锐而详细的3D形状，同时保持网状表面清洁和光滑，并显着缩小生成的和手工制作的3D形状之间的间隙。就纹理产生而言，它通过基于蛋白质的渲染（PBR）升级，这是一种从Hunyuan3d 2.0涂料模型扩展的新型多视图架构。我们的广泛评估表明，Hunyuan3d 2.5在形状和端到端纹理生成中都显着优于先前的方法。

Title: Mr. Snuffleupagus at SemEval-2025 Task 4: Unlearning Factual Knowledge from LLMs Using Adaptive RMU

• Authors: Arjun Dosajh, Mihika Sanghi
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.16548
• Pdf URL: https://arxiv.org/pdf/2506.16548
• Copy Paste: [[2506.16548]] Mr. Snuffleupagus at SemEval-2025 Task 4: Unlearning Factual Knowledge from LLMs Using Adaptive RMU(https://arxiv.org/abs/2506.16548)
• Keywords: generation
• Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation. However, their tendency to memorize training data raises concerns regarding privacy, copyright compliance, and security, particularly in cases involving Personally Identifiable Information (PII). Effective machine unlearning techniques are essential to mitigate these risks, yet existing methods remain underdeveloped for LLMs due to their open-ended output space. In this work, we apply the Adaptive Representation Misdirection Unlearning (RMU) technique to unlearn sensitive information from LLMs. Through extensive experiments, we analyze the effects of unlearning across different decoder layers to determine the most effective regions for sensitive information removal. Our technique ranked 4th on the official leaderboard of both 1B parameter and 7B parameter models.
• 摘要：大型语言模型（LLM）在自然语言理解和产生中表现出了显着的能力。但是，他们记住培训数据的趋势引起了人们对隐私，版权合规性和安全性的担忧，尤其是在涉及个人身份信息（PII）的情况下。有效的机器学习技术对于减轻这些风险至关重要，但由于其开放式输出空间，现有方法仍未发育。在这项工作中，我们将自适应表示误导（RMU）技术应用于LLM的敏感信息。通过广泛的实验，我们分析了跨不同解码器层学习的影响，以确定去除敏感信息的最有效区域。我们的技术在1B参数和7B参数模型的官方排行榜上排名第四。

Title: SafeTriage: Facial Video De-identification for Privacy-Preserving Stroke Triage

• Authors: Tongan Cai, Haomiao Ni, Wenchao Ma, Yuan Xue, Qian Ma, Rachel Leicht, Kelvin Wong, John Volpi, Stephen T.C. Wong, James Z. Wang, Sharon X. Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16578
• Pdf URL: https://arxiv.org/pdf/2506.16578
• Copy Paste: [[2506.16578]] SafeTriage: Facial Video De-identification for Privacy-Preserving Stroke Triage(https://arxiv.org/abs/2506.16578)
• Keywords: generative
• Abstract: Effective stroke triage in emergency settings often relies on clinicians' ability to identify subtle abnormalities in facial muscle coordination. While recent AI models have shown promise in detecting such patterns from patient facial videos, their reliance on real patient data raises significant ethical and privacy challenges -- especially when training robust and generalizable models across institutions. To address these concerns, we propose SafeTriage, a novel method designed to de-identify patient facial videos while preserving essential motion cues crucial for stroke diagnosis. SafeTriage leverages a pretrained video motion transfer (VMT) model to map the motion characteristics of real patient faces onto synthetic identities. This approach retains diagnostically relevant facial dynamics without revealing the patients' identities. To mitigate the distribution shift between normal population pre-training videos and patient population test videos, we introduce a conditional generative model for visual prompt tuning, which adapts the input space of the VMT model to ensure accurate motion transfer without needing to fine-tune the VMT model backbone. Comprehensive evaluation, including quantitative metrics and clinical expert assessments, demonstrates that SafeTriage-produced synthetic videos effectively preserve stroke-relevant facial patterns, enabling reliable AI-based triage. Our evaluations also show that SafeTriage provides robust privacy protection while maintaining diagnostic accuracy, offering a secure and ethically sound foundation for data sharing and AI-driven clinical analysis in neurological disorders.
• 摘要：在紧急情况下，有效的中风分类通常取决于临床医生鉴定面部肌肉协调中微妙异常的能力。尽管最近的AI模型在检测患者面部视频的这种模式方面已显示出希望，但他们对实际患者数据的依赖会引起重大的道德和隐私挑战，尤其是当培训机构培训强大且可推广的模型时。为了解决这些问题，我们提出了Safetrage，这是一种新型方法，旨在识别患者的面部视频，同时保留对中风诊断至关重要的基本运动提示。 Safetrage利用验证的视频运动转移（VMT）模型将真实患者面部的运动特征映射到合成身份。这种方法保留了诊断相关的面部动力，而没有揭示患者的身份。为了减轻正常人群预训练视频和患者人群测试视频之间的分布变化，我们引入了一种有条件的生成模型，用于视觉及时调整，该模型适应了VMT模型的输入空间，以确保准确的运动转移，而无需微调VMT模型骨架。全面的评估，包括定量指标和临床专家评估，表明，安全制作的合成视频有效地保留了与中风相关的面部模式，从而实现了可靠的基于AI的分类。我们的评估还表明，Safetriage在保持诊断准确性的同时提供了强大的隐私保护，为数据共享和AI驱动的神经系统疾病中的临床分析提供了安全和道德上合理的基础。

Title: MetaQAP -- A Meta-Learning Approach for Quality-Aware Pretraining in Image Quality Assessment

• Authors: Muhammad Azeem Aslam, Muhammad Hamza, Nisar Ahmed, Gulshan Saleem, Zhu Shuangtong, Hu Hongfei, Xu Wei, Saba Aslam, Wang Jun
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2506.16601
• Pdf URL: https://arxiv.org/pdf/2506.16601
• Copy Paste: [[2506.16601]] MetaQAP -- A Meta-Learning Approach for Quality-Aware Pretraining in Image Quality Assessment(https://arxiv.org/abs/2506.16601)
• Keywords: quality assessment
• Abstract: Image Quality Assessment (IQA) is a critical task in a wide range of applications but remains challenging due to the subjective nature of human perception and the complexity of real-world image distortions. This study proposes MetaQAP, a novel no-reference IQA model designed to address these challenges by leveraging quality-aware pre-training and meta-learning. The model performs three key contributions: pre-training Convolutional Neural Networks (CNNs) on a quality-aware dataset, implementing a quality-aware loss function to optimize predictions, and integrating a meta-learner to form an ensemble model that effectively combines predictions from multiple base models. Experimental evaluations were conducted on three benchmark datasets: LiveCD, KonIQ-10K, and BIQ2021. The proposed MetaQAP model achieved exceptional performance with Pearson Linear Correlation Coefficient (PLCC) and Spearman Rank Order Correlation Coefficient (SROCC) scores of 0.9885/0.9812 on LiveCD, 0.9702/0.9658 on KonIQ-10K, and 0.884/0.8765 on BIQ2021, outperforming existing IQA methods. Cross-dataset evaluations further demonstrated the generalizability of the model, with PLCC and SROCC scores ranging from 0.6721 to 0.8023 and 0.6515 to 0.7805, respectively, across diverse datasets. The ablation study confirmed the significance of each model component, revealing substantial performance degradation when critical elements such as the meta-learner or quality-aware loss function were omitted. MetaQAP not only addresses the complexities of authentic distortions but also establishes a robust and generalizable framework for practical IQA applications. By advancing the state-of-the-art in no-reference IQA, this research provides valuable insights and methodologies for future improvements and extensions in the field.
• 摘要：图像质量评估（IQA）在广泛的应用中是一项关键任务，但由于人类感知的主观性质和现实世界图像扭曲的复杂性，因此仍然具有挑战性。这项研究提出了一种新型的No-No-Reference IQA模型Metaqap，旨在通过利用质量吸引的预培训和元学习来解决这些挑战。该模型执行三个关键贡献：训练前卷积神经网络（CNN）在质量感知的数据集中，实施质量感知的损失函数以优化预测，并集成元学习者以形成合奏模型，以有效地结合了来自多个基本模型的预测。实验评估是在三个基准数据集上进行的：LiveCD，KONIQ-10K和BIQ2021。提出的元数据模型在liveCD上以0.9885/0.9812为0.9702/0.9658在KONIQ-10K上和0.884/0.884/0.87765上使用的0.9702/0.9658在LIVECD上达到0.9885/0.9812，以0.9702/0.9658在LIVECD上获得了0.9885/0.9812的出色表现。跨数据库评估进一步证明了该模型的普遍性，PLCC和SROCC分别分别为0.6721至0.8023和0.6515至0.7805。消融研究证实了每个模型成分的重要性，揭示了当省略元素或质量感知损失函数等关键要素时，效果降低了。 Metaqap不仅解决了真实扭曲的复杂性，而且还为实用的IQA应用程序建立了一个可靠且可推广的框架。通过推进No-Reference IQA的最新技术，这项研究为未来的改进和扩展提供了宝贵的见解和方法。

Title: Relational Deep Learning: Challenges, Foundations and Next-Generation Architectures

• Authors: Vijay Prakash Dwivedi, Charilaos Kanatsoulis, Shenyang Huang, Jure Leskovec
• Subjects: cs.LG, cs.AI, cs.DB
• Abstract URL: https://arxiv.org/abs/2506.16654
• Pdf URL: https://arxiv.org/pdf/2506.16654
• Copy Paste: [[2506.16654]] Relational Deep Learning: Challenges, Foundations and Next-Generation Architectures(https://arxiv.org/abs/2506.16654)
• Keywords: generation
• Abstract: Graph machine learning has led to a significant increase in the capabilities of models that learn on arbitrary graph-structured data and has been applied to molecules, social networks, recommendation systems, and transportation, among other domains. Data in multi-tabular relational databases can also be constructed as 'relational entity graphs' for Relational Deep Learning (RDL) - a new blueprint that enables end-to-end representation learning without traditional feature engineering. Compared to arbitrary graph-structured data, relational entity graphs have key properties: (i) their structure is defined by primary-foreign key relationships between entities in different tables, (ii) the structural connectivity is a function of the relational schema defining a database, and (iii) the graph connectivity is temporal and heterogeneous in nature. In this paper, we provide a comprehensive review of RDL by first introducing the representation of relational databases as relational entity graphs, and then reviewing public benchmark datasets that have been used to develop and evaluate recent GNN-based RDL models. We discuss key challenges including large-scale multi-table integration and the complexities of modeling temporal dynamics and heterogeneous data, while also surveying foundational neural network methods and recent architectural advances specialized for relational entity graphs. Finally, we explore opportunities to unify these distinct modeling challenges, highlighting how RDL converges multiple sub-fields in graph machine learning towards the design of foundation models that can transform the processing of relational data.
• 摘要：Graph Machine Learning导致模型的功能显着提高，这些模型的功能在任意图形结构数据上学习，并已应用于分子，社交网络，推荐系统和运输等分子。多型关系数据库中的数据也可以构造为关系深度学习的“关系实体图”（RDL） - 一种新的蓝图，可以在没有传统功能工程的情况下进行端到端表示学习。与任意的图形结构数据相比，关系实体图具有关键属性：（i）它们的结构由不同表中实体之间的主要外国密钥关系定义，（ii）结构连接性是定义数据库的关系架构的函数，（iii）图形连接性是时间和异性。在本文中，我们首先引入关系数据库作为关系实体图表，然后审查已用于开发和评估基于GNN的RDL模型的公共基准数据集，从而对RDL进行了全面的评论。我们讨论了关键挑战，包括大规模的多桌集成以及建模时间动力学和异质数据的复杂性，同时还测量了基本神经网络方法和最新的建筑进步，专门用于关系实体图。最后，我们探索了统一这些独特的建模挑战的机会，并强调了RDL如何将图形机学习中的多个子场收敛到可以改变关系数据处理的基础模型的设计。

Title: Mesh-Informed Neural Operator : A Transformer Generative Approach

• Authors: Yaozhong Shi, Zachary E. Ross, Domniki Asimaki, Kamyar Azizzadenesheli
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.16656
• Pdf URL: https://arxiv.org/pdf/2506.16656
• Copy Paste: [[2506.16656]] Mesh-Informed Neural Operator : A Transformer Generative Approach(https://arxiv.org/abs/2506.16656)
• Keywords: generative
• Abstract: Generative models in function spaces, situated at the intersection of generative modeling and operator learning, are attracting increasing attention due to their immense potential in diverse scientific and engineering applications. While functional generative models are theoretically domain- and discretization-agnostic, current implementations heavily rely on the Fourier Neural Operator (FNO), limiting their applicability to regular grids and rectangular domains. To overcome these critical limitations, we introduce the Mesh-Informed Neural Operator (MINO). By leveraging graph neural operators and cross-attention mechanisms, MINO offers a principled, domain- and discretization-agnostic backbone for generative modeling in function spaces. This advancement significantly expands the scope of such models to more diverse applications in generative, inverse, and regression tasks. Furthermore, MINO provides a unified perspective on integrating neural operators with general advanced deep learning architectures. Finally, we introduce a suite of standardized evaluation metrics that enable objective comparison of functional generative models, addressing another critical gap in the field.
• 摘要：位于生成建模和操作员学习的交集的功能空间中的生成模型，由于其在多种科学和工程应用中的巨大潜力，引起了人们的关注。虽然功能生成模型在理论上是域和离散化的敏捷性，但当前的实现严重依赖于傅立叶神经操作员（FNO），从而将其适用性限制在常规网格和矩形域。为了克服这些临界局限性，我们介绍了网状信息的神经操作员（MINO）。通过利用图形神经操作员和交叉注意机制，Mino提供了一种原则上的，域和离散的敏捷骨架，用于在功能空间中的生成建模。这一进步将此类模型的范围大大扩展到生成，反向和回归任务中更多样化的应用程序。此外，Mino提供了将神经操作员与一般高级深度学习体系结构相结合的统一观点。最后，我们介绍了一系列标准化评估指标，可以对功能生成模型进行客观比较，并解决该领域的另一个关键差距。

Title: Private Training & Data Generation by Clustering Embeddings

• Authors: Felix Zhou, Samson Zhou, Vahab Mirrokni, Alessandro Epasto, Vincent Cohen-Addad
• Subjects: cs.LG, cs.CR, stat.ML
• Abstract URL: https://arxiv.org/abs/2506.16661
• Pdf URL: https://arxiv.org/pdf/2506.16661
• Copy Paste: [[2506.16661]] Private Training & Data Generation by Clustering Embeddings(https://arxiv.org/abs/2506.16661)
• Keywords: generation
• Abstract: Deep neural networks often use large, high-quality datasets to achieve high performance on many machine learning tasks. When training involves potentially sensitive data, this process can raise privacy concerns, as large models have been shown to unintentionally memorize and reveal sensitive information, including reconstructing entire training samples. Differential privacy (DP) provides a robust framework for protecting individual data and in particular, a new approach to privately training deep neural networks is to approximate the input dataset with a privately generated synthetic dataset, before any subsequent training algorithm. We introduce a novel principled method for DP synthetic image embedding generation, based on fitting a Gaussian Mixture Model (GMM) in an appropriate embedding space using DP clustering. Our method provably learns a GMM under separation conditions. Empirically, a simple two-layer neural network trained on synthetically generated embeddings achieves state-of-the-art (SOTA) classification accuracy on standard benchmark datasets. Additionally, we demonstrate that our method can generate realistic synthetic images that achieve downstream classification accuracy comparable to SOTA methods. Our method is quite general, as the encoder and decoder modules can be freely substituted to suit different tasks. It is also highly scalable, consisting only of subroutines that scale linearly with the number of samples and/or can be implemented efficiently in distributed systems.
• 摘要：深度神经网络通常使用大型高质量数据集来实现许多机器学习任务的高性能。当培训涉及潜在的敏感数据时，此过程可能会引起隐私问题，因为大型模型被证明是无意中记住并揭示敏感信息的，包括重建整个培训样本。差异隐私（DP）提供了一个可靠的框架来保护单个数据，尤其是，在任何后续培训算法之前，使用私人生成的合成数据集私有培训深度神经网络的新方法是用私人生成的合成数据集近似输入数据集。我们基于使用DP群集在适当的嵌入空间中拟合的高斯混合模型（GMM），引入了一种新颖的DP合成图像嵌入生成的原则性方法。事实证明，我们的方法在分离条件下学习了GMM。从经验上讲，一个简单的两层神经网络在合成生成的嵌入式上训练了标准基准数据集上的最先进（SOTA）分类精度。此外，我们证明我们的方法可以生成现实的合成图像，以实现与SOTA方法相当的下游分类精度。我们的方法非常通用，因为可以免费替换编码器和解码器模块以适合不同的任务。它也是高度可扩展的，仅包括与样品数量线性缩放和/或可以在分布式系统中有效实现的子例程。

Title: How to Train your Text-to-Image Model: Evaluating Design Choices for Synthetic Training Captions

• Authors: Manuel Brack, Sudeep Katakol, Felix Friedrich, Patrick Schramowski, Hareesh Ravi, Kristian Kersting, Ajinkya Kale
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.16679
• Pdf URL: https://arxiv.org/pdf/2506.16679
• Copy Paste: [[2506.16679]] How to Train your Text-to-Image Model: Evaluating Design Choices for Synthetic Training Captions(https://arxiv.org/abs/2506.16679)
• Keywords: generation
• Abstract: Training data is at the core of any successful text-to-image models. The quality and descriptiveness of image text are crucial to a model's performance. Given the noisiness and inconsistency in web-scraped datasets, recent works shifted towards synthetic training captions. While this setup is generally believed to produce more capable models, current literature does not provide any insights into its design choices. This study closes this gap by systematically investigating how different synthetic captioning strategies impact the downstream performance of text-to-image models. Our experiments demonstrate that dense, high-quality captions enhance text alignment but may introduce trade-offs in output aesthetics and diversity. Conversely, captions of randomized lengths yield balanced improvements across aesthetics and alignment without compromising sample diversity. We also demonstrate that varying caption distributions introduce significant shifts in the output bias of a trained model. Our findings underscore the importance of caption design in achieving optimal model performance and provide practical insights for more effective training data strategies in text-to-image generation.
• 摘要：培训数据是任何成功的文本模型模型的核心。图像文本的质量和描述性对于模型的性能至关重要。鉴于网络绑带数据集的噪声和不一致，最近的作品转向了合成培训字幕。尽管通常认为这种设置会产生更有能力的模型，但当前的文献并不能对其设计选择提供任何见解。这项研究通过系统地研究不同的合成字幕策略如何影响文本模型的下游性能，从而缩小了这一差距。我们的实验表明，密集，高质量的字幕增强了文本对齐，但可能会引入产出美学和多样性方面的权衡。相反，随机长度的标题在不影响样本多样性的情况下会产生跨美学和对齐方式的平衡改进。我们还证明，不同的标题分布引入了训练有素的模型的输出偏置的显着转移。我们的发现强调了标题设计在实现最佳模型性能中的重要性，并为文本到图像生成的更有效的培训数据策略提供了实用的见解。

Title: Fast and Stable Diffusion Planning through Variational Adaptive Weighting

• Authors: Zhiying Qiu, Tao Lin
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.16688
• Pdf URL: https://arxiv.org/pdf/2506.16688
• Copy Paste: [[2506.16688]] Fast and Stable Diffusion Planning through Variational Adaptive Weighting(https://arxiv.org/abs/2506.16688)
• Keywords: generative
• Abstract: Diffusion models have recently shown promise in offline RL. However, these methods often suffer from high training costs and slow convergence, particularly when using transformer-based denoising backbones. While several optimization strategies have been proposed -- such as modified noise schedules, auxiliary prediction targets, and adaptive loss weighting -- challenges remain in achieving stable and efficient training. In particular, existing loss weighting functions typically rely on neural network approximators, which can be ineffective in early training phases due to limited generalization capacity of MLPs when exposed to sparse feedback in the early training stages. In this work, we derive a variationally optimal uncertainty-aware weighting function and introduce a closed-form polynomial approximation method for its online estimation under the flow-based generative modeling framework. We integrate our method into a diffusion planning pipeline and evaluate it on standard offline RL benchmarks. Experimental results on Maze2D and Kitchen tasks show that our method achieves competitive performance with up to 10 times fewer training steps, highlighting its practical effectiveness.
• 摘要：扩散模型最近在离线RL中显示了有望。但是，这些方法通常会遭受高训练成本和缓慢的收敛性，尤其是在使用基于变压器的deno骨架时。尽管已经提出了几种优化策略，例如修改后的噪声时间表，辅助预测目标以及自适应减肥体重 - 在实现稳定且高效的训练方面仍然存在挑战。特别是，现有的减肥功能通常依赖于神经网络近似值，由于MLP在早期训练阶段暴露于稀疏反馈时，由于MLP的概括能力有限，因此在早期训练阶段可能无效。在这项工作中，我们得出了一种变异最佳的不确定性感知加权函数，并在基于流量的生成建模框架下引入了其在线估计的封闭形式多项式近似方法。我们将方法集成到扩散计划管道中，并根据标准离线RL基准进行评估。 Maze2D和厨房任务的实验结果表明，我们的方法以较少的训练步骤达到了10倍的竞争性能，突出了其实际有效性。

Title: Language-driven Description Generation and Common Sense Reasoning for Video Action Recognition

• Authors: Xiaodan Hu, Chuhang Zou, Suchen Wang, Jaechul Kim, Narendra Ahuja
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16701
• Pdf URL: https://arxiv.org/pdf/2506.16701
• Copy Paste: [[2506.16701]] Language-driven Description Generation and Common Sense Reasoning for Video Action Recognition(https://arxiv.org/abs/2506.16701)
• Keywords: generation
• Abstract: Recent video action recognition methods have shown excellent performance by adapting large-scale pre-trained language-image models to the video domain. However, language models contain rich common sense priors - the scene contexts that humans use to constitute an understanding of objects, human-object interactions, and activities - that have not been fully exploited. In this paper, we introduce a framework incorporating language-driven common sense priors to identify cluttered video action sequences from monocular views that are often heavily occluded. We propose: (1) A video context summary component that generates candidate objects, activities, and the interactions between objects and activities; (2) A description generation module that describes the current scene given the context and infers subsequent activities, through auxiliary prompts and common sense reasoning; (3) A multi-modal activity recognition head that combines visual and textual cues to recognize video actions. We demonstrate the effectiveness of our approach on the challenging Action Genome and Charades datasets.
• 摘要：最近的视频动作识别方法通过将大规模训练的语言图像模型调整到视频域，表现出了出色的性能。但是，语言模型包含丰富的常识先验 - 人类用来构成对物体，人类对象相互作用和活动的场景上下文 - 尚未得到充分利用。在本文中，我们介绍了一个结合语言驱动的常识先验的框架，以从通常被严重遮挡的单眼视图中识别出混乱的视频动作序列。我们建议：（1）视频上下文摘要组件，该组件生成候选对象，活动以及对象与活动之间的交互； （2）描述通过辅助提示和常识推理描述当前场景的生成模块，该模块描述了当前场景并进化了随后的活动； （3）组合视觉和文本提示以识别视频动作的多模式活动识别头。我们证明了方法对具有挑战性的动作基因组和CHARADES数据集的有效性。

Title: Uncertainty-Aware Variational Information Pursuit for Interpretable Medical Image Analysis

• Authors: Md Nahiduzzaman, Ruwan Tennakoon, Steven Korevaar, Zongyuan Ge, Alireza Bab-Hadiashar
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16742
• Pdf URL: https://arxiv.org/pdf/2506.16742
• Copy Paste: [[2506.16742]] Uncertainty-Aware Variational Information Pursuit for Interpretable Medical Image Analysis(https://arxiv.org/abs/2506.16742)
• Keywords: generation
• Abstract: In medical imaging, AI decision-support systems must balance accuracy and interpretability to build user trust and support effective clinical decision-making. Recently, Variational Information Pursuit (V-IP) and its variants have emerged as interpretable-by-design modeling techniques, aiming to explain AI decisions in terms of human-understandable, clinically relevant concepts. However, existing V-IP methods overlook instance-level uncertainties in query-answer generation, which can arise from model limitations (epistemic uncertainty) or variability in expert responses (aleatoric uncertainty). This paper introduces Uncertainty-Aware V-IP (UAV-IP), a novel framework that integrates uncertainty quantification into the V-IP process. We evaluate UAV-IP across four medical imaging datasets, PH2, Derm7pt, BrEaST, and SkinCon, demonstrating an average AUC improvement of approximately 3.2% while generating 20% more concise explanations compared to baseline V-IP, without sacrificing informativeness. These findings highlight the importance of uncertainty-aware reasoning in interpretable by design models for robust and reliable medical decision-making.
• 摘要：在医学成像中，AI决策支持系统必须平衡准确性和解释性，以建立用户信任并支持有效的临床决策。最近，变异信息追求（V-IP）及其变体已成为可解释的逐日设计建模技术，旨在根据人类理解，临床相关的概念来解释AI决策。但是，现有的V-IP方法忽略了查询 - 答案产生中实例级别的不确定性，这可能是由模型限制（认知不确定性）或专家响应的可变性（态度不确定性）引起的。本文介绍了不确定性感知的V-IP（UAV-IP），该框架将不确定性定量整合到V-IP过程中。我们在四个医学成像数据集（PH2，DERM7PT，乳房和肤色）上评估了无人机IP，与基线V-IP相比，平均AUC改善约为3.2％，而无需牺牲信息而产生的简洁解释多20％。这些发现强调了不确定性感知推理在设计模型中对强大而可靠的医疗决策制定的重要性。

Title: Noise-Informed Diffusion-Generated Image Detection with Anomaly Attention

• Authors: Weinan Guan, Wei Wang, Bo Peng, Ziwen He, Jing Dong, Haonan Cheng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16743
• Pdf URL: https://arxiv.org/pdf/2506.16743
• Copy Paste: [[2506.16743]] Noise-Informed Diffusion-Generated Image Detection with Anomaly Attention(https://arxiv.org/abs/2506.16743)
• Keywords: generation
• Abstract: With the rapid development of image generation technologies, especially the advancement of Diffusion Models, the quality of synthesized images has significantly improved, raising concerns among researchers about information security. To mitigate the malicious abuse of diffusion models, diffusion-generated image detection has proven to be an effective this http URL, a key challenge for forgery detection is generalising to diffusion models not seen during training. In this paper, we address this problem by focusing on image noise. We observe that images from different diffusion models share similar noise patterns, distinct from genuine images. Building upon this insight, we introduce a novel Noise-Aware Self-Attention (NASA) module that focuses on noise regions to capture anomalous patterns. To implement a SOTA detection model, we incorporate NASA into Swin Transformer, forming an novel detection architecture NASA-Swin. Additionally, we employ a cross-modality fusion embedding to combine RGB and noise images, along with a channel mask strategy to enhance feature learning from both modalities. Extensive experiments demonstrate the effectiveness of our approach in enhancing detection capabilities for diffusion-generated images. When encountering unseen generation methods, our approach achieves the state-of-the-art this http URL code is available at this https URL.
• 摘要：随着图像产生技术的快速发展，尤其是扩散模型的发展，合成图像的质量已大大提高，从而引起了研究人员对信息安全的关注。为了减轻对扩散模型的恶意滥用，扩散生成的图像检测已被证明是有效的HTTP URL，伪造检测的主要挑战是对训练期间看不到的扩散模型的普遍性。在本文中，我们通过关注图像噪声来解决此问题。我们观察到，来自不同扩散模型的图像共享相似的噪声模式，这与真实图像不同。在这种见解的基础上，我们引入了一种新颖的噪音感知自我注意力（NASA）模块，该模块的重点是噪声区域以捕获异常模式。为了实现SOTA检测模型，我们将NASA纳入Swin Transformer，形成了一种新型的检测体系结构NASA-SWIN。此外，我们采用交叉模式融合嵌入来结合RGB和噪声图像，以及通道掩码策略，以增强两种模式的特征学习。广泛的实验证明了我们方法在增强扩散生成图像的检测能力方面的有效性。在遇到看不见的生成方法时，我们的方法可以在此HTTPS URL上获得此HTTP URL代码的最新方法。

Title: Infrared and Visible Image Fusion Based on Implicit Neural Representations

• Authors: Shuchen Sun, Ligen Shi, Chang Liu, Lina Wu, Jun Qiu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16773
• Pdf URL: https://arxiv.org/pdf/2506.16773
• Copy Paste: [[2506.16773]] Infrared and Visible Image Fusion Based on Implicit Neural Representations(https://arxiv.org/abs/2506.16773)
• Keywords: super-resolution
• Abstract: Infrared and visible light image fusion aims to combine the strengths of both modalities to generate images that are rich in information and fulfill visual or computational requirements. This paper proposes an image fusion method based on Implicit Neural Representations (INR), referred to as INRFuse. This method parameterizes a continuous function through a neural network to implicitly represent the multimodal information of the image, breaking through the traditional reliance on discrete pixels or explicit features. The normalized spatial coordinates of the infrared and visible light images serve as inputs, and multi-layer perceptrons is utilized to adaptively fuse the features of both modalities, resulting in the output of the fused image. By designing multiple loss functions, the method jointly optimizes the similarity between the fused image and the original images, effectively preserving the thermal radiation information of the infrared image while maintaining the texture details of the visible light image. Furthermore, the resolution-independent characteristic of INR allows for the direct fusion of images with varying resolutions and achieves super-resolution reconstruction through high-density coordinate queries. Experimental results indicate that INRFuse outperforms existing methods in both subjective visual quality and objective evaluation metrics, producing fused images with clear structures, natural details, and rich information without the necessity for a training dataset.
• 摘要：红外且可见的光图像融合旨在结合两种方式的优势，以生成丰富的信息并满足视觉或计算要求的图像。本文提出了一种基于隐式神经表示（INR）的图像融合方法，称为inrfuse。此方法通过神经网络参数将连续的函数参数为隐式表示图像的多模式信息，从而破坏了对离散像素或显式特征的传统依赖。红外和可见光图像的归一化空间坐标用作输入，多层感知器可用于适应两种模态的特征，从而导致融合图像的输出。通过设计多个损失功能，该方法共同优化了融合图像和原始图像之间的相似性，从而有效地保留了红外图像的热辐射信息，同时保持可见光光图像的纹理细节。此外，INR的独立特征可以直接融合具有不同分辨率的图像，并通过高密度坐标查询实现超分辨率重建。实验结果表明，Inrfuse在主观视觉质量和客观评估指标中的表现优于现有方法，从而在无需进行培训数据集的情况下生成具有清晰结构，自然细节和丰富信息的融合图像。

Title: PQCAD-DM: Progressive Quantization and Calibration-Assisted Distillation for Extremely Efficient Diffusion Model

• Authors: Beomseok Ko, Hyeryung Jang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.16776
• Pdf URL: https://arxiv.org/pdf/2506.16776
• Copy Paste: [[2506.16776]] PQCAD-DM: Progressive Quantization and Calibration-Assisted Distillation for Extremely Efficient Diffusion Model(https://arxiv.org/abs/2506.16776)
• Keywords: generation, generative
• Abstract: Diffusion models excel in image generation but are computational and resource-intensive due to their reliance on iterative Markov chain processes, leading to error accumulation and limiting the effectiveness of naive compression techniques. In this paper, we propose PQCAD-DM, a novel hybrid compression framework combining Progressive Quantization (PQ) and Calibration-Assisted Distillation (CAD) to address these challenges. PQ employs a two-stage quantization with adaptive bit-width transitions guided by a momentum-based mechanism, reducing excessive weight perturbations in low-precision. CAD leverages full-precision calibration datasets during distillation, enabling the student to match full-precision performance even with a quantized teacher. As a result, PQCAD-DM achieves a balance between computational efficiency and generative quality, halving inference time while maintaining competitive performance. Extensive experiments validate PQCAD-DM's superior generative capabilities and efficiency across diverse datasets, outperforming fixed-bit quantization methods.
• 摘要：扩散模型在图像生成中表现出色，但由于其依赖迭代马尔可夫链过程而具有计算和资源密集型，从而导致错误积累并限制了天真的压缩技术的有效性。在本文中，我们提出了一个新型的混合压缩框架PQCAD-DM，结合了进行性量化（PQ）和校准辅助蒸馏（CAD）来应对这些挑战。 PQ采用了两阶段量化，并以基于动量的机制为指导的自适应位宽度转变，从而减少了低精度的重量扰动。 CAD在蒸馏过程中利用完整精确的校准数据集，使学生即使与量化的老师都可以匹配完整精确的表现。结果，PQCAD-DM在计算效率和生成质量之间取得了平衡，在保持竞争性能的同时将推理时间减半。广泛的实验验证了PQCAD-DM的出色生成能力和各种数据集的效率，表现优于固定位量化方法。

Title: Revisiting LoRA through the Lens of Parameter Redundancy: Spectral Encoding Helps

• Authors: Jiashun Cheng, Aochuan Chen, Nuo Chen, Ziqi Gao, Yuhan Li, Jia Li, Fugee Tsung
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.16787
• Pdf URL: https://arxiv.org/pdf/2506.16787
• Copy Paste: [[2506.16787]] Revisiting LoRA through the Lens of Parameter Redundancy: Spectral Encoding Helps(https://arxiv.org/abs/2506.16787)
• Keywords: generation
• Abstract: Low-Rank Adaptation (LoRA) has emerged as a prominent technique for fine-tuning large foundation models. Despite its successes, the substantial parameter redundancy, which limits the capacity and efficiency of LoRA, has been recognized as a bottleneck. In this work, we systematically investigate the impact of redundancy in fine-tuning LoRA and reveal that reducing density redundancy does not degrade expressiveness. Based on this insight, we introduce \underline{S}pectral-\underline{e}ncoding \underline{L}ow-\underline{R}ank \underline{A}daptation (SeLoRA), which harnesses the robust expressiveness of spectral bases to re-parameterize LoRA from a sparse spectral subspace. Designed with simplicity, SeLoRA enables seamless integration with various LoRA variants for performance boosting, serving as a scalable plug-and-play framework. Extensive experiments substantiate that SeLoRA achieves greater efficiency with fewer parameters, delivering superior performance enhancements over strong baselines on various downstream tasks, including commonsense reasoning, math reasoning, and code generation.
• 摘要：低级适应性（LORA）已成为微调大型基础模型的重要技术。尽管取得了成功，但限制洛拉的容量和效率的实质参数冗余已被认为是瓶颈。在这项工作中，我们系统地研究了冗余在微调洛拉中的影响，并揭示了降低密度冗余不会降低表现力。基于这个见解，我们介绍\下划线{s} pectr- \下划线{ Selora以简单的方式设计，可以通过各种Lora变体进行无缝集成，以提高性能，并用作可扩展的插件框架。广泛的实验证实了Selora可以通过更少的参数实现更高的效率，从而在各种下游任务（包括常识推理，数学推理和代码生成）上提供了优于强大基线的卓越性能。

Title: RealSR-R1: Reinforcement Learning for Real-World Image Super-Resolution with Vision-Language Chain-of-Thought

• Authors: Junbo Qiao, Miaomiao Cai, Wei Li, Yutong Liu, Xudong Huang, Gaoqi He, Jiao Xie, Jie Hu, Xinghao Chen, Shaohui Lin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16796
• Pdf URL: https://arxiv.org/pdf/2506.16796
• Copy Paste: [[2506.16796]] RealSR-R1: Reinforcement Learning for Real-World Image Super-Resolution with Vision-Language Chain-of-Thought(https://arxiv.org/abs/2506.16796)
• Keywords: restoration, super-resolution, generation
• Abstract: Real-World Image Super-Resolution is one of the most challenging task in image restoration. However, existing methods struggle with an accurate understanding of degraded image content, leading to reconstructed results that are both low-fidelity and unnatural. We present RealSR-R1 in this work, which empowers the RealSR models with understanding and reasoning capabilities. Inspired by the success of Chain of Thought (CoT) in large language models (LLMs), we simulate the human process of handling degraded images and propose the VLCoT framework, which integrates vision and language reasoning. The framework aims to precisely restore image details by progressively generating more comprehensive text and higher-resolution images. To overcome the challenge of traditional supervised learning CoT failing to generalize to real-world scenarios, we introduce, for the first time, Group Relative Policy Optimization (GRPO) into the Real-World Image Super-Resolution task. We propose VLCoT-GRPO as a solution, which designs four reward functions: (1) Format reward, used to standardize the CoT process; (2) Degradation reward, to incentivize accurate degradation estimation; (3) Understanding reward, to ensure the accuracy of the generated content; and (4) Generation reward, where we propose using a visual expert model to evaluate the quality of generated images, encouraging the model to generate more realistic images. Extensive experiments demonstrate that our proposed RealSR-R1 can generate realistic details and accurately understand image content, particularly in semantically rich scenes or images with severe degradation.
• 摘要：现实世界图像超分辨率是图像恢复中最具挑战性的任务之一。但是，现有的方法努力对降级图像含量的准确理解，从而导致既低保真又不自然的重建结果。我们在这项工作中介绍了Realsr-R1，该工作使RealSR模型具有理解和推理能力。受到大型语言模型（LLMS）的思想链（COT）成功的启发，我们模拟了处理降级图像的人类过程，并提出了整合视觉和语言推理的VLCOT框架。该框架旨在通过逐步生成更全面的文本和更高分辨率的图像来精确恢复图像细节。为了克服传统的监督学习婴儿cot的挑战，无法推广到现实世界中的情况，我们首次将小组相对策略优化（GRPO）介绍到现实世界图像超级分辨率任务中。我们建议VLCOT-GRPO作为解决方案，该解决方案设计了四个奖励功能：（1）格式奖励，用于标准化COT过程； （2）降解奖励，激励准确的降解估计； （3）理解奖励，以确保生成内容的准确性； （4）生成奖励，我们建议使用视觉专家模型评估生成的图像的质量，从而鼓励模型生成更真实的图像。广泛的实验表明，我们提出的REALSR-R1可以生成逼真的细节并准确理解图像内容，尤其是在具有严重降解的语义富裕场景或图像中。

Title: Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentation

• Authors: Riccardo Corvi, Davide Cozzolino, Ekta Prashnani, Shalini De Mello, Koki Nagano, Luisa Verdoliva
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16802
• Pdf URL: https://arxiv.org/pdf/2506.16802
• Copy Paste: [[2506.16802]] Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentation(https://arxiv.org/abs/2506.16802)
• Keywords: generation, generative
• Abstract: Synthetic video generation is progressing very rapidly. The latest models can produce very realistic high-resolution videos that are virtually indistinguishable from real ones. Although several video forensic detectors have been recently proposed, they often exhibit poor generalization, which limits their applicability in a real-world scenario. Our key insight to overcome this issue is to guide the detector towards seeing what really matters. In fact, a well-designed forensic classifier should focus on identifying intrinsic low-level artifacts introduced by a generative architecture rather than relying on high-level semantic flaws that characterize a specific model. In this work, first, we study different generative architectures, searching and identifying discriminative features that are unbiased, robust to impairments, and shared across models. Then, we introduce a novel forensic-oriented data augmentation strategy based on the wavelet decomposition and replace specific frequency-related bands to drive the model to exploit more relevant forensic cues. Our novel training paradigm improves the generalizability of AI-generated video detectors, without the need for complex algorithms and large datasets that include multiple synthetic generators. To evaluate our approach, we train the detector using data from a single generative model and test it against videos produced by a wide range of other models. Despite its simplicity, our method achieves a significant accuracy improvement over state-of-the-art detectors and obtains excellent results even on very recent generative models, such as NOVA and FLUX. Code and data will be made publicly available.
• 摘要：合成视频生成的进展非常迅速。最新模型可以生成非常现实的高分辨率视频，这些视频几乎与真实的视频无法区分。尽管最近提出了几个视频取证探测器，但它们经常表现出较差的概括，这限制了其在现实情况下的适用性。我们克服这个问题的主要见解是指导检测器查看真正重要的事情。实际上，设计良好的法医分类器应集中于识别生成架构引入的内在低级伪像，而不是依靠特定特定模型的高级语义缺陷。在这项工作中，首先，我们研究了不同的生成体系结构，搜索和识别公正的歧视性特征，可损害的强​​大和跨模型共享。然后，我们基于小波分解引入了一种新型的面向法医的数据增强策略，并替换了与频率相关的频段，以驱动模型以利用更相关的法医提示。我们的新型训练范式提高了AI生成的视频检测器的普遍性，而无需复杂的算法和包括多个合成发生器的大型数据集。为了评估我们的方法，我们使用来自单个生成模型的数据训练检测器，并根据各种其他模型对视频进行测试。尽管它很简单，但我们的方法仍比最先进的探测器获得了显着的准确性改善，并且即使在最近的生成模型（例如Nova和Flux）中，也获得了出色的结果。代码和数据将公开可用。

Title: FOCUS: Unified Vision-Language Modeling for Interactive Editing Driven by Referential Segmentation

• Authors: Fan Yang, Yousong Zhu, Xin Li, Yufei Zhan, Hongyin Zhao, Shurong Zheng, Yaowei Wang, Ming Tang, Jinqiao Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16806
• Pdf URL: https://arxiv.org/pdf/2506.16806
• Copy Paste: [[2506.16806]] FOCUS: Unified Vision-Language Modeling for Interactive Editing Driven by Referential Segmentation(https://arxiv.org/abs/2506.16806)
• Keywords: generation, generative
• Abstract: Recent Large Vision Language Models (LVLMs) demonstrate promising capabilities in unifying visual understanding and generative modeling, enabling both accurate content understanding and flexible editing. However, current approaches treat "what to see" and "how to edit" separately: they either perform isolated object segmentation or utilize segmentation masks merely as conditional prompts for local edit generation tasks, often relying on multiple disjointed models. To bridge these gaps, we introduce FOCUS, a unified LVLM that integrates segmentation-aware perception and controllable object-centric generation within an end-to-end framework. FOCUS employs a dual-branch visual encoder to simultaneously capture global semantic context and fine-grained spatial details. In addition, we leverage a MoVQGAN-based visual tokenizer to produce discrete visual tokens that enhance generation quality. To enable accurate and controllable image editing, we propose a progressive multi-stage training pipeline, where segmentation masks are jointly optimized and used as spatial condition prompts to guide the diffusion decoder. This strategy aligns visual encoding, segmentation, and generation modules, effectively bridging segmentation-aware perception with fine-grained visual synthesis. Extensive experiments across three core tasks, including multimodal understanding, referring segmentation accuracy, and controllable image generation, demonstrate that FOCUS achieves strong performance by jointly optimizing visual perception and generative capabilities.
• 摘要：最近的大型视觉语言模型（LVLM）展示了统一视觉理解和生成建模的有希望的能力，从而实现了准确的内容理解和灵活的编辑。但是，当前的方法将“看到的内容”和“如何分别进行编辑”视为：他们要么仅作为本地编辑生成任务的条件提示，要么将隔离的对象分割或利用分割掩码，通常依赖于多个分离的模型。为了弥合这些差距，我们引入了焦点，这是一个统一的LVLM，该LVLM在端到端的框架内集成了细分感知感知和以对象为中心的可控生成。 Focus采用双分支视觉编码器同时捕获全球语义上下文和细粒的空间细节。此外，我们利用基于MOVQGAN的视觉令牌制作来产生离散的视觉令牌，从而提高发电质量。为了启用准确且可控的图像编辑，我们提出了一条渐进的多阶段训练管道，其中分割掩模是共同优化的，并用作空间条件提示，以指导扩散解码器。该策略将视觉编码，分割和生成模块对齐，有效地弥合了分割感知感知，并与细粒度的视觉合成。跨三个核心任务的广泛实验，包括多模式理解，参考细分精度和可控的图像生成，这表明焦点通过共同优化视觉感知和生成能力来实现强大的性能。

Title: Loupe: A Generalizable and Adaptive Framework for Image Forgery Detection

• Authors: Yuchu Jiang, Jiaming Chu, Jian Zhao, Xin Zhang, Xu Yang, Lei Jin, Chi Zhang, Xuelong Li
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.16819
• Pdf URL: https://arxiv.org/pdf/2506.16819
• Copy Paste: [[2506.16819]] Loupe: A Generalizable and Adaptive Framework for Image Forgery Detection(https://arxiv.org/abs/2506.16819)
• Keywords: generative
• Abstract: The proliferation of generative models has raised serious concerns about visual content forgery. Existing deepfake detection methods primarily target either image-level classification or pixel-wise localization. While some achieve high accuracy, they often suffer from limited generalization across manipulation types or rely on complex architectures. In this paper, we propose Loupe, a lightweight yet effective framework for joint deepfake detection and localization. Loupe integrates a patch-aware classifier and a segmentation module with conditional queries, allowing simultaneous global authenticity classification and fine-grained mask prediction. To enhance robustness against distribution shifts of test set, Loupe introduces a pseudo-label-guided test-time adaptation mechanism by leveraging patch-level predictions to supervise the segmentation head. Extensive experiments on the DDL dataset demonstrate that Loupe achieves state-of-the-art performance, securing the first place in the IJCAI 2025 Deepfake Detection and Localization Challenge with an overall score of 0.846. Our results validate the effectiveness of the proposed patch-level fusion and conditional query design in improving both classification accuracy and spatial localization under diverse forgery patterns. The code is available at this https URL.
• 摘要：生成模型的扩散引起了人们对视觉内容伪造的严重关注。现有的DeepFake检测方法主要针对图像级分类或像素定位。尽管有些人获得了很高的精度，但他们通常会遭受在操纵类型中的概括有限或依靠复杂的架构。在本文中，我们提出了Loupe，这是一个轻巧但有效的框架，用于联合深层检测和定位。 Loupe与条件查询集成了贴片感知分类器和分割模块，从而允许同时进行全局真实性分类和细粒度的掩码预测。为了增强对测试集的分布变化的鲁棒性，Loupe通过利用斑块级预测来监督分割头，引入了伪标签引导的测试时间适应机制。 DDL数据集的广泛实验表明，Loupe实现了最先进的性能，确保了IJCAI 2025 DeepFake检测和本地化挑战的第一名，总分为0.846。我们的结果证明了拟议的斑块级融合和有条件查询设计在提高分类准确性和空间定位下的有效性。该代码可在此HTTPS URL上找到。

Title: Reward-Agnostic Prompt Optimization for Text-to-Image Diffusion Models

• Authors: Semin Kim, Yeonwoo Cha, Jaehoon Yoo, Seunghoon Hong
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.16853
• Pdf URL: https://arxiv.org/pdf/2506.16853
• Copy Paste: [[2506.16853]] Reward-Agnostic Prompt Optimization for Text-to-Image Diffusion Models(https://arxiv.org/abs/2506.16853)
• Keywords: generation
• Abstract: We investigate a general approach for improving user prompts in text-to-image (T2I) diffusion models by finding prompts that maximize a reward function specified at test-time. Although diverse reward models are used for evaluating image generation, existing automated prompt engineering methods typically target specific reward configurations. Consequently, these specialized designs exhibit suboptimal performance when applied to new prompt engineering scenarios involving different reward models. To address this limitation, we introduce RATTPO (Reward-Agnostic Test-Time Prompt Optimization), a flexible test-time optimization method applicable across various reward scenarios without modification. RATTPO iteratively searches for optimized prompts by querying large language models (LLMs) \textit{without} requiring reward-specific task descriptions. Instead, it uses the optimization trajectory and a novel reward-aware feedback signal (termed a "hint") as context. Empirical results demonstrate the versatility of RATTPO, effectively enhancing user prompts across diverse reward setups that assess various generation aspects, such as aesthetics, general human preference, or spatial relationships between objects. RATTPO surpasses other test-time search baselines in search efficiency, using up to 3.5 times less inference budget, and, given sufficient inference budget, achieves performance comparable to learning-based baselines that require reward-specific fine-tuning. The code is available at this https URL.
• 摘要：我们通过查找提示提示在测试时指定的奖励功能，研究了一种通用方法，以改善用户提示（T2I）扩散模型。尽管多种奖励模型用于评估图像生成，但现有的自动化及时工程方法通常针对特定的奖励配置。因此，这些专业设计将在涉及不同奖励模型的新及时工程场景上应用时表现出次优性能。为了解决此限制，我们介绍了Rattpo（奖励 - 敏捷的测试时间提示优化），这是一种适用于各种奖励场景的灵活测试时间优化方法，而无需修改。 Rattpo迭代通过查询大型语言模型（LLMS）\ textIt {而不}需要需要特定于奖励的任务描述来搜索优化的提示。取而代之的是，它使用优化轨迹和新颖的奖励感知反馈信号（称为“提示”）作为上下文。经验结果证明了rattpo的多功能性，有效地增强了各种奖励设置的提示，这些奖励设置评估了各种一代方面，例如美学，一般的人类偏好或对象之间的空间关系。 Rattpo在搜索效率方面超过了其他测试时间搜索基线，使用推理预算少的3.5倍，并且鉴于推理预算足够，可以实现与需要基于学习的基线相媲美的绩效，这些基线需要特定于奖励的微调。该代码可在此HTTPS URL上找到。

Title: Visual-Instructed Degradation Diffusion for All-in-One Image Restoration

• Authors: Wenyang Luo, Haina Qin, Zewen Chen, Libin Wang, Dandan Zheng, Yuming Li, Yufan Liu, Bing Li, Weiming Hu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.16960
• Pdf URL: https://arxiv.org/pdf/2506.16960
• Copy Paste: [[2506.16960]] Visual-Instructed Degradation Diffusion for All-in-One Image Restoration(https://arxiv.org/abs/2506.16960)
• Keywords: restoration
• Abstract: Image restoration tasks like deblurring, denoising, and dehazing usually need distinct models for each degradation type, restricting their generalization in real-world scenarios with mixed or unknown degradations. In this work, we propose \textbf{Defusion}, a novel all-in-one image restoration framework that utilizes visual instruction-guided degradation diffusion. Unlike existing methods that rely on task-specific models or ambiguous text-based priors, Defusion constructs explicit \textbf{visual instructions} that align with the visual degradation patterns. These instructions are grounded by applying degradations to standardized visual elements, capturing intrinsic degradation features while agnostic to image semantics. Defusion then uses these visual instructions to guide a diffusion-based model that operates directly in the degradation space, where it reconstructs high-quality images by denoising the degradation effects with enhanced stability and generalizability. Comprehensive experiments demonstrate that Defusion outperforms state-of-the-art methods across diverse image restoration tasks, including complex and real-world degradations.
• 摘要：图像恢复任务（例如去毛，去核和去悬式）通常需要为每种降解类型的不同模型，从而限制了它们在现实世界中具有混合或未知降解的现实情况。在这项工作中，我们提出了\ textbf {defusion}，这是一种新颖的多合一图像恢复框架，利用视觉指导引导的降解扩散。与依赖于特定于任务模型或基于文本的先验模棱两可的现有方法不同，报道构造了与视觉降级模式一致的显式\ textbf {Visture指令}。这些说明是通过将降解施加到标准化的视觉元素上的基础，从而在不可知论到图像语义上捕获了内在的降解特征。然后，滥用使用这些视觉指令来指导基于扩散的模型，该模型直接在退化空间中运行，在该模型中，它通过降低降解效应而以增强的稳定性和普遍性来重建高质量的图像。全面的实验表明，滥用的表现优于各种图像恢复任务（包括复杂和现实世界中的降解）的最先进方法。

Title: Reversing Flow for Image Restoration

• Authors: Haina Qin, Wenyang Luo, Libin Wang, Dandan Zheng, Jingdong Chen, Ming Yang, Bing Li, Weiming Hu
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2506.16961
• Pdf URL: https://arxiv.org/pdf/2506.16961
• Copy Paste: [[2506.16961]] Reversing Flow for Image Restoration(https://arxiv.org/abs/2506.16961)
• Keywords: restoration, generative
• Abstract: Image restoration aims to recover high-quality (HQ) images from degraded low-quality (LQ) ones by reversing the effects of degradation. Existing generative models for image restoration, including diffusion and score-based models, often treat the degradation process as a stochastic transformation, which introduces inefficiency and complexity. In this work, we propose ResFlow, a novel image restoration framework that models the degradation process as a deterministic path using continuous normalizing flows. ResFlow augments the degradation process with an auxiliary process that disambiguates the uncertainty in HQ prediction to enable reversible modeling of the degradation process. ResFlow adopts entropy-preserving flow paths and learns the augmented degradation flow by matching the velocity field. ResFlow significantly improves the performance and speed of image restoration, completing the task in fewer than four sampling steps. Extensive experiments demonstrate that ResFlow achieves state-of-the-art results across various image restoration benchmarks, offering a practical and efficient solution for real-world applications.
• 摘要：图像恢复旨在通过逆转降解效果来从降解的低质量（LQ）降解的高质量图像中恢复高质量的图像。现有用于图像恢复的生成模型，包括扩散和基于得分的模型，通常将降解过程视为一种随机转化，引入了效率低下和复杂性。在这项工作中，我们提出了Resflow，这是一种新型的图像恢复框架，该框架将降解过程建模为使用连续归一化流量的确定性路径。 RESFlow通过一个辅助过程来增强降解过程，该过程消除了HQ预测中的不确定性，以实现对降解过程的可逆建模。 ResFlow采用熵保护流路径，并通过匹配速度场来学习增强的降解流。 ResFlow显着提高了图像恢复的性能和速度，以不到四个采样步骤完成任务。广泛的实验表明，ResFlow在各种图像恢复基准中实现最新结果，为现实世界应用提供了实用有效的解决方案。

Title: Robust Reinforcement Learning for Discrete Compositional Generation via General Soft Operators

• Authors: Marco Jiralerspong, Esther Derman, Danilo Vucetic, Nikolay Malkin, Bilun Sun, Tianyu Zhang, Pierre-Luc Bacon, Gauthier Gidel
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.17007
• Pdf URL: https://arxiv.org/pdf/2506.17007
• Copy Paste: [[2506.17007]] Robust Reinforcement Learning for Discrete Compositional Generation via General Soft Operators(https://arxiv.org/abs/2506.17007)
• Keywords: generation
• Abstract: A major bottleneck in scientific discovery involves narrowing a large combinatorial set of objects, such as proteins or molecules, to a small set of promising candidates. While this process largely relies on expert knowledge, recent methods leverage reinforcement learning (RL) to enhance this filtering. They achieve this by estimating proxy reward functions from available datasets and using regularization to generate more diverse candidates. These reward functions are inherently uncertain, raising a particularly salient challenge for scientific discovery. In this work, we show that existing methods, often framed as sampling proportional to a reward function, are inadequate and yield suboptimal candidates, especially in large search spaces. To remedy this issue, we take a robust RL approach and introduce a unified operator that seeks robustness to the uncertainty of the proxy reward function. This general operator targets peakier sampling distributions while encompassing known soft RL operators. It also leads us to a novel algorithm that identifies higher-quality, diverse candidates in both synthetic and real-world tasks. Ultimately, our work offers a new, flexible perspective on discrete compositional generation tasks. Code: this https URL.
• 摘要：科学发现中的主要瓶颈涉及缩小一组蛋白质或分子等大型对象的组合，以缩小一小群有前途的候选人。尽管此过程很大程度上取决于专家知识，但最近的方法利用了加强学习（RL）来增强这种过滤。他们通过估计可用数据集的代理奖励功能并使用正则化来产生更多不同的候选人来实现这一目标。这些奖励功能本质上是不确定的，这引起了科学发现的特别突出挑战。在这项工作中，我们表明，通常将与奖励功能成正比的采样框起来的现有方法不足和产生次优候选者，尤其是在大型搜索空间中。为了解决这个问题，我们采用了强大的RL方法，并介绍了一个统一的操作员，该操作员寻求鲁棒性，以解决代理奖励功能的不确定性。该通用操作员靶向峰采样分布，同时涵盖已知的软RL操作员。它还使我们采用了一种新颖的算法，该算法识别了合成和现实世界任务中更高质量的不同候选者。最终，我们的工作为离散的成分生成任务提供了新的灵活的观点。代码：此HTTPS URL。

Title: The Hidden Cost of an Image: Quantifying the Energy Consumption of AI Image Generation

• Authors: Giulia Bertazzini, Chiara Albisani, Daniele Baracchi, Dasara Shullani, Roberto Verdecchia
• Subjects: cs.LG, cs.MM
• Abstract URL: https://arxiv.org/abs/2506.17016
• Pdf URL: https://arxiv.org/pdf/2506.17016
• Copy Paste: [[2506.17016]] The Hidden Cost of an Image: Quantifying the Energy Consumption of AI Image Generation(https://arxiv.org/abs/2506.17016)
• Keywords: generation
• Abstract: With the growing adoption of AI image generation, in conjunction with the ever-increasing environmental resources demanded by AI, we are urged to answer a fundamental question: What is the environmental impact hidden behind each image we generate? In this research, we present a comprehensive empirical experiment designed to assess the energy consumption of AI image generation. Our experiment compares 17 state-of-the-art image generation models by considering multiple factors that could affect their energy consumption, such as model quantization, image resolution, and prompt length. Additionally, we consider established image quality metrics to study potential trade-offs between energy consumption and generated image quality. Results show that image generation models vary drastically in terms of the energy they consume, with up to a 46x difference. Image resolution affects energy consumption inconsistently, ranging from a 1.3x to 4.7x increase when doubling resolution. U-Net-based models tend to consume less than Transformer-based one. Model quantization instead results to deteriorate the energy efficiency of most models, while prompt length and content have no statistically significant impact. Improving image quality does not always come at the cost of a higher energy consumption, with some of the models producing the highest quality images also being among the most energy efficient ones.
• 摘要：随着AI图像产生的越来越多，以及AI要求的不断增长的环境资源，我们敦促回答一个基本问题：我们产生的每个图像背后隐藏的环境影响是什么？在这项研究中，我们提出了一项旨在评估AI图像产生能源消耗的综合经验实验。我们的实验通过考虑可能影响其能耗的多种因素（例如模型量化，图像分辨率和及时长度）来比较17个最先进的图像生成模型。此外，我们考虑建立的图像质量指标，以研究能耗和产生的图像质量之间的潜在权衡。结果表明，图像生成模型随消耗的能量而差异很大，最大差异为46倍。分辨率分辨率不一致地影响能源消耗，分辨率增加了1.3倍至4.7倍。基于U-NET的模型往往比基于变压器的模型少消耗。相反，模型量化会导致大多数模型的能源效率恶化，而及时长度和内容没有统计学上的显着影响。提高图像质量并不总是以更高的能源消耗为代价，其中一些产生最高质量图像的模型也是最节能的图像之一。

Title: Unsupervised Image Super-Resolution Reconstruction Based on Real-World Degradation Patterns

• Authors: Yiyang Tie, Hong Zhu, Yunyun Luo, Jing Shi
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2506.17027
• Pdf URL: https://arxiv.org/pdf/2506.17027
• Copy Paste: [[2506.17027]] Unsupervised Image Super-Resolution Reconstruction Based on Real-World Degradation Patterns(https://arxiv.org/abs/2506.17027)
• Keywords: super-resolution
• Abstract: The training of real-world super-resolution reconstruction models heavily relies on datasets that reflect real-world degradation patterns. Extracting and modeling degradation patterns for super-resolution reconstruction using only real-world low-resolution (LR) images remains a challenging task. When synthesizing datasets to simulate real-world degradation, relying solely on degradation extraction methods fails to capture both blur and diverse noise characteristics across varying LR distributions, as well as more implicit degradations such as color gamut shifts. Conversely, domain translation alone cannot accurately approximate real-world blur characteristics due to the significant degradation domain gap between synthetic and real data. To address these challenges, we propose a novel TripleGAN framework comprising two strategically designed components: The FirstGAN primarily focuses on narrowing the domain gap in blur characteristics, while the SecondGAN performs domain-specific translation to approximate target-domain blur properties and learn additional degradation patterns. The ThirdGAN is trained on pseudo-real data generated by the FirstGAN and SecondGAN to reconstruct real-world LR images. Extensive experiments on the RealSR and DRealSR datasets demonstrate that our method exhibits clear advantages in quantitative metrics while maintaining sharp reconstructions without over-smoothing artifacts. The proposed framework effectively learns real-world degradation patterns from LR observations and synthesizes aligned datasets with corresponding degradation characteristics, thereby enabling the trained network to achieve superior performance in reconstructing high-quality SR images from real-world LR inputs.
• 摘要：对现实世界的超分辨率重建模型的培训在很大程度上依赖于反映现实世界下降模式的数据集。仅使用现实世界低分辨率（LR）图像的超分辨率重建的提取和建模降解模式仍然是一项具有挑战性的任务。当合成数据集以模拟现实世界中的降级时，仅依靠降解提取方法无法捕获不同LR分布的模糊和不同的噪声特性，以及更隐含的降解，例如颜色范围的变化。相反，由于合成数据和真实数据之间的显着降解域间隙，仅域翻译不能准确地近似现实世界的模糊特征。为了应对这些挑战，我们提出了一个新型的TripleGAR框架，其中包括两个战略性设计的组件：第一gan主要侧重于缩小模糊特征中的域间隙，而第二ang则执行特定于域特异性的翻译以近似目标模糊特性并学习其他降级模式。第三gan接受了Firstgan和Secondgan生成的伪真实数据的训练，以重建现实世界LR图像。对REALSR和DREALSR数据集进行的广泛实验表明，我们的方法在定量指标方面具有明显的优势，同时保持了尖锐的重建，而无需过度光滑的工件。所提出的框架有效地从LR观测值中学习了现实世界中的退化模式，并综合了具有相应的降级特性的对齐数据集，从而使训练有素的网络能够从现实世界LR输入中重建高质量的SR图像来实现卓越的性能。

Title: Assembler: Scalable 3D Part Assembly via Anchor Point Diffusion

• Authors: Wang Zhao, Yan-Pei Cao, Jiale Xu, Yuejiang Dong, Ying Shan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.17074
• Pdf URL: https://arxiv.org/pdf/2506.17074
• Copy Paste: [[2506.17074]] Assembler: Scalable 3D Part Assembly via Anchor Point Diffusion(https://arxiv.org/abs/2506.17074)
• Keywords: generation, generative
• Abstract: We present Assembler, a scalable and generalizable framework for 3D part assembly that reconstructs complete objects from input part meshes and a reference image. Unlike prior approaches that mostly rely on deterministic part pose prediction and category-specific training, Assembler is designed to handle diverse, in-the-wild objects with varying part counts, geometries, and structures. It addresses the core challenges of scaling to general 3D part assembly through innovations in task formulation, representation, and data. First, Assembler casts part assembly as a generative problem and employs diffusion models to sample plausible configurations, effectively capturing ambiguities arising from symmetry, repeated parts, and multiple valid assemblies. Second, we introduce a novel shape-centric representation based on sparse anchor point clouds, enabling scalable generation in Euclidean space rather than SE(3) pose prediction. Third, we construct a large-scale dataset of over 320K diverse part-object assemblies using a synthesis and filtering pipeline built on existing 3D shape repositories. Assembler achieves state-of-the-art performance on PartNet and is the first to demonstrate high-quality assembly for complex, real-world objects. Based on Assembler, we further introduce an interesting part-aware 3D modeling system that generates high-resolution, editable objects from images, demonstrating potential for interactive and compositional design. Project page: this https URL
• 摘要：我们提出了汇编器，这是3D零件组件的可扩展且可推广的框架，可从输入零件网格和参考图像中重建完整的对象。与先前的方法不同，主要依赖确定性零件姿势预测和特定于类别的培训不同，汇编器旨在处理具有不同零件计数，几何形状和结构的多样化的野外物体。它通过任务制定，表示和数据的创新来解决将扩展到一般3D部分组装的核心挑战。首先，汇编器将零件组装作为生成问题，并采用扩散模型来采样合理的配置，有效地捕获了由对称性，重复的零件和多个有效的组件引起的歧义。其次，我们基于稀疏的锚点云引入了一种新颖的中心表示形式，可以在欧几里得空间而不是SE（3）姿势预测中可扩展产生。第三，我们使用建立在现有的3D Shape存储库上的合成和过滤管道来构建一个超过320k各种零件对象组件的大规模数据集。汇编器可以在Partnet上实现最新的性能，并且是第一个为复杂的现实世界对象展示高质量组装的人。基于汇编器，我们进一步引入了一个有趣的部分感知的3D建模系统，该系统从图像中生成高分辨率，可编辑的对象，从而证明了交互式和组成设计的潜力。项目页面：此HTTPS URL

Title: RGBTrack: Fast, Robust Depth-Free 6D Pose Estimation and Tracking

• Authors: Teng Guo, Jingjin Yu
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2506.17119
• Pdf URL: https://arxiv.org/pdf/2506.17119
• Copy Paste: [[2506.17119]] RGBTrack: Fast, Robust Depth-Free 6D Pose Estimation and Tracking(https://arxiv.org/abs/2506.17119)
• Keywords: generative
• Abstract: We introduce a robust framework, RGBTrack, for real-time 6D pose estimation and tracking that operates solely on RGB data, thereby eliminating the need for depth input for such dynamic and precise object pose tracking tasks. Building on the FoundationPose architecture, we devise a novel binary search strategy combined with a render-and-compare mechanism to efficiently infer depth and generate robust pose hypotheses from true-scale CAD models. To maintain stable tracking in dynamic scenarios, including rapid movements and occlusions, RGBTrack integrates state-of-the-art 2D object tracking (XMem) with a Kalman filter and a state machine for proactive object pose recovery. In addition, RGBTrack's scale recovery module dynamically adapts CAD models of unknown scale using an initial depth estimate, enabling seamless integration with modern generative reconstruction techniques. Extensive evaluations on benchmark datasets demonstrate that RGBTrack's novel depth-free approach achieves competitive accuracy and real-time performance, making it a promising practical solution candidate for application areas including robotics, augmented reality, and computer vision. The source code for our implementation will be made publicly available at this https URL.
• 摘要：我们引入了一个强大的框架RGBTRACK，以实现仅在RGB数据上运行的实时6D姿势估计和跟踪，从而消除了对这种动态和精确对象姿势跟踪任务的深度输入的需求。在基础架构架构的基础上，我们设计了一种新颖的二进制搜索策略，并结合了渲染和能力机制，以有效地推断深度并从真实规模的CAD模型中产生强大的姿势假设。为了在动态场景中保持稳定的跟踪，包括快速运动和遮挡，RGBTRACK将最新的2D对象跟踪（XMEM）与Kalman滤镜和用于主动对象姿势恢复的状态机集成在一起。此外，RGBTRACK的比例恢复模块使用初始深度估算方法动态调整了未知刻度的CAD模型，从而可以与现代生成重建技术无缝集成。对基准数据集的广泛评估表明，RGBTRACK的新颖无深度方法可实现竞争精度和实时性能，使其成为包括机器人技术，增强现实和计算机视觉在内的应用领域的实用解决方案候选者。我们实施的源代码将在此HTTPS URL上公开可用。

Title: Dynamic Watermark Generation for Digital Images using Perimeter Gated SPAD Imager PUFs

• Authors: Md Sakibur Sajal, Marc Dandin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.17134
• Pdf URL: https://arxiv.org/pdf/2506.17134
• Copy Paste: [[2506.17134]] Dynamic Watermark Generation for Digital Images using Perimeter Gated SPAD Imager PUFs(https://arxiv.org/abs/2506.17134)
• Keywords: generation
• Abstract: Digital image watermarks as a security feature can be derived from the imager's physically unclonable functions (PUFs) by utilizing the manufacturing variations, i.e., the dark signal non-uniformity (DSNU). While a few demonstrations focused on the CMOS image sensors (CIS) and active pixel sensors (APS), single photon avalanche diode (SPAD) imagers have never been investigated for this purpose. In this work, we have proposed a novel watermarking technique using perimeter gated SPAD (pgSPAD) imagers. We utilized the DSNU of three 64 x 64 pgSPAD imager chips, fabricated in a 0.35 {\mu}m standard CMOS process and analyzed the simulated watermarks for standard test images from publicly available database. Our observation shows that both source identification and tamper detection can be achieved using the proposed source-scene-specific dynamic watermarks with a controllable sensitivity-robustness trade-off.
• 摘要：数字图像水印作为安全特征可以通过使用制造变化，即暗信号不均匀（DSNU）来源自成像师的物理不可吻合的功能（PUF）。虽然一些示威活动集中在CMOS图像传感器（CI）和主动像素传感器（AP）上，但为此目的从未对单个光子雪崩（SPAD）成像器进行研究。在这项工作中，我们提出了一种使用外围盖式Spad（PGSPAD）成像仪的新型水印技术。我们利用了三个64 x 64 PGSPAD成像片芯片的DSNU，该芯片在0.35 {\ MU} M标准CMOS过程中制造，并分析了来自公开可用数据库的标准测试图像的模拟水印。我们的观察结果表明，使用拟议的源源特异性动态水印，可以通过可控的灵敏度 - 舒适性权衡来实现来源识别和篡改检测。

Title: Consistent Sampling and Simulation: Molecular Dynamics with Energy-Based Diffusion Models

• Authors: Michael Plainer, Hao Wu, Leon Klein, Stephan Günnemann, Frank Noé
• Subjects: cs.LG, cs.AI, physics.chem-ph, physics.comp-ph, stat.ML
• Abstract URL: https://arxiv.org/abs/2506.17139
• Pdf URL: https://arxiv.org/pdf/2506.17139
• Copy Paste: [[2506.17139]] Consistent Sampling and Simulation: Molecular Dynamics with Energy-Based Diffusion Models(https://arxiv.org/abs/2506.17139)
• Keywords: generative
• Abstract: Diffusion models have recently gained significant attention due to their effectiveness in various scientific domains, including biochemistry. When trained on equilibrium molecular distributions, diffusion models provide both: a generative procedure to sample equilibrium conformations and associated forces derived from the model's scores. However, using the forces for coarse-grained molecular dynamics simulations uncovers inconsistencies in the samples generated via classical diffusion inference and simulation, despite both originating from the same model. Particularly at the small diffusion timesteps required for simulations, diffusion models fail to satisfy the Fokker-Planck equation, which governs how the score should evolve over time. We interpret this deviation as an indication of the observed inconsistencies and propose an energy-based diffusion model with a Fokker-Planck-derived regularization term enforcing consistency. We demonstrate the effectiveness of our approach on toy systems, alanine dipeptide, and introduce a state-of-the-art transferable Boltzmann emulator for dipeptides that supports simulation and demonstrates enhanced consistency and efficient sampling.
• 摘要：由于其在包括生物化学在内的各种科学领域的有效性，扩散模型最近引起了人们的关注。当对平衡分子分布进行训练时，扩散模型既提供：样品平衡构象和源自模型得分的相关力的生成程序。但是，使用这些力进行粗粒分子动力学模拟，尽管两者都来自同一模型，但通过经典扩散推断和模拟产生的样品中发现了不一致的样本。特别是在模拟所需的小扩散时间步中，扩散模型无法满足Fokker-Planck方程，该方程控制了分数应如何随着时间的推移而发展。我们将这种偏差解释为观察到的不一致的指示，并提出了一个具有福克 - 普朗克衍生的正则化项的基于能量的扩散模型。我们证明了方法对玩具系统，丙氨酸二肽的有效性，并引入了可转移的玻尔兹曼仿真器的最先进的二肽，以支持模拟并证明增强的一致性和有效的采样。

Title: Deep generative models as the probability transformation functions

• Authors: Vitalii Bondar, Vira Babenko, Roman Trembovetskyi, Yurii Korobeinyk, Viktoriya Dzyuba
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.17171
• Pdf URL: https://arxiv.org/pdf/2506.17171
• Copy Paste: [[2506.17171]] Deep generative models as the probability transformation functions(https://arxiv.org/abs/2506.17171)
• Keywords: generative
• Abstract: This paper introduces a unified theoretical perspective that views deep generative models as probability transformation functions. Despite the apparent differences in architecture and training methodologies among various types of generative models - autoencoders, autoregressive models, generative adversarial networks, normalizing flows, diffusion models, and flow matching - we demonstrate that they all fundamentally operate by transforming simple predefined distributions into complex target data distributions. This unifying perspective facilitates the transfer of methodological improvements between model architectures and provides a foundation for developing universal theoretical approaches, potentially leading to more efficient and effective generative modeling techniques.
• 摘要：本文介绍了一个统一的理论观点，该观点将深层生成模型视为概率转换函数。尽管在各种类型的生成模型之间的体系结构和训练方法明显差异（自动编码器，自动回归模型，生成对抗网络，正常化的流量，扩散模型和流量匹配） - 我们证明它们都通过将简单的预定性预定分布转换为复杂的目标数据分布来从根本上运行。这种统一的观点促进了模型体系结构之间方法学改进的转移，并为开发通用理论方法提供了基础，这有可能导致更有效，有效的生成建模技术。

Title: Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Condition

• Authors: Jiaqi Li, Junshu Tang, Zhiyong Xu, Longhuang Wu, Yuan Zhou, Shuai Shao, Tianbao Yu, Zhiguo Cao, Qinglin Lu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.17201
• Pdf URL: https://arxiv.org/pdf/2506.17201
• Copy Paste: [[2506.17201]] Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Condition(https://arxiv.org/abs/2506.17201)
• Keywords: generation
• Abstract: Recent advances in diffusion-based and controllable video generation have enabled high-quality and temporally coherent video synthesis, laying the groundwork for immersive interactive gaming experiences. However, current methods face limitations in dynamics, generality, long-term consistency, and efficiency, which limit the ability to create various gameplay videos. To address these gaps, we introduce Hunyuan-GameCraft, a novel framework for high-dynamic interactive video generation in game environments. To achieve fine-grained action control, we unify standard keyboard and mouse inputs into a shared camera representation space, facilitating smooth interpolation between various camera and movement operations. Then we propose a hybrid history-conditioned training strategy that extends video sequences autoregressively while preserving game scene information. Additionally, to enhance inference efficiency and playability, we achieve model distillation to reduce computational overhead while maintaining consistency across long temporal sequences, making it suitable for real-time deployment in complex interactive environments. The model is trained on a large-scale dataset comprising over one million gameplay recordings across over 100 AAA games, ensuring broad coverage and diversity, then fine-tuned on a carefully annotated synthetic dataset to enhance precision and control. The curated game scene data significantly improves the visual fidelity, realism and action controllability. Extensive experiments demonstrate that Hunyuan-GameCraft significantly outperforms existing models, advancing the realism and playability of interactive game video generation.
• 摘要：基于扩散和可控的视频生成的最新进展使高质量和时间连贯的视频综合，为沉浸式互动游戏体验奠定了基础。但是，当前的方法面临动态，一般性，长期一致性和效率的限制，这些方法限制了创建各种游戏视频的能力。为了解决这些差距，我们介绍了Hunyuan-Gamecraft，这是一个新颖的框架，用于游戏环境中高动态互动视频的生成。为了获得细粒度的动作控制，我们将标准键盘和鼠标输入统一到共享的相机表示空间中，从而促进了各种相机和运动操作之间的平滑插值。然后，我们提出了一个混合历史条件的培训策略，该策略在保留游戏场景信息的同时，自动摄取视频序列。此外，为了提高推断效率和可玩性，我们实现了模型蒸馏以减少计算开销，同时保持跨时间序列的一致性，从而适合在复杂的交互式环境中实时部署。该模型在一个大规模数据集上进行了培训，该数据集在100多个AAA游戏中包含超过100万个游戏录制，从而确保了广泛的覆盖范围和多样性，然后在精心注释的合成数据集中进行了微调，以增强精度和控制。精心策划的游戏场景数据可显着提高视觉保真度，现实主义和动作可控性。广泛的实验表明，Hunyuan-Gamecraft极大地胜过现有模型，推动了交互式游戏视频生成的现实性和可玩性。

Title: UniFork: Exploring Modality Alignment for Unified Multimodal Understanding and Generation

• Authors: Teng Li, Quanfeng Lu, Lirui Zhao, Hao Li, Xizhou Zhu, Yu Qiao, Jun Zhang, Wenqi Shao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.17202
• Pdf URL: https://arxiv.org/pdf/2506.17202
• Copy Paste: [[2506.17202]] UniFork: Exploring Modality Alignment for Unified Multimodal Understanding and Generation(https://arxiv.org/abs/2506.17202)
• Keywords: generation
• Abstract: Unified image understanding and generation has emerged as a promising paradigm in multimodal artificial intelligence. Despite recent progress, the optimal architectural design for such unified models remains an open challenge. In this work, we start by analyzing the modality alignment behaviors of task-specific expert models for understanding and generation, as well as current unified models. Our analysis reveals a crucial observation: understanding tasks benefit from a progressively increasing modality alignment across network depth, which helps build up semantic information for better comprehension; In contrast, generation tasks follow a different trend: modality alignment increases in the early layers but decreases in the deep layers to recover spatial details. These divergent alignment patterns create a fundamental conflict in fully shared Transformer backbones, where a uniform representational flow often leads to performance compromises across two tasks. Motivated by this finding, we introduce UniFork, a novel Y-shaped architecture that shares the shallow layers for cross-task representation learning, while employing task-specific branches in deeper layers to avoid task interference. This design effectively balances shared learning and task specialization. Through extensive ablation experiments, we demonstrate that Unifork consistently outperforms conventional fully shared Transformer architectures, and achieves performance on par with or better than task-specific models.
• 摘要：统一的图像理解和产生已成为多模式人工智能中有希望的范式。尽管取得了最近的进步，但这种统一模型的最佳体系结构设计仍然是一个开放的挑战。在这项工作中，我们首先分析特定于理解和生成的任务专家模型以及当前统一模型的模式对齐行为。我们的分析揭示了一个至关重要的观察：理解任务受益于跨网络深度的逐渐增加的方式对齐，这有助于构建语义信息以更好地理解；相比之下，生成任务遵循不同的趋势：早期层的模态对准增加，但深层层的减少以恢复空间细节。这些不同的一致性模式在完全共享的变压器骨架中造成了基本冲突，在此，统一的代表性流程通常会导致跨两个任务的绩效妥协。在这一发现的激励下，我们引入了Unifork，这是一种新颖的Y形体系结构，分享了交叉任务代表学习的浅层层，同时在更深层次的层中采用特定于任务的分支来避免任务干扰。该设计有效地平衡了共享的学习和任务专业化。通过广泛的消融实验，我们证明了统一的表现始终优于常规的完全共享的变压器体系结构，并且与特定于任务的模型相比，在效果上取得了比较或更好的效果。

Title: Long-term Traffic Simulation with Interleaved Autoregressive Motion and Scenario Generation

• Authors: Xiuyu Yang, Shuhan Tan, Philipp Krähenbühl
• Subjects: cs.CV, cs.AI, cs.RO
• Abstract URL: https://arxiv.org/abs/2506.17213
• Pdf URL: https://arxiv.org/pdf/2506.17213
• Copy Paste: [[2506.17213]] Long-term Traffic Simulation with Interleaved Autoregressive Motion and Scenario Generation(https://arxiv.org/abs/2506.17213)
• Keywords: generation
• Abstract: An ideal traffic simulator replicates the realistic long-term point-to-point trip that a self-driving system experiences during deployment. Prior models and benchmarks focus on closed-loop motion simulation for initial agents in a scene. This is problematic for long-term simulation. Agents enter and exit the scene as the ego vehicle enters new regions. We propose InfGen, a unified next-token prediction model that performs interleaved closed-loop motion simulation and scene generation. InfGen automatically switches between closed-loop motion simulation and scene generation mode. It enables stable long-term rollout simulation. InfGen performs at the state-of-the-art in short-term (9s) traffic simulation, and significantly outperforms all other methods in long-term (30s) simulation. The code and model of InfGen will be released at this https URL
• 摘要：理想的交通模拟器复制了自动驾驶系统在部署过程中经历的现实的长期点对点之旅。先前的模型和基准将场景中初始代理的闭环运动模拟着眼于闭环运动模拟。这对于长期模拟是有问题的。当自我车辆进入新区域时，代理商进入并退出了现场。我们提出了Infgen，这是一个统一的下一步预测模型，该模型执行交错的闭环运动模拟和场景生成。 Infgen自动在闭环运动模拟和场景生成模式之间进行切换。它可以实现稳定的长期推出模拟。 Infgen在短期（9S）交通模拟中在最新的技术中执行，并在长期（30s）模拟中明显优于所有其他方法。 Infgen的代码和模型将在此HTTPS URL上发布

Title: Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens

• Authors: Zeyuan Yang, Xueyang Yu, Delin Chen, Maohao Shen, Chuang Gan
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.17218
• Pdf URL: https://arxiv.org/pdf/2506.17218
• Copy Paste: [[2506.17218]] Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens(https://arxiv.org/abs/2506.17218)
• Keywords: generation
• Abstract: Vision-language models (VLMs) excel at multimodal understanding, yet their text-only decoding forces them to verbalize visual reasoning, limiting performance on tasks that demand visual imagination. Recent attempts train VLMs to render explicit images, but the heavy image-generation pre-training often hinders the reasoning ability. Inspired by the way humans reason with mental imagery-the internal construction and manipulation of visual cues-we investigate whether VLMs can reason through interleaved multimodal trajectories without producing explicit images. To this end, we present a Machine Mental Imagery framework, dubbed as Mirage, which augments VLM decoding with latent visual tokens alongside ordinary text. Concretely, whenever the model chooses to ``think visually'', it recasts its hidden states as next tokens, thereby continuing a multimodal trajectory without generating pixel-level images. Begin by supervising the latent tokens through distillation from ground-truth image embeddings, we then switch to text-only supervision to make the latent trajectory align tightly with the task objective. A subsequent reinforcement learning stage further enhances the multimodal reasoning capability. Experiments on diverse benchmarks demonstrate that Mirage unlocks stronger multimodal reasoning without explicit image generation.
• 摘要：视觉语言模型（VLMS）在多模式理解方面表现出色，但它们的文本解码迫使他们言行视觉推理，从而限制了需要视觉想象的任务的性能。最近的尝试训练VLM来呈现明确的图像，但是沉重的图像生成预训练通常会阻碍推理能力。受到人类推理的方式的启发，内部构造和操纵视觉提示，研究VLM是否可以通过交织的多模式轨迹来推理而无需产生明确的图像。为此，我们提出了一个被称为Mirage的机器心理图像框架，该框架将VLM与普通文本一起增强VLM解码。具体而言，每当模型选择``视觉上思考''时，都将其隐藏状态重新铸造为“隔壁”，从而在不生成像素级图像的情况下继续进行多模式轨迹。首先，通过从地面图像嵌入中蒸馏来监督潜在令牌，然后我们切换到仅文本的监督，以使潜在的轨迹与任务目标紧密地保持一致。随后的增强学习阶段进一步增强了多模式推理能力。对不同基准测试的实验表明，幻影在没有明确的图像产生的情况下解锁了更强的多模式推理。

Title: Emergent Temporal Correspondences from Video Diffusion Transformers

• Authors: Jisu Nam, Soowon Son, Dahyun Chung, Jiyoung Kim, Siyoon Jin, Junhwa Hur, Seungryong Kim
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.17220
• Pdf URL: https://arxiv.org/pdf/2506.17220
• Copy Paste: [[2506.17220]] Emergent Temporal Correspondences from Video Diffusion Transformers(https://arxiv.org/abs/2506.17220)
• Keywords: generation
• Abstract: Recent advancements in video diffusion models based on Diffusion Transformers (DiTs) have achieved remarkable success in generating temporally coherent videos. Yet, a fundamental question persists: how do these models internally establish and represent temporal correspondences across frames? We introduce DiffTrack, the first quantitative analysis framework designed to answer this question. DiffTrack constructs a dataset of prompt-generated video with pseudo ground-truth tracking annotations and proposes novel evaluation metrics to systematically analyze how each component within the full 3D attention mechanism of DiTs (e.g., representations, layers, and timesteps) contributes to establishing temporal correspondences. Our analysis reveals that query-key similarities in specific, but not all, layers play a critical role in temporal matching, and that this matching becomes increasingly prominent during the denoising process. We demonstrate practical applications of DiffTrack in zero-shot point tracking, where it achieves state-of-the-art performance compared to existing vision foundation and self-supervised video models. Further, we extend our findings to motion-enhanced video generation with a novel guidance method that improves temporal consistency of generated videos without additional training. We believe our work offers crucial insights into the inner workings of video DiTs and establishes a foundation for further research and applications leveraging their temporal understanding.
• 摘要：基于扩散变压器（DIT）的视频扩散模型的最新进步在生成时间连贯的视频方面取得了巨大的成功。但是，一个基本问题仍然存在：这些模型如何在内部建立和表示跨帧的时间对应关系？我们介绍了Diffrack，这是第一个旨在回答这个问题的定量分析框架。 Diffrack构建了一个迅速生成的视频的数据集，其中具有伪基真实跟踪注释的注释，并提出了新颖的评估指标，以系统地分析DIT的完整3D注意机制中的每个组件如何（例如，表示，层，层和时间段）如何为建立时间的相应性提供了贡献。我们的分析表明，特定但不是全部的查询键相似性在时间匹配中起着至关重要的作用，并且这种匹配在转化过程中变得越来越突出。我们展示了Diffrack在零击点跟踪中的实际应用，与现有视觉基础和自我监督的视频模型相比，它可以在其中实现最先进的性能。此外，我们通过一种新颖的指导方法将发现扩展到运动增强视频的生成，该方法在没有额外培训的情况下提高了生成视频的时间一致性。我们认为，我们的工作为视频点的内部运作提供了重要的见解，并为利用其时间理解的进一步研究和应用建立了基础。