2025-07-03

Title: Few-Shot Inspired Generative Zero-Shot Learning

• Authors: Md Shakil Ahamed Shohag, Q. M. Jonathan Wu, Farhad Pourpanah
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.01026
• Pdf URL: https://arxiv.org/pdf/2507.01026
• Copy Paste: [[2507.01026]] Few-Shot Inspired Generative Zero-Shot Learning(https://arxiv.org/abs/2507.01026)
• Keywords: generative
• Abstract: Generative zero-shot learning (ZSL) methods typically synthesize visual features for unseen classes using predefined semantic attributes, followed by training a fully supervised classification model. While effective, these methods require substantial computational resources and extensive synthetic data, thereby relaxing the original ZSL assumptions. In this paper, we propose FSIGenZ, a few-shot-inspired generative ZSL framework that reduces reliance on large-scale feature synthesis. Our key insight is that class-level attributes exhibit instance-level variability, i.e., some attributes may be absent or partially visible, yet conventional ZSL methods treat them as uniformly present. To address this, we introduce Model-Specific Attribute Scoring (MSAS), which dynamically re-scores class attributes based on model-specific optimization to approximate instance-level variability without access to unseen data. We further estimate group-level prototypes as clusters of instances based on MSAS-adjusted attribute scores, which serve as representative synthetic features for each unseen class. To mitigate the resulting data imbalance, we introduce a Dual-Purpose Semantic Regularization (DPSR) strategy while training a semantic-aware contrastive classifier (SCC) using these prototypes. Experiments on SUN, AwA2, and CUB benchmarks demonstrate that FSIGenZ achieves competitive performance using far fewer synthetic features.
• 摘要：生成零射学习（ZSL）方法通常使用预定义的语义属性合成视觉特征，然后训练完全监督的分类模型。尽管有效，但这些方法需要大量的计算资源和广泛的合成数据，从而放松了原始的ZSL假设。在本文中，我们提出了FSIGENZ，这是一些受弹性的生成ZSL框架，可降低对大规模特征合成的依赖。我们的主要见解是，类属性属性表现出实例级别的可变性，即可能不存在或部分可见某些属性，但常规的ZSL方法将其视为均匀存在的。为了解决这个问题，我们介绍了特定于模型的属性评分（MSA），该属性得分（MSA）基于特定于模型的优化，动态评分类属性，以近似实例级别的可变性，而无需访问看不见的数据。我们将进一步估计组级原型作为基于MSA调整的属性分数的实例集群，这些属性得分是每个看不见类的代表性合成特征。为了减轻所得数据不平衡，我们在使用这些原型训练语义意识的对比分类器（SCC）时引入了双重用途语义正则化（DPSR）策略。在Sun，Awa2和Cub基准上进行的实验表明，Fsigenz使用较少的合成特征实现了竞争性能。

Title: PathCoT: Chain-of-Thought Prompting for Zero-shot Pathology Visual Reasoning

• Authors: Junjie Zhou, Yingli Zuo, Shichang Feng, Peng Wan, Qi Zhu, Daoqiang Zhang, Wei Shao
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2507.01029
• Pdf URL: https://arxiv.org/pdf/2507.01029
• Copy Paste: [[2507.01029]] PathCoT: Chain-of-Thought Prompting for Zero-shot Pathology Visual Reasoning(https://arxiv.org/abs/2507.01029)
• Keywords: generative
• Abstract: With the development of generative artificial intelligence and instruction tuning techniques, multimodal large language models (MLLMs) have made impressive progress on general reasoning tasks. Benefiting from the chain-of-thought (CoT) methodology, MLLMs can solve the visual reasoning problem step-by-step. However, existing MLLMs still face significant challenges when applied to pathology visual reasoning tasks: (1) LLMs often underperforms because they lack domain-specific information, which can lead to model hallucinations. (2) The additional reasoning steps in CoT may introduce errors, leading to the divergence of answers. To address these limitations, we propose PathCoT, a novel zero-shot CoT prompting method which integrates the pathology expert-knowledge into the reasoning process of MLLMs and incorporates self-evaluation to mitigate divergence of answers. Specifically, PathCoT guides the MLLM with prior knowledge to perform as pathology experts, and provides comprehensive analysis of the image with their domain-specific knowledge. By incorporating the experts' knowledge, PathCoT can obtain the answers with CoT reasoning. Furthermore, PathCoT incorporates a self-evaluation step that assesses both the results generated directly by MLLMs and those derived through CoT, finally determining the reliable answer. The experimental results on the PathMMU dataset demonstrate the effectiveness of our method on pathology visual understanding and reasoning.
• 摘要：随着生成人工智能和教学调整技术的发展，多模式大语模型（MLLM）在一般推理任务上取得了令人印象深刻的进步。受益于经过经过思考链（COT）方法，MLLM可以逐步解决视觉推理问题。但是，现有的MLLM在应用于病理学视觉推理任务时仍然面临重大挑战：（1）LLMS通常不足，因为它们缺乏特定于领域的信息，这可能导致模型幻觉。 （2）COT中的其他推理步骤可能会引入错误，从而导致答案的分歧。为了解决这些局限性，我们提出了一种新型的零摄氏COT促进方法Pathcot，将病理学专家知识整合到MLLM的推理过程中，并结合了自我评估以减轻答案的分歧。具体而言，Pathcot用先验知识指导MLLM作为病理专家，并通过特定于领域的知识对图像进行全面分析。通过纳入专家的知识，Pathcot可以通过COT推理获得答案。此外，Pathcot结合了一个自我评估步骤，该步骤评估了MLLM直接产生的结果和通过COT得出的结果，最终确定了可靠的答案。 PATHMMU数据集的实验结果证明了我们方法对病理的视觉理解和推理的有效性。

Title: Data-driven Insights for Informed Decision-Making: Applying LSTM Networks for Robust Electricity Forecasting in Libya

• Authors: Asma Agaal, Mansour Essgaer, Hend M. Farkash, Zulaiha Ali Othman
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.01034
• Pdf URL: https://arxiv.org/pdf/2507.01034
• Copy Paste: [[2507.01034]] Data-driven Insights for Informed Decision-Making: Applying LSTM Networks for Robust Electricity Forecasting in Libya(https://arxiv.org/abs/2507.01034)
• Keywords: generation
• Abstract: Accurate electricity forecasting is crucial for grid stability and energy planning, especially in Benghazi, Libya, where frequent load shedding, generation deficits, and infrastructure limitations persist. This study proposes a data-driven approach to forecast electricity load, generation, and deficits for 2025 using historical data from 2019 (a year marked by instability) and 2023 (a more stable year). Multiple time series models were applied, including ARIMA, seasonal ARIMA, dynamic regression ARIMA, exponential smoothing, extreme gradient boosting, and Long Short-Term Memory (LSTM) neural networks. The dataset was enhanced through missing value imputation, outlier smoothing, and log transformation. Performance was assessed using mean squared error, root mean squared error, mean absolute error, and mean absolute percentage error. LSTM outperformed all other models, showing strong capabilities in modeling non-stationary and seasonal patterns. A key contribution of this work is an optimized LSTM framework that integrates exogenous factors such as temperature and humidity, offering robust performance in forecasting multiple electricity indicators. These results provide practical insights for policymakers and grid operators to enable proactive load management and resource planning in data-scarce, volatile regions.
• 摘要：准确的电力预测对于网格稳定性和能源规划至关重要，尤其是在利比亚班加西（Benghazi，Libya），那里频繁的负荷脱落，发电缺陷和基础设施限制持续存在。这项研究提出了一种数据驱动的方法，以使用2019年以来的历史数据（以不稳定为标志的一年）和2023年（更稳定的一年）来预测2025年的电力负载，产生和缺陷。应用了多个时间序列模型，包括Arima，季节性Arima，动态回归Arima，指数平滑，极端梯度增强和长期记忆（LSTM）神经网络。通过缺少价值插补，离群平滑和日志转换来增强数据集。使用均方误差，均方根误差，平均绝对误差和平均绝对百分比误差评估性能。 LSTM的表现优于所有其他模型，在建模非平稳和季节性模式时表现出强大的功能。这项工作的一个关键贡献是一个优化的LSTM框架，该框架整合了诸如温度和湿度之类的外源性因素，从而在预测多种电力指标方面提供了强大的性能。这些结果为政策制定者和网格操作员提供了实用的见解，以在数据筛选，挥发性区域实现主动的负载管理和资源计划。

Title: Research on Low-Latency Inference and Training Efficiency Optimization for Graph Neural Network and Large Language Model-Based Recommendation Systems

• Authors: Yushang Zhao, Haotian Lyu, Yike Peng, Aijia Sun, Feng Jiang, Xinyue Han
• Subjects: cs.LG, cs.AI, cs.PF
• Abstract URL: https://arxiv.org/abs/2507.01035
• Pdf URL: https://arxiv.org/pdf/2507.01035
• Copy Paste: [[2507.01035]] Research on Low-Latency Inference and Training Efficiency Optimization for Graph Neural Network and Large Language Model-Based Recommendation Systems(https://arxiv.org/abs/2507.01035)
• Keywords: generation
• Abstract: The incessant advent of online services demands high speed and efficient recommender systems (ReS) that can maintain real-time performance along with processing very complex user-item interactions. The present study, therefore, considers computational bottlenecks involved in hybrid Graph Neural Network (GNN) and Large Language Model (LLM)-based ReS with the aim optimizing their inference latency and training efficiency. An extensive methodology was used: hybrid GNN-LLM integrated architecture-optimization strategies(quantization, LoRA, distillation)-hardware acceleration (FPGA, DeepSpeed)-all under R 4.4.2. Experimental improvements were significant, with the optimal Hybrid + FPGA + DeepSpeed configuration reaching 13.6% more accuracy (NDCG@10: 0.75) at 40-60ms of latency, while LoRA brought down training time by 66% (3.8 hours) in comparison to the non-optimized baseline. Irrespective of domain, such as accuracy or efficiency, it can be established that hardware-software co-design and parameter-efficient tuning permit hybrid models to outperform GNN or LLM approaches implemented independently. It recommends the use of FPGA as well as LoRA for real-time deployment. Future work should involve federated learning along with advanced fusion architectures for better scalability and privacy preservation. Thus, this research marks the fundamental groundwork concerning next-generation ReS balancing low-latency response with cutting-edge personalization.
• 摘要：在线服务的不断出现需要高速和高效的推荐系统（RES），这些系统可以维持实时性能以及处理非常复杂的用户项目交互。因此，本研究考虑了涉及混合图神经网络（GNN）和大语言模型（LLM）的RES的计算瓶颈，其目标是优化其推理潜伏期和训练效率。使用了一种广泛的方法：混合GNN-LLM集成体系结构优化策略（量化，LORA，蒸馏） - 软件加速度（FPGA，DEEPSPEED） - 在R 4.4.2下。实验改进是显着的，最佳混合 + FPGA + DEEPSPEED配置在40-60ms的延迟下达到了13.6％的精度（NDCG@10：0.75），而Lora则将训练时间降低了66％（3.8小时）。无论域（例如准确性或效率），都可以确定，硬件软件的共同设计和参数有效的调整允许混合模型超过GNN或LLM方法独立实施的方法。它建议使用FPGA以及LORA进行实时部署。未来的工作应涉及联合学习以及先进的融合体系结构，以更好地可扩展性和隐私保护。因此，这项研究标志着关于下一代RES平衡低延迟响应与尖端个性化的基本基础。

Title: Sensing Cardiac Health Across Scenarios and Devices: A Multi-Modal Foundation Model Pretrained on Heterogeneous Data from 1.7 Million Individuals

• Authors: Xiao Gu, Wei Tang, Jinpei Han, Veer Sangha, Fenglin Liu, Shreyank N Gowda, Antonio H. Ribeiro, Patrick Schwab, Kim Branson, Lei Clifton, Antonio Luiz P. Ribeiro, Zhangdaihong Liu, David A. Clifton
• Subjects: cs.LG, cs.AI, eess.SP
• Abstract URL: https://arxiv.org/abs/2507.01045
• Pdf URL: https://arxiv.org/pdf/2507.01045
• Copy Paste: [[2507.01045]] Sensing Cardiac Health Across Scenarios and Devices: A Multi-Modal Foundation Model Pretrained on Heterogeneous Data from 1.7 Million Individuals(https://arxiv.org/abs/2507.01045)
• Keywords: generative
• Abstract: Cardiac biosignals, such as electrocardiograms (ECG) and photoplethysmograms (PPG), are of paramount importance for the diagnosis, prevention, and management of cardiovascular diseases, and have been extensively used in a variety of clinical tasks. Conventional deep learning approaches for analyzing these signals typically rely on homogeneous datasets and static bespoke models, limiting their robustness and generalizability across diverse clinical settings and acquisition protocols. In this study, we present a cardiac sensing foundation model (CSFM) that leverages advanced transformer architectures and a generative, masked pretraining strategy to learn unified representations from vast, heterogeneous health records. Our model is pretrained on an innovative multi-modal integration of data from multiple large-scale datasets (including MIMIC-III-WDB, MIMIC-IV-ECG, and CODE), comprising cardiac signals and the corresponding clinical or machine-generated text reports from approximately 1.7 million individuals. We demonstrate that the embeddings derived from our CSFM not only serve as effective feature extractors across diverse cardiac sensing scenarios, but also enable seamless transfer learning across varying input configurations and sensor modalities. Extensive evaluations across diagnostic tasks, demographic information recognition, vital sign measurement, clinical outcome prediction, and ECG question answering reveal that CSFM consistently outperforms traditional one-modal-one-task approaches. Notably, CSFM exhibits robust performance across multiple ECG lead configurations from standard 12-lead systems to single-lead setups, and in scenarios where only ECG, only PPG, or a combination thereof is available. These findings highlight the potential of CSFM as a versatile and scalable solution, for comprehensive cardiac monitoring.
• 摘要：心脏生物信号（例如心电图（ECG）和光摄取图（PPG））对于心血管疾病的诊断，预防和管理至关重要，并且已在各种临床任务中广泛使用。分析这些信号的常规深度学习方法通​​常依赖于均匀的数据集和静态定制模型，从而限制了它们在各种临床环境和获取协议中的鲁棒性和概括性。在这项研究中，我们提出了一种心脏传感基础模型（CSFM），该模型利用高级变压器体系结构和一种生成的，掩盖的预处理策略，以从广泛的，异构的健康记录中学习统一的表示。我们的模型是根据来自多个大规模数据集（包括模拟物-III-WDB，Mimic-IV-ECG和代码）的数据的创新多模式集成的预估计，其中包括心脏信号以及来自大约170万个人的相应临床或机器生成的文本报告。我们证明，源自CSFM的嵌入不仅是各种心脏传感场景中的有效提取器，而且还可以在不同的输入配置和传感器模态上实现无缝传输学习。跨诊断任务，人口统计信息识别，生命体征测量，临床结果预测和ECG问题回答的广泛评估表明，CSFM始终胜过传统的一式一式任务方法。值得注意的是，CSFM在从标准的12铅系统到单个铅设置的多个ECG铅配置中表现出强大的性能，并且在只有ECG，仅PPG或其组合的情况下。这些发现突出了CSFM作为多功能和可扩展解决方案的潜力，用于全面的心脏监测。

Title: Variational Digital Twins

• Authors: Logan A. Burnett, Umme Mahbuba Nabila, Majdi I. Radaideh
• Subjects: cs.LG, eess.SY
• Abstract URL: https://arxiv.org/abs/2507.01047
• Pdf URL: https://arxiv.org/pdf/2507.01047
• Copy Paste: [[2507.01047]] Variational Digital Twins(https://arxiv.org/abs/2507.01047)
• Keywords: generation
• Abstract: While digital twins (DT) hold promise for providing real-time insights into complex energy assets, much of the current literature either does not offer a clear framework for information exchange between the model and the asset, lacks key features needed for real-time implementation, or gives limited attention to model uncertainty. Here, we aim to solve these gaps by proposing a variational digital twin (VDT) framework that augments standard neural architectures with a single Bayesian output layer. This lightweight addition, along with a novel VDT updating algorithm, lets a twin update in seconds on commodity GPUs while producing calibrated uncertainty bounds that can inform experiment design, control algorithms, and model reliability. The VDT is evaluated on four energy-sector problems. For critical-heat-flux prediction, uncertainty-driven active learning reaches R2 = 0.98 using 47 % fewer experiments and one-third the training time of random sampling. A three-year renewable-generation twin maintains R2 > 0.95 for solar output and curbs error growth for volatile wind forecasts via monthly updates that process only one month of data at a time. A nuclear reactor transient cooldown twin reconstructs thermocouple signals with R2 > 0.99 and preserves accuracy after 50 % sensor loss, demonstrating robustness to degraded instrumentation. Finally, a physics-informed Li-ion battery twin, retrained after every ten discharges, lowers voltage mean-squared error by an order of magnitude relative to the best static model while adapting its credible intervals as the cell approaches end-of-life. These results demonstrate that combining modest Bayesian augmentation with efficient update schemes turns conventional surrogates into uncertainty-aware, data-efficient, and computationally tractable DTs, paving the way for dependable models across industrial and scientific energy systems.
• 摘要：尽管数字双胞胎（DT）有望提供对复杂能源资产的实时见解，但当前的许多文献都没有为模型和资产之间的信息交换提供明确的框架，缺乏实时实施所需的关键功能，或者对模型不确定性的关注有限。在这里，我们旨在通过提出一个变异数字双胞胎（VDT）框架来解决这些空白，该框架可以通过单个贝叶斯输出层增强标准神经体系结构。这种轻巧的添加以及一种新颖的VDT更新算法，可以在商品GPU的几秒钟内进行双重更新，同时产生校准的不确定性界限，这些范围可以为实验设计，控制算法和模型可靠性提供信息。 VDT对四个能源部门问题进行了评估。对于关键热量预测，使用少47％的实验和三分之一的训练时间，不确定性驱动的主动学习达到R2 = 0.98。三年的可再生生成双胞胎可通过每月更新一次，用于挥发性风向预测的太阳能输出的R2> 0.95，一次仅一次数据进行了一个月的数据。核反应堆瞬态冷却双胞胎双胞胎以R2> 0.99的形式重建热电偶信号，并在传感器丢失50％后保持精度，表明对降级仪器的稳健性。最后，每十个放电后进行物理信息的锂离子电池双胞胎，相对于最佳的静态模型，将均值误差降低，同时随着单元的接近寿命而适应其可靠的间隔。这些结果表明，将谦虚的贝叶斯增强与有效的更新方案相结合，将常规替代物变成不确定性的，数据效率和计算上可触及的DTS，为跨工业和科学能源系统的可靠模型铺平了道路。

Title: Loop2Net: Data-Driven Generation and Optimization of Airfoil CFD Meshes from Sparse Boundary Coordinates

• Authors: Lushun Fan, Yuqin Xia, Jun Li, Karl Jenkins
• Subjects: cs.LG, physics.flu-dyn
• Abstract URL: https://arxiv.org/abs/2507.01057
• Pdf URL: https://arxiv.org/pdf/2507.01057
• Copy Paste: [[2507.01057]] Loop2Net: Data-Driven Generation and Optimization of Airfoil CFD Meshes from Sparse Boundary Coordinates(https://arxiv.org/abs/2507.01057)
• Keywords: generation
• Abstract: In this study, an innovative intelligent optimization system for mesh quality is proposed, which is based on a deep convolutional neural network architecture, to achieve mesh generation and optimization. The core of the study is the Loop2Net generator and loss function, it predicts the mesh based on the given wing coordinates. And the model's performance is continuously optimised by two key loss functions during the training. Then discipline by adding penalties, the goal of mesh generation was finally reached.
• 摘要：在这项研究中，提出了针对网格质量的创新智能优化系统，该系统基于深度卷积神经网络体系结构，以实现网格的生成和优化。研究的核心是Loop2NET发生器和损耗函数，它根据给定的机翼坐标预测网格。在训练过程中，通过两个关键损失功能不断优化模型的性能。然后，通过增加惩罚来纪律，最终达到了网状生成的目标。

Title: Good Enough to Learn: LLM-based Anomaly Detection in ECU Logs without Reliable Labels

• Authors: Bogdan Bogdan, Arina Cazacu, Laura Vasilie
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.01077
• Pdf URL: https://arxiv.org/pdf/2507.01077
• Copy Paste: [[2507.01077]] Good Enough to Learn: LLM-based Anomaly Detection in ECU Logs without Reliable Labels(https://arxiv.org/abs/2507.01077)
• Keywords: generative
• Abstract: Anomaly detection often relies on supervised or clustering approaches, with limited success in specialized domains like automotive communication systems where scalable solutions are essential. We propose a novel decoder-only Large Language Model (LLM) to detect anomalies in Electronic Control Unit (ECU) communication logs. Our approach addresses two key challenges: the lack of LLMs tailored for ECU communication and the complexity of inconsistent ground truth data. By learning from UDP communication logs, we formulate anomaly detection simply as identifying deviations in time from normal behavior. We introduce an entropy regularization technique that increases model's uncertainty in known anomalies while maintaining consistency in similar scenarios. Our solution offers three novelties: a decoder-only anomaly detection architecture, a way to handle inconsistent labeling, and an adaptable LLM for different ECU communication use cases. By leveraging the generative capabilities of decoder-only models, we present a new technique that addresses the high cost and error-prone nature of manual labeling through a more scalable system that is able to learn from a minimal set of examples, while improving detection accuracy in complex communication environments.
• 摘要：异常检测通常依赖于监督或聚类方法，而在可扩展解决方案至关重要的汽车通信系统（例如汽车通信系统）中，成功的成功有限。我们提出了一种新型的仅解码器大型语言模型（LLM），以检测电子控制单元（ECU）通信日志中的异常。我们的方法解决了两个关键挑战：缺乏针对ECU通信的LLMS和不一致的地面真相数据的复杂性。通过从UDP通信日志中学习，我们简单地提出异常检测，以确定及时与正常行为的偏差。我们介绍了一种熵正则化技术，该技术增加了已知异常的模型不确定性，同时在类似情况下保持一致性。我们的解决方案提供了三个新颖性：仅解码器的异常检测体系结构，一种处理不一致的标签的方法以及用于不同ECU通信用例的适应性LLM。通过利用仅解码器模型的生成能力，我们提出了一种新技术，该技术通过更可扩展的系统来解决手动标记的高成本和错误性质，该系统能够从最小的示例中学习，同时提高复杂通信环境中的检测准确性。

Title: Geometry-aware 4D Video Generation for Robot Manipulation

• Authors: Zeyi Liu, Shuang Li, Eric Cousineau, Siyuan Feng, Benjamin Burchfiel, Shuran Song
• Subjects: cs.CV, cs.AI, cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2507.01099
• Pdf URL: https://arxiv.org/pdf/2507.01099
• Copy Paste: [[2507.01099]] Geometry-aware 4D Video Generation for Robot Manipulation(https://arxiv.org/abs/2507.01099)
• Keywords: generation
• Abstract: Understanding and predicting the dynamics of the physical world can enhance a robot's ability to plan and interact effectively in complex environments. While recent video generation models have shown strong potential in modeling dynamic scenes, generating videos that are both temporally coherent and geometrically consistent across camera views remains a significant challenge. To address this, we propose a 4D video generation model that enforces multi-view 3D consistency of videos by supervising the model with cross-view pointmap alignment during training. This geometric supervision enables the model to learn a shared 3D representation of the scene, allowing it to predict future video sequences from novel viewpoints based solely on the given RGB-D observations, without requiring camera poses as inputs. Compared to existing baselines, our method produces more visually stable and spatially aligned predictions across multiple simulated and real-world robotic datasets. We further show that the predicted 4D videos can be used to recover robot end-effector trajectories using an off-the-shelf 6DoF pose tracker, supporting robust robot manipulation and generalization to novel camera viewpoints.
• 摘要：理解和预测物理世界的动态可以增强机器人在复杂环境中有效计划和互动的能力。尽管最近的视频生成模型在建模动态场景方面表现出强大的潜力，但在摄像机视图上生成了既有时间一致又几何一致的视频仍然是一个重大挑战。为了解决这个问题，我们提出了一个4D视频生成模型，该模型通过在训练过程中使用跨视图指数对齐来监督模型，从而实现视频的多视图3D一致性。这种几何监督使模型能够学习场景的共享3D表示，从而使其能够仅基于给定的RGB-D观测值从新颖的观点来预测未来的视频序列，而无需将摄像头姿势作为输入。与现有基线相比，我们的方法在多个模拟和现实世界的机器人数据集中产生更稳定和空间对齐的预测。我们进一步表明，预测的4D视频可用于使用现成的6DOF姿势跟踪器恢复机器人最终效应器轨迹，从而支持强大的机器人操纵和对新型相机视点的概括。

Title: Spectral Manifold Harmonization for Graph Imbalanced Regression

• Authors: Brenda Nogueira, Gabe Gomes, Meng Jiang, Nitesh V. Chawla, Nuno Moniz
• Subjects: cs.LG, q-bio.MN
• Abstract URL: https://arxiv.org/abs/2507.01132
• Pdf URL: https://arxiv.org/pdf/2507.01132
• Copy Paste: [[2507.01132]] Spectral Manifold Harmonization for Graph Imbalanced Regression(https://arxiv.org/abs/2507.01132)
• Keywords: generation
• Abstract: Graph-structured data is ubiquitous in scientific domains, where models often face imbalanced learning settings. In imbalanced regression, domain preferences focus on specific target value ranges representing the most scientifically valuable cases; we observe a significant lack of research. In this paper, we present Spectral Manifold Harmonization (SMH), a novel approach for addressing this imbalanced regression challenge on graph-structured data by generating synthetic graph samples that preserve topological properties while focusing on often underrepresented target distribution regions. Conventional methods fail in this context because they either ignore graph topology in case generation or do not target specific domain ranges, resulting in models biased toward average target values. Experimental results demonstrate the potential of SMH on chemistry and drug discovery benchmark datasets, showing consistent improvements in predictive performance for target domain ranges.
• 摘要：图形结构的数据在科学领域无处不在，在科学领域，模型通常面临不平衡的学习设置。在不平衡的回归中，领域的偏好集中在代表最有价值案例的特定目标价值范围上；我们观察到严重缺乏研究。在本文中，我们提出了光谱歧管统一（SMH），这是一种通过生成保留拓扑特性的合成图样品，同时着重于通常代表性不足的目标分布区域，以解决图形结构数据不平衡的回归挑战的新方法。常规方法在这种情况下失败，因为它们要么忽略了案例生成的图形拓扑，要么不针对特定域范围，从而导致模型偏向平均目标值。实验结果表明，SMH在化学和药物发现基准数据集上的潜力，显示目标域范围的预测性能持续提高。

Title: Diffusion Explorer: Interactive Exploration of Diffusion Models

• Authors: Alec Helbling, Duen Horng Chau
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.01178
• Pdf URL: https://arxiv.org/pdf/2507.01178
• Copy Paste: [[2507.01178]] Diffusion Explorer: Interactive Exploration of Diffusion Models(https://arxiv.org/abs/2507.01178)
• Keywords: generation
• Abstract: Diffusion models have been central to the development of recent image, video, and even text generation systems. They posses striking geometric properties that can be faithfully portrayed in low-dimensional settings. However, existing resources for explaining diffusion either require an advanced theoretical foundation or focus on their neural network architectures rather than their rich geometric properties. We introduce Diffusion Explorer, an interactive tool to explain the geometric properties of diffusion models. Users can train 2D diffusion models in the browser and observe the temporal dynamics of their sampling process. Diffusion Explorer leverages interactive animation, which has been shown to be a powerful tool for making engaging visualizations of dynamic systems, making it well suited to explaining diffusion models which represent stochastic processes that evolve over time. Diffusion Explorer is open source and a live demo is available at this http URL.
• 摘要：扩散模型对于最近的图像，视频甚至文本生成系统的开发至关重要。他们具有惊人的几何特性，可以在低维设置中忠实地描绘。但是，现有的用于解释扩散的资源要么需要先进的理论基础，要么专注于其神经网络体系结构，而不是其丰富的几何特性。我们介绍了扩散资源管理器，这是一种交互式工具，可以解释扩散模型的几何特性。用户可以在浏览器中训练2D扩散模型，并观察其采样过程的时间动态。扩散探索器利用交互式动画，该动画已被证明是使动态系统引人入胜的功能强大的工具，使其非常适合解释代表随着时间时间发展的随机过程的扩散模型。扩散资源管理器是开源的，可以在此HTTP URL上获得实时演示。

Title: AIGVE-MACS: Unified Multi-Aspect Commenting and Scoring Model for AI-Generated Video Evaluation

• Authors: Xiao Liu, Jiawei Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.01255
• Pdf URL: https://arxiv.org/pdf/2507.01255
• Copy Paste: [[2507.01255]] AIGVE-MACS: Unified Multi-Aspect Commenting and Scoring Model for AI-Generated Video Evaluation(https://arxiv.org/abs/2507.01255)
• Keywords: generation
• Abstract: The rapid advancement of AI-generated video models has created a pressing need for robust and interpretable evaluation frameworks. Existing metrics are limited to producing numerical scores without explanatory comments, resulting in low interpretability and human evaluation alignment. To address those challenges, we introduce AIGVE-MACS, a unified model for AI-Generated Video Evaluation(AIGVE), which can provide not only numerical scores but also multi-aspect language comment feedback in evaluating these generated videos. Central to our approach is AIGVE-BENCH 2, a large-scale benchmark comprising 2,500 AI-generated videos and 22,500 human-annotated detailed comments and numerical scores across nine critical evaluation aspects. Leveraging AIGVE-BENCH 2, AIGVE-MACS incorporates recent Vision-Language Models with a novel token-wise weighted loss and a dynamic frame sampling strategy to better align with human evaluators. Comprehensive experiments across supervised and zero-shot benchmarks demonstrate that AIGVE-MACS achieves state-of-the-art performance in both scoring correlation and comment quality, significantly outperforming prior baselines including GPT-4o and VideoScore. In addition, we further showcase a multi-agent refinement framework where feedback from AIGVE-MACS drives iterative improvements in video generation, leading to 53.5% quality enhancement. This work establishes a new paradigm for comprehensive, human-aligned evaluation of AI-generated videos. We release the AIGVE-BENCH 2 and AIGVE-MACS at this https URL.
• 摘要：AI生成的视频模型的快速发展使人们对强大且可解释的评估框架迫切需要。现有指标仅限于产生数值得分而没有解释性评论，从而导致低解释性和人类评估一致性。为了应对这些挑战，我们介绍了AIGVE-MAC，这是AI生成的视频评估（AIGVE）的统一模型，它不仅可以提供数值得分，还可以提供评估这些生成的视频的多相关语言评论反馈。我们方法的核心是Aigve Bench 2，这是一个大规模的基准测试，其中包括2500个AI生成的视频以及22,500个跨9个批判性评估方面的人类注销的详细评论和数值分数。 AIGVE-MAC利用AIGVE BENCH 2，将最新的视觉模型与新颖的令牌加权损失和动态框架采样策略结合在一起，以更好地与人类评估者保持一致。跨受监督和零射基准测试的全面实验表明，AIGVE-MAC在评分相关性和评论质量中都取得了最先进的性能，从而超过了包括GPT-4O和Videoscore在内的先前基线。此外，我们进一步展示了一个多代理改进框架，其中AIGVE-MACS的反馈可以推动视频生成的迭代改进，从而导致53.5％的质量增强。这项工作建立了一个新的范式，用于对AI生成的视频进行全面的，人类的一致评估。我们在此HTTPS URL上释放AIGVE基座2和AIGVE-MAC。

Title: Frequency Domain-Based Diffusion Model for Unpaired Image Dehazing

• Authors: Chengxu Liu, Lu Qi, Jinshan Pan, Xueming Qian, Ming-Hsuan Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.01275
• Pdf URL: https://arxiv.org/pdf/2507.01275
• Copy Paste: [[2507.01275]] Frequency Domain-Based Diffusion Model for Unpaired Image Dehazing(https://arxiv.org/abs/2507.01275)
• Keywords: generative
• Abstract: Unpaired image dehazing has attracted increasing attention due to its flexible data requirements during model training. Dominant methods based on contrastive learning not only introduce haze-unrelated content information, but also ignore haze-specific properties in the frequency domain (\ie,~haze-related degradation is mainly manifested in the amplitude spectrum). To address these issues, we propose a novel frequency domain-based diffusion model, named \ours, for fully exploiting the beneficial knowledge in unpaired clear data. In particular, inspired by the strong generative ability shown by Diffusion Models (DMs), we tackle the dehazing task from the perspective of frequency domain reconstruction and perform the DMs to yield the amplitude spectrum consistent with the distribution of clear images. To implement it, we propose an Amplitude Residual Encoder (ARE) to extract the amplitude residuals, which effectively compensates for the amplitude gap from the hazy to clear domains, as well as provide supervision for the DMs training. In addition, we propose a Phase Correction Module (PCM) to eliminate artifacts by further refining the phase spectrum during dehazing with a simple attention mechanism. Experimental results demonstrate that our \ours outperforms other state-of-the-art methods on both synthetic and real-world datasets.
• 摘要：由于模型训练期间的灵活数据要求，未配对的图像除尘吸引了越来越多的关注。基于对比度学习的主要方法不仅引入了与雾化无关的内容信息，而且还忽略了频域中的雾霾特异性特性（\ ie，〜与Haze相关的降解主要在振幅光谱中表现出来）。为了解决这些问题，我们提出了一个新颖的基于频域的扩散模型，名为\ outs，以充分利用未配对的清晰数据中的有益知识。特别是，受到扩散模型（DMS）表现出的强生成能力的启发，我们从频域重建的角度解决了飞行任务，并执行DMS以产生与透明图像的分布一致的振幅光谱。为了实施它，我们提出了一个振幅残差编码器（是）提取振幅残留物，从而有效地补偿了从朦胧到清除域的振幅差距，并为DMS培训提供了监督。此外，我们提出了一个相校正模块（PCM），以通过简单的注意机制进一步精炼飞行过程中的相光谱来消除伪影。实验结果表明，我们的\我们的\在合成数据集和现实世界中的其他最先进方法的表现。

Title: DiffusionLight-Turbo: Accelerated Light Probes for Free via Single-Pass Chrome Ball Inpainting

• Authors: Worameth Chinchuthakun, Pakkapon Phongthawee, Amit Raj, Varun Jampani, Pramook Khungurn, Supasorn Suwajanakorn
• Subjects: cs.CV, cs.GR, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.01305
• Pdf URL: https://arxiv.org/pdf/2507.01305
• Copy Paste: [[2507.01305]] DiffusionLight-Turbo: Accelerated Light Probes for Free via Single-Pass Chrome Ball Inpainting(https://arxiv.org/abs/2507.01305)
• Keywords: generation
• Abstract: We introduce a simple yet effective technique for estimating lighting from a single low-dynamic-range (LDR) image by reframing the task as a chrome ball inpainting problem. This approach leverages a pre-trained diffusion model, Stable Diffusion XL, to overcome the generalization failures of existing methods that rely on limited HDR panorama datasets. While conceptually simple, the task remains challenging because diffusion models often insert incorrect or inconsistent content and cannot readily generate chrome balls in HDR format. Our analysis reveals that the inpainting process is highly sensitive to the initial noise in the diffusion process, occasionally resulting in unrealistic outputs. To address this, we first introduce DiffusionLight, which uses iterative inpainting to compute a median chrome ball from multiple outputs to serve as a stable, low-frequency lighting prior that guides the generation of a high-quality final result. To generate high-dynamic-range (HDR) light probes, an Exposure LoRA is fine-tuned to create LDR images at multiple exposure values, which are then merged. While effective, DiffusionLight is time-intensive, requiring approximately 30 minutes per estimation. To reduce this overhead, we introduce DiffusionLight-Turbo, which reduces the runtime to about 30 seconds with minimal quality loss. This 60x speedup is achieved by training a Turbo LoRA to directly predict the averaged chrome balls from the iterative process. Inference is further streamlined into a single denoising pass using a LoRA swapping technique. Experimental results that show our method produces convincing light estimates across diverse settings and demonstrates superior generalization to in-the-wild scenarios. Our code is available at this https URL
• 摘要：我们引入了一种简单而有效的技术，用于通过将任务重新构架为镀铬球涂上问题，从而从单个低动力范围（LDR）图像估算照明。这种方法利用了预先训练的扩散模型稳定的扩散XL，以克服依赖于有限的HDR Panorama数据集的现有方法的泛化失败。尽管从概念上讲简单，但任务仍然具有挑战性，因为扩散模型通常插入不正确或不一致的内容，并且无法轻易以HDR格式产生镀铬球。我们的分析表明，在扩散过程中的初始噪声高度敏感，有时会导致不切实际的输出。为了解决这个问题，我们首先引入扩散光，它使用迭代插图来计算来自多个输出的中间镀铬球，以作为稳定的低频照明，这是指引导产生高质量的最终结果。为了生成高动力范围（HDR）的光探针，对曝光lora进行微调以在多个曝光值下创建LDR图像，然后将其合并。虽然有效，但扩散光是时间密集型，每个估计需要大约30分钟。为了减少这个开销，我们引入了扩散光 - 涡轮增压，这将运行时间降低到大约30秒，而质量损失最少。通过训练涡轮洛拉直接预测迭代过程中的平均镀铬球来实现这一60倍加速。使用洛拉交换技术将推论进一步简化为单个denoising通过。实验结果表明我们的方法会在各种环境中产生令人信服的光估计，并证明对野外场景的概括。我们的代码可在此HTTPS URL上找到

Title: Efficient Kilometer-Scale Precipitation Downscaling with Conditional Wavelet Diffusion

• Authors: Chugang Yi, Minghan Yu, Weikang Qian, Yixin Wen, Haizhao Yang
• Subjects: cs.LG, physics.ao-ph
• Abstract URL: https://arxiv.org/abs/2507.01354
• Pdf URL: https://arxiv.org/pdf/2507.01354
• Copy Paste: [[2507.01354]] Efficient Kilometer-Scale Precipitation Downscaling with Conditional Wavelet Diffusion(https://arxiv.org/abs/2507.01354)
• Keywords: super-resolution, generative
• Abstract: Effective hydrological modeling and extreme weather analysis demand precipitation data at a kilometer-scale resolution, which is significantly finer than the 10 km scale offered by standard global products like IMERG. To address this, we propose the Wavelet Diffusion Model (WDM), a generative framework that achieves 10x spatial super-resolution (downscaling to 1 km) and delivers a 9x inference speedup over pixel-based diffusion models. WDM is a conditional diffusion model that learns the learns the complex structure of precipitation from MRMS radar data directly in the wavelet domain. By focusing on high-frequency wavelet coefficients, it generates exceptionally realistic and detailed 1-km precipitation fields. This wavelet-based approach produces visually superior results with fewer artifacts than pixel-space models, and delivers a significant gains in sampling efficiency. Our results demonstrate that WDM provides a robust solution to the dual challenges of accuracy and speed in geoscience super-resolution, paving the way for more reliable hydrological forecasts.
• 摘要：有效的水文建模和极端天气分析要求以公里尺度的分辨率进行降水数据，这比标准全球产品（如Imerg）所提供的10 km量表明显细得多。为了解决这个问题，我们提出了小波扩散模型（WDM），这是一个实现10倍空间超分辨率（降尺度至1 km）的生成框架，并在基于像素的扩散模型上提供了9倍的推理加速。 WDM是一个有条件的扩散模型，它可以从小波域中直接从MRMS雷达数据中学习降水的复杂结构。通过专注于高频小波系数，它产生了异常现实且详细的1公里降水场。这种基于小波的方法在视觉上产生的效果比像素空间模型少，并且在采样效率方面带来了显着提高。我们的结果表明，WDM为地球科学超级分辨率的准确性和速度的双重挑战提供了强有力的解决方案，为更可靠的水文预测铺平了道路。

Title: 3D Gaussian Splatting Driven Multi-View Robust Physical Adversarial Camouflage Generation

• Authors: Tianrui Lou, Xiaojun Jia, Siyuan Liang, Jiawei Liang, Ming Zhang, Yanjun Xiao, Xiaochun Cao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.01367
• Pdf URL: https://arxiv.org/pdf/2507.01367
• Copy Paste: [[2507.01367]] 3D Gaussian Splatting Driven Multi-View Robust Physical Adversarial Camouflage Generation(https://arxiv.org/abs/2507.01367)
• Keywords: generation
• Abstract: Physical adversarial attack methods expose the vulnerabilities of deep neural networks and pose a significant threat to safety-critical scenarios such as autonomous driving. Camouflage-based physical attack is a more promising approach compared to the patch-based attack, offering stronger adversarial effectiveness in complex physical environments. However, most prior work relies on mesh priors of the target object and virtual environments constructed by simulators, which are time-consuming to obtain and inevitably differ from the real world. Moreover, due to the limitations of the backgrounds in training images, previous methods often fail to produce multi-view robust adversarial camouflage and tend to fall into sub-optimal solutions. Due to these reasons, prior work lacks adversarial effectiveness and robustness across diverse viewpoints and physical environments. We propose a physical attack framework based on 3D Gaussian Splatting (3DGS), named PGA, which provides rapid and precise reconstruction with few images, along with photo-realistic rendering capabilities. Our framework further enhances cross-view robustness and adversarial effectiveness by preventing mutual and self-occlusion among Gaussians and employing a min-max optimization approach that adjusts the imaging background of each viewpoint, helping the algorithm filter out non-robust adversarial features. Extensive experiments validate the effectiveness and superiority of PGA. Our code is available at:this https URL.
• 摘要：身体对抗攻击方法暴露了深神经网络的脆弱性，并对诸如自动驾驶等安全至关重要的情况构成了重大威胁。与基于斑块的攻击相比，基于伪装的物理攻击是一种更有前途的方法，在复杂的物理环境中具有更强的对抗性效果。但是，大多数先前的工作都依赖于模拟器构建的目标对象和虚拟环境的网格先验，这些环境既耗时，又与现实世界无可避免地有所不同。此外，由于训练图像中背景的局限性，以前的方法通常无法产生多视图的强大对抗性伪装，并且倾向于属于优化的解决方案。由于这些原因，先前的工作缺乏各种观点和物理环境中的对抗性和鲁棒性。我们提出了一个基于3D高斯碎片（3DG）的物理攻击框架，名为PGA，该框架提供了几乎没有图像的快速而精确的重建，以及照片真实的渲染能力。我们的框架进一步增强了跨视图的鲁棒性和对抗性的有效性，通过防止高斯人之间的相互和自我辨别，并采用最小 - 最大优化方法来调整每个观点的成像背景，从而帮助算法过滤掉非体做的对抗特征。广泛的实验验证了PGA的有效性和优势。我们的代码可用：此HTTPS URL。

Title: Activation Reward Models for Few-Shot Model Alignment

• Authors: Tianning Chai, Chancharik Mitra, Brandon Huang, Gautam Rajendrakumar Gare, Zhiqiu Lin, Assaf Arbelle, Leonid Karlinsky, Rogerio Feris, Trevor Darrell, Deva Ramanan, Roei Herzig
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.01368
• Pdf URL: https://arxiv.org/pdf/2507.01368
• Copy Paste: [[2507.01368]] Activation Reward Models for Few-Shot Model Alignment(https://arxiv.org/abs/2507.01368)
• Keywords: generative
• Abstract: Aligning Large Language Models (LLMs) and Large Multimodal Models (LMMs) to human preferences is a central challenge in improving the quality of the models' generative outputs for real-world applications. A common approach is to use reward modeling to encode preferences, enabling alignment via post-training using reinforcement learning. However, traditional reward modeling is not easily adaptable to new preferences because it requires a separate reward model, commonly trained on large preference datasets. To address this, we introduce Activation Reward Models (Activation RMs) -- a novel few-shot reward modeling method that leverages activation steering to construct well-aligned reward signals using minimal supervision and no additional model finetuning. Activation RMs outperform existing few-shot reward modeling approaches such as LLM-as-a-judge with in-context learning, voting-based scoring, and token probability scoring on standard reward modeling benchmarks. Furthermore, we demonstrate the effectiveness of Activation RMs in mitigating reward hacking behaviors, highlighting their utility for safety-critical applications. Toward this end, we propose PreferenceHack, a novel few-shot setting benchmark, the first to test reward models on reward hacking in a paired preference format. Finally, we show that Activation RM achieves state-of-the-art performance on this benchmark, surpassing even GPT-4o.
• 摘要：将大型语言模型（LLMS）和大型多模型模型（LMM）与人类偏好保持一致，这是提高模型生成性输出质量的核心挑战。一种常见的方法是使用奖励建模来编码偏好，从而通过增强学习通过后培训来对齐。但是，传统的奖励建模不容易适应新的偏好，因为它需要单独的奖励模型，通常在大型偏好数据集上接受培训。为了解决这个问题，我们介绍了激活奖励模型（激活RMS） - 一种新颖的几声奖励建模方法，利用激活转向来利用最小的监督构建良好的奖励信号，没有其他模型填充。激活RMS的表现优于现有的几季奖励建模方法，例如使用c-as-a-a-a-a-a-a-a-a-a-as-a-a-a-Gudge，基于投票的评分以及对标准奖励建模基准测试的概率评分。此外，我们证明了激活RMS在减轻奖励黑客行为方面的有效性，从而强调了其对安全至关重要的应用的效用。为此，我们提出了prefereHACK，这是一种新颖的几弹性设定基准，第一个以配对的偏好格式测试奖励黑客奖励模型的奖励模型。最后，我们表明，激活RM在此基准测试中实现最先进的性能，甚至超过GPT-4O。

Title: FixTalk: Taming Identity Leakage for High-Quality Talking Head Generation in Extreme Cases

• Authors: Shuai Tan, Bill Gong, Bin Ji, Ye Pan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.01390
• Pdf URL: https://arxiv.org/pdf/2507.01390
• Copy Paste: [[2507.01390]] FixTalk: Taming Identity Leakage for High-Quality Talking Head Generation in Extreme Cases(https://arxiv.org/abs/2507.01390)
• Keywords: generation
• Abstract: Talking head generation is gaining significant importance across various domains, with a growing demand for high-quality rendering. However, existing methods often suffer from identity leakage (IL) and rendering artifacts (RA), particularly in extreme cases. Through an in-depth analysis of previous approaches, we identify two key insights: (1) IL arises from identity information embedded within motion features, and (2) this identity information can be leveraged to address RA. Building on these findings, this paper introduces FixTalk, a novel framework designed to simultaneously resolve both issues for high-quality talking head generation. Firstly, we propose an Enhanced Motion Indicator (EMI) to effectively decouple identity information from motion features, mitigating the impact of IL on generated talking heads. To address RA, we introduce an Enhanced Detail Indicator (EDI), which utilizes the leaked identity information to supplement missing details, thus fixing the artifacts. Extensive experiments demonstrate that FixTalk effectively mitigates IL and RA, achieving superior performance compared to state-of-the-art methods.
• 摘要：随着对高质量渲染的需求的增长，会说话的产量在各个领域都具有重要的重视。但是，现有方法通常会遭受身份泄漏（IL）和渲染工件（RA）的困扰，尤其是在极端情况下。通过对先前方法的深入分析，我们确定了两个关键见解：（1）IL来自运动特征中嵌入的身份信息，并且（2）可以利用此身份信息来解决RA。在这些发现的基础上，本文介绍了Fixtalk，这是一个新颖的框架，旨在同时解决这两个问题，以供高质量的说话型脑海。首先，我们提出了一个增强的运动指标（EMI），以有效地将身份信息从运动特征解脱出来，从而减轻IL对产生的说话头的影响。为了解决RA，我们介绍了一个增强的细节指标（EDI），该指标利用泄漏的身份信息来补充缺失的细节，从而修复了工件。广泛的实验表明，与最先进的方法相比，夹具有效地减轻了IL和RA，实现了卓越的性能。

Title: CaptionSmiths: Flexibly Controlling Language Pattern in Image Captioning

• Authors: Kuniaki Saito, Donghyun Kim, Kwanyong Park, Atsushi Hashimoto, Yoshitaka Ushiku
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.01409
• Pdf URL: https://arxiv.org/pdf/2507.01409
• Copy Paste: [[2507.01409]] CaptionSmiths: Flexibly Controlling Language Pattern in Image Captioning(https://arxiv.org/abs/2507.01409)
• Keywords: generative
• Abstract: An image captioning model flexibly switching its language pattern, e.g., descriptiveness and length, should be useful since it can be applied to diverse applications. However, despite the dramatic improvement in generative vision-language models, fine-grained control over the properties of generated captions is not easy due to two reasons: (i) existing models are not given the properties as a condition during training and (ii) existing models cannot smoothly transition its language pattern from one state to the other. Given this challenge, we propose a new approach, CaptionSmiths, to acquire a single captioning model that can handle diverse language patterns. First, our approach quantifies three properties of each caption, length, descriptiveness, and uniqueness of a word, as continuous scalar values, without human annotation. Given the values, we represent the conditioning via interpolation between two endpoint vectors corresponding to the extreme states, e.g., one for a very short caption and one for a very long caption. Empirical results demonstrate that the resulting model can smoothly change the properties of the output captions and show higher lexical alignment than baselines. For instance, CaptionSmiths reduces the error in controlling caption length by 506\% despite better lexical alignment. Code will be available on this https URL.
• 摘要：图像字幕模型可以灵活地切换其语言模式，例如描述性和长度，因为它可以应用于不同的应用程序。但是，尽管生成视觉模型的改善有了显着的改善，但由于两个原因，对生成字幕的属性的细粒度控制并不容易：（i）现有模型在培训期间没有作为条件作为条件，并且（ii）现有模型无法将其语言模式从一种状态顺畅地将其语言模式顺畅地转换为另一种状态。考虑到这一挑战，我们提出了一种新方法，字幕史密斯，以获取可以处理多种语言模式的单个字幕模型。首先，我们的方法量化了每个标题的三个特性，长度，描述性和单词的唯一性，作为连续标量值，而没有人类注释。鉴于值，我们表示与极端状态相对应的两个端点向量之间的插值调节，例如一个非常短的字幕，一个用于很长的标题。经验结果表明，所得模型可以平稳地更改输出字幕的性能，并显示出比基线更高的词汇比对。例如，尽管词汇比对更好，字幕史密斯将控制字幕长度的误差降低了506 \％。代码将在此HTTPS URL上可用。

Title: DocShaDiffusion: Diffusion Model in Latent Space for Document Image Shadow Removal

• Authors: Wenjie Liu, Bingshu Wang, Ze Wang, C.L. Philip Chen
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.01422
• Pdf URL: https://arxiv.org/pdf/2507.01422
• Copy Paste: [[2507.01422]] DocShaDiffusion: Diffusion Model in Latent Space for Document Image Shadow Removal(https://arxiv.org/abs/2507.01422)
• Keywords: generation
• Abstract: Document shadow removal is a crucial task in the field of document image enhancement. However, existing methods tend to remove shadows with constant color background and ignore color shadows. In this paper, we first design a diffusion model in latent space for document image shadow removal, called DocShaDiffusion. It translates shadow images from pixel space to latent space, enabling the model to more easily capture essential features. To address the issue of color shadows, we design a shadow soft-mask generation module (SSGM). It is able to produce accurate shadow mask and add noise into shadow regions specially. Guided by the shadow mask, a shadow mask-aware guided diffusion module (SMGDM) is proposed to remove shadows from document images by supervising the diffusion and denoising process. We also propose a shadow-robust perceptual feature loss to preserve details and structures in document images. Moreover, we develop a large-scale synthetic document color shadow removal dataset (SDCSRD). It simulates the distribution of realistic color shadows and provides powerful supports for the training of models. Experiments on three public datasets validate the proposed method's superiority over state-of-the-art. Our code and dataset will be publicly available.
• 摘要：文档删除是文档图像增强领域的至关重要任务。但是，现有方法倾向于删除具有恒定颜色背景的阴影，而忽略了颜色阴影。在本文中，我们首先在潜在空间中设计一个扩散模型，用于文档图像删除，称为DocShadiffusion。它将阴影图像从像素空间转换为潜在空间，从而使模型更容易捕获基本功能。为了解决颜色阴影的问题，我们设计了一个阴影软遮罩生成模块（SSGM）。它能够产生准确的影子面膜，并特别在阴影区域中添加噪音。在阴影面罩的指导下，提议通过监督扩散和降解过程来从文档图像中删除阴影。我们还提出了阴影般的感知特征损失，以保留文档图像中的细节和结构。此外，我们开发了一个大规模的合成文档颜色阴影去除数据集（SDCSRD）。它模拟了逼真的颜色阴影的分布，并为训练模型提供了强大的支持。在三个公共数据集上进行的实验验证了所提出的方法优于最先进的方法。我们的代码和数据集将公开可用。

Title: DiffMark: Diffusion-based Robust Watermark Against Deepfakes

• Authors: Chen Sun, Haiyang Sun, Zhiqing Guo, Yunfeng Diao, Liejun Wang, Dan Ma, Gaobo Yang, Keqin Li
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2507.01428
• Pdf URL: https://arxiv.org/pdf/2507.01428
• Copy Paste: [[2507.01428]] DiffMark: Diffusion-based Robust Watermark Against Deepfakes(https://arxiv.org/abs/2507.01428)
• Keywords: generation
• Abstract: Deepfakes pose significant security and privacy threats through malicious facial manipulations. While robust watermarking can aid in authenticity verification and source tracking, existing methods often lack the sufficient robustness against Deepfake manipulations. Diffusion models have demonstrated remarkable performance in image generation, enabling the seamless fusion of watermark with image during generation. In this study, we propose a novel robust watermarking framework based on diffusion model, called DiffMark. By modifying the training and sampling scheme, we take the facial image and watermark as conditions to guide the diffusion model to progressively denoise and generate corresponding watermarked image. In the construction of facial condition, we weight the facial image by a timestep-dependent factor that gradually reduces the guidance intensity with the decrease of noise, thus better adapting to the sampling process of diffusion model. To achieve the fusion of watermark condition, we introduce a cross information fusion (CIF) module that leverages a learnable embedding table to adaptively extract watermark features and integrates them with image features via cross-attention. To enhance the robustness of the watermark against Deepfake manipulations, we integrate a frozen autoencoder during training phase to simulate Deepfake manipulations. Additionally, we introduce Deepfake-resistant guidance that employs specific Deepfake model to adversarially guide the diffusion sampling process to generate more robust watermarked images. Experimental results demonstrate the effectiveness of the proposed DiffMark on typical Deepfakes. Our code will be available at this https URL.
• 摘要：通过恶意的面部操纵，深击构成了巨大的安全性和隐私威胁。尽管强大的水印可以帮助进行真实性验证和源跟踪，但现有方法通常缺乏针对深泡操作的足够鲁棒性。扩散模型在图像产生中表现出了显着的性能，从而使水印与图像的无缝融合在一起。在这项研究中，我们提出了一个基于扩散模型的新型鲁棒水印框架，称为diffmark。通过修改训练和采样方案，我们将面部图像和水印作为指导扩散模型逐渐降解并生成相应的水印图像。在面部条件的构建中，我们通过依赖时间段的因子加重面部图像，该因子逐渐通过噪声减少而逐渐降低引导强度，从而更好地适应扩散模型的采样过程。为了达到水印条件的融合，我们引入了一个跨信息融合（CIF）模块，该模块利用可学习的嵌入式表来适应水印特征，并通过交叉注意将其与图像特征集成在一起。为了增强水印对DeepFake操作的鲁棒性，我们在训练阶段集成了冷冻的自动编码器，以模拟深击操作。此外，我们引入了耐深泡沫的指导，该指导采用特定的深层模型来对抗指导扩散抽样过程，以生成更强大的水印图像。实验结果证明了所提出的差异对典型深击的有效性。我们的代码将在此HTTPS URL上可用。

Title: Representation Entanglement for Generation:Training Diffusion Transformers Is Much Easier Than You Think

• Authors: Ge Wu, Shen Zhang, Ruijing Shi, Shanghua Gao, Zhenyuan Chen, Lei Wang, Zhaowei Chen, Hongcheng Gao, Yao Tang, Jian Yang, Ming-Ming Cheng, Xiang Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.01467
• Pdf URL: https://arxiv.org/pdf/2507.01467
• Copy Paste: [[2507.01467]] Representation Entanglement for Generation:Training Diffusion Transformers Is Much Easier Than You Think(https://arxiv.org/abs/2507.01467)
• Keywords: generation
• Abstract: REPA and its variants effectively mitigate training challenges in diffusion models by incorporating external visual representations from pretrained models, through alignment between the noisy hidden projections of denoising networks and foundational clean image representations. We argue that the external alignment, which is absent during the entire denoising inference process, falls short of fully harnessing the potential of discriminative representations. In this work, we propose a straightforward method called Representation Entanglement for Generation (REG), which entangles low-level image latents with a single high-level class token from pretrained foundation models for denoising. REG acquires the capability to produce coherent image-class pairs directly from pure noise, substantially improving both generation quality and training efficiency. This is accomplished with negligible additional inference overhead, requiring only one single additional token for denoising (<0.5\% increase in FLOPs and latency). The inference process concurrently reconstructs both image latents and their corresponding global semantics, where the acquired semantic knowledge actively guides and enhances the image generation process. On ImageNet 256$\times$256, SiT-XL/2 + REG demonstrates remarkable convergence acceleration, achieving $\textbf{63}\times$ and $\textbf{23}\times$ faster training than SiT-XL/2 and SiT-XL/2 + REPA, respectively. More impressively, SiT-L/2 + REG trained for merely 400K iterations outperforms SiT-XL/2 + REPA trained for 4M iterations ($\textbf{10}\times$ longer). Code is available at: this https URL.
• 摘要：REPA及其变体通过结合预审计模型的外部视觉表示，通过在嘈杂的隐藏隐藏式预测网络和基础清洁图像表示之间进行对齐，从而有效地减轻了扩散模型中的训练挑战。我们认为，在整个剥离推理过程中不存在的外部对齐方式没有充分利用判别代表的潜力。在这项工作中，我们提出了一种简单的方法，称为“代表纠缠（REG）”（REG），该方法将低级图像潜在的潜在的纠缠与验证的基础模型的单个高级类别令牌纠缠在一起。 Reg从纯噪声直接获得了产生连贯的图像级对的能力，从而显着提高了发电质量和训练效率。这是通过可忽略的额外推理开销来实现的，只需要一个额外的令牌才能进行降级（拖鞋和潜伏期增加了0.5％）。推理过程同时重建了图像潜在及其相应的全局语义，在该语义上，获得的语义知识会积极指导并增强图像生成过程。在Imagenet 256 $ \ times $ 256上，SIT-XL/2 + REG展示了显着的融合加速度，达到$ \ textbf {63} \ times $和$ \ textbf {23} \ textbf {23} \ times $ $ $ $ $比SIT-XL/2和SIT-XL/2和SIT-XL/2 + REPA分别更快。更令人印象深刻的是，SIT-L/2 + REG仅针对400K迭代而受过训练的SIT-XL/2 + REPA经过4M迭代培训（$ \ textbf {10} \ times $更长）。代码可用：此HTTPS URL。

Title: AVC-DPO: Aligned Video Captioning via Direct Preference Optimization

• Authors: Jiyang Tang, Hengyi Li, Yifan Du, Wayne Xin Zhao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.01492
• Pdf URL: https://arxiv.org/pdf/2507.01492
• Copy Paste: [[2507.01492]] AVC-DPO: Aligned Video Captioning via Direct Preference Optimization(https://arxiv.org/abs/2507.01492)
• Keywords: generation
• Abstract: Although video multimodal large language models (video MLLMs) have achieved substantial progress in video captioning tasks, it remains challenging to adjust the focal emphasis of video captions according to human preferences. To address this limitation, we propose Aligned Video Captioning via Direct Preference Optimization (AVC-DPO), a post-training framework designed to enhance captioning capabilities in video MLLMs through preference alignment. Our approach designs enhanced prompts that specifically target temporal dynamics and spatial information-two key factors that humans care about when watching a video-thereby incorporating human-centric preferences. AVC-DPO leverages the same foundation model's caption generation responses under varied prompt conditions to conduct preference-aware training and caption alignment. Using this framework, we have achieved exceptional performance in the LOVE@CVPR'25 Workshop Track 1A: Video Detailed Captioning Challenge, achieving first place on the Video Detailed Captioning (VDC) benchmark according to the VDCSCORE evaluation metric.
• 摘要：尽管视频多模式大型语言模型（视频MLLM）在视频字幕任务中取得了重大进展，但根据人类的偏好调整视频字幕的重点仍然很具有挑战性。为了解决此限制，我们建议通过直接偏好优化（AVC-DPO）进行对齐视频字幕，这是一个训练后框架，旨在通过偏好对齐来增强视频MLLM中的字幕功能。我们的方法设计增强了提示，这些提示专门针对时间动态和空间信息两个关键因素，即人们在观看视频中关心的关键因素 - 以结合以人为中心的偏好的视频。 AVC-DPO利用相同的基础模型在各种及时条件下的字幕产生响应来进行偏好感知的培训和字幕对准。使用此框架，我们在Love@cvpr'25车间轨道1A中取得了出色的表现：视频详细的字幕挑战，在视频详细字幕（VDC）基准测试中获得了第一名，该标题（VDC）基准根据VDCSCORE评估公制。

Title: Loss Functions in Diffusion Models: A Comparative Study

• Authors: Dibyanshu Kumar, Philipp Vaeth, Magda Gregorová
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.01516
• Pdf URL: https://arxiv.org/pdf/2507.01516
• Copy Paste: [[2507.01516]] Loss Functions in Diffusion Models: A Comparative Study(https://arxiv.org/abs/2507.01516)
• Keywords: generative
• Abstract: Diffusion models have emerged as powerful generative models, inspiring extensive research into their underlying mechanisms. One of the key questions in this area is the loss functions these models shall train with. Multiple formulations have been introduced in the literature over the past several years with some links and some critical differences stemming from various initial considerations. In this paper, we explore the different target objectives and corresponding loss functions in detail. We present a systematic overview of their relationships, unifying them under the framework of the variational lower bound objective. We complement this theoretical analysis with an empirical study providing insights into the conditions under which these objectives diverge in performance and the underlying factors contributing to such deviations. Additionally, we evaluate how the choice of objective impacts the model ability to achieve specific goals, such as generating high-quality samples or accurately estimating likelihoods. This study offers a unified understanding of loss functions in diffusion models, contributing to more efficient and goal-oriented model designs in future research.
• 摘要：扩散模型已成为强大的生成模型，激发了对其潜在机制的广泛研究。该领域的关键问题之一是这些模型应训练的损失功能。在过去的几年中，文献中已经引入了多种表述，并具有一些链接以及各种初始考虑的一些关键差异。在本文中，我们详细探讨了不同的目标目标和相应的损失功能。我们介绍了他们的关系的系统概述，将它们统一在变化下限目标的框架下。我们通过一项经验研究对这种理论分析进行补充，该研究提供了有关这些目标在绩效方面差异和导致此类偏差的基本因素的条件的见解。此外，我们评估了目标的选择如何影响模型能力实现特定目标的能力，例如生成高质量的样本或准确估计了可能性。这项研究提供了对扩散模型中损失功能的统一理解，从而有助于将来的研究中更高效，面向目标的模型设计。

Title: Chargax: A JAX Accelerated EV Charging Simulator

• Authors: Koen Ponse, Jan Felix Kleuker, Aske Plaat, Thomas Moerland
• Subjects: cs.LG, cs.AI, eess.SY
• Abstract URL: https://arxiv.org/abs/2507.01522
• Pdf URL: https://arxiv.org/pdf/2507.01522
• Copy Paste: [[2507.01522]] Chargax: A JAX Accelerated EV Charging Simulator(https://arxiv.org/abs/2507.01522)
• Keywords: generation
• Abstract: Deep Reinforcement Learning can play a key role in addressing sustainable energy challenges. For instance, many grid systems are heavily congested, highlighting the urgent need to enhance operational efficiency. However, reinforcement learning approaches have traditionally been slow due to the high sample complexity and expensive simulation requirements. While recent works have effectively used GPUs to accelerate data generation by converting environments to JAX, these works have largely focussed on classical toy problems. This paper introduces Chargax, a JAX-based environment for realistic simulation of electric vehicle charging stations designed for accelerated training of RL agents. We validate our environment in a variety of scenarios based on real data, comparing reinforcement learning agents against baselines. Chargax delivers substantial computational performance improvements of over 100x-1000x over existing environments. Additionally, Chargax' modular architecture enables the representation of diverse real-world charging station configurations.
• 摘要：深厚的强化学习可以在应对可持续能源挑战方面发挥关键作用。例如，许多网格系统非常拥挤，强调了提高操作效率的迫切需求。但是，由于样本复杂性高和昂贵的模拟要求，传统上，加强学习方法一直很慢。尽管最近的作品通过将环境转换为JAX来有效地利用GPU来加速数据生成，但这些作品主要集中在经典的玩具问题上。本文介绍了Chargax，Chargax是一种基于JAX的环境，用于对电动汽车充电站进行逼真的模拟，该环境旨在加速RL代理的培训。我们基于真实数据在各种情况下验证环境，将强化学习剂与基线进行比较。 Chargax在现有环境中提供了超过100x-1000x的大量计算绩效改进。此外，Chargax的模块化体系结构可以代表各种现实世界充电站配置的表示。

Title: A Gift from the Integration of Discriminative and Diffusion-based Generative Learning: Boundary Refinement Remote Sensing Semantic Segmentation

• Authors: Hao Wang, Keyan Hu, Xin Guo, Haifeng Li, Chao Tao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.01573
• Pdf URL: https://arxiv.org/pdf/2507.01573
• Copy Paste: [[2507.01573]] A Gift from the Integration of Discriminative and Diffusion-based Generative Learning: Boundary Refinement Remote Sensing Semantic Segmentation(https://arxiv.org/abs/2507.01573)
• Keywords: generative
• Abstract: Remote sensing semantic segmentation must address both what the ground objects are within an image and where they are located. Consequently, segmentation models must ensure not only the semantic correctness of large-scale patches (low-frequency information) but also the precise localization of boundaries between patches (high-frequency information). However, most existing approaches rely heavily on discriminative learning, which excels at capturing low-frequency features, while overlooking its inherent limitations in learning high-frequency features for semantic segmentation. Recent studies have revealed that diffusion generative models excel at generating high-frequency details. Our theoretical analysis confirms that the diffusion denoising process significantly enhances the model's ability to learn high-frequency features; however, we also observe that these models exhibit insufficient semantic inference for low-frequency features when guided solely by the original image. Therefore, we integrate the strengths of both discriminative and generative learning, proposing the Integration of Discriminative and diffusion-based Generative learning for Boundary Refinement (IDGBR) framework. The framework first generates a coarse segmentation map using a discriminative backbone model. This map and the original image are fed into a conditioning guidance network to jointly learn a guidance representation subsequently leveraged by an iterative denoising diffusion process refining the coarse segmentation. Extensive experiments across five remote sensing semantic segmentation datasets (binary and multi-class segmentation) confirm our framework's capability of consistent boundary refinement for coarse results from diverse discriminative architectures. The source code will be available at this https URL.
• 摘要：遥感语义分割必须解决图像中的地面对象及其位置。因此，分割模型不仅必须确保大规模补丁的语义正确性（低频信息），还必须确保贴片之间边界的精确定位（高频信息）。但是，大多数现有的方法在很大程度上都依赖于判别性学习，该学习擅长捕获低频功能，同时忽略了其在学习用于语义分段的高频特征方面的固有局限性。最近的研究表明，扩散生成模型在产生高频细节方面表现出色。我们的理论分析证实，扩散降解过程可显着增强该模型学习高频特征的能力。但是，我们还观察到，这些模型仅由原始图像引导时表现出对低频特征的语义推断。因此，我们整合了歧视性学习和生成性学习的优势，并提出了基于歧视性和基于扩散的生成学习以进行边界改进（IDGBR）框架的整合。该框架首先使用区分骨干模型生成粗分割图。该地图和原始图像被馈入条件指导网络，以共同学习指导表示，随后通过迭代授予扩散过程来完善粗分割。在五个遥感语义分段数据集（二进制和多级分段）上进行的大量实验证实了我们框架的一致边界改进能力，从而获得了各种歧视性架构的粗略结果。源代码将在此HTTPS URL上可用。

Title: Survivability of Backdoor Attacks on Unconstrained Face Recognition Systems

• Authors: Quentin Le Roux, Yannick Teglia, Teddy Furon, Philippe Loubet-Moundi, Eric Bourbao
• Subjects: cs.CV, cs.AI, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.01607
• Pdf URL: https://arxiv.org/pdf/2507.01607
• Copy Paste: [[2507.01607]] Survivability of Backdoor Attacks on Unconstrained Face Recognition Systems(https://arxiv.org/abs/2507.01607)
• Keywords: generation
• Abstract: The widespread use of deep learning face recognition raises several security concerns. Although prior works point at existing vulnerabilities, DNN backdoor attacks against real-life, unconstrained systems dealing with images captured in the wild remain a blind spot of the literature. This paper conducts the first system-level study of backdoors in deep learning-based face recognition systems. This paper yields four contributions by exploring the feasibility of DNN backdoors on these pipelines in a holistic fashion. We demonstrate for the first time two backdoor attacks on the face detection task: face generation and face landmark shift attacks. We then show that face feature extractors trained with large margin losses also fall victim to backdoor attacks. Combining our models, we then show using 20 possible pipeline configurations and 15 attack cases that a single backdoor enables an attacker to bypass the entire function of a system. Finally, we provide stakeholders with several best practices and countermeasures.
• 摘要：深度学习面部识别的广泛使用引起了一些安全问题。尽管先前的作品指出了现有的漏洞，但DNN的后门攻击对现实生活，无限制的系统处理在野外捕获的图像仍然是文献的盲点。本文在基于深度学习的面部识别系统中对后门进行了首次系统级研究。本文通过以整体方式探讨了这些管道上DNN后门的可行性，从而产生了四个贡献。我们首次在面部检测任务上进行了两次后门攻击：面部生成和面部标志性转移攻击。然后，我们证明接受了较大利润损失训练的面部功能提取器也成为后门攻击的受害者。结合我们的模型，我们使用20个可能的管道配置和15个攻击案例显示，单个后门使攻击者能够绕过系统的整个功能。最后，我们为利益相关者提供了几种最佳实践和对策。

Title: Perception-Oriented Latent Coding for High-Performance Compressed Domain Semantic Inference

• Authors: Xu Zhang, Ming Lu, Yan Chen, Zhan Ma
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2507.01608
• Pdf URL: https://arxiv.org/pdf/2507.01608
• Copy Paste: [[2507.01608]] Perception-Oriented Latent Coding for High-Performance Compressed Domain Semantic Inference(https://arxiv.org/abs/2507.01608)
• Keywords: generative
• Abstract: In recent years, compressed domain semantic inference has primarily relied on learned image coding models optimized for mean squared error (MSE). However, MSE-oriented optimization tends to yield latent spaces with limited semantic richness, which hinders effective semantic inference in downstream tasks. Moreover, achieving high performance with these models often requires fine-tuning the entire vision model, which is computationally intensive, especially for large models. To address these problems, we introduce Perception-Oriented Latent Coding (POLC), an approach that enriches the semantic content of latent features for high-performance compressed domain semantic inference. With the semantically rich latent space, POLC requires only a plug-and-play adapter for fine-tuning, significantly reducing the parameter count compared to previous MSE-oriented methods. Experimental results demonstrate that POLC achieves rate-perception performance comparable to state-of-the-art generative image coding methods while markedly enhancing performance in vision tasks, with minimal fine-tuning overhead. Code is available at this https URL.
• 摘要：近年来，压缩域语义推断主要依赖于针对均方误差（MSE）优化的学习图像编码模型。但是，面向MSE的优化倾向于产生具有有限语义丰富的潜在空间，这阻碍了下游任务的有效语义推断。此外，通过这些模型实现高性能通常需要对整个视觉模型进行微调，这是计算密集型的，尤其是对于大型模型。为了解决这些问题，我们介绍了面向感知的潜在编码（POLC），该方法丰富了高性能压缩域语义推断的潜在特征的语义内容。使用语义丰富的潜在空间，POLC仅需要一个用于微调的插件适配器，与以前面向MSE的方法相比，参数计数大大降低了。实验结果表明，POLC的速率感知性能与最先进的生成图像编码方法相当，同时显着提高了视觉任务的性能，并以最少的微调开销。代码可在此HTTPS URL上找到。

Title: Dance Dance ConvLSTM

• Authors: Miguel O'Malley
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.01644
• Pdf URL: https://arxiv.org/pdf/2507.01644
• Copy Paste: [[2507.01644]] Dance Dance ConvLSTM(https://arxiv.org/abs/2507.01644)
• Keywords: generation
• Abstract: \textit{Dance Dance Revolution} is a rhythm game consisting of songs and accompanying choreography, referred to as charts. Players press arrows on a device referred to as a dance pad in time with steps determined by the song's chart. In 2017, the authors of Dance Dance Convolution (DDC) developed an algorithm for the automatic generation of \textit{Dance Dance Revolution} charts, utilizing a CNN-LSTM architecture. We introduce Dance Dance ConvLSTM (DDCL), a new method for the automatic generation of DDR charts using a ConvLSTM based model, which improves upon the DDC methodology and substantially increases the accuracy of chart generation.
• 摘要：\ textit {舞蹈革命}是一款节奏游戏，由歌曲和随附的编舞组成，称为图表。玩家按及时地称为舞蹈垫的设备上的箭头，并由歌曲图表确定的步骤。 2017年，舞蹈舞蹈卷积（DDC）的作者开发了一种使用CNN-LSTM Architecture的自动生成\ Textit {Dance Dance Revolution}图表的算法。我们介绍了舞蹈convlstm（DDCL），这是一种使用基于ConvlSTM的模型自动生成DDR图表的新方法，该模型可改善DDC方法论，并大大提高了图表生成的准确性。

Title: Autoregressive Image Generation with Linear Complexity: A Spatial-Aware Decay Perspective

• Authors: Yuxin Mao, Zhen Qin, Jinxing Zhou, Hui Deng, Xuyang Shen, Bin Fan, Jing Zhang, Yiran Zhong, Yuchao Dai
• Subjects: cs.CV, cs.AI, cs.MM
• Abstract URL: https://arxiv.org/abs/2507.01652
• Pdf URL: https://arxiv.org/pdf/2507.01652
• Copy Paste: [[2507.01652]] Autoregressive Image Generation with Linear Complexity: A Spatial-Aware Decay Perspective(https://arxiv.org/abs/2507.01652)
• Keywords: generation
• Abstract: Autoregressive (AR) models have garnered significant attention in image generation for their ability to effectively capture both local and global structures within visual data. However, prevalent AR models predominantly rely on the transformer architectures, which are beset by quadratic computational complexity concerning input sequence length and substantial memory overhead due to the necessity of maintaining key-value caches. Although linear attention mechanisms have successfully reduced this burden in language models, our initial experiments reveal that they significantly degrade image generation quality because of their inability to capture critical long-range dependencies in visual data. We propose Linear Attention with Spatial-Aware Decay (LASAD), a novel attention mechanism that explicitly preserves genuine 2D spatial relationships within the flattened image sequences by computing position-dependent decay factors based on true 2D spatial location rather than 1D sequence positions. Based on this mechanism, we present LASADGen, an autoregressive image generator that enables selective attention to relevant spatial contexts with linear complexity. Experiments on ImageNet show LASADGen achieves state-of-the-art image generation performance and computational efficiency, bridging the gap between linear attention's efficiency and spatial understanding needed for high-quality generation.
• 摘要：自回归（AR）模型已在图像生成中引起了极大的关注，因为它们有效地捕获视觉数据中的本地和全球结构的能力。但是，普遍的AR模型主要依赖于变压器体系结构，由于必须维护钥匙值缓存，因此受到输入序列长度和大量内存开销的二次计算复杂性。尽管线性注意机制已成功地减轻了语言模型的负担，但我们的初始实验表明，由于它们无法捕获视觉数据中的关键长期依赖性，因此它们显着降低了图像的产生质量。我们提出了线性注意力的线性注意（LASAD），这是一种新型的注意机制，通过根据真实的2D空间位置而不是1D序列位置来计算位置依赖性衰减因子，从而在扁平的图像序列中明确保留了真正的2D空间关系。基于这种机制，我们提出了lasadgen，这是一种自回归图像发生器，可以选择性地关注具有线性复杂性的相关空间上下文。 Imagenet上的实验表明，Lasadgen实现了最先进的图像生成性能和计算效率，从而弥合了线性注意力的效率和高质量生成所需的空间理解之间的差距。

Title: AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Training

• Authors: Zhenyu Han, Ansheng You, Haibo Wang, Kui Luo, Guang Yang, Wenqi Shi, Menglong Chen, Sicheng Zhang, Zeshun Lan, Chunshi Deng, Huazhong Ji, Wenjie Liu, Yu Huang, Yixiang Zhang, Chenyi Pan, Jing Wang, Xin Huang, Chunsheng Li, Jianping Wu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.01663
• Pdf URL: https://arxiv.org/pdf/2507.01663
• Copy Paste: [[2507.01663]] AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Training(https://arxiv.org/abs/2507.01663)
• Keywords: generation
• Abstract: Reinforcement learning (RL) has become a pivotal technology in the post-training phase of large language models (LLMs). Traditional task-colocated RL frameworks suffer from significant scalability bottlenecks, while task-separated RL frameworks face challenges in complex dataflows and the corresponding resource idling and workload imbalance. Moreover, most existing frameworks are tightly coupled with LLM training or inference engines, making it difficult to support custom-designed engines. To address these challenges, we propose AsyncFlow, an asynchronous streaming RL framework for efficient post-training. Specifically, we introduce a distributed data storage and transfer module that provides a unified data management and fine-grained scheduling capability in a fully streamed manner. This architecture inherently facilitates automated pipeline overlapping among RL tasks and dynamic load balancing. Moreover, we propose a producer-consumer-based asynchronous workflow engineered to minimize computational idleness by strategically deferring parameter update process within staleness thresholds. Finally, the core capability of AsynFlow is architecturally decoupled from underlying training and inference engines and encapsulated by service-oriented user interfaces, offering a modular and customizable user experience. Extensive experiments demonstrate an average of 1.59 throughput improvement compared with state-of-the-art baseline. The presented architecture in this work provides actionable insights for next-generation RL training system designs.
• 摘要：强化学习（RL）已成为大语言模型（LLMS）的训练后阶段中的关键技术。传统的任务重新分配的RL框架具有明显的可扩展性瓶颈，而任务分隔的RL框架在复杂的数据流中面临挑战，以及相应的资源闲置和工作负载不平衡。此外，大多数现有的框架与LLM培训或推理引擎紧密结合，因此很难支持定制设计的发动机。为了应对这些挑战，我们提出了异步流，这是一个异步流的RL RL框架，用于有效的后训练。具体而言，我们引入了分布式数据存储和传输模块，该模块以完全流的方式提供了统一的数据管理和细粒度调度功能。该体系结构固有地促进了RL任务和动态负载平衡之间的自动管道重叠。此外，我们提出了一个基于生产者 - 消费者的异步工作流，该工作流程设计，该工作流程通过策略性地推迟稳定性阈值的参数更新过程来最大程度地减少计算闲置。最后，异步的核心能力在架构上与基础培训和推理引擎脱钩，并由面向服务的用户界面封装，提供模块化且可自定义的用户体验。广泛的实验表明，与最先进的基线相比，平均吞吐量改善了1.59。这项工作中提出的架构为下一代RL培训系统设计提供了可行的见解。

Title: ECCV 2024 W-CODA: 1st Workshop on Multimodal Perception and Comprehension of Corner Cases in Autonomous Driving

• Authors: Kai Chen, Ruiyuan Gao, Lanqing Hong, Hang Xu, Xu Jia, Holger Caesar, Dengxin Dai, Bingbing Liu, Dzmitry Tsishkou, Songcen Xu, Chunjing Xu, Qiang Xu, Huchuan Lu, Dit-Yan Yeung
• Subjects: cs.CV, cs.AI, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.01735
• Pdf URL: https://arxiv.org/pdf/2507.01735
• Copy Paste: [[2507.01735]] ECCV 2024 W-CODA: 1st Workshop on Multimodal Perception and Comprehension of Corner Cases in Autonomous Driving(https://arxiv.org/abs/2507.01735)
• Keywords: generation
• Abstract: In this paper, we present details of the 1st W-CODA workshop, held in conjunction with the ECCV 2024. W-CODA aims to explore next-generation solutions for autonomous driving corner cases, empowered by state-of-the-art multimodal perception and comprehension techniques. 5 Speakers from both academia and industry are invited to share their latest progress and opinions. We collect research papers and hold a dual-track challenge, including both corner case scene understanding and generation. As the pioneering effort, we will continuously bridge the gap between frontier autonomous driving techniques and fully intelligent, reliable self-driving agents robust towards corner cases.
• 摘要：在本文中，我们介绍了与ECCV 2024一起举行的第一W-CODA研讨会的详细信息。W-CODA旨在探索以自动驾驶角案例的下一代解决方案，并由最先进的多模式感知和理解技术增强。邀请了5位来自学术界和行业的发言人分享他们的最新进展和意见。我们收集研究论文并面临双轨挑战，包括角落案例的理解和发电。作为开创性的努力，我们将不断弥合前沿自动驾驶技术与完全聪明，可靠的自动驾驶代理之间的差距，以固定在角落案件上。

Title: HOI-Dyn: Learning Interaction Dynamics for Human-Object Motion Diffusion

• Authors: Lin Wu, Zhixiang Chen, Jianglin Lan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.01737
• Pdf URL: https://arxiv.org/pdf/2507.01737
• Copy Paste: [[2507.01737]] HOI-Dyn: Learning Interaction Dynamics for Human-Object Motion Diffusion(https://arxiv.org/abs/2507.01737)
• Keywords: generation
• Abstract: Generating realistic 3D human-object interactions (HOIs) remains a challenging task due to the difficulty of modeling detailed interaction dynamics. Existing methods treat human and object motions independently, resulting in physically implausible and causally inconsistent behaviors. In this work, we present HOI-Dyn, a novel framework that formulates HOI generation as a driver-responder system, where human actions drive object responses. At the core of our method is a lightweight transformer-based interaction dynamics model that explicitly predicts how objects should react to human motion. To further enforce consistency, we introduce a residual-based dynamics loss that mitigates the impact of dynamics prediction errors and prevents misleading optimization signals. The dynamics model is used only during training, preserving inference efficiency. Through extensive qualitative and quantitative experiments, we demonstrate that our approach not only enhances the quality of HOI generation but also establishes a feasible metric for evaluating the quality of generated interactions.
• 摘要：由于难以对详细的相互作用动态建模，生成现实的3D人类对象相互作用（HOI）仍然是一项艰巨的任务。现有方法独立处理人类和对象运动，从而使身体上不可行和因果不一致。在这项工作中，我们提出了Hoi-dyn，这是一个新颖的框架，该框架将HOI生成作为驾驶员响应系统，其中人类行动驱动对象响应。我们方法的核心是一个基于轻巧的变压器的相互作用动力学模型，该模型明确预测对象应如何反应人类运动。为了进一步执行一致性，我们引入了基于残差的动态损失，以减轻动态预测误差的影响并防止误导性优化信号。动力学模型仅在训练过程中使用，并保留推理效率。通过广泛的定性和定量实验，我们证明了我们的方法不仅可以提高HOI生成的质量，而且还建立了可行的指标来评估产生的相互作用的质量。

Title: Rethinking Discrete Tokens: Treating Them as Conditions for Continuous Autoregressive Image Synthesis

• Authors: Peng Zheng, Junke Wang, Yi Chang, Yizhou Yu, Rui Ma, Zuxuan Wu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.01756
• Pdf URL: https://arxiv.org/pdf/2507.01756
• Copy Paste: [[2507.01756]] Rethinking Discrete Tokens: Treating Them as Conditions for Continuous Autoregressive Image Synthesis(https://arxiv.org/abs/2507.01756)
• Keywords: generation
• Abstract: Recent advances in large language models (LLMs) have spurred interests in encoding images as discrete tokens and leveraging autoregressive (AR) frameworks for visual generation. However, the quantization process in AR-based visual generation models inherently introduces information loss that degrades image fidelity. To mitigate this limitation, recent studies have explored to autoregressively predict continuous tokens. Unlike discrete tokens that reside in a structured and bounded space, continuous representations exist in an unbounded, high-dimensional space, making density estimation more challenging and increasing the risk of generating out-of-distribution artifacts. Based on the above findings, this work introduces DisCon (Discrete-Conditioned Continuous Autoregressive Model), a novel framework that reinterprets discrete tokens as conditional signals rather than generation targets. By modeling the conditional probability of continuous representations conditioned on discrete tokens, DisCon circumvents the optimization challenges of continuous token modeling while avoiding the information loss caused by quantization. DisCon achieves a gFID score of 1.38 on ImageNet 256$\times$256 generation, outperforming state-of-the-art autoregressive approaches by a clear margin.
• 摘要：大型语言模型（LLM）的最新进展激发了将图像作为离散令牌编码和利用自动回归（AR）框架的兴趣。但是，基于AR的视觉生成模型中的量化过程固有地引入了信息丢失，从而降低了图像保真度。为了减轻这一限制，最近的研究探索了自动加工的连续令牌。与驻留在结构化和有界空间中的离散令牌不同，连续表示存在在无限的高维空间中，从而使密度估计更具挑战性并增加产生过分分发工件的风险。基于上面的发现，这项工作引入了Discon（离散条件连续自回归模型），这是一个新颖的框架，将离散令牌重新诠释为条件信号而不是生成目标。通过对以离散令牌为条件的连续表示的条件概率进行建模，可以折断连续令牌建模的优化挑战，同时避免了量化造成的信息损失。 Discon在Imagenet 256 $ \ Times Generation上获得1.38的GFID分数，超过了最先进的自动回归方法的明确利润。

Title: Enhanced Generative Model Evaluation with Clipped Density and Coverage

• Authors: Nicolas Salvy, Hugues Talbot, Bertrand Thirion
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2507.01761
• Pdf URL: https://arxiv.org/pdf/2507.01761
• Copy Paste: [[2507.01761]] Enhanced Generative Model Evaluation with Clipped Density and Coverage(https://arxiv.org/abs/2507.01761)
• Keywords: generative
• Abstract: Although generative models have made remarkable progress in recent years, their use in critical applications has been hindered by their incapacity to reliably evaluate sample quality. Quality refers to at least two complementary concepts: fidelity and coverage. Current quality metrics often lack reliable, interpretable values due to an absence of calibration or insufficient robustness to outliers. To address these shortcomings, we introduce two novel metrics, Clipped Density and Clipped Coverage. By clipping individual sample contributions and, for fidelity, the radii of nearest neighbor balls, our metrics prevent out-of-distribution samples from biasing the aggregated values. Through analytical and empirical calibration, these metrics exhibit linear score degradation as the proportion of poor samples increases. Thus, they can be straightforwardly interpreted as equivalent proportions of good samples. Extensive experiments on synthetic and real-world datasets demonstrate that Clipped Density and Clipped Coverage outperform existing methods in terms of robustness, sensitivity, and interpretability for evaluating generative models.
• 摘要：尽管近年来生成模型取得了显着的进展，但由于它们无能可靠地评估样本质量的能力，它们在关键应用中的使用受到了阻碍。质量至少是指两个互补概念：保真度和覆盖范围。当前的质量指标通常由于缺乏对异常值的校准或不足的鲁棒性而缺乏可靠的，可解释的值。为了解决这些缺点，我们介绍了两个新颖的指标，剪裁密度和覆盖范围。通过删除单个样本贡献，为了保真为最近的邻居球的半径，我们的指标可以防止分布样品偏置汇总值。通过分析和经验校准，随着贫困样品比例的增加，这些指标表现出线性评分降解。因此，它们可以直接解释为相当的好样品比例。关于合成和现实世界数据集的广泛实验表明，在鲁棒性，灵敏度和评估生成模型的可解释性方面，剪辑密度和剪辑覆盖率优于现有方法。

Title: FreeLoRA: Enabling Training-Free LoRA Fusion for Autoregressive Multi-Subject Personalization

• Authors: Peng Zheng, Ye Wang, Rui Ma, Zuxuan Wu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.01792
• Pdf URL: https://arxiv.org/pdf/2507.01792
• Copy Paste: [[2507.01792]] FreeLoRA: Enabling Training-Free LoRA Fusion for Autoregressive Multi-Subject Personalization(https://arxiv.org/abs/2507.01792)
• Keywords: generation, generative
• Abstract: Subject-driven image generation plays a crucial role in applications such as virtual try-on and poster design. Existing approaches typically fine-tune pretrained generative models or apply LoRA-based adaptations for individual subjects. However, these methods struggle with multi-subject personalization, as combining independently adapted modules often requires complex re-tuning or joint optimization. We present FreeLoRA, a simple and generalizable framework that enables training-free fusion of subject-specific LoRA modules for multi-subject personalization. Each LoRA module is adapted on a few images of a specific subject using a Full Token Tuning strategy, where it is applied across all tokens in the prompt to encourage weakly supervised token-content alignment. At inference, we adopt Subject-Aware Inference, activating each module only on its corresponding subject tokens. This enables training-free fusion of multiple personalized subjects within a single image, while mitigating overfitting and mutual interference between subjects. Extensive experiments show that FreeLoRA achieves strong performance in both subject fidelity and prompt consistency.
• 摘要：主题驱动的图像生成在虚拟试验和海报设计等应用中起着至关重要的作用。现有方法通常是对预及生成模型的微调或对个别受试者的基于洛拉的适应性。但是，这些方法在多主体个性化方面遇到了困难，因为组合独立适应的模块通常需要复杂的重新调整或关节优化。我们提出了Freelora，这是一个简单且可概括的框架，可实现对特定主题的Lora模块的无培训融合，以进行多个对象个性化。每个LORA模块都使用完整的令牌调整策略对特定主题的几张图像进行了调整，在该策略中，它在所有令牌中都应用于所有令牌，以鼓励弱监督的令牌包含对准。在推论时，我们采用主题感知的推断，仅在其相应的主题令牌上激活每个模块。这使得可以在单个图像中对多个个性化受试者进行无训练融合，同时减轻受试者之间的过度拟合和相互干扰。广泛的实验表明，Freelora在受试者的保真度和迅速的一致性方面都达到了强劲的表现。

Title: LoRA Fine-Tuning Without GPUs: A CPU-Efficient Meta-Generation Framework for LLMs

• Authors: Reza Arabpour, Haitz Sáez de Ocáriz Borde, Anastasis Kratsios
• Subjects: cs.LG, cs.AI, cs.CL, stat.ML
• Abstract URL: https://arxiv.org/abs/2507.01806
• Pdf URL: https://arxiv.org/pdf/2507.01806
• Copy Paste: [[2507.01806]] LoRA Fine-Tuning Without GPUs: A CPU-Efficient Meta-Generation Framework for LLMs(https://arxiv.org/abs/2507.01806)
• Keywords: generation
• Abstract: Low-Rank Adapters (LoRAs) have transformed the fine-tuning of Large Language Models (LLMs) by enabling parameter-efficient updates. However, their widespread adoption remains limited by the reliance on GPU-based training. In this work, we propose a theoretically grounded approach to LoRA fine-tuning designed specifically for users with limited computational resources, particularly those restricted to standard laptop CPUs. Our method learns a meta-operator that maps any input dataset, represented as a probability distribution, to a set of LoRA weights by leveraging a large bank of pre-trained adapters for the Mistral-7B-Instruct-v0.2 model. Instead of performing new gradient-based updates, our pipeline constructs adapters via lightweight combinations of existing LoRAs directly on CPU. While the resulting adapters do not match the performance of GPU-trained counterparts, they consistently outperform the base Mistral model on downstream tasks, offering a practical and accessible alternative to traditional GPU-based fine-tuning.
• 摘要：低级适配器（Loras）通过启用参数有效的更新来改变大语模型（LLMS）的微调。但是，他们的广泛采用仍然受到对基于GPU的培训的依赖的限制。在这项工作中，我们为专门为有限的计算资源（尤其是限于标准笔记本电脑CPU的用户）设计的Lora微型调整提出了一种理论上扎根的方法。我们的方法学习了一个元操作员，该元操作员通过利用大量的预先训练的适配器来绘制一组lora权重的任何输入数据集，以用于Mistral-7b-Instruct-V0.2模型。我们的管道没有执行新的基于梯度的更新，而是通过直接在CPU上的现有Loras的轻量级组合来构造适配器。尽管所得的适配器与受GPU训练的对应物的性能不符，但它们在下游任务上始终超过基本的Mistral模型，提供了基于GPU的传统基于GPU的实用替代方案。

Title: Out-of-Distribution Detection Methods Answer the Wrong Questions

• Authors: Yucen Lily Li, Daohan Lu, Polina Kirichenko, Shikai Qiu, Tim G. J. Rudner, C. Bayan Bruss, Andrew Gordon Wilson
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2507.01831
• Pdf URL: https://arxiv.org/pdf/2507.01831
• Copy Paste: [[2507.01831]] Out-of-Distribution Detection Methods Answer the Wrong Questions(https://arxiv.org/abs/2507.01831)
• Keywords: generative
• Abstract: To detect distribution shifts and improve model safety, many out-of-distribution (OOD) detection methods rely on the predictive uncertainty or features of supervised models trained on in-distribution data. In this paper, we critically re-examine this popular family of OOD detection procedures, and we argue that these methods are fundamentally answering the wrong questions for OOD detection. There is no simple fix to this misalignment, since a classifier trained only on in-distribution classes cannot be expected to identify OOD points; for instance, a cat-dog classifier may confidently misclassify an airplane if it contains features that distinguish cats from dogs, despite generally appearing nothing alike. We find that uncertainty-based methods incorrectly conflate high uncertainty with being OOD, while feature-based methods incorrectly conflate far feature-space distance with being OOD. We show how these pathologies manifest as irreducible errors in OOD detection and identify common settings where these methods are ineffective. Additionally, interventions to improve OOD detection such as feature-logit hybrid methods, scaling of model and data size, epistemic uncertainty representation, and outlier exposure also fail to address this fundamental misalignment in objectives. We additionally consider unsupervised density estimation and generative models for OOD detection, which we show have their own fundamental limitations.
• 摘要：为了检测分布变化并提高模型安全性，许多分布（OOD）检测方法依赖于对分布数据进行训练的监督模型的预测不确定性或特征。在本文中，我们重新检查了这个流行的OOD检测程序家族，我们认为这些方法从根本上回答了错误的OOD检测问题。这种未对准并没有简单的修复，因为不能期望仅在分配类中训练的分类器识别出OOD点；例如，如果猫狗分类器包含将猫与狗区分开的特征，尽管它通常没有任何相似之处，猫狗分类器可能会自信地错误分类。我们发现，基于不确定性的方法错误地将高不确定性与OOD混合在一起，而基于特征的方法将远距离的特征空间距离与OOD混合在一起。我们展示了这些病理如何在OOD检测中表现为不可证明的错误，并确定这些方法无效的常见设置。此外，改善OOD检测的干预措施，例如特征逻辑混合方法，模型和数据规模的缩放，认知不确定性表示以及离群较高的暴露也无法解决目标中的这一基本未对准。我们还考虑了无监督的密度估计和OOD检测的生成模型，我们显示它们具有自己的基本局限性。

Title: Towards Foundation Auto-Encoders for Time-Series Anomaly Detection

• Authors: Gastón García González, Pedro Casas, Emilio Martínez, Alicia Fernández
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.01875
• Pdf URL: https://arxiv.org/pdf/2507.01875
• Copy Paste: [[2507.01875]] Towards Foundation Auto-Encoders for Time-Series Anomaly Detection(https://arxiv.org/abs/2507.01875)
• Keywords: generative
• Abstract: We investigate a novel approach to time-series modeling, inspired by the successes of large pretrained foundation models. We introduce FAE (Foundation Auto-Encoders), a foundation generative-AI model for anomaly detection in time-series data, based on Variational Auto-Encoders (VAEs). By foundation, we mean a model pretrained on massive amounts of time-series data which can learn complex temporal patterns useful for accurate modeling, forecasting, and detection of anomalies on previously unseen datasets. FAE leverages VAEs and Dilated Convolutional Neural Networks (DCNNs) to build a generic model for univariate time-series modeling, which could eventually perform properly in out-of-the-box, zero-shot anomaly detection applications. We introduce the main concepts of FAE, and present preliminary results in different multi-dimensional time-series datasets from various domains, including a real dataset from an operational mobile ISP, and the well known KDD 2021 Anomaly Detection dataset.
• 摘要：我们研究了一种新颖的时间序列建模方法，灵感来自大型审计基础模型的成功。我们介绍了基于变异自动编码器（VAES）的时间序列数据的基础生成-AI模型FAE（Foundation Auto-编码器）。根据基础，我们的意思是对大量时间序列数据进行了预测的模型，该模型可以学习复杂的时间模式，可用于准确的建模，预测和检测以前看不见的数据集对异常的检测。 FAE利用VAE和扩张的卷积神经网络（DCNN）来构建单变量时间序列建模的通用模型，这些模型最终可以在开箱即用的，零拍的异常检测应用程序中正确执行。我们介绍了FAE的主要概念，并以来自各个域的不同多维时间序列数据集（包括来自操作的移动ISP的真实数据集）以及众所周知的KDD 2021 Anomaly检测数据集中的不同多维时间序列数据集。

Title: Self-Reinforcing Prototype Evolution with Dual-Knowledge Cooperation for Semi-Supervised Lifelong Person Re-Identification

• Authors: Kunlun Xu, Fan Zhuo, Jiangmeng Li, Xu Zou, Jiahuan Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.01884
• Pdf URL: https://arxiv.org/pdf/2507.01884
• Copy Paste: [[2507.01884]] Self-Reinforcing Prototype Evolution with Dual-Knowledge Cooperation for Semi-Supervised Lifelong Person Re-Identification(https://arxiv.org/abs/2507.01884)
• Keywords: generation
• Abstract: Current lifelong person re-identification (LReID) methods predominantly rely on fully labeled data streams. However, in real-world scenarios where annotation resources are limited, a vast amount of unlabeled data coexists with scarce labeled samples, leading to the Semi-Supervised LReID (Semi-LReID) problem where LReID methods suffer severe performance degradation. Existing LReID methods, even when combined with semi-supervised strategies, suffer from limited long-term adaptation performance due to struggling with the noisy knowledge occurring during unlabeled data utilization. In this paper, we pioneer the investigation of Semi-LReID, introducing a novel Self-Reinforcing Prototype Evolution with Dual-Knowledge Cooperation framework (SPRED). Our key innovation lies in establishing a self-reinforcing cycle between dynamic prototype-guided pseudo-label generation and new-old knowledge collaborative purification to enhance the utilization of unlabeled data. Specifically, learnable identity prototypes are introduced to dynamically capture the identity distributions and generate high-quality pseudo-labels. Then, the dual-knowledge cooperation scheme integrates current model specialization and historical model generalization, refining noisy pseudo-labels. Through this cyclic design, reliable pseudo-labels are progressively mined to improve current-stage learning and ensure positive knowledge propagation over long-term learning. Experiments on the established Semi-LReID benchmarks show that our SPRED achieves state-of-the-art performance. Our source code is available at this https URL
• 摘要：当前的终生人士重新识别（LREID）方法主要依赖于完全标记的数据流。但是，在注释资源有限的现实情况下，大量未标记的数据共存，标有稀缺的样品，导致半监督的LREID（半透明）问题，其中LREID方法遭受了严重的性能降低。现有的LREID方法，即使与半监督的策略结合使用，由于在未标记的数据利用过程中遇到嘈杂的知识而陷入困境，因此长期适应性绩效有限。在本文中，我们开创了对半朗德的研究，引入了一种新颖的自我增强原型演化，并使用双重知识合作框架（Spred）引入了。我们的关键创新在于在动态原型引导的伪标签生成和新老知识协作净化之间建立一个自我增强周期，以增强未标记数据的利用。具体而言，引入了可学习的身份原型以动态捕获身份分布并生成高质量的伪标记。然后，双重知识合作方案集成了当前的模型专业化和历史模型概括，并完善了嘈杂的伪标记。通过这种循环设计，可靠的伪标签被逐步开采，以改善当前阶段的学习并确保在长期学习中积极知识传播。在既定的半统计基准中进行的实验表明，我们的SPRS达到了最先进的性能。我们的源代码可在此HTTPS URL上找到

Title: Reasoning to Edit: Hypothetical Instruction-Based Image Editing with Visual Reasoning

• Authors: Qingdong He, Xueqin Chen, Chaoyi Wang, Yanjie Pan, Xiaobin Hu, Zhenye Gan, Yabiao Wang, Chengjie Wang, Xiangtai Li, Jiangning Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.01908
• Pdf URL: https://arxiv.org/pdf/2507.01908
• Copy Paste: [[2507.01908]] Reasoning to Edit: Hypothetical Instruction-Based Image Editing with Visual Reasoning(https://arxiv.org/abs/2507.01908)
• Keywords: generation
• Abstract: Instruction-based image editing (IIE) has advanced rapidly with the success of diffusion models. However, existing efforts primarily focus on simple and explicit instructions to execute editing operations such as adding, deleting, moving, or swapping objects. They struggle to handle more complex implicit hypothetical instructions that require deeper reasoning to infer plausible visual changes and user intent. Additionally, current datasets provide limited support for training and evaluating reasoning-aware editing capabilities. Architecturally, these methods also lack mechanisms for fine-grained detail extraction that support such reasoning. To address these limitations, we propose Reason50K, a large-scale dataset specifically curated for training and evaluating hypothetical instruction reasoning image editing, along with ReasonBrain, a novel framework designed to reason over and execute implicit hypothetical instructions across diverse scenarios. Reason50K includes over 50K samples spanning four key reasoning scenarios: Physical, Temporal, Causal, and Story reasoning. ReasonBrain leverages Multimodal Large Language Models (MLLMs) for editing guidance generation and a diffusion model for image synthesis, incorporating a Fine-grained Reasoning Cue Extraction (FRCE) module to capture detailed visual and textual semantics essential for supporting instruction reasoning. To mitigate the semantic loss, we further introduce a Cross-Modal Enhancer (CME) that enables rich interactions between the fine-grained cues and MLLM-derived features. Extensive experiments demonstrate that ReasonBrain consistently outperforms state-of-the-art baselines on reasoning scenarios while exhibiting strong zero-shot generalization to conventional IIE tasks. Our dataset and code will be released publicly.
• 摘要：基于指导的图像编辑（IIE）随着扩散模型的成功迅速发展。但是，现有的努力主要关注简单和明确的指令，以执行编辑操作，例如添加，删除，移动或交换对象。他们努力处理更复杂的隐性假设指示，这些指令需要更深入的推理来推断出合理的视觉变化和用户意图。此外，当前的数据集为培训和评估推理意识的编辑功能提供了有限的支持。从结构上讲，这些方法还缺乏支持这种推理的细粒细节提取的机制。为了解决这些局限性，我们提出了Reason 50k，这是一个专门策划培训和评估假设教学推理图像编辑的大规模数据集，以及Reasonbrain，这是一个新颖的框架，旨在推理和执行跨不同场景的隐性假设指示。 Reason 50k包括超过50k样本，涵盖了四个关键推理方案：物理，时间，因果和故事推理。理性脑利用多模式大语言模型（MLLM）进行编辑指导生成和图像合成的扩散模型，并结合了细粒度的推理提示提取（FRCE）模块，以捕获支持指导推理的详细视觉和文本语义。为了减轻语义损失，我们进一步引入了一个跨模式增强器（CME），该增强器（CME）可以在细粒度提示和MLLM衍生的特征之间进行丰富的相互作用。广泛的实验表明，理性脑在推理方案上始终优于最先进的基线，同时表现出对常规IIE任务的强烈零弹性概括。我们的数据集和代码将公开发布。

Title: CI-VID: A Coherent Interleaved Text-Video Dataset

• Authors: Yiming Ju, Jijin Hu, Zhengxiong Luo, Haoge Deng, hanyu Zhao, Li Du, Chengwei Wu, Donglin Hao, Xinlong Wang, Tengfei Pan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.01938
• Pdf URL: https://arxiv.org/pdf/2507.01938
• Copy Paste: [[2507.01938]] CI-VID: A Coherent Interleaved Text-Video Dataset(https://arxiv.org/abs/2507.01938)
• Keywords: generation
• Abstract: Text-to-video (T2V) generation has recently attracted considerable attention, resulting in the development of numerous high-quality datasets that have propelled progress in this area. However, existing public datasets are primarily composed of isolated text-video (T-V) pairs and thus fail to support the modeling of coherent multi-clip video sequences. To address this limitation, we introduce CI-VID, a dataset that moves beyond isolated text-to-video (T2V) generation toward text-and-video-to-video (TV2V) generation, enabling models to produce coherent, multi-scene video sequences. CI-VID contains over 340,000 samples, each featuring a coherent sequence of video clips with text captions that capture both the individual content of each clip and the transitions between them, enabling visually and textually grounded generation. To further validate the effectiveness of CI-VID, we design a comprehensive, multi-dimensional benchmark incorporating human evaluation, VLM-based assessment, and similarity-based metrics. Experimental results demonstrate that models trained on CI-VID exhibit significant improvements in both accuracy and content consistency when generating video sequences. This facilitates the creation of story-driven content with smooth visual transitions and strong temporal coherence, underscoring the quality and practical utility of the CI-VID dataset We release the CI-VID dataset and the accompanying code for data construction and evaluation at: this https URL
• 摘要：文本到视频（T2V）的一代最近引起了广泛的关注，导致发展了许多高质量的数据集，这些数据集已推动了这一领域的进步。但是，现有的公共数据集主要由孤立的文本视频（T-V）对组成，因此无法支持连贯的多CLIP视频序列的建模。为了解决此限制，我们介绍了CI-VID，这是一个数据集，该数据集超越了孤立的文本对视频（T2V）生成，向文本和视频到视频（TV2V）生成，使模型能够产生连贯的多场景视频序列。 CI-VID包含340,000多个样本，每个样本都包含一系列连贯的视频剪辑序列，并带有文本字幕，可捕获每个剪辑的单个内容以及它们之间的过渡，从而在视觉上和文本基础上启用。为了进一步验证CI-VID的有效性，我们设计了一个全面的，多维的基准，该基准包括人类评估，基于VLM的评估和基于相似性的指标。实验结果表明，在生成视频序列时，经过CI-VID训练的模型在准确性和内容一致性方面都具有显着提高。这有助于创建以平稳的视觉过渡和较强的时间连贯性来创建故事驱动的内容，强调CI-VID数据集的质量和实用性，我们释放了CI-VID数据集以及随附的数据构建和评估代码：

Title: LongAnimation: Long Animation Generation with Dynamic Global-Local Memory

• Authors: Nan Chen, Mengqi Huang, Yihao Meng, Zhendong Mao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.01945
• Pdf URL: https://arxiv.org/pdf/2507.01945
• Copy Paste: [[2507.01945]] LongAnimation: Long Animation Generation with Dynamic Global-Local Memory(https://arxiv.org/abs/2507.01945)
• Keywords: generation
• Abstract: Animation colorization is a crucial part of real animation industry production. Long animation colorization has high labor costs. Therefore, automated long animation colorization based on the video generation model has significant research value. Existing studies are limited to short-term colorization. These studies adopt a local paradigm, fusing overlapping features to achieve smooth transitions between local segments. However, the local paradigm neglects global information, failing to maintain long-term color consistency. In this study, we argue that ideal long-term color consistency can be achieved through a dynamic global-local paradigm, i.e., dynamically extracting global color-consistent features relevant to the current generation. Specifically, we propose LongAnimation, a novel framework, which mainly includes a SketchDiT, a Dynamic Global-Local Memory (DGLM), and a Color Consistency Reward. The SketchDiT captures hybrid reference features to support the DGLM module. The DGLM module employs a long video understanding model to dynamically compress global historical features and adaptively fuse them with the current generation features. To refine the color consistency, we introduce a Color Consistency Reward. During inference, we propose a color consistency fusion to smooth the video segment transition. Extensive experiments on both short-term (14 frames) and long-term (average 500 frames) animations show the effectiveness of LongAnimation in maintaining short-term and long-term color consistency for open-domain animation colorization task. The code can be found at this https URL.
• 摘要：动画着色是真实动画行业生产的关键部分。长时间的动画着色的人工成本很高。因此，基于视频生成模型的自动长动画着色具有重要的研究值。现有研究仅限于短期着色。这些研究采用了当地范式，融合了重叠的特征，以实现本地细分市场之间的平稳过渡。但是，本地范式忽略了全球信息，无法保持长期颜色一致性。在这项研究中，我们认为可以通过动态的全局本地范式，即动态提取与当前一代相关的全球颜色一致性特征来实现理想的长期颜色一致性。具体来说，我们提出了一个新颖的框架，主要包括sketchdit，动态的全部本地内存（DGLM）和颜色一致性奖励。 SketchDit捕获了混合参考功能，以支持DGLM模块。 DGLM模块采用较长的视频理解模型来动态压缩全球历史特征，并将其与当前一代特征融合在一起。为了完善颜色一致性，我们引入了颜色一致性奖励。在推断期间，我们提出了颜色一致性融合以平滑视频段过渡。在短期（14帧）和长期（平均500帧）动画上进行了广泛的实验，显示了longanimation在保持短期和长期颜色一致性方面的有效性，以实现开放域动画着色任务。该代码可以在此HTTPS URL上找到。

Title: Test-Time Scaling with Reflective Generative Model

• Authors: Zixiao Wang, Yuxin Wang, Xiaorui Wang, Mengting Xing, Jie Gao, Jianjun Xu, Guangcan Liu, Chenhui Jin, Zhuo Wang, Shengzhuo Zhang, Hongtao Xie
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2507.01951
• Pdf URL: https://arxiv.org/pdf/2507.01951
• Copy Paste: [[2507.01951]] Test-Time Scaling with Reflective Generative Model(https://arxiv.org/abs/2507.01951)
• Keywords: generative
• Abstract: We introduce our first reflective generative model MetaStone-S1, which obtains OpenAI o3's performance via the self-supervised process reward model (SPRM). Through sharing the backbone network and using task-specific heads for next token prediction and process scoring respectively, SPRM successfully integrates the policy model and process reward model(PRM) into a unified interface without extra process annotation, reducing over 99% PRM parameters for efficient reasoning. Equipped with SPRM, MetaStone-S1 is naturally suitable for test time scaling (TTS), and we provide three reasoning effort modes (low, medium, and high), based on the controllable thinking length. Moreover, we empirically establish a scaling law that reveals the relationship between total thinking computation and TTS performance. Experiments demonstrate that our MetaStone-S1 achieves comparable performance to OpenAI-o3-mini's series with only 32B parameter size. To support the research community, we have open-sourced MetaStone-S1 at this https URL.
• 摘要：我们介绍了我们的第一个反射生成模型metastone-s1，该模型通过自我监督的过程奖励模型（SPRM）获得OpenAI O3的性能。通过共享骨干网络并使用特定于任务的头部进行隔壁预测和过程评分，SPRM成功地将策略​​模型和流程奖励模型（PRM）集成到统一的界面中，而无需额外的流程注释，减少了超过99％的PRM参数以获得有效的推理。 Metastone-S1配备了SPRM，自然适合测试时间缩放（TTS），我们根据可控的思维长度提供了三种推理工作模式（低，中，中高）。此外，我们从经验上建立了扩展定律，该法律揭示了总思维计算与TTS绩效之间的关系。实验表明，我们的Metastone-S1的性能与OpenAI-O3-Mini的系列相当，只有32B参数大小。为了支持研究社区，我们在此HTTPS URL上有开源的Metastone-S1。

Title: How Well Does GPT-4o Understand Vision? Evaluating Multimodal Foundation Models on Standard Computer Vision Tasks

• Authors: Rahul Ramachandran, Ali Garjani, Roman Bachmann, Andrei Atanov, Oğuzhan Fatih Kar, Amir Zamir
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.01955
• Pdf URL: https://arxiv.org/pdf/2507.01955
• Copy Paste: [[2507.01955]] How Well Does GPT-4o Understand Vision? Evaluating Multimodal Foundation Models on Standard Computer Vision Tasks(https://arxiv.org/abs/2507.01955)
• Keywords: generation
• Abstract: Multimodal foundation models, such as GPT-4o, have recently made remarkable progress, but it is not clear where exactly these models stand in terms of understanding vision. In this paper, we benchmark the performance of popular multimodal foundation models (GPT-4o, o4-mini, Gemini 1.5 Pro and Gemini 2.0 Flash, Claude 3.5 Sonnet, Qwen2-VL, Llama 3.2) on standard computer vision tasks (semantic segmentation, object detection, image classification, depth and surface normal prediction) using established datasets (e.g., COCO, ImageNet and its variants, etc). The main challenges to performing this are: 1) most models are trained to output text and cannot natively express versatile domains, such as segments or 3D geometry, and 2) many leading models are proprietary and accessible only at an API level, i.e., there is no weight access to adapt them. We address these challenges by translating standard vision tasks into equivalent text-promptable and API-compatible tasks via prompt chaining to create a standardized benchmarking framework. We observe that 1) the models are not close to the state-of-the-art specialist models at any task. However, 2) they are respectable generalists; this is remarkable as they are presumably trained on primarily image-text-based tasks. 3) They perform semantic tasks notably better than geometric ones. 4) While the prompt-chaining techniques affect performance, better models exhibit less sensitivity to prompt variations. 5) GPT-4o performs the best among non-reasoning models, securing the top position in 4 out of 6 tasks, 6) reasoning models, e.g. o3, show improvements in geometric tasks, and 7) a preliminary analysis of models with native image generation, like the latest GPT-4o, shows they exhibit quirks like hallucinations and spatial misalignments.
• 摘要：多模式的基础模型（例如GPT-4O）最近取得了显着的进步，但尚不清楚这些模型在理解视觉方面的确切位置。在本文中，我们基于流行的多式联系基础模型（GPT-4O，O4-Mini，Gemini 1.5 Pro和Gemini 2.0 Flash，Claude 3.5 SONNET，QWEN2-VL，LLAMA 3.2）在标准计算机视觉任务上（使用语义段，对象检测，图像分类，depsife，ccote）（CRE）（eco）（eco）（eco）（eco）（eco）（eco）（eco）（eco）（CLAMA 3.2）变体等）。执行此操作的主要挑战是：1）大多数模型都接受了输出文本的培训，并且不能本地表达通用的域，例如片段或3D几何，2）许多领先的模型仅在API级别上可访问，即没有重量访问它们适应它们。我们通过将标准视觉任务转换为同等文本和API兼容的任务来解决这些挑战。我们观察到1）在任何任务下，模型都不接近最先进的专家模型。但是，2）他们是受人尊敬的通才；这是非常了不起的，因为它们大概是对主要基于图像文本的任务进行培训的。 3）他们执行语义任务比几何任务要好。 4）虽然迅速链接技术会影响性能，但更好的模型对迅速变化的敏感性较小。 5）GPT-4O在非争议模型中表现最好，在6个任务中的4个任务中获得最高位置，6）推理模型，例如O3显示了几何任务的改进，以及7）对具有天然图像产生的模型的初步分析，例如最新的GPT-4O，表明它们表现出幻觉和空间未对准等怪癖。

Title: Locality-aware Parallel Decoding for Efficient Autoregressive Image Generation

• Authors: Zhuoyang Zhang, Luke J. Huang, Chengyue Wu, Shang Yang, Kelly Peng, Yao Lu, Song Han
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.01957
• Pdf URL: https://arxiv.org/pdf/2507.01957
• Copy Paste: [[2507.01957]] Locality-aware Parallel Decoding for Efficient Autoregressive Image Generation(https://arxiv.org/abs/2507.01957)
• Keywords: generation
• Abstract: We present Locality-aware Parallel Decoding (LPD) to accelerate autoregressive image generation. Traditional autoregressive image generation relies on next-patch prediction, a memory-bound process that leads to high latency. Existing works have tried to parallelize next-patch prediction by shifting to multi-patch prediction to accelerate the process, but only achieved limited parallelization. To achieve high parallelization while maintaining generation quality, we introduce two key techniques: (1) Flexible Parallelized Autoregressive Modeling, a novel architecture that enables arbitrary generation ordering and degrees of parallelization. It uses learnable position query tokens to guide generation at target positions while ensuring mutual visibility among concurrently generated tokens for consistent parallel decoding. (2) Locality-aware Generation Ordering, a novel schedule that forms groups to minimize intra-group dependencies and maximize contextual support, enhancing generation quality. With these designs, we reduce the generation steps from 256 to 20 (256$\times$256 res.) and 1024 to 48 (512$\times$512 res.) without compromising quality on the ImageNet class-conditional generation, and achieving at least 3.4$\times$ lower latency than previous parallelized autoregressive models.
• 摘要：我们提出局部感知的并行解码（LPD），以加速自回归图像产生。传统的自回归图像生成依赖于下一步预测，这是一个导致较高延迟的内存过程。现有的作品试图通过转移到多斑点预测以加速过程来并行化下一点预测，但仅实现了有限的并行化。为了在保持发电质量的同时实现高平行化，我们引入了两种关键技术：（1）灵活的平行自回归建模，这是一种新型的体系结构，可实现任意生成排序和并行程度。它使用可学习的位置查询令牌来指导目标位置的生成，同时确保同时生成的令牌之间相互可见性，以始终如一地平行解码。 （2）局部感知的生成订购，这是一个新的时间表，形成了群体，以最大程度地减少组内依赖性并最大化上下文支持，从而提高生成质量。通过这些设计，我们将生成步骤从256降低到20（256 $ \ times $ 256 res。）和1024降至48（512 $ \ times $ 512 res。），而不会在ImageNet类别条件生成上造成质量，并且比以前的平行自动审进型至少达到3.4 $ \ tims $延迟。