2025-07-25

Title: Knowledge Abstraction for Knowledge-based Semantic Communication: A Generative Causality Invariant Approach

• Authors: Minh-Duong Nguyen, Quoc-Viet Pham, Nguyen H. Tran, Hoang-Khoi Do, Duy T. Ngo, Won-Joo Hwang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.17784
• Pdf URL: https://arxiv.org/pdf/2507.17784
• Copy Paste: [[2507.17784]] Knowledge Abstraction for Knowledge-based Semantic Communication: A Generative Causality Invariant Approach(https://arxiv.org/abs/2507.17784)
• Keywords: generative
• Abstract: In this study, we design a low-complexity and generalized AI model that can capture common knowledge to improve data reconstruction of the channel decoder for semantic communication. Specifically, we propose a generative adversarial network that leverages causality-invariant learning to extract causal and non-causal representations from the data. Causal representations are invariant and encompass crucial information to identify the data's label. They can encapsulate semantic knowledge and facilitate effective data reconstruction at the receiver. Moreover, the causal mechanism ensures that learned representations remain consistent across different domains, making the system reliable even with users collecting data from diverse domains. As user-collected data evolves over time causing knowledge divergence among users, we design sparse update protocols to improve the invariant properties of the knowledge while minimizing communication overheads. Three key observations were drawn from our empirical evaluations. Firstly, causality-invariant knowledge ensures consistency across different devices despite the diverse training data. Secondly, invariant knowledge has promising performance in classification tasks, which is pivotal for goal-oriented semantic communications. Thirdly, our knowledge-based data reconstruction highlights the robustness of our decoder, which surpasses other state-of-the-art data reconstruction and semantic compression methods in terms of Peak Signal-to-Noise Ratio (PSNR).
• 摘要：在这项研究中，我们设计了一个低复杂性和广义的AI模型，该模型可以捕获常识，以改善通道解码器进行语义通信的数据重建。具体而言，我们提出了一个生成的对抗网络，该网络利用因果关系学习来从数据中提取因果和非因果关系。因果表示是不变的，并且包含关键信息以识别数据的标签。它们可以封装语义知识并促进接收器的有效数据重建。此外，因果机制可确保在不同领域中学习的表示形式保持一致，即使用户从不同领域收集数据也使系统可靠。随着用户收集的数据随着时间的流逝而演变，导致用户之间的知识差异，我们设计了稀疏的更新协议，以提高知识的不变属性，同时最大程度地减少通信开销。从我们的经验评估中得出了三个关键观察。首先，尽管有多样化的培训数据，但因果关系不变知识可确保不同设备的一致性。其次，不变知识在分类任务中具有有希望的表现，这对于面向目标的语义通信至关重要。第三，我们基于知识的数据重建突出了我们解码器的鲁棒性，该解码器超过了其他最先进的数据重建和语义压缩方法，而语义压缩方法的峰值信噪比（PSNR）。

Title: LSDM: LLM-Enhanced Spatio-temporal Diffusion Model for Service-Level Mobile Traffic Prediction

• Authors: Shiyuan Zhang, Tong Li, Zhu Xiao, Hongyang Du, Kaibin Huang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.17795
• Pdf URL: https://arxiv.org/pdf/2507.17795
• Copy Paste: [[2507.17795]] LSDM: LLM-Enhanced Spatio-temporal Diffusion Model for Service-Level Mobile Traffic Prediction(https://arxiv.org/abs/2507.17795)
• Keywords: generative
• Abstract: Service-level mobile traffic prediction for individual users is essential for network efficiency and quality of service enhancement. However, current prediction methods are limited in their adaptability across different urban environments and produce inaccurate results due to the high uncertainty in personal traffic patterns, the lack of detailed environmental context, and the complex dependencies among different network services. These challenges demand advanced modeling techniques that can capture dynamic traffic distributions and rich environmental features. Inspired by the recent success of diffusion models in distribution modeling and Large Language Models (LLMs) in contextual understanding, we propose an LLM-Enhanced Spatio-temporal Diffusion Model (LSDM). LSDM integrates the generative power of diffusion models with the adaptive learning capabilities of transformers, augmented by the ability to capture multimodal environmental information for modeling service-level patterns and dynamics. Extensive evaluations on real-world service-level datasets demonstrate that the model excels in traffic usage predictions, showing outstanding generalization and adaptability. After incorporating contextual information via LLM, the performance improves by at least 2.83% in terms of the coefficient of determination. Compared to models of a similar type, such as CSDI, the root mean squared error can be reduced by at least 8.29%. The code and dataset will be available at: this https URL.
• 摘要：服务级别的单个用户的服务级移动流量预测对于网络效率和服务质量增强至关重要。但是，由于个人交通模式的高度不确定性，缺乏详细的环境环境以及不同网络服务之间的复杂依赖性，目前的预测方法在不同城市环境中的适应性受到限制，并产生不准确的结果。这些挑战需要高级建模技术，可以捕获动态的交通分布和丰富的环境特征。受到扩散模型在分布模型和大型语言模型（LLM）中的最新成功的启发，我们提出了一个LLM增强时空扩散模型（LSDM）。 LSDM将扩散模型的生成力与变压器的自适应学习能力集成在一起，并通过捕获多模式环境信息来建模服务级别的模式和动力学的能力增强。对现实世界服务级数据集的广泛评估表明，该模型在流量使用预测中擅长，显示出杰出的概括和适应性。通过LLM合并上下文信息后，就确定系数而言，绩效至少提高了2.83％。与类似类型的模型（例如CSDI）相比，根平方误差至少可以减少8.29％。代码和数据集将在以下网址提供：此HTTPS URL。

Title: CoCAI: Copula-based Conformal Anomaly Identification for Multivariate Time-Series

• Authors: Nicholas A. Pearson, Francesca Zanello, Davide Russo, Luca Bortolussi, Francesca Cairoli
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2507.17796
• Pdf URL: https://arxiv.org/pdf/2507.17796
• Copy Paste: [[2507.17796]] CoCAI: Copula-based Conformal Anomaly Identification for Multivariate Time-Series(https://arxiv.org/abs/2507.17796)
• Keywords: generative
• Abstract: We propose a novel framework that harnesses the power of generative artificial intelligence and copula-based modeling to address two critical challenges in multivariate time-series analysis: delivering accurate predictions and enabling robust anomaly detection. Our method, Copula-based Conformal Anomaly Identification for Multivariate Time-Series (CoCAI), leverages a diffusion-based model to capture complex dependencies within the data, enabling high quality forecasting. The model's outputs are further calibrated using a conformal prediction technique, yielding predictive regions which are statistically valid, i.e., cover the true target values with a desired confidence level. Starting from these calibrated forecasts, robust outlier detection is performed by combining dimensionality reduction techniques with copula-based modeling, providing a statistically grounded anomaly score. CoCAI benefits from an offline calibration phase that allows for minimal overhead during deployment and delivers actionable results rooted in established theoretical foundations. Empirical tests conducted on real operational data derived from water distribution and sewerage systems confirm CoCAI's effectiveness in accurately forecasting target sequences of data and in identifying anomalous segments within them.
• 摘要：我们提出了一个新颖的框架，该框架利用了生成人工智能和基于Copula的建模的力量，以解决多元时间序列分析中的两个关键挑战：提供准确的预测并实现可靠的异常检测。我们的基于COPULA的方法的多元时间序列（COCAI）的方法利用基于扩散的模型来捕获数据中的复杂依赖性，从而实现了高质量的预测。使用保形预测技术进一步校准了模型的输出，从而产生统计上有效的预测区域，即以所需的置信度覆盖真实的目标值。从这些校准的预测开始，通过将尺寸降低技术与基于Copula的建模相结合，提供了统计扎根的异常评分，从而进行了鲁棒的离群检测。 Cocai受益于离线校准阶段，该阶段允许在部署过程中最小的开销，并提供植根于已建立的理论基础的可行结果。对源自水分布和污水处理系统得出的实际操作数据进行的经验测试证实了Cocai在准确预测数据的目标序列以及识别其中的异常段时的有效性。

Title: GenSelect: A Generative Approach to Best-of-N

• Authors: Shubham Toshniwal, Ivan Sorokin, Aleksander Ficek, Ivan Moshkov, Igor Gitman
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2507.17797
• Pdf URL: https://arxiv.org/pdf/2507.17797
• Copy Paste: [[2507.17797]] GenSelect: A Generative Approach to Best-of-N(https://arxiv.org/abs/2507.17797)
• Keywords: generative
• Abstract: Generative reward models with parallel sampling have enabled effective test-time scaling for reasoning tasks. Current approaches employ pointwise scoring of individual solutions or pairwise comparisons. However, pointwise methods underutilize LLMs' comparative abilities, while pairwise methods scale inefficiently with larger sampling budgets. We introduce GenSelect, where the LLM uses long reasoning to select the best solution among N candidates. This leverages LLMs' comparative strengths while scaling efficiently across parallel sampling budgets. For math reasoning, we demonstrate that reasoning models, such as QwQ and DeepSeek-R1-0528, excel at GenSelect, outperforming existing scoring approaches with simple prompting.
• 摘要：具有并行抽样的生成奖励模型已实现了推理任务的有效测试时间缩放。当前的方法采用单个解决方案或成对比较的指定评分。但是，点的方法不足以利用LLM的比较能力，而成对方法随着较大的采样预算的效率低下。我们介绍了Genselect，其中LLM使用长期推理来选择N候选者的最佳解决方案。这利用LLM的比较优势，同时在平行采样预算中有效地扩展。对于数学推理，我们证明了推理模型，例如QWQ和DeepSeek-R1-0528，在Genselect上出色，并以简单的提示优于现有的评分方法。

Title: Wasserstein GAN-Based Precipitation Downscaling with Optimal Transport for Enhancing Perceptual Realism

• Authors: Kenta Shiraishi, Yuka Muto, Atsushi Okazaki, Shunji Kotsuki
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.17798
• Pdf URL: https://arxiv.org/pdf/2507.17798
• Copy Paste: [[2507.17798]] Wasserstein GAN-Based Precipitation Downscaling with Optimal Transport for Enhancing Perceptual Realism(https://arxiv.org/abs/2507.17798)
• Keywords: generative
• Abstract: High-resolution (HR) precipitation prediction is essential for reducing damage from stationary and localized heavy rainfall; however, HR precipitation forecasts using process-driven numerical weather prediction models remains challenging. This study proposes using Wasserstein Generative Adversarial Network (WGAN) to perform precipitation downscaling with an optimal transport cost. In contrast to a conventional neural network trained with mean squared error, the WGAN generated visually realistic precipitation fields with fine-scale structures even though the WGAN exhibited slightly lower performance on conventional evaluation metrics. The learned critic of WGAN correlated well with human perceptual realism. Case-based analysis revealed that large discrepancies in critic scores can help identify both unrealistic WGAN outputs and potential artifacts in the reference data. These findings suggest that the WGAN framework not only improves perceptual realism in precipitation downscaling but also offers a new perspective for evaluating and quality-controlling precipitation datasets.
• 摘要：高分辨率（HR）降水预测对于减少固定和局部大雨的破坏至关重要；但是，使用过程驱动的数值天气预测模型的人力资源降水预测仍然具有挑战性。这项研究建议使用Wasserstein生成的对抗网络（WGAN）以最佳的运输成本进行降水缩小。与经过平均误差训练的常规神经网络相反，沃甘（Wgan）产生了具有优质结构的视觉逼真的降水场，即使恩格在常规评估指标上表现出较低的性能略低。博学的沃根批评家与人类的感知现实主义息息相关。基于案例的分析表明，评论家分数的巨大差异可以帮助识别参考数据中不现实的wgan输出和潜在的伪像。这些发现表明，智慧框架不仅可以改善降水降级的感知现实主义，而且还提供了一种用于评估和质量控制降水数据集的新观点。

Title: Lumina-mGPT 2.0: Stand-Alone AutoRegressive Image Modeling

• Authors: Yi Xin, Juncheng Yan, Qi Qin, Zhen Li, Dongyang Liu, Shicheng Li, Victor Shea-Jay Huang, Yupeng Zhou, Renrui Zhang, Le Zhuo, Tiancheng Han, Xiaoqing Sun, Siqi Luo, Mengmeng Wang, Bin Fu, Yuewen Cao, Hongsheng Li, Guangtao Zhai, Xiaohong Liu, Yu Qiao, Peng Gao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.17801
• Pdf URL: https://arxiv.org/pdf/2507.17801
• Copy Paste: [[2507.17801]] Lumina-mGPT 2.0: Stand-Alone AutoRegressive Image Modeling(https://arxiv.org/abs/2507.17801)
• Keywords: generation, generative
• Abstract: We present Lumina-mGPT 2.0, a stand-alone, decoder-only autoregressive model that revisits and revitalizes the autoregressive paradigm for high-quality image generation and beyond. Unlike existing approaches that rely on pretrained components or hybrid architectures, Lumina-mGPT 2.0 is trained entirely from scratch, enabling unrestricted architectural design and licensing freedom. It achieves generation quality on par with state-of-the-art diffusion models such as DALL-E 3 and SANA, while preserving the inherent flexibility and compositionality of autoregressive modeling. Our unified tokenization scheme allows the model to seamlessly handle a wide spectrum of tasks-including subject-driven generation, image editing, controllable synthesis, and dense prediction-within a single generative framework. To further boost usability, we incorporate efficient decoding strategies like inference-time scaling and speculative Jacobi sampling to improve quality and speed, respectively. Extensive evaluations on standard text-to-image benchmarks (e.g., GenEval, DPG) demonstrate that Lumina-mGPT 2.0 not only matches but in some cases surpasses diffusion-based models. Moreover, we confirm its multi-task capabilities on the Graph200K benchmark, with the native Lumina-mGPT 2.0 performing exceptionally well. These results position Lumina-mGPT 2.0 as a strong, flexible foundation model for unified multimodal generation. We have released our training details, code, and models at this https URL.
• 摘要：我们提出了Lumina-MGPT 2.0，这是一种独立的，仅解码器的自动回归模型，可重新审视和振兴高质量图像生成及以后的自回归范式。与依赖于验证组件或混合体系结构的现有方法不同，Lumina-Mgpt 2.0完全从头开始训练，从而实现了不受限制的建筑设计和许可自由。它以最先进的扩散模型（例如Dall-E 3和Sana）达到生长质量，同时保留自回归建模的固有灵活性和组成性。我们统一的象征化方案允许该模型无缝处理广泛的任务，包括主题驱动的生成，图像编辑，可控的合成和与单个生成框架的密集预测。为了进一步提高可用性，我们结合了有效的解码策略，例如推理时间缩放和投机性雅各比采样，分别提高质量和速度。对标准文本对图像基准（例如Geneval，DPG）的广泛评估表明，Lumina-MGPT 2.0不仅匹配，而且在某些情况下超过了基于扩散的模型。此外，我们在Graph200K基准测试中确认了其多任务功能，而本机Lumina-MGPT 2.0表现出色。这些结果将Lumina-MGPT 2.0定位为统一多模式生成的强，灵活的基础模型。我们已经在此HTTPS URL上发布了培训详细信息，代码和模型。

Title: SV3.3B: A Sports Video Understanding Model for Action Recognition

• Authors: Sai Varun Kodathala, Yashwanth Reddy Vutukoori, Rakesh Vunnam
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.17844
• Pdf URL: https://arxiv.org/pdf/2507.17844
• Copy Paste: [[2507.17844]] SV3.3B: A Sports Video Understanding Model for Action Recognition(https://arxiv.org/abs/2507.17844)
• Keywords: generation
• Abstract: This paper addresses the challenge of automated sports video analysis, which has traditionally been limited by computationally intensive models requiring server-side processing and lacking fine-grained understanding of athletic movements. Current approaches struggle to capture the nuanced biomechanical transitions essential for meaningful sports analysis, often missing critical phases like preparation, execution, and follow-through that occur within seconds. To address these limitations, we introduce SV3.3B, a lightweight 3.3B parameter video understanding model that combines novel temporal motion difference sampling with self-supervised learning for efficient on-device deployment. Our approach employs a DWT-VGG16-LDA based keyframe extraction mechanism that intelligently identifies the 16 most representative frames from sports sequences, followed by a V-DWT-JEPA2 encoder pretrained through mask-denoising objectives and an LLM decoder fine-tuned for sports action description generation. Evaluated on a subset of the NSVA basketball dataset, SV3.3B achieves superior performance across both traditional text generation metrics and sports-specific evaluation criteria, outperforming larger closed-source models including GPT-4o variants while maintaining significantly lower computational requirements. Our model demonstrates exceptional capability in generating technically detailed and analytically rich sports descriptions, achieving 29.2% improvement over GPT-4o in ground truth validation metrics, with substantial improvements in information density, action complexity, and measurement precision metrics essential for comprehensive athletic analysis. Model Available at this https URL.
• 摘要：本文解决了自动体育视频分析的挑战，传统上，该视频分析受到计算密集型模型的限制，需要服务器端处理以及对运动运动缺乏细粒度的了解。当前的方法难以捕获有意义的运动分析所必需的细微生物力学转变，通常会缺少临界阶段，例如在几秒钟内发生的准备，执行和跟进。为了解决这些限制，我们介绍了SV3.3B，这是一种轻巧的3.3b参数视频理解模型，将新颖的时间运动差异采样与自我监督的学习结合在一起，以实现有效的智障部署。我们的方法采用了基于DWT-VGG16-LDA的钥匙扣提取机制，该机制智能地识别了体育序列中的16个最具代表性的框架，然后是通过掩盖否定目标预测的V-DWT-JEPA2编码器，并通过否定的LLM解码器进行了微调，以进行体育行动描述。 SV3.3B在NSVA篮球数据集的一部分中进行了评估，在传统的文本生成指标和特定于运动的评估标准中都取得了卓越的性能，超过了包括GPT-4O变体（包括GPT-4O变体）的较大封闭式模型，同时保持较低的计算要求。我们的模型表明，在生成技术详细和分析丰富的运动描述方面具有出色的能力，在地面真相验证指标中，与GPT-4O相比，相比具有29.2％的提高，并且信息密度密度，动作复杂性和测量精度度量指标可实现全面运动分析所必需的精确指标。在此HTTPS URL上可用的型号。

Title: Detail++: Training-Free Detail Enhancer for Text-to-Image Diffusion Models

• Authors: Lifeng Chen, Jiner Wang, Zihao Pan, Beier Zhu, Xiaofeng Yang, Chi Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.17853
• Pdf URL: https://arxiv.org/pdf/2507.17853
• Copy Paste: [[2507.17853]] Detail++: Training-Free Detail Enhancer for Text-to-Image Diffusion Models(https://arxiv.org/abs/2507.17853)
• Keywords: generation
• Abstract: Recent advances in text-to-image (T2I) generation have led to impressive visual results. However, these models still face significant challenges when handling complex prompt, particularly those involving multiple subjects with distinct attributes. Inspired by the human drawing process, which first outlines the composition and then incrementally adds details, we propose Detail++, a training-free framework that introduces a novel Progressive Detail Injection (PDI) strategy to address this limitation. Specifically, we decompose a complex prompt into a sequence of simplified sub-prompts, guiding the generation process in stages. This staged generation leverages the inherent layout-controlling capacity of self-attention to first ensure global composition, followed by precise refinement. To achieve accurate binding between attributes and corresponding subjects, we exploit cross-attention mechanisms and further introduce a Centroid Alignment Loss at test time to reduce binding noise and enhance attribute consistency. Extensive experiments on T2I-CompBench and a newly constructed style composition benchmark demonstrate that Detail++ significantly outperforms existing methods, particularly in scenarios involving multiple objects and complex stylistic conditions.
• 摘要：文本对图像（T2i）一代的最新进展导致了令人印象深刻的视觉结果。但是，在处理复杂提示时，这些模型仍然面临重大挑战，尤其是那些涉及多个具有不同属性的主题的挑战。受到人类绘图过程的启发，该过程首先概述了构图，然后逐步添加了细节，我们提出了细节++，这是一个无训练的框架，该框架介绍了一种新型的渐进细节注入（PDI）策略来解决此限制。具体而言，我们将复杂的提示分解为一系列简化的子奖，以分阶段指导生成过程。这一阶段的一代利用了自我注意的固有的布局控制能力首先确保全球构图，然后进行精确的完善。为了在属性和相应受试者之间获得准确的结合，我们利用了交叉注意机制，并在测试时间进一步引入了质心对齐损失，以减少结合噪声并增强属性一致性。关于T2i-Compbench和新构建的样式组成基准的广泛实验表明，细节++显着优于现有方法，尤其是在涉及多个对象和复杂风格条件的情况下。

Title: Towards Facilitated Fairness Assessment of AI-based Skin Lesion Classifiers Through GenAI-based Image Synthesis

• Authors: Ko Watanabe. Stanislav Frolov. Adriano Lucieri. Andreas Dengel
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.17860
• Pdf URL: https://arxiv.org/pdf/2507.17860
• Copy Paste: [[2507.17860]] Towards Facilitated Fairness Assessment of AI-based Skin Lesion Classifiers Through GenAI-based Image Synthesis(https://arxiv.org/abs/2507.17860)
• Keywords: generative
• Abstract: Recent advancements in Deep Learning and its application on the edge hold great potential for the revolution of routine screenings for skin cancers like Melanoma. Along with the anticipated benefits of this technology, potential dangers arise from unforseen and inherent biases. Thus, assessing and improving the fairness of such systems is of utmost importance. A key challenge in fairness assessment is to ensure that the evaluation dataset is sufficiently representative of different Personal Identifiable Information (PII) (sex, age, and race) and other minority groups. Against the backdrop of this challenge, this study leverages the state-of-the-art Generative AI (GenAI) LightningDiT model to assess the fairness of publicly available melanoma classifiers. The results suggest that fairness assessment using highly realistic synthetic data is a promising direction. Yet, our findings indicate that verifying fairness becomes difficult when the melanoma-detection model used for evaluation is trained on data that differ from the dataset underpinning the synthetic images. Nonetheless, we propose that our approach offers a valuable new avenue for employing synthetic data to gauge and enhance fairness in medical-imaging GenAI systems.
• 摘要：深度学习及其在边缘上的应用的最新进展具有巨大的潜力，可以为像黑色素瘤这样的皮肤癌进行常规筛查的革命。除了这项技术的预期益处外，潜在的危险是不可预见的和固有的偏见。因此，评估和改善此类系统的公平性至关重要。公平评估中的一个关键挑战是确保评估数据集充分代表不同的个人可识别信息（PII）（性别，年龄和种族）和其他少数群体。在这项挑战的背景下，这项研究利用了最先进的生成AI（Genai）Lightningdit模型来评估公开可用的黑色素瘤分类器的公平性。结果表明，使用高度现实的合成数据进行公平评估是一个有希望的方向。然而，我们的发现表明，当用于评估的黑色素瘤检测模型对与合成图像的基础的数据集进行培训时，验证公平性变得困难。尽管如此，我们建议我们的方法为使用合成数据来衡量和增强医疗成像Genai系统的公平性提供了宝贵的新途径。

Title: Look the Other Way: Designing 'Positive' Molecules with Negative Data via Task Arithmetic

• Authors: Rıza Özçelik, Sarah de Ruiter, Francesca Grisoni
• Subjects: cs.LG, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2507.17876
• Pdf URL: https://arxiv.org/pdf/2507.17876
• Copy Paste: [[2507.17876]] Look the Other Way: Designing 'Positive' Molecules with Negative Data via Task Arithmetic(https://arxiv.org/abs/2507.17876)
• Keywords: generative
• Abstract: The scarcity of molecules with desirable properties (i.e., 'positive' molecules) is an inherent bottleneck for generative molecule design. To sidestep such obstacle, here we propose molecular task arithmetic: training a model on diverse and abundant negative examples to learn 'property directions' $--$ without accessing any positively labeled data $--$ and moving models in the opposite property directions to generate positive molecules. When analyzed on 20 zero-shot design experiments, molecular task arithmetic generated more diverse and successful designs than models trained on positive molecules. Moreover, we employed molecular task arithmetic in dual-objective and few-shot design tasks. We find that molecular task arithmetic can consistently increase the diversity of designs while maintaining desirable design properties. With its simplicity, data efficiency, and performance, molecular task arithmetic bears the potential to become the $\textit{de-facto}$ transfer learning strategy for de novo molecule design.
• 摘要：具有理想特性（即“阳性”分子）的分子的稀缺是生成分子设计的固有瓶颈。为了避开这种障碍，我们在这里提出了分子任务算术：培训一个模型，以学习“属性方向” $ - $ - $ - $，而无需访问任何标记的数据$ - $ - $ - 在相反的属性方向上移动模型以产生正分子。当对20个零射击设计实验进行分析时，分子任务算术比在正分子上训练的模型更多样化和成功的设计。此外，我们采用了分子任务算术在双目标设计任务和少量设计任务中。我们发现分子任务算术可以始终如一地增加设计的多样性，同时保持理想的设计特性。凭借其简单，数据效率和性能，分子任务算术具有成为从头分子设计的$ \ textit {de-facto} $传递学习策略的潜力。

Title: Fourier Neural Operators for Non-Markovian Processes:Approximation Theorems and Experiments

• Authors: Wonjae Lee, Taeyoung Kim, Hyungbin Park
• Subjects: cs.LG, math.NA
• Abstract URL: https://arxiv.org/abs/2507.17887
• Pdf URL: https://arxiv.org/pdf/2507.17887
• Copy Paste: [[2507.17887]] Fourier Neural Operators for Non-Markovian Processes:Approximation Theorems and Experiments(https://arxiv.org/abs/2507.17887)
• Keywords: generation
• Abstract: This paper introduces an operator-based neural network, the mirror-padded Fourier neural operator (MFNO), designed to learn the dynamics of stochastic systems. MFNO extends the standard Fourier neural operator (FNO) by incorporating mirror padding, enabling it to handle non-periodic inputs. We rigorously prove that MFNOs can approximate solutions of path-dependent stochastic differential equations and Lipschitz transformations of fractional Brownian motions to an arbitrary degree of accuracy. Our theoretical analysis builds on Wong--Zakai type theorems and various approximation techniques. Empirically, the MFNO exhibits strong resolution generalization--a property rarely seen in standard architectures such as LSTMs, TCNs, and DeepONet. Furthermore, our model achieves performance that is comparable or superior to these baselines while offering significantly faster sample path generation than classical numerical schemes.
• 摘要：本文介绍了一个基于操作员的神经网络，镜面的傅立叶神经操作员（MFNO），旨在学习随机系统的动力学。 MFNO通过合并镜填充来扩展标准的傅立叶神经操作员（FNO），从而使其能够处理非周期性输入。我们严格地证明，MFNO可以近似路径依赖性的随机微分方程和LIPSCHITZ的解决方案，将分数Brownian Motions的Lipschitz转换达到任意程度的准确性。我们的理论分析建立在Wong-Zakai类型定理和各种近似技术的基础上。从经验上讲，MFNO表现出强大的分辨率概括，这是在LSTM，TCN和DEADONET等标准体系结构中很少见的。此外，我们的模型可实现与这些基线相当或优越的性能，同时提供的样品路径生成速度明显比经典的数值方案更快。

Title: DiNAT-IR: Exploring Dilated Neighborhood Attention for High-Quality Image Restoration

• Authors: Hanzhou Liu, Binghan Li, Chengkai Liu, Mi Lu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.17892
• Pdf URL: https://arxiv.org/pdf/2507.17892
• Copy Paste: [[2507.17892]] DiNAT-IR: Exploring Dilated Neighborhood Attention for High-Quality Image Restoration(https://arxiv.org/abs/2507.17892)
• Keywords: restoration
• Abstract: Transformers, with their self-attention mechanisms for modeling long-range dependencies, have become a dominant paradigm in image restoration tasks. However, the high computational cost of self-attention limits scalability to high-resolution images, making efficiency-quality trade-offs a key research focus. To address this, Restormer employs channel-wise self-attention, which computes attention across channels instead of spatial dimensions. While effective, this approach may overlook localized artifacts that are crucial for high-quality image restoration. To bridge this gap, we explore Dilated Neighborhood Attention (DiNA) as a promising alternative, inspired by its success in high-level vision tasks. DiNA balances global context and local precision by integrating sliding-window attention with mixed dilation factors, effectively expanding the receptive field without excessive overhead. However, our preliminary experiments indicate that directly applying this global-local design to the classic deblurring task hinders accurate visual restoration, primarily due to the constrained global context understanding within local attention. To address this, we introduce a channel-aware module that complements local attention, effectively integrating global context without sacrificing pixel-level precision. The proposed DiNAT-IR, a Transformer-based architecture specifically designed for image restoration, achieves competitive results across multiple benchmarks, offering a high-quality solution for diverse low-level computer vision problems.
• 摘要：变形金刚具有建模长期依赖性的自我发项机制，已成为图像恢复任务中的主要范式。但是，自我注意力的高计算成本将可扩展性限制为高分辨率图像，从而使效率质量的权衡成为关键的焦点。为了解决这个问题，Restormer采用了渠道自我注意力，该自我注意力跨通道而不是空间维度计算注意力。尽管有效，但这种方法可能会忽略对于高质量图像恢复至关重要的局部工件。为了弥合这一差距，我们探索了扩张的邻里注意力（DINA），这是一种有希望的选择，灵感来自于其在高级视觉任务中的成功。 Dina通过将滑动窗口的关注与混合扩张因子相结合，从而有效地扩大了接受场而没有过多的开销，从而平衡了全球环境和局部精度。但是，我们的初步实验表明，将这种全局本地设计直接应用于经典的脱张任务上，主要是由于本地注意力中的全球环境理解受到限制。为了解决这个问题，我们介绍了一个频道感知的模块，该模块可以补充本地注意力，从而有效地整合了全局上下文，而无需牺牲像素级的精度。拟议的Dinat-IR是一种专门设计用于图像修复的构造，它在多个基准测试中取得了竞争成果，为各种低级计算机视觉问题提供了高质量的解决方案。

Title: Deep learning-aided inverse design of porous metamaterials

• Authors: Phu Thien Nguyen, Yousef Heider, Dennis M. Kochmann, Fadi Aldakheel
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.17907
• Pdf URL: https://arxiv.org/pdf/2507.17907
• Copy Paste: [[2507.17907]] Deep learning-aided inverse design of porous metamaterials(https://arxiv.org/abs/2507.17907)
• Keywords: generation, generative
• Abstract: The ultimate aim of the study is to explore the inverse design of porous metamaterials using a deep learning-based generative framework. Specifically, we develop a property-variational autoencoder (pVAE), a variational autoencoder (VAE) augmented with a regressor, to generate structured metamaterials with tailored hydraulic properties, such as porosity and permeability. While this work uses the lattice Boltzmann method (LBM) to generate intrinsic permeability tensor data for limited porous microstructures, a convolutional neural network (CNN) is trained using a bottom-up approach to predict effective hydraulic properties. This significantly reduces the computational cost compared to direct LBM simulations. The pVAE framework is trained on two datasets: a synthetic dataset of artificial porous microstructures and CT-scan images of volume elements from real open-cell foams. The encoder-decoder architecture of the VAE captures key microstructural features, mapping them into a compact and interpretable latent space for efficient structure-property exploration. The study provides a detailed analysis and interpretation of the latent space, demonstrating its role in structure-property mapping, interpolation, and inverse design. This approach facilitates the generation of new metamaterials with desired properties. The datasets and codes used in this study will be made open-access to support further research.
• 摘要：该研究的最终目的是使用基于深度学习的生成框架探索多孔超材料的逆设计。具体而言，我们开发了一种具有回归器的变异自动编码器（VAE）的属性变量自动编码器（PVAE），以生成具有量身定制的液压性能（例如孔隙率和渗透性）的结构化的超材料。虽然这项工作使用晶格玻尔兹曼方法（LBM）来生成有限多孔微观结构的固有通透性张量数据，但使用自下而上的方法对卷积神经网络（CNN）进行培训，以预测有效的液压特性。与直接LBM模拟相比，这大大降低了计算成本。 PVAE框架在两个数据集上进行了训练：人工多孔微观结构的合成数据集以及来自真实开放式泡沫的体积元素的CT扫描图像。 VAE的编码器解码器结构捕获了关键的微观结构特征，将它们映射到一个紧凑而可解释的潜在空间中，以进行有效的结构 - 培训探索。该研究提供了对潜在空间的详细分析和解释，证明了其在结构 - 特性映射，插值和逆设计中的作用。这种方法促进了具有所需特性的新型超材料的产生。本研究中使用的数据集和代码将被开放访问以支持进一步的研究。

Title: Multimodal Fine-grained Reasoning for Post Quality Evaluation

• Authors: Xiaoxu Guo, Siyan Liang, Yachao Cui, Juxiang Zhou, Lei Wang, Han Cao
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.17934
• Pdf URL: https://arxiv.org/pdf/2507.17934
• Copy Paste: [[2507.17934]] Multimodal Fine-grained Reasoning for Post Quality Evaluation(https://arxiv.org/abs/2507.17934)
• Keywords: quality assessment
• Abstract: Accurately assessing post quality requires complex relational reasoning to capture nuanced topic-post relationships. However, existing studies face three major limitations: (1) treating the task as unimodal categorization, which fails to leverage multimodal cues and fine-grained quality distinctions; (2) introducing noise during deep multimodal fusion, leading to misleading signals; and (3) lacking the ability to capture complex semantic relationships like relevance and comprehensiveness. To address these issues, we propose the Multimodal Fine-grained Topic-post Relational Reasoning (MFTRR) framework, which mimics human cognitive processes. MFTRR reframes post-quality assessment as a ranking task and incorporates multimodal data to better capture quality variations. It consists of two key modules: (1) the Local-Global Semantic Correlation Reasoning Module, which models fine-grained semantic interactions between posts and topics at both local and global levels, enhanced by a maximum information fusion mechanism to suppress noise; and (2) the Multi-Level Evidential Relational Reasoning Module, which explores macro- and micro-level relational cues to strengthen evidence-based reasoning. We evaluate MFTRR on three newly constructed multimodal topic-post datasets and the public Lazada-Home dataset. Experimental results demonstrate that MFTRR significantly outperforms state-of-the-art baselines, achieving up to 9.52% NDCG@3 improvement over the best unimodal method on the Art History dataset.
• 摘要：准确评估后质量需要复杂的关系推理，以捕获细微的主题 - 台阶关系。但是，现有的研究面临三个主要局限性：（1）将任务视为单峰分类，这无法利用多模式提示和细粒度的质量区分； （2）在深层多模式融合过程中引入噪声，导致误导信号； （3）缺乏捕获相关性和全面性等复杂语义关系的能力。为了解决这些问题，我们提出了多模式的细粒主题 - 关系 - 关系推理（MFTRR）框架，该框架模仿了人类的认知过程。 MFTRR将质量后评估作为一项排名任务，并结合了多模式数据以更好地捕获质量变化。它由两个关键模块组成：（1）局部全球语义相关推理模块，该模块模拟了局部和全局级别的帖子和主题之间的细粒语义相互作用，并通过最大的信息融合机制来抑制噪声； （2）多级证据关系推理模块，探索宏观和微观关系线索以加强基于证据的推理。我们评估了三个新建的多模式主题post数据集和公共Lazada-home数据集的MFTRR。实验结果表明，MFTRR显着胜过最先进的基线，比在ART历史数据集上的最佳单峰方法提高了9.52％的NDCG@3改进。

Title: GRR-CoCa: Leveraging LLM Mechanisms in Multimodal Model Architectures

• Authors: Jake R. Patock, Nicole Catherine Lewis, Kevin McCoy, Christina Gomez, Canling Chen, Lorenzo Luzi
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2507.18009
• Pdf URL: https://arxiv.org/pdf/2507.18009
• Copy Paste: [[2507.18009]] GRR-CoCa: Leveraging LLM Mechanisms in Multimodal Model Architectures(https://arxiv.org/abs/2507.18009)
• Keywords: generation, generative
• Abstract: State-of-the-art (SOTA) image and text generation models are multimodal models that have many similarities to large language models (LLMs). Despite achieving strong performances, leading foundational multimodal model architectures frequently lag behind the architectural sophistication of contemporary LLMs. We propose GRR-CoCa, an improved SOTA Contrastive Captioner (CoCa) model that incorporates Gaussian error gated linear units, root mean squared normalization, and rotary positional embedding into the textual decoders and the vision transformer (ViT) encoder. Each architectural modification has been shown to improve model performance in LLMs, but has yet to be adopted in CoCa. We benchmarked GRR-CoCa against Baseline CoCa, a model with the same modified textual decoders but with CoCa's original ViT encoder. We used standard pretraining and fine-tuning workflows to benchmark the models on contrastive and generative tasks. Our GRR-CoCa significantly outperformed Baseline CoCa on the pretraining dataset and three diverse fine-tuning datasets. Pretraining improvements were 27.25% in contrastive loss, 3.71% in perplexity, and 7.15% in CoCa loss. The average fine-tuning improvements were 13.66% in contrastive loss, 5.18% in perplexity, and 5.55% in CoCa loss. We show that GRR-CoCa's modified architecture improves performance and generalization across vision-language domains.
• 摘要：最新的（SOTA）图像和文本生成模型是与大语言模型（LLMS）具有许多相似之处的多模型模型。尽管表现出色，但领先的基础多模型架构经常落后于当代LLM的建筑精致。我们提出了GRR-COCA，这是一种改进的SOTA对比字幕（COCA）模型，该模型结合了高斯错误门控线性单元，均方根平方的归一化以及旋转位置嵌入文本解码器和视觉变压器（VIT）编码器。已经证明，每种架构修饰都可以改善LLM中的模型性能，但在可口可乐中尚未采用。我们对基线可口可乐进行了基准测试，该模型具有相同的修改文本解码器，但带有可口可乐的原始vit编码器。我们使用标准预处理和微调工作流来对比和生成任务进行基准测试模型。我们的GRR-COCA在预处理数据集和三个不同的微调数据集上的表现明显优于基线可口可乐。对比度损失的预处理改善为27.25％，困惑性为3.71％，可口可乐损失为7.15％。对比损失的平均微调改善为13.66％，困惑性为5.18％，可口可乐损失为5.55％。我们表明，GRR-COCA的修改架构可改善视觉域之间的性能和概括。

Title: Celeb-DF++: A Large-scale Challenging Video DeepFake Benchmark for Generalizable Forensics

• Authors: Yuezun Li, Delong Zhu, Xinjie Cui, Siwei Lyu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.18015
• Pdf URL: https://arxiv.org/pdf/2507.18015
• Copy Paste: [[2507.18015]] Celeb-DF++: A Large-scale Challenging Video DeepFake Benchmark for Generalizable Forensics(https://arxiv.org/abs/2507.18015)
• Keywords: generation
• Abstract: The rapid advancement of AI technologies has significantly increased the diversity of DeepFake videos circulating online, posing a pressing challenge for \textit{generalizable forensics}, \ie, detecting a wide range of unseen DeepFake types using a single model. Addressing this challenge requires datasets that are not only large-scale but also rich in forgery diversity. However, most existing datasets, despite their scale, include only a limited variety of forgery types, making them insufficient for developing generalizable detection methods. Therefore, we build upon our earlier Celeb-DF dataset and introduce {Celeb-DF++}, a new large-scale and challenging video DeepFake benchmark dedicated to the generalizable forensics challenge. Celeb-DF++ covers three commonly encountered forgery scenarios: Face-swap (FS), Face-reenactment (FR), and Talking-face (TF). Each scenario contains a substantial number of high-quality forged videos, generated using a total of 22 various recent DeepFake methods. These methods differ in terms of architectures, generation pipelines, and targeted facial regions, covering the most prevalent DeepFake cases witnessed in the wild. We also introduce evaluation protocols for measuring the generalizability of 24 recent detection methods, highlighting the limitations of existing detection methods and the difficulty of our new dataset.
• 摘要：AI技术的快速发展显着提高了在线流传的Deepfake视频的多样性，对\ textit {prencyable forensics}（\ ie）提出了紧迫的挑战，\ ie \ ie，使用单个模型检测了广泛的未见深击类型。应对这一挑战需要不仅大规模，而且富裕的伪造多样性的数据集。但是，大多数现有的数据集虽然规模尺度，但仅包括有限的伪造类型，因此不足以开发可推广的检测方法。因此，我们基于较早的Celeb-DF数据集并介绍{Celeb-df ++}，这是一种新的大规模且具有挑战性的视频DeepFake基准，专门针对可推广的取证挑战。 Celeb-DF ++涵盖了三个通常遇到的伪造场景：面部折扣（FS），面对面练习（FR）和Talking-Face（TF）。每种情况都包含大量高质量的锻造视频，这些视频总共使用了22种各种最近的深击方法生成。这些方法在建筑，发电管道和有针对性的面部区域方面有所不同，涵盖了野外最普遍的深层捕捞案例。我们还介绍了评估协议，以衡量24种最近检测方法的普遍性，从而强调了现有检测方法的局限性和我们新数据集的难度。

Title: High-fidelity 3D Gaussian Inpainting: preserving multi-view consistency and photorealistic details

• Authors: Jun Zhou, Dinghao Li, Nannan Li, Mingjie Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.18023
• Pdf URL: https://arxiv.org/pdf/2507.18023
• Copy Paste: [[2507.18023]] High-fidelity 3D Gaussian Inpainting: preserving multi-view consistency and photorealistic details(https://arxiv.org/abs/2507.18023)
• Keywords: restoration
• Abstract: Recent advancements in multi-view 3D reconstruction and novel-view synthesis, particularly through Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS), have greatly enhanced the fidelity and efficiency of 3D content creation. However, inpainting 3D scenes remains a challenging task due to the inherent irregularity of 3D structures and the critical need for maintaining multi-view consistency. In this work, we propose a novel 3D Gaussian inpainting framework that reconstructs complete 3D scenes by leveraging sparse inpainted views. Our framework incorporates an automatic Mask Refinement Process and region-wise Uncertainty-guided Optimization. Specifically, we refine the inpainting mask using a series of operations, including Gaussian scene filtering and back-projection, enabling more accurate localization of occluded regions and realistic boundary restoration. Furthermore, our Uncertainty-guided Fine-grained Optimization strategy, which estimates the importance of each region across multi-view images during training, alleviates multi-view inconsistencies and enhances the fidelity of fine details in the inpainted results. Comprehensive experiments conducted on diverse datasets demonstrate that our approach outperforms existing state-of-the-art methods in both visual quality and view consistency.
• 摘要：多视图3D重建和新型视图合成的最新进展，尤其是通过神经辐射场（NERF）和3D高斯分裂（3DGS），可以极大地提高了3D含量创建的忠诚度和效率。但是，由于3D结构的固有不规则性以及维持多视图一致性的关键需求，介入3D场景仍然是一项具有挑战性的任务。在这项工作中，我们提出了一个新颖的3D高斯indpaining框架，该框架通过利用稀疏的成分观点来重建完整的3D场景。我们的框架结合了自动掩码完善过程和智慧不确定性引导的优化。具体而言，我们使用一系列操作（包括高斯场景过滤和反向注射）来完善介入面罩，从而更准确地定位了遮挡区域和现实的边界恢复。此外，我们的不确定性引导的细颗粒优化策略估计了培训期间每个区域跨多视图图像的重要性，促使多视图矛盾并提高了未成熟结果中细节的忠诚度。在各种数据集上进行的全面实验表明，我们的方法在视觉质量和视图一致性方面都优于现有的最新方法。

Title: Enhancing Scene Transition Awareness in Video Generation via Post-Training

• Authors: Hanwen Shen, Jiajie Lu, Yupeng Cao, Xiaonan Yang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.18046
• Pdf URL: https://arxiv.org/pdf/2507.18046
• Copy Paste: [[2507.18046]] Enhancing Scene Transition Awareness in Video Generation via Post-Training(https://arxiv.org/abs/2507.18046)
• Keywords: generation
• Abstract: Recent advances in AI-generated video have shown strong performance on \emph{text-to-video} tasks, particularly for short clips depicting a single scene. However, current models struggle to generate longer videos with coherent scene transitions, primarily because they cannot infer when a transition is needed from the prompt. Most open-source models are trained on datasets consisting of single-scene video clips, which limits their capacity to learn and respond to prompts requiring multiple scenes. Developing scene transition awareness is essential for multi-scene generation, as it allows models to identify and segment videos into distinct clips by accurately detecting transitions. To address this, we propose the \textbf{Transition-Aware Video} (TAV) dataset, which consists of preprocessed video clips with multiple scene transitions. Our experiment shows that post-training on the \textbf{TAV} dataset improves prompt-based scene transition understanding, narrows the gap between required and generated scenes, and maintains image quality.
• 摘要：AI生成的视频的最新进展显示出在\ emph {text-to-video}任务上的出色表现，特别是对于描绘单个场景的短剪辑。但是，当前的模型很难通过连贯的场景过渡生成更长的视频，这主要是因为它们无法从提示中进行过渡时推断。大多数开源模型都在由单场景视频剪辑组成的数据集上进行培训，该数据集限制了他们学习和响应需要多个场景提示的能力。开发场景过渡意识对于多场景生成至关重要，因为它允许模型通过准确检测过渡来识别和将视频识别为不同的剪辑。为了解决这个问题，我们提出了\ textbf {transition-ware-ware视频}（tav）数据集，该数据集由具有多个场景过渡的预处理视频剪辑组成。我们的实验表明，\ textbf {Tav}数据集对培训的培训可改善基于提示的场景过渡的理解，缩小所需场景和生成场景之间的差距，并保持图像质量。

Title: BokehDiff: Neural Lens Blur with One-Step Diffusion

• Authors: Chengxuan Zhu, Qingnan Fan, Qi Zhang, Jinwei Chen, Huaqi Zhang, Chao Xu, Boxin Shi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.18060
• Pdf URL: https://arxiv.org/pdf/2507.18060
• Copy Paste: [[2507.18060]] BokehDiff: Neural Lens Blur with One-Step Diffusion(https://arxiv.org/abs/2507.18060)
• Keywords: generative
• Abstract: We introduce BokehDiff, a novel lens blur rendering method that achieves physically accurate and visually appealing outcomes, with the help of generative diffusion prior. Previous methods are bounded by the accuracy of depth estimation, generating artifacts in depth discontinuities. Our method employs a physics-inspired self-attention module that aligns with the image formation process, incorporating depth-dependent circle of confusion constraint and self-occlusion effects. We adapt the diffusion model to the one-step inference scheme without introducing additional noise, and achieve results of high quality and fidelity. To address the lack of scalable paired data, we propose to synthesize photorealistic foregrounds with transparency with diffusion models, balancing authenticity and scene diversity.
• 摘要：我们介绍了Bokehdiff，这是一种新颖的镜头模糊渲染方法，在生成扩散的帮助下，可以实现身体上准确且具有视觉吸引力的结果。先前的方法受深度估计的准确性，在深度不连续性中产生伪影。我们的方法采用了一个由物理启发的自我发场模块，该模块与图像形成过程保持一致，并结合了混淆约束和自我封度效应的深度依赖性圆圈。我们将扩散模型调整到一步推理方案中，而无需引入额外的噪声，并实现了高质量和忠诚的结果。为了解决缺乏可扩展的配对数据，我们建议通过透明度与扩散模型，平衡真实性和场景多样性合成逼真的前景。

Title: Adapting Large VLMs with Iterative and Manual Instructions for Generative Low-light Enhancement

• Authors: Xiaoran Sun, Liyan Wang, Cong Wang, Yeying Jin, Kin-man Lam, Zhixun Su, Yang Yang, Jinshan Pan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.18064
• Pdf URL: https://arxiv.org/pdf/2507.18064
• Copy Paste: [[2507.18064]] Adapting Large VLMs with Iterative and Manual Instructions for Generative Low-light Enhancement(https://arxiv.org/abs/2507.18064)
• Keywords: restoration, generation, generative
• Abstract: Most existing low-light image enhancement (LLIE) methods rely on pre-trained model priors, low-light inputs, or both, while neglecting the semantic guidance available from normal-light images. This limitation hinders their effectiveness in complex lighting conditions. In this paper, we propose VLM-IMI, a novel framework that leverages large vision-language models (VLMs) with iterative and manual instructions (IMIs) for LLIE. VLM-IMI incorporates textual descriptions of the desired normal-light content as enhancement cues, enabling semantically informed restoration. To effectively integrate cross-modal priors, we introduce an instruction prior fusion module, which dynamically aligns and fuses image and text features, promoting the generation of detailed and semantically coherent outputs. During inference, we adopt an iterative and manual instruction strategy to refine textual instructions, progressively improving visual quality. This refinement enhances structural fidelity, semantic alignment, and the recovery of fine details under extremely low-light conditions. Extensive experiments across diverse scenarios demonstrate that VLM-IMI outperforms state-of-the-art methods in both quantitative metrics and perceptual quality. The source code is available at this https URL.
• 摘要：大多数现有的低光图像增强（LLIE）方法依赖于预先训练的模型先验，低光输入或两者兼而有之，同时忽略了正常光图像可用的语义指导。这种限制阻碍了它们在复杂的照明条件下的有效性。在本文中，我们提出了VLM-IMI，这是一个新颖的框架，该框架利用Llie的迭代和手动说明（IMIS）利用大型视觉模型（VLM）。 VLM-IMI将所需的正常光含量的文本描述作为增强提示，从而实现了语义知情的恢复。为了有效地整合跨模式先验，我们引入了一个指令先验的融合模块，该模块会动态对齐和融合图像和文本特征，从而促进详细和语义上连贯的输出的生成。在推断期间，我们采用迭代和手动指导策略来完善文本说明，逐步提高视觉质量。这种完善增强了结构保真度，语义对准以及在极低光线条件下的细节的恢复。跨不同场景的广泛实验表明，VLM-IMI在定量指标和感知质量方面都优于最先进的方法。源代码可在此HTTPS URL上找到。

Title: T2VWorldBench: A Benchmark for Evaluating World Knowledge in Text-to-Video Generation

• Authors: Yubin Chen, Xuyang Guo, Zhenmei Shi, Zhao Song, Jiahao Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.18107
• Pdf URL: https://arxiv.org/pdf/2507.18107
• Copy Paste: [[2507.18107]] T2VWorldBench: A Benchmark for Evaluating World Knowledge in Text-to-Video Generation(https://arxiv.org/abs/2507.18107)
• Keywords: generation
• Abstract: Text-to-video (T2V) models have shown remarkable performance in generating visually reasonable scenes, while their capability to leverage world knowledge for ensuring semantic consistency and factual accuracy remains largely understudied. In response to this challenge, we propose T2VWorldBench, the first systematic evaluation framework for evaluating the world knowledge generation abilities of text-to-video models, covering 6 major categories, 60 subcategories, and 1,200 prompts across a wide range of domains, including physics, nature, activity, culture, causality, and object. To address both human preference and scalable evaluation, our benchmark incorporates both human evaluation and automated evaluation using vision-language models (VLMs). We evaluated the 10 most advanced text-to-video models currently available, ranging from open source to commercial models, and found that most models are unable to understand world knowledge and generate truly correct videos. These findings point out a critical gap in the capability of current text-to-video models to leverage world knowledge, providing valuable research opportunities and entry points for constructing models with robust capabilities for commonsense reasoning and factual generation.
• 摘要：文本到视频（T2V）模型在产生视觉合理的场景时表现出了出色的性能，同时，它们利用世界知识来确保语义一致性和事实准确性的能力在很大程度上被忽略了。为了应对这一挑战，我们提出了T2VWorldBench，这是评估文本到视频模型的世界知识产生能力的第一个系统评估框架，涵盖了6个主要类别，60个子类别和1,200个提示，包括各种领域，包括物理学，自然，活动，文化，因果关系，物质和物体。为了解决人类偏好和可扩展评估，我们的基准测试同时使用视觉语言模型（VLM）进行了人类评估和自动化评估。我们评估了目前可用的10个最先进的文本到视频模型，从开源到商业模型，发现大多数模型无法理解世界知识并生成真正正确的视频。这些发现指出了当前文本到视频模型的能力，以利用世界知识的能力，为建造具有强大能力的强大推理和事实生成的模型提供了宝贵的研究机会和切入点。

Title: Degradation-Consistent Learning via Bidirectional Diffusion for Low-Light Image Enhancement

• Authors: Jinhong He, Minglong Xue, Zhipu Liu, Mingliang Zhou, Aoxiang Ning, Palaiahnakote Shivakumara
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2507.18144
• Pdf URL: https://arxiv.org/pdf/2507.18144
• Copy Paste: [[2507.18144]] Degradation-Consistent Learning via Bidirectional Diffusion for Low-Light Image Enhancement(https://arxiv.org/abs/2507.18144)
• Keywords: restoration, generation, generative
• Abstract: Low-light image enhancement aims to improve the visibility of degraded images to better align with human visual perception. While diffusion-based methods have shown promising performance due to their strong generative capabilities. However, their unidirectional modelling of degradation often struggles to capture the complexity of real-world degradation patterns, leading to structural inconsistencies and pixel misalignments. To address these challenges, we propose a bidirectional diffusion optimization mechanism that jointly models the degradation processes of both low-light and normal-light images, enabling more precise degradation parameter matching and enhancing generation quality. Specifically, we perform bidirectional diffusion-from low-to-normal light and from normal-to-low light during training and introduce an adaptive feature interaction block (AFI) to refine feature representation. By leveraging the complementarity between these two paths, our approach imposes an implicit symmetry constraint on illumination attenuation and noise distribution, facilitating consistent degradation learning and improving the models ability to perceive illumination and detail degradation. Additionally, we design a reflection-aware correction module (RACM) to guide color restoration post-denoising and suppress overexposed regions, ensuring content consistency and generating high-quality images that align with human visual perception. Extensive experiments on multiple benchmark datasets demonstrate that our method outperforms state-of-the-art methods in both quantitative and qualitative evaluations while generalizing effectively to diverse degradation scenarios. Code at this https URL
• 摘要：低光图像增强旨在提高降解图像的可见性，以更好地与人类视觉感知保持一致。尽管基于扩散的方法由于其强大的生成能力而显示出令人鼓舞的性能。但是，他们的降解的单向建模通常努力捕获现实世界降解模式的复杂性，从而导致结构上的不一致和像素不对准。为了应对这些挑战，我们提出了一种双向扩散优化机制，该机制共同模拟了弱光和正常光图像的降解过程，从而实现了更精确的降级参数匹配和增强发电质量。具体而言，我们在训练过程中进行双向扩散，从低正常光到正常光，并从正常光到低光，并引入自适应特征相互作用块（AFI）以完善特征表示。通过利用这两条路径之间的互补性，我们的方法对照明衰减和噪声分布施加了隐式对称性限制，从而促进了一致的降级学习，并提高了模型能够感知照明和细节退化的模型。此外，我们设计了一个反射感知的校正模块（RACM），以指导降落后的颜色修复并抑制过度曝光区域，确保内容一致性并产生与人类视觉感知相符的高质量图像。在多个基准数据集上进行的广泛实验表明，我们的方法在定量和定性评估中均优于最先进的方法，同时有效地将其推广到各种降级方案。此https url中的代码

Title: TeEFusion: Blending Text Embeddings to Distill Classifier-Free Guidance

• Authors: Minghao Fu, Guo-Hua Wang, Xiaohao Chen, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.18192
• Pdf URL: https://arxiv.org/pdf/2507.18192
• Copy Paste: [[2507.18192]] TeEFusion: Blending Text Embeddings to Distill Classifier-Free Guidance(https://arxiv.org/abs/2507.18192)
• Keywords: generation
• Abstract: Recent advances in text-to-image synthesis largely benefit from sophisticated sampling strategies and classifier-free guidance (CFG) to ensure high-quality generation. However, CFG's reliance on two forward passes, especially when combined with intricate sampling algorithms, results in prohibitively high inference costs. To address this, we introduce TeEFusion (\textbf{Te}xt \textbf{E}mbeddings \textbf{Fusion}), a novel and efficient distillation method that directly incorporates the guidance magnitude into the text embeddings and distills the teacher model's complex sampling strategy. By simply fusing conditional and unconditional text embeddings using linear operations, TeEFusion reconstructs the desired guidance without adding extra parameters, simultaneously enabling the student model to learn from the teacher's output produced via its sophisticated sampling approach. Extensive experiments on state-of-the-art models such as SD3 demonstrate that our method allows the student to closely mimic the teacher's performance with a far simpler and more efficient sampling strategy. Consequently, the student model achieves inference speeds up to 6$\times$ faster than the teacher model, while maintaining image quality at levels comparable to those obtained through the teacher's complex sampling approach. The code is publicly available at \href{this https URL}{this http URL}.
• 摘要：文本到图像合成的最新进展在很大程度上受益于复杂的抽样策略和无分类器指导（CFG），以确保高质量的生成。但是，CFG对两个正向通行证的依赖，尤其是在与复杂的采样算法结合使用时，导致了高度高的推理成本。为了解决这个问题，我们介绍了Teefusion（\ textbf {te} XT \ textbf {e} mbeddings \ textbf {fusion}），这是一种新颖而有效的蒸馏方法，将指导幅度直接结合到教师模型的复杂采样策略中。通过简单地使用线性操作融合条件和无条件的文本嵌入，Teefusion可以在不添加额外参数的情况下重建所需的指导，同时使学生模型通过其复杂的采样方法从教师的输出中学习。对SD3等最先进模型的广泛实验表明，我们的方法使学生可以通过更简单，更有效的采样策略密切模仿教师的表现。因此，学生模型的推理速度比教师模型快6 $ \ times $，同时将图像质量保持在与教师的复杂抽样方法相当的水平上。该代码在\ href {this HTTPS url} {this HTTP url}上公开可用。

Title: Improving Large Vision-Language Models' Understanding for Field Data

• Authors: Xiaomei Zhang, Hanyu Zheng, Xiangyu Zhu, Jinghuan Wei, Junhong Zou, Zhen Lei, Zhaoxiang Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.18311
• Pdf URL: https://arxiv.org/pdf/2507.18311
• Copy Paste: [[2507.18311]] Improving Large Vision-Language Models' Understanding for Field Data(https://arxiv.org/abs/2507.18311)
• Keywords: generation
• Abstract: Large Vision-Language Models (LVLMs) have shown impressive capabilities across a range of tasks that integrate visual and textual understanding, such as image captioning and visual question answering. These models are trained on large-scale image and video datasets paired with text, enabling them to bridge visual perception and natural language processing. However, their application to scientific domains, especially in interpreting complex field data commonly used in the natural sciences, remains underexplored. In this work, we introduce FieldLVLM, a novel framework designed to improve large vision-language models' understanding of field data. FieldLVLM consists of two main components: a field-aware language generation strategy and a data-compressed multimodal model tuning. The field-aware language generation strategy leverages a special-purpose machine learning pipeline to extract key physical features from field data, such as flow classification, Reynolds number, and vortex patterns. This information is then converted into structured textual descriptions that serve as a dataset. The data-compressed multimodal model tuning focuses on LVLMs with these generated datasets, using a data compression strategy to reduce the complexity of field inputs and retain only the most informative values. This ensures compatibility with the models language decoder and guides its learning more effectively. Experimental results on newly proposed benchmark datasets demonstrate that FieldLVLM significantly outperforms existing methods in tasks involving scientific field data. Our findings suggest that this approach opens up new possibilities for applying large vision-language models to scientific research, helping bridge the gap between large models and domain-specific discovery.
• 摘要：大型视觉模型（LVLM）在一系列任务中显示出令人印象深刻的功能，这些任务集成了视觉和文本理解，例如图像字幕和视觉问题回答。这些模型经过大规模图像和视频数据集的培训，使它们能够桥接视觉感知和自然语言处理。但是，它们在科学领域中的应用，尤其是在解释自然科学中常用的复杂领域数据中，仍然没有得到充实的态度。在这项工作中，我们介绍了FieldLVLM，这是一个新颖的框架，旨在提高大型视觉模型对现场数据的理解。 fieldLVLM由两个主要组成部分组成：一种现场感知语言生成策略和数据压缩的多模式模型调整。现场意见的语言生成策略利用专用机器学习管道从现场数据（例如流量分类，雷诺数和涡流模式）中提取关键的物理特征。然后将这些信息转换为结构化的文本描述，这些描述用作数据集。使用数据压缩策略来减少现场输入的复杂性并仅保留最有用的值，因此使用这些生成的数据集的数据压缩的多模式调整将重点放在LVLM上。这样可以确保与模型语言解码器的兼容性，并更有效地指导其学习。对新提出的基准数据集的实验结果表明，FieldLVLM在涉及科学现场数据的任务中的现有方法显着优于现有方法。我们的发现表明，这种方法为将大型视觉模型应用于科学研究开辟了新的可能性，有助于弥合大型模型和特定领域的发现之间的差距。

Title: MVG4D: Image Matrix-Based Multi-View and Motion Generation for 4D Content Creation from a Single Image

• Authors: Xiaotian Chen, DongFu Yin, Fei Richard Yu, Xuanchen Li, Xinhao Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.18371
• Pdf URL: https://arxiv.org/pdf/2507.18371
• Copy Paste: [[2507.18371]] MVG4D: Image Matrix-Based Multi-View and Motion Generation for 4D Content Creation from a Single Image(https://arxiv.org/abs/2507.18371)
• Keywords: generation, generative
• Abstract: Advances in generative modeling have significantly enhanced digital content creation, extending from 2D images to complex 3D and 4D scenes. Despite substantial progress, producing high-fidelity and temporally consistent dynamic 4D content remains a challenge. In this paper, we propose MVG4D, a novel framework that generates dynamic 4D content from a single still image by combining multi-view synthesis with 4D Gaussian Splatting (4D GS). At its core, MVG4D employs an image matrix module that synthesizes temporally coherent and spatially diverse multi-view images, providing rich supervisory signals for downstream 3D and 4D reconstruction. These multi-view images are used to optimize a 3D Gaussian point cloud, which is further extended into the temporal domain via a lightweight deformation network. Our method effectively enhances temporal consistency, geometric fidelity, and visual realism, addressing key challenges in motion discontinuity and background degradation that affect prior 4D GS-based methods. Extensive experiments on the Objaverse dataset demonstrate that MVG4D outperforms state-of-the-art baselines in CLIP-I, PSNR, FVD, and time efficiency. Notably, it reduces flickering artifacts and sharpens structural details across views and time, enabling more immersive AR/VR experiences. MVG4D sets a new direction for efficient and controllable 4D generation from minimal inputs.
• 摘要：生成建模的进步显着增强了数字内容的创建，从2D图像扩展到复杂的3D和4D场景。尽管取得了长足的进步，但产生高保真性和时间一致的动态4D内容仍然是一个挑战。在本文中，我们提出了MVG4D，这是一个新颖的框架，通过将多视图合成与4D高斯脱落（4D GS）相结合（4D GS）来生成动态的4D内容。 MVG4D以其核心使用图像矩阵模块，该模块综合了时间相干和空间上不同的多视图图像，为下游3D和4D重建提供了丰富的监督信号。这些多视图图像用于优化3D高斯点云，该云通过轻质变形网络进一步扩展到时间域。我们的方法有效地增强了时间的一致性，几何保真度和视觉现实主义，从而解决了影响基于4D GS方法的运动不连续性和背景降解中的关键挑战。 OBJAVERSE数据集的广泛实验表明，MVG4D在夹子I，PSNR，FVD和时间效率中的表现优于最先进的基准。值得注意的是，它减少了闪烁的文物，并在视图和时间范围内培养结构细节，从而实现了更多的身临其境的AR/VR体验。 MVG4D为从最小输入的高效和可控制的4D生成设定了一个新的方向。

Title: A Comprehensive Review of Diffusion Models in Smart Agriculture: Progress, Applications, and Challenges

• Authors: Xing Hua, Haodong Chen, Qianqian Duan, Danfeng Hong, Ruijiao Li, Huiliang Shang, Linghua Jiang, Haima Yang, Dawei Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.18376
• Pdf URL: https://arxiv.org/pdf/2507.18376
• Copy Paste: [[2507.18376]] A Comprehensive Review of Diffusion Models in Smart Agriculture: Progress, Applications, and Challenges(https://arxiv.org/abs/2507.18376)
• Keywords: generation, generative
• Abstract: With the global population growing and arable land resources becoming increasingly scarce,smart agriculture and precision agriculture have emerged as key directions for the future ofagricultural this http URL intelligence (AI) technologies, particularly deep learning models, have found widespread applications in areas such as crop monitoring and pest detection. As an emerging generative model, diffusion models have shown significant promise in tasks like agricultural image processing, data augmentation, and remote sensing. Compared to traditional generative adversarial networks (GANs), diffusion models offer superior training stability and generation quality, effectively addressing challenges such as limited agricultural data and imbalanced image samples. This paper reviews the latest advancements in the application of diffusion models in agriculture, focusing on their potential in crop pest and disease detection, remote sensing image enhancement, crop growth prediction, and agricultural resource management. Experimental results demonstrate that diffusion models significantly improve model accuracy and robustness in data augmentation, image generation, and denoising, especially in complex environments. Despite challenges related to computational efficiency and generalization capabilities, diffusion models are expected to play an increasingly important role in smart and precision agriculture as technology advances, providing substantial support for the sustainable development of global agriculture.
• 摘要：随着全球人口的增长和可耕地的土地资源越来越稀缺，智能农业和精确农业已经成为质业未来的关键方向，这种HTTP URL智能（AI）技术，尤其是深度学习模型，在农作物监测和PEST监测等领域中发现了广泛的应用。作为一种新兴生成模型，扩散模型在诸如农业图像处理，数据增强和遥感之类的任务中表现出了巨大的希望。与传统的生成对抗网络（GAN）相比，扩散模型提供了卓越的训练稳定性和发电质量，从而有效地应对诸如有限的农业数据和不平衡图像样本等挑战。本文回顾了扩散模型在农业中的应用中的最新进步，重点是它们在作物有害生物和疾病检测中的潜力，遥感图像增强，作物生长预测和农业资源管理。实验结果表明，扩散模型可显着提高数据增强，图像产生和降解性的模型准确性和鲁棒性，尤其是在复杂的环境中。尽管与计算效率和概括能力相关的挑战，但随着技术的发展，扩散模型有望在智能和精确农业中发挥越来越重要的作用，为全球农业的可持续发展提供了大量支持。

Title: Towards Consistent Long-Term Pose Generation

• Authors: Yayuan Li, Filippos Bellos, Jason Corso
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.18382
• Pdf URL: https://arxiv.org/pdf/2507.18382
• Copy Paste: [[2507.18382]] Towards Consistent Long-Term Pose Generation(https://arxiv.org/abs/2507.18382)
• Keywords: generation
• Abstract: Current approaches to pose generation rely heavily on intermediate representations, either through two-stage pipelines with quantization or autoregressive models that accumulate errors during inference. This fundamental limitation leads to degraded performance, particularly in long-term pose generation where maintaining temporal coherence is crucial. We propose a novel one-stage architecture that directly generates poses in continuous coordinate space from minimal context - a single RGB image and text description - while maintaining consistent distributions between training and inference. Our key innovation is eliminating the need for intermediate representations or token-based generation by operating directly on pose coordinates through a relative movement prediction mechanism that preserves spatial relationships, and a unified placeholder token approach that enables single-forward generation with identical behavior during training and inference. Through extensive experiments on Penn Action and First-Person Hand Action Benchmark (F-PHAB) datasets, we demonstrate that our approach significantly outperforms existing quantization-based and autoregressive methods, especially in long-term generation scenarios.
• 摘要：当前的姿势生成方法在很大程度上依赖于中间表示，要么通过具有量化的两阶段管道或自回旋模型，这些模型会在推理过程中累积错误。这种基本的局限性导致了降解的性能，特别是在长期姿势产生中，维持时间连贯性至关重要。我们提出了一种新颖的单阶段体系结构，该结构直接从最小情况下 - 单个RGB图像和文本描述中直接生成姿势，同时保持训练和推理之间的一致分布。我们的关键创新是通过通过相对运动预测机制直接在姿势坐标上操作，从而消除了中间表示或基于令牌的生成的需求，该预测机制保留了空间关系，并在训练和推理过程中具有相同的行为行为，从而使统一的占位持有人代币方法具有单一的产生。通过对Penn Action和第一人称手动作基准（F-PHAB）数据集进行的广泛实验，我们证明我们的方法显着优于现有的基于量化的和自动回应方法，尤其是在长期生成方案中。

Title: Iwin Transformer: Hierarchical Vision Transformer using Interleaved Windows

• Authors: Simin Huo, Ning Li
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.18405
• Pdf URL: https://arxiv.org/pdf/2507.18405
• Copy Paste: [[2507.18405]] Iwin Transformer: Hierarchical Vision Transformer using Interleaved Windows(https://arxiv.org/abs/2507.18405)
• Keywords: generation
• Abstract: We introduce Iwin Transformer, a novel position-embedding-free hierarchical vision transformer, which can be fine-tuned directly from low to high resolution, through the collaboration of innovative interleaved window attention and depthwise separable convolution. This approach uses attention to connect distant tokens and applies convolution to link neighboring tokens, enabling global information exchange within a single module, overcoming Swin Transformer's limitation of requiring two consecutive blocks to approximate global attention. Extensive experiments on visual benchmarks demonstrate that Iwin Transformer exhibits strong competitiveness in tasks such as image classification (87.4 top-1 accuracy on ImageNet-1K), semantic segmentation and video action recognition. We also validate the effectiveness of the core component in Iwin as a standalone module that can seamlessly replace the self-attention module in class-conditional image generation. The concepts and methods introduced by the Iwin Transformer have the potential to inspire future research, like Iwin 3D Attention in video generation. The code and models are available at this https URL.
• 摘要：我们介绍了Iwin Transformer，这是一种新型的无位置插入的层次视觉变压器，可以通过创新的交织的窗户注意力和深度分离卷积的协作，直接从低分辨率到高分辨率进行微调。这种方法使用注意力连接遥远的令牌并应用卷积来链接相邻的令牌，从而在单个模块中实现了全局信息交换，从而克服了Swin Transformer的限制，即需要两个连续的块以近似全球关注。对视觉基准测试的广泛实验表明，IWIN变压器在图像分类（Imagenet-1K上的87.4 Top-1精度），语义分割和视频动作识别等任务中表现出强大的竞争力。我们还验证了伊温（Iwin）中核心成分的有效性为独立模块，该模块可以无缝替代类条件图像生成中的自我发项模块。 IWIN Transformer引入的概念和方法具有激发未来研究的潜力，例如Iwin 3D在视频生成中的关注。代码和模型可在此HTTPS URL上找到。

Title: PDB-Eval: An Evaluation of Large Multimodal Models for Description and Explanation of Personalized Driving Behavior

• Authors: Junda Wu, Jessica Echterhoff, Kyungtae Han, Amr Abdelraouf, Rohit Gupta, Julian McAuley
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.18447
• Pdf URL: https://arxiv.org/pdf/2507.18447
• Copy Paste: [[2507.18447]] PDB-Eval: An Evaluation of Large Multimodal Models for Description and Explanation of Personalized Driving Behavior(https://arxiv.org/abs/2507.18447)
• Keywords: generative
• Abstract: Understanding a driver's behavior and intentions is important for potential risk assessment and early accident prevention. Safety and driver assistance systems can be tailored to individual drivers' behavior, significantly enhancing their effectiveness. However, existing datasets are limited in describing and explaining general vehicle movements based on external visual evidence. This paper introduces a benchmark, PDB-Eval, for a detailed understanding of Personalized Driver Behavior, and aligning Large Multimodal Models (MLLMs) with driving comprehension and reasoning. Our benchmark consists of two main components, PDB-X and PDB-QA. PDB-X can evaluate MLLMs' understanding of temporal driving scenes. Our dataset is designed to find valid visual evidence from the external view to explain the driver's behavior from the internal view. To align MLLMs' reasoning abilities with driving tasks, we propose PDB-QA as a visual explanation question-answering task for MLLM instruction fine-tuning. As a generic learning task for generative models like MLLMs, PDB-QA can bridge the domain gap without harming MLLMs' generalizability. Our evaluation indicates that fine-tuning MLLMs on fine-grained descriptions and explanations can effectively bridge the gap between MLLMs and the driving domain, which improves zero-shot performance on question-answering tasks by up to 73.2%. We further evaluate the MLLMs fine-tuned on PDB-X in Brain4Cars' intention prediction and AIDE's recognition tasks. We observe up to 12.5% performance improvements on the turn intention prediction task in Brain4Cars, and consistent performance improvements up to 11.0% on all tasks in AIDE.
• 摘要：了解驾驶员的行为和意图对于潜在的风险评估和早期预防事故非常重要。安全和驱动程序辅助系统可以针对单个驾驶员的行为量身定制，从而大大提高了其有效性。但是，现有数据集在描述和解释基于外部视觉证据的通用车辆运动方面受到限制。本文介绍了一个基准，即PDB-eval，以详细了解个性化的驾驶员行为，并将大型多模型模型（MLLM）与驾驶理解和推理保持一致。我们的基准由两个主要组件PDB-X和PDB-QA组成。 PDB-X可以评估MLLM对时间驾驶场景的理解。我们的数据集旨在从外部视图中找到有效的视觉证据，以从内部视图解释驾驶员的行为。为了使MLLMS的推理能力与驾驶任务相结合，我们建议PDB-QA作为MLLM指令微调的视觉解释提问任务。作为MLLM等生成模型的一项通用学习任务，PDB-QA可以弥合域间隙而不会损害MLLM的可推广性。我们的评估表明，对细粒度描述和解释的微调MLLM可以有效地弥合MLLM和驾驶领域之间的差距，从而提高了提问任务上的零击性能高达73.2％。我们进一步评估了Brain4CARS意图预测和助手的识别任务中对PDB-X的MLLM微调。我们在Brain4Car的转弯意向预测任务上观察到高达12.5％的绩效提高，并且在助手中所有任务的一致性提高了高达11.0％。

Title: CRUISE: Cooperative Reconstruction and Editing in V2X Scenarios using Gaussian Splatting

• Authors: Haoran Xu, Saining Zhang, Peishuo Li, Baijun Ye, Xiaoxue Chen, Huan-ang Gao, Jv Zheng, Xiaowei Song, Ziqiao Peng, Run Miao, Jinrang Jia, Yifeng Shi, Guangqi Yi, Hang Zhao, Hao Tang, Hongyang Li, Kaicheng Yu, Hao Zhao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.18473
• Pdf URL: https://arxiv.org/pdf/2507.18473
• Copy Paste: [[2507.18473]] CRUISE: Cooperative Reconstruction and Editing in V2X Scenarios using Gaussian Splatting(https://arxiv.org/abs/2507.18473)
• Keywords: generation
• Abstract: Vehicle-to-everything (V2X) communication plays a crucial role in autonomous driving, enabling cooperation between vehicles and infrastructure. While simulation has significantly contributed to various autonomous driving tasks, its potential for data generation and augmentation in V2X scenarios remains underexplored. In this paper, we introduce CRUISE, a comprehensive reconstruction-and-synthesis framework designed for V2X driving environments. CRUISE employs decomposed Gaussian Splatting to accurately reconstruct real-world scenes while supporting flexible editing. By decomposing dynamic traffic participants into editable Gaussian representations, CRUISE allows for seamless modification and augmentation of driving scenes. Furthermore, the framework renders images from both ego-vehicle and infrastructure views, enabling large-scale V2X dataset augmentation for training and evaluation. Our experimental results demonstrate that: 1) CRUISE reconstructs real-world V2X driving scenes with high fidelity; 2) using CRUISE improves 3D detection across ego-vehicle, infrastructure, and cooperative views, as well as cooperative 3D tracking on the V2X-Seq benchmark; and 3) CRUISE effectively generates challenging corner cases.
• 摘要：车辆到所有（V2X）的通信在自动驾驶中起着至关重要的作用，从而使车辆和基础设施之间的合作能够合作。虽然模拟显着促进了各种自主驾驶任务，但在V2X方案中，其数据生成和增强的潜力仍然没有被忽视。在本文中，我们介绍了Cruise，这是为V2X驾驶环境设计的全面重建和合成框架。 Cruise采用分解的高斯分裂来准确地重建现实世界的场景，同时支持灵活的编辑。通过将动态的交通参与者分解为可编辑的高斯表示形式，Cruise可以无缝修改和增强驾驶场景。此外，该框架还从自我车辆和基础设施视图中提供图像，从而使大规模的V2X数据集扩大进行培训和评估。我们的实验结果表明：1）Cruise以高保真度重建了现实世界中的V2X驾驶场景； 2）使用Cruise改善了跨自我车辆，基础设施和合作视图的3D检测，以及在V2X-Seq基准上进行的合作3D跟踪； 3）巡航有效地产生了具有挑战性的角案例。

Title: COT-AD: Cotton Analysis Dataset

• Authors: Akbar Ali, Mahek Vyas, Soumyaratna Debnath, Chanda Grover Kamra, Jaidev Sanjay Khalane, Reuben Shibu Devanesan, Indra Deep Mastan, Subramanian Sankaranarayanan, Pankaj Khanna, Shanmuganathan Raman
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.18532
• Pdf URL: https://arxiv.org/pdf/2507.18532
• Copy Paste: [[2507.18532]] COT-AD: Cotton Analysis Dataset(https://arxiv.org/abs/2507.18532)
• Keywords: restoration, generative
• Abstract: This paper presents COT-AD, a comprehensive Dataset designed to enhance cotton crop analysis through computer vision. Comprising over 25,000 images captured throughout the cotton growth cycle, with 5,000 annotated images, COT-AD includes aerial imagery for field-scale detection and segmentation and high-resolution DSLR images documenting key diseases. The annotations cover pest and disease recognition, vegetation, and weed analysis, addressing a critical gap in cotton-specific agricultural datasets. COT-AD supports tasks such as classification, segmentation, image restoration, enhancement, deep generative model-based cotton crop synthesis, and early disease management, advancing data-driven crop management
• 摘要：本文介绍了COT-AD，这是一个综合数据集，旨在通过计算机视觉增强棉花作物分析。在整个棉花生长周期中捕获的25,000张图像，带有5,000张带注释的图像，包括用于现场尺度检测和分割的空中图像以及记录关键疾病的高分辨率DSLR图像。注释涵盖了害虫和疾病识别，植被和杂草分析，以解决棉花特异性农业数据集的临界差距。 COT-AD支持分类，细分，图像修复，增强，基于模型的棉花作物合成和早期疾病管理等任务，推进数据驱动的作物管理

Title: Elucidating the Design Space of Arbitrary-Noise-Based Diffusion Models

• Authors: Xingyu Qiu, Mengying Yang, Xinghua Ma, Dong Liang, Yuzhen Li, Fanding Li, Gongning Luo, Wei Wang, Kuanquan Wang, Shuo Li
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.18534
• Pdf URL: https://arxiv.org/pdf/2507.18534
• Copy Paste: [[2507.18534]] Elucidating the Design Space of Arbitrary-Noise-Based Diffusion Models(https://arxiv.org/abs/2507.18534)
• Keywords: restoration
• Abstract: EDM elucidates the unified design space of diffusion models, yet its fixed noise patterns restricted to pure Gaussian noise, limit advancements in image restoration. Our study indicates that forcibly injecting Gaussian noise corrupts the degraded images, overextends the image transformation distance, and increases restoration complexity. To address this problem, our proposed EDA Elucidates the Design space of Arbitrary-noise-based diffusion models. Theoretically, EDA expands the freedom of noise pattern while preserving the original module flexibility of EDM, with rigorous proof that increased noise complexity incurs no additional computational overhead during restoration. EDA is validated on three typical tasks: MRI bias field correction (global smooth noise), CT metal artifact reduction (global sharp noise), and natural image shadow removal (local boundary-aware noise). With only 5 sampling steps, EDA outperforms most task-specific methods and achieves state-of-the-art performance in bias field correction and shadow removal.
• 摘要：EDM阐明了扩散模型的统一设计空间，但其固定噪声模式限于纯高斯噪声，限制了图像恢复的进步。我们的研究表明，强行注射高斯噪声会破坏降解的图像，过度扩展图像转换距离并增加恢复复杂性。为了解决这个问题，我们提出的EDA阐明了基于任意噪声的扩散模型的设计空间。从理论上讲，EDA在保留EDM的原始模块灵活性的同时扩大了噪声模式的自由度，并具有严格的证据，表明噪声复杂性增加会导致恢复过程中没有额外的计算开销。 EDA在三个典型任务上得到了验证：MRI偏置场校正（全球平滑噪声），CT金属伪像减少（全局尖锐噪声）和自然图像阴影去除（局部边界感知噪声）。只有5个采样步骤，EDA胜过大多数特定于任务的方法，并在偏置场校正和阴影去除方面实现了最先进的性能。

Title: TTS-VAR: A Test-Time Scaling Framework for Visual Auto-Regressive Generation

• Authors: Zhekai Chen, Ruihang Chu, Yukang Chen, Shiwei Zhang, Yujie Wei, Yingya Zhang, Xihui Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.18537
• Pdf URL: https://arxiv.org/pdf/2507.18537
• Copy Paste: [[2507.18537]] TTS-VAR: A Test-Time Scaling Framework for Visual Auto-Regressive Generation(https://arxiv.org/abs/2507.18537)
• Keywords: generation
• Abstract: Scaling visual generation models is essential for real-world content creation, yet requires substantial training and computational expenses. Alternatively, test-time scaling has garnered growing attention due to resource efficiency and promising performance. In this work, we present TTS-VAR, the first general test-time scaling framework for visual auto-regressive (VAR) models, modeling the generation process as a path searching problem. To dynamically balance computational efficiency with exploration capacity, we first introduce an adaptive descending batch size schedule throughout the causal generation process. Besides, inspired by VAR's hierarchical coarse-to-fine multi-scale generation, our framework integrates two key components: (i) At coarse scales, we observe that generated tokens are hard for evaluation, possibly leading to erroneous acceptance of inferior samples or rejection of superior samples. Noticing that the coarse scales contain sufficient structural information, we propose clustering-based diversity search. It preserves structural variety through semantic feature clustering, enabling later selection on samples with higher potential. (ii) In fine scales, resampling-based potential selection prioritizes promising candidates using potential scores, which are defined as reward functions incorporating multi-scale generation history. Experiments on the powerful VAR model Infinity show a notable 8.7% GenEval score improvement (from 0.69 to 0.75). Key insights reveal that early-stage structural features effectively influence final quality, and resampling efficacy varies across generation scales. Code is available at this https URL.
• 摘要：扩展视觉生成模型对于实际创建现实世界的内容至关重要，但需要大量的培训和计算费用。另外，由于资源效率和有希望的性能，测试时间的扩展已吸引了人们日益增长的关注。在这项工作中，我们提出了TTS-VAR，这是视觉自动回归（VAR）模型的第一个通用测试时间缩放框架，将生成过程建模为路径搜索问题。为了动态地平衡计算效率与勘探能力，我们首先在整个因果生成过程中引入了自适应下降批量尺寸时间表。此外，受到VAR分层的粗到1个多尺度生成的启发，我们的框架集成了两个关键组成部分：（i）在粗尺度上，我们观察到，生成的代币很难评估，可能导致对下样品的错误接受或拒绝上级样品。注意到粗尺度包含足够的结构信息，我们提出了基于聚类的多样性搜索。它通过语义特征聚类来保留结构变化，从而使后来在具有较高潜力的样品上进行选择。 （ii）在细大的情况下，基于基于潜在的潜在选择优先考虑有前途的候选人使用潜在的分数，这些分数被定义为结合了多尺度生成历史的奖励函数。强大的VAR模型无穷大的实验显示出显着的8.7％的Geneval评分提高（从0.69到0.75）。关键见解表明，早期结构特征有效地影响了最终质量，并且重新采样功效随着一代量表而异。代码可在此HTTPS URL上找到。

Title: Demystify Protein Generation with Hierarchical Conditional Diffusion Models

• Authors: Zinan Ling, Yi Shi, Da Yan, Yang Zhou, Bo Hui
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.18603
• Pdf URL: https://arxiv.org/pdf/2507.18603
• Copy Paste: [[2507.18603]] Demystify Protein Generation with Hierarchical Conditional Diffusion Models(https://arxiv.org/abs/2507.18603)
• Keywords: generation
• Abstract: Generating novel and functional protein sequences is critical to a wide range of applications in biology. Recent advancements in conditional diffusion models have shown impressive empirical performance in protein generation tasks. However, reliable generations of protein remain an open research question in de novo protein design, especially when it comes to conditional diffusion models. Considering the biological function of a protein is determined by multi-level structures, we propose a novel multi-level conditional diffusion model that integrates both sequence-based and structure-based information for efficient end-to-end protein design guided by specified functions. By generating representations at different levels simultaneously, our framework can effectively model the inherent hierarchical relations between different levels, resulting in an informative and discriminative representation of the generated protein. We also propose a Protein-MMD, a new reliable evaluation metric, to evaluate the quality of generated protein with conditional diffusion models. Our new metric is able to capture both distributional and functional similarities between real and generated protein sequences while ensuring conditional consistency. We experiment with the benchmark datasets, and the results on conditional protein generation tasks demonstrate the efficacy of the proposed generation framework and evaluation metric.
• 摘要：生成新颖和功能性蛋白质序列对于在生物学中的广泛应用至关重要。有条件扩散模型的最新进展显示出在蛋白质生成任务中令人印象深刻的经验表现。但是，可靠的蛋白质几代仍然是从头蛋白质设计中的一个开放研究问题，尤其是在有条件扩散模型方面。考虑到蛋白质的生物学功能是由多级结构确定的，我们提出了一个新型的多级条件扩散模型，该模型同时集成了基于序列的信息和基于结构的信息，以通过指定功能指导的有效端到端蛋白质设计。通过同时在不同级别生成表示形式，我们的框架可以有效地对不同级别之间的固有层次关系进行建模，从而导致生成的蛋白质的信息和歧视性表示。我们还提出了一种新的可靠评估度量蛋白-MMD，以评估有条件扩散模型的蛋白质质量。我们的新指标能够捕获实际蛋白序列和生成的蛋白质序列之间的分布和功能相似性，同时确保有条件的一致性。我们尝试了基准数据集，并且有条件蛋白质生成任务的结果证明了提出的生成框架和评估度量的功效。

Title: SynC: Synthetic Image Caption Dataset Refinement with One-to-many Mapping for Zero-shot Image Captioning

• Authors: Si-Woo Kim, MinJu Jeon, Ye-Chan Kim, Soeun Lee, Taewhan Kim, Dong-Jin Kim
• Subjects: cs.CV, cs.AI, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.18616
• Pdf URL: https://arxiv.org/pdf/2507.18616
• Copy Paste: [[2507.18616]] SynC: Synthetic Image Caption Dataset Refinement with One-to-many Mapping for Zero-shot Image Captioning(https://arxiv.org/abs/2507.18616)
• Keywords: generation
• Abstract: Zero-shot Image Captioning (ZIC) increasingly utilizes synthetic datasets generated by text-to-image (T2I) models to mitigate the need for costly manual annotation. However, these T2I models often produce images that exhibit semantic misalignments with their corresponding input captions (e.g., missing objects, incorrect attributes), resulting in noisy synthetic image-caption pairs that can hinder model training. Existing dataset pruning techniques are largely designed for removing noisy text in web-crawled data. However, these methods are ill-suited for the distinct challenges of synthetic data, where captions are typically well-formed, but images may be inaccurate representations. To address this gap, we introduce SynC, a novel framework specifically designed to refine synthetic image-caption datasets for ZIC. Instead of conventional filtering or regeneration, SynC focuses on reassigning captions to the most semantically aligned images already present within the synthetic image pool. Our approach employs a one-to-many mapping strategy by initially retrieving multiple relevant candidate images for each caption. We then apply a cycle-consistency-inspired alignment scorer that selects the best image by verifying its ability to retrieve the original caption via image-to-text retrieval. Extensive evaluations demonstrate that SynC consistently and significantly improves performance across various ZIC models on standard benchmarks (MS-COCO, Flickr30k, NoCaps), achieving state-of-the-art results in several scenarios. SynC offers an effective strategy for curating refined synthetic data to enhance ZIC.
• 摘要：零拍图像字幕（ZIC）越来越多地利用由文本到图像（T2I）模型生成的合成数据集，以减轻对昂贵的手动注释的需求。但是，这些T2I模型通常会产生具有语义未对准的图像，其相应的输入字幕（例如缺失对象，错误属性），从而导致嘈杂的合成图像捕获对，可以阻碍模型训练。现有的数据集修剪技术在很大程度上设计用于删除网络爬行数据中的嘈杂文本。但是，这些方法不适合合成数据的独特挑战，其中字幕通常是构成良好的，但图像可能是不准确的表示。为了解决这一差距，我们引入了Sync，这是一个专门设计的新型框架，旨在完善ZIC的合成图像捕获数据集。 Sync不是传统的过滤或再生，而是重点是将字幕重新分配到合成图像池中已经存在的最具语义对齐图像。我们的方法通过最初为每个标题检索多个相关候选图像，采用一对多的映射策略。然后，我们应用一个受周期性启发的对齐得分手，该得分手通过验证其通过图像到文本检索来检索原始字幕的能力来选择最佳图像。广泛的评估表明，在标准基准（MS-Coco，Flickr30k，nocaps）上，同步始终如一，显着提高了各种ZIC模型的性能，在几种情况下实现了最先进的结果。同步提供了一种有效的策略，用于策划精制的合成数据以增强ZIC。

Title: 3D Software Synthesis Guided by Constraint-Expressive Intermediate Representation

• Authors: Shuqing Li, Anson Y. Lam, Yun Peng, Wenxuan Wang, Michael R. Lyu
• Subjects: cs.CV, cs.AI, cs.MM, cs.SE
• Abstract URL: https://arxiv.org/abs/2507.18625
• Pdf URL: https://arxiv.org/pdf/2507.18625
• Copy Paste: [[2507.18625]] 3D Software Synthesis Guided by Constraint-Expressive Intermediate Representation(https://arxiv.org/abs/2507.18625)
• Keywords: generation
• Abstract: Graphical user interface (UI) software has undergone a fundamental transformation from traditional two-dimensional (2D) desktop/web/mobile interfaces to spatial three-dimensional (3D) environments. While existing work has made remarkable success in automated 2D software generation, such as HTML/CSS and mobile app interface code synthesis, the generation of 3D software still remains under-explored. Current methods for 3D software generation usually generate the 3D environments as a whole and cannot modify or control specific elements in the software. Furthermore, these methods struggle to handle the complex spatial and semantic constraints inherent in the real world. To address the challenges, we present Scenethesis, a novel requirement-sensitive 3D software synthesis approach that maintains formal traceability between user specifications and generated 3D software. Scenethesis is built upon ScenethesisLang, a domain-specific language that serves as a granular constraint-aware intermediate representation (IR) to bridge natural language requirements and executable 3D software. It serves both as a comprehensive scene description language enabling fine-grained modification of 3D software elements and as a formal constraint-expressive specification language capable of expressing complex spatial constraints. By decomposing 3D software synthesis into stages operating on ScenethesisLang, Scenethesis enables independent verification, targeted modification, and systematic constraint satisfaction. Our evaluation demonstrates that Scenethesis accurately captures over 80% of user requirements and satisfies more than 90% of hard constraints while handling over 100 constraints simultaneously. Furthermore, Scenethesis achieves a 42.8% improvement in BLIP-2 visual evaluation scores compared to the state-of-the-art method.
• 摘要：图形用户界面（UI）软件已经经历了从传统的二维（2D）桌面/Web/移动接口到空间三维（3D）环境的基本转换。尽管现有的工作在自动2D软件生成中取得了显着成功，例如HTML/CSS和移动应用接口代码综合，但3D软件的生成仍然尚未探索。 3D软件生成的当前方法通常会生成整个3D环境，并且无法修改或控制软件中的特定元素。此外，这些方法难以处理现实世界中固有的复杂空间和语义约束。为了应对挑战，我们提出了一种新型需求敏感的3D软件综合方法，该方法可在用户规格和生成的3D软件之间保持正式的可追溯性。 SceneThesis是建立在SceneThestheslang上的，这是一种特定于领域的语言，可作为颗粒状约束的中间表示（IR），以桥接自然语言要求和可执行的3D软件。它既用作全面的场景说明语言，可以对3D软件元素进行细粒度的修改，又可以作为一种正式的约束规范语言，能够表达复杂的空间约束。通过将3D软件合成分解为在场景上运行的阶段，场景可以实现独立的验证，有针对性的修改和系统的约束满意度。我们的评估表明，场景精确捕获了80％以上的用户需求，并同时处理100多个约束时满足了90％以上的硬约束。此外，与最先进的方法相比，BLIP-2视觉评估得分的景观可提高42.8％。

Title: Captain Cinema: Towards Short Movie Generation

• Authors: Junfei Xiao, Ceyuan Yang, Lvmin Zhang, Shengqu Cai, Yang Zhao, Yuwei Guo, Gordon Wetzstein, Maneesh Agrawala, Alan Yuille, Lu Jiang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.18634
• Pdf URL: https://arxiv.org/pdf/2507.18634
• Copy Paste: [[2507.18634]] Captain Cinema: Towards Short Movie Generation(https://arxiv.org/abs/2507.18634)
• Keywords: generation
• Abstract: We present Captain Cinema, a generation framework for short movie generation. Given a detailed textual description of a movie storyline, our approach firstly generates a sequence of keyframes that outline the entire narrative, which ensures long-range coherence in both the storyline and visual appearance (e.g., scenes and characters). We refer to this step as top-down keyframe planning. These keyframes then serve as conditioning signals for a video synthesis model, which supports long context learning, to produce the spatio-temporal dynamics between them. This step is referred to as bottom-up video synthesis. To support stable and efficient generation of multi-scene long narrative cinematic works, we introduce an interleaved training strategy for Multimodal Diffusion Transformers (MM-DiT), specifically adapted for long-context video data. Our model is trained on a specially curated cinematic dataset consisting of interleaved data pairs. Our experiments demonstrate that Captain Cinema performs favorably in the automated creation of visually coherent and narrative consistent short movies in high quality and efficiency. Project page: this https URL
• 摘要：我们介绍了Cinema上尉，这是短电影一代的一代框架。鉴于电影故事情节的详细文本描述，我们的方法首先生成了一系列概述整个叙述的序列，从而确保了故事情节和视觉外观（例如，场景和角色）的远距离连贯性。我们将此步骤称为自上而下的密钥帧规划。然后，这些关键帧充当视频综合模型的条件信号，该模型支持长上下文学习，以产生它们之间的时空动力学。此步骤称为自下而上的视频综合。为了支持多场叙事叙事电影作品的稳定而有效的生成，我们引入了多模式扩散变压器（MM-DIT）的交织训练策略，专门针对长篇文化视频数据。我们的模型在由交织数据对组成的特殊策划的电影数据集上进行了训练。我们的实验表明，Cinema船长在自动创作以高质量和效率的视觉连贯和叙事的短片创作中表现出色。项目页面：此HTTPS URL