2025-05-12

Title: From Events to Enhancement: A Survey on Event-Based Imaging Technologies

• Authors: Yunfan Lu, Xiaogang Xu, Pengteng Li, Yusheng Wang, Yi Cui, Huizai Yao, Hui Xiong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05488
• Pdf URL: https://arxiv.org/pdf/2505.05488
• Copy Paste: [[2505.05488]] From Events to Enhancement: A Survey on Event-Based Imaging Technologies(https://arxiv.org/abs/2505.05488)
• Keywords: generation
• Abstract: Event cameras offering high dynamic range and low latency have emerged as disruptive technologies in imaging. Despite growing research on leveraging these benefits for different imaging tasks, a comprehensive study of recently advances and challenges are still lacking. This limits the broader understanding of how to utilize events in universal imaging applications. In this survey, we first introduce a physical model and the characteristics of different event sensors as the foundation. Following this, we highlight the advancement and interaction of image/video enhancement tasks with events. Additionally, we explore advanced tasks, which capture richer light information with events, \eg~light field estimation, multi-view generation, and photometric. Finally, we discuss new challenges and open questions offering a perspective for this rapidly evolving field. More continuously updated resources are at this link: this https URL
• 摘要：提供高动态范围和低潜伏期的事件摄像机已成为成像中的破坏性技术。尽管越来越多地利用这些好处来完成不同的成像任务，但仍缺乏对最近进步和挑战的全面研究。这限制了对如何利用通用成像应用中事件的广泛理解。在这项调查中，我们首先介绍了一个物理模型和不同事件传感器作为基础的特征。在此之后，我们重点介绍了图像/视频增强任务与事件的进步和互动。此外，我们探索了高级任务，这些任务捕获了带有事件的富裕光信息，例如光场估计，多视图生成和光度法。最后，我们讨论了新的挑战和开放问题，为这个快速发展的领域提供了观点。此链接更加连续更新的资源：此HTTPS URL

Title: Learning 3D Persistent Embodied World Models

• Authors: Siyuan Zhou, Yilun Du, Yuncong Yang, Lei Han, Peihao Chen, Dit-Yan Yeung, Chuang Gan
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2505.05495
• Pdf URL: https://arxiv.org/pdf/2505.05495
• Copy Paste: [[2505.05495]] Learning 3D Persistent Embodied World Models(https://arxiv.org/abs/2505.05495)
• Keywords: generation
• Abstract: The ability to simulate the effects of future actions on the world is a crucial ability of intelligent embodied agents, enabling agents to anticipate the effects of their actions and make plans accordingly. While a large body of existing work has explored how to construct such world models using video models, they are often myopic in nature, without any memory of a scene not captured by currently observed images, preventing agents from making consistent long-horizon plans in complex environments where many parts of the scene are partially observed. We introduce a new persistent embodied world model with an explicit memory of previously generated content, enabling much more consistent long-horizon simulation. During generation time, our video diffusion model predicts RGB-D video of the future observations of the agent. This generation is then aggregated into a persistent 3D map of the environment. By conditioning the video model on this 3D spatial map, we illustrate how this enables video world models to faithfully simulate both seen and unseen parts of the world. Finally, we illustrate the efficacy of such a world model in downstream embodied applications, enabling effective planning and policy learning.
• 摘要：模拟未来行动对世界的影响的能力是智能体现的代理人的关键能力，使代理人能够预测其行动的影响并相应地制定计划。尽管大量现有作品探索了如何使用视频模型来构建这样的世界模型，但它们通常是近视的，而没有任何记忆，而没有当前观察到的图像未捕获的场景，从而阻止了代理在复杂的环境中制定一致的长距离计划，在这些环境中，许多部分都被部分观察到了部分场景。我们引入了一种新的持久性体现世界模型，并具有对先前生成的内容的明确记忆，从而实现了更加一致的长匹马模拟。在一代期间，我们的视频扩散模型预测了代理商未来观察结果的RGB-D视频。然后将这一代人汇总成环境的持久3D图。通过在这张3D空间地图上调节视频模型，我们说明了如何使视频世界模型忠实地模拟世界各地的各个部分。最后，我们说明了这种世界模型在下游体现的应用程序中的功效，从而实现了有效的计划和政策学习。

Title: Preliminary Explorations with GPT-4o(mni) Native Image Generation

• Authors: Pu Cao, Feng Zhou, Junyi Ji, Qingye Kong, Zhixiang Lv, Mingjian Zhang, Xuekun Zhao, Siqi Wu, Yinghui Lin, Qing Song, Lu Yang
• Subjects: cs.CV, cs.AI, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.05501
• Pdf URL: https://arxiv.org/pdf/2505.05501
• Copy Paste: [[2505.05501]] Preliminary Explorations with GPT-4o(mni) Native Image Generation(https://arxiv.org/abs/2505.05501)
• Keywords: generation
• Abstract: Recently, the visual generation ability by GPT-4o(mni) has been unlocked by OpenAI. It demonstrates a very remarkable generation capability with excellent multimodal condition understanding and varied task instructions. In this paper, we aim to explore the capabilities of GPT-4o across various tasks. Inspired by previous study, we constructed a task taxonomy along with a carefully curated set of test samples to conduct a comprehensive qualitative test. Benefiting from GPT-4o's powerful multimodal comprehension, its image-generation process demonstrates abilities surpassing those of traditional image-generation tasks. Thus, regarding the dimensions of model capabilities, we evaluate its performance across six task categories: traditional image generation tasks, discriminative tasks, knowledge-based generation, commonsense-based generation, spatially-aware image generation, and temporally-aware image generation. These tasks not only assess the quality and conditional alignment of the model's outputs but also probe deeper into GPT-4o's understanding of real-world concepts. Our results reveal that GPT-4o performs impressively well in general-purpose synthesis tasks, showing strong capabilities in text-to-image generation, visual stylization, and low-level image processing. However, significant limitations remain in its ability to perform precise spatial reasoning, instruction-grounded generation, and consistent temporal prediction. Furthermore, when faced with knowledge-intensive or domain-specific scenarios, such as scientific illustrations or mathematical plots, the model often exhibits hallucinations, factual errors, or structural inconsistencies. These findings suggest that while GPT-4o marks a substantial advancement in unified multimodal generation, there is still a long way to go before it can be reliably applied to professional or safety-critical domains.
• 摘要：最近，Openai解锁了GPT-4O（MNI）的视觉产生能力。它表现出非常出色的生成能力，具有出色的多模式条件理解和各种任务指令。在本文中，我们旨在探讨各种任务中GPT-4O的功能。受到先前研究的启发，我们构建了一项任务分类法以及一组精心策划的测试样本，以进行全面的定性测试。受益于GPT-4O强大的多模式理解，其图像生成过程表明了能力超过传统图像生成任务的能力。因此，关于模型功能的维度，我们评估了其跨六个任务类别的性能：传统的图像生成任务，歧视性任务，基于知识的生成，基于常识的生成，空间感知的图像生成以及时间感知的图像生成。这些任务不仅评估了模型输出的质量和条件对齐，而且还对GPT-4O对现实世界概念的理解进行了更深入的探讨。我们的结果表明，GPT-4O在通用综合任务中表现出色，显示出很强的文本到图像生成，视觉风格和低级图像处理的能力。然而，执行精确的空间推理，指导的产生和一致的时间预测的能力仍然存在显着局限性。此外，当面对知识密集型或特定领域的场景（例如科学插图或数学图）时，该模型通常会表现出幻觉，事实错误或结构上的不一致。这些发现表明，尽管GPT-4O标志着统一的多模式生成的重大进步，但在可靠地应用于专业或安全至关重要的领域之前，还有很长的路要走。

Title: Apply Hierarchical-Chain-of-Generation to Complex Attributes Text-to-3D Generation

• Authors: Yiming Qin, Zhu Xu, Yang Liu
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.05505
• Pdf URL: https://arxiv.org/pdf/2505.05505
• Copy Paste: [[2505.05505]] Apply Hierarchical-Chain-of-Generation to Complex Attributes Text-to-3D Generation(https://arxiv.org/abs/2505.05505)
• Keywords: generation
• Abstract: Recent text-to-3D models can render high-quality assets, yet they still stumble on objects with complex attributes. The key obstacles are: (1) existing text-to-3D approaches typically lift text-to-image models to extract semantics via text encoders, while the text encoder exhibits limited comprehension ability for long descriptions, leading to deviated cross-attention focus, subsequently wrong attribute binding in generated results. (2) Occluded object parts demand a disciplined generation order and explicit part disentanglement. Though some works introduce manual efforts to alleviate the above issues, their quality is unstable and highly reliant on manual information. To tackle above problems, we propose a automated method Hierarchical-Chain-of-Generation (HCoG). It leverages a large language model to decompose the long description into blocks representing different object parts, and orders them from inside out according to occlusions, forming a hierarchical chain. Within each block we first coarsely create components, then precisely bind attributes via target-region localization and corresponding 3D Gaussian kernel optimization. Between blocks, we introduce Gaussian Extension and Label Elimination to seamlessly generate new parts by extending new Gaussian kernels, re-assigning semantic labels, and eliminating unnecessary kernels, ensuring that only relevant parts are added without disrupting previously optimized parts. Experiments confirm that HCoG yields structurally coherent, attribute-faithful 3D objects with complex attributes. The code is available at this https URL .
• 摘要：最近的文本到3D模型可以渲染高质量的资产，但它们仍然偶然发现具有复杂属性的对象。关键障碍是：（1）现有的文本到3D方法通常会提起文本对图像模型，以通过文本编码提取语义，而文本编码器对长描述表现出有限的理解能力，从而导致偏离的交叉探测焦点，随后在生成的结果中绑定了错误的属性。 （2）封闭的对象零件需要纪律处分的生成顺序和明确的零件分离。尽管一些作品介绍了手动努力来减轻上述问题，但它们的质量是不稳定的，并且高度依赖手动信息。为了解决上述问题，我们提出了一种自动方法层次结构链（HCOG）。它利用大型语言模型将长描述分解为代表不同对象部分的块，并根据遮挡从内而外订购，形成层次结构链。在每个块中，我们首先要粗略地创建组件，然后通过目标区域定位和相应的3D高斯内核优化来精确地结合属性。在块之间，我们通过扩展新的高斯内核，重新分配语义标签并消除不必要的核，从而无缝地产生新零件，从而无缝生成新的部分，从而确保仅添加相关零件而不会破坏先前优化的部分，从而无缝生成新部分。实验证实，HCOG产生具有复杂属性的结构相干的属性 - 信仰3D对象。该代码可在此HTTPS URL上找到。

Title: Occupancy World Model for Robots

• Authors: Zhang Zhang, Qiang Zhang, Wei Cui, Shuai Shi, Yijie Guo, Gang Han, Wen Zhao, Jingkai Sun, Jiahang Cao, Jiaxu Wang, Hao Cheng, Xiaozhu Ju, Zhengping Che, Renjing Xu, Jian Tang
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2505.05512
• Pdf URL: https://arxiv.org/pdf/2505.05512
• Copy Paste: [[2505.05512]] Occupancy World Model for Robots(https://arxiv.org/abs/2505.05512)
• Keywords: generative
• Abstract: Understanding and forecasting the scene evolutions deeply affect the exploration and decision of embodied agents. While traditional methods simulate scene evolutions through trajectory prediction of potential instances, current works use the occupancy world model as a generative framework for describing fine-grained overall scene dynamics. However, existing methods cluster on the outdoor structured road scenes, while ignoring the exploration of forecasting 3D occupancy scene evolutions for robots in indoor scenes. In this work, we explore a new framework for learning the scene evolutions of observed fine-grained occupancy and propose an occupancy world model based on the combined spatio-temporal receptive field and guided autoregressive transformer to forecast the scene evolutions, called RoboOccWorld. We propose the Conditional Causal State Attention (CCSA), which utilizes camera poses of next state as conditions to guide the autoregressive transformer to adapt and understand the indoor robotics scenarios. In order to effectively exploit the spatio-temporal cues from historical observations, Hybrid Spatio-Temporal Aggregation (HSTA) is proposed to obtain the combined spatio-temporal receptive field based on multi-scale spatio-temporal windows. In addition, we restructure the OccWorld-ScanNet benchmark based on local annotations to facilitate the evaluation of the indoor 3D occupancy scene evolution prediction task. Experimental results demonstrate that our RoboOccWorld outperforms state-of-the-art methods in indoor 3D occupancy scene evolution prediction task. The code will be released soon.
• 摘要：理解和预测场景的演变深深影响体现代理的探索和决策。尽管传统方法通过潜在实例的轨迹预测模拟场景的演变，但当前的作品将占用世界模型用作描述细粒整体场景动态的生成框架。但是，现有方法聚集在室外结构化的道路场景上，同时忽略了室内场景中机器人的预测3D占用场景演变的探索。在这项工作中，我们探索了一个新的框架，以学习观察到的精细占用的现场演变，并根据当时的时空接收领域提出了一个占用世界模型，并指导自动回归自动回归变形金刚预测场景的演变，称为RobOocCworld。我们提出有条件的因果状态注意力（CCSA），该病因利用下一个状态的相机姿势作为指导自回旋变压器的条件，以适应和了解室内机器人机器人方案。为了从历史观察中有效利用时空线索，提出混合时空聚集（HSTA）以获得基于多尺度时空时空窗户的合并时空接受场。此外，我们基于本地注释来重组OCCWorld-Scannet基准测试，以促进评估室内3D占用场景现场演变预测任务。实验结果表明，我们的Robooccworld在室内3D占用场景演变预测任务中的最先进方法。该代码将很快发布。

Title: OXSeg: Multidimensional attention UNet-based lip segmentation using semi-supervised lip contours

• Authors: Hanie Moghaddasi, Christina Chambers, Sarah N. Mattson, Jeffrey R. Wozniak, Claire D. Coles, Raja Mukherjee, Michael Suttie
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.05531
• Pdf URL: https://arxiv.org/pdf/2505.05531
• Copy Paste: [[2505.05531]] OXSeg: Multidimensional attention UNet-based lip segmentation using semi-supervised lip contours(https://arxiv.org/abs/2505.05531)
• Keywords: generation, generative
• Abstract: Lip segmentation plays a crucial role in various domains, such as lip synchronization, lipreading, and diagnostics. However, the effectiveness of supervised lip segmentation is constrained by the availability of lip contour in the training phase. A further challenge with lip segmentation is its reliance on image quality , lighting, and skin tone, leading to inaccuracies in the detected boundaries. To address these challenges, we propose a sequential lip segmentation method that integrates attention UNet and multidimensional input. We unravel the micro-patterns in facial images using local binary patterns to build multidimensional inputs. Subsequently, the multidimensional inputs are fed into sequential attention UNets, where the lip contour is reconstructed. We introduce a mask generation method that uses a few anatomical landmarks and estimates the complete lip contour to improve segmentation accuracy. This mask has been utilized in the training phase for lip segmentation. To evaluate the proposed method, we use facial images to segment the upper lips and subsequently assess lip-related facial anomalies in subjects with fetal alcohol syndrome (FAS). Using the proposed lip segmentation method, we achieved a mean dice score of 84.75%, and a mean pixel accuracy of 99.77% in upper lip segmentation. To further evaluate the method, we implemented classifiers to identify those with FAS. Using a generative adversarial network (GAN), we reached an accuracy of 98.55% in identifying FAS in one of the study populations. This method could be used to improve lip segmentation accuracy, especially around Cupid's bow, and shed light on distinct lip-related characteristics of FAS.
• 摘要：唇部分割在各种领域中起着至关重要的作用，例如唇部同步，唇读和诊断。但是，监督唇部分割的有效性受训练阶段的唇轮廓的可用性限制。唇部分割的另一个挑战是它依赖图像质量，照明和肤色，导致检测到的边界不准确。为了应对这些挑战，我们提出了一种顺序的唇部分割方法，该方法集成了注意力UNET和多维输入。我们使用局部二进制图案在面部图像中揭示微观图像，以构建多维输入。随后，将多维输入馈入唇形轮廓的顺序注意UNET中。我们引入了一种掩模生成方法，该方法使用了一些解剖学地标，并估算完整的唇轮廓以提高分割精度。该面具已在培训阶段用于唇部分割。为了评估所提出的方法，我们使用面部图像来分割上唇，然后评估胎儿酒精综合征（FAS）受试者中与唇部相关的面部异常。使用提出的唇部分割方法，我们达到的平均骰子得分为84.75％，平均像素精度为99.77％，在上唇分割中。为了进一步评估该方法，我们实施了分类器来识别使用FA的分类器。使用生成的对抗网络（GAN），我们在识别其中一个研究人群中识别FAS的准确度达到了98.55％。该方法可用于提高唇部分割精度，尤其是在丘比特的弓周围，并阐明了FAS的不同唇部相关特性。

Title: Rethinking Graph Contrastive Learning through Relative Similarity Preservation

• Authors: Zhiyuan Ning, Pengfei Wang, Ziyue Qiao, Pengyang Wang, Yuanchun Zhou
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.05533
• Pdf URL: https://arxiv.org/pdf/2505.05533
• Copy Paste: [[2505.05533]] Rethinking Graph Contrastive Learning through Relative Similarity Preservation(https://arxiv.org/abs/2505.05533)
• Keywords: generation
• Abstract: Graph contrastive learning (GCL) has achieved remarkable success by following the computer vision paradigm of preserving absolute similarity between augmented views. However, this approach faces fundamental challenges in graphs due to their discrete, non-Euclidean nature -- view generation often breaks semantic validity and similarity verification becomes unreliable. Through analyzing 11 real-world graphs, we discover a universal pattern transcending the homophily-heterophily dichotomy: label consistency systematically diminishes as structural distance increases, manifesting as smooth decay in homophily graphs and oscillatory decay in heterophily graphs. We establish theoretical guarantees for this pattern through random walk theory, proving label distribution convergence and characterizing the mechanisms behind different decay behaviors. This discovery reveals that graphs naturally encode relative similarity patterns, where structurally closer nodes exhibit collectively stronger semantic relationships. Leveraging this insight, we propose RELGCL, a novel GCL framework with complementary pairwise and listwise implementations that preserve these inherent patterns through collective similarity objectives. Extensive experiments demonstrate that our method consistently outperforms 20 existing approaches across both homophily and heterophily graphs, validating the effectiveness of leveraging natural relative similarity over artificial absolute similarity.
• 摘要：图形对比度学习（GCL）通过遵循计算机视觉范例的范围，即保持增强视图之间的绝对相似性，取得了巨大的成功。但是，这种方法由于其离散的非欧国性性质而面临图表中的基本挑战 - 视图发电通常会破坏语义有效性和相似性验证变得不可靠。通过分析11个真实图形，我们发现了一种超越同质性二分法的通用模式：随着结构距离的增加，标签的一致性会系统地减小，在同质图中表现为平滑衰减，在杂型图中的振荡衰减。我们通过随机行走理论为这种模式建立理论保证，证明标签分布收敛并表征不同衰减行为背后的机制。这一发现表明，图表自然编码相对相似性模式，其中结构上更接近的节点表现出更强的语义关系。利用这种见解，我们提出了RelgCl，这是一种具有互补成对和列表的新型GCL框架，通过集体相似性目标来保留这些固有的模式。广泛的实验表明，我们的方法始终胜过同层和异质图的20种现有方法，从而验证了利用自然相对相似性而不是人工绝对相似性的有效性。

Title: Prompt to Polyp: Clinically-Aware Medical Image Synthesis with Diffusion Models

• Authors: Mikhail Chaichuk, Sushant Gautam, Steven Hicks, Elena Tutubalina
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.05573
• Pdf URL: https://arxiv.org/pdf/2505.05573
• Copy Paste: [[2505.05573]] Prompt to Polyp: Clinically-Aware Medical Image Synthesis with Diffusion Models(https://arxiv.org/abs/2505.05573)
• Keywords: generation
• Abstract: The generation of realistic medical images from text descriptions has significant potential to address data scarcity challenges in healthcare AI while preserving patient privacy. This paper presents a comprehensive study of text-to-image synthesis in the medical domain, comparing two distinct approaches: (1) fine-tuning large pre-trained latent diffusion models and (2) training small, domain-specific models. We introduce a novel model named MSDM, an optimized architecture based on Stable Diffusion that integrates a clinical text encoder, variational autoencoder, and cross-attention mechanisms to better align medical text prompts with generated images. Our study compares two approaches: fine-tuning large pre-trained models (FLUX, Kandinsky) versus training compact domain-specific models (MSDM). Evaluation across colonoscopy (MedVQA-GI) and radiology (ROCOv2) datasets reveals that while large models achieve higher fidelity, our optimized MSDM delivers comparable quality with lower computational costs. Quantitative metrics and qualitative evaluations by medical experts reveal strengths and limitations of each approach.
• 摘要：从文本描述中产生的现实医学图像具有很大的潜力，可以在保留患者隐私的同时解决医疗保健AI中的数据稀缺挑战。本文介绍了医学领域中文本对图像合成的全面研究，比较了两种不同的方法：（1）微调大型预训练的潜在扩散模型和（2）培训小型域特异性模型。我们介绍了一种名为MSDM的新型模型，该模型是一种基于稳定扩散的优化体系结构，该结构集成了临床文本编码器，变分自动编码器和跨注意机制，以更好地使医疗文本提示与生成的图像更好地对齐。我们的研究比较了两种方法：微调大型预训练模型（Flux，Kandinsky）与训练紧凑型域特异性模型（MSDM）。跨结肠镜检查（MEDVQA-GI）和放射学（ROCOV2）数据集的评估表明，尽管大型模型具有更高的保真度，但我们优化的MSDM可提供可比的质量，而计算成本较低。医学专家的定量指标和定性评估揭示了每种方法的优势和局限性。

Title: ReactDance: Progressive-Granular Representation for Long-Term Coherent Reactive Dance Generation

• Authors: Jingzhong Lin, Yuanyuan Qi, Xinru Li, Wenxuan Huang, Xiangfeng Xu, Bangyan Li, Xuejiao Wang, Gaoqi He
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05589
• Pdf URL: https://arxiv.org/pdf/2505.05589
• Copy Paste: [[2505.05589]] ReactDance: Progressive-Granular Representation for Long-Term Coherent Reactive Dance Generation(https://arxiv.org/abs/2505.05589)
• Keywords: generation
• Abstract: Reactive dance generation (RDG) produces follower movements conditioned on guiding dancer and music while ensuring spatial coordination and temporal coherence. However, existing methods overemphasize global constraints and optimization, overlooking local information, such as fine-grained spatial interactions and localized temporal context. Therefore, we present ReactDance, a novel diffusion-based framework for high-fidelity RDG with long-term coherence and multi-scale controllability. Unlike existing methods that struggle with interaction fidelity, synchronization, and temporal consistency in duet synthesis, our approach introduces two key innovations: 1)Group Residual Finite Scalar Quantization (GRFSQ), a multi-scale disentangled motion representation that captures interaction semantics from coarse body rhythms to fine-grained joint dynamics, and 2)Blockwise Local Context (BLC), a sampling strategy eliminating error accumulation in long sequence generation via local block causal masking and periodic positional encoding. Built on the decoupled multi-scale GRFSQ representation, we implement a diffusion model withLayer-Decoupled Classifier-free Guidance (LDCFG), allowing granular control over motion semantics across scales. Extensive experiments on standard benchmarks demonstrate that ReactDance surpasses existing methods, achieving state-of-the-art performance.
• 摘要：反应性舞蹈产生（RDG）产生的追随者运动以指导舞者和音乐为条件，同时确保空间协调和时间连贯性。但是，现有的方法过于强调全局约束和优化，忽略了本地信息，例如细粒度的空间相互作用和局部时间上下文。因此，我们提出了ReactDance，这是一种基于长期连贯性和多尺度可控性的高保真RDG的新型基于扩散的框架。 Unlike existing methods that struggle with interaction fidelity, synchronization, and temporal consistency in duet synthesis, our approach introduces two key innovations: 1)Group Residual Finite Scalar Quantization (GRFSQ), a multi-scale disentangled motion representation that captures interaction semantics from coarse body rhythms to fine-grained joint dynamics, and 2)Blockwise Local Context (BLC), a sampling strategy eliminating error通过局部区块因果掩蔽和周期性位置编码以长序列产生积累。我们建立在脱钩的多尺度GRFSQ表示上，我们实现了使用layerer-decopled的无分类器指南（LDCFG）实现扩散模型，从而允许跨量表对运动语义的粒状控制。对标准基准测试的广泛实验表明，反应超过现有方法，实现了最新的性能。

Title: A Preliminary Study for GPT-4o on Image Restoration

• Authors: Hao Yang, Yan Yang, Ruikun Zhang, Liyuan Pan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05621
• Pdf URL: https://arxiv.org/pdf/2505.05621
• Copy Paste: [[2505.05621]] A Preliminary Study for GPT-4o on Image Restoration(https://arxiv.org/abs/2505.05621)
• Keywords: restoration, generation
• Abstract: OpenAI's GPT-4o model, integrating multi-modal inputs and outputs within an autoregressive architecture, has demonstrated unprecedented performance in image generation. In this work, we investigate its potential impact on the image restoration community. We present the first systematic evaluation of GPT-4o across diverse restoration tasks. Our experiments reveal that, although restoration outputs from GPT-4o are visually appealing, they often suffer from pixel-level structural fidelity when compared to ground-truth images. Common issues are variations in image proportions, shifts in object positions and quantities, and changes in this http URL address it, taking image dehazing, derainning, and low-light enhancement as representative case studies, we show that GPT-4o's outputs can serve as powerful visual priors, substantially enhancing the performance of existing dehazing networks. It offers practical guidelines and a baseline framework to facilitate the integration of GPT-4o into future image restoration pipelines. We hope the study on GPT-4o image restoration will accelerate innovation in the broader field of image generation areas. To support further research, we will release GPT-4o-restored images from over 10 widely used image restoration datasets.
• 摘要：OpenAI的GPT-4O模型，将多模式输入和输出集成到自回归体系结构中，在图像生成中表现出了前所未有的性能。在这项工作中，我们研究了其对图像恢复社区的潜在影响。我们介绍了跨不同恢复任务的GPT-4O的首次系统评估。我们的实验表明，尽管GPT-4O的恢复输出在视觉上吸引人，但与地面真实图像相比，它们通常会遭受像素级的结构保真度。常见的问题是图像比例的变化，对象位置和数量的变化以及该HTTP URL的变化解决了它，以代表性的案例研究为代表性案例研究，将图像去除，降低和低光增强为代表性，我们表明GPT-4O的输出可以充当强大的视觉效果，从而实质上增强了现有降临网络的性能。它提供了实用的准则和基线框架，以促进将GPT-4O集成到未来的图像恢复管道中。我们希望有关GPT-4O图像恢复的研究能够加快图像生成领域的广泛领域的创新。为了支持进一步的研究，我们将从超过10个广泛使用的图像恢复数据集中释放GPT-4O-RESTOR的图像。

Title: Looking Beyond Language Priors: Enhancing Visual Comprehension and Attention in Multimodal Models

• Authors: Aarti Ghatkesar, Uddeshya Upadhyay, Ganesh Venkatesh
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.05626
• Pdf URL: https://arxiv.org/pdf/2505.05626
• Copy Paste: [[2505.05626]] Looking Beyond Language Priors: Enhancing Visual Comprehension and Attention in Multimodal Models(https://arxiv.org/abs/2505.05626)
• Keywords: generation
• Abstract: Achieving deep alignment between vision and language remains a central challenge for Multimodal Large Language Models (MLLMs). These models often fail to fully leverage visual input, defaulting to strong language priors. Our approach first provides insights into how MLLMs internally build visual understanding of image regions and then introduces techniques to amplify this capability. Specifically, we explore techniques designed both to deepen the model's understanding of visual content and to ensure that these visual insights actively guide language generation. We demonstrate the superior multimodal understanding of our resultant model through a detailed upstream analysis quantifying its ability to predict visually-dependent tokens as well as 10 pt boost on visually challenging tasks.
• 摘要：对于多模式大语模型（MLLM）来说，实现视觉和语言之间的深入对齐仍然是一个核心挑战。这些模型通常无法完全利用视觉输入，默认为强大的语言先验。我们的方法首先提供了有关MLLM在内部如何建立对图像区域的视觉理解的见解，然后引入了扩大此功能的技术。具体来说，我们探索技术既旨在加深模型对视觉内容的理解，又可以确保这些视觉见解积极指导语言生成。我们通过详细的上游分析量化了其预测视觉依赖性令牌的能力，并在视觉上具有挑战性的任务上提高了10 pt的能力，从而证明了对我们产生模型的高模式理解。

Title: VR-RAG: Open-vocabulary Species Recognition with RAG-Assisted Large Multi-Modal Models

• Authors: Faizan Farooq Khan, Jun Chen, Youssef Mohamed, Chun-Mei Feng, Mohamed Elhoseiny
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05635
• Pdf URL: https://arxiv.org/pdf/2505.05635
• Copy Paste: [[2505.05635]] VR-RAG: Open-vocabulary Species Recognition with RAG-Assisted Large Multi-Modal Models(https://arxiv.org/abs/2505.05635)
• Keywords: generation
• Abstract: Open-vocabulary recognition remains a challenging problem in computer vision, as it requires identifying objects from an unbounded set of categories. This is particularly relevant in nature, where new species are discovered every year. In this work, we focus on open-vocabulary bird species recognition, where the goal is to classify species based on their descriptions without being constrained to a predefined set of taxonomic categories. Traditional benchmarks like CUB-200-2011 and Birdsnap have been evaluated in a closed-vocabulary paradigm, limiting their applicability to real-world scenarios where novel species continually emerge. We show that the performance of current systems when evaluated under settings closely aligned with open-vocabulary drops by a huge margin. To address this gap, we propose a scalable framework integrating structured textual knowledge from Wikipedia articles of 11,202 bird species distilled via GPT-4o into concise, discriminative summaries. We propose Visual Re-ranking Retrieval-Augmented Generation(VR-RAG), a novel, retrieval-augmented generation framework that uses visual similarities to rerank the top m candidates retrieved by a set of multimodal vision language encoders. This allows for the recognition of unseen taxa. Extensive experiments across five established classification benchmarks show that our approach is highly effective. By integrating VR-RAG, we improve the average performance of state-of-the-art Large Multi-Modal Model QWEN2.5-VL by 15.4% across five benchmarks. Our approach outperforms conventional VLM-based approaches, which struggle with unseen species. By bridging the gap between encyclopedic knowledge and visual recognition, our work advances open-vocabulary recognition, offering a flexible, scalable solution for biodiversity monitoring and ecological research.
• 摘要：在计算机视觉中，开放式视频计师识别仍然是一个具有挑战性的问题，因为它需要从一组无限类别中识别对象。这在本质上尤其重要，每年都会发现新物种。在这项工作中，我们专注于开放式鸟类鸟类的识别，其目标是根据其描述对物种进行分类，而不被限制为预定义的一组分类学类别。在封闭式范围的范式中评估了诸如CUB-200-200-200-200-200-200-200-200-和BirdsNAP的传统基准，从而将其适用性限制在新型物种不断出现的现实情况下。我们表明，在设置下与开放式摄影液滴相一致的设置下进行评估时，当前系统的性能较大。为了解决这一差距，我们提出了一个可扩展的框架，将通过GPT-4O蒸馏的11,202种鸟类物种的Wikipedia物品整合到简洁的，歧视性的摘要中。我们提出了视觉重新排列的检索效果生成（VR-rag），这是一个新颖的检索生成框架，使用视觉相似性来重新读取一组由一组多模式视觉语言编码器检索的顶级M候选者。这允许识别看不见的分类单元。五个既定分类基准的广泛实验表明，我们的方法非常有效。通过整合VR-rag，我们在五个基准测试中提高了最先进的大型多模型QWEN2.5-VL的平均性能，提高了15.4％。我们的方法表现优于传统的基于VLM的方法，这些方法与看不见的物种斗争。通过弥合百科全书知识和视觉识别之间的差距，我们的工作进步了开放式视频识别，为生物多样性监测和生态研究提供了灵活，可扩展的解决方案。

Title: Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval

• Authors: Alexander Most, Joseph Winjum, Ayan Biswas, Shawn Jones, Nishath Rajiv Ranasinghe, Dan O'Malley, Manish Bhattarai
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.05666
• Pdf URL: https://arxiv.org/pdf/2505.05666
• Copy Paste: [[2505.05666]] Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval(https://arxiv.org/abs/2505.05666)
• Keywords: generation
• Abstract: Retrieval-Augmented Generation (RAG) has become a popular technique for enhancing the reliability and utility of Large Language Models (LLMs) by grounding responses in external documents. Traditional RAG systems rely on Optical Character Recognition (OCR) to first process scanned documents into text. However, even state-of-the-art OCRs can introduce errors, especially in degraded or complex documents. Recent vision-language approaches, such as ColPali, propose direct visual embedding of documents, eliminating the need for OCR. This study presents a systematic comparison between a vision-based RAG system (ColPali) and more traditional OCR-based pipelines utilizing Llama 3.2 (90B) and Nougat OCR across varying document qualities. Beyond conventional retrieval accuracy metrics, we introduce a semantic answer evaluation benchmark to assess end-to-end question-answering performance. Our findings indicate that while vision-based RAG performs well on documents it has been fine-tuned on, OCR-based RAG is better able to generalize to unseen documents of varying quality. We highlight the key trade-offs between computational efficiency and semantic accuracy, offering practical guidance for RAG practitioners in selecting between OCR-dependent and vision-based document retrieval systems in production environments.
• 摘要：检索增强的一代（RAG）已成为一种流行的技术，可以通过将外部文档中的响应接地来增强大语言模型（LLM）的可靠性和实用性。传统的抹布系统依靠光学特征识别（OCR）将扫描文档首次处理为文本。但是，即使是最先进的OCR也会引入错误，尤其是在退化或复杂的文档中。最近的视力语言方法，例如Colpali，提出了文档的直接视觉嵌入，消除了对OCR的需求。这项研究介绍了基于视觉的抹布系统（COLPALI）和更传统的基于OCR的管道，利用Llama 3.2（90b）和各种文档质量的Nougat OCR进行了系统比较。除了传统的检索准确性指标外，我们还引入了语义答案评估基准，以评估端到端的提问性能。我们的发现表明，尽管基于视觉的抹布在对其进行微调的文档上表现良好，但基于OCR的抹布可以更好地概括地看不见质量的文档。我们强调了计算效率和语义准确性之间的关键权衡，为抹布从业者提供了实用指导，以在生产环境中的OCR依赖性和基于视觉的文档检索系统之间选择。

Title: InstanceGen: Image Generation with Instance-level Instructions

• Authors: Etai Sella, Yanir Kleiman, Hadar Averbuch-Elor
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05678
• Pdf URL: https://arxiv.org/pdf/2505.05678
• Copy Paste: [[2505.05678]] InstanceGen: Image Generation with Instance-level Instructions(https://arxiv.org/abs/2505.05678)
• Keywords: generation, generative
• Abstract: Despite rapid advancements in the capabilities of generative models, pretrained text-to-image models still struggle in capturing the semantics conveyed by complex prompts that compound multiple objects and instance-level attributes. Consequently, we are witnessing growing interests in integrating additional structural constraints, %leveraging additional structural inputs typically in the form of coarse bounding boxes, to better guide the generation process in such challenging cases. In this work, we take the idea of structural guidance a step further by making the observation that contemporary image generation models can directly provide a plausible \emph{fine-grained} structural initialization. We propose a technique that couples this image-based structural guidance with LLM-based instance-level instructions, yielding output images that adhere to all parts of the text prompt, including object counts, instance-level attributes, and spatial relations between instances.
• 摘要：尽管生成模型的能力取得了迅速的进步，但预验证的文本对图像模型仍在捕获复杂提示传达的语义，这些提示将多个对象和实例级级属性弥补。因此，我们目睹了越来越多的兴趣整合其他结构约束，百分比通常以粗边界框的形式利用其他结构投入，以更好地指导这种挑战性的情况下的生成过程。在这项工作中，我们将结构指导的想法进一步发展，通过观察当代图像生成模型可以直接提供合理的\ emph {细粒}结构初始化。我们提出了一种技术，该技术将基于图像的结构指导与基于LLM的实例级指令结合，产生粘附于文本提示的所有部分的输出图像，包括对象计数，实例级属性和实例之间的空间关系。

Title: Automated Learning of Semantic Embedding Representations for Diffusion Models

• Authors: Limai Jiang, Yunpeng Cai
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05732
• Pdf URL: https://arxiv.org/pdf/2505.05732
• Copy Paste: [[2505.05732]] Automated Learning of Semantic Embedding Representations for Diffusion Models(https://arxiv.org/abs/2505.05732)
• Keywords: generative
• Abstract: Generative models capture the true distribution of data, yielding semantically rich representations. Denoising diffusion models (DDMs) exhibit superior generative capabilities, though efficient representation learning for them are lacking. In this work, we employ a multi-level denoising autoencoder framework to expand the representation capacity of DDMs, which introduces sequentially consistent Diffusion Transformers and an additional timestep-dependent encoder to acquire embedding representations on the denoising Markov chain through self-conditional diffusion learning. Intuitively, the encoder, conditioned on the entire diffusion process, compresses high-dimensional data into directional vectors in latent under different noise levels, facilitating the learning of image embeddings across all timesteps. To verify the semantic adequacy of embeddings generated through this approach, extensive experiments are conducted on various datasets, demonstrating that optimally learned embeddings by DDMs surpass state-of-the-art self-supervised representation learning methods in most cases, achieving remarkable discriminative semantic representation quality. Our work justifies that DDMs are not only suitable for generative tasks, but also potentially advantageous for general-purpose deep learning applications.
• 摘要：生成模型捕获了数据的真实分布，从而产生了语义丰富的表示。剥离扩散模型（DDMS）具有出色的生成能力，尽管缺乏有效的代表性学习。在这项工作中，我们采用多级denoising自动编码器框架来扩展DDMS的表示能力，该框架引入了顺序一致的扩散变压器和额外的TimeStep依赖性编码器，以通过自我调节性扩散学习来获取在DeNoise Markov链上的嵌入表示。从直觉上讲，编码器以整个扩散过程为条件，将高维数据压缩到不同噪声水平下潜在的定向矢量中，从而促进了所有时间段中图像嵌入的学习。为了验证通过这种方法生成的嵌入的语义充足性，在各种数据集上进行了广泛的实验，证明DDMS通过DDMS超越最新的自我监督的最新自我监督表示表示方法在大多数情况下实现了显着的歧视性语义表示质量。我们的工作证明，DDM不仅适用于生成任务，而且对于通用深度学习应用程序也有可能有利。

Title: Harnessing LLMs Explanations to Boost Surrogate Models in Tabular Data Classification

• Authors: Ruxue Shi, Hengrui Gu, Xu Shen, Xin Wang
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2505.05744
• Pdf URL: https://arxiv.org/pdf/2505.05744
• Copy Paste: [[2505.05744]] Harnessing LLMs Explanations to Boost Surrogate Models in Tabular Data Classification(https://arxiv.org/abs/2505.05744)
• Keywords: generation
• Abstract: Large Language Models (LLMs) have shown remarkable ability in solving complex tasks, making them a promising tool for enhancing tabular learning. However, existing LLM-based methods suffer from high resource requirements, suboptimal demonstration selection, and limited interpretability, which largely hinder their prediction performance and application in the real world. To overcome these problems, we propose a novel in-context learning framework for tabular prediction. The core idea is to leverage the explanations generated by LLMs to guide a smaller, locally deployable Surrogate Language Model (SLM) to make interpretable tabular predictions. Specifically, our framework mainly involves three stages: (i) Post Hoc Explanation Generation, where LLMs are utilized to generate explanations for question-answer pairs in candidate demonstrations, providing insights into the reasoning behind the answer. (ii) Post Hoc Explanation-Guided Demonstrations Selection, which utilizes explanations generated by LLMs to guide the process of demonstration selection from candidate demonstrations. (iii) Post Hoc Explanation-Guided Interpretable SLM Prediction, which utilizes the demonstrations obtained in step (ii) as in-context and merges corresponding explanations as rationales to improve the performance of SLM and guide the model to generate interpretable outputs. Experimental results highlight the framework's effectiveness, with an average accuracy improvement of 5.31% across various tabular datasets in diverse domains.
• 摘要：大型语言模型（LLMS）在解决复杂的任务方面表现出色，使它们成为增强表格学习的有前途的工具。但是，现有的基于LLM的方法遭受了高资源需求，次优的演示选择以及有限的解释性，这在很大程度上阻碍了其在现实世界中的预测性能和应用。为了克服这些问题，我们为表格预测提出了一个新颖的文化学习框架。核心思想是利用LLMS生成的解释来指导较小的本地可部署的替代语言模型（SLM）来做出可解释的表格预测。具体而言，我们的框架主要涉及三个阶段：（i）事后解释生成，其中LLM被用于在候选示范中为提问 - 答案对的解释，从而提供了有关答案背后的推理的见解。 （ii）事后解释引导的演示选择，该选择利用LLMS生成的解释来指导候选人示范中的演示选择过程。 （iii）事后解释引导的可解释的SLM预测，该预测利用步骤（ii）中获得的演示为中文，并将相应的解释作为理由合并，以提高SLM的性能，并指导模型生成可解释的输出。实验结果突出了该框架的有效性，在不同域中各个表格数据集的平均准确性提高了5.31％。

Title: Accelerating Diffusion Transformer via Increment-Calibrated Caching with Channel-Aware Singular Value Decomposition

• Authors: Zhiyuan Chen, Keyi Li, Yifan Jia, Le Ye, Yufei Ma
• Subjects: cs.CV, cs.LG, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.05829
• Pdf URL: https://arxiv.org/pdf/2505.05829
• Copy Paste: [[2505.05829]] Accelerating Diffusion Transformer via Increment-Calibrated Caching with Channel-Aware Singular Value Decomposition(https://arxiv.org/abs/2505.05829)
• Keywords: generation, generative
• Abstract: Diffusion transformer (DiT) models have achieved remarkable success in image generation, thanks for their exceptional generative capabilities and scalability. Nonetheless, the iterative nature of diffusion models (DMs) results in high computation complexity, posing challenges for deployment. Although existing cache-based acceleration methods try to utilize the inherent temporal similarity to skip redundant computations of DiT, the lack of correction may induce potential quality degradation. In this paper, we propose increment-calibrated caching, a training-free method for DiT acceleration, where the calibration parameters are generated from the pre-trained model itself with low-rank approximation. To deal with the possible correction failure arising from outlier activations, we introduce channel-aware Singular Value Decomposition (SVD), which further strengthens the calibration effect. Experimental results show that our method always achieve better performance than existing naive caching methods with a similar computation resource budget. When compared with 35-step DDIM, our method eliminates more than 45% computation and improves IS by 12 at the cost of less than 0.06 FID increase. Code is available at this https URL.
• 摘要：扩散变压器（DIT）模型在图像生成方面取得了巨大的成功，感谢其出色的生成能力和可扩展性。但是，扩散模型（DMS）的迭代性质会导致高计算复杂性，从而对部署构成了挑战。尽管现有的基于缓存的加速方法试图利用跳过DIT的冗余计算的固有时间相似性，但缺乏校正可能会导致潜在的质量降解。在本文中，我们提出了增量校准的缓存，这是一种用于DIT加速度的无训练方法，其中校准参数是从具有低级别近似值的预训练模型本身生成的。为了处理异常激活引起的可能的校正失败，我们引入了通道感知的奇异值分解（SVD），从而进一步增强了校准效果。实验结果表明，我们的方法始终比具有类似计算资源预算的现有幼稚缓存方法更好。与35步DDIM相比，我们的方法消除了45％以上的计算，并提高了12个以上的成本小于0.06 FID。代码可在此HTTPS URL上找到。

Title: Decoupling Multi-Contrast Super-Resolution: Pairing Unpaired Synthesis with Implicit Representations

• Authors: Hongyu Rui, Yinzhe Wu, Fanwen Wang, Jiahao Huang, Liutao Yang, Zi Wang, Guang Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05855
• Pdf URL: https://arxiv.org/pdf/2505.05855
• Copy Paste: [[2505.05855]] Decoupling Multi-Contrast Super-Resolution: Pairing Unpaired Synthesis with Implicit Representations(https://arxiv.org/abs/2505.05855)
• Keywords: super-resolution
• Abstract: Magnetic Resonance Imaging (MRI) is critical for clinical diagnostics but is often limited by long acquisition times and low signal-to-noise ratios, especially in modalities like diffusion and functional MRI. The multi-contrast nature of MRI presents a valuable opportunity for cross-modal enhancement, where high-resolution (HR) modalities can serve as references to boost the quality of their low-resolution (LR) counterparts-motivating the development of Multi-Contrast Super-Resolution (MCSR) techniques. Prior work has shown that leveraging complementary contrasts can improve SR performance; however, effective feature extraction and fusion across modalities with varying resolutions remains a major challenge. Moreover, existing MCSR methods often assume fixed resolution settings and all require large, perfectly paired training datasets-conditions rarely met in real-world clinical environments. To address these challenges, we propose a novel Modular Multi-Contrast Super-Resolution (MCSR) framework that eliminates the need for paired training data and supports arbitrary upscaling. Our method decouples the MCSR task into two stages: (1) Unpaired Cross-Modal Synthesis (U-CMS), which translates a high-resolution reference modality into a synthesized version of the target contrast, and (2) Unsupervised Super-Resolution (U-SR), which reconstructs the final output using implicit neural representations (INRs) conditioned on spatial coordinates. This design enables scale-agnostic and anatomically faithful reconstruction by bridging un-paired cross-modal synthesis with unsupervised resolution enhancement. Experiments show that our method achieves superior performance at 4x and 8x upscaling, with improved fidelity and anatomical consistency over existing baselines. Our framework demonstrates strong potential for scalable, subject-specific, and data-efficient MCSR in real-world clinical settings.
• 摘要：磁共振成像（MRI）对于临床诊断至关重要，但通常受到较长的获取时间和低信噪比的限制，尤其是在扩散和功能性MRI等模态下。 MRI的多对比性质为跨模式增强提供了宝贵的机会，在这种情况下，高分辨率（HR）方式可以作为提高其低分辨率（LR）对应物的质量的参考，以动机的方式能够发展多对比度超级分辨率（MCSR）技术。先前的工作表明，利用互补对比可以提高SR性能。但是，具有各种分辨率的有效特征提取和融合是一个主要的挑战。此外，现有的MCSR方法通常采用固定的分辨率设置，并且所有这些都需要在现实世界中临床环境中很少满足的大型，完美配对的培训数据集条件。为了应对这些挑战，我们提出了一种新型的模块化多对比度超分辨率（MCSR）框架，该框架消除了对配对训练数据的需求，并支持任意提高。我们的方法将MCSR任务分为两个阶段：（1）未配合的跨模式合成（U-CMS），将高分辨率的参考模式转化为目标对比的合成版本，以及（2）无用的超分辨率（U-SR），使用隐式的Neural Auterantions（Inrals）进行了依次（Inrs）条件，将最终的输出重建。该设计通过桥接未配合的跨模式合成和无监督的分辨率增强，可以使尺度不足和解剖学上的忠实重建。实验表明，我们的方法在4倍和8倍的上升效果下实现了卓越的性能，对现有基线的保真度和解剖学一致性提高。我们的框架表明，在现实世界中临床环境中，具有可扩展，特定于学科和数据有效的MCSR的强大潜力。

Title: Generative Discovery of Partial Differential Equations by Learning from Math Handbooks

• Authors: Hao Xu, Yuntian Chen, Rui Cao, Tianning Tang, Mengge Du, Jian Li, Adrian H. Callaghan, Dongxiao Zhang
• Subjects: cs.LG, cs.AI, physics.comp-ph
• Abstract URL: https://arxiv.org/abs/2505.05869
• Pdf URL: https://arxiv.org/pdf/2505.05869
• Copy Paste: [[2505.05869]] Generative Discovery of Partial Differential Equations by Learning from Math Handbooks(https://arxiv.org/abs/2505.05869)
• Keywords: generation, generative
• Abstract: Data driven discovery of partial differential equations (PDEs) is a promising approach for uncovering the underlying laws governing complex systems. However, purely data driven techniques face the dilemma of balancing search space with optimization efficiency. This study introduces a knowledge guided approach that incorporates existing PDEs documented in a mathematical handbook to facilitate the discovery process. These PDEs are encoded as sentence like structures composed of operators and basic terms, and used to train a generative model, called EqGPT, which enables the generation of free form PDEs. A loop of generation evaluation optimization is constructed to autonomously identify the most suitable PDE. Experimental results demonstrate that this framework can recover a variety of PDE forms with high accuracy and computational efficiency, particularly in cases involving complex temporal derivatives or intricate spatial terms, which are often beyond the reach of conventional methods. The approach also exhibits generalizability to irregular spatial domains and higher dimensional settings. Notably, it succeeds in discovering a previously unreported PDE governing strongly nonlinear surface gravity waves propagating toward breaking, based on real world experimental data, highlighting its applicability to practical scenarios and its potential to support scientific discovery.
• 摘要：数据驱动的部分微分方程（PDE）是揭示有关复杂系统的基本法律的有前途方法。但是，纯粹的数据驱动技术面临着平衡搜索空间与优化效率的困境。这项研究介绍了一种知识引导的方法，该方法结合了数学手册中记录的现有PDE，以促进发现过程。这些PDE被编码为句子，例如由操作员和基本术语组成的结构，用于训练称为eqgpt的生成模型，该模型可以生成自由形式的PDE。构建了生成评估优化的循环，以自主识别最合适的PDE。实验结果表明，该框架可以以高精度和计算效率恢复各种PDE形式，尤其是在涉及复杂的时间衍生物或复杂的空间术语的情况下，这通常超出了常规方法的范围。该方法还表现出对不规则空间结构域和较高维度设置的普遍性。值得注意的是，基于现实世界的实验数据，它成功地发现了一个以前未报告的PDE，该PDE强烈地管理着强烈的非线性表面重力波，这些重力波倾向于破坏破裂，从而强调了其对实际场景的适用性及其在支持科学发现的潜力。

Title: A 3D pocket-aware and evolutionary conserved interaction guided diffusion model for molecular optimization

• Authors: Anjie Qiao, Hao Zhang, Qianmu Yuan, Qirui Deng, Jingtian Su, Weifeng Huang, Huihao Zhou, Guo-Bo Li, Zhen Wang, Jinping Lei
• Subjects: cs.LG, physics.chem-ph, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2505.05874
• Pdf URL: https://arxiv.org/pdf/2505.05874
• Copy Paste: [[2505.05874]] A 3D pocket-aware and evolutionary conserved interaction guided diffusion model for molecular optimization(https://arxiv.org/abs/2505.05874)
• Keywords: generation
• Abstract: Generating molecules that bind to specific protein targets via diffusion models has shown good promise for structure-based drug design and molecule optimization. Especially, the diffusion models with binding interaction guidance enables molecule generation with high affinity through forming favorable interaction within protein pocket. However, the generated molecules may not form interactions with the highly conserved residues, which are important for protein functions and bioactivities of the ligands. Herein, we developed a new 3D target-aware diffusion model DiffDecip, which explicitly incorporates the protein-ligand binding interactions and evolutionary conservation information of protein residues into both diffusion and sampling process, for molecule optimization through scaffold decoration. The model performance revealed that DiffDecip outperforms baseline model DiffDec on molecule optimization towards higher affinity through forming more non-covalent interactions with highly conserved residues in the protein pocket.
• 摘要：通过扩散模型与特定蛋白质靶标结合的分子对基于结构的药物设计和分子优化显示了良好的希望。特别是，具有结合相互作用引导的扩散模型可以通过在蛋白质袋中形成有利的相互作用来产生高亲和力的分子。但是，产生的分子可能不会与高度保守的残基形成相互作用，这对于配体的蛋白质功能和生物活性很重要。在此，我们开发了一种新的3D目标扩散模型DiffDecip，该模型明确地融合了蛋白质 - 配体的结合相互作用和蛋白质残基的进化保护信息，以通过支架装饰进行分子优化分子优化分子。该模型性能表明，通过与蛋白质口袋中高度保守的残基形成更非共价的相互作用，DIFFDECIP优先于分子优化对更高亲和力的基线模型差异。

Title: Examining the Source of Defects from a Mechanical Perspective for 3D Anomaly Detection

• Authors: Hanzhe Liang, Aoran Wang, Jie Zhou, Xin Jin, Can Gao, Jinbao Wang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.05901
• Pdf URL: https://arxiv.org/pdf/2505.05901
• Copy Paste: [[2505.05901]] Examining the Source of Defects from a Mechanical Perspective for 3D Anomaly Detection(https://arxiv.org/abs/2505.05901)
• Keywords: generation
• Abstract: In this paper, we go beyond identifying anomalies only in structural terms and think about better anomaly detection motivated by anomaly causes. Most anomalies are regarded as the result of unpredictable defective forces from internal and external sources, and their opposite forces are sought to correct the anomalies. We introduced a Mechanics Complementary framework for 3D anomaly detection (MC4AD) to generate internal and external Corrective forces for each point. A Diverse Anomaly-Generation (DA-Gen) module is first proposed to simulate various anomalies. Then, we present a Corrective Force Prediction Network (CFP-Net) with complementary representations for point-level representation to simulate the different contributions of internal and external corrective forces. A combined loss was proposed, including a new symmetric loss and an overall loss, to constrain the corrective forces properly. As a highlight, we consider 3D anomaly detection in industry more comprehensively, creating a hierarchical quality control strategy based on a three-way decision and contributing a dataset named Anomaly-IntraVariance with intraclass variance to evaluate the model. On the proposed and existing five datasets, we obtained nine state-of-the-art performers with the minimum parameters and the fastest inference speed. The source is available at this https URL
• 摘要：在本文中，我们超越了仅以结构性术语来识别异常，而是考虑出由异常原因促进的更好的异常检测。大多数异常被视为内部和外部来源不可预测的有缺陷力的结果，并寻求相反的力来纠正异常。我们引入了3D异常检测（MC4AD）的力学补充框架，以为每个点产生内部和外部纠正力。首先提出了多种异常（DA-GEN）模块来模拟各种异常。然后，我们提出一个具有互补表示的纠正力预测网络（CFP-NET），用于点级表示，以模拟内部和外部纠正力的不同贡献。提出了合并的损失，包括新的对称损失和整体损失，以适当限制纠正力。作为一个亮点，我们更全面地考虑了行业中的3D异常检测，创建了基于三向决策的层次结构质量控制策略，并贡献了一个名为Anomaly Intravariance的数据集，并使用Intraclass差异来评估该模型。在拟议的五个数据集和现有的五个数据集上，我们获得了九个具有最低参数和最快推理速度的最先进的表演者。该源可在此HTTPS URL上找到

Title: Autoencoder-Based Hybrid Replay for Class-Incremental Learning

• Authors: Milad Khademi Nori, Il-Min Kim, Guanghui Wang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05926
• Pdf URL: https://arxiv.org/pdf/2505.05926
• Copy Paste: [[2505.05926]] Autoencoder-Based Hybrid Replay for Class-Incremental Learning(https://arxiv.org/abs/2505.05926)
• Keywords: generative
• Abstract: In class-incremental learning (CIL), effective incremental learning strategies are essential to mitigate task confusion and catastrophic forgetting, especially as the number of tasks $t$ increases. Current exemplar replay strategies impose $\mathcal{O}(t)$ memory/compute complexities. We propose an autoencoder-based hybrid replay (AHR) strategy that leverages our new hybrid autoencoder (HAE) to function as a compressor to alleviate the requirement for large memory, achieving $\mathcal{O}(0.1 t)$ at the worst case with the computing complexity of $\mathcal{O}(t)$ while accomplishing state-of-the-art performance. The decoder later recovers the exemplar data stored in the latent space, rather than in raw format. Additionally, HAE is designed for both discriminative and generative modeling, enabling classification and replay capabilities, respectively. HAE adopts the charged particle system energy minimization equations and repulsive force algorithm for the incremental embedding and distribution of new class centroids in its latent space. Our results demonstrate that AHR consistently outperforms recent baselines across multiple benchmarks while operating with the same memory/compute budgets. The source code is included in the supplementary material and will be open-sourced upon publication.
• 摘要：在课堂开发学习（CIL）中，有效的增量学习策略对于减轻任务困惑和灾难性遗忘至关重要，尤其是随着任务的数量$ t $的增加。当前的示例重播策略强加了$ \ MATHCAL {O}（T）$内存/计算复杂性。我们提出了一种基于自动编码器的混合式重播（AHR）策略，该策略利用我们的新混合动力自动编码器（HAE）作为压缩机起作用，以减轻对$ \ Mathcal的计算复杂性，可以实现$ \ MATHCAL {O}（O}（O}（0.1 T）$）$ \ - e-t-earcal-the $ i}（$ t-e}（$ the）$ i}（$ t）（t）解码器后来恢复了存储在潜在空间中的示例数据，而不是以原始格式。此外，HAE是为判别和生成性建模设计的，分别启用了分类和重播功能。 HAE采用带电的粒子系统能量最小化方程和排斥力算法，用于在其潜在空间中新类质心的增量嵌入和分布。我们的结果表明，AHR在使用相同的内存/计算预算操作的同时，在多个基准测试中始终优于最近的基线。源代码包含在补充材料中，并将在出版后开源。

Title: Offline Multi-agent Reinforcement Learning via Score Decomposition

• Authors: Dan Qiao, Wenhao Li, Shanchao Yang, Hongyuan Zha, Baoxiang Wang
• Subjects: cs.LG, cs.MA
• Abstract URL: https://arxiv.org/abs/2505.05968
• Pdf URL: https://arxiv.org/pdf/2505.05968
• Copy Paste: [[2505.05968]] Offline Multi-agent Reinforcement Learning via Score Decomposition(https://arxiv.org/abs/2505.05968)
• Keywords: generative
• Abstract: Offline multi-agent reinforcement learning (MARL) faces critical challenges due to distributional shifts, further exacerbated by the high dimensionality of joint action spaces and the diversity in coordination strategies and quality among agents. Conventional approaches, including independent learning frameworks and value decomposition methods based on pessimistic principles, remain susceptible to out-of-distribution (OOD) joint actions and often yield suboptimal performance. Through systematic analysis of prevalent offline MARL benchmarks, we identify that this limitation primarily stems from the inherently multimodal nature of joint collaborative policies induced by offline data collection. To address these challenges, we propose a novel two-stage framework: First, we employ a diffusion-based generative model to explicitly capture the complex behavior policy, enabling accurate modeling of diverse multi-agent coordination patterns. Second, we introduce a sequential score function decomposition mechanism to regularize individual policies and enable decentralized execution. Extensive experiments on continuous control tasks demonstrate state-of-the-art performance across multiple standard offline MARL benchmarks, outperforming existing methods by 26.3\% in normalized returns. Our approach provides new insights into offline coordination and equilibrium selection in cooperative multi-agent systems.
• 摘要：离线多机构增强学习（MARL）由于分配的转变而面临着关键的挑战，这进一步加剧了联合行动空间的高维度以及代理商之间的协调策略和质量的多样性。传统方法，包括基于悲观原则的独立学习框架和价值分解方法，仍然容易受到分数过失（OOD）的关节行动的影响，并且通常会产生次优的性能。通过对普遍离线MARL基准测试的系统分析，我们确定该限制主要源于离线数据收集引起的联合协作策略的固有多模式性质。为了应对这些挑战，我们提出了一个新颖的两阶段框架：首先，我们采用基于扩散的生成模型来明确捕获复杂的行为策略，从而可以准确地建模多种多样的协调模式。其次，我们引入了一个顺序得分函数分解机制，以使单个策略正规化并实现分散执行。关于连续控制任务的广泛实验表明，在归一化回报中，多个标准离线基准的最先进的性能优于现有方法的26.3 \％。我们的方法为合作多代理系统中的离线协调和均衡选择提供了新的见解。

Title: Towards Better Cephalometric Landmark Detection with Diffusion Data Generation

• Authors: Dongqian Guo, Wencheng Han, Pang Lyu, Yuxi Zhou, Jianbing Shen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.06055
• Pdf URL: https://arxiv.org/pdf/2505.06055
• Copy Paste: [[2505.06055]] Towards Better Cephalometric Landmark Detection with Diffusion Data Generation(https://arxiv.org/abs/2505.06055)
• Keywords: generation
• Abstract: Cephalometric landmark detection is essential for orthodontic diagnostics and treatment planning. Nevertheless, the scarcity of samples in data collection and the extensive effort required for manual annotation have significantly impeded the availability of diverse datasets. This limitation has restricted the effectiveness of deep learning-based detection methods, particularly those based on large-scale vision models. To address these challenges, we have developed an innovative data generation method capable of producing diverse cephalometric X-ray images along with corresponding annotations without human intervention. To achieve this, our approach initiates by constructing new cephalometric landmark annotations using anatomical priors. Then, we employ a diffusion-based generator to create realistic X-ray images that correspond closely with these annotations. To achieve precise control in producing samples with different attributes, we introduce a novel prompt cephalometric X-ray image dataset. This dataset includes real cephalometric X-ray images and detailed medical text prompts describing the images. By leveraging these detailed prompts, our method improves the generation process to control different styles and attributes. Facilitated by the large, diverse generated data, we introduce large-scale vision detection models into the cephalometric landmark detection task to improve accuracy. Experimental results demonstrate that training with the generated data substantially enhances the performance. Compared to methods without using the generated data, our approach improves the Success Detection Rate (SDR) by 6.5%, attaining a notable 82.2%. All code and data are available at: this https URL
• 摘要：头标的地标检测对于正畸诊断和治疗计划至关重要。然而，数据收集中样本的稀缺性以及手动注释所需的广泛努力极大地阻碍了各种数据集的可用性。这种限制限制了基于深度学习的检测方法的有效性，尤其是基于大型视觉模型的方法。为了应对这些挑战，我们开发了一种创新的数据生成方法，能够产生不同的头皮计量学X射线图像以及相应的注释而无需人工干预。为了实现这一目标，我们的方法通过使用解剖学先验来构建新的头标有地标注释来启动。然后，我们采用基于扩散的生成器来创建与这些注释紧密相对应的现实X射线图像。为了在产生具有不同属性的样品时获得精确的控制，我们引入了一种新颖的迅速头形X射线图像数据集。该数据集包括真实的头部计量X射线图像和详细的医学文本提示，描述了图像。通过利用这些详细的提示，我们的方法改善了控制不同样式和属性的生成过程。在大型，多样化的数据中，我们将大规模视觉检测模型引入了头标的地标检测任务，以提高准确性。实验结果表明，使用生成数据的训练大大提高了性能。与不使用生成数据的方法相比，我们的方法将成功检测率（SDR）提高了6.5％，获得了显着的82.2％。所有代码和数据均可在以下网址提供：此HTTPS URL

Title: Photovoltaic Defect Image Generator with Boundary Alignment Smoothing Constraint for Domain Shift Mitigation

• Authors: Dongying Li, Binyi Su, Hua Zhang, Yong Li, Haiyong Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.06117
• Pdf URL: https://arxiv.org/pdf/2505.06117
• Copy Paste: [[2505.06117]] Photovoltaic Defect Image Generator with Boundary Alignment Smoothing Constraint for Domain Shift Mitigation(https://arxiv.org/abs/2505.06117)
• Keywords: generation, generative
• Abstract: Accurate defect detection of photovoltaic (PV) cells is critical for ensuring quality and efficiency in intelligent PV manufacturing systems. However, the scarcity of rich defect data poses substantial challenges for effective model training. While existing methods have explored generative models to augment datasets, they often suffer from instability, limited diversity, and domain shifts. To address these issues, we propose PDIG, a Photovoltaic Defect Image Generator based on Stable Diffusion (SD). PDIG leverages the strong priors learned from large-scale datasets to enhance generation quality under limited data. Specifically, we introduce a Semantic Concept Embedding (SCE) module that incorporates text-conditioned priors to capture the relational concepts between defect types and their appearances. To further enrich the domain distribution, we design a Lightweight Industrial Style Adaptor (LISA), which injects industrial defect characteristics into the SD model through cross-disentangled attention. At inference, we propose a Text-Image Dual-Space Constraints (TIDSC) module, enforcing the quality of generated images via positional consistency and spatial smoothing alignment. Extensive experiments demonstrate that PDIG achieves superior realism and diversity compared to state-of-the-art methods. Specifically, our approach improves Frechet Inception Distance (FID) by 19.16 points over the second-best method and significantly enhances the performance of downstream defect detection tasks.
• 摘要：光伏（PV）细胞的准确缺陷检测对于确保智能PV制造系统的质量和效率至关重要。但是，丰富的缺陷数据的稀缺性为有效的模型培训带来了重大挑战。尽管现有方法探索了生成模型以增强数据集，但它们通常会遭受不稳定性，有限的多样性和域转移的困扰。为了解决这些问题，我们提出了基于稳定扩散（SD）的光伏缺陷图像发生器PDIG。 PDIG利用了从大规模数据集中学到的强大先验，以在有限的数据下提高发电质量。具体而言，我们引入了一个语义概念嵌入（SCE）模块，该模块结合了文本条件的先验，以捕获缺陷类型及其外观之间的关系概念。为了进一步丰富域分布，我们设计了一个轻巧的工业风格适配器（LISA），该适配器（LISA）通过跨详细信息的注意力将工业缺陷特征注入SD模型。在推断时，我们提出了一个文本图像双空间约束（TIDSC）模块，通过位置一致性和空间平滑对齐来实施生成的图像的质量。广泛的实验表明，与最先进的方法相比，PDIG实现了优越的现实主义和多样性。具体来说，我们的方法将特雷希特的距离（FID）提高了19.16点，在第二好的方法上，显着提高了下游缺陷检测任务的性能。