2025-08-01

Title: Neural Autoregressive Modeling of Brain Aging

• Authors: Ridvan Yesiloglu, Wei Peng, Md Tauhidul Islam, Ehsan Adeli
• Subjects: cs.LG, eess.IV, q-bio.NC
• Abstract URL: https://arxiv.org/abs/2507.22954
• Pdf URL: https://arxiv.org/pdf/2507.22954
• Copy Paste: [[2507.22954]] Neural Autoregressive Modeling of Brain Aging(https://arxiv.org/abs/2507.22954)
• Keywords: generation, generative
• Abstract: Brain aging synthesis is a critical task with broad applications in clinical and computational neuroscience. The ability to predict the future structural evolution of a subject's brain from an earlier MRI scan provides valuable insights into aging trajectories. Yet, the high-dimensionality of data, subtle changes of structure across ages, and subject-specific patterns constitute challenges in the synthesis of the aging brain. To overcome these challenges, we propose NeuroAR, a novel brain aging simulation model based on generative autoregressive transformers. NeuroAR synthesizes the aging brain by autoregressively estimating the discrete token maps of a future scan from a convenient space of concatenated token embeddings of a previous and future scan. To guide the generation, it concatenates into each scale the subject's previous scan, and uses its acquisition age and the target age at each block via cross-attention. We evaluate our approach on both the elderly population and adolescent subjects, demonstrating superior performance over state-of-the-art generative models, including latent diffusion models (LDM) and generative adversarial networks, in terms of image fidelity. Furthermore, we employ a pre-trained age predictor to further validate the consistency and realism of the synthesized images with respect to expected aging patterns. NeuroAR significantly outperforms key models, including LDM, demonstrating its ability to model subject-specific brain aging trajectories with high fidelity.
• 摘要：脑老化合成是临床和计算神经科学中广泛应用的关键任务。从较早的MRI扫描中预测受试者大脑未来结构演变的能力为衰老轨迹提供了宝贵的见解。然而，数据的高维度，跨年龄的结构的细微变化以及特定于主题的模式构成了衰老大脑综合的挑战。为了克服这些挑战，我们提出了Neuroar，这是一种基于产生自回旋变压器的新型脑老化模拟模型。 NeuroAR通过自动估算以前扫描和将来扫描的串联令牌嵌入的方便空间来估算未来扫描的离散令牌图，从而构成了衰老大脑的衰老大脑。为了指导一代，它将其连接到每个尺度上的对象先前的扫描，并通过跨注意使用其获取年龄和每个块的目标年龄。我们评估了对老年人群和青少年受试者的方法，在图像保真度方面表明了超过最先进的生成模型，包括潜在扩散模型（LDM）和生成对抗性网络。此外，我们采用了预先训练的年龄预测因子，以进一步验证合成图像相对于预期衰老模式的一致性和现实性。 NeuroAR明显胜过包括LDM在内的关键模型，证明了其具有高忠诚度的主体特异性脑老化轨迹的能力。

Title: Investigating the Invertibility of Multimodal Latent Spaces: Limitations of Optimization-Based Methods

• Authors: Siwoo Park
• Subjects: cs.LG, cs.AI, cs.CV, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2507.23010
• Pdf URL: https://arxiv.org/pdf/2507.23010
• Copy Paste: [[2507.23010]] Investigating the Invertibility of Multimodal Latent Spaces: Limitations of Optimization-Based Methods(https://arxiv.org/abs/2507.23010)
• Keywords: generation, generative
• Abstract: This paper investigates the inverse capabilities and broader utility of multimodal latent spaces within task-specific AI (Artificial Intelligence) models. While these models excel at their designed forward tasks (e.g., text-to-image generation, audio-to-text transcription), their potential for inverse mappings remains largely unexplored. We propose an optimization-based framework to infer input characteristics from desired outputs, applying it bidirectionally across Text-Image (BLIP, Flux.1-dev) and Text-Audio (Whisper-Large-V3, Chatterbox-TTS) modalities. Our central hypothesis posits that while optimization can guide models towards inverse tasks, their multimodal latent spaces will not consistently support semantically meaningful and perceptually coherent inverse mappings. Experimental results consistently validate this hypothesis. We demonstrate that while optimization can force models to produce outputs that align textually with targets (e.g., a text-to-image model generating an image that an image captioning model describes correctly, or an ASR model transcribing optimized audio accurately), the perceptual quality of these inversions is chaotic and incoherent. Furthermore, when attempting to infer the original semantic input from generative models, the reconstructed latent space embeddings frequently lack semantic interpretability, aligning with nonsensical vocabulary tokens. These findings highlight a critical limitation. multimodal latent spaces, primarily optimized for specific forward tasks, do not inherently possess the structure required for robust and interpretable inverse mappings. Our work underscores the need for further research into developing truly semantically rich and invertible multimodal latent spaces.
• 摘要：本文研究了特定于特定于AI（人工智能）模型中多模式潜在空间的逆功能和更广泛的效用。尽管这些模型在其设计的远期任务（例如文本到图像生成，音频到文本转录）上表现出色，但它们的反向映射的潜力仍然在很大程度上没有探索。我们提出了一个基于优化的框架，以从所需的输出中推断输入特性，并在文本图像（Blip，Flux.1-DEV）和文本ADIO（Whisper-Large-V3，Chatterbox-TTS）模态上进行双向应用。我们的中心假设认为，尽管优化可以指导模型朝向逆任务，但它们的多模式潜在空间将不会始终如一地支持语义上有意义且具有感知相干的逆映射。实验结果始终验证这一假设。我们证明，尽管优化可以迫使模型在文本上与目标保持一致的输出（例如，图像字幕模型可以正确描述的图像或ASR模型准确地转录优化音频的图像模型，但这些反everterions的感知质量是混乱和不稳定的。此外，当试图从生成模型中推断出原始的语义输入时，重建的潜在空间嵌入经常缺乏语义解释性，与荒谬的词汇令牌保持一致。这些发现突出了一个关键的限制。多模式的潜在空间主要针对特定的远期任务进行了优化，并不固有地具有可靠且可解释的逆映射所需的结构。我们的工作强调了进一步研究的必要性，以发展真正具有语义上富裕和可逆的多模式潜在空间。

Title: Modeling Human Gaze Behavior with Diffusion Models for Unified Scanpath Prediction

• Authors: Giuseppe Cartella, Vittorio Cuculo, Alessandro D'Amelio, Marcella Cornia, Giuseppe Boccignone, Rita Cucchiara
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.23021
• Pdf URL: https://arxiv.org/pdf/2507.23021
• Copy Paste: [[2507.23021]] Modeling Human Gaze Behavior with Diffusion Models for Unified Scanpath Prediction(https://arxiv.org/abs/2507.23021)
• Keywords: generation
• Abstract: Predicting human gaze scanpaths is crucial for understanding visual attention, with applications in human-computer interaction, autonomous systems, and cognitive robotics. While deep learning models have advanced scanpath prediction, most existing approaches generate averaged behaviors, failing to capture the variability of human visual exploration. In this work, we present ScanDiff, a novel architecture that combines diffusion models with Vision Transformers to generate diverse and realistic scanpaths. Our method explicitly models scanpath variability by leveraging the stochastic nature of diffusion models, producing a wide range of plausible gaze trajectories. Additionally, we introduce textual conditioning to enable task-driven scanpath generation, allowing the model to adapt to different visual search objectives. Experiments on benchmark datasets show that ScanDiff surpasses state-of-the-art methods in both free-viewing and task-driven scenarios, producing more diverse and accurate scanpaths. These results highlight its ability to better capture the complexity of human visual behavior, pushing forward gaze prediction research. Source code and models are publicly available at this https URL.
• 摘要：预测人类凝视扫描路径对于理解视觉关注至关重要，并在人类计算机相互作用，自主系统和认知机器人中的应用至关重要。尽管深度学习模型具有先进的扫描预测，但大多数现有方法都会产生平均行为，无法捕获人类视觉探索的可变性。在这项工作中，我们提出了史加达（Scandiff），这是一种新颖的体系结构，将扩散模型与视觉变压器结合在一起，以产生多样化和逼真的扫描路径。我们的方法通过利用扩散模型的随机性质来显式模拟扫描可变性，从而产生广泛的合理凝视轨迹。此外，我们介绍文本调节以启用任务驱动的扫描路径生成，从而使模型适应了不同的视觉搜索目标。基准数据集上的实验表明，在自由观看和任务驱动的方案中，扫描仪都超过了最新方法，从而产生了更加多样和准确的扫描路径。这些结果突出了其更好地捕获人类视觉行为复杂性，推动前进的预测研究的能力。源代码和模型可在此HTTPS URL上公开可用。

Title: Recovering Diagnostic Value: Super-Resolution-Aided Echocardiographic Classification in Resource-Constrained Imaging

• Authors: Krishan Agyakari Raja Babu, Om Prabhu, Annu, Mohanasankar Sivaprakasam
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.23027
• Pdf URL: https://arxiv.org/pdf/2507.23027
• Copy Paste: [[2507.23027]] Recovering Diagnostic Value: Super-Resolution-Aided Echocardiographic Classification in Resource-Constrained Imaging(https://arxiv.org/abs/2507.23027)
• Keywords: super-resolution, generative
• Abstract: Automated cardiac interpretation in resource-constrained settings (RCS) is often hindered by poor-quality echocardiographic imaging, limiting the effectiveness of downstream diagnostic models. While super-resolution (SR) techniques have shown promise in enhancing magnetic resonance imaging (MRI) and computed tomography (CT) scans, their application to echocardiography-a widely accessible but noise-prone modality-remains underexplored. In this work, we investigate the potential of deep learning-based SR to improve classification accuracy on low-quality 2D echocardiograms. Using the publicly available CAMUS dataset, we stratify samples by image quality and evaluate two clinically relevant tasks of varying complexity: a relatively simple Two-Chamber vs. Four-Chamber (2CH vs. 4CH) view classification and a more complex End-Diastole vs. End-Systole (ED vs. ES) phase classification. We apply two widely used SR models-Super-Resolution Generative Adversarial Network (SRGAN) and Super-Resolution Residual Network (SRResNet), to enhance poor-quality images and observe significant gains in performance metric-particularly with SRResNet, which also offers computational efficiency. Our findings demonstrate that SR can effectively recover diagnostic value in degraded echo scans, making it a viable tool for AI-assisted care in RCS, achieving more with less.
• 摘要：资源约束设置（RCS）中的自动化心脏解释通常受到超声心动图质量成像的阻碍，从而限制了下游诊断模型的有效性。尽管超分辨率（SR）技术在增强磁共振成像（MRI）和计算机断层扫描（CT）扫描方面已显示出希望，但它们应用于超声心动图 - 广泛可访问但容易发生噪声的方式毫无疑问。在这项工作中，我们研究了基于深度学习的SR提高低质量2D超声心动图的分类准确性的潜力。使用公开可用的CAMUS数据集，我们通过图像质量对样品进行分层，并评估两项临床上相关的复杂性：一个相对简单的两次室与四腔（2CH vs. 4CH）视图分类和更复杂的端端与端端与端 - 端 - 端 - 恒星（ed vs. vs. vs. vs. vs. vs. vs. vs. vs. ES）。我们应用了两个广泛使用的SR模型 - 分辨率生成对抗网络（SRGAN）和超分辨率残留网络（SRRESNET），以增强质量不佳的图像，并使用SRResnet观察性能度量的显着提高，这也提供了计算效率。我们的发现表明，SR可以有效地在退化的回声扫描中恢复诊断价值，从而使其成为RCS中AI辅助护理的可行工具，从而更少实现。

Title: Reference-Guided Diffusion Inpainting For Multimodal Counterfactual Generation

• Authors: Alexandru Buburuzan
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.23058
• Pdf URL: https://arxiv.org/pdf/2507.23058
• Copy Paste: [[2507.23058]] Reference-Guided Diffusion Inpainting For Multimodal Counterfactual Generation(https://arxiv.org/abs/2507.23058)
• Keywords: generation
• Abstract: Safety-critical applications, such as autonomous driving and medical image analysis, require extensive multimodal data for rigorous testing. Synthetic data methods are gaining prominence due to the cost and complexity of gathering real-world data, but they demand a high degree of realism and controllability to be useful. This work introduces two novel methods for synthetic data generation in autonomous driving and medical image analysis, namely MObI and AnydoorMed, respectively. MObI is a first-of-its-kind framework for Multimodal Object Inpainting that leverages a diffusion model to produce realistic and controllable object inpaintings across perceptual modalities, demonstrated simultaneously for camera and lidar. Given a single reference RGB image, MObI enables seamless object insertion into existing multimodal scenes at a specified 3D location, guided by a bounding box, while maintaining semantic consistency and multimodal coherence. Unlike traditional inpainting methods that rely solely on edit masks, this approach uses 3D bounding box conditioning to ensure accurate spatial positioning and realistic scaling. AnydoorMed extends this paradigm to the medical imaging domain, focusing on reference-guided inpainting for mammography scans. It leverages a diffusion-based model to inpaint anomalies with impressive detail preservation, maintaining the reference anomaly's structural integrity while semantically blending it with the surrounding tissue. Together, these methods demonstrate that foundation models for reference-guided inpainting in natural images can be readily adapted to diverse perceptual modalities, paving the way for the next generation of systems capable of constructing highly realistic, controllable and multimodal counterfactual scenarios.
• 摘要：安全至关重要的应用，例如自动驾驶和医疗图像分析，需要广泛的多模式数据来进行严格的测试。由于收集现实世界数据的成本和复杂性，合成数据方法正在变得突出，但它们要求高度的现实主义和可控性才能有用。这项工作分别介绍了两种用于自动驾驶和医学图像分析中合成数据生成的新方法，即Mobi和Anydoormed。 MOBI是用于多模式对象介绍的首个框架，该框架利用扩散模型在感知方式上产生真实和可控的对象，并同时证明了相机和激光镜头。给定单个参考RGB图像，MOBI可以在指定的3D位置，在边界框的指导下，将无缝对象插入到现有的多模式场景中，同时保持语义一致性和多模式相干性。与仅依靠编辑面膜的传统涂漆方法不同，此方法使用3D边界框调节来确保准确的空间定位和逼真的缩放。 Anydoormed将此范式扩展到医学成像领域，重点是引导乳房X线摄影扫描的介绍。它利用基于扩散的模型来保持具有令人印象深刻的细节保护的涂料异常，从而保持参考异常的结构完整性，同时将其与周围的组织融合。总之，这些方法表明，自然图像中参考引导涂漆的基础模型可以很容易地适应各种感知方式，为下一代能够构建高度现实，可控制和多模式反事实场景的系统铺平了道路。

Title: Scalable Generative Modeling of Weighted Graphs

• Authors: Richard Williams, Eric Nalisnick, Andrew Holbrook
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.23111
• Pdf URL: https://arxiv.org/pdf/2507.23111
• Copy Paste: [[2507.23111]] Scalable Generative Modeling of Weighted Graphs(https://arxiv.org/abs/2507.23111)
• Keywords: generative
• Abstract: Weighted graphs are ubiquitous throughout biology, chemistry, and the social sciences, motivating the development of generative models for abstract weighted graph data using deep neural networks. However, most current deep generative models are either designed for unweighted graphs and are not easily extended to weighted topologies or incorporate edge weights without consideration of a joint distribution with topology. Furthermore, learning a distribution over weighted graphs must account for complex nonlocal dependencies between both the edges of the graph and corresponding weights of each edge. We develop an autoregressive model BiGG-E, a nontrivial extension of the BiGG model, that learns a joint distribution over weighted graphs while still exploiting sparsity to generate a weighted graph with $n$ nodes and $m$ edges in $O((n + m)\log n)$ time. Simulation studies and experiments on a variety of benchmark datasets demonstrate that BiGG-E best captures distributions over weighted graphs while remaining scalable and computationally efficient.
• 摘要：加权图在整个生物学，化学和社会科学中都是无处不在的，激发了使用深神经网络抽象加权图数据的生成模型的开发。但是，大多数当前的深层生成模型要么是为未加权图设计的，因此不容易扩展到加权拓扑，或者在不考虑与拓扑的关节分布的情况下结合边缘权重。此外，在加权图上学习分布必须考虑到图形边缘和每个边缘的相应权重之间的复杂非局部依赖关系。我们开发了一种自回归模型BigG-E，BigG-E是BigG型号的非平凡扩展，该模型在加权图上学习了联合分布，同时仍利用稀疏度以生成带有$ n $ nodes的加权图，并以$ o（（n + m）\ log n）$时间产生$ n $ nodes和$ m $ edges。对各种基准数据集的仿真研究和实验表明，BIGG-E最佳捕获了加权图上的分布，同时保持可扩展性和计算效率。

Title: Details Matter for Indoor Open-vocabulary 3D Instance Segmentation

• Authors: Sanghun Jung, Jingjing Zheng, Ke Zhang, Nan Qiao, Albert Y. C. Chen, Lu Xia, Chi Liu, Yuyin Sun, Xiao Zeng, Hsiang-Wei Huang, Byron Boots, Min Sun, Cheng-Hao Kuo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23134
• Pdf URL: https://arxiv.org/pdf/2507.23134
• Copy Paste: [[2507.23134]] Details Matter for Indoor Open-vocabulary 3D Instance Segmentation(https://arxiv.org/abs/2507.23134)
• Keywords: generation
• Abstract: Unlike closed-vocabulary 3D instance segmentation that is often trained end-to-end, open-vocabulary 3D instance segmentation (OV-3DIS) often leverages vision-language models (VLMs) to generate 3D instance proposals and classify them. While various concepts have been proposed from existing research, we observe that these individual concepts are not mutually exclusive but complementary. In this paper, we propose a new state-of-the-art solution for OV-3DIS by carefully designing a recipe to combine the concepts together and refining them to address key challenges. Our solution follows the two-stage scheme: 3D proposal generation and instance classification. We employ robust 3D tracking-based proposal aggregation to generate 3D proposals and remove overlapped or partial proposals by iterative merging/removal. For the classification stage, we replace the standard CLIP model with Alpha-CLIP, which incorporates object masks as an alpha channel to reduce background noise and obtain object-centric representation. Additionally, we introduce the standardized maximum similarity (SMS) score to normalize text-to-proposal similarity, effectively filtering out false positives and boosting precision. Our framework achieves state-of-the-art performance on ScanNet200 and S3DIS across all AP and AR metrics, even surpassing an end-to-end closed-vocabulary method.
• 摘要：与封闭的唱机3D实例分割不同，通常受过训练的端到端，开放式3D实例分割（OV-3DIS）通常利用视觉语言模型（VLMS）生成3D实例建议并对其进行分类。尽管已经从现有研究提出了各种概念，但我们观察到这些个别概念不是相互排斥的，而是互补的。在本文中，我们通过仔细设计一种将概念结合在一起并完善它们以应对关键挑战的食谱，为OV-3DIS提出了一种新的最先进解决方案。我们的解决方案遵循两阶段方案：3D提案生成和实例分类。我们采用强大的3D跟踪提案聚合来生成3D建议，并通过迭代合并/删除来删除重叠或部分建议。对于分类阶段，我们用alpha-clip替换标准夹模型，该模型将对象掩模作为Alpha通道，以减少背景噪声并获得以对象为中心的表示。此外，我们引入了标准化的最大相似性（SMS）分数，以使文本到宽敞的相似性归一化，有效地滤除了误报并提高精度。我们的框架在所有AP和AR指标上都达到了Scannet200和S3DIS的最先进性能，甚至超过了端到端的闭合录音库方法。

Title: AI paradigm for solving differential equations: first-principles data generation and scale-dilation operator AI solver

• Authors: Xiangshu Gong, Zhiqiang Xie, Xiaowei Jin, Chen Wang, Yanling Qu, Wangmeng Zuo, Hui Li
• Subjects: cs.LG, physics.comp-ph
• Abstract URL: https://arxiv.org/abs/2507.23141
• Pdf URL: https://arxiv.org/pdf/2507.23141
• Copy Paste: [[2507.23141]] AI paradigm for solving differential equations: first-principles data generation and scale-dilation operator AI solver(https://arxiv.org/abs/2507.23141)
• Keywords: generation
• Abstract: Many problems are governed by differential equations (DEs). Artificial intelligence (AI) is a new path for solving DEs. However, data is very scarce and existing AI solvers struggle with approximation of high frequency components (AHFC). We propose an AI paradigm for solving diverse DEs, including DE-ruled first-principles data generation methodology and scale-dilation operator (SDO) AI solver. Using either prior knowledge or random fields, we generate solutions and then substitute them into the DEs to derive the sources and initial/boundary conditions through balancing DEs, thus producing arbitrarily vast amount of, first-principles-consistent training datasets at extremely low computational cost. We introduce a reversible SDO that leverages the Fourier transform of the multiscale solutions to fix AHFC, and design a spatiotemporally coupled, attention-based Transformer AI solver of DEs with SDO. An upper bound on the Hessian condition number of the loss function is proven to be proportional to the squared 2-norm of the solution gradient, revealing that SDO yields a smoother loss landscape, consequently fixing AHFC with efficient training. Extensive tests on diverse DEs demonstrate that our AI paradigm achieves consistently superior accuracy over state-of-the-art methods. This work makes AI solver of DEs to be truly usable in broad nature and engineering fields.
• 摘要：许多问题由微分方程（DES）支配。人工智能（AI）是解决DES的新途径。但是，数据非常稀缺，现有的AI求解器与高频组件（AHFC）的近似作用。我们提出了一个用于解决多样化DES的AI范式，包括脱离的第一原理数据生成方法论和规模稀释操作员（SDO）AI求解器。使用先验知识或随机字段，我们生成解决方案，然后将它们替换为DES，以通过平衡DES来得出来源和初始/边界条件，从而以极低的计算成本生成任意大量的第一原理辅助培训数据集。我们介绍了可逆的SDO，该SDO利用了多尺度解决方案的傅立叶变换来修复AHFC，并设计了一个时空耦合的，基于注意力的变压器AI求解器DES与SDO。事实证明，在Hessian条件数量上的上限与溶液梯度的平方的2型态成正比，这表明SDO产生了更平滑的损失景观，因此通过有效的训练固定AHFC。对多样性的广泛测试表明，我们的AI范式比最先进的方法始终达到卓越的精度。这项工作使DES的AI求解器在广阔的自然和工程领域中真正可用。

Title: X-NeMo: Expressive Neural Motion Reenactment via Disentangled Latent Attention

• Authors: Xiaochen Zhao, Hongyi Xu, Guoxian Song, You Xie, Chenxu Zhang, Xiu Li, Linjie Luo, Jinli Suo, Yebin Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23143
• Pdf URL: https://arxiv.org/pdf/2507.23143
• Copy Paste: [[2507.23143]] X-NeMo: Expressive Neural Motion Reenactment via Disentangled Latent Attention(https://arxiv.org/abs/2507.23143)
• Keywords: generation
• Abstract: We propose X-NeMo, a novel zero-shot diffusion-based portrait animation pipeline that animates a static portrait using facial movements from a driving video of a different individual. Our work first identifies the root causes of the key issues in prior approaches, such as identity leakage and difficulty in capturing subtle and extreme expressions. To address these challenges, we introduce a fully end-to-end training framework that distills a 1D identity-agnostic latent motion descriptor from driving image, effectively controlling motion through cross-attention during image generation. Our implicit motion descriptor captures expressive facial motion in fine detail, learned end-to-end from a diverse video dataset without reliance on pretrained motion detectors. We further enhance expressiveness and disentangle motion latents from identity cues by supervising their learning with a dual GAN decoder, alongside spatial and color augmentations. By embedding the driving motion into a 1D latent vector and controlling motion via cross-attention rather than additive spatial guidance, our design eliminates the transmission of spatial-aligned structural clues from the driving condition to the diffusion backbone, substantially mitigating identity leakage. Extensive experiments demonstrate that X-NeMo surpasses state-of-the-art baselines, producing highly expressive animations with superior identity resemblance. Our code and models are available for research.
• 摘要：我们提出了X-Nemo，这是一种基于零射传散射的动画管道的小说，它使用来自其他个体的驱动视频的面部运动来动画静态肖像。我们的工作首先确定了先前方法中关键问题的根本原因，例如身份泄漏和难以捕获微妙和极端表情的困难。为了应对这些挑战，我们引入了一个完全端到端的训练框架，该框架将1D身份 - 静态的潜在运动描述符提取到驱动图像，从而在图像生成过程中通过交叉注意来有效地控制运动。我们的隐式运动描述符细节捕获了表达的面部运动，从多样化的视频数据集端到端学习，而无需依赖预审前的运动探测器。我们通过双重GAN解码器以及空间和颜色增强来进一步增强表达性和脱离身份线索的脱离身份线索。通过将驱动运动嵌入1D潜在矢量并通过交叉注意而不是加性空间指导控制运动，我们的设计消除了空间对齐的结构线索从驱动条件到扩散骨架的传播，从而大大减轻了身份泄漏。广泛的实验表明，X-Nemo超过了最先进的基线，产生具有出色身份相似的高度表达动画。我们的代码和模型可用于研究。

Title: Towards High-Resolution Alignment and Super-Resolution of Multi-Sensor Satellite Imagery

• Authors: Philip Wootaek Shin, Vishal Gaur, Rahul Ramachandran, Manil Maskey, Jack Sampson, Vijaykrishnan Narayanan, Sujit Roy
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23150
• Pdf URL: https://arxiv.org/pdf/2507.23150
• Copy Paste: [[2507.23150]] Towards High-Resolution Alignment and Super-Resolution of Multi-Sensor Satellite Imagery(https://arxiv.org/abs/2507.23150)
• Keywords: super-resolution
• Abstract: High-resolution satellite imagery is essential for geospatial analysis, yet differences in spatial resolution across satellite sensors present challenges for data fusion and downstream applications. Super-resolution techniques can help bridge this gap, but existing methods rely on artificially downscaled images rather than real sensor data and are not well suited for heterogeneous satellite sensors with differing spectral, temporal characteristics. In this work, we develop a preliminary framework to align and Harmonized Landsat Sentinel 30m(HLS 30) imagery using Harmonized Landsat Sentinel 10m(HLS10) as a reference from the HLS dataset. Our approach aims to bridge the resolution gap between these sensors and improve the quality of super-resolved Landsat imagery. Quantitative and qualitative evaluations demonstrate the effectiveness of our method, showing its potential for enhancing satellite-based sensing applications. This study provides insights into the feasibility of heterogeneous satellite image super-resolution and highlights key considerations for future advancements in the field.
• 摘要：高分辨率卫星图像对于地理空间分析至关重要，但是跨卫星传感器的空间分辨率差异提出了数据融合和下游应用的挑战。超分辨率技术可以帮助弥合这一差距，但是现有的方法依赖于人为的缩小图像而不是真实的传感器数据，并且不适合具有不同光谱，时间特征的异质卫星传感器。在这项工作中，我们开发了一个初步框架，以使用HLS数据集的参考，以使用协调的Landsat Sentinel 10M（HLS10）来对齐和协调Landsat Sentinel 30m（HLS 30）图像。我们的方法旨在弥合这些传感器之间的分辨率差距，并提高超级分辨Landsat图像的质量。定量和定性评估证明了我们方法的有效性，显示了其增强基于卫星的传感应用的潜力。这项研究提供了有关异质卫星图像超分辨率可行性的见解，并突出了该领域未来进步的关键注意事项。

Title: FuseTen: A Generative Model for Daily 10 m Land Surface Temperature Estimation from Spatio-Temporal Satellite Observations

• Authors: Sofiane Bouaziz, Adel Hafiane, Raphael Canals, Rachid Nedjai
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23154
• Pdf URL: https://arxiv.org/pdf/2507.23154
• Copy Paste: [[2507.23154]] FuseTen: A Generative Model for Daily 10 m Land Surface Temperature Estimation from Spatio-Temporal Satellite Observations(https://arxiv.org/abs/2507.23154)
• Keywords: generative
• Abstract: Urban heatwaves, droughts, and land degradation are pressing and growing challenges in the context of climate change. A valuable approach to studying them requires accurate spatio-temporal information on land surface conditions. One of the most important variables for assessing and understanding these phenomena is Land Surface Temperature (LST), which is derived from satellites and provides essential information about the thermal state of the Earth's surface. However, satellite platforms inherently face a trade-off between spatial and temporal resolutions. To bridge this gap, we propose FuseTen, a novel generative framework that produces daily LST observations at a fine 10 m spatial resolution by fusing spatio-temporal observations derived from Sentinel-2, Landsat 8, and Terra MODIS. FuseTen employs a generative architecture trained using an averaging-based supervision strategy grounded in physical principles. It incorporates attention and normalization modules within the fusion process and uses a PatchGAN discriminator to enforce realism. Experiments across multiple dates show that FuseTen outperforms linear baselines, with an average 32.06% improvement in quantitative metrics and 31.42% in visual fidelity. To the best of our knowledge, this is the first non-linear method to generate daily LST estimates at such fine spatial resolution.
• 摘要：在气候变化的背景下，城市热浪，干旱和土地退化正在紧迫和日益增长的挑战。研究它们的一种有价值的方法需要有关土地表面条件的准确时空信息。评估和理解这些现象的最重要变量之一是陆地温度（LST），该变量源自卫星，并提供有关地球表面热状态的基本信息。但是，卫星平台固有地面临空间和时间分辨率之间的权衡。为了弥合这一差距，我们提出了Fuseten，这是一种新型的生成框架，每天通过融合了Sentinel-2，Landsat 8和Terra Modis的时空观测来以10 m的空间分辨率进行每日LST观测。 Fuseten采用了一种生成架构，该架构使用基于物理原则的基于平均的监督策略进行了培训。它将注意力和归一化模块纳入融合过程中，并使用Patchgan歧视器来实现现实主义。跨多个日期的实验表明，Fuseten的表现优于线性基准，平均定量指标提高了32.06％，视觉保真度为31.42％。据我们所知，这是第一种以如此精细的空间分辨率生成每日LST估计值的第一种非线性方法。

Title: Single Image Rain Streak Removal Using Harris Corner Loss and R-CBAM Network

• Authors: Jongwook Si, Sungyoung Kim
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2507.23185
• Pdf URL: https://arxiv.org/pdf/2507.23185
• Copy Paste: [[2507.23185]] Single Image Rain Streak Removal Using Harris Corner Loss and R-CBAM Network(https://arxiv.org/abs/2507.23185)
• Keywords: restoration
• Abstract: The problem of single-image rain streak removal goes beyond simple noise suppression, requiring the simultaneous preservation of fine structural details and overall visual quality. In this study, we propose a novel image restoration network that effectively constrains the restoration process by introducing a Corner Loss, which prevents the loss of object boundaries and detailed texture information during restoration. Furthermore, we propose a Residual Convolutional Block Attention Module (R-CBAM) Block into the encoder and decoder to dynamically adjust the importance of features in both spatial and channel dimensions, enabling the network to focus more effectively on regions heavily affected by rain streaks. Quantitative evaluations conducted on the Rain100L and Rain100H datasets demonstrate that the proposed method significantly outperforms previous approaches, achieving a PSNR of 33.29 dB on Rain100L and 26.16 dB on Rain100H.
• 摘要：单位图降雨切断的问题超出了简单的噪声抑制，需要同时保存精细的结构细节和整体视觉质量。在这项研究中，我们提出了一个新型的图像恢复网络，该网络通过引入角损失有效地限制了恢复过程，从而防止了物体边界的丧失和恢复过程中详细的纹理信息。此外，我们提出了一个残留的卷积块注意模块（R-CBAM）阻止编码器和解码器，以动态调整空间和通道维度中特征的重要性，从而使网络能够更有效地专注于受雨条纹严重影响的区域。在RAIN100L和RAIN100H数据集上进行的定量评估表明，所提出的方法显着胜过以前的方法，在Rain100L上达到了33.29 dB的PSNR，在Rain100H上达到了26.16 dB。

Title: Multi-Modal Motion Retrieval by Learning a Fine-Grained Joint Embedding Space

• Authors: Shiyao Yu, Zi-An Wang, Kangning Yin, Zheng Tian, Mingyuan Zhang, Weixin Si, Shihao Zou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23188
• Pdf URL: https://arxiv.org/pdf/2507.23188
• Copy Paste: [[2507.23188]] Multi-Modal Motion Retrieval by Learning a Fine-Grained Joint Embedding Space(https://arxiv.org/abs/2507.23188)
• Keywords: generation
• Abstract: Motion retrieval is crucial for motion acquisition, offering superior precision, realism, controllability, and editability compared to motion generation. Existing approaches leverage contrastive learning to construct a unified embedding space for motion retrieval from text or visual modality. However, these methods lack a more intuitive and user-friendly interaction mode and often overlook the sequential representation of most modalities for improved retrieval performance. To address these limitations, we propose a framework that aligns four modalities -- text, audio, video, and motion -- within a fine-grained joint embedding space, incorporating audio for the first time in motion retrieval to enhance user immersion and convenience. This fine-grained space is achieved through a sequence-level contrastive learning approach, which captures critical details across modalities for better alignment. To evaluate our framework, we augment existing text-motion datasets with synthetic but diverse audio recordings, creating two multi-modal motion retrieval datasets. Experimental results demonstrate superior performance over state-of-the-art methods across multiple sub-tasks, including an 10.16% improvement in R@10 for text-to-motion retrieval and a 25.43% improvement in R@1 for video-to-motion retrieval on the HumanML3D dataset. Furthermore, our results show that our 4-modal framework significantly outperforms its 3-modal counterpart, underscoring the potential of multi-modal motion retrieval for advancing motion acquisition.
• 摘要：运动检索对于运动采集至关重要，与运动产生相比，具有卓越的精度，现实主义，可控性和可编辑性。现有方法利用对比度学习来构建统一的嵌入空间，以从文本或视觉方式中检索运动。但是，这些方法缺乏更直观和用户友好的交互模式，并且经常忽略大多数模式的顺序表示，以改善检索性能。为了解决这些限制，我们提出了一个框架，该框架在细粒度的接头嵌入空间内与四种模式（文本，音频，视频和运动）保持一致，首次在运动检索中结合音频以增强用户的沉浸和便利性。通过序列级的对比学习方法实现了这个细粒度的空间，该方法捕获了跨模态的关键细节，以更好地对齐。为了评估我们的框架，我们通过合成但多样化的音频记录来增强现有的文本动态数据集，从而创建两个多模式运动检索数据集。实验结果表明，在多个子任务中，比最先进的方法优于最先进的方法，包括在文本到动作检索的R@10提高10.16％，R@1的25.43％改善了HumanML3D数据集的视频对动作检索。此外，我们的结果表明，我们的4模式框架的表现明显优于其3型模式的对手，这突显了多模式运动检索在进行运动采集方面的潜力。

Title: Zero-Shot Document Understanding using Pseudo Table of Contents-Guided Retrieval-Augmented Generation

• Authors: Hyeon Seong Jeong, Sangwoo Jo, Byeong Hyun Yoon, Yoonseok Heo, Haedong Jeong, Taehoon Kim
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.23217
• Pdf URL: https://arxiv.org/pdf/2507.23217
• Copy Paste: [[2507.23217]] Zero-Shot Document Understanding using Pseudo Table of Contents-Guided Retrieval-Augmented Generation(https://arxiv.org/abs/2507.23217)
• Keywords: generation
• Abstract: Understanding complex multimodal documents remains challenging due to their structural inconsistencies and limited training data availability. We introduce \textit{DocsRay}, a training-free document understanding system that integrates pseudo Table of Contents (TOC) generation with hierarchical Retrieval-Augmented Generation (RAG). Our approach leverages multimodal Large Language Models' (LLMs) native capabilities to seamlessly process documents containing diverse elements such as text, images, charts, and tables without requiring specialized models or additional training. DocsRay's framework synergistically combines three key techniques: (1) a semantic structuring module using prompt-based LLM interactions to generate a hierarchical pseudo-TOC, (2) zero-shot multimodal analysis that converts diverse document elements into unified, text-centric representations using the inherent capabilities of multimodal LLMs, and (3) an efficient two-stage hierarchical retrieval system that reduces retrieval complexity from $O(N)$ to $O(S + k_1 \cdot N_s)$. Evaluated on documents averaging 49.4 pages and 20,971 textual tokens, DocsRay reduced query latency from 3.89 to 2.12 seconds, achieving a 45% efficiency improvement. On the MMLongBench-Doc benchmark, DocsRay-Pro attains an accuracy of 64.7%, substantially surpassing previous state-of-the-art results.
• 摘要：由于其结构上的不一致和培训数据的有限，了解复杂的多模式文档仍然具有挑战性。我们介绍了\ textit {docsray}，这是一个无培训的文档理解系统，将伪目录（TOC）生成与层次结构检索发电（RAG）集成在一起。我们的方法利用多模式大型语言模型（LLMS）本地功能无缝处理包含不同元素的文档，例如文本，图像，图表和表格，而无需专门模型或其他培训。 DOCSRAY的框架协同结合了三个关键技术：（1）使用基于及时的LLM交互的语义结构模块生成层次的伪-TOC，（2）零摄像的多模式分析，将多种文档元素转换为统一的文本式，有效的两个系统的固有能力，以将多种文档元素转换为多模型的固有能力，并（3）HYMS和3）（3）（3）（3）（3）3）将检索复杂性从$ O（N）$降低到$ O（S + K_1 \ CDOT N_S）$。对平均49.4页和20,971个文本令牌的文档进行了评估，DocsRay将查询延迟从3.89降低到2.12秒，从而提高了45％的效率。在Mmlongbench-Doc基准测试中，Docsray-Pro的准确性为64.7％，实质上超过了先前的最新结果。

Title: Toward Safe, Trustworthy and Realistic Augmented Reality User Experience

• Authors: Yanming Xiu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23226
• Pdf URL: https://arxiv.org/pdf/2507.23226
• Copy Paste: [[2507.23226]] Toward Safe, Trustworthy and Realistic Augmented Reality User Experience(https://arxiv.org/abs/2507.23226)
• Keywords: quality assessment
• Abstract: As augmented reality (AR) becomes increasingly integrated into everyday life, ensuring the safety and trustworthiness of its virtual content is critical. Our research addresses the risks of task-detrimental AR content, particularly that which obstructs critical information or subtly manipulates user perception. We developed two systems, ViDDAR and VIM-Sense, to detect such attacks using vision-language models (VLMs) and multimodal reasoning modules. Building on this foundation, we propose three future directions: automated, perceptually aligned quality assessment of virtual content; detection of multimodal attacks; and adaptation of VLMs for efficient and user-centered deployment on AR devices. Overall, our work aims to establish a scalable, human-aligned framework for safeguarding AR experiences and seeks feedback on perceptual modeling, multimodal AR content implementation, and lightweight model adaptation.
• 摘要：随着增强现实（AR）越来越多地整合到日常生活中，确保其虚拟内容的安全性和可信赖性至关重要。我们的研究解决了任务涉及AR含量的风险，尤其是阻碍关键信息或巧妙地操纵用户感知的风险。我们开发了两个系统，即Viddar和Vim-Sense，用于使用视觉模型（VLM）和多模式推理模块检测此类攻击。在这个基础的基础上，我们提出了三个未来的方向：虚拟内容的自动化，感知质量评估；检测多模式攻击；并改编VLM，以在AR设备上进行高效和以用户为中心的部署。总体而言，我们的工作旨在建立一个可扩展的人类一致的框架，以保护AR体验，并寻求有关感知建模，多模式AR内容实现和轻量级模型适应的反馈。

Title: Generalized Reinforcement Learning for Retriever-Specific Query Rewriter with Unstructured Real-World Documents

• Authors: Sungguk Cha, DongWook Kim, Taeseung Hahn, Mintae Kim, Youngsub Han, Byoung-Ki Jeon
• Subjects: cs.CV, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.23242
• Pdf URL: https://arxiv.org/pdf/2507.23242
• Copy Paste: [[2507.23242]] Generalized Reinforcement Learning for Retriever-Specific Query Rewriter with Unstructured Real-World Documents(https://arxiv.org/abs/2507.23242)
• Keywords: generation
• Abstract: Retrieval-Augmented Generation (RAG) systems rely heavily on effective query formulation to unlock external knowledge, yet optimizing queries for diverse, unstructured real-world documents remains a challenge. We introduce \textbf{RL-QR}, a reinforcement learning framework for retriever-specific query rewriting that eliminates the need for human-annotated datasets and extends applicability to both text-only and multi-modal databases. By synthesizing scenario-question pairs and leveraging Generalized Reward Policy Optimization (GRPO), RL-QR trains query rewriters tailored to specific retrievers, enhancing retrieval performance across varied domains. Experiments on industrial in-house data demonstrate significant improvements, with $\text{RL-QR}_{\text{multi-modal}}$ achieving an 11\% relative gain in NDCG@3 for multi-modal RAG and $\text{RL-QR}_{\text{lexical}}$ yielding a 9\% gain for lexical retrievers. However, challenges persist with semantic and hybrid retrievers, where rewriters failed to improve performance, likely due to training misalignments. Our findings highlight RL-QR's potential to revolutionize query optimization for RAG systems, offering a scalable, annotation-free solution for real-world retrieval tasks, while identifying avenues for further refinement in semantic retrieval contexts.
• 摘要：检索增强的生成（RAG）系统在很大程度上依赖有效的查询公式来解锁外部知识，但优化查询对多样化的，非结构化的现实世界文档仍然是一个挑战。我们介绍了\ textbf {rl-qr}，这是一种用于检索器特定查询重写的强化学习框架，以消除对人类注销数据集的需求，并将适用性扩展到文本和多模式数据库。通过综合方案问题对并利用广义奖励策略优化（GRPO），RL-QR训练了针对特定猎犬量身定制的查询重写器，从而增强了各种域之间的检索性能。对工业内部数据的实验表现出重大改进，并具有$ \ text {rl-qr} _ {\ text {\ text {multi-modapal}} $在ndcg@3中获得11 \％的相对增益，用于多模式rag和$ \ text {rl-qr}猎犬。但是，语义和混合猎犬的挑战持续存在，在这种语义和混合猎犬中，重写者无法提高性能，这可能是由于训练未对准而造成的。我们的发现突出了RL-QR对抹布系统的查询优化的潜力，为真实世界检索任务提供了可扩展的，无注释的解决方案，同时识别了在语义检索环境中进一步完善的途径。

Title: UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editing

• Authors: Hao Tang, Chenwei Xie, Xiaoyi Bao, Tingyu Weng, Pandeng Li, Yun Zheng, Liwei Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23278
• Pdf URL: https://arxiv.org/pdf/2507.23278
• Copy Paste: [[2507.23278]] UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editing(https://arxiv.org/abs/2507.23278)
• Keywords: generation
• Abstract: In this paper, we propose UniLIP, which extends CLIP to reconstruction, generation and editing, thereby building a unified tokenizer upon its exceptional comprehension capabilities. Previous CLIP-based unified methods often require additional diffusion decoders or quantization to support reconstruction and generation tasks, leading to inconsistent reconstruction or degradation of original comprehension this http URL contrast, we introduce a two-stage training scheme and a self-distillation strategy that progressively integrates reconstruction capabilities into CLIP, allowing it to maintain original comprehension performance while achieving effective image reconstruction. Furthermore, we propose a dual-condition architecture to connect the MLLM and diffusion transformer, using both learnable queries and the last layer multimodal hidden states as joint conditions. This method not only enables the utilization of the MLLM's strong reasoning capabilities in generation tasks, but also maximizes the exploitation of the rich information in UniLIP features during editing tasks. In text-to-image generation tasks, UniLIP obtains scores of 0.87 and 0.53 on GenEval and WISE benchmark respectively, surpassing all previous unified models of similar scale. In image editing, UniLIP also achieves a score of 3.62 on the ImgEdit Benchmark, surpassing recent state-of-the-art models such as BAGEL and UniWorld-V1. UniLIP effectively expand the application scope of CLIP, enabling continuous CLIP features to not only serve as the optimal choice for understanding tasks but also achieve highly competitive performance in generation and editing tasks.
• 摘要：在本文中，我们提出了UNILIP，将剪辑扩展到重建，生成和编辑，从而在其出色的理解能力上构建统一的令牌。以前的基于片段的统一方法通常需要额外的扩散解码器或量化来支持重建和生成任务，从而导致不一致的重建或对原始理解的重新构建或退化，我们引入了两阶段的训练方案，并引入了两阶段的培训方案，并逐步将重建能力逐步整合到剪辑中，以使其能够促进剪辑，从而使其能够有效地进行启动。此外，我们建议使用可学习的查询和最后一层的多模式隐藏状态作为关节条件，以连接MLLM和扩散变压器，以连接MLLM和扩散变压器。这种方法不仅可以利用MLLM在生成任务中强大的推理能力，而且还可以在编辑任务期间最大程度地利用UNILIP功能中丰富的信息的利用。在文本到图像生成任务中，Unip分别在Geneval和Wise基准分别获得0.87和0.53的得分，超过了所有以前所有类似规模的统一模型。在图像编辑中，Unilip在IMGEDIT基准测试上也达到了3.62的得分，超过了最近最新的模型，例如Bagel和Uniworld-V1。 Unip有效地扩大了剪辑的应用程序范围，从而使连续的剪辑功能不仅可以作为理解任务的最佳选择，而且还可以在生成和编辑任务中实现高度竞争性的性能。

Title: PriorFusion: Unified Integration of Priors for Robust Road Perception in Autonomous Driving

• Authors: Xuewei Tang, Mengmeng Yang, Tuopu Wen, Peijin Jia, Le Cui, Mingshang Luo, Kehua Sheng, Bo Zhang, Diange Yang, Kun Jiang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23309
• Pdf URL: https://arxiv.org/pdf/2507.23309
• Copy Paste: [[2507.23309]] PriorFusion: Unified Integration of Priors for Robust Road Perception in Autonomous Driving(https://arxiv.org/abs/2507.23309)
• Keywords: generative
• Abstract: With the growing interest in autonomous driving, there is an increasing demand for accurate and reliable road perception technologies. In complex environments without high-definition map support, autonomous vehicles must independently interpret their surroundings to ensure safe and robust decision-making. However, these scenarios pose significant challenges due to the large number, complex geometries, and frequent occlusions of road elements. A key limitation of existing approaches lies in their insufficient exploitation of the structured priors inherently present in road elements, resulting in irregular, inaccurate predictions. To address this, we propose PriorFusion, a unified framework that effectively integrates semantic, geometric, and generative priors to enhance road element perception. We introduce an instance-aware attention mechanism guided by shape-prior features, then construct a data-driven shape template space that encodes low-dimensional representations of road elements, enabling clustering to generate anchor points as reference priors. We design a diffusion-based framework that leverages these prior anchors to generate accurate and complete predictions. Experiments on large-scale autonomous driving datasets demonstrate that our method significantly improves perception accuracy, particularly under challenging conditions. Visualization results further confirm that our approach produces more accurate, regular, and coherent predictions of road elements.
• 摘要：随着对自动驾驶的兴趣，人们对准确可靠的道路知觉技术的需求不断增长。在没有高清地图支持的复杂环境中，自动驾驶汽车必须独立解释其周围环境，以确保安全和强大的决策。但是，由于大量，复杂的几何形状和频繁的道路元素阻塞，这些情况构成了重大挑战。现有方法的关键局限性在于它们对道路元素固有存在的结构化先验的利用不足，导致了不规则的，不准确的预测。为了解决这个问题，我们提出了先验融合，这是一个有效整合语义，几何和生成先验的统一框架，以增强道路元素的感知。我们介绍了一种以形状优先特征为指导的实例感知机制，然后构建一个数据驱动的形状模板空间，该模板空间编码道路元素的低维表示，从而使聚类可以生成锚点作为参考先验。我们设计了一个基于扩散的框架，该框架利用这些先前的锚来生成准确而完整的预测。大规模自动驾驶数据集的实验表明，我们的方法显着提高了感知的准确性，尤其是在具有挑战性的条件下。可视化结果进一步证实，我们的方法对道路元素产生了更准确，规则和连贯的预测。

Title: The Cow of Rembrandt - Analyzing Artistic Prompt Interpretation in Text-to-Image Models

• Authors: Alfio Ferrara, Sergio Picascia, Elisabetta Rocchetti
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23313
• Pdf URL: https://arxiv.org/pdf/2507.23313
• Copy Paste: [[2507.23313]] The Cow of Rembrandt - Analyzing Artistic Prompt Interpretation in Text-to-Image Models(https://arxiv.org/abs/2507.23313)
• Keywords: generative
• Abstract: Text-to-image diffusion models have demonstrated remarkable capabilities in generating artistic content by learning from billions of images, including popular artworks. However, the fundamental question of how these models internally represent concepts, such as content and style in paintings, remains unexplored. Traditional computer vision assumes content and style are orthogonal, but diffusion models receive no explicit guidance about this distinction during training. In this work, we investigate how transformer-based text-to-image diffusion models encode content and style concepts when generating artworks. We leverage cross-attention heatmaps to attribute pixels in generated images to specific prompt tokens, enabling us to isolate image regions influenced by content-describing versus style-describing tokens. Our findings reveal that diffusion models demonstrate varying degrees of content-style separation depending on the specific artistic prompt and style requested. In many cases, content tokens primarily influence object-related regions while style tokens affect background and texture areas, suggesting an emergent understanding of the content-style distinction. These insights contribute to our understanding of how large-scale generative models internally represent complex artistic concepts without explicit supervision. We share the code and dataset, together with an exploratory tool for visualizing attention maps at this https URL.
• 摘要：文本到图像扩散模型通过从数十亿图像（包括流行的艺术品）中学习来生成艺术内容方面表现出了非凡的功能。但是，这些模型在内部如何代表概念（例如绘画中的内容和样式）的基本问题仍然没有探索。传统的计算机视觉假设内容和样式是正交的，但是扩散模型在训练过程中没有明确的指导。在这项工作中，我们研究了基于变压器的文本对图像扩散模型如何在生成艺术品时编码内容和样式概念。我们利用跨意图的热图将生成图像中的像素归因于特定的提示令牌，从而使我们能够隔离受内容描述与样式描述令牌影响的图像区域。我们的发现表明，扩散模型表明，根据所要求的特定艺术提示和样式，内容式分离程度不同。在许多情况下，内容令牌主要影响与对象相关的区域，而样式令牌会影响背景和纹理区域，这表明对内容式的区别有深刻的理解。这些见解有助于我们理解在没有明确监督的情况下内部代表复杂艺术概念的大规模生成模型。我们共享代码和数据集，以及一个探索性工具，可在此HTTPS URL上可视化注意力图。

Title: Good Learners Think Their Thinking: Generative PRM Makes Large Reasoning Model More Efficient Math Learner

• Authors: Tao He, Rongchuan Mu, Lizi Liao, Yixin Cao, Ming Liu, Bing Qin
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.23317
• Pdf URL: https://arxiv.org/pdf/2507.23317
• Copy Paste: [[2507.23317]] Good Learners Think Their Thinking: Generative PRM Makes Large Reasoning Model More Efficient Math Learner(https://arxiv.org/abs/2507.23317)
• Keywords: generative
• Abstract: Large reasoning models (LRMs) have recently shown promise in solving complex math problems when optimized with Reinforcement Learning (RL). But conventional approaches rely on outcome-only rewards that provide sparse feedback, resulting in inefficient optimization process. In this work, we investigate the function of process reward models (PRMs) to accelerate the RL training for LRMs. We propose a novel intrinsic signal-driven generative process evaluation mechanism operating at the thought level to address major bottlenecks in RL-based training. Specifically, instead of requiring PRMs to know how to solve problems, our method uses intrinsic signals in solutions to judge stepwise correctness and aggregate contiguous correct/incorrect steps into coherent 'thought' units. This structured, thought-level rewards enable more reliable credit assignment by reducing ambiguity in step segmentation and alleviating reward hacking. We further introduce a capability-adaptive reward mechanism that dynamically balances exploration and exploitation based on the LRM's current proficiency, guiding learning without stifling creative trial-and-error. These innovations are integrated into a new off-policy RL algorithm, TP-GRPO, which extends grouped proximal optimization with process-based rewards and improves training efficiency. Experiments on 1.5B and 7B parameter LRMs demonstrate that our method achieves higher problem-solving accuracy with significantly fewer training samples than outcome-only reward baselines. The results validate that well-structured process rewards can substantially accelerate LRM optimization in math reasoning tasks. Code is available at this https URL.
• 摘要：大型推理模型（LRMS）最近在解决增强学习（RL）时解决了解决复杂数学问题的希望。但是，传统的方法依赖于提供稀疏反馈的仅结局奖励，从而导致优化过程效率低下。在这项工作中，我们研究了过程奖励模型（PRM）的功能，以加速LRMS的RL培训。我们提出了一种在思想水平上运行的新型固有信号驱动的生成过程评估机制，以解决基于RL的训练中的主要瓶颈。具体而言，我们的方法不需要PRMS知道如何解决问题，而是在解决方案中使用固有的信号来判断逐步正确性，并将连续的“思想”单位串联正确/不正确的步骤汇总。这种结构化的，思想级别的奖励可以通过减少步骤细分和减轻奖励黑客的歧义来实现更可靠的信用分配。我们进一步介绍了一种能力自适应奖励机制，该机制基于LRM的当前水平，动态平衡探索和剥削，指导学习而不扼杀创意的试验和错误。这些创新被整合到一种新的非政策RL算法TP-GRPO中，该算法将基于过程的奖励扩展到分组的近端优化并提高培训效率。 1.5b和7b参数LRMS的实验表明，我们的方法可实现更高的问题解决准确性，而训练样本明显少于仅限结果奖励基准。结果证明了结构良好的过程奖励可以在数学推理任务中基本上加速LRM优化。代码可在此HTTPS URL上找到。

Title: Who is a Better Talker: Subjective and Objective Quality Assessment for AI-Generated Talking Heads

• Authors: Yingjie Zhou, Jiezhang Cao, Zicheng Zhang, Farong Wen, Yanwei Jiang, Jun Jia, Xiaohong Liu, Xiongkuo Min, Guangtao Zhai
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2507.23343
• Pdf URL: https://arxiv.org/pdf/2507.23343
• Copy Paste: [[2507.23343]] Who is a Better Talker: Subjective and Objective Quality Assessment for AI-Generated Talking Heads(https://arxiv.org/abs/2507.23343)
• Keywords: generation, quality assessment
• Abstract: Speech-driven methods for portraits are figuratively known as "Talkers" because of their capability to synthesize speaking mouth shapes and facial movements. Especially with the rapid development of the Text-to-Image (T2I) models, AI-Generated Talking Heads (AGTHs) have gradually become an emerging digital human media. However, challenges persist regarding the quality of these talkers and AGTHs they generate, and comprehensive studies addressing these issues remain limited. To address this gap, this paper presents the largest AGTH quality assessment dataset THQA-10K to date, which selects 12 prominent T2I models and 14 advanced talkers to generate AGTHs for 14 prompts. After excluding instances where AGTH generation is unsuccessful, the THQA-10K dataset contains 10,457 AGTHs. Then, volunteers are recruited to subjectively rate the AGTHs and give the corresponding distortion categories. In our analysis for subjective experimental results, we evaluate the performance of talkers in terms of generalizability and quality, and also expose the distortions of existing AGTHs. Finally, an objective quality assessment method based on the first frame, Y-T slice and tone-lip consistency is proposed. Experimental results show that this method can achieve state-of-the-art (SOTA) performance in AGTH quality assessment. The work is released at this https URL.
• 摘要：语音驱动的肖像方法被形象称为“谈话者”，因为它们可以合成口腔形状和面部运动的能力。特别是随着文本形象（T2I）模型的快速发展，AI生成的会说话的头（AGTH）逐渐成为新兴的数字人类媒体。但是，关于它们产生的这些说话者和AGTH的质量的挑战仍然存在，解决这些问题的全面研究仍然有限。为了解决这一差距，本文介绍了迄今为止最大的AGTH质量评估数据集THQA-10K，该数据集THQA-10K迄今为止选择了12种著名的T2I型号和14个高级讲话者，以生成14个提示的AGTH。在排除了第三代失败的实例之后，THQA-10K数据集包含10,457个agths。然后，招募志愿者以主观评价AGTH并给出相应的失真类别。在我们的主观实验结果分析中，我们评估了说话者在概括性和质量方面的表现，并暴露了现有AGTH的扭曲。最后，提出了基于第一帧的客观质量评估方法，提出了Y-T切片和音调唇的一致性。实验结果表明，此方法可以在AGTH质量评估中实现最先进的（SOTA）性能。该作品在此HTTPS URL上发布。

Title: IN45023 Neural Network Design Patterns in Computer Vision Seminar Report, Summer 2025

• Authors: Radu-Andrei Bourceanu, Neil De La Fuente, Jan Grimm, Andrei Jardan, Andriy Manucharyan, Cornelius Weiss, Roman Pflugfelder
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23357
• Pdf URL: https://arxiv.org/pdf/2507.23357
• Copy Paste: [[2507.23357]] IN45023 Neural Network Design Patterns in Computer Vision Seminar Report, Summer 2025(https://arxiv.org/abs/2507.23357)
• Keywords: generation, generative
• Abstract: This report analyzes the evolution of key design patterns in computer vision by examining six influential papers. The analy- sis begins with foundational architectures for image recognition. We review ResNet, which introduced residual connections to overcome the vanishing gradient problem and enable effective training of significantly deeper convolutional networks. Subsequently, we examine the Vision Transformer (ViT), which established a new paradigm by applying the Transformer ar- chitecture to sequences of image patches, demonstrating the efficacy of attention-based models for large-scale image recogni- tion. Building on these visual representation backbones, we investigate generative models. Generative Adversarial Networks (GANs) are analyzed for their novel adversarial training process, which challenges a generator against a discriminator to learn complex data distributions. Then, Latent Diffusion Models (LDMs) are covered, which improve upon prior generative methods by performing a sequential denoising process in a perceptually compressed latent space. LDMs achieve high-fidelity synthesis with greater computational efficiency, representing the current state-of-the-art for image generation. Finally, we explore self-supervised learning techniques that reduce dependency on labeled data. DINO is a self-distillation framework in which a student network learns to match the output of a momentum-updated teacher, yielding features with strong k-NN classification performance. We conclude with Masked Autoencoders (MAE), which utilize an asymmetric encoder-decoder design to reconstruct heavily masked inputs, providing a highly scalable and effective method for pre-training large-scale vision models.
• 摘要：该报告通过检查六篇有影响力的论文来分析计算机视觉中关键设计模式的演变。分析始于图像识别的基础体系结构。我们审查了重新连接，该重新连接引入了剩余连接，以克服消失的梯度问题，并有效地培训了更深层次的卷积网络。随后，我们检查了视觉变压器（VIT），该变压器（VIT）通过将变压器的插曲应用于图像贴片序列来建立新的范式，从而证明了基于注意力的模型对大规模图像识别的功效。在这些视觉表示主机的基础上，我们研究了生成模型。分析了生成对抗网络（GAN）的新型对抗训练过程，该过程向发电机挑战了歧视者，以学习复杂的数据分布。然后，涵盖了潜在扩散模型（LDMS），通过在感知压缩的潜在空间中执行顺序降解过程，可以改善先前的生成方法。 LDMS具有更高的计算效率实现高保真综合，代表了图像生成的当前最新技术。最后，我们探索自我监督的学习技术，以减少对标记数据的依赖。 Dino是一个自我验证框架，在该框架中，学生网络学会与势头更新的老师的输出相匹配，从而产生具有强大K-NN分类性能的功能。我们以蒙面自动编码器（MAE）的形式结束，该自动编码器利用不对称的编码器设计设计重建了重建屏蔽的输入，为预训练大规模视觉模型提供了一种高度可扩展有效的方法。

Title: UniEmo: Unifying Emotional Understanding and Generation with Learnable Expert Queries

• Authors: Yijie Zhu, Lingsen Zhang, Zitong Yu, Rui Shao, Tao Tan, Liqiang Nie
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23372
• Pdf URL: https://arxiv.org/pdf/2507.23372
• Copy Paste: [[2507.23372]] UniEmo: Unifying Emotional Understanding and Generation with Learnable Expert Queries(https://arxiv.org/abs/2507.23372)
• Keywords: generation
• Abstract: Emotional understanding and generation are often treated as separate tasks, yet they are inherently complementary and can mutually enhance each other. In this paper, we propose the UniEmo, a unified framework that seamlessly integrates these two tasks. The key challenge lies in the abstract nature of emotions, necessitating the extraction of visual representations beneficial for both tasks. To address this, we propose a hierarchical emotional understanding chain with learnable expert queries that progressively extracts multi-scale emotional features, thereby serving as a foundational step for unification. Simultaneously, we fuse these expert queries and emotional representations to guide the diffusion model in generating emotion-evoking images. To enhance the diversity and fidelity of the generated emotional images, we further introduce the emotional correlation coefficient and emotional condition loss into the fusion process. This step facilitates fusion and alignment for emotional generation guided by the understanding. In turn, we demonstrate that joint training allows the generation component to provide implicit feedback to the understanding part. Furthermore, we propose a novel data filtering algorithm to select high-quality and diverse emotional images generated by the well-trained model, which explicitly feedback into the understanding part. Together, these generation-driven dual feedback processes enhance the model's understanding capacity. Extensive experiments show that UniEmo significantly outperforms state-of-the-art methods in both emotional understanding and generation tasks. The code for the proposed method is available at this https URL.
• 摘要：情感理解和产生通常被视为单独的任务，但它们本质上是互补的，可以相互增强。在本文中，我们提出了Uniemo，这是一个无缝整合这两个任务的统一框架。关键挑战在于情绪的抽象性质，因此需要提取视觉表示对这两个任务有益。为了解决这个问题，我们通过可学习的专家查询提出了一个层次的情感理解链，该链逐渐提取多尺度的情感特征，从而成为统一的基础步骤。同时，我们融合了这些专家的疑问和情感表示，以指导传播模型产生情绪引起的图像。为了增强产生的情感图像的多样性和忠诚，我们进一步将情感相关系数和情感状况损失引入融合过程中。此步骤促进了以理解为指导的情感产生的融合和结盟。反过来，我们证明了联合培训使生成组件可以为理解部分提供隐式反馈。此外，我们提出了一种新型的数据过滤算法，以选择训练有素的模型产生的高质量和多样化的情感图像，该模型明确反馈到理解部分。这些一代驱动的双重反馈过程一起增强了模型的理解能力。广泛的实验表明，在情感理解和发电任务中，Uniemo显着优于最先进的方法。该方法的代码可在此HTTPS URL上获得。

Title: Policy Learning from Large Vision-Language Model Feedback without Reward Modeling

• Authors: Tung M. Luu, Donghoon Lee, Younghwan Lee, Chang D. Yoo
• Subjects: cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2507.23391
• Pdf URL: https://arxiv.org/pdf/2507.23391
• Copy Paste: [[2507.23391]] Policy Learning from Large Vision-Language Model Feedback without Reward Modeling(https://arxiv.org/abs/2507.23391)
• Keywords: generation
• Abstract: Offline reinforcement learning (RL) provides a powerful framework for training robotic agents using pre-collected, suboptimal datasets, eliminating the need for costly, time-consuming, and potentially hazardous online interactions. This is particularly useful in safety-critical real-world applications, where online data collection is expensive and impractical. However, existing offline RL algorithms typically require reward labeled data, which introduces an additional bottleneck: reward function design is itself costly, labor-intensive, and requires significant domain expertise. In this paper, we introduce PLARE, a novel approach that leverages large vision-language models (VLMs) to provide guidance signals for agent training. Instead of relying on manually designed reward functions, PLARE queries a VLM for preference labels on pairs of visual trajectory segments based on a language task description. The policy is then trained directly from these preference labels using a supervised contrastive preference learning objective, bypassing the need to learn explicit reward models. Through extensive experiments on robotic manipulation tasks from the MetaWorld, PLARE achieves performance on par with or surpassing existing state-of-the-art VLM-based reward generation methods. Furthermore, we demonstrate the effectiveness of PLARE in real-world manipulation tasks with a physical robot, further validating its practical applicability.
• 摘要：离线增强学习（RL）为使用预先收集的次优数据集训练机器人代理提供了一个有力的框架，从而消除了对昂贵，耗时和潜在危险在线互动的需求。这在关键的现实世界中特别有用，在线数据收集昂贵且不切实际。但是，现有的离线RL算法通常需要奖励标记的数据，该数据引入了额外的瓶颈：奖励功能设计本身是昂贵，劳动力密集的，并且需要重要的领域专业知识。在本文中，我们介绍了Plare，这是一种新型方法，利用大型视觉模型（VLMS）为代理培训提供指导信号。 Plare不用依靠手动设计的奖励功能，而是根据语言任务描述对VLM查询VLM以在成对的视觉轨迹段上查询偏好标签。然后，使用有监督的对比偏好学习目标直接从这些偏好标签中培训该政策，绕开了学习明确奖励模型的需求。通过对Metaworld的机器人操纵任务进行的广泛实验，Plare可以在与现有的最新VLM基于VLM的奖励生成方法相同或超越现有的绩效。此外，我们通过物理机器人证明了斑点在实际操纵任务中的有效性，从而进一步验证了其实际适用性。

Title: Out-of-Distribution Detection in Medical Imaging via Diffusion Trajectories

• Authors: Lemar Abdi, Francisco Caetano, Amaan Valiuddin, Christiaan Viviers, Hamdi Joudeh, Fons van der Sommen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23411
• Pdf URL: https://arxiv.org/pdf/2507.23411
• Copy Paste: [[2507.23411]] Out-of-Distribution Detection in Medical Imaging via Diffusion Trajectories(https://arxiv.org/abs/2507.23411)
• Keywords: generative
• Abstract: In medical imaging, unsupervised out-of-distribution (OOD) detection offers an attractive approach for identifying pathological cases with extremely low incidence rates. In contrast to supervised methods, OOD-based approaches function without labels and are inherently robust to data imbalances. Current generative approaches often rely on likelihood estimation or reconstruction error, but these methods can be computationally expensive, unreliable, and require retraining if the inlier data changes. These limitations hinder their ability to distinguish nominal from anomalous inputs efficiently, consistently, and robustly. We propose a reconstruction-free OOD detection method that leverages the forward diffusion trajectories of a Stein score-based denoising diffusion model (SBDDM). By capturing trajectory curvature via the estimated Stein score, our approach enables accurate anomaly scoring with only five diffusion steps. A single SBDDM pre-trained on a large, semantically aligned medical dataset generalizes effectively across multiple Near-OOD and Far-OOD benchmarks, achieving state-of-the-art performance while drastically reducing computational cost during inference. Compared to existing methods, SBDDM achieves a relative improvement of up to 10.43% and 18.10% for Near-OOD and Far-OOD detection, making it a practical building block for real-time, reliable computer-aided diagnosis.
• 摘要：在医学成像中，无监督的分布（OOD）检测提供了一种有吸引力的方法来识别发病率极低的病理病例。与监督方法相反，基于OOD的方法无标记的功能，并且对数据失衡具有固有的稳健性。当前的生成方法通常依赖于似然估计或重建错误，但是这些方法在计算上可能是昂贵的，不可靠的，并且如果近距离数据发生变化，则需要重新训练。这些局限性阻碍了它们有效，始终如一，稳健地区分异常输入的能力。我们提出了一种无重建的OOD检测方法，该方法利用基于Stein得分的DeNoising扩散模型（SBDDM）的正向扩散轨迹。通过通过估计的Stein分数捕获轨迹曲率，我们的方法仅能使用五个扩散步骤进行准确的异常评分。在大型的，具有语义上的大型医学数据集中进行了预训练的单个SBDDM在多个近似和Far-ood基准中有效地概括了，从而实现了最先进的性能，同时大大降低了推断期间的计算成本。与现有方法相比，SBDDM的相对改善可在附近和Far-OOD检测中获得高达10.43％和18.10％，这使其成为实时，可靠的计算机辅助诊断的实用构建块。

Title: Online Estimation of Table-Top Grown Strawberry Mass in Field Conditions with Occlusions

• Authors: Jinshan Zhen, Yuanyue Ge, Tianxiao Zhu, Hui Zhao, Ya Xiong
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2507.23487
• Pdf URL: https://arxiv.org/pdf/2507.23487
• Copy Paste: [[2507.23487]] Online Estimation of Table-Top Grown Strawberry Mass in Field Conditions with Occlusions(https://arxiv.org/abs/2507.23487)
• Keywords: generative
• Abstract: Accurate mass estimation of table-top grown strawberries under field conditions remains challenging due to frequent occlusions and pose variations. This study proposes a vision-based pipeline integrating RGB-D sensing and deep learning to enable non-destructive, real-time and online mass estimation. The method employed YOLOv8-Seg for instance segmentation, Cycle-consistent generative adversarial network (CycleGAN) for occluded region completion, and tilt-angle correction to refine frontal projection area calculations. A polynomial regression model then mapped the geometric features to mass. Experiments demonstrated mean mass estimation errors of 8.11% for isolated strawberries and 10.47% for occluded cases. CycleGAN outperformed large mask inpainting (LaMa) model in occlusion recovery, achieving superior pixel area ratios (PAR) (mean: 0.978 vs. 1.112) and higher intersection over union (IoU) scores (92.3% vs. 47.7% in the [0.9-1] range). This approach addresses critical limitations of traditional methods, offering a robust solution for automated harvesting and yield monitoring with complex occlusion patterns.
• 摘要：由于频繁的阻塞和姿势变化，在田间条件下对桌面种植的草莓的准确质量估计仍然具有挑战性。这项研究提出了基于视觉的管道，将RGB-D传感和深度学习整合在一起，以实现非破坏性，实时和在线质量估计。该方法采用Yolov8-Seg，例如分割，周期矛盾的生成对抗网络（Cyclegan）进行遮挡区域完成，以及倾斜角度校正以进行精炼的额叶投影面积计算。然后，多项式回归模型将几何特征映射到质量。实验表明，隔离草莓的平均质量估计误差为8.11％，闭塞病例的平均质量估计误差为10.47％。在遮挡恢复中，Cyclegan优于大面罩（LAMA）模型，达到上像素面积比（PAR）（平均值：0.978 vs. 1.112）和较高的联合（IOU）分数（92.3％vs. 47.7％在[0.9-1-1]范围内）。这种方法解决了传统方法的临界局限性，为自动收获提供了强大的解决方案，并具有复杂的遮挡模式。

Title: GraphRAG-R1: Graph Retrieval-Augmented Generation with Process-Constrained Reinforcement Learning

• Authors: Chuanyue Yu, Kuo Zhao, Yuhan Li, Heng Chang, Mingjian Feng, Xiangzhe Jiang, Yufei Sun, Jia Li, Yuzhi Zhang, Jianxin Li, Ziwei Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.23581
• Pdf URL: https://arxiv.org/pdf/2507.23581
• Copy Paste: [[2507.23581]] GraphRAG-R1: Graph Retrieval-Augmented Generation with Process-Constrained Reinforcement Learning(https://arxiv.org/abs/2507.23581)
• Keywords: generation
• Abstract: Graph Retrieval-Augmented Generation (GraphRAG) has shown great effectiveness in enhancing the reasoning abilities of LLMs by leveraging graph structures for knowledge representation and modeling complex real-world relationships. However, existing GraphRAG methods still face significant bottlenecks when handling complex problems that require multi-hop reasoning, as their query and retrieval phases are largely based on pre-defined heuristics and do not fully utilize the reasoning potentials of LLMs. To address this problem, we propose GraphRAG-R1, an adaptive GraphRAG framework by training LLMs with process-constrained outcome-based reinforcement learning (RL) to enhance the multi-hop reasoning ability. Our method can decompose complex problems, autonomously invoke retrieval tools to acquire necessary information, and perform effective reasoning. Specifically, we utilize a modified version of Group Relative Policy Optimization (GRPO) that supports rollout-with-thinking capability. Next, we design two process-constrained reward functions. To handle the shallow retrieval problem, we design a Progressive Retrieval Attenuation (PRA) reward to encourage essential retrievals. Then, to handle the over-thinking problem, we design Cost-Aware F1 (CAF) reward to balance the model performance with computational costs. We further design a phase-dependent training strategy, containing three training stages corresponding to cold start and these two rewards. Lastly, our method adopts a hybrid graph-textual retrieval to improve the reasoning capacity. Extensive experimental results demonstrate that GraphRAG-R1 boosts LLM capabilities in solving complex reasoning problems compared to state-of-the-art GraphRAG methods on both in-domain and out-of-domain datasets. Furthermore, our framework can be flexibly integrated with various existing retrieval methods, consistently delivering performance improvements.
• 摘要：图形检索增强的生成（GraphRag）通过利用图形结构来实现知识表示和建模复杂的现实世界关系，在增强LLM的推理能力方面表现出很大的有效性。但是，现有的GraphRag方法在处理需要多跳推理的复杂问题时仍然面临着重要的瓶颈，因为它们的查询和检索阶段在很大程度上基于预定的启发式方法，并且不完全利用LLMS的推理潜力。为了解决此问题，我们提出了通过训练基于过程受到结果的增强增强学习（RL）来提高多跳推理能力的llms graphRag-r1。我们的方法可以分解复杂的问题，自主调用检索工具以获取必要的信息并执行有效的推理。具体而言，我们利用了支持与思考能力的推出功能的组相对策略优化（GRPO）的修改版本。接下来，我们设计两个过程受限的奖励功能。为了解决浅检索问题，我们设计了渐进的检索衰减（PRA）奖励以鼓励基本的检索。然后，为了解决过度思考的问题，我们设计成本吸引的F1（CAF）奖励，以平衡模型性能与计算成本。我们进一步设计了一个依赖相关的训练策略，其中包含与冷启动和这两个奖励相对应的三个训练阶段。最后，我们的方法采用混合图形文本检索以提高推理能力。广泛的实验结果表明，与最新的域内和室外数据集上的最新绘图rag方法相比，GraphRag-R1在解决复杂的推理问题方面提高了LLM的能力。此外，我们的框架可以通过各种现有的检索方法灵活地集成，从而始终如一地提高性能。

Title: MoGA: 3D Generative Avatar Prior for Monocular Gaussian Avatar Reconstruction

• Authors: Zijian Dong, Longteng Duan, Jie Song, Michael J. Black, Andreas Geiger
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23597
• Pdf URL: https://arxiv.org/pdf/2507.23597
• Copy Paste: [[2507.23597]] MoGA: 3D Generative Avatar Prior for Monocular Gaussian Avatar Reconstruction(https://arxiv.org/abs/2507.23597)
• Keywords: generative
• Abstract: We present MoGA, a novel method to reconstruct high-fidelity 3D Gaussian avatars from a single-view image. The main challenge lies in inferring unseen appearance and geometric details while ensuring 3D consistency and realism. Most previous methods rely on 2D diffusion models to synthesize unseen views; however, these generated views are sparse and inconsistent, resulting in unrealistic 3D artifacts and blurred appearance. To address these limitations, we leverage a generative avatar model, that can generate diverse 3D avatars by sampling deformed Gaussians from a learned prior distribution. Due to the limited amount of 3D training data such a 3D model alone cannot capture all image details of unseen identities. Consequently, we integrate it as a prior, ensuring 3D consistency by projecting input images into its latent space and enforcing additional 3D appearance and geometric constraints. Our novel approach formulates Gaussian avatar creation as a model inversion process by fitting the generative avatar to synthetic views from 2D diffusion models. The generative avatar provides a meaningful initialization for model fitting, enforces 3D regularization, and helps in refining pose estimation. Experiments show that our method surpasses state-of-the-art techniques and generalizes well to real-world scenarios. Our Gaussian avatars are also inherently animatable
• 摘要：我们提出了Moga，这是一种从单视图像中重建高保真3D高斯化身的新方法。主要挑战在于推断出看不见的外观和几何细节，同时确保3D一致性和现实主义。大多数以前的方法都依赖2D扩散模型来综合看不见的视图。但是，这些产生的视图稀疏且不一致，导致了不切实际的3D伪像和外观模糊。为了解决这些局限性，我们利用一种生成的化身模型，可以通过从学到的先前分布中抽样变形的高斯人来产生不同的3D化身。由于3D训练数据的数量有限，因此仅3D模型无法捕获所有看不见身份的图像细节。因此，我们将其整合为先验，通过将输入图像投射到其潜在空间中并实施其他3D外观和几何约束来确保3D一致性。我们的新方法通过将生成化的化身拟合到来自2D扩散模型的合成视图，从而将高斯化身的创建作为模型反转过程。生成化的化身为模型拟合，强制执行3D正则化提供了有意义的初始化，并有助于完善姿势估计。实验表明，我们的方法超过了最先进的技术，并将其概括为现实情况。我们的高斯化身也是天生的动画

Title: EB-gMCR: Energy-Based Generative Modeling for Signal Unmixing and Multivariate Curve Resolution

• Authors: Yu-Tang Chang, Shih-Fang Chen
• Subjects: cs.LG, cs.CE
• Abstract URL: https://arxiv.org/abs/2507.23600
• Pdf URL: https://arxiv.org/pdf/2507.23600
• Copy Paste: [[2507.23600]] EB-gMCR: Energy-Based Generative Modeling for Signal Unmixing and Multivariate Curve Resolution(https://arxiv.org/abs/2507.23600)
• Keywords: generative
• Abstract: Signal unmixing analysis decomposes data into basic patterns and is widely applied in chemical and biological research. Multivariate curve resolution (MCR), a branch of signal unmixing, separates mixed chemical signals into base patterns (components) and their concentrations, playing a key role in understanding composition. Classical MCR is typically framed as matrix factorization (MF) and requires a user-specified component count, usually unknown in real data. As dataset size or component count increases, the scalability and reliability of MF-based MCR face significant challenges. This study reformulates MCR as a generative process (gMCR), and introduces an energy-based deep learning solver, EB-gMCR, that automatically discovers the smallest component set able to reconstruct the data faithfully. EB-gMCR starts from a large candidate pool (e.g., 1024 spectra) and employs a differentiable gating network to retain only active components while estimating their concentrations. On noisy synthetic datasets containing up to 256 latent sources, EB-gMCR maintained R^2 >= 0.98 and recovered the component count within 5% of the ground truth; at lower noise it achieved R^2 >= 0.99 with near exact component estimation. Additional chemical priors, such as non-negativity or nonlinear mixing, enter as simple plug-in functions, enabling adaptation to other instruments or domains without altering the core learning process. By uniting high-capacity generative modeling and hard component selection, EB-gMCR offers a practical route to large-scale signal unmixing analysis, including chemical library-driven scenarios. The source code is available at this https URL.
• 摘要：信号分析分析将数据分解为基本模式，并广泛应用于化学和生物学研究中。信号解体的一个分支多变量曲线分辨率（MCR）将混合的化学信号分为基础图案（组件）及其浓度，在理解组成方面起着关键作用。经典MCR通常被构架为矩阵分解（MF），需要用户指定的组件计数，通常在实际数据中未知。随着数据集大小或组件计数的增加，基于MF的MCR的可伸缩性和可靠性面临着重大挑战。这项研究将MCR重新定为生成过程（GMCR），并引入了基于能量的深度学习求解器EB-GMCR，该求EB-GMCR会自动发现能够忠实地重建数据的最小组件。 EB-GMCR从大型候选池（例如1024个光谱）开始，并采用可区分的门控网络仅保留活跃组件，同时估计其浓度。在包含多达256个潜在来源的嘈杂的合成数据集中，EB-GMCR保持r^2> = 0.98，并在地面真相的5％之内恢复了组件计数；在较低的噪声下，它实现了R^2> = 0.99，并具有接近精确的组件估计。其他化学先验（例如非阴性或非线性混合）以简单的插件功能输入，可以适应其他仪器或域，而无需更改核心学习过程。通过团结高容量生成的建模和硬组件选择，EB-GMCR提供了实用的大规模信号解散分析的途径，包括化学库驱动的方案。源代码可在此HTTPS URL上找到。

Title: L-GTA: Latent Generative Modeling for Time Series Augmentation

• Authors: Luis Roque, Carlos Soares, Vitor Cerqueira, Luis Torgo
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.23615
• Pdf URL: https://arxiv.org/pdf/2507.23615
• Copy Paste: [[2507.23615]] L-GTA: Latent Generative Modeling for Time Series Augmentation(https://arxiv.org/abs/2507.23615)
• Keywords: generative
• Abstract: Data augmentation is gaining importance across various aspects of time series analysis, from forecasting to classification and anomaly detection tasks. We introduce the Latent Generative Transformer Augmentation (L-GTA) model, a generative approach using a transformer-based variational recurrent autoencoder. This model uses controlled transformations within the latent space of the model to generate new time series that preserve the intrinsic properties of the original dataset. L-GTA enables the application of diverse transformations, ranging from simple jittering to magnitude warping, and combining these basic transformations to generate more complex synthetic time series datasets. Our evaluation of several real-world datasets demonstrates the ability of L-GTA to produce more reliable, consistent, and controllable augmented data. This translates into significant improvements in predictive accuracy and similarity measures compared to direct transformation methods.
• 摘要：从预测到分类和异常检测任务的时间序列分析的各个方面，数据增强变得重要。我们介绍了使用基于变压器的复发自动编码器的潜在生成变压器增强（L-GTA）模型，这是一种生成方法。该模型使用模型潜在空间内的受控转换来生成新的时间序列，以保留原始数据集的内在属性。 L-GTA实现了各种变换的应用，从简单的抖动到幅度的扭曲，以及结合这些基本转换以生成更复杂的合成时间序列数据集。我们对几个现实世界数据集的评估证明了L-GTA产生更可靠，一致和可控的增强数据的能力。与直接转化方法相比，这转化为预测准确性和相似性度量的显着改善。

Title: DivControl: Knowledge Diversion for Controllable Image Generation

• Authors: Yucheng Xie, Fu Feng, Ruixiao Shi, Jing Wang, Yong Rui, Xin Geng
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.23620
• Pdf URL: https://arxiv.org/pdf/2507.23620
• Copy Paste: [[2507.23620]] DivControl: Knowledge Diversion for Controllable Image Generation(https://arxiv.org/abs/2507.23620)
• Keywords: generation
• Abstract: Diffusion models have advanced from text-to-image (T2I) to image-to-image (I2I) generation by incorporating structured inputs such as depth maps, enabling fine-grained spatial control. However, existing methods either train separate models for each condition or rely on unified architectures with entangled representations, resulting in poor generalization and high adaptation costs for novel conditions. To this end, we propose DivControl, a decomposable pretraining framework for unified controllable generation and efficient adaptation. DivControl factorizes ControlNet via SVD into basic components-pairs of singular vectors-which are disentangled into condition-agnostic learngenes and condition-specific tailors through knowledge diversion during multi-condition training. Knowledge diversion is implemented via a dynamic gate that performs soft routing over tailors based on the semantics of condition instructions, enabling zero-shot generalization and parameter-efficient adaptation to novel conditions. To further improve condition fidelity and training efficiency, we introduce a representation alignment loss that aligns condition embeddings with early diffusion features. Extensive experiments demonstrate that DivControl achieves state-of-the-art controllability with 36.4$\times$ less training cost, while simultaneously improving average performance on basic conditions. It also delivers strong zero-shot and few-shot performance on unseen conditions, demonstrating superior scalability, modularity, and transferability.
• 摘要：扩散模型通过合并结构化输入（例如深度图），从而实现了细粒的空间控制，从文本到图像（T2I）到图像对图像（I2i）的生成。但是，现有方法要么为每种条件训练单独的模型，要么依靠具有纠缠表示的统一体系结构，从而导致概括和新颖条件的高适应性成本。为此，我们提出了DivContol，这是一个可分解的预处理框架，用于统一可控的生成和有效的适应。 DivConconol通过SVD将控制网络分配到单数矢量的基本组件对中 - 在多条件培训期间，通过知识转移将其分解为条件 - 不合骨学习基因和特定条件的裁缝。知识转移是通过动态门实现的，该动态门根据条件指令的语义对裁缝执行软路由，从而使零弹性概括和对新条件的参数有效适应。为了进一步提高条件保真度和训练效率，我们引入了表示对准损失，该损失将嵌入条件嵌入与早期扩散特征保持一致。广泛的实验表明，Divcontol以36.4 $ \ tims $少$培训成本实现最新的可控性，同时提高基本条件下的平均绩效。它还在看不见的条件下提供了强劲的零击和几乎没有射击的性能，表明了卓越的可伸缩性，模块化和可传递性。

Title: Adaptively Distilled ControlNet: Accelerated Training and Superior Sampling for Medical Image Synthesis

• Authors: Kunpeng Qiu, Zhiying Zhou, Yongxin Guo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23652
• Pdf URL: https://arxiv.org/pdf/2507.23652
• Copy Paste: [[2507.23652]] Adaptively Distilled ControlNet: Accelerated Training and Superior Sampling for Medical Image Synthesis(https://arxiv.org/abs/2507.23652)
• Keywords: generation
• Abstract: Medical image annotation is constrained by privacy concerns and labor-intensive labeling, significantly limiting the performance and generalization of segmentation models. While mask-controllable diffusion models excel in synthesis, they struggle with precise lesion-mask alignment. We propose \textbf{Adaptively Distilled ControlNet}, a task-agnostic framework that accelerates training and optimization through dual-model distillation. Specifically, during training, a teacher model, conditioned on mask-image pairs, regularizes a mask-only student model via predicted noise alignment in parameter space, further enhanced by adaptive regularization based on lesion-background ratios. During sampling, only the student model is used, enabling privacy-preserving medical image generation. Comprehensive evaluations on two distinct medical datasets demonstrate state-of-the-art performance: TransUNet improves mDice/mIoU by 2.4%/4.2% on KiTS19, while SANet achieves 2.6%/3.5% gains on Polyps, highlighting its effectiveness and superiority. Code is available at GitHub.
• 摘要：医疗图像注释受到隐私问题和劳动密集型标签的限制，严重限制了分割模型的性能和概括。尽管可控制的扩散模型在合成方面表现出色，但它们在精确的病变掩盖对准方面挣扎。我们提出\ textbf {自适应蒸馏的controlnet}，这是一个任务无关的框架，可通过双模型蒸馏加速训练和优化。具体而言，在训练期间，以面膜图像对为条件的教师模型通过参数空间中的预测噪声对准纯学生模型正规化，并通过基于病变背景比率的自适应正则化进一步增强。在抽样过程中，仅使用学生模型，可以生成隐私的医学图像。对两个不同的医疗数据集的全面评估表明了最先进的性能：Transunet在Kits19上将MDICE/MIOU提高了2.4％/4.2％，而SANET在息肉上获得了2.6％/3.5％的增长，强调了其有效性和优势。代码可在GitHub上找到。

Title: DepMicroDiff: Diffusion-Based Dependency-Aware Multimodal Imputation for Microbiome Data

• Authors: Rabeya Tus Sadia, Qiang Cheng
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23676
• Pdf URL: https://arxiv.org/pdf/2507.23676
• Copy Paste: [[2507.23676]] DepMicroDiff: Diffusion-Based Dependency-Aware Multimodal Imputation for Microbiome Data(https://arxiv.org/abs/2507.23676)
• Keywords: generative
• Abstract: Microbiome data analysis is essential for understanding host health and disease, yet its inherent sparsity and noise pose major challenges for accurate imputation, hindering downstream tasks such as biomarker discovery. Existing imputation methods, including recent diffusion-based models, often fail to capture the complex interdependencies between microbial taxa and overlook contextual metadata that can inform imputation. We introduce DepMicroDiff, a novel framework that combines diffusion-based generative modeling with a Dependency-Aware Transformer (DAT) to explicitly capture both mutual pairwise dependencies and autoregressive relationships. DepMicroDiff is further enhanced by VAE-based pretraining across diverse cancer datasets and conditioning on patient metadata encoded via a large language model (LLM). Experiments on TCGA microbiome datasets show that DepMicroDiff substantially outperforms state-of-the-art baselines, achieving higher Pearson correlation (up to 0.712), cosine similarity (up to 0.812), and lower RMSE and MAE across multiple cancer types, demonstrating its robustness and generalizability for microbiome imputation.
• 摘要：微生物组数据分析对于理解宿主健康和疾病至关重要，但是它固有的稀疏性和噪声对准确的插补构成了重大挑战，从而阻碍了下游任务，例如生物标志物发现。现有的插补方法，包括最近基于扩散的模型，通常无法捕获微生物分类单元和忽略可以告知插补的上下文元数据之间的复杂相互依赖性。我们介绍了Depmicrodiff，这是一个新颖的框架，将基于扩散的生成建模与依赖性感知变压器（DAT）结合在一起，以明确捕获共同成对的依赖性和自回旋关系。通过在各种癌症数据集中进行基于VAE的预处理以及对通过大型语言模型（LLM）编码的患者元数据进行条件的条件，进一步增强了Depmicrodiff。 TCGA微生物组数据集的实验表明，Depmicrodiff的表现要优于最先进的基线，实现了较高的Pearson相关性（高达0.712），余弦相似性（高达0.812）（高达0.812），并且在多种癌症中较低的RMSE和MAE，展示了其强大的微生物和一般性的微生不足。

Title: I2V-GS: Infrastructure-to-Vehicle View Transformation with Gaussian Splatting for Autonomous Driving Data Generation

• Authors: Jialei Chen, Wuhao Xu, Sipeng He, Baoru Huang, Dongchun Ren
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23683
• Pdf URL: https://arxiv.org/pdf/2507.23683
• Copy Paste: [[2507.23683]] I2V-GS: Infrastructure-to-Vehicle View Transformation with Gaussian Splatting for Autonomous Driving Data Generation(https://arxiv.org/abs/2507.23683)
• Keywords: generation
• Abstract: Vast and high-quality data are essential for end-to-end autonomous driving systems. However, current driving data is mainly collected by vehicles, which is expensive and inefficient. A potential solution lies in synthesizing data from real-world images. Recent advancements in 3D reconstruction demonstrate photorealistic novel view synthesis, highlighting the potential of generating driving data from images captured on the road. This paper introduces a novel method, I2V-GS, to transfer the Infrastructure view To the Vehicle view with Gaussian Splatting. Reconstruction from sparse infrastructure viewpoints and rendering under large view transformations is a challenging problem. We adopt the adaptive depth warp to generate dense training views. To further expand the range of views, we employ a cascade strategy to inpaint warped images, which also ensures inpainting content is consistent across views. To further ensure the reliability of the diffusion model, we utilize the cross-view information to perform a confidenceguided optimization. Moreover, we introduce RoadSight, a multi-modality, multi-view dataset from real scenarios in infrastructure views. To our knowledge, I2V-GS is the first framework to generate autonomous driving datasets with infrastructure-vehicle view transformation. Experimental results demonstrate that I2V-GS significantly improves synthesis quality under vehicle view, outperforming StreetGaussian in NTA-Iou, NTL-Iou, and FID by 45.7%, 34.2%, and 14.9%, respectively.
• 摘要：庞大而高质量的数据对于端到端的自动驾驶系统至关重要。但是，当前的驾驶数据主要由昂贵且效率低下的车辆收集。潜在的解决方案在于从现实世界图像中综合数据。 3D重建中的最新进展表明了感性的新型视图综合，突出了从道路上捕获的图像中生成驱动数据的潜力。本文介绍了一种新颖的方法I2V-GS，将基础设施视图传递给了高斯裂开的车辆视图。从稀疏的基础架构观点和大型视图转换下渲染的重建是一个具有挑战性的问题。我们采用自适应深度翘曲来产生密集的训练观点。为了进一步扩大视图范围，我们采用级联策略来注册扭曲的图像，这也确保了整个视图的内容一致。为了进一步确保扩散模型的可靠性，我们利用跨视图信息来执行信心指导的优化。此外，我们介绍了Roadsight，这是一种来自基础架构视图中实际场景的多模式，多视图数据集。据我们所知，I2V-GS是使用基础架构 - 车辆视图转换生成自动驾驶数据集的第一个框架。实验结果表明，I2V-GS在车辆视图下显着提高了合成质量，在NTA-IOU，NTL-IOU和FID中的表现分别超过45.7％，34.2％和14.9％。

Title: UniLDiff: Unlocking the Power of Diffusion Priors for All-in-One Image Restoration

• Authors: Zihan Cheng, Liangtai Zhou, Dian Chen, Ni Tang, Xiaotong Luo, Yanyun Qu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23685
• Pdf URL: https://arxiv.org/pdf/2507.23685
• Copy Paste: [[2507.23685]] UniLDiff: Unlocking the Power of Diffusion Priors for All-in-One Image Restoration(https://arxiv.org/abs/2507.23685)
• Keywords: restoration, generative
• Abstract: All-in-One Image Restoration (AiOIR) has emerged as a promising yet challenging research direction. To address its core challenges, we propose a novel unified image restoration framework based on latent diffusion models (LDMs). Our approach structurally integrates low-quality visual priors into the diffusion process, unlocking the powerful generative capacity of diffusion models for diverse degradations. Specifically, we design a Degradation-Aware Feature Fusion (DAFF) module to enable adaptive handling of diverse degradation types. Furthermore, to mitigate detail loss caused by the high compression and iterative sampling of LDMs, we design a Detail-Aware Expert Module (DAEM) in the decoder to enhance texture and fine-structure recovery. Extensive experiments across multi-task and mixed degradation settings demonstrate that our method consistently achieves state-of-the-art performance, highlighting the practical potential of diffusion priors for unified image restoration. Our code will be released.
• 摘要：多合一的图像恢复（Aioir）已成为一个有前途但充满挑战的研究方向。为了应对其核心挑战，我们提出了一个基于潜在扩散模型（LDMS）的新型统一图像恢复框架。我们的方法在结构上将低质量的视觉先验整合到扩散过程中，从而释放了扩散模型的强大生成能力，以促进各种降解。具体而言，我们设计了降解感知功能融合（DAFF）模块，以实现各种降解类型的自适应处理。此外，为了减轻由LDM的高压缩和迭代采样引起的细节损失，我们在解码器中设计了一个细节感知的专家模块（DAEM），以增强纹理和精细结构恢复。跨多任务和混合降解设置进行的广泛实验表明，我们的方法始终达到最先进的性能，突出了扩散先验的实践潜力，以实现统一图像恢复。我们的代码将发布。

Title: DiffuMatch: Category-Agnostic Spectral Diffusion Priors for Robust Non-rigid Shape Matching

• Authors: Emery Pierson, Lei Li, Angela Dai, Maks Ovsjanikov
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23715
• Pdf URL: https://arxiv.org/pdf/2507.23715
• Copy Paste: [[2507.23715]] DiffuMatch: Category-Agnostic Spectral Diffusion Priors for Robust Non-rigid Shape Matching(https://arxiv.org/abs/2507.23715)
• Keywords: generative
• Abstract: Deep functional maps have recently emerged as a powerful tool for solving non-rigid shape correspondence tasks. Methods that use this approach combine the power and flexibility of the functional map framework, with data-driven learning for improved accuracy and generality. However, most existing methods in this area restrict the learning aspect only to the feature functions and still rely on axiomatic modeling for formulating the training loss or for functional map regularization inside the networks. This limits both the accuracy and the applicability of the resulting approaches only to scenarios where assumptions of the axiomatic models hold. In this work, we show, for the first time, that both in-network regularization and functional map training can be replaced with data-driven methods. For this, we first train a generative model of functional maps in the spectral domain using score-based generative modeling, built from a large collection of high-quality maps. We then exploit the resulting model to promote the structural properties of ground truth functional maps on new shape collections. Remarkably, we demonstrate that the learned models are category-agnostic, and can fully replace commonly used strategies such as enforcing Laplacian commutativity or orthogonality of functional maps. Our key technical contribution is a novel distillation strategy from diffusion models in the spectral domain. Experiments demonstrate that our learned regularization leads to better results than axiomatic approaches for zero-shot non-rigid shape matching. Our code is available at: this https URL
• 摘要：深度功能图最近已成为解决非刚性形状对应任务的强大工具。使用这种方法的方法将功能地图框架的功能和灵活性与数据驱动的学习相结合，以提高准确性和通用性。但是，该领域中的大多数现有方法仅将学习方面仅限于特征功能，并且仍然依靠公理建模来制定训练损失或网络内部的功能地图正则化。这仅限于所得方法的准确性和适用性，仅适用于公理模型假设所具有的场景。在这项工作中，我们首次表明，网络正规化和功能地图训练都可以用数据驱动的方法代替。为此，我们首先使用基于得分的生成模型来训练光谱域中功能图的生成模型，该模型是由大量高质量地图构建的。然后，我们利用所得模型来促进新形状集合上地面真实功能图的结构特性。值得注意的是，我们证明了学识渊博的模型是类别不可知的，并且可以完全替换常用的策略，例如执行Laplacian的通勤性或功能图的正交性。我们的关键技术贡献是从光谱域中扩散模型的一种新颖的蒸馏策略。实验表明，与零射击非刚性形状匹配的公理方法相比，我们学到的正则化可以取得更好的结果。我们的代码可用：此HTTPS URL

Title: Gaussian Variation Field Diffusion for High-fidelity Video-to-4D Synthesis

• Authors: Bowen Zhang, Sicheng Xu, Chuxin Wang, Jiaolong Yang, Feng Zhao, Dong Chen, Baining Guo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.23785
• Pdf URL: https://arxiv.org/pdf/2507.23785
• Copy Paste: [[2507.23785]] Gaussian Variation Field Diffusion for High-fidelity Video-to-4D Synthesis(https://arxiv.org/abs/2507.23785)
• Keywords: generation
• Abstract: In this paper, we present a novel framework for video-to-4D generation that creates high-quality dynamic 3D content from single video inputs. Direct 4D diffusion modeling is extremely challenging due to costly data construction and the high-dimensional nature of jointly representing 3D shape, appearance, and motion. We address these challenges by introducing a Direct 4DMesh-to-GS Variation Field VAE that directly encodes canonical Gaussian Splats (GS) and their temporal variations from 3D animation data without per-instance fitting, and compresses high-dimensional animations into a compact latent space. Building upon this efficient representation, we train a Gaussian Variation Field diffusion model with temporal-aware Diffusion Transformer conditioned on input videos and canonical GS. Trained on carefully-curated animatable 3D objects from the Objaverse dataset, our model demonstrates superior generation quality compared to existing methods. It also exhibits remarkable generalization to in-the-wild video inputs despite being trained exclusively on synthetic data, paving the way for generating high-quality animated 3D content. Project page: this https URL.
• 摘要：在本文中，我们为视频到4D的新框架提供了一个新颖的框架，该框架从单个视频输入中创建了高质量的动态3D内容。直接4D扩散建模由于昂贵的数据构建以及共同代表3D形状，外观和运动的高维质而极具挑战性。我们通过引入直接编码规范高斯夹层（GS）的直接4Dmesh-to-GS变化场来解决这些挑战，及其从3D动画数据中的时间变化而无需每天拟合，并将高维动画压缩到一个紧凑的潜在潜在空间中。在这种有效的表示的基础上，我们使用在输入视频和规范GS的条件下，使用时间感知的扩散变压器来训练高斯变化场扩散模型。与现有方法相比，我们的模型对来自Objaverse数据集进行了精心策划的动画3D对象的培训。尽管接受了合成数据的培训，但它也表现出对野外视频输入的显着概括，为生成高质量的动画3D内容铺平了道路。项目页面：此HTTPS URL。