2025-01-20

Title: VideoWorld: Exploring Knowledge Learning from Unlabeled Videos

• Authors: Zhongwei Ren, Yunchao Wei, Xun Guo, Yao Zhao, Bingyi Kang, Jiashi Feng, Xiaojie Jin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.09781
• Pdf URL: https://arxiv.org/pdf/2501.09781
• Copy Paste: [[2501.09781]] VideoWorld: Exploring Knowledge Learning from Unlabeled Videos(https://arxiv.org/abs/2501.09781)
• Keywords: generation, generative
• Abstract: This work explores whether a deep generative model can learn complex knowledge solely from visual input, in contrast to the prevalent focus on text-based models like large language models (LLMs). We develop VideoWorld, an auto-regressive video generation model trained on unlabeled video data, and test its knowledge acquisition abilities in video-based Go and robotic control tasks. Our experiments reveal two key findings: (1) video-only training provides sufficient information for learning knowledge, including rules, reasoning and planning capabilities, and (2) the representation of visual change is crucial for knowledge acquisition. To improve both the efficiency and efficacy of this process, we introduce the Latent Dynamics Model (LDM) as a key component of VideoWorld. Remarkably, VideoWorld reaches a 5-dan professional level in the Video-GoBench with just a 300-million-parameter model, without relying on search algorithms or reward mechanisms typical in reinforcement learning. In robotic tasks, VideoWorld effectively learns diverse control operations and generalizes across environments, approaching the performance of oracle models in CALVIN and RLBench. This study opens new avenues for knowledge acquisition from visual data, with all code, data, and models open-sourced for further research.
• 摘要：这项研究探索了深度生成模型是否可以仅从视觉输入中学习复杂知识，这与普遍关注的基于文本的模型（如大型语言模型 (LLM)）形成鲜明对比。我们开发了 VideoWorld，这是一种在未标记视频数据上训练的自回归视频生成模型，并在基于视频的围棋和机器人控制任务中测试了其知识获取能力。我们的实验揭示了两个关键发现：（1）纯视频训练为学习知识提供了足够的信息，包括规则、推理和规划能力；（2）视觉变化的表示对于知识获取至关重要。为了提高这一过程的效率和功效，我们引入了潜在动力学模型 (LDM) 作为 VideoWorld 的关键组件。值得注意的是，VideoWorld 仅使用 3 亿参数模型就达到了 Video-GoBench 中的 5 段专业水平，而无需依赖强化学习中常见的搜索算法或奖励机制。在机器人任务中，VideoWorld 有效地学习了各种控制操作并跨环境进行泛化，接近 CALVIN 和 RLBench 中 oracle 模型的性能。这项研究为从视觉数据中获取知识开辟了新的途径，所有代码、数据和模型均开源以供进一步研究。

Title: Lossy Compression with Pretrained Diffusion Models

• Authors: Jeremy Vonderfecht, Feng Liu
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2501.09815
• Pdf URL: https://arxiv.org/pdf/2501.09815
• Copy Paste: [[2501.09815]] Lossy Compression with Pretrained Diffusion Models(https://arxiv.org/abs/2501.09815)
• Keywords: generative
• Abstract: We apply the DiffC algorithm (Theis et al. 2022) to Stable Diffusion 1.5, 2.1, XL, and Flux-dev, and demonstrate that these pretrained models are remarkably capable lossy image compressors. A principled algorithm for lossy compression using pretrained diffusion models has been understood since at least Ho et al. 2020, but challenges in reverse-channel coding have prevented such algorithms from ever being fully implemented. We introduce simple workarounds that lead to the first complete implementation of DiffC, which is capable of compressing and decompressing images using Stable Diffusion in under 10 seconds. Despite requiring no additional training, our method is competitive with other state-of-the-art generative compression methods at low ultra-low bitrates.
• 摘要：我们将 DiffC 算法 (Theis 等人，2022) 应用于 Stable Diffusion 1.5、2.1、XL 和 Flux-dev，并证明这些预训练模型是性能卓越的有损图像压缩器。自 Ho 等人 2020 年以来，人们已经了解了使用预训练扩散模型进行有损压缩的原理算法，但反向通道编码中的挑战阻碍了此类算法的完全实现。我们介绍了一些简单的解决方法，从而首次完整实现了 DiffC，它能够在 10 秒内使用 Stable Diffusion 压缩和解压缩图像。尽管不需要额外的训练，但我们的方法在低超低比特率下与其他最先进的生成压缩方法具有竞争力。

Title: Generalized Single-Image-Based Morphing Attack Detection Using Deep Representations from Vision Transformer

• Authors: Haoyu Zhang, Raghavendra Ramachandra, Kiran Raja, Christoph Busch
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.09817
• Pdf URL: https://arxiv.org/pdf/2501.09817
• Copy Paste: [[2501.09817]] Generalized Single-Image-Based Morphing Attack Detection Using Deep Representations from Vision Transformer(https://arxiv.org/abs/2501.09817)
• Keywords: generation
• Abstract: Face morphing attacks have posed severe threats to Face Recognition Systems (FRS), which are operated in border control and passport issuance use cases. Correspondingly, morphing attack detection algorithms (MAD) are needed to defend against such attacks. MAD approaches must be robust enough to handle unknown attacks in an open-set scenario where attacks can originate from various morphing generation algorithms, post-processing and the diversity of printers/scanners. The problem of generalization is further pronounced when the detection has to be made on a single suspected image. In this paper, we propose a generalized single-image-based MAD (S-MAD) algorithm by learning the encoding from Vision Transformer (ViT) architecture. Compared to CNN-based architectures, ViT model has the advantage on integrating local and global information and hence can be suitable to detect the morphing traces widely distributed among the face region. Extensive experiments are carried out on face morphing datasets generated using publicly available FRGC face datasets. Several state-of-the-art (SOTA) MAD algorithms, including representative ones that have been publicly evaluated, have been selected and benchmarked with our ViT-based approach. Obtained results demonstrate the improved detection performance of the proposed S-MAD method on inter-dataset testing (when different data is used for training and testing) and comparable performance on intra-dataset testing (when the same data is used for training and testing) experimental protocol.
• 摘要：人脸变形攻击对边境管制和护照签发用例中运行的人脸识别系统 (FRS) 构成了严重威胁。相应地，需要变形攻击检测算法 (MAD) 来防御此类攻击。MAD 方法必须足够强大，才能在开放场景中处理未知攻击，其中攻击可能源自各种变形生成算法、后处理和打印机/扫描仪的多样性。当必须对单个可疑图像进行检测时，泛化问题更加突出。在本文中，我们通过学习 Vision Transformer (ViT) 架构的编码，提出了一种基于单图像的广义 MAD (S-MAD) 算法。与基于 CNN 的架构相比，ViT 模型在整合局部和全局信息方面具有优势，因此适合检测广泛分布在人脸区域中的变形痕迹。在使用公开可用的 FRGC 人脸数据集生成的人脸变形数据集上进行了广泛的实验。我们选择了几种最先进的 (SOTA) MAD 算法，包括已公开评估的代表性算法，并使用基于 ViT 的方法对其进行了基准测试。所得结果表明，所提出的 S-MAD 方法在数据集间测试（当使用不同的数据进行训练和测试时）上的检测性能有所提高，并且在数据集内测试（当使用相同的数据进行训练和测试时）实验协议上具有可比的性能。

Title: BN-Pool: a Bayesian Nonparametric Approach to Graph Pooling

• Authors: Daniele Castellana, Filippo Maria Bianchi
• Subjects: cs.LG, math.PR
• Abstract URL: https://arxiv.org/abs/2501.09821
• Pdf URL: https://arxiv.org/pdf/2501.09821
• Copy Paste: [[2501.09821]] BN-Pool: a Bayesian Nonparametric Approach to Graph Pooling(https://arxiv.org/abs/2501.09821)
• Keywords: generative
• Abstract: We introduce BN-Pool, the first clustering-based pooling method for Graph Neural Networks (GNNs) that adaptively determines the number of supernodes in a coarsened graph. By leveraging a Bayesian non-parametric framework, BN-Pool employs a generative model capable of partitioning graph nodes into an unbounded number of clusters. During training, we learn the node-to-cluster assignments by combining the supervised loss of the downstream task with an unsupervised auxiliary term, which encourages the reconstruction of the original graph topology while penalizing unnecessary proliferation of clusters. This adaptive strategy allows BN-Pool to automatically discover an optimal coarsening level, offering enhanced flexibility and removing the need to specify sensitive pooling ratios. We show that BN-Pool achieves superior performance across diverse benchmarks.
• 摘要：我们引入了 BN-Pool，这是第一个基于聚类的图神经网络 (GNN) 池化方法，它可以自适应地确定粗化图中的超节点数量。通过利用贝叶斯非参数框架，BN-Pool 采用一种生成模型，能够将图节点划分为无限数量的簇。在训练期间，我们通过将下游任务的监督损失与无监督辅助项相结合来学习节点到簇的分配，这鼓励重建原始图拓扑，同时惩罚不必要的簇扩散。这种自适应策略允许 BN-Pool 自动发现最佳粗化级别，提供增强的灵活性，并且无需指定敏感的池化比率。我们表明 BN-Pool 在各种基准测试中都实现了卓越的性能。

Title: PIXELS: Progressive Image Xemplar-based Editing with Latent Surgery

• Authors: Shristi Das Biswas, Matthew Shreve, Xuelu Li, Prateek Singhal, Kaushik Roy
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.09826
• Pdf URL: https://arxiv.org/pdf/2501.09826
• Copy Paste: [[2501.09826]] PIXELS: Progressive Image Xemplar-based Editing with Latent Surgery(https://arxiv.org/abs/2501.09826)
• Keywords: generation
• Abstract: Recent advancements in language-guided diffusion models for image editing are often bottle-necked by cumbersome prompt engineering to precisely articulate desired changes. An intuitive alternative calls on guidance from in-the-wild image exemplars to help users bring their imagined edits to life. Contemporary exemplar-based editing methods shy away from leveraging the rich latent space learnt by pre-existing large text-to-image (TTI) models and fall back on training with curated objective functions to achieve the task. Though somewhat effective, this demands significant computational resources and lacks compatibility with diverse base models and arbitrary exemplar count. On further investigation, we also find that these techniques restrict user control to only applying uniform global changes over the entire edited region. In this paper, we introduce a novel framework for progressive exemplar-driven editing with off-the-shelf diffusion models, dubbed PIXELS, to enable customization by providing granular control over edits, allowing adjustments at the pixel or region level. Our method operates solely during inference to facilitate imitative editing, enabling users to draw inspiration from a dynamic number of reference images, or multimodal prompts, and progressively incorporate all the desired changes without retraining or fine-tuning existing TTI models. This capability of fine-grained control opens up a range of new possibilities, including selective modification of individual objects and specifying gradual spatial changes. We demonstrate that PIXELS delivers high-quality edits efficiently, leading to a notable improvement in quantitative metrics as well as human evaluation. By making high-quality image editing more accessible, PIXELS has the potential to enable professional-grade edits to a wider audience with the ease of using any open-source image generation model.
• 摘要：用于图像编辑的语言引导扩散模型的最新进展通常受到繁琐的提示工程的阻碍，无法精确表达所需的更改。一种直观的替代方法是利用野生图像样本的指导来帮助用户将他们想象的编辑变为现实。当代基于样本的编辑方法回避利用预先存在的大型文本到图像 (TTI) 模型学习到的丰富潜在空间，而是依靠使用精心策划的目标函数进行训练来完成任务。虽然这有点有效，但它需要大量的计算资源，并且与各种基础模型和任意样本计数缺乏兼容性。在进一步调查中，我们还发现这些技术将用户控制限制为仅对整个编辑区域应用统一的全局更改。在本文中，我们介绍了一种使用现成的扩散模型进行渐进式样本驱动编辑的新框架，称为 PIXELS，通过提供对编辑的细粒度控制来实现自定义，允许在像素或区域级别进行调整。我们的方法仅在推理过程中运行，以促进模仿编辑，使用户能够从动态数量的参考图像或多模式提示中汲取灵感，并逐步整合所有所需的更改，而无需重新训练或微调现有的 TTI 模型。这种细粒度控制能力开辟了一系列新的可能性，包括选择性修改单个对象和指定渐进的空间变化。我们证明 PIXELS 可以高效地提供高质量的编辑，从而显着改善定量指标和人工评估。通过使高质量图像编辑更容易获得，PIXELS 有可能让更广泛的受众轻松使用任何开源图像生成模型进行专业级编辑。

Title: ASTRA: A Scene-aware TRAnsformer-based model for trajectory prediction

• Authors: Izzeddin Teeti, Aniket Thomas, Munish Monga, Sachin Kumar, Uddeshya Singh, Andrew Bradley, Biplab Banerjee, Fabio Cuzzolin
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.09878
• Pdf URL: https://arxiv.org/pdf/2501.09878
• Copy Paste: [[2501.09878]] ASTRA: A Scene-aware TRAnsformer-based model for trajectory prediction(https://arxiv.org/abs/2501.09878)
• Keywords: generative
• Abstract: We present ASTRA (A} Scene-aware TRAnsformer-based model for trajectory prediction), a light-weight pedestrian trajectory forecasting model that integrates the scene context, spatial dynamics, social inter-agent interactions and temporal progressions for precise forecasting. We utilised a U-Net-based feature extractor, via its latent vector representation, to capture scene representations and a graph-aware transformer encoder for capturing social interactions. These components are integrated to learn an agent-scene aware embedding, enabling the model to learn spatial dynamics and forecast the future trajectory of pedestrians. The model is designed to produce both deterministic and stochastic outcomes, with the stochastic predictions being generated by incorporating a Conditional Variational Auto-Encoder (CVAE). ASTRA also proposes a simple yet effective weighted penalty loss function, which helps to yield predictions that outperform a wide array of state-of-the-art deterministic and generative models. ASTRA demonstrates an average improvement of 27%/10% in deterministic/stochastic settings on the ETH-UCY dataset, and 26% improvement on the PIE dataset, respectively, along with seven times fewer parameters than the existing state-of-the-art model (see Figure 1). Additionally, the model's versatility allows it to generalize across different perspectives, such as Bird's Eye View (BEV) and Ego-Vehicle View (EVV).
• 摘要：我们提出了 ASTRA（一种基于场景感知 TRANSformer 的轨迹预测模型），这是一种轻量级行人轨迹预测模型，它集成了场景背景、空间动态、社交代理间交互和时间进展，以实现精确预测。我们利用基于 U-Net 的特征提取器（通过其潜在向量表示）来捕获场景表示，并利用图形感知 Transformer 编码器来捕获社交互动。这些组件集成在一起以学习代理场景感知嵌入，使模型能够学习空间动态并预测行人的未来轨迹。该模型旨在产生确定性和随机性结果，其中随机预测是通过结合条件变分自动编码器 (CVAE) 生成的。ASTRA 还提出了一种简单但有效的加权惩罚损失函数，这有助于产生优于各种最先进的确定性和生成性模型的预测。 ASTRA 在 ETH-UCY 数据集的确定性/随机设置中分别实现了 27%/10% 的平均改进，在 PIE 数据集上实现了 26% 的改进，同时参数数量比现有的最先进模型少了七倍（见图 1）。此外，该模型的多功能性使其能够跨不同视角进行推广，例如鸟瞰图 (BEV) 和自车视图 (EVV)。

Title: IE-Bench: Advancing the Measurement of Text-Driven Image Editing for Human Perception Alignment

• Authors: Shangkun Sun, Bowen Qu, Xiaoyu Liang, Songlin Fan, Wei Gao
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.09927
• Pdf URL: https://arxiv.org/pdf/2501.09927
• Copy Paste: [[2501.09927]] IE-Bench: Advancing the Measurement of Text-Driven Image Editing for Human Perception Alignment(https://arxiv.org/abs/2501.09927)
• Keywords: generation, quality assessment
• Abstract: Recent advances in text-driven image editing have been significant, yet the task of accurately evaluating these edited images continues to pose a considerable challenge. Different from the assessment of text-driven image generation, text-driven image editing is characterized by simultaneously conditioning on both text and a source image. The edited images often retain an intrinsic connection to the original image, which dynamically change with the semantics of the text. However, previous methods tend to solely focus on text-image alignment or have not aligned with human perception. In this work, we introduce the Text-driven Image Editing Benchmark suite (IE-Bench) to enhance the assessment of text-driven edited images. IE-Bench includes a database contains diverse source images, various editing prompts and the corresponding results different editing methods, and total 3,010 Mean Opinion Scores (MOS) provided by 25 human subjects. Furthermore, we introduce IE-QA, a multi-modality source-aware quality assessment method for text-driven image editing. To the best of our knowledge, IE-Bench offers the first IQA dataset and model tailored for text-driven image editing. Extensive experiments demonstrate IE-QA's superior subjective-alignments on the text-driven image editing task compared with previous metrics. We will make all related data and code available to the public.
• 摘要：文本驱动的图像编辑最近取得了重大进展，但准确评估这些编辑后的图像仍然是一项艰巨的挑战。与文本驱动的图像生成的评估不同，文本驱动的图像编辑的特点是同时对文本和源图像进行条件处理。编辑后的图像通常保留与原始图像的内在联系，这些联系会随着文本的语义而动态变化。然而，以前的方法往往只关注文本-图像对齐，或者与人类感知不一致。在这项工作中，我们引入了文本驱动的图像编辑基准套件 (IE-Bench) 来增强对文本驱动的编辑图像的评估。IE-Bench 包括一个数据库，其中包含各种源图像、各种编辑提示和不同编辑方法的相应结果，以及 25 名人类受试者提供的总共 3,010 个平均意见分数 (MOS)。此外，我们引入了 IE-QA，这是一种用于文本驱动图像编辑的多模态源感知质量评估方法。据我们所知，IE-Bench 提供了第一个针对文本驱动的图像编辑量身定制的 IQA 数据集和模型。大量实验表明，与之前的指标相比，IE-QA 在文本驱动的图像编辑任务上具有更出色的主观对齐效果。我们将向公众开放所有相关数据和代码。

Title: Discrete Prior-based Temporal-coherent Content Prediction for Blind Face Video Restoration

• Authors: Lianxin Xie, Bingbing Zheng, Wen Xue, Yunfei Zhang, Le Jiang, Ruotao Xu, Si Wu, Hau-San Wong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.09960
• Pdf URL: https://arxiv.org/pdf/2501.09960
• Copy Paste: [[2501.09960]] Discrete Prior-based Temporal-coherent Content Prediction for Blind Face Video Restoration(https://arxiv.org/abs/2501.09960)
• Keywords: restoration
• Abstract: Blind face video restoration aims to restore high-fidelity details from videos subjected to complex and unknown degradations. This task poses a significant challenge of managing temporal heterogeneity while at the same time maintaining stable face attributes. In this paper, we introduce a Discrete Prior-based Temporal-Coherent content prediction transformer to address the challenge, and our model is referred to as DP-TempCoh. Specifically, we incorporate a spatial-temporal-aware content prediction module to synthesize high-quality content from discrete visual priors, conditioned on degraded video tokens. To further enhance the temporal coherence of the predicted content, a motion statistics modulation module is designed to adjust the content, based on discrete motion priors in terms of cross-frame mean and variance. As a result, the statistics of the predicted content can match with that of real videos over time. By performing extensive experiments, we verify the effectiveness of the design elements and demonstrate the superior performance of our DP-TempCoh in both synthetically and naturally degraded video restoration.
• 摘要：盲脸视频修复旨在从受到复杂和未知退化影响的视频中恢复高保真细节。这项任务带来了重大挑战，即在管理时间异质性的同时保持稳定的面部属性。在本文中，我们引入了一种基于离散先验的时间相干内容预测变换器来应对这一挑战，我们的模型称为 DP-TempCoh。具体来说，我们结合了时空感知内容预测模块，以退化视频标记为条件，从离散视觉先验合成高质量内容。为了进一步增强预测内容的时间连贯性，设计了一个运动统计调制模块，以根据跨帧均值和方差的离散运动先验来调整内容。因此，预测内容的统计数据可以与真实视频的统计数据随时间的变化相匹配。通过进行大量实验，我们验证了设计元素的有效性，并展示了我们的 DP-TempCoh 在合成和自然退化视频修复中的卓越性能。

Title: RichSpace: Enriching Text-to-Video Prompt Space via Text Embedding Interpolation

• Authors: Yuefan Cao, Chengyue Gong, Xiaoyu Li, Yingyu Liang, Zhizhou Sha, Zhenmei Shi, Zhao Song
• Subjects: cs.CV, cs.AI, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.09982
• Pdf URL: https://arxiv.org/pdf/2501.09982
• Copy Paste: [[2501.09982]] RichSpace: Enriching Text-to-Video Prompt Space via Text Embedding Interpolation(https://arxiv.org/abs/2501.09982)
• Keywords: generation
• Abstract: Text-to-video generation models have made impressive progress, but they still struggle with generating videos with complex features. This limitation often arises from the inability of the text encoder to produce accurate embeddings, which hinders the video generation model. In this work, we propose a novel approach to overcome this challenge by selecting the optimal text embedding through interpolation in the embedding space. We demonstrate that this method enables the video generation model to produce the desired videos. Additionally, we introduce a simple algorithm using perpendicular foot embeddings and cosine similarity to identify the optimal interpolation embedding. Our findings highlight the importance of accurate text embeddings and offer a pathway for improving text-to-video generation performance.
• 摘要：文本到视频生成模型取得了令人瞩目的进展，但它们在生成具有复杂特征的视频方面仍然举步维艰。这种限制通常源于文本编码器无法生成准确的嵌入，从而阻碍了视频生成模型。在这项工作中，我们提出了一种新方法来克服这一挑战，即通过在嵌入空间中进行插值来选择最佳文本嵌入。我们证明这种方法使视频生成模型能够生成所需的视频。此外，我们引入了一种使用垂直脚嵌入和余弦相似度来确定最佳插值嵌入的简单算法。我们的研究结果强调了准确文本嵌入的重要性，并为提高文本到视频生成性能提供了一条途径。

Title: Deep Learning for Early Alzheimer Disease Detection with MRI Scans

• Authors: Mohammad Rafsan, Tamer Oraby, Upal Roy, Sanjeev Kumar, Hansapani Rodrigo
• Subjects: cs.CV, cs.AI, eess.IV
• Abstract URL: https://arxiv.org/abs/2501.09999
• Pdf URL: https://arxiv.org/pdf/2501.09999
• Copy Paste: [[2501.09999]] Deep Learning for Early Alzheimer Disease Detection with MRI Scans(https://arxiv.org/abs/2501.09999)
• Keywords: generative
• Abstract: Alzheimer's Disease is a neurodegenerative condition characterized by dementia and impairment in neurological function. The study primarily focuses on the individuals above age 40, affecting their memory, behavior, and cognitive processes of the brain. Alzheimer's disease requires diagnosis by a detailed assessment of MRI scans and neuropsychological tests of the patients. This project compares existing deep learning models in the pursuit of enhancing the accuracy and efficiency of AD diagnosis, specifically focusing on the Convolutional Neural Network, Bayesian Convolutional Neural Network, and the U-net model with the Open Access Series of Imaging Studies brain MRI dataset. Besides, to ensure robustness and reliability in the model evaluations, we address the challenge of imbalance in data. We then perform rigorous evaluation to determine strengths and weaknesses for each model by considering sensitivity, specificity, and computational efficiency. This comparative analysis would shed light on the future role of AI in revolutionizing AD diagnostics but also paved ways for future innovation in medical imaging and the management of neurodegenerative diseases.
• 摘要：阿尔茨海默病是一种神经退行性疾病，其特征是痴呆和神经功能受损。该研究主要关注 40 岁以上的个体，影响他们的记忆、行为和大脑的认知过程。阿尔茨海默病需要通过对患者的 MRI 扫描和神经心理学测试进行详细评估来诊断。该项目比较了现有的深度学习模型，以提高 AD 诊断的准确性和效率，特别关注卷积神经网络、贝叶斯卷积神经网络和 U-net 模型与开放获取系列成像研究脑 MRI 数据集。此外，为了确保模型评估的稳健性和可靠性，我们解决了数据不平衡的挑战。然后，我们通过考虑敏感性、特异性和计算效率进行严格的评估，以确定每个模型的优缺点。这种比较分析将揭示人工智能在彻底改变 AD 诊断方面未来的作用，同时也为未来医学成像和神经退行性疾病管理的创新铺平了道路。

Title: Mitigating Hallucinations on Object Attributes using Multiview Images and Negative Instructions

• Authors: Zhijie Tan, Yuzhi Li, Shengwei Meng, Xiang Yuan, Weiping Li, Tong Mo, Bingce Wang, Xu Chu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.10011
• Pdf URL: https://arxiv.org/pdf/2501.10011
• Copy Paste: [[2501.10011]] Mitigating Hallucinations on Object Attributes using Multiview Images and Negative Instructions(https://arxiv.org/abs/2501.10011)
• Keywords: generation
• Abstract: Current popular Large Vision-Language Models (LVLMs) are suffering from Hallucinations on Object Attributes (HoOA), leading to incorrect determination of fine-grained attributes in the input images. Leveraging significant advancements in 3D generation from a single image, this paper proposes a novel method to mitigate HoOA in LVLMs. This method utilizes multiview images sampled from generated 3D representations as visual prompts for LVLMs, thereby providing more visual information from other viewpoints. Furthermore, we observe the input order of multiple multiview images significantly affects the performance of LVLMs. Consequently, we have devised Multiview Image Augmented VLM (MIAVLM), incorporating a Multiview Attributes Perceiver (MAP) submodule capable of simultaneously eliminating the influence of input image order and aligning visual information from multiview images with Large Language Models (LLMs). Besides, we designed and employed negative instructions to mitigate LVLMs' bias towards ``Yes" responses. Comprehensive experiments demonstrate the effectiveness of our method.
• 摘要：当前流行的大型视觉语言模型 (LVLM) 受到对象属性幻觉 (HoOA) 的影响，导致对输入图像中的细粒度属性的判断不正确。本文利用单幅图像 3D 生成的重大进步，提出了一种缓解 LVLM 中 HoOA 的新方法。该方法利用从生成的 3D 表示中采样的多视图图像作为 LVLM 的视觉提示，从而从其他视角提供更多视觉信息。此外，我们观察到多个多视图图像的输入顺序显著影响 LVLM 的性能。因此，我们设计了多视图图像增强 VLM (MIAVLM)，结合了多视图属性感知器 (MAP) 子模块，能够同时消除输入图像顺序的影响并将来自多视图图像的视觉信息与大型语言模型 (LLM) 对齐。此外，我们设计并使用了负面指令来缓解 LVLM 对“是”回答的偏见。全面的实验证明了我们方法的有效性。

Title: DiffuEraser: A Diffusion Model for Video Inpainting

• Authors: Xiaowen Li, Haolan Xue, Peiran Ren, Liefeng Bo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10018
• Pdf URL: https://arxiv.org/pdf/2501.10018
• Copy Paste: [[2501.10018]] DiffuEraser: A Diffusion Model for Video Inpainting(https://arxiv.org/abs/2501.10018)
• Keywords: generation, generative
• Abstract: Recent video inpainting algorithms integrate flow-based pixel propagation with transformer-based generation to leverage optical flow for restoring textures and objects using information from neighboring frames, while completing masked regions through visual Transformers. However, these approaches often encounter blurring and temporal inconsistencies when dealing with large masks, highlighting the need for models with enhanced generative capabilities. Recently, diffusion models have emerged as a prominent technique in image and video generation due to their impressive performance. In this paper, we introduce DiffuEraser, a video inpainting model based on stable diffusion, designed to fill masked regions with greater details and more coherent structures. We incorporate prior information to provide initialization and weak conditioning,which helps mitigate noisy artifacts and suppress hallucinations. Additionally, to improve temporal consistency during long-sequence inference, we expand the temporal receptive fields of both the prior model and DiffuEraser, and further enhance consistency by leveraging the temporal smoothing property of Video Diffusion Models. Experimental results demonstrate that our proposed method outperforms state-of-the-art techniques in both content completeness and temporal consistency while maintaining acceptable efficiency.
• 摘要：最近的视频修复算法将基于流的像素传播与基于变换器的生成相结合，利用光流使用来自相邻帧的信息恢复纹理和对象，同时通过视觉变换器完成蒙版区域。然而，这些方法在处理大型蒙版时经常会遇到模糊和时间不一致的问题，这凸显了对具有增强生成能力的模型的需求。最近，扩散模型因其出色的性能而成为图像和视频生成中的一种重要技术。在本文中，我们介绍了基于稳定扩散的视频修复模型 DiffuEraser，旨在用更多细节和更连贯的结构填充蒙版区域。我们结合先验信息来提供初始化和弱条件，这有助于减轻噪声伪影并抑制幻觉。此外，为了提高长序列推理过程中的时间一致性，我们扩展了先验模型和 DiffuEraser 的时间感受野，并通过利用视频扩散模型的时间平滑特性进一步增强一致性。实验结果表明，我们提出的方法在内容完整性和时间一致性方面都优于最先进的技术，同时保持了可接受的效率。

Title: CLIP-PCQA: Exploring Subjective-Aligned Vision-Language Modeling for Point Cloud Quality Assessment

• Authors: Yating Liu, Yujie Zhang, Ziyu Shan, Yiling Xu
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2501.10071
• Pdf URL: https://arxiv.org/pdf/2501.10071
• Copy Paste: [[2501.10071]] CLIP-PCQA: Exploring Subjective-Aligned Vision-Language Modeling for Point Cloud Quality Assessment(https://arxiv.org/abs/2501.10071)
• Keywords: quality assessment
• Abstract: In recent years, No-Reference Point Cloud Quality Assessment (NR-PCQA) research has achieved significant progress. However, existing methods mostly seek a direct mapping function from visual data to the Mean Opinion Score (MOS), which is contradictory to the mechanism of practical subjective evaluation. To address this, we propose a novel language-driven PCQA method named CLIP-PCQA. Considering that human beings prefer to describe visual quality using discrete quality descriptions (e.g., "excellent" and "poor") rather than specific scores, we adopt a retrieval-based mapping strategy to simulate the process of subjective assessment. More specifically, based on the philosophy of CLIP, we calculate the cosine similarity between the visual features and multiple textual features corresponding to different quality descriptions, in which process an effective contrastive loss and learnable prompts are introduced to enhance the feature extraction. Meanwhile, given the personal limitations and bias in subjective experiments, we further covert the feature similarities into probabilities and consider the Opinion Score Distribution (OSD) rather than a single MOS as the final target. Experimental results show that our CLIP-PCQA outperforms other State-Of-The-Art (SOTA) approaches.
• 摘要：近年来，无参考点云质量评估（NR-PCQA）研究取得了重大进展。然而，现有方法大多寻求从视觉数据到平均意见分数（MOS）的直接映射函数，这与实际主观评价机制相矛盾。为了解决这个问题，我们提出了一种新的语言驱动的PCQA方法CLIP-PCQA。考虑到人类更喜欢使用离散的质量描述（例如“优秀”和“差”）而不是特定的分数来描述视觉质量，我们采用基于检索的映射策略来模拟主观评估的过程。更具体地说，基于CLIP的理念，我们计算视觉特征与对应于不同质量描述的多个文本特征之间的余弦相似度，在此过程中引入有效的对比损失和可学习的提示来增强特征提取。同时，考虑到主观实验中的个人局限性和偏见，我们进一步将特征相似度转化为概率，并将意见分数分布（OSD）而不是单个MOS视为最终目标。实验结果表明，我们的 CLIP-PCQA 优于其他最先进的 (SOTA) 方法。

Title: DiffVSR: Enhancing Real-World Video Super-Resolution with Diffusion Models for Advanced Visual Quality and Temporal Consistency

• Authors: Xiaohui Li, Yihao Liu, Shuo Cao, Ziyan Chen, Shaobin Zhuang, Xiangyu Chen, Yinan He, Yi Wang, Yu Qiao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10110
• Pdf URL: https://arxiv.org/pdf/2501.10110
• Copy Paste: [[2501.10110]] DiffVSR: Enhancing Real-World Video Super-Resolution with Diffusion Models for Advanced Visual Quality and Temporal Consistency(https://arxiv.org/abs/2501.10110)
• Keywords: restoration, super-resolution, generation
• Abstract: Diffusion models have demonstrated exceptional capabilities in image generation and restoration, yet their application to video super-resolution faces significant challenges in maintaining both high fidelity and temporal consistency. We present DiffVSR, a diffusion-based framework for real-world video super-resolution that effectively addresses these challenges through key innovations. For intra-sequence coherence, we develop a multi-scale temporal attention module and temporal-enhanced VAE decoder that capture fine-grained motion details. To ensure inter-sequence stability, we introduce a noise rescheduling mechanism with an interweaved latent transition approach, which enhances temporal consistency without additional training overhead. We propose a progressive learning strategy that transitions from simple to complex degradations, enabling robust optimization despite limited high-quality video data. Extensive experiments demonstrate that DiffVSR delivers superior results in both visual quality and temporal consistency, setting a new performance standard in real-world video super-resolution.
• 摘要：扩散模型在图像生成和恢复方面表现出了卓越的能力，但它们在视频超分辨率中的应用面临着保持高保真度和时间一致性的重大挑战。我们提出了 DiffVSR，这是一个基于扩散的真实世界视频超分辨率框架，它通过关键创新有效地解决了这些挑战。对于序列内连贯性，我们开发了一个多尺度时间注意模块和时间增强 VAE 解码器，可以捕获细粒度的运动细节。为了确保序列间稳定性，我们引入了一种噪声重新调度机制和交织潜在转换方法，这可以在不增加训练开销的情况下增强时间一致性。我们提出了一种渐进式学习策略，从简单到复杂的退化过渡，尽管高质量视频数据有限，但仍能实现稳健的优化。大量实验表明，DiffVSR 在视觉质量和时间一致性方面均提供了卓越的结果，为真实世界的视频超分辨率树立了新的性能标准。

Title: A Vision-Language Framework for Multispectral Scene Representation Using Language-Grounded Features

• Authors: Enes Karanfil, Nevrez Imamoglu, Erkut Erdem, Aykut Erdem
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10144
• Pdf URL: https://arxiv.org/pdf/2501.10144
• Copy Paste: [[2501.10144]] A Vision-Language Framework for Multispectral Scene Representation Using Language-Grounded Features(https://arxiv.org/abs/2501.10144)
• Keywords: generation
• Abstract: Scene understanding in remote sensing often faces challenges in generating accurate representations for complex environments such as various land use areas or coastal regions, which may also include snow, clouds, or haze. To address this, we present a vision-language framework named Spectral LLaVA, which integrates multispectral data with vision-language alignment techniques to enhance scene representation and description. Using the BigEarthNet v2 dataset from Sentinel-2, we establish a baseline with RGB-based scene descriptions and further demonstrate substantial improvements through the incorporation of multispectral information. Our framework optimizes a lightweight linear projection layer for alignment while keeping the vision backbone of SpectralGPT frozen. Our experiments encompass scene classification using linear probing and language modeling for jointly performing scene classification and description generation. Our results highlight Spectral LLaVA's ability to produce detailed and accurate descriptions, particularly for scenarios where RGB data alone proves inadequate, while also enhancing classification performance by refining SpectralGPT features into semantically meaningful representations.
• 摘要：遥感中的场景理解通常面临生成复杂环境（例如各种土地使用区域或沿海地区，其中可能还包括雪、云或雾霾）的准确表示的挑战。为了解决这个问题，我们提出了一个名为 Spectral LLaVA 的视觉语言框架，它将多光谱数据与视觉语言对齐技术相结合，以增强场景表示和描述。使用来自 Sentinel-2 的 BigEarthNet v2 数据集，我们建立了基于 RGB 的场景描述基线，并通过结合多光谱信息进一步展示了实质性的改进。我们的框架优化了轻量级线性投影层以进行对齐，同时保持 SpectralGPT 的视觉主干冻结。我们的实验包括使用线性探测和语言建模进行场景分类，以联合执行场景分类和描述生成。我们的结果突出了 Spectral LLaVA 生成详细而准确描述的能力，特别是在仅使用 RGB 数据不足以满足需要的场景中，同时还通过将 SpectralGPT 特征细化为语义上有意义的表示来提高分类性能。

Title: Region-wise stacking ensembles for estimating brain-age using MRI

• Authors: Georgios Antonopoulos, Shammi More, Simon B. Eickhoff, Federico Raimondo, Kaustubh R. Patil
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.10153
• Pdf URL: https://arxiv.org/pdf/2501.10153
• Copy Paste: [[2501.10153]] Region-wise stacking ensembles for estimating brain-age using MRI(https://arxiv.org/abs/2501.10153)
• Keywords: generative
• Abstract: Predictive modeling using structural magnetic resonance imaging (MRI) data is a prominent approach to study brain-aging. Machine learning algorithms and feature extraction methods have been employed to improve predictions and explore healthy and accelerated aging e.g. neurodegenerative and psychiatric disorders. The high-dimensional MRI data pose challenges to building generalizable and interpretable models as well as for data privacy. Common practices are resampling or averaging voxels within predefined parcels, which reduces anatomical specificity and biological interpretability as voxels within a region may differently relate to aging. Effectively, naive fusion by averaging can result in information loss and reduced accuracy. We present a conceptually novel two-level stacking ensemble (SE) approach. The first level comprises regional models for predicting individuals' age based on voxel-wise information, fused by a second-level model yielding final predictions. Eight data fusion scenarios were explored using as input Gray matter volume (GMV) estimates from four datasets covering the adult lifespan. Performance, measured using mean absolute error (MAE), R2, correlation and prediction bias, showed that SE outperformed the region-wise averages. The best performance was obtained when first-level regional predictions were obtained as out-of-sample predictions on the application site with second-level models trained on independent and site-specific data (MAE=4.75 vs baseline regional mean GMV MAE=5.68). Performance improved as more datasets were used for training. First-level predictions showed improved and more robust aging signal providing new biological insights and enhanced data privacy. Overall, the SE improves accuracy compared to the baseline while preserving or enhancing data privacy.
• 摘要：使用结构磁共振成像 (MRI) 数据进行预测建模是研究大脑衰老的一种重要方法。机器学习算法和特征提取方法已被用于改进预测并探索健康和加速衰老，例如神经退行性疾病和精神疾病。高维 MRI 数据对构建可推广和可解释的模型以及数据隐私提出了挑战。常见的做法是在预定义的包裹内重新采样或平均体素，这会降低解剖特异性和生物学可解释性，因为区域内的体素可能与衰老有不同的关系。实际上，通过平均进行的简单融合可能会导致信息丢失和准确性降低。我们提出了一种概念上新颖的两级堆叠集成 (SE) 方法。第一级包括基于体素信息预测个体年龄的区域模型，由第二级模型融合以得出最终预测。使用来自四个涵盖成年人寿命的数据集的灰质体积 (GMV) 估计值作为输入，探索了八种数据融合场景。使用平均绝对误差 (MAE)、R2、相关性和预测偏差来衡量性能，结果表明 SE 的表现优于区域平均值。当第一级区域预测作为应用站点的样本外预测获得，而第二级模型在独立和特定站点的数据上进行训练时，可获得最佳性能（MAE=4.75 vs 基线区域平均 GMV MAE=5.68）。随着更多数据集用于训练，性能得到改善。第一级预测显示出改进和更强大的衰老信号，从而提供了新的生物学见解并增强了数据隐私。总体而言，SE 与基线相比提高了准确性，同时保留或增强了数据隐私。

Title: Hypercone Assisted Contour Generation for Out-of-Distribution Detection

• Authors: Annita Vapsi, Andrés Muñoz, Nancy Thomas, Keshav Ramani, Daniel Borrajo
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.10209
• Pdf URL: https://arxiv.org/pdf/2501.10209
• Copy Paste: [[2501.10209]] Hypercone Assisted Contour Generation for Out-of-Distribution Detection(https://arxiv.org/abs/2501.10209)
• Keywords: generation
• Abstract: Recent advances in the field of out-of-distribution (OOD) detection have placed great emphasis on learning better representations suited to this task. While there are distance-based approaches, distributional awareness has seldom been exploited for better performance. We present HAC$_k$-OOD, a novel OOD detection method that makes no distributional assumption about the data, but automatically adapts to its distribution. Specifically, HAC$_k$-OOD constructs a set of hypercones by maximizing the angular distance to neighbors in a given data-point's vicinity to approximate the contour within which in-distribution (ID) data-points lie. Experimental results show state-of-the-art FPR@95 and AUROC performance on Near-OOD detection and on Far-OOD detection on the challenging CIFAR-100 benchmark without explicitly training for OOD performance.
• 摘要：分布外 (OOD) 检测领域的最新进展非常重视学习更适合此任务的表示。虽然存在基于距离的方法，但很少利用分布意识来提高性能。我们提出了 HAC$_k$-OOD，这是一种新颖的 OOD 检测方法，它不对数据做任何分布假设，但会自动适应其分布。具体而言，HAC$_k$-OOD 通过最大化给定数据点附近邻居的角距离来构造一组超锥，以近似分布内 (ID) 数据点所在的轮廓。实验结果表明，在具有挑战性的 CIFAR-100 基准上，在近 OOD 检测和远 OOD 检测中，无需明确训练 OOD 性能，即可实现最先进的 FPR@95 和 AUROC 性能。

Title: Disharmony: Forensics using Reverse Lighting Harmonization

• Authors: Philip Wootaek Shin, Jack Sampson, Vijaykrishnan Narayanan, Andres Marquez, Mahantesh Halappanavar
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10212
• Pdf URL: https://arxiv.org/pdf/2501.10212
• Copy Paste: [[2501.10212]] Disharmony: Forensics using Reverse Lighting Harmonization(https://arxiv.org/abs/2501.10212)
• Keywords: generation
• Abstract: Content generation and manipulation approaches based on deep learning methods have seen significant advancements, leading to an increased need for techniques to detect whether an image has been generated or edited. Another area of research focuses on the insertion and harmonization of objects within images. In this study, we explore the potential of using harmonization data in conjunction with a segmentation model to enhance the detection of edited image regions. These edits can be either manually crafted or generated using deep learning methods. Our findings demonstrate that this approach can effectively identify such edits. Existing forensic models often overlook the detection of harmonized objects in relation to the background, but our proposed Disharmony Network addresses this gap. By utilizing an aggregated dataset of harmonization techniques, our model outperforms existing forensic networks in identifying harmonized objects integrated into their backgrounds, and shows potential for detecting various forms of edits, including virtual try-on tasks.
• 摘要：基于深度学习方法的内容生成和操作方法取得了重大进展，导致对检测图像是否已生成或编辑的技术的需求增加。另一个研究领域侧重于在图像中插入和协调对象。在本研究中，我们探索了将协调数据与分割模型结合使用以增强对已编辑图像区域的检测的潜力。这些编辑可以是手工制作的，也可以是使用深度学习方法生成的。我们的研究结果表明，这种方法可以有效识别此类编辑。现有的取证模型通常会忽略与背景相关的协调对象的检测，但我们提出的不和谐网络解决了这一问题。通过利用协调技术的聚合数据集，我们的模型在识别集成到背景中的协调对象方面优于现有的取证网络，并显示出检测各种形式的编辑（包括虚拟试穿任务）的潜力。

Title: Modelling Activity Scheduling Behaviour with Deep Generative Machine Learning

• Authors: Fred Shone, Tim Hillel
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.10221
• Pdf URL: https://arxiv.org/pdf/2501.10221
• Copy Paste: [[2501.10221]] Modelling Activity Scheduling Behaviour with Deep Generative Machine Learning(https://arxiv.org/abs/2501.10221)
• Keywords: generative
• Abstract: We model human activity scheduling behaviour using a deep generative machine learning approach. Activity schedules, which represent the activities and associated travel behaviours of individuals, are a core component of many applied models in the transport, energy and epidemiology domains. Our data driven approach learns human preferences and scheduling logic without the need for complex interacting combinations of sub-models and custom-rules, this makes our approach significantly faster and simpler to operate that existing approaches. We find activity schedule data combines aspects of both continuous image data and also discrete text data, requiring novel approaches. We additionally contribute a novel schedule representation and comprehensive evaluation framework for generated schedules. Evaluation shows our approach is able to rapidly generate large, diverse and realistic synthetic samples of activity schedules.
• 摘要：我们使用深度生成机器学习方法对人类活动安排行为进行建模。活动时间表代表个人的活动和相关旅行行为，是交通、能源和流行病学领域许多应用模型的核心组成部分。我们的数据驱动方法可以学习人类的偏好和调度逻辑，而无需复杂的子模型和自定义规则交互组合，这使得我们的方法比现有方法更快、更易于操作。我们发现活动时间表数据结合了连续图像数据和离散文本数据的方面，需要新颖的方法。我们还为生成的时间表提供了一种新颖的时间表表示和全面的评估框架。评估表明，我们的方法能够快速生成大量、多样化和逼真的活动时间表合成样本。

Title: Counterfactual Explanations for k-means and Gaussian Clustering

• Authors: Georgios Vardakas, Antonia Karra, Evaggelia Pitoura, Aristidis Likas
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.10234
• Pdf URL: https://arxiv.org/pdf/2501.10234
• Copy Paste: [[2501.10234]] Counterfactual Explanations for k-means and Gaussian Clustering(https://arxiv.org/abs/2501.10234)
• Keywords: generation
• Abstract: Counterfactuals have been recognized as an effective approach to explain classifier decisions. Nevertheless, they have not yet been considered in the context of clustering. In this work, we propose the use of counterfactuals to explain clustering solutions. First, we present a general definition for counterfactuals for model-based clustering that includes plausibility and feasibility constraints. Then we consider the counterfactual generation problem for k-means and Gaussian clustering assuming Euclidean distance. Our approach takes as input the factual, the target cluster, a binary mask indicating actionable or immutable features and a plausibility factor specifying how far from the cluster boundary the counterfactual should be placed. In the k-means clustering case, analytical mathematical formulas are presented for computing the optimal solution, while in the Gaussian clustering case (assuming full, diagonal, or spherical covariances) our method requires the numerical solution of a nonlinear equation with a single parameter only. We demonstrate the advantages of our approach through illustrative examples and quantitative experimental comparisons.
• 摘要：反事实已被公认为解释分类器决策的有效方法。然而，它们尚未在聚类的背景下被考虑。在这项工作中，我们建议使用反事实来解释聚类解决方案。首先，我们为基于模型的聚类提出了反事实的一般定义，其中包括合理性和可行性约束。然后，我们考虑假设欧几里得距离的 k 均值和高斯聚类的反事实生成问题。我们的方法将事实、目标聚类、指示可操作或不可变特征的二进制掩码以及指定反事实应放置在离聚类边界多远的合理性因子作为输入。在 k 均值聚类情况下，给出了用于计算最优解的分析数学公式，而在高斯聚类情况下（假设全、对角或球面协方差），我们的方法只需要对具有单个参数的非线性方程进行数值解。我们通过说明性示例和定量实验比较展示了我们方法的优势。

Title: GSTAR: Gaussian Surface Tracking and Reconstruction

• Authors: Chengwei Zheng, Lixin Xue, Juan Zarate, Jie Song
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10283
• Pdf URL: https://arxiv.org/pdf/2501.10283
• Copy Paste: [[2501.10283]] GSTAR: Gaussian Surface Tracking and Reconstruction(https://arxiv.org/abs/2501.10283)
• Keywords: generation
• Abstract: 3D Gaussian Splatting techniques have enabled efficient photo-realistic rendering of static scenes. Recent works have extended these approaches to support surface reconstruction and tracking. However, tracking dynamic surfaces with 3D Gaussians remains challenging due to complex topology changes, such as surfaces appearing, disappearing, or splitting. To address these challenges, we propose GSTAR, a novel method that achieves photo-realistic rendering, accurate surface reconstruction, and reliable 3D tracking for general dynamic scenes with changing topology. Given multi-view captures as input, GSTAR binds Gaussians to mesh faces to represent dynamic objects. For surfaces with consistent topology, GSTAR maintains the mesh topology and tracks the meshes using Gaussians. In regions where topology changes, GSTAR adaptively unbinds Gaussians from the mesh, enabling accurate registration and the generation of new surfaces based on these optimized Gaussians. Additionally, we introduce a surface-based scene flow method that provides robust initialization for tracking between frames. Experiments demonstrate that our method effectively tracks and reconstructs dynamic surfaces, enabling a range of applications. Our project page with the code release is available at this https URL.
• 摘要：3D 高斯 Splatting 技术已实现静态场景的高效照片级逼真渲染。最近的研究扩展了这些方法以支持表面重建和跟踪。然而，由于复杂的拓扑变化（例如表面出现、消失或分裂），使用 3D 高斯跟踪动态表面仍然具有挑战性。为了应对这些挑战，我们提出了 GSTAR，这是一种新颖的方法，可实现照片级逼真渲染、准确的表面重建和可靠的 3D 跟踪，适用于拓扑不断变化的一般动态场景。给定多视图捕获作为输入，GSTAR 将高斯绑定到网格面以表示动态对象。对于具有一致拓扑的表面，GSTAR 保持网格拓扑并使用高斯跟踪网格。在拓扑发生变化的区域，GSTAR 自适应地将高斯从网格中解绑，从而实现精确配准并基于这些优化的高斯生成新表面。此外，我们引入了一种基于表面的场景流方法，为帧间跟踪提供强大的初始化。实验表明，我们的方法可以有效地跟踪和重建动态表面，从而实现一系列应用。我们的项目页面及代码发布可通过此 https URL 访问。

Title: DiffStereo: High-Frequency Aware Diffusion Model for Stereo Image Restoration

• Authors: Huiyun Cao, Yuan Shi, Bin Xia, Xiaoyu Jin, Wenming Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10325
• Pdf URL: https://arxiv.org/pdf/2501.10325
• Copy Paste: [[2501.10325]] DiffStereo: High-Frequency Aware Diffusion Model for Stereo Image Restoration(https://arxiv.org/abs/2501.10325)
• Keywords: restoration, super-resolution, generative
• Abstract: Diffusion models (DMs) have achieved promising performance in image restoration but haven't been explored for stereo images. The application of DM in stereo image restoration is confronted with a series of challenges. The need to reconstruct two images exacerbates DM's computational cost. Additionally, existing latent DMs usually focus on semantic information and remove high-frequency details as redundancy during latent compression, which is precisely what matters for image restoration. To address the above problems, we propose a high-frequency aware diffusion model, DiffStereo for stereo image restoration as the first attempt at DM in this domain. Specifically, DiffStereo first learns latent high-frequency representations (LHFR) of HQ images. DM is then trained in the learned space to estimate LHFR for stereo images, which are fused into a transformer-based stereo image restoration network providing beneficial high-frequency information of corresponding HQ images. The resolution of LHFR is kept the same as input images, which preserves the inherent texture from distortion. And the compression in channels alleviates the computational burden of DM. Furthermore, we devise a position encoding scheme when integrating the LHFR into the restoration network, enabling distinctive guidance in different depths of the restoration network. Comprehensive experiments verify that by combining generative DM and transformer, DiffStereo achieves both higher reconstruction accuracy and better perceptual quality on stereo super-resolution, deblurring, and low-light enhancement compared with state-of-the-art methods.
• 摘要：扩散模型 (DM) 在图像恢复方面取得了良好的效果，但尚未在立体图像中得到探索。DM 在立体图像恢复中的应用面临着一系列挑战。重建两幅图像的需求加剧了 DM 的计算成本。此外，现有的潜在 DM 通常关注语义信息，并在潜在压缩过程中将高频细节作为冗余删除，而这恰恰是图像恢复的关键。为了解决上述问题，我们提出了一种用于立体图像恢复的高频感知扩散模型 DiffStereo，这是该领域 DM 的首次尝试。具体而言，DiffStereo 首先学习 HQ 图像的潜在高频表示 (LHFR)。然后在学习空间中训练 DM 以估计立体图像的 LHFR，这些 LHFR 被融合到基于变换器的立体图像恢复网络中，提供相应 HQ 图像的有益高频信息。LHFR 的分辨率与输入图像保持相同，从而保留了固有纹理免受失真。并且通道中的压缩减轻了 DM 的计算负担。此外，我们在将 LHFR 集成到恢复网络时设计了一种位置编码方案，从而能够在恢复网络的不同深度中进行不同的引导。综合实验验证了通过结合生成式 DM 和 Transformer，与最先进的方法相比，DiffStereo 在立体超分辨率、去模糊和低光增强方面实现了更高的重建精度和更好的感知质量。

Title: Credit Risk Identification in Supply Chains Using Generative Adversarial Networks

• Authors: Zizhou Zhang, Xinshi Li, Yu Cheng, Zhenrui Chen, Qianying Liu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.10348
• Pdf URL: https://arxiv.org/pdf/2501.10348
• Copy Paste: [[2501.10348]] Credit Risk Identification in Supply Chains Using Generative Adversarial Networks(https://arxiv.org/abs/2501.10348)
• Keywords: generation, generative
• Abstract: Credit risk management within supply chains has emerged as a critical research area due to its significant implications for operational stability and financial sustainability. The intricate interdependencies among supply chain participants mean that credit risks can propagate across networks, with impacts varying by industry. This study explores the application of Generative Adversarial Networks (GANs) to enhance credit risk identification in supply chains. GANs enable the generation of synthetic credit risk scenarios, addressing challenges related to data scarcity and imbalanced datasets. By leveraging GAN-generated data, the model improves predictive accuracy while effectively capturing dynamic and temporal dependencies in supply chain data. The research focuses on three representative industries-manufacturing (steel), distribution (pharmaceuticals), and services (e-commerce) to assess industry-specific credit risk contagion. Experimental results demonstrate that the GAN-based model outperforms traditional methods, including logistic regression, decision trees, and neural networks, achieving superior accuracy, recall, and F1 scores. The findings underscore the potential of GANs in proactive risk management, offering robust tools for mitigating financial disruptions in supply chains. Future research could expand the model by incorporating external market factors and supplier relationships to further enhance predictive capabilities. Keywords- Generative Adversarial Networks (GANs); Supply Chain Risk; Credit Risk Identification; Machine Learning; Data Augmentation
• 摘要：供应链中的信用风险管理已成为一个重要的研究领域，因为它对运营稳定性和财务可持续性具有重要意义。供应链参与者之间错综复杂的相互依赖关系意味着信用风险可以跨网络传播，其影响因行业而异。本研究探讨了生成对抗网络 (GAN) 在增强供应链信用风险识别中的应用。GAN 能够生成合成信用风险场景，解决与数据稀缺和数据集不平衡相关的挑战。通过利用 GAN 生成的数据，该模型提高了预测准确性，同时有效地捕获了供应链数据中的动态和时间依赖关系。该研究重点关注三个代表性行业——制造业（钢铁）、分销业（制药）和服务业（电子商务），以评估特定行业的信用风险传染。实验结果表明，基于 GAN 的模型优于传统方法，包括逻辑回归、决策树和神经网络，实现了卓越的准确率、召回率和 F1 分数。研究结果强调了 GAN 在主动风险管理方面的潜力，为缓解供应链中的财务中断提供了强有力的工具。未来的研究可以通过纳入外部市场因素和供应商关系来扩展该模型，以进一步增强预测能力。关键词- 生成对抗网络 (GAN)；供应链风险；信用风险识别；机器学习；数据增强