2025-04-10

Title: Ternarization of Vision Language Models for use on edge devices

• Authors: Ben Crulis, Cyril De Runz, Barthelemy Serres, Gilles Venturini
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.06298
• Pdf URL: https://arxiv.org/pdf/2504.06298
• Copy Paste: [[2504.06298]] Ternarization of Vision Language Models for use on edge devices(https://arxiv.org/abs/2504.06298)
• Keywords: generation
• Abstract: We propose a process to compress a pre-trained Vision Language Model into a ternary version of itself instead of training a ternary model from scratch. A new initialization scheme from pre-trained weights based on the k-means algorithm is proposed to reduce the ternarization time. We implement different custom operators for executing the ternary model on the TensorFlow Lite Engine. We compare the original model with its ternary and binary versions in terms of memory consumption, inference speed and perplexity. We find that the ternary model using our custom ternary matrix multiplication operator provides a good compromise in term of memory usage and perplexity, while having the fastest token generation speed.
• 摘要：我们提出了一个过程，将预训练的视觉语言模型压缩为自身的三元版本，而不是从头开始训练三元模型。提出了一种基于K-均值算法的预训练的重量的新初始化方案，以减少三元化时间。我们实施了不同的自定义运算符，以在Tensorflow Lite引擎上执行三元模型。我们将原始模型与其三元版和二进制版本进行比较，从记忆消耗，推理速度和困惑方面。我们发现，使用自定义三元矩阵乘法运算符的三元模型在内存使用和困惑方面提供了良好的折衷，同时具有最快的代币生成速度。

Title: Well2Flow: Reconstruction of reservoir states from sparse wells using score-based generative models

• Authors: Shiqin Zeng, Haoyun Li, Abhinav Prakash Gahlot, Felix J. Herrmann
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.06305
• Pdf URL: https://arxiv.org/pdf/2504.06305
• Copy Paste: [[2504.06305]] Well2Flow: Reconstruction of reservoir states from sparse wells using score-based generative models(https://arxiv.org/abs/2504.06305)
• Keywords: generative
• Abstract: This study investigates the use of score-based generative models for reservoir simulation, with a focus on reconstructing spatially varying permeability and saturation fields in saline aquifers, inferred from sparse observations at two well locations. By modeling the joint distribution of permeability and saturation derived from high-fidelity reservoir simulations, the proposed neural network is trained to learn the complex spatiotemporal dynamics governing multiphase fluid flow in porous media. During inference, the framework effectively reconstructs both permeability and saturation fields by conditioning on sparse vertical profiles extracted from well log data. This approach introduces a novel methodology for incorporating physical constraints and well log guidance into generative models, significantly enhancing the accuracy and physical plausibility of the reconstructed subsurface states. Furthermore, the framework demonstrates strong generalization capabilities across varying geological scenarios, highlighting its potential for practical deployment in data-scarce reservoir management tasks.
• 摘要：这项研究研究了基于分数的生成模型用于储层模拟，重点是重建盐水含水层中空间变化的渗透性和饱和场，这是根据两个井位置的稀疏观测来推断的。通过对高保真储层模拟得出的渗透性和饱和度的联合分布进行建模，训练了拟议的神经网络，以了解多孔介质中多相流体流的复杂时空动力学。在推断期间，框架通过根据从井的日志数据中提取的稀疏垂直轮廓进行调节来有效地重建渗透性和饱和场。这种方法引入了一种新颖的方法，用于将物理约束和良好的日志指导纳入生成模型中，从而显着提高了重建的地下状态的准确性和物理合理性。此外，该框架在不同的地质场景中表现出强大的概括能力，强调了其在数据砂金储层管理任务中实际部署的潜力。

Title: Optimizing Large Language Models: Metrics, Energy Efficiency, and Case Study Insights

• Authors: Tahniat Khan, Soroor Motie, Sedef Akinli Kocak, Shaina Raza
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.06307
• Pdf URL: https://arxiv.org/pdf/2504.06307
• Copy Paste: [[2504.06307]] Optimizing Large Language Models: Metrics, Energy Efficiency, and Case Study Insights(https://arxiv.org/abs/2504.06307)
• Keywords: generative
• Abstract: The rapid adoption of large language models (LLMs) has led to significant energy consumption and carbon emissions, posing a critical challenge to the sustainability of generative AI technologies. This paper explores the integration of energy-efficient optimization techniques in the deployment of LLMs to address these environmental concerns. We present a case study and framework that demonstrate how strategic quantization and local inference techniques can substantially lower the carbon footprints of LLMs without compromising their operational effectiveness. Experimental results reveal that these methods can reduce energy consumption and carbon emissions by up to 45\% post quantization, making them particularly suitable for resource-constrained environments. The findings provide actionable insights for achieving sustainability in AI while maintaining high levels of accuracy and responsiveness.
• 摘要：大型语言模型（LLM）的迅速采用导致了大量的能耗和碳排放，对生成AI技术的可持续性构成了关键的挑战。本文探讨了在LLMS部署中的节能优化技术的整合，以解决这些环境问题。我们提出了一个案例研究和框架，该案例研究表明战略量化和局部推理技术如何在不损害其运营效果的情况下大大降低LLM的碳足迹。实验结果表明，这些方法可以在量化后最多将能耗和碳排放量减少45％，从而使其特别适合于资源受限的环境。这些发现为实现AI的可持续性提供了可行的见解，同时保持了高度的准确性和响应能力。

Title: DMol: A Schedule-Driven Diffusion Model for Highly Efficient and Versatile Molecule Generation

• Authors: Peizhi Niu, Yu-Hsiang Wang, Vishal Rana, Chetan Rupakheti, Abhishek Pandey, Olgica Milenkovic
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.06312
• Pdf URL: https://arxiv.org/pdf/2504.06312
• Copy Paste: [[2504.06312]] DMol: A Schedule-Driven Diffusion Model for Highly Efficient and Versatile Molecule Generation(https://arxiv.org/abs/2504.06312)
• Keywords: generation
• Abstract: We introduce a new graph diffusion model for small molecule generation, \emph{DMol}, which outperforms the state-of-the-art DiGress model in terms of validity by roughly $1.5\%$ across all benchmarking datasets while reducing the number of diffusion steps by at least $10$-fold, and the running time to roughly one half. The performance improvements are a result of a careful change in the objective function and a ``graph noise" scheduling approach which, at each diffusion step, allows one to only change a subset of nodes of varying size in the molecule graph. Another relevant property of the method is that it can be easily combined with junction-tree-like graph representations that arise by compressing a collection of relevant ring structures into supernodes. Unlike classical junction-tree techniques that involve VAEs and require complicated reconstruction steps, compressed DMol directly performs graph diffusion on a graph that compresses only a carefully selected set of frequent carbon rings into supernodes, which results in straightforward sample generation. This compressed DMol method offers additional validity improvements over generic DMol of roughly $2\%$, increases the novelty of the method, and further improves the running time due to reductions in the graph size.
• 摘要：我们引入了一个针对小分子生成的新图扩散模型，\ emph {dmol}，该模型在所有基准测试数据集中，在所有基准测试数据集中，以大约1.5 \％$的价格优于最先进的离题模型，同时将扩散步骤的数量减少至少10美元，而运行时间至少为$ fold-fold-fold-fold-fold-fold-fort-fortim-fort-fort。性能改进是由于目标函数仔细改变和``图形噪声''调度方法的结果，在每个扩散步骤中，它只能更改分子图中不同大小的节点的子集。 techniques that involve VAEs and require complicated reconstruction steps, compressed DMol directly performs graph diffusion on a graph that compresses only a carefully selected set of frequent carbon rings into supernodes, which results in straightforward sample generation. This compressed DMol method offers additional validity improvements over generic DMol of roughly $2\%$, increases the novelty of the method, and further improves the running time due to reductions in the graph 尺寸。

Title: Accelerating LLM Inference Throughput via Asynchronous KV Cache Prefetching

• Authors: Yanhao Dong, Yubo Miao, Weinan Li, Xiao Zheng, Chao Wang, Feng Lyu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.06319
• Pdf URL: https://arxiv.org/pdf/2504.06319
• Copy Paste: [[2504.06319]] Accelerating LLM Inference Throughput via Asynchronous KV Cache Prefetching(https://arxiv.org/abs/2504.06319)
• Keywords: generation
• Abstract: Large Language Models (LLMs) exhibit pronounced memory-bound characteristics during inference due to High Bandwidth Memory (HBM) bandwidth constraints. In this paper, we propose an L2 Cache-oriented asynchronous KV Cache prefetching method to break through the memory bandwidth bottleneck in LLM inference through computation-load overlap. By strategically scheduling idle memory bandwidth during active computation windows, our method proactively prefetches required KV Cache into GPU L2 cache, enabling high-speed L2 cache hits for subsequent accesses and effectively hiding HBM access latency within computational cycles. Extensive experiments on NVIDIA H20 GPUs demonstrate that the proposed method achieves 2.15x improvement in attention kernel efficiency and up to 1.97x end-to-end throughput enhancement, surpassing state-of-the-art baseline FlashAttention-3. Notably, our solution maintains orthogonality to existing optimization techniques and can be integrated with current inference frameworks, providing a scalable latency-hiding solution for next-generation LLM inference engines.
• 摘要：大型语言模型（LLM）在推理期间由于高带宽内存（HBM）带宽约束而表现出明显的记忆结合特性。在本文中，我们提出了一个面向L2的异步KV缓存预摘要方法，以通过计算载荷重叠在LLM推理中突破内存带宽瓶颈。通过在主动计算窗口中策略性地安排空闲内存带宽，我们的方法主动预取需要KV缓存到GPU L2缓存中，从而使高速L2高速缓存命中可用于后续访问，并有效地将HBM访问延迟隐藏在计算周期内。对NVIDIA H20 GPU进行的广泛实验表明，所提出的方法可提高注意力内核效率，并提高了1.97倍的端到端吞吐量增强，超过了先进的基线闪光体3。值得注意的是，我们的解决方案维持对现有优化技术的正交性，并且可以与当前的推理框架集成，为下一代LLM推理引擎提供了可扩展的延迟隐藏解决方案。

Title: Unifying Autoregressive and Diffusion-Based Sequence Generation

• Authors: Nima Fathi, Torsten Scholak, Pierre-André Noël
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.06416
• Pdf URL: https://arxiv.org/pdf/2504.06416
• Copy Paste: [[2504.06416]] Unifying Autoregressive and Diffusion-Based Sequence Generation(https://arxiv.org/abs/2504.06416)
• Keywords: generation
• Abstract: We present significant extensions to diffusion-based sequence generation models, blurring the line with autoregressive language models. We introduce hyperschedules, which assign distinct noise schedules to individual token positions, generalizing both autoregressive models (e.g., GPT) and conventional diffusion models (e.g., SEDD, MDLM) as special cases. Second, we propose two hybrid token-wise noising processes that interpolate between absorbing and uniform processes, enabling the model to fix past mistakes, and we introduce a novel inference algorithm that leverages this new feature in a simplified context inspired from MDLM. To support efficient training and inference, we design attention masks compatible with KV-caching. Our methods achieve state-of-the-art perplexity and generate diverse, high-quality sequences across standard benchmarks, suggesting a promising path for autoregressive diffusion-based sequence generation.
• 摘要：我们为基于扩散的序列生成模型提供了显着的扩展，将线路与自回归语言模型模糊。我们介绍了超级修剪，这些hyperschedules将独特的噪声时间表分配给单个令牌位置，从而将自回归模型（例如GPT）和常规扩散模型（例如SEDD，MDLM）概括为特殊情况。其次，我们提出了两个在吸收和统一过程之间插值的混合令牌的no脉过程，使模型能够解决过去的错误，并引入了一种新型的推理算法，该算法在MDLM启发的简化上下文中利用这一新功能。为了支持有效的训练和推理，我们设计了与KV摄影兼容的注意力面罩。我们的方法达到了最新的困惑，并在标准基准范围内产生了多样的高质量序列，这表明基于自回归扩散的序列产生的有希望的途径。

Title: Releasing Differentially Private Event Logs Using Generative Models

• Authors: Frederik Wangelik, Majid Rafiei, Mahsa Pourbafrani, Wil M.P. van der Aalst
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2504.06418
• Pdf URL: https://arxiv.org/pdf/2504.06418
• Copy Paste: [[2504.06418]] Releasing Differentially Private Event Logs Using Generative Models(https://arxiv.org/abs/2504.06418)
• Keywords: generative
• Abstract: In recent years, the industry has been witnessing an extended usage of process mining and automated event data analysis. Consequently, there is a rising significance in addressing privacy apprehensions related to the inclusion of sensitive and private information within event data utilized by process mining algorithms. State-of-the-art research mainly focuses on providing quantifiable privacy guarantees, e.g., via differential privacy, for trace variants that are used by the main process mining techniques, e.g., process discovery. However, privacy preservation techniques designed for the release of trace variants are still insufficient to meet all the demands of industry-scale utilization. Moreover, ensuring privacy guarantees in situations characterized by a high occurrence of infrequent trace variants remains a challenging endeavor. In this paper, we introduce two novel approaches for releasing differentially private trace variants based on trained generative models. With TraVaG, we leverage \textit{Generative Adversarial Networks} (GANs) to sample from a privatized implicit variant distribution. Our second method employs \textit{Denoising Diffusion Probabilistic Models} that reconstruct artificial trace variants from noise via trained Markov chains. Both methods offer industry-scale benefits and elevate the degree of privacy assurances, particularly in scenarios featuring a substantial prevalence of infrequent variants. Also, they overcome the shortcomings of conventional privacy preservation techniques, such as bounding the length of variants and introducing fake variants. Experimental results on real-life event data demonstrate that our approaches surpass state-of-the-art techniques in terms of privacy guarantees and utility preservation.
• 摘要：近年来，该行业一直目睹了流程挖掘和自动化事件数据分析的广泛使用。因此，解决与通过过程挖掘算法所使用的事件数据中纳入敏感和私人信息有关的隐私忧虑的重要性。最先进的研究主要集中于提供可量化的隐私保证，例如通过差异隐私，用于主要过程挖掘技术（例如过程发现）使用的痕量变体。但是，为释放痕量变体而设计的隐私保护技术仍然不足以满足行业规模利用的所有需求。此外，确保在以很高的痕量变体发生的情况下确保隐私保证仍然是一项艰巨的努力。在本文中，我们介绍了两种新型方法，用于根据训练有素的生成模型释放不同的私有微量变体。使用travag，我们利用\ textit {生成对抗网络}（gans）来从私有化的隐式变体分布中进行采样。我们的第二种方法采用\ textit {denoising扩散概率模型}，通过训练有素的马尔可夫链从噪声中重建人工痕量变体。两种方法都提供行业规模的好处并提高了隐私保证的程度，尤其是在很少出现不经常变体的情况下。此外，他们克服了常规隐私保护技术的缺点，例如界定变体的长度和引入假变体。现实事件数据的实验结果表明，我们的方法在隐私保证和公用事业保护方面超过了最新技术。

Title: D-Feat Occlusions: Diffusion Features for Robustness to Partial Visual Occlusions in Object Recognition

• Authors: Rupayan Mallick, Sibo Dong, Nataniel Ruiz, Sarah Adel Bargal
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06432
• Pdf URL: https://arxiv.org/pdf/2504.06432
• Copy Paste: [[2504.06432]] D-Feat Occlusions: Diffusion Features for Robustness to Partial Visual Occlusions in Object Recognition(https://arxiv.org/abs/2504.06432)
• Keywords: generation
• Abstract: Applications of diffusion models for visual tasks have been quite noteworthy. This paper targets making classification models more robust to occlusions for the task of object recognition by proposing a pipeline that utilizes a frozen diffusion model. Diffusion features have demonstrated success in image generation and image completion while understanding image context. Occlusion can be posed as an image completion problem by deeming the pixels of the occluder to be `missing.' We hypothesize that such features can help hallucinate object visual features behind occluding objects, and hence we propose using them to enable models to become more occlusion robust. We design experiments to include input-based augmentations as well as feature-based augmentations. Input-based augmentations involve finetuning on images where the occluder pixels are inpainted, and feature-based augmentations involve augmenting classification features with intermediate diffusion features. We demonstrate that our proposed use of diffusion-based features results in models that are more robust to partial object occlusions for both Transformers and ConvNets on ImageNet with simulated occlusions. We also propose a dataset that encompasses real-world occlusions and demonstrate that our method is more robust to partial object occlusions.
• 摘要：值得注意的是，扩散模型在视觉任务中的应用是非常值得注意的。本文通过提出利用冷冻扩散模型的管道，将分类模型更加健壮，以使对象识别任务的遮挡更加健壮。扩散特征在理解图像上下文的同时，在图像生成和图像完成方面取得了成功。通过认为封闭器的像素为“缺失”，可以将遮挡作为图像完成问题。我们假设这样的功能可以帮助幻觉对象在阻塞物体后面的视觉特征，因此我们建议使用它们使模型变得更加牢固。我们设计实验以包括基于输入的增强和基于功能的增强。基于输入的增强涉及对封闭式像素的图像进行填充，而基于特征的增强涉及具有中间扩散特征的分类特征。我们证明，我们提出的基于扩散的特征的使用会导致模型对具有模拟闭合的ImageNet上的变压器和反向网络的部分对象遮挡更强大。我们还提出了一个包含现实世界阻塞的数据集，并证明我们的方法对部分对象的闭合更强大。

Title: Implementation of a Zed 2i Stereo Camera for High-Frequency Shoreline Change and Coastal Elevation Monitoring

• Authors: José A. Pilartes-Congo, Matthew Kastl, Michael J. Starek, Marina Vicens-Miquel, Philippe Tissot
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06464
• Pdf URL: https://arxiv.org/pdf/2504.06464
• Copy Paste: [[2504.06464]] Implementation of a Zed 2i Stereo Camera for High-Frequency Shoreline Change and Coastal Elevation Monitoring(https://arxiv.org/abs/2504.06464)
• Keywords: generation
• Abstract: The increasing population, thus financial interests, in coastal areas have increased the need to monitor coastal elevation and shoreline change. Though several resources exist to obtain this information, they often lack the required temporal resolution for short-term monitoring (e.g., every hour). To address this issue, this study implements a low-cost ZED 2i stereo camera system and close-range photogrammetry to collect images for generating 3D point clouds, digital surface models (DSMs) of beach elevation, and georectified imagery at a localized scale and high temporal resolution. The main contributions of this study are (i) intrinsic camera calibration, (ii) georectification and registration of acquired imagery and point cloud, (iii) generation of the DSM of the beach elevation, and (iv) a comparison of derived products against those from uncrewed aircraft system structure-from-motion photogrammetry. Preliminary results show that despite its limitations, the ZED 2i can provide the desired mapping products at localized and high temporal scales. The system achieved a mean reprojection error of 0.20 px, a point cloud registration of 27 cm, a vertical error of 37.56 cm relative to ground truth, and georectification root mean square errors of 2.67 cm and 2.81 cm for x and y.
• 摘要：沿海地区的人口增加，因此财务利益增加了监测沿海高度和海岸线变化的需求。尽管存在几种资源来获取此信息，但他们通常缺乏短期监控所需的时间分辨率（例如，每小时）。为了解决此问题，本研究实现了低成本的ZED 2I立体声摄像机系统和近距离摄影测量法，以收集图像，以生成3D点云，海滩海拔的数字表面模型（DSM）以及以局部规模和高时间分辨率的地理图像。这项研究的主要贡献是（i）固有的摄像机校准，（ii）获得的图像和点云的地理化和注册，（iii）海滩海拔的DSM产生，以及（iv）与未螺旋的飞机系统结构结构结构的派生产品的比较。初步结果表明，尽管有局限性，但ZED 2i可以在局部和高时间尺度上提供所需的映射产品。该系统达到了0.20 px的平均再投影误差，相对于地面真理的垂直误差为37.56 cm，垂直误差为37.56 cm，而Georectification均方根均值均方根误差为2.67 cm，x和y的垂直误差为2.67 cm和2.81 cm。

Title: Rethinking LayerNorm in Image Restoration Transformers

• Authors: MinKyu Lee, Sangeek Hyun, Woojin Jun, Hyunjun Kim, Jiwoo Chung, Jae-Pil Heo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06629
• Pdf URL: https://arxiv.org/pdf/2504.06629
• Copy Paste: [[2504.06629]] Rethinking LayerNorm in Image Restoration Transformers(https://arxiv.org/abs/2504.06629)
• Keywords: restoration
• Abstract: This work investigates abnormal feature behaviors observed in image restoration (IR) Transformers. Specifically, we identify two critical issues: feature entropy becoming excessively small and feature magnitudes diverging up to a million-fold scale. We pinpoint the root cause to the per-token normalization aspect of conventional LayerNorm, which disrupts essential spatial correlations and internal feature statistics. To address this, we propose a simple normalization strategy tailored for IR Transformers. Our approach applies normalization across the entire spatio-channel dimension, effectively preserving spatial correlations. Additionally, we introduce an input-adaptive rescaling method that aligns feature statistics to the unique statistical requirements of each input. Experimental results verify that this combined strategy effectively resolves feature divergence, significantly enhancing both the stability and performance of IR Transformers across various IR tasks.
• 摘要：这项工作研究了图像恢复（IR）变压器中观察到的异常特征行为。具体来说，我们确定了两个关键问题：特征熵变得过于小，并且特征大小的变化达到了一百万倍。我们将根本原因指定为常规上层的直接归一化方面，这破坏了基本的空间相关性和内部特征统计。为了解决这个问题，我们提出了针对红外变压器量身定制的简单归一化策略。我们的方法在整个时空通道维度上采用归一化，从而有效地保留了空间相关性。此外，我们引入了一种输入自适应恢复方法，该方法将统计数据与每个输入的唯一统计要求保持一致。实验结果验证了这种组合策略有效地解决特征差异，从而显着增强了IR在各种IR任务中IR变压器的稳定性和性能。

Title: PosterMaker: Towards High-Quality Product Poster Generation with Accurate Text Rendering

• Authors: Yifan Gao, Zihang Lin, Chuanbin Liu, Min Zhou, Tiezheng Ge, Bo Zheng, Hongtao Xie
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06632
• Pdf URL: https://arxiv.org/pdf/2504.06632
• Copy Paste: [[2504.06632]] PosterMaker: Towards High-Quality Product Poster Generation with Accurate Text Rendering(https://arxiv.org/abs/2504.06632)
• Keywords: generation
• Abstract: Product posters, which integrate subject, scene, and text, are crucial promotional tools for attracting customers. Creating such posters using modern image generation methods is valuable, while the main challenge lies in accurately rendering text, especially for complex writing systems like Chinese, which contains over 10,000 individual characters. In this work, we identify the key to precise text rendering as constructing a character-discriminative visual feature as a control signal. Based on this insight, we propose a robust character-wise representation as control and we develop TextRenderNet, which achieves a high text rendering accuracy of over 90%. Another challenge in poster generation is maintaining the fidelity of user-specific products. We address this by introducing SceneGenNet, an inpainting-based model, and propose subject fidelity feedback learning to further enhance fidelity. Based on TextRenderNet and SceneGenNet, we present PosterMaker, an end-to-end generation framework. To optimize PosterMaker efficiently, we implement a two-stage training strategy that decouples text rendering and background generation learning. Experimental results show that PosterMaker outperforms existing baselines by a remarkable margin, which demonstrates its effectiveness.
• 摘要：整合主题，场景和文本的产品海报是吸引客户的关键促销工具。使用现代图像生成方法创建此类海报非常有价值，而主要的挑战在于准确地渲染文本，尤其是对于诸如中文（包含10,000多个字符）的复杂写作系统。在这项工作中，我们将精确文本渲染的关键确定为构建角色 - 歧义视觉特征作为控制信号。基于这种见解，我们提出了一个强大的角色表示，并开发了Textrendernet，它的文本渲染精度高达90％以上。海报生成的另一个挑战是维持特定用户产品的忠诚度。我们通过介绍基于介入的模型SceneGennet来解决这一问题，并提出主题保真反馈学习以进一步提高忠诚度。根据Textrendernet和SceneGennet，我们提出了端到端一代框架Postermaker。为了有效地优化Postermaker，我们实施了两阶段的培训策略，该策略将文本渲染和背景生成学习取消。实验结果表明，Postermaker的表现优于现有基线，这表明其有效性。

Title: Crafting Query-Aware Selective Attention for Single Image Super-Resolution

• Authors: Junyoung Kim, Youngrok Kim, Siyeol Jung, Donghyun Min
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06634
• Pdf URL: https://arxiv.org/pdf/2504.06634
• Copy Paste: [[2504.06634]] Crafting Query-Aware Selective Attention for Single Image Super-Resolution(https://arxiv.org/abs/2504.06634)
• Keywords: super-resolution
• Abstract: Single Image Super-Resolution (SISR) reconstructs high-resolution images from low-resolution inputs, enhancing image details. While Vision Transformer (ViT)-based models improve SISR by capturing long-range dependencies, they suffer from quadratic computational costs or employ selective attention mechanisms that do not explicitly focus on query-relevant regions. Despite these advancements, prior work has overlooked how selective attention mechanisms should be effectively designed for SISR. We propose SSCAN, which dynamically selects the most relevant key-value windows based on query similarity, ensuring focused feature extraction while maintaining efficiency. In contrast to prior approaches that apply attention globally or heuristically, our method introduces a query-aware window selection strategy that better aligns attention computation with important image regions. By incorporating fixed-sized windows, SSCAN reduces memory usage and enforces linear token-to-token complexity, making it scalable for large images. Our experiments demonstrate that SSCAN outperforms existing attention-based SISR methods, achieving up to 0.14 dB PSNR improvement on urban datasets, guaranteeing both computational efficiency and reconstruction quality in SISR.
• 摘要：单图像超分辨率（SISR）从低分辨率输入中重建高分辨率图像，从而增强图像细节。虽然Vision Transformer（VIT）模型通过捕获远程依赖性来改善SISR，但它们遭受了二次计算成本或采用选择性的注意机制，这些机制并未明确关注与查询相关的区域。尽管取得了这些进步，但先前的工作还是忽略了如何有效地为SISR设计选择性的注意机制。我们提出了SSCAN，该SSCAN基于查询相似性动态选择最相关的键值窗口，从而确保集中的特征提取同时保持效率。与以前在全球或启发式上应用注意力的方法相反，我们的方法引入了一种查询感知的窗口选择策略，该策略可以更好地将注意力计算与重要图像区域保持一致。通过合并固定尺寸的窗口，SSCAN可以减少内存使用量并实施线性令牌到to的复杂性，从而使其可扩展到大图像。我们的实验表明，SSCAN的表现优于现有的基于注意力的SISR方法，在Urban数据集上最多可改善0.14 dB PSNR，保证了SISR的计算效率和重建质量。

Title: Patch Matters: Training-free Fine-grained Image Caption Enhancement via Local Perception

• Authors: Ruotian Peng, Haiying He, Yake Wei, Yandong Wen, Di Hu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06666
• Pdf URL: https://arxiv.org/pdf/2504.06666
• Copy Paste: [[2504.06666]] Patch Matters: Training-free Fine-grained Image Caption Enhancement via Local Perception(https://arxiv.org/abs/2504.06666)
• Keywords: generation
• Abstract: High-quality image captions play a crucial role in improving the performance of cross-modal applications such as text-to-image generation, text-to-video generation, and text-image retrieval. To generate long-form, high-quality captions, many recent studies have employed multimodal large language models (MLLMs). However, current MLLMs often produce captions that lack fine-grained details or suffer from hallucinations, a challenge that persists in both open-source and closed-source models. Inspired by Feature-Integration theory, which suggests that attention must focus on specific regions to integrate visual information effectively, we propose a \textbf{divide-then-aggregate} strategy. Our method first divides the image into semantic and spatial patches to extract fine-grained details, enhancing the model's local perception of the image. These local details are then hierarchically aggregated to generate a comprehensive global description. To address hallucinations and inconsistencies in the generated captions, we apply a semantic-level filtering process during hierarchical aggregation. This training-free pipeline can be applied to both open-source models (LLaVA-1.5, LLaVA-1.6, Mini-Gemini) and closed-source models (Claude-3.5-Sonnet, GPT-4o, GLM-4V-Plus). Extensive experiments demonstrate that our method generates more detailed, reliable captions, advancing multimodal description generation without requiring model retraining. The source code are available at this https URL
• 摘要：高质量的图像标题在改善跨模式应用的性能（例如文本到图像生成，文本到视频生成和文本图像检索）方面起着至关重要的作用。为了产生长形式的高质量标题，许多最近的研究采用了多模式模型（MLLM）。但是，当前的MLLM通常会产生缺乏细节细节或幻觉的标题，这一挑战始终存在于开源和封闭源模型中。受特征融合理论的启发，该理论表明，注意力必须集中在特定区域以有效地整合视觉信息，我们提出了\ textbf {divide-then-aggregate}策略。我们的方法首先将图像分为语义和空间斑块，以提取细粒细节，从而增强了模型对图像的本地感知。然后将这些本地详细信息分层汇总以生成全面的全球描述。为了解决生成的字幕中的幻觉和不一致，我们在层次聚合过程中应用了语义级过滤过程。该无训练管道可以应用于开源模型（LLAVA-1.5，LLAVA-1.6，MINI-GEMINI）和闭合源模型（Claude-3.5-Sonnet，GPT-4O，GPT-4O，GLM-4V-Plus）。广泛的实验表明，我们的方法生成更详细的，可靠的字幕，不需要模型再培训就可以推进多模式描述生成。源代码可在此HTTPS URL上找到

Title: RAGME: Retrieval Augmented Video Generation for Enhanced Motion Realism

• Authors: Elia Peruzzo, Dejia Xu, Xingqian Xu, Humphrey Shi, Nicu Sebe
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06672
• Pdf URL: https://arxiv.org/pdf/2504.06672
• Copy Paste: [[2504.06672]] RAGME: Retrieval Augmented Video Generation for Enhanced Motion Realism(https://arxiv.org/abs/2504.06672)
• Keywords: generation
• Abstract: Video generation is experiencing rapid growth, driven by advances in diffusion models and the development of better and larger datasets. However, producing high-quality videos remains challenging due to the high-dimensional data and the complexity of the task. Recent efforts have primarily focused on enhancing visual quality and addressing temporal inconsistencies, such as flickering. Despite progress in these areas, the generated videos often fall short in terms of motion complexity and physical plausibility, with many outputs either appearing static or exhibiting unrealistic motion. In this work, we propose a framework to improve the realism of motion in generated videos, exploring a complementary direction to much of the existing literature. Specifically, we advocate for the incorporation of a retrieval mechanism during the generation phase. The retrieved videos act as grounding signals, providing the model with demonstrations of how the objects move. Our pipeline is designed to apply to any text-to-video diffusion model, conditioning a pretrained model on the retrieved samples with minimal fine-tuning. We demonstrate the superiority of our approach through established metrics, recently proposed benchmarks, and qualitative results, and we highlight additional applications of the framework.
• 摘要：在扩散模型的进步和更好和更大的数据集的发展驱动的驱动到驱动的是，视频生产正在经历快速增长。但是，由于高维数据和任务的复杂性，制作高质量的视频仍然具有挑战性。最近的努力主要集中在提高视觉质量和解决时间矛盾之处，例如闪烁。尽管在这些领域取得了进展，但生成的视频通常在运动复杂性和物理上的合理性方面缺乏，许多输出要么出现静态或表现出不切实际的运动。在这项工作中，我们提出了一个框架，以改善产生的视频中运动的现实主义，探索许多现有文献的互补方向。具体而言，我们主张在生成阶段掺入检索机制。检索到的视频是接地信号，为模型提供了对象如何移动的演示。我们的管道旨在适用于任何文本到视频扩散模型，以最小的微调对检索到的样品进行验证的模型。我们通过既定的指标，最近提出的基准和定性结果来证明我们的方法的优势，我们重点介绍了该框架的其他应用。

Title: CAT: Circular-Convolutional Attention for Sub-Quadratic Transformers

• Authors: Yoshihiro Yamada
• Subjects: cs.LG, cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06704
• Pdf URL: https://arxiv.org/pdf/2504.06704
• Copy Paste: [[2504.06704]] CAT: Circular-Convolutional Attention for Sub-Quadratic Transformers(https://arxiv.org/abs/2504.06704)
• Keywords: generation
• Abstract: Transformers have driven remarkable breakthroughs in natural language processing and computer vision, yet their standard attention mechanism still imposes O(N^2) complexity, hindering scalability to longer sequences. We introduce Circular-convolutional ATtention (CAT), a Fourier-based approach that efficiently applies circular convolutions to reduce complexity without sacrificing representational power. CAT achieves O(NlogN) computations, requires fewer learnable parameters by streamlining fully-connected layers, and introduces no heavier operations, resulting in consistent accuracy improvements and about a 10% speedup in naive PyTorch implementations on large-scale benchmarks such as ImageNet-1k and WikiText-103. Grounded in an engineering-isomorphism framework, CAT's design not only offers practical efficiency and ease of implementation but also provides insights to guide the development of next-generation, high-performance Transformer architectures. Finally, our ablation studies highlight the key conditions underlying CAT's success, shedding light on broader principles for scalable attention mechanisms.
• 摘要：变形金刚在自然语言处理和计算机视觉方面取得了显着突破，但是它们的标准注意机制仍然施加O（n^2）复杂性，阻碍了对更长序列的可扩展性。我们引入了循环 - 跨跨性关注（CAT），这是一种基于傅立叶的方法，可有效地采用循环卷积以降低复杂性而无需牺牲代表力。 CAT达到O（NLOGN）计算，通过简化完全连接的层，需要更少的可学习参数，并且没有引入较重的操作，从而可以提高准确性，并且在幼稚的Pytorch实现的大规模基准测试中，诸如Imagenet-1K和Wikitext-103。 Cat的设计以工程形式的框架为基础，不仅提供了实用的效率和易于实施，而且还提供了见解，以指导下一代，高性能的变压器体系结构的发展。最后，我们的消融研究突出了猫成功的关键条件，阐明了更广泛的可扩展注意机制的原则。

Title: Plastic tensor networks for interpretable generative modeling

• Authors: Katsuya O. Akamatsu, Kenji Harada, Tsuyoshi Okubo, Naoki Kawashima
• Subjects: cs.LG, cond-mat.stat-mech
• Abstract URL: https://arxiv.org/abs/2504.06722
• Pdf URL: https://arxiv.org/pdf/2504.06722
• Copy Paste: [[2504.06722]] Plastic tensor networks for interpretable generative modeling(https://arxiv.org/abs/2504.06722)
• Keywords: generative
• Abstract: A structural optimization scheme for a single-layer nonnegative adaptive tensor tree (NATT) that models a target probability distribution is proposed. The NATT scheme, by construction, has the advantage that it is interpretable as a probabilistic graphical model. We consider the NATT scheme and a recently proposed Born machine adaptive tensor tree (BMATT) optimization scheme and demonstrate their effectiveness on a variety of generative modeling tasks where the objective is to infer the hidden structure of a provided dataset. Our results show that in terms of minimizing the negative log-likelihood, the single-layer scheme has model performance comparable to the Born machine scheme, though not better. The tasks include deducing the structure of binary bitwise operations, learning the internal structure of random Bayesian networks given only visible sites, and a real-world example related to hierarchical clustering where a cladogram is constructed from mitochondrial DNA sequences. In doing so, we also show the importance of the choice of network topology and the versatility of a least-mutual information criterion in selecting a candidate structure for a tensor tree, as well as discuss aspects of these tensor tree generative models including their information content and interpretability.
• 摘要：提出了一种模拟目标概率分布的单层非负自适应张量树（Natt）的结构优化方案。 Natt方案通过构造具有一个优势，即可以解释为概率图形模型。我们考虑了Natt方案和最近提出的Born机器自适应张量树（BMATT）优化方案，并证明了它们在各种生成建模任务上的有效性，目的是推断提供的数据集的隐藏结构。我们的结果表明，在最大程度地减少负模样的情况下，单层方案的模型性能与Born Machine方案相当，尽管不是更好。这些任务包括推论二进制比位操作的结构，学习仅给出可见位点的随机贝叶斯网络的内部结构，以及与分层聚类有关的现实世界示例，其中clastrogram是由线粒体DNA序列构建的。在此过程中，我们还展示了网络拓扑选择的重要性以及在为张量树选择候选结构时的最小信息标准的多功能性，并讨论了这些张量树生成模型的各个方面，包括其信息内容和解释性。

Title: Compass Control: Multi Object Orientation Control for Text-to-Image Generation

• Authors: Rishbuh Parihar, Vaibhav Agrawal, Sachidanand VS, R. Venkatesh Babu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06752
• Pdf URL: https://arxiv.org/pdf/2504.06752
• Copy Paste: [[2504.06752]] Compass Control: Multi Object Orientation Control for Text-to-Image Generation(https://arxiv.org/abs/2504.06752)
• Keywords: generation
• Abstract: Existing approaches for controlling text-to-image diffusion models, while powerful, do not allow for explicit 3D object-centric control, such as precise control of object orientation. In this work, we address the problem of multi-object orientation control in text-to-image diffusion models. This enables the generation of diverse multi-object scenes with precise orientation control for each object. The key idea is to condition the diffusion model with a set of orientation-aware \textbf{compass} tokens, one for each object, along with text tokens. A light-weight encoder network predicts these compass tokens taking object orientation as the input. The model is trained on a synthetic dataset of procedurally generated scenes, each containing one or two 3D assets on a plain background. However, direct training this framework results in poor orientation control as well as leads to entanglement among objects. To mitigate this, we intervene in the generation process and constrain the cross-attention maps of each compass token to its corresponding object regions. The trained model is able to achieve precise orientation control for a) complex objects not seen during training and b) multi-object scenes with more than two objects, indicating strong generalization capabilities. Further, when combined with personalization methods, our method precisely controls the orientation of the new object in diverse contexts. Our method achieves state-of-the-art orientation control and text alignment, quantified with extensive evaluations and a user study.
• 摘要：现有的控制文本对图像扩散模型的方法虽然功能强大，但不允许明确的3D对象以中心的控制，例如对对象方向的精确控制。在这项工作中，我们解决了文本到图像扩散模型中多对象方向控制的问题。这使得为​​每个对象具有精确的方向控制的不同多对象场景的生成。关键思想是用一组方向敏感\ textbf {compass}令牌来调节扩散模型，每个对象一个以及文本令牌。轻巧的编码器网络预测这些指南针将对象取向作为输入。该模型在程序生成的场景的合成数据集上进行了训练，每个数据集在普通背景上包含一个或两个3D资产。但是，直接培训该框架会导致方向控制不良，并导致物体之间的纠缠。为了减轻这种情况，我们干预生成过程，并将每个指南令牌的交叉注意地图限制为其相应的对象区域。受过训练的模型能够实现a）在训练中未见的复杂对象的精确取向控制，b）具有两个以上对象的多对象场景，表明强大的概括能力。此外，当与个性化方法结合使用时，我们的方法精确地控制了不同环境中新对象的方向。我们的方法实现了最新的方向控制和文本对齐方式，并通过广泛的评估和用户研究来量化。

Title: FANeRV: Frequency Separation and Augmentation based Neural Representation for Video

• Authors: Li Yu, Zhihui Li, Jimin Xiao, Moncef Gabbouj
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06755
• Pdf URL: https://arxiv.org/pdf/2504.06755
• Copy Paste: [[2504.06755]] FANeRV: Frequency Separation and Augmentation based Neural Representation for Video(https://arxiv.org/abs/2504.06755)
• Keywords: restoration
• Abstract: Neural representations for video (NeRV) have gained considerable attention for their strong performance across various video tasks. However, existing NeRV methods often struggle to capture fine spatial details, resulting in vague reconstructions. In this paper, we present a Frequency Separation and Augmentation based Neural Representation for video (FANeRV), which addresses these limitations with its core Wavelet Frequency Upgrade this http URL block explicitly separates input frames into high and low-frequency components using discrete wavelet transform, followed by targeted enhancement using specialized modules. Finally, a specially designed gated network effectively fuses these frequency components for optimal reconstruction. Additionally, convolutional residual enhancement blocks are integrated into the later stages of the network to balance parameter distribution and improve the restoration of high-frequency details. Experimental results demonstrate that FANeRV significantly improves reconstruction performance and excels in multiple tasks, including video compression, inpainting, and interpolation, outperforming existing NeRV methods.
• 摘要：视频（神经）的神经表示因其在各种视频任务中的出色表现而引起了极大的关注。但是，现有的神经方法通常难以捕获精细的空间细节，从而导致模糊的重建。在本文中，我们提出了视频（FANERV）的频率分离和基于增强的神经表示，该神经表示通过其核心小波频率升级该HTTP URL块将输入框架明确将输入框架分为高和低频率的组件，并使用离散的小波变换，随后是使用专用模块的靶向增强。最后，一个专门设计的门网网络有效地融合了这些频率组件以进行最佳重建。此外，将卷积残差增强块集成到网络的后期，以平衡参数分布并改善高频细节的恢复。实验结果表明，FANERV显着提高了重建性能，并在多个任务中脱颖而出，包括视频压缩，介入和插值，表现优于现有的神经方法。

Title: A Meaningful Perturbation Metric for Evaluating Explainability Methods

• Authors: Danielle Cohen, Hila Chefer, Lior Wolf
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06800
• Pdf URL: https://arxiv.org/pdf/2504.06800
• Copy Paste: [[2504.06800]] A Meaningful Perturbation Metric for Evaluating Explainability Methods(https://arxiv.org/abs/2504.06800)
• Keywords: generation
• Abstract: Deep neural networks (DNNs) have demonstrated remarkable success, yet their wide adoption is often hindered by their opaque decision-making. To address this, attribution methods have been proposed to assign relevance values to each part of the input. However, different methods often produce entirely different relevance maps, necessitating the development of standardized metrics to evaluate them. Typically, such evaluation is performed through perturbation, wherein high- or low-relevance regions of the input image are manipulated to examine the change in prediction. In this work, we introduce a novel approach, which harnesses image generation models to perform targeted perturbation. Specifically, we focus on inpainting only the high-relevance pixels of an input image to modify the model's predictions while preserving image fidelity. This is in contrast to existing approaches, which often produce out-of-distribution modifications, leading to unreliable results. Through extensive experiments, we demonstrate the effectiveness of our approach in generating meaningful rankings across a wide range of models and attribution methods. Crucially, we establish that the ranking produced by our metric exhibits significantly higher correlation with human preferences compared to existing approaches, underscoring its potential for enhancing interpretability in DNNs.
• 摘要：深度神经网络（DNNS）取得了巨大的成功，但是他们不透明的决策通常会阻碍他们的广泛采用。为了解决这个问题，已经提出了归因方法将相关值分配给输入的每个部分。但是，不同的方法通常会产生完全不同的相关图图，因此需要开发标准化指标来评估它们。通常，这种评估是通过扰动进行的，其中操纵输入图像的高或低相位区域以检查预测的变化。在这项工作中，我们介绍了一种新颖的方法，该方法利用图像生成模型执行靶向扰动。具体而言，我们专注于仅介绍输入图像的高相关像素，以在保留图像保真度的同时修改模型的预测。这与现有方法相反，后者通常会产生分布式的修改，从而导致不可靠的结果。通过广泛的实验，我们证明了方法在跨广泛模型和归因方法中产生有意义的排名方面的有效性。至关重要的是，我们确定与现有方法相比，我们的度量标准所产生的排名与人类偏好的相关性明显更高，强调了其增强DNN中可解释性的潜力。

Title: MonoPlace3D: Learning 3D-Aware Object Placement for 3D Monocular Detection

• Authors: Rishubh Parihar, Srinjay Sarkar, Sarthak Vora, Jogendra Kundu, R. Venkatesh Babu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06801
• Pdf URL: https://arxiv.org/pdf/2504.06801
• Copy Paste: [[2504.06801]] MonoPlace3D: Learning 3D-Aware Object Placement for 3D Monocular Detection(https://arxiv.org/abs/2504.06801)
• Keywords: generation
• Abstract: Current monocular 3D detectors are held back by the limited diversity and scale of real-world datasets. While data augmentation certainly helps, it's particularly difficult to generate realistic scene-aware augmented data for outdoor settings. Most current approaches to synthetic data generation focus on realistic object appearance through improved rendering techniques. However, we show that where and how objects are positioned is just as crucial for training effective 3D monocular detectors. The key obstacle lies in automatically determining realistic object placement parameters - including position, dimensions, and directional alignment when introducing synthetic objects into actual scenes. To address this, we introduce MonoPlace3D, a novel system that considers the 3D scene content to create realistic augmentations. Specifically, given a background scene, MonoPlace3D learns a distribution over plausible 3D bounding boxes. Subsequently, we render realistic objects and place them according to the locations sampled from the learned distribution. Our comprehensive evaluation on two standard datasets KITTI and NuScenes, demonstrates that MonoPlace3D significantly improves the accuracy of multiple existing monocular 3D detectors while being highly data efficient.
• 摘要：当前的单眼3D检测器被现实世界数据集的多样性和规模有限。虽然数据增强肯定会有所帮助，但要为室外设置生成现实的场景增强数据特别困难。通过改进的渲染技术，大多数合成数据生成的方法都集中在逼真的对象外观上。但是，我们表明对象的定位位置和如何定位对于训练有效的3D单眼探测器同样至关重要。关键障碍在于自动确定现实的对象放置参数（包括位置，尺寸和定向对齐），当将合成对象引入实际场景时。为了解决这个问题，我们介绍了Monoplace3D，这是一个新型系统，它考虑了3D场景内容以创建现实的增强。具体而言，在背景场景中，monoplace3d了解了合理的3D边界框上的分布。随后，我们呈现逼真的对象，并根据从学习分布中采样的位置放置它们。我们对两个标准数据集Kitti和Nuscenes的全面评估表明，Monoplace3D显着提高了多个现有单眼3D检测器的准确性，同时高度数据效率。

Title: DyDiT++: Dynamic Diffusion Transformers for Efficient Visual Generation

• Authors: Wangbo Zhao, Yizeng Han, Jiasheng Tang, Kai Wang, Hao Luo, Yibing Song, Gao Huang, Fan Wang, Yang You
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06803
• Pdf URL: https://arxiv.org/pdf/2504.06803
• Copy Paste: [[2504.06803]] DyDiT++: Dynamic Diffusion Transformers for Efficient Visual Generation(https://arxiv.org/abs/2504.06803)
• Keywords: generation
• Abstract: Diffusion Transformer (DiT), an emerging diffusion model for visual generation, has demonstrated superior performance but suffers from substantial computational costs. Our investigations reveal that these costs primarily stem from the \emph{static} inference paradigm, which inevitably introduces redundant computation in certain \emph{diffusion timesteps} and \emph{spatial regions}. To overcome this inefficiency, we propose \textbf{Dy}namic \textbf{Di}ffusion \textbf{T}ransformer (DyDiT), an architecture that \emph{dynamically} adjusts its computation along both \emph{timestep} and \emph{spatial} dimensions. Specifically, we introduce a \emph{Timestep-wise Dynamic Width} (TDW) approach that adapts model width conditioned on the generation timesteps. In addition, we design a \emph{Spatial-wise Dynamic Token} (SDT) strategy to avoid redundant computation at unnecessary spatial locations. TDW and SDT can be seamlessly integrated into DiT and significantly accelerates the generation process. Building on these designs, we further enhance DyDiT in three key aspects. First, DyDiT is integrated seamlessly with flow matching-based generation, enhancing its versatility. Furthermore, we enhance DyDiT to tackle more complex visual generation tasks, including video generation and text-to-image generation, thereby broadening its real-world applications. Finally, to address the high cost of full fine-tuning and democratize technology access, we investigate the feasibility of training DyDiT in a parameter-efficient manner and introduce timestep-based dynamic LoRA (TD-LoRA). Extensive experiments on diverse visual generation models, including DiT, SiT, Latte, and FLUX, demonstrate the effectiveness of DyDiT.
• 摘要：扩散变压器（DIT）是一个新兴的视觉生成扩散模型，它表现出了卓越的性能，但遭受了实质性的计算成本。我们的研究表明，这些成本主要源于\ emph {static}的推论范式，这不可避免地在某些\ emph {forfusion timeSteps}和\ emph {fastial empatial区域}中引入了冗余计算。为了克服这种低效率，我们提出\ textbf {dy} namic \ textbf {di} ffusion \ textbf {t} ransformer（dydit），这是一个\ emph {emph {dynamedaly}沿\ emph {timph {timeestepp}和\ emph}的体系结构。具体来说，我们引入了一个\ emph {timeStep-wise动态宽度}（TDW）方法，该方法适应了在生成时间段上的模型宽度。此外，我们设计了一个\ emph {空间动态令牌}（SDT）策略，以避免在不必要的空间位置进行冗余计算。 TDW和SDT可以无缝集成到DIT中，并显着加速生成过程。在这些设计的基础上，我们在三个关键方面进一步增强了Dydit。首先，Dydit与基于流匹配的一代无缝集成，从而增强了其多功能性。此外，我们增强了DYDIT，以应对更复杂的视觉生成任务，包括视频生成和文本图像生成，从而扩大其现实世界的应用。最后，为了解决全面微调和民主化技术访问的高昂成本，我们以参数有效的方式调查了培训dydit的可行性，并介绍了基于时间段的动态洛拉（TD-lora）。在包括DIT，SIT，Taitte和Flux在内的各种视觉生成模型上进行了广泛的实验，证明了Dydit的有效性。

Title: Classifying the Unknown: In-Context Learning for Open-Vocabulary Text and Symbol Recognition

• Authors: Tom Simon, William Mocaer, Pierrick Tranouez, Clement Chatelain, Thierry Paquet
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06841
• Pdf URL: https://arxiv.org/pdf/2504.06841
• Copy Paste: [[2504.06841]] Classifying the Unknown: In-Context Learning for Open-Vocabulary Text and Symbol Recognition(https://arxiv.org/abs/2504.06841)
• Keywords: generation
• Abstract: We introduce Rosetta, a multimodal model that leverages Multimodal In-Context Learning (MICL) to classify sequences of novel script patterns in documents by leveraging minimal examples, thus eliminating the need for explicit retraining. To enhance contextual learning, we designed a dataset generation process that ensures varying degrees of contextual informativeness, improving the model's adaptability in leveraging context across different scenarios. A key strength of our method is the use of a Context-Aware Tokenizer (CAT), which enables open-vocabulary classification. This allows the model to classify text and symbol patterns across an unlimited range of classes, extending its classification capabilities beyond the scope of its training alphabet of patterns. As a result, it unlocks applications such as the recognition of new alphabets and languages. Experiments on synthetic datasets demonstrate the potential of Rosetta to successfully classify Out-Of-Distribution visual patterns and diverse sets of alphabets and scripts, including but not limited to Chinese, Greek, Russian, French, Spanish, and Japanese.
• 摘要：我们介绍了Rosetta，这是一种多模型，利用多模式的内在学习（MICL）通过利用最小示例，从而在文档中对新型脚本模式进行分类，从而消除了对显式再培训的需求。为了增强上下文学习，我们设计了一个数据集生成过程，该过程可确保不同程度的上下文信息性，从而改善了模型在不同情况下利用上下文中的适应性。我们方法的关键优势是使用上下文感知的令牌剂（CAT），这可以实现开放式播放性分类。这使该模型可以在无限范围的类中对文本和符号模式进行分类，从而将其分类功能扩展到其模式训练字母的范围之外。结果，它解锁了诸如识别新字母和语言的应用程序。关于合成数据集的实验表明，罗塞塔（Rosetta）成功地将分布外视觉模式和各种字母和脚本集进行了分类，包括但不限于中文，希腊语，俄罗斯，法语，西班牙语和日语。

Title: EIDT-V: Exploiting Intersections in Diffusion Trajectories for Model-Agnostic, Zero-Shot, Training-Free Text-to-Video Generation

• Authors: Diljeet Jagpal, Xi Chen, Vinay P. Namboodiri
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.06861
• Pdf URL: https://arxiv.org/pdf/2504.06861
• Copy Paste: [[2504.06861]] EIDT-V: Exploiting Intersections in Diffusion Trajectories for Model-Agnostic, Zero-Shot, Training-Free Text-to-Video Generation(https://arxiv.org/abs/2504.06861)
• Keywords: generation
• Abstract: Zero-shot, training-free, image-based text-to-video generation is an emerging area that aims to generate videos using existing image-based diffusion models. Current methods in this space require specific architectural changes to image generation models, which limit their adaptability and scalability. In contrast to such methods, we provide a model-agnostic approach. We use intersections in diffusion trajectories, working only with the latent values. We could not obtain localized frame-wise coherence and diversity using only the intersection of trajectories. Thus, we instead use a grid-based approach. An in-context trained LLM is used to generate coherent frame-wise prompts; another is used to identify differences between frames. Based on these, we obtain a CLIP-based attention mask that controls the timing of switching the prompts for each grid cell. Earlier switching results in higher variance, while later switching results in more coherence. Therefore, our approach can ensure appropriate control between coherence and variance for the frames. Our approach results in state-of-the-art performance while being more flexible when working with diverse image-generation models. The empirical analysis using quantitative metrics and user studies confirms our model's superior temporal consistency, visual fidelity and user satisfaction, thus providing a novel way to obtain training-free, image-based text-to-video generation.
• 摘要：零射击，无训练，基于图像的文本对视频生成是一个新兴领域，旨在使用现有的基于图像的扩散模型生成视频。该空间中的当前方法需要对图像生成模型进行特定的体系结构更改，从而限制其适应性和可扩展性。与此类方法相反，我们提供了一种模型不可屈服的方法。我们在扩散轨迹中使用相交，仅与潜在值一起工作。我们无法仅使用轨迹的交汇处获得局部框架的连贯性和多样性。因此，我们使用基于网格的方法。受培训的LLM用于生成连贯的框架提示；另一个用于识别帧之间的差异。基于这些，我们获得了一个基于夹的注意力掩码，该掩模可以控制切换每个网格单元提示的时间。较早的切换会导致较高的差异，而后来切换会导致更连贯性。因此，我们的方法可以确保框架的连贯性和差异之间的适当控制。我们的方法会导致最先进的性能，同时在使用各种图像生成模型时更加灵活。使用定量指标和用户研究的经验分析证实了我们模型的卓越时间一致性，视觉保真度和用户满意度，从而提供了一种新颖的方式来获得无训练的，基于图像的文本到视频生成。

Title: ColorizeDiffusion v2: Enhancing Reference-based Sketch Colorization Through Separating Utilities

• Authors: Dingkun Yan, Xinrui Wang, Yusuke Iwasawa, Yutaka Matsuo, Suguru Saito, Jiaxian Guo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06895
• Pdf URL: https://arxiv.org/pdf/2504.06895
• Copy Paste: [[2504.06895]] ColorizeDiffusion v2: Enhancing Reference-based Sketch Colorization Through Separating Utilities(https://arxiv.org/abs/2504.06895)
• Keywords: generation
• Abstract: Reference-based sketch colorization methods have garnered significant attention due to their potential applications in the animation production industry. However, most existing methods are trained with image triplets of sketch, reference, and ground truth that are semantically and spatially well-aligned, while real-world references and sketches often exhibit substantial misalignment. This mismatch in data distribution between training and inference leads to overfitting, consequently resulting in spatial artifacts and significant degradation in overall colorization quality, limiting potential applications of current methods for general purposes. To address this limitation, we conduct an in-depth analysis of the \textbf{carrier}, defined as the latent representation facilitating information transfer from reference to sketch. Based on this analysis, we propose a novel workflow that dynamically adapts the carrier to optimize distinct aspects of colorization. Specifically, for spatially misaligned artifacts, we introduce a split cross-attention mechanism with spatial masks, enabling region-specific reference injection within the diffusion process. To mitigate semantic neglect of sketches, we employ dedicated background and style encoders to transfer detailed reference information in the latent feature space, achieving enhanced spatial control and richer detail synthesis. Furthermore, we propose character-mask merging and background bleaching as preprocessing steps to improve foreground-background integration and background generation. Extensive qualitative and quantitative evaluations, including a user study, demonstrate the superior performance of our proposed method compared to existing approaches. An ablation study further validates the efficacy of each proposed component.
• 摘要：基于参考的草图着色方法由于动画生产行业的潜在应用而引起了极大的关注。但是，大多数现有的方法都经过素描，参考和地面真理的图像三重态，这些图像在语义和空间上良好，而现实世界中的参考和草图经常表现出很大的错位。训练和推理之间的数据分布不匹配会导致过度拟合，因此导致空间伪像和整体着色质量的显着降解，从而限制了当前方法的潜在应用。为了解决此限制，我们对\ textbf {carrier}进行了深入分析，该分析定义为潜在表示促进信息从参考到草图的转移。基于此分析，我们提出了一种新型的工作流程，该新工作流程动态适应载体以优化着色的不同方面。具体而言，对于空间未对准的伪影，我们引入了带有空间遮罩的分裂跨注意机制，在扩散过程中为特定于区域的参考注射提供了。为了减轻对草图的语义忽略，我们采用专门的背景和样式编码器来传输潜在特征空间中的详细参考信息，从而实现增强的空间控制和更丰富的细节综合。此外，我们提出角色掩盖合并和背景漂白，作为预处理步骤，以改善前后背景的集成和背景生产。包括用户研究在内的广泛的定性和定量评估表明，与现有方法相比，我们提出的方法的出色性能。一项消融研究进一步验证了每个提出的组件的功效。

Title: MedSegFactory: Text-Guided Generation of Medical Image-Mask Pairs

• Authors: Jiawei Mao, Yuhan Wang, Yucheng Tang, Daguang Xu, Kang Wang, Yang Yang, Zongwei Zhou, Yuyin Zhou
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.06897
• Pdf URL: https://arxiv.org/pdf/2504.06897
• Copy Paste: [[2504.06897]] MedSegFactory: Text-Guided Generation of Medical Image-Mask Pairs(https://arxiv.org/abs/2504.06897)
• Keywords: generation
• Abstract: This paper presents MedSegFactory, a versatile medical synthesis framework that generates high-quality paired medical images and segmentation masks across modalities and tasks. It aims to serve as an unlimited data repository, supplying image-mask pairs to enhance existing segmentation tools. The core of MedSegFactory is a dual-stream diffusion model, where one stream synthesizes medical images and the other generates corresponding segmentation masks. To ensure precise alignment between image-mask pairs, we introduce Joint Cross-Attention (JCA), enabling a collaborative denoising paradigm by dynamic cross-conditioning between streams. This bidirectional interaction allows both representations to guide each other's generation, enhancing consistency between generated pairs. MedSegFactory unlocks on-demand generation of paired medical images and segmentation masks through user-defined prompts that specify the target labels, imaging modalities, anatomical regions, and pathological conditions, facilitating scalable and high-quality data generation. This new paradigm of medical image synthesis enables seamless integration into diverse medical imaging workflows, enhancing both efficiency and accuracy. Extensive experiments show that MedSegFactory generates data of superior quality and usability, achieving competitive or state-of-the-art performance in 2D and 3D segmentation tasks while addressing data scarcity and regulatory constraints.
• 摘要：本文介绍了MedSegFactory，这是一种多功能的医学合成框架，可在模态和任务中生成高质量的配对医学图像和分割面具。它旨在用作无限的数据存储库，提供图像面罩对以增强现有的分割工具。 MedSegfactory的核心是双流扩散模型，其中一个流综合了医学图像，另一个流则生成相应的分割掩码。为了确保图像面罩对之间的精确对齐，我们引入了联合交叉注意（JCA），从而通过在流之间进行动态交叉调节，从而实现了协作的剥落范式。这种双向相互作用使两种表示都可以指导彼此的生成，从而增强了生成对之间的一致性。 MedSegFactory通过用户定义的提示来解锁配对的医学图像和分割掩盖的需求，这些提示指定目标标签，成像方式，解剖区域和病理状况，从而促进可扩展和高质量数据的生成。这种新的医学图像合成范式可以使各种医学成像工作流程无缝集成，从而提高效率和准确性。广泛的实验表明，MedSegFactory生成了质量和可用性的数据，在2D和3D分割任务中实现了竞争性或最先进的性能，同时解决数据稀缺性和监管约束。

Title: SIGMAN:Scaling 3D Human Gaussian Generation with Millions of Assets

• Authors: Yuhang Yang, Fengqi Liu, Yixing Lu, Qin Zhao, Pingyu Wu, Wei Zhai, Ran Yi, Yang Cao, Lizhuang Ma, Zheng-Jun Zha, Junting Dong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06982
• Pdf URL: https://arxiv.org/pdf/2504.06982
• Copy Paste: [[2504.06982]] SIGMAN:Scaling 3D Human Gaussian Generation with Millions of Assets(https://arxiv.org/abs/2504.06982)
• Keywords: generation
• Abstract: 3D human digitization has long been a highly pursued yet challenging task. Existing methods aim to generate high-quality 3D digital humans from single or multiple views, but remain primarily constrained by current paradigms and the scarcity of 3D human assets. Specifically, recent approaches fall into several paradigms: optimization-based and feed-forward (both single-view regression and multi-view generation with reconstruction). However, they are limited by slow speed, low quality, cascade reasoning, and ambiguity in mapping low-dimensional planes to high-dimensional space due to occlusion and invisibility, respectively. Furthermore, existing 3D human assets remain small-scale, insufficient for large-scale training. To address these challenges, we propose a latent space generation paradigm for 3D human digitization, which involves compressing multi-view images into Gaussians via a UV-structured VAE, along with DiT-based conditional generation, we transform the ill-posed low-to-high-dimensional mapping problem into a learnable distribution shift, which also supports end-to-end inference. In addition, we employ the multi-view optimization approach combined with synthetic data to construct the HGS-1M dataset, which contains $1$ million 3D Gaussian assets to support the large-scale training. Experimental results demonstrate that our paradigm, powered by large-scale training, produces high-quality 3D human Gaussians with intricate textures, facial details, and loose clothing deformation.
• 摘要：3D人类数字化长期以来一直是一项高度追求但又具有挑战性的任务。现有的方法旨在从单个或多种视图中产生高质量的3D数字人类，但仍主要受到当前范式和3D人类资产的稀缺性的限制。具体而言，最近的方法属于几种范式：基于优化和进料（均具有重建的单视回归和多视图生成）。但是，它们受到慢速，低质量，级联推理和模棱两可的限制，分别由于遮挡和隐形性，将低维平面映射到高维空间。此外，现有的3D人力资产仍然很小，不足以进行大规模培训。为了应对这些挑战，我们提出了3D人类数字化的潜在太空生成范式，涉及通过紫外线结构的VAE将多视图图像压缩到高斯，以及基于DIT的有条件一代，我们将不足的低维度映射问题转变为可学习的分配转移，这也支持最终的质地。此外，我们采用多视图优化方法与合成数据结合使用，构建HGS-1M数据集，该数据集包含100万美元的3D高斯资产来支持大规模培训。实验结果表明，我们的范式由大规模训练提供动力，产生具有复杂质地，面部细节和宽松衣服变形的高质量的3D人类高斯人。

Title: Enhancing Metabolic Syndrome Prediction with Hybrid Data Balancing and Counterfactuals

• Authors: Sanyam Paresh Shah, Abdullah Mamun, Shovito Barua Soumma, Hassan Ghasemzadeh
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.06987
• Pdf URL: https://arxiv.org/pdf/2504.06987
• Copy Paste: [[2504.06987]] Enhancing Metabolic Syndrome Prediction with Hybrid Data Balancing and Counterfactuals(https://arxiv.org/abs/2504.06987)
• Keywords: generation
• Abstract: Metabolic Syndrome (MetS) is a cluster of interrelated risk factors that significantly increases the risk of cardiovascular diseases and type 2 diabetes. Despite its global prevalence, accurate prediction of MetS remains challenging due to issues such as class imbalance, data scarcity, and methodological inconsistencies in existing studies. In this paper, we address these challenges by systematically evaluating and optimizing machine learning (ML) models for MetS prediction, leveraging advanced data balancing techniques and counterfactual analysis. Multiple ML models, including XGBoost, Random Forest, TabNet, etc., were trained and compared under various data balancing techniques such as random oversampling (ROS), SMOTE, ADASYN, and CTGAN. Additionally, we introduce MetaBoost, a novel hybrid framework that integrates SMOTE, ADASYN, and CTGAN, optimizing synthetic data generation through weighted averaging and iterative weight tuning to enhance the model's performance (achieving a 1.14% accuracy improvement over individual balancing techniques). A comprehensive counterfactual analysis is conducted to quantify feature-level changes required to shift individuals from high-risk to low-risk categories. The results indicate that blood glucose (50.3%) and triglycerides (46.7%) were the most frequently modified features, highlighting their clinical significance in MetS risk reduction. Additionally, probabilistic analysis shows elevated blood glucose (85.5% likelihood) and triglycerides (74.9% posterior probability) as the strongest predictors. This study not only advances the methodological rigor of MetS prediction but also provides actionable insights for clinicians and researchers, highlighting the potential of ML in mitigating the public health burden of metabolic syndrome.
• 摘要：代谢综合征（METS）是一个相互关联的风险因素群，可显着增加心血管疾病和2型糖尿病的风险。尽管其全球流行率，但由于诸如阶级失衡，数据稀缺性和方法论上不一致之类的问题，对Mets的准确预测仍然具有挑战性。在本文中，我们通过系统地评估和优化METS预测的机器学习（ML）模型来解决这些挑战，利用高级数据平衡技术和反事实分析。在各种数据平衡技术（例如随机过采样（ROS），Smote），Smote，Adasyn和Ctgan（包括XGBoost，随机森林，Tabnet等）在内的多种ML模型进行了培训和比较。此外，我们介绍了一种新型的混合框架，该框架整合了Smote，Adasyn和Ctgan，通过加权平均和迭代权重调整来优化合成数据生成，以增强模型的性能（在单个平衡技术中获得1.14％的精度提高了1.14％的精度）。进行了全面的反事实分析，以量化个人从高风险转向低风险类别所需的特征级别的变化。结果表明，血糖（50.3％）和甘油三酸酯（46.7％）是最经常修改的特征，突出了它们在METS风险降低中的临床意义。此外，概率分析表明，血糖升高（可能性为85.5％）和甘油三酸酯（74.9％后概率）是最强的预测因子。这项研究不仅提高了Mets预测的方法论，而且还为临床医生和研究人员提供了可行的见解，强调了ML在减轻代谢综合征的公共卫生负担方面的潜力。

Title: A Unified Agentic Framework for Evaluating Conditional Image Generation

• Authors: Jifang Wang, Xue Yang, Longyue Wang, Zhenran Xu, Yiyu Wang, Yaowei Wang, Weihua Luo, Kaifu Zhang, Baotian Hu, Min Zhang
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2504.07046
• Pdf URL: https://arxiv.org/pdf/2504.07046
• Copy Paste: [[2504.07046]] A Unified Agentic Framework for Evaluating Conditional Image Generation(https://arxiv.org/abs/2504.07046)
• Keywords: generation
• Abstract: Conditional image generation has gained significant attention for its ability to personalize content. However, the field faces challenges in developing task-agnostic, reliable, and explainable evaluation metrics. This paper introduces CIGEval, a unified agentic framework for comprehensive evaluation of conditional image generation tasks. CIGEval utilizes large multimodal models (LMMs) as its core, integrating a multi-functional toolbox and establishing a fine-grained evaluation framework. Additionally, we synthesize evaluation trajectories for fine-tuning, empowering smaller LMMs to autonomously select appropriate tools and conduct nuanced analyses based on tool outputs. Experiments across seven prominent conditional image generation tasks demonstrate that CIGEval (GPT-4o version) achieves a high correlation of 0.4625 with human assessments, closely matching the inter-annotator correlation of 0.47. Moreover, when implemented with 7B open-source LMMs using only 2.3K training trajectories, CIGEval surpasses the previous GPT-4o-based state-of-the-art method. Case studies on GPT-4o image generation highlight CIGEval's capability in identifying subtle issues related to subject consistency and adherence to control guidance, indicating its great potential for automating evaluation of image generation tasks with human-level reliability.
• 摘要：有条件的图像产生因其个性化内容的能力而引起了极大的关注。但是，该领域在开发任务不合时宜，可靠且可解释的评估指标方面面临挑战。本文介绍了Cigeval，这是一个统一的代理框架，用于全面评估有条件的图像生成任务。 CIGEVAL利用大型多模型（LMM）作为其核心，集成了多功能工具箱并建立了细粒度的评估框架。此外，我们合成评估轨迹进行微调，赋予较小的LMMS自主选择适当的工具并根据工具输出进行细微的分析。七个突出的条件图像生成任务进行的实验表明，CIGEVAL（GPT-4O版本）与人类评估的高度相关性高度为0.4625，与0.47的通道间相关性紧密匹配。此外，当仅使用2.3k训练轨迹使用7B开源LMMS实施时，Cigeval超过了以前的基于GPT-4O的最新方法。 GPT-4O图像生成的案例研究突出了Cigeval在确定与主体一致性和依从性控制指导相关的微妙问题方面的能力，这表明其具有自动化评估图像生成任务的巨大潜力。

Title: To Backtrack or Not to Backtrack: When Sequential Search Limits Model Reasoning

• Authors: Tian Qin, David Alvarez-Melis, Samy Jelassi, Eran Malach
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.07052
• Pdf URL: https://arxiv.org/pdf/2504.07052
• Copy Paste: [[2504.07052]] To Backtrack or Not to Backtrack: When Sequential Search Limits Model Reasoning(https://arxiv.org/abs/2504.07052)
• Keywords: generation
• Abstract: Recent advancements in large language models have significantly improved their reasoning abilities, particularly through techniques involving search and backtracking. Backtracking naturally scales test-time compute by enabling sequential, linearized exploration via long chain-of-thought (CoT) generation. However, this is not the only strategy for scaling test-time compute: parallel sampling with best-of-n selection provides an alternative that generates diverse solutions simultaneously. Despite the growing adoption of sequential search, its advantages over parallel sampling--especially under a fixed compute budget remain poorly understood. In this paper, we systematically compare these two approaches on two challenging reasoning tasks: CountDown and Sudoku. Surprisingly, we find that sequential search underperforms parallel sampling on CountDown but outperforms it on Sudoku, suggesting that backtracking is not universally beneficial. We identify two factors that can cause backtracking to degrade performance: (1) training on fixed search traces can lock models into suboptimal strategies, and (2) explicit CoT supervision can discourage "implicit" (non-verbalized) reasoning. Extending our analysis to reinforcement learning (RL), we show that models with backtracking capabilities benefit significantly from RL fine-tuning, while models without backtracking see limited, mixed gains. Together, these findings challenge the assumption that backtracking universally enhances LLM reasoning, instead revealing a complex interaction between task structure, training data, model scale, and learning paradigm.
• 摘要：大型语言模型的最新进展已大大提高了其推理能力，尤其是通过涉及搜索和回溯技术的技术。通过长链（COT）生成启用顺序的线性探索，自然缩放测试时间计算。但是，这并不是缩放测试时间计算的唯一策略：与最佳N选择的并行抽样提供了一种同时生成多种解决方案的替代方案。尽管顺序搜索的采用越来越大，但其优势比平行采样的优势 - 尤其是在固定的计算预算下，人们的理解仍然很少。在本文中，我们会系统地比较有关两种具有挑战性的推理任务的这两种方法：倒计时和Sudoku。令人惊讶的是，我们发现顺式搜索表现不佳在倒计时的平行采样，但在Sudoku上的表现优于它，这表明回溯并不是普遍有益的。我们确定了可能导致回溯以降低性能的两个因素：（1）固定搜索轨迹的培训可以将模型锁定为次优策略，并且（2）明确的COT监督可以阻止“隐式”（非语言）推理。将我们的分析扩展到增强学习（RL），我们表明具有回溯功能的模型从RL微调中受益匪浅，而没有回溯的模型请参见有限的混合收益。这些发现共同挑战了以下假设：回溯性普遍增强LLM推理，而是揭示了任务结构，训练数据，模型量表和学习范式之间的复杂相互作用。

Title: Detecting AI-generated Artwork

• Authors: Meien Li, Mark Stamp
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.07078
• Pdf URL: https://arxiv.org/pdf/2504.07078
• Copy Paste: [[2504.07078]] Detecting AI-generated Artwork(https://arxiv.org/abs/2504.07078)
• Keywords: generative
• Abstract: The high efficiency and quality of artwork generated by Artificial Intelligence (AI) has created new concerns and challenges for human artists. In particular, recent improvements in generative AI have made it difficult for people to distinguish between human-generated and AI-generated art. In this research, we consider the potential utility of various types of Machine Learning (ML) and Deep Learning (DL) models in distinguishing AI-generated artwork from human-generated artwork. We focus on three challenging artistic styles, namely, baroque, cubism, and expressionism. The learning models we test are Logistic Regression (LR), Support Vector Machine (SVM), Multilayer Perceptron (MLP), and Convolutional Neural Network (CNN). Our best experimental results yield a multiclass accuracy of 0.8208 over six classes, and an impressive accuracy of 0.9758 for the binary classification problem of distinguishing AI-generated from human-generated art.
• 摘要：人工智能（AI）产生的高效和质量为人类艺术家带来了新的关注和挑战。特别是，最近的生成AI的改进使人们难以区分人类生成和AI生成的艺术。在这项研究中，我们考虑了各种类型的机器学习（ML）和深度学习（DL）模型的潜在效用，以区分AI生成的艺术品与人类生成的艺术品。我们专注于三种具有挑战性的艺术风格，即巴洛克式，立体主义和表现主义。我们测试的学习模型是逻辑回归（LR），支持向量机（SVM），多层感知器（MLP）和卷积神经网络（CNN）。我们的最佳实验结果在六个类别上产生了0.8208的多类精确度，对于将AI生成的AI与人类生成的ART区分开的二进制分类问题的令人印象深刻的精度为0.9758。

Title: GenDoP: Auto-regressive Camera Trajectory Generation as a Director of Photography

• Authors: Mengchen Zhang, Tong Wu, Jing Tan, Ziwei Liu, Gordon Wetzstein, Dahua Lin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.07083
• Pdf URL: https://arxiv.org/pdf/2504.07083
• Copy Paste: [[2504.07083]] GenDoP: Auto-regressive Camera Trajectory Generation as a Director of Photography(https://arxiv.org/abs/2504.07083)
• Keywords: generation
• Abstract: Camera trajectory design plays a crucial role in video production, serving as a fundamental tool for conveying directorial intent and enhancing visual storytelling. In cinematography, Directors of Photography meticulously craft camera movements to achieve expressive and intentional framing. However, existing methods for camera trajectory generation remain limited: Traditional approaches rely on geometric optimization or handcrafted procedural systems, while recent learning-based methods often inherit structural biases or lack textual alignment, constraining creative synthesis. In this work, we introduce an auto-regressive model inspired by the expertise of Directors of Photography to generate artistic and expressive camera trajectories. We first introduce DataDoP, a large-scale multi-modal dataset containing 29K real-world shots with free-moving camera trajectories, depth maps, and detailed captions in specific movements, interaction with the scene, and directorial intent. Thanks to the comprehensive and diverse database, we further train an auto-regressive, decoder-only Transformer for high-quality, context-aware camera movement generation based on text guidance and RGBD inputs, named GenDoP. Extensive experiments demonstrate that compared to existing methods, GenDoP offers better controllability, finer-grained trajectory adjustments, and higher motion stability. We believe our approach establishes a new standard for learning-based cinematography, paving the way for future advancements in camera control and filmmaking. Our project website: this https URL.
• 摘要：相机轨迹设计在视频制作中起着至关重要的作用，它是传达导演意图和增强视觉讲故事的基本工具。在摄影中，精心制作的摄影摄像机运动的导演可以实现表达和故意的框架。但是，现有的相机轨迹生成方法仍然有限：传统方法依赖于几何优化或手工制作的程序系统，而最近的基于学习的方法通常继承了结构偏见或缺乏文本一致性，从而限制了创造性的综合。在这项工作中，我们介绍了一种自动回归模型，该模型灵感来自摄影导演的专业知识，以产生艺术和表现力的相机轨迹。我们首先介绍DataDop，这是一种大型多模式数据集，其中包含29k现实世界的照片，带有自由移动的摄像头轨迹，深度图和特定动作中的详细标题，与场景的交互以及导演意图。多亏了全面和多样化的数据库，我们进一步培训了一种自动回归，仅解码器的变压器，用于基于文本指南和RGBD输入的高质量，上下文醒目的相机运动生成，名为Gendop。广泛的实验表明，与现有方法相比，Gendop提供了更好的可控性，更细粒度的轨迹调整和更高的运动稳定性。我们认为，我们的方法为基于学习的摄影建立了新的标准，为未来的摄像机控制和电影制作铺平了道路。我们的项目网站：此HTTPS URL。

Title: Identifying Unknown Stochastic Dynamics via Finite expression methods

• Authors: Senwei Liang, Chunmei Wang, Xingjian Xu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.07085
• Pdf URL: https://arxiv.org/pdf/2504.07085
• Copy Paste: [[2504.07085]] Identifying Unknown Stochastic Dynamics via Finite expression methods(https://arxiv.org/abs/2504.07085)
• Keywords: generative
• Abstract: Modeling stochastic differential equations (SDEs) is crucial for understanding complex dynamical systems in various scientific fields. Recent methods often employ neural network-based models, which typically represent SDEs through a combination of deterministic and stochastic terms. However, these models usually lack interpretability and have difficulty generalizing beyond their training domain. This paper introduces the Finite Expression Method (FEX), a symbolic learning approach designed to derive interpretable mathematical representations of the deterministic component of SDEs. For the stochastic component, we integrate FEX with advanced generative modeling techniques to provide a comprehensive representation of SDEs. The numerical experiments on linear, nonlinear, and multidimensional SDEs demonstrate that FEX generalizes well beyond the training domain and delivers more accurate long-term predictions compared to neural network-based methods. The symbolic expressions identified by FEX not only improve prediction accuracy but also offer valuable scientific insights into the underlying dynamics of the systems, paving the way for new scientific discoveries.
• 摘要：建模随机微分方程（SDE）对于理解各种科学领域的复杂动力系统至关重要。最近的方法通常采用基于神经网络的模型，该模型通常通过确定性和随机术语的结合来代表SDE。但是，这些模型通常缺乏可解释性，并且难以推广其训练领域。本文介绍了有限表达方法（FEX），这是一种符号学习方法，旨在得出SDES确定性组成部分的可解释数学表示。对于随机组件，我们将FEX与先进的生成建模技术集成在一起，以提供SDE的全面表示。在线性，非线性和多维SDE上进行的数值实验表明，与基于神经网络的方法相比，FEX超出了训练域，并提供了更准确的长期预测。 FEX确定的符号表达式不仅提高了预测准确性，而且还为系统的潜在动态提供了宝贵的科学见解，为新的科学发现铺平了道路。

Title: OmniCaptioner: One Captioner to Rule Them All

• Authors: Yiting Lu, Jiakang Yuan, Zhen Li, Shitian Zhao, Qi Qin, Xinyue Li, Le Zhuo, Licheng Wen, Dongyang Liu, Yuewen Cao, Xiangchao Yan, Xin Li, Botian Shi, Tao Chen, Zhibo Chen, Lei Bai, Bo Zhang, Peng Gao
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2504.07089
• Pdf URL: https://arxiv.org/pdf/2504.07089
• Copy Paste: [[2504.07089]] OmniCaptioner: One Captioner to Rule Them All(https://arxiv.org/abs/2504.07089)
• Keywords: generation
• Abstract: We propose OmniCaptioner, a versatile visual captioning framework for generating fine-grained textual descriptions across a wide variety of visual domains. Unlike prior methods limited to specific image types (e.g., natural images or geometric visuals), our framework provides a unified solution for captioning natural images, visual text (e.g., posters, UIs, textbooks), and structured visuals (e.g., documents, tables, charts). By converting low-level pixel information into semantically rich textual representations, our framework bridges the gap between visual and textual modalities. Our results highlight three key advantages: (i) Enhanced Visual Reasoning with LLMs, where long-context captions of visual modalities empower LLMs, particularly the DeepSeek-R1 series, to reason effectively in multimodal scenarios; (ii) Improved Image Generation, where detailed captions improve tasks like text-to-image generation and image transformation; and (iii) Efficient Supervised Fine-Tuning (SFT), which enables faster convergence with less data. We believe the versatility and adaptability of OmniCaptioner can offer a new perspective for bridging the gap between language and visual modalities.
• 摘要：我们提出了Omnicaptioner，这是一个多功能的视觉字幕框架，用于在各种视觉域中生成细粒度的文本描述。与限于特定图像类型（例如自然图像或几何视觉效果）的先前方法不同，我们的框架为字幕自然图像，视觉文本（例如，海报，UIS，教科书）和结构化视觉效果（例如文档，表，表格，图表，图表）提供了统一的解决方案。通过将低级像素信息转换为语义上丰富的文本表示形式，我们的框架弥合了视觉和文本方式之间的差距。我们的结果突出了三个关键优势：（i）通过LLMS增强视觉推理，其中的长篇文章字幕赋予LLMS，尤其是DeepSeek-R1系列，以在多模式的情况下有效地推理； （ii）改进的图像生成，其中详细的标题改善了文本对图像生成和图像转换等任务； （iii）有效监督的微调（SFT），可以更快地收敛。我们认为，Omnicaptioner的多功能性和适应性可以为弥合语言和视觉方式之间的差距提供新的观点。

Title: Sculpting Subspaces: Constrained Full Fine-Tuning in LLMs for Continual Learning

• Authors: Nikhil Shivakumar Nayak, Krishnateja Killamsetty, Ligong Han, Abhishek Bhandwaldar, Prateek Chanda, Kai Xu, Hao Wang, Aldo Pareja, Oleg Silkin, Mustafa Eyceoz, Akash Srivastava
• Subjects: cs.LG, cs.AI, cs.CL, math.PR, stat.ML
• Abstract URL: https://arxiv.org/abs/2504.07097
• Pdf URL: https://arxiv.org/pdf/2504.07097
• Copy Paste: [[2504.07097]] Sculpting Subspaces: Constrained Full Fine-Tuning in LLMs for Continual Learning(https://arxiv.org/abs/2504.07097)
• Keywords: generation
• Abstract: Continual learning in large language models (LLMs) is prone to catastrophic forgetting, where adapting to new tasks significantly degrades performance on previously learned ones. Existing methods typically rely on low-rank, parameter-efficient updates that limit the model's expressivity and introduce additional parameters per task, leading to scalability issues. To address these limitations, we propose a novel continual full fine-tuning approach leveraging adaptive singular value decomposition (SVD). Our method dynamically identifies task-specific low-rank parameter subspaces and constrains updates to be orthogonal to critical directions associated with prior tasks, thus effectively minimizing interference without additional parameter overhead or storing previous task gradients. We evaluate our approach extensively on standard continual learning benchmarks using both encoder-decoder (T5-Large) and decoder-only (LLaMA-2 7B) models, spanning diverse tasks including classification, generation, and reasoning. Empirically, our method achieves state-of-the-art results, up to 7% higher average accuracy than recent baselines like O-LoRA, and notably maintains the model's general linguistic capabilities, instruction-following accuracy, and safety throughout the continual learning process by reducing forgetting to near-negligible levels. Our adaptive SVD framework effectively balances model plasticity and knowledge retention, providing a practical, theoretically grounded, and computationally scalable solution for continual learning scenarios in large language models.
• 摘要：在大型语言模型（LLM）中的持续学习容易遭受灾难性的遗忘，在这些遗忘中，适应新任务会大大降低以前学到的任务的表现。现有方法通常依赖于限制模型表现力并每个任务引入其他参数的低级参数效率更新，从而导致可扩展性问题。为了解决这些局限性，我们提出了一种新型的持续全面微调方法，利用适应性奇异价值分解（SVD）。我们的方法动态识别特定任务的低级别参数子空间，并将更新与与先前任务相关的关键方向进行正交，从而有效地最大程度地减少干扰而没有其他参数开销或存储先前的任务梯度。我们使用编码器编码器（T5-Large）和仅解码器（Llama-2 7b）模型对标准持续学习基准进行了广泛评估我们的方法，涵盖了各种任务，包括分类，发电和推理。从经验上讲，我们的方法可实现最先进的结果，比最近的基线（如O-Lora）高出7％的平均准确性，并且在整个持续学习过程中，通过降低忘记近乎不可识别的水平，尤其是该模型的一般语言能力，指导遵循的准确性和安全性。我们的自适应SVD框架有效地平衡了模型的可塑性和知识保留，从而为大型语言模型中的连续学习场景提供了实用的，理论上的基础和计算可扩展的解决方案。