2026-02-02

Title: Attention Isn't All You Need for Emotion Recognition:Domain Features Outperform Transformers on the EAV Dataset

• Authors: Anmol Guragain
• Subjects: cs.LG, cs.CV, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2601.22161
• Pdf URL: https://arxiv.org/pdf/2601.22161
• Copy Paste: [[2601.22161]] Attention Isn't All You Need for Emotion Recognition:Domain Features Outperform Transformers on the EAV Dataset(https://arxiv.org/abs/2601.22161)
• Keywords: transformer
• Abstract: We present a systematic study of multimodal emotion recognition using the EAV dataset, investigating whether complex attention mechanisms improve performance on small datasets. We implement three model categories: baseline transformers (M1), novel factorized attention mechanisms (M2), and improved CNN baselines (M3). Our experiments show that sophisticated attention mechanisms consistently underperform on small datasets. M2 models achieved 5 to 13 percentage points below baselines due to overfitting and destruction of pretrained features. In contrast, simple domain-appropriate modifications proved effective: adding delta MFCCs to the audio CNN improved accuracy from 61.9\% to \textbf{65.56\%} (+3.66pp), while frequency-domain features for EEG achieved \textbf{67.62\%} (+7.62pp over the paper baseline). Our vision transformer baseline (M1) reached \textbf{75.30\%}, exceeding the paper's ViViT result (74.5\%) through domain-specific pretraining, and vision delta features achieved \textbf{72.68\%} (+1.28pp over the paper CNN). These findings demonstrate that for small-scale emotion recognition, domain knowledge and proper implementation outperform architectural complexity.

Title: Do Open-Vocabulary Detectors Transfer to Aerial Imagery? A Comparative Evaluation

• Authors: Christos Tsourveloudis
• Subjects: cs.CV, cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2601.22164
• Pdf URL: https://arxiv.org/pdf/2601.22164
• Copy Paste: [[2601.22164]] Do Open-Vocabulary Detectors Transfer to Aerial Imagery? A Comparative Evaluation(https://arxiv.org/abs/2601.22164)
• Keywords: fair
• Abstract: Open-vocabulary object detection (OVD) enables zero-shot recognition of novel categories through vision-language models, achieving strong performance on natural images. However, transferability to aerial imagery remains unexplored. We present the first systematic benchmark evaluating five state-of-the-art OVD models on the LAE-80C aerial dataset (3,592 images, 80 categories) under strict zero-shot conditions. Our experimental protocol isolates semantic confusion from visual localization through Global, Oracle, and Single-Category inference modes. Results reveal severe domain transfer failure: the best model (OWLv2) achieves only 27.6% F1-score with 69% false positive rate. Critically, reducing vocabulary size from 80 to 3.2 classes yields 15x improvement, demonstrating that semantic confusion is the primary bottleneck. Prompt engineering strategies such as domain-specific prefixing and synonym expansion, fail to provide meaningful performance gains. Performance varies dramatically across datasets (F1: 0.53 on DIOR, 0.12 on FAIR1M), exposing brittleness to imaging conditions. These findings establish baseline expectations and highlight the need for domain-adaptive approaches in aerial OVD.

Title: In Vino Veritas and Vulnerabilities: Examining LLM Safety via Drunk Language Inducement

• Authors: Anudeex Shetty, Aditya Joshi, Salil S. Kanhere
• Subjects: cs.CL, cs.AI, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22169
• Pdf URL: https://arxiv.org/pdf/2601.22169
• Copy Paste: [[2601.22169]] In Vino Veritas and Vulnerabilities: Examining LLM Safety via Drunk Language Inducement(https://arxiv.org/abs/2601.22169)
• Keywords: privacy, robust, large language model
• Abstract: Humans are susceptible to undesirable behaviours and privacy leaks under the influence of alcohol. This paper investigates drunk language, i.e., text written under the influence of alcohol, as a driver for safety failures in large language models (LLMs). We investigate three mechanisms for inducing drunk language in LLMs: persona-based prompting, causal fine-tuning, and reinforcement-based post-training. When evaluated on 5 LLMs, we observe a higher susceptibility to jailbreaking on JailbreakBench (even in the presence of defences) and privacy leaks on ConfAIde, where both benchmarks are in English, as compared to the base LLMs as well as previously reported approaches. Via a robust combination of manual evaluation and LLM-based evaluators and analysis of error categories, our findings highlight a correspondence between human-intoxicated behaviour, and anthropomorphism in LLMs induced with drunk language. The simplicity and efficiency of our drunk language inducement approaches position them as potential counters for LLM safety tuning, highlighting significant risks to LLM safety.

Title: ShellForge: Adversarial Co-Evolution of Webshell Generation and Multi-View Detection for Robust Webshell Defense

• Authors: Yizhong Ding
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22182
• Pdf URL: https://arxiv.org/pdf/2601.22182
• Copy Paste: [[2601.22182]] ShellForge: Adversarial Co-Evolution of Webshell Generation and Multi-View Detection for Robust Webshell Defense(https://arxiv.org/abs/2601.22182)
• Keywords: protect, defense, attack, robust
• Abstract: Webshells remain a primary foothold for attackers to compromise servers, particularly within PHP ecosystems. However, existing detection mechanisms often struggle to keep pace with rapid variant evolution and sophisticated obfuscation techniques that camouflage malicious intent. Furthermore, many current defenses suffer from high false-alarm rates when encountering benign administrative scripts that employ heavy obfuscation for intellectual property protection. To address these challenges, we present ShellForge, an adversarial co-evolution framework that couples automated webshell generation with multi-view detection to continuously harden defensive boundaries. The framework operates through an iterative co-training loop where a generator and a detector mutually reinforce each other via the exchange of hard samples. The generator is optimized through supervised fine-tuning and preference-based reinforcement learning to synthesize functional, highly evasive variants. Simultaneously, we develop a multi-view fusion detector that integrates semantic features from long-string compression, structural features from pruned abstract syntax trees, and global statistical indicators such as Shannon entropy. To minimize false positives, ShellForge utilizes a LLM-based transformation to create de-malicious samples--scripts that retain complex obfuscation patterns but lack harmful payloads--serving as high-quality hard negatives during training. Evaluations on the public FWOID benchmark demonstrate that ShellForge significantly enhances defensive robustness. Upon convergence, the detector maintains a 0.981 F1-score while the generator achieves a 0.939 evasion rate against commercial engines on VirusTotal.

Title: FedAdaVR: Adaptive Variance Reduction for Robust Federated Learning under Limited Client Participation

• Authors: S M Ruhul Kabir Howlader, Xiao Chen, Yifei Xie, Lu Liu
• Subjects: cs.LG, cs.DC
• Abstract URL: https://arxiv.org/abs/2601.22204
• Pdf URL: https://arxiv.org/pdf/2601.22204
• Copy Paste: [[2601.22204]] FedAdaVR: Adaptive Variance Reduction for Robust Federated Learning under Limited Client Participation(https://arxiv.org/abs/2601.22204)
• Keywords: robust, federate
• Abstract: Federated learning (FL) encounters substantial challenges due to heterogeneity, leading to gradient noise, client drift, and partial client participation errors, the last of which is the most pervasive but remains insufficiently addressed in current literature. In this paper, we propose FedAdaVR, a novel FL algorithm aimed at solving heterogeneity issues caused by sporadic client participation by incorporating an adaptive optimiser with a variance reduction technique. This method takes advantage of the most recent stored updates from clients, even when they are absent from the current training round, thereby emulating their presence. Furthermore, we propose FedAdaVR-Quant, which stores client updates in quantised form, significantly reducing the memory requirements (by 50%, 75%, and 87.5%) of FedAdaVR while maintaining equivalent model performance. We analyse the convergence behaviour of FedAdaVR under general nonconvex conditions and prove that our proposed algorithm can eliminate partial client participation error. Extensive experiments conducted on multiple datasets, under both independent and identically distributed (IID) and non-IID settings, demonstrate that FedAdaVR consistently outperforms state-of-the-art baseline methods.

Title: Causal Imitation Learning Under Measurement Error and Distribution Shift

• Authors: Shi Bo, AmirEmad Ghassami
• Subjects: cs.LG, stat.ME, stat.ML
• Abstract URL: https://arxiv.org/abs/2601.22206
• Pdf URL: https://arxiv.org/pdf/2601.22206
• Copy Paste: [[2601.22206]] Causal Imitation Learning Under Measurement Error and Distribution Shift(https://arxiv.org/abs/2601.22206)
• Keywords: robust
• Abstract: We study offline imitation learning (IL) when part of the decision-relevant state is observed only through noisy measurements and the distribution may change between training and deployment. Such settings induce spurious state-action correlations, so standard behavioral cloning (BC) -- whether conditioning on raw measurements or ignoring them -- can converge to systematically biased policies under distribution shift. We propose a general framework for IL under measurement error, inspired by explicitly modeling the causal relationships among the variables, yielding a target that retains a causal interpretation and is robust to distribution shift. Building on ideas from proximal causal inference, we introduce \texttt{CausIL}, which treats noisy state observations as proxy variables, and we provide identification conditions under which the target policy is recoverable from demonstrations without rewards or interactive expert queries. We develop estimators for both discrete and continuous state spaces; for continuous settings, we use an adversarial procedure over RKHS function classes to learn the required parameters. We evaluate \texttt{CausIL} on semi-simulated longitudinal data from the PhysioNet/Computing in Cardiology Challenge 2019 cohort and demonstrate improved robustness to distribution shift compared to BC baselines.

Title: Latent Spherical Flow Policy for Reinforcement Learning with Combinatorial Actions

• Authors: Lingkai Kong, Anagha Satish, Hezi Jiang, Akseli Kangaslahti, Andrew Ma, Wenbo Chen, Mingxiao Song, Lily Xu, Milind Tambe
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22211
• Pdf URL: https://arxiv.org/pdf/2601.22211
• Copy Paste: [[2601.22211]] Latent Spherical Flow Policy for Reinforcement Learning with Combinatorial Actions(https://arxiv.org/abs/2601.22211)
• Keywords: generative
• Abstract: Reinforcement learning (RL) with combinatorial action spaces remains challenging because feasible action sets are exponentially large and governed by complex feasibility constraints, making direct policy parameterization impractical. Existing approaches embed task-specific value functions into constrained optimization programs or learn deterministic structured policies, sacrificing generality and policy expressiveness. We propose a solver-induced \emph{latent spherical flow policy} that brings the expressiveness of modern generative policies to combinatorial RL while guaranteeing feasibility by design. Our method, LSFlow, learns a \emph{stochastic} policy in a compact continuous latent space via spherical flow matching, and delegates feasibility to a combinatorial optimization solver that maps each latent sample to a valid structured action. To improve efficiency, we train the value network directly in the latent space, avoiding repeated solver calls during policy optimization. To address the piecewise-constant and discontinuous value landscape induced by solver-based action selection, we introduce a smoothed Bellman operator that yields stable, well-defined learning targets. Empirically, our approach outperforms state-of-the-art baselines by an average of 20.6\% across a range of challenging combinatorial RL tasks.

Title: Geometry without Position? When Positional Embeddings Help and Hurt Spatial Reasoning

• Authors: Jian Shi, Michael Birsak, Wenqing Cui, Zhenyu Li, Peter Wonka
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22231
• Pdf URL: https://arxiv.org/pdf/2601.22231
• Copy Paste: [[2601.22231]] Geometry without Position? When Positional Embeddings Help and Hurt Spatial Reasoning(https://arxiv.org/abs/2601.22231)
• Keywords: transformer
• Abstract: This paper revisits the role of positional embeddings (PEs) within vision transformers (ViTs) from a geometric perspective. We show that PEs are not mere token indices but effectively function as geometric priors that shape the spatial structure of the representation. We introduce token-level diagnostics that measure how multi-view geometric consistency in ViT representation depends on consitent PEs. Through extensive experiments on 14 foundation ViT models, we reveal how PEs influence multi-view geometry and spatial reasoning. Our findings clarify the role of PEs as a causal mechanism that governs spatial structure in ViT representations. Our code is provided in this https URL

Title: A Systematic Literature Review on LLM Defenses Against Prompt Injection and Jailbreaking: Expanding NIST Taxonomy

• Authors: Pedro H. Barcha Correia, Ryan W. Achjian, Diego E. G. Caetano de Oliveira, Ygor Acacio Maria, Victor Takashi Hayashi, Marcos Lopes, Charles Christian Miers, Marcos A. Simplicio Jr
• Subjects: cs.CR, cs.AI, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22240
• Pdf URL: https://arxiv.org/pdf/2601.22240
• Copy Paste: [[2601.22240]] A Systematic Literature Review on LLM Defenses Against Prompt Injection and Jailbreaking: Expanding NIST Taxonomy(https://arxiv.org/abs/2601.22240)
• Keywords: security, defense, attack, generative, large language model
• Abstract: The rapid advancement and widespread adoption of generative artificial intelligence (GenAI) and large language models (LLMs) has been accompanied by the emergence of new security vulnerabilities and challenges, such as jailbreaking and other prompt injection attacks. These maliciously crafted inputs can exploit LLMs, causing data leaks, unauthorized actions, or compromised outputs, for instance. As both offensive and defensive prompt injection techniques evolve quickly, a structured understanding of mitigation strategies becomes increasingly important. To address that, this work presents the first systematic literature review on prompt injection mitigation strategies, comprehending 88 studies. Building upon NIST's report on adversarial machine learning, this work contributes to the field through several avenues. First, it identifies studies beyond those documented in NIST's report and other academic reviews and surveys. Second, we propose an extension to NIST taxonomy by introducing additional categories of defenses. Third, by adopting NIST's established terminology and taxonomy as a foundation, we promote consistency and enable future researchers to build upon the standardized taxonomy proposed in this work. Finally, we provide a comprehensive catalog of the reviewed prompt injection defenses, documenting their reported quantitative effectiveness across specific LLMs and attack datasets, while also indicating which solutions are open-source and model-agnostic. This catalog, together with the guidelines presented herein, aims to serve as a practical resource for researchers advancing the field of adversarial machine learning and for developers seeking to implement effective defenses in production systems.

Title: Is Hierarchical Quantization Essential for Optimal Reconstruction?

• Authors: Shirin Reyhanian, Laurenz Wiskott
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22244
• Pdf URL: https://arxiv.org/pdf/2601.22244
• Copy Paste: [[2601.22244]] Is Hierarchical Quantization Essential for Optimal Reconstruction?(https://arxiv.org/abs/2601.22244)
• Keywords: generative
• Abstract: Vector-quantized variational autoencoders (VQ-VAEs) are central to models that rely on high reconstruction fidelity, from neural compression to generative pipelines. Hierarchical extensions, such as VQ-VAE2, are often credited with superior reconstruction performance because they split global and local features across multiple levels. However, since higher levels derive all their information from lower levels, they should not carry additional reconstructive content beyond what the lower-level already encodes. Combined with recent advances in training objectives and quantization mechanisms, this leads us to ask whether a single-level VQ-VAE, with matched representational budget and no codebook collapse, can equal the reconstruction fidelity of its hierarchical counterpart. Although the multi-scale structure of hierarchical models may improve perceptual quality in downstream tasks, the effect of hierarchy on reconstruction accuracy, isolated from codebook utilization and overall representational capacity, remains empirically underexamined. We revisit this question by comparing a two-level VQ-VAE and a capacity-matched single-level model on high-resolution ImageNet images. Consistent with prior observations, we confirm that inadequate codebook utilization limits single-level VQ-VAEs and that overly high-dimensional embeddings destabilize quantization and increase codebook collapse. We show that lightweight interventions such as initialization from data, periodic reset of inactive codebook vectors, and systematic tuning of codebook hyperparameters significantly reduce collapse. Our results demonstrate that when representational budgets are matched, and codebook collapse is mitigated, single-level VQ-VAEs can match the reconstruction fidelity of hierarchical variants, challenging the assumption that hierarchical quantization is inherently superior for high-quality reconstructions.

Title: MirrorMark: A Distortion-Free Multi-Bit Watermark for Large Language Models

• Authors: Ya Jiang, Massieh Kordi Boroujeny, Surender Suresh Kumar, Kai Zeng
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22246
• Pdf URL: https://arxiv.org/pdf/2601.22246
• Copy Paste: [[2601.22246]] MirrorMark: A Distortion-Free Multi-Bit Watermark for Large Language Models(https://arxiv.org/abs/2601.22246)
• Keywords: robust, watermark, large language model
• Abstract: As large language models (LLMs) become integral to applications such as question answering and content creation, reliable content attribution has become increasingly important. Watermarking is a promising approach, but existing methods either provide only binary signals or distort the sampling distribution, degrading text quality; distortion-free approaches, in turn, often suffer from weak detectability or robustness. We propose MirrorMark, a multi-bit and distortion-free watermark for LLMs. By mirroring sampling randomness in a measure-preserving manner, MirrorMark embeds multi-bit messages without altering the token probability distribution, preserving text quality by design. To improve robustness, we introduce a context-based scheduler that balances token assignments across message positions while remaining resilient to insertions and deletions. We further provide a theoretical analysis of the equal error rate to interpret empirical performance. Experiments show that MirrorMark matches the text quality of non-watermarked generation while achieving substantially stronger detectability: with 54 bits embedded in 300 tokens, it improves bit accuracy by 8-12% and correctly identifies up to 11% more watermarked texts at 1% false positive rate.

Title: FunPRM: Function-as-Step Process Reward Model with Meta Reward Correction for Code Generation

• Authors: Ruiyi Zhang, Peijia Qin, Qi Cao, Eric Xue, Pengtao Xie
• Subjects: cs.LG, cs.SE
• Abstract URL: https://arxiv.org/abs/2601.22249
• Pdf URL: https://arxiv.org/pdf/2601.22249
• Copy Paste: [[2601.22249]] FunPRM: Function-as-Step Process Reward Model with Meta Reward Correction for Code Generation(https://arxiv.org/abs/2601.22249)
• Keywords: large language model
• Abstract: Code generation is a core application of large language models (LLMs), yet LLMs still frequently fail on complex programming tasks. Given its success in mathematical reasoning, test-time scaling approaches such as Process Reward Model (PRM)-based Best-of-N selection offer a promising way to improve performance. However, existing PRMs remain ineffective for code generation due to the lack of meaningful step decomposition in code and the noise of Monte Carlo-estimated partial-solution correctness scores (rewards). To address these challenges, we propose FunPRM. FunPRM prompts LLMs to encourage modular code generation organized into functions, with functions treated as PRM reasoning steps. Furthermore, FunPRM introduces a novel meta-learning-based reward correction mechanism that leverages clean final-solution rewards obtained via a unit-test-based evaluation system to purify noisy partial-solution rewards. Experiments on LiveCodeBench and BigCodeBench demonstrate that FunPRM consistently outperforms existing test-time scaling methods across five base LLMs, notably achieving state-of-the-art performance on LiveCodeBench when combined with O4-mini. Furthermore, FunPRM produces code that is more readable and reusable for developers.

Title: Symmetry Breaking in Transformers for Efficient and Interpretable Training

• Authors: Eva Silverstein, Daniel Kunin, Vasudev Shyam
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22257
• Pdf URL: https://arxiv.org/pdf/2601.22257
• Copy Paste: [[2601.22257]] Symmetry Breaking in Transformers for Efficient and Interpretable Training(https://arxiv.org/abs/2601.22257)
• Keywords: interpretability, transformer
• Abstract: The attention mechanism in its standard implementation contains extraneous rotational degrees of freedom that are carried through computation but do not affect model activations or outputs. We introduce a simple symmetry-breaking protocol that inserts a preferred direction into this rotational space through batchwise-sampled, unlearned query and value biases. This modification has two theoretically motivated and empirically validated consequences. First, it can substantially improve the performance of simple, memory-efficient optimizers, narrowing -- and in some cases closing -- the gap to successful but more complex memory-intensive adaptive methods. We demonstrate this by pretraining 124M parameter transformer models with four optimization algorithms (AdamW, SOAP, SGDM, and Energy Conserving Descent(ECD)) and evaluating both validation loss and downstream logical reasoning. Second, it enables an interpretable use of otherwise redundant rotational degrees of freedom, selectively amplifying semantically meaningful token classes within individual attention heads. Overall, our results show that minimal, principled architectural changes can simultaneously improve performance and interpretability.

Title: Privacy-Preserving Sensor-Based Human Activity Recognition for Low-Resource Healthcare Using Classical Machine Learning

• Authors: Ramakant Kumar, Pravin Kumar
• Subjects: cs.LG, cs.NI
• Abstract URL: https://arxiv.org/abs/2601.22265
• Pdf URL: https://arxiv.org/pdf/2601.22265
• Copy Paste: [[2601.22265]] Privacy-Preserving Sensor-Based Human Activity Recognition for Low-Resource Healthcare Using Classical Machine Learning(https://arxiv.org/abs/2601.22265)
• Keywords: privacy, robust
• Abstract: Limited access to medical infrastructure forces elderly and vulnerable patients to rely on home-based care, often leading to neglect and poor adherence to therapeutic exercises such as yoga or physiotherapy. To address this gap, we propose a low-cost and automated human activity recognition (HAR) framework based on wearable inertial sensors and machine learning. Activity data, including walking, walking upstairs, walking downstairs, sitting, standing, and lying, were collected using accelerometer and gyroscope measurements. Four classical classifiers, Logistic Regression, Random Forest, Support Vector Machine (SVM), and k-Nearest Neighbors (k-NN), were evaluated and compared with the proposed Support Tensor Machine (STM). Experimental results show that SVM achieved an accuracy of 93.33 percent, while Logistic Regression, Random Forest, and k-NN achieved 91.11 percent. In contrast, STM significantly outperformed these models, achieving a test accuracy of 96.67 percent and the highest cross-validation accuracy of 98.50 percent. Unlike conventional methods, STM leverages tensor representations to preserve spatio-temporal motion dynamics, resulting in robust classification across diverse activities. The proposed framework demonstrates strong potential for remote healthcare, elderly assistance, child activity monitoring, yoga feedback, and smart home wellness, offering a scalable solution for low-resource and rural healthcare settings.

Title: Task-Uniform Convergence and Backward Transfer in Federated Domain-Incremental Learning with Partial Participation

• Authors: Longtao Xu, Jian Li
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22274
• Pdf URL: https://arxiv.org/pdf/2601.22274
• Copy Paste: [[2601.22274]] Task-Uniform Convergence and Backward Transfer in Federated Domain-Incremental Learning with Partial Participation(https://arxiv.org/abs/2601.22274)
• Keywords: privacy, federate
• Abstract: Real-world federated systems seldom operate on static data: input distributions drift while privacy rules forbid raw-data sharing. We study this setting as Federated Domain-Incremental Learning (FDIL), where (i) clients are heterogeneous, (ii) tasks arrive sequentially with shifting domains, yet (iii) the label space remains fixed. Two theoretical pillars remain missing for FDIL under realistic deployment: a guarantee of backward knowledge transfer (BKT) and a convergence rate that holds across the sequence of all tasks with partial participation. We introduce SPECIAL (Server-Proximal Efficient Continual Aggregation for Learning), a simple, memory-free FDIL algorithm that adds a single server-side ``anchor'' to vanilla FedAvg: in each round, the server nudges the uniformly sampled participated clients update toward the previous global model with a lightweight proximal term. This anchor curbs cumulative drift without replay buffers, synthetic data, or task-specific heads, keeping communication and model size unchanged. Our theory shows that SPECIAL (i) preserves earlier tasks: a BKT bound caps any increase in prior-task loss by a drift-controlled term that shrinks with more rounds, local epochs, and participating clients; and (ii) learns efficiently across all tasks: the first communication-efficient non-convex convergence rate for FDIL with partial participation, O((E/NT)^(1/2)), with E local epochs, T communication rounds, and N participated clients per round, matching single-task FedAvg while explicitly separating optimization variance from inter-task drift. Experimental results further demonstrate the effectiveness of SPECIAL.

Title: VMonarch: Efficient Video Diffusion Transformers with Structured Attention

• Authors: Cheng Liang, Haoxian Chen, Liang Hou, Qi Fan, Gangshan Wu, Xin Tao, Limin Wang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22275
• Pdf URL: https://arxiv.org/pdf/2601.22275
• Copy Paste: [[2601.22275]] VMonarch: Efficient Video Diffusion Transformers with Structured Attention(https://arxiv.org/abs/2601.22275)
• Keywords: diffusion, transformer
• Abstract: The quadratic complexity of the attention mechanism severely limits the context scalability of Video Diffusion Transformers (DiTs). We find that the highly sparse spatio-temporal attention patterns exhibited in Video DiTs can be naturally represented by the Monarch matrix. It is a class of structured matrices with flexible sparsity, enabling sub-quadratic attention via an alternating minimization algorithm. Accordingly, we propose VMonarch, a novel attention mechanism for Video DiTs that enables efficient computation over the dynamic sparse patterns with structured Monarch matrices. First, we adapt spatio-temporal Monarch factorization to explicitly capture the intra-frame and inter-frame correlations of the video data. Second, we introduce a recomputation strategy to mitigate artifacts arising from instabilities during alternating minimization of Monarch matrices. Third, we propose a novel online entropy algorithm fused into FlashAttention, enabling fast Monarch matrix updates for long sequences. Extensive experiments demonstrate that VMonarch achieves comparable or superior generation quality to full attention on VBench after minimal tuning. It overcomes the attention bottleneck in Video DiTs, reduces attention FLOPs by a factor of 17.5, and achieves a speedup of over 5x in attention computation for long videos, surpassing state-of-the-art sparse attention methods at 90% sparsity.

Title: SurrogateSHAP: Training-Free Contributor Attribution for Text-to-Image (T2I) Models

• Authors: Mingyu Lu, Soham Gadgil, Chris Lin, Chanwoo Kim, Su-In Lee
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22276
• Pdf URL: https://arxiv.org/pdf/2601.22276
• Copy Paste: [[2601.22276]] SurrogateSHAP: Training-Free Contributor Attribution for Text-to-Image (T2I) Models(https://arxiv.org/abs/2601.22276)
• Keywords: fair, diffusion, generative
• Abstract: As Text-to-Image (T2I) diffusion models are increasingly used in real-world creative workflows, a principled framework for valuing contributors who provide a collection of data is essential for fair compensation and sustainable data marketplaces. While the Shapley value offers a theoretically grounded approach to attribution, it faces a dual computational bottleneck: (i) the prohibitive cost of exhaustive model retraining for each sampled subset of players (i.e., data contributors) and (ii) the combinatorial number of subsets needed to estimate marginal contributions due to contributor interactions. To this end, we propose SurrogateSHAP, a retraining-free framework that approximates the expensive retraining game through inference from a pretrained model. To further improve efficiency, we employ a gradient-boosted tree to approximate the utility function and derive Shapley values analytically from the tree-based model. We evaluate SurrogateSHAP across three diverse attribution tasks: (i) image quality for DDPM-CFG on CIFAR-20, (ii) aesthetics for Stable Diffusion on Post-Impressionist artworks, and (iii) product diversity for FLUX.1 on Fashion-Product data. Across settings, SurrogateSHAP outperforms prior methods while substantially reducing computational overhead, consistently identifying influential contributors across multiple utility metrics. Finally, we demonstrate that SurrogateSHAP effectively localizes data sources responsible for spurious correlations in clinical images, providing a scalable path toward auditing safety-critical generative models.

Title: Prepare Reasoning Language Models for Multi-Agent Debate with Self-Debate Reinforcement Learning

• Authors: Chenxi Liu, Yanshuo Chen, Ruibo Chen, Tianyi Xiong, Tong Zheng, Heng Huang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22297
• Pdf URL: https://arxiv.org/pdf/2601.22297
• Copy Paste: [[2601.22297]] Prepare Reasoning Language Models for Multi-Agent Debate with Self-Debate Reinforcement Learning(https://arxiv.org/abs/2601.22297)
• Keywords: large language model
• Abstract: The reasoning abilities of large language models (LLMs) have been substantially improved by reinforcement learning with verifiable rewards (RLVR). At test time, collaborative reasoning through Multi-Agent Debate (MAD) has emerged as a promising approach for enhancing LLM performance. However, current RLVR methods typically train LLMs to solve problems in isolation, without explicitly preparing them to synthesize and benefit from different rationales that arise during debate. In this work, we propose Self-Debate Reinforcement Learning (SDRL), a training framework that equips a single LLM with strong standalone problem-solving ability and the capability to learn from diverse reasoning trajectories in MAD. Given a prompt, SDRL first samples multiple candidate solutions, then constructs a debate context with diverse reasoning paths and generates second-turn responses conditioned on this context. Finally, SDRL jointly optimizes both the initial and debate-conditioned responses, yielding a model that is effective as both a standalone solver and a debate participant. Experiments across multiple base models and reasoning benchmarks show that SDRL improves overall MAD performance while simultaneously strengthening single model reasoning.

Title: Conformal Prediction for Generative Models via Adaptive Cluster-Based Density Estimation

• Authors: Qidong Yang, Qianyu Julie Zhu, Jonathan Giezendanner, Youssef Marzouk, Stephen Bates, Sherrie Wang
• Subjects: cs.LG, cs.AI, physics.ao-ph
• Abstract URL: https://arxiv.org/abs/2601.22298
• Pdf URL: https://arxiv.org/pdf/2601.22298
• Copy Paste: [[2601.22298]] Conformal Prediction for Generative Models via Adaptive Cluster-Based Density Estimation(https://arxiv.org/abs/2601.22298)
• Keywords: generative
• Abstract: Conditional generative models map input variables to complex, high-dimensional distributions, enabling realistic sample generation in a diverse set of domains. A critical challenge with these models is the absence of calibrated uncertainty, which undermines trust in individual outputs for high-stakes applications. To address this issue, we propose a systematic conformal prediction approach tailored to conditional generative models, leveraging density estimation on model-generated samples. We introduce a novel method called CP4Gen, which utilizes clustering-based density estimation to construct prediction sets that are less sensitive to outliers, more interpretable, and of lower structural complexity than existing methods. Extensive experiments on synthetic datasets and real-world applications, including climate emulation tasks, demonstrate that CP4Gen consistently achieves superior performance in terms of prediction set volume and structural simplicity. Our approach offers practitioners a powerful tool for uncertainty estimation associated with conditional generative models, particularly in scenarios demanding rigorous and interpretable prediction sets.

Title: Coarse-to-Real: Generative Rendering for Populated Dynamic Scenes

• Authors: Gonzalo Gomez-Nogales, Yicong Hong, Chongjian Ge, Marc Comino-Trinidad, Dan Casas, Yi Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22301
• Pdf URL: https://arxiv.org/pdf/2601.22301
• Copy Paste: [[2601.22301]] Coarse-to-Real: Generative Rendering for Populated Dynamic Scenes(https://arxiv.org/abs/2601.22301)
• Keywords: generative
• Abstract: Traditional rendering pipelines rely on complex assets, accurate materials and lighting, and substantial computational resources to produce realistic imagery, yet they still face challenges in scalability and realism for populated dynamic scenes. We present C2R (Coarse-to-Real), a generative rendering framework that synthesizes real-style urban crowd videos from coarse 3D simulations. Our approach uses coarse 3D renderings to explicitly control scene layout, camera motion, and human trajectories, while a learned neural renderer generates realistic appearance, lighting, and fine-scale dynamics guided by text prompts. To overcome the lack of paired training data between coarse simulations and real videos, we adopt a two-phase mixed CG-real training strategy that learns a strong generative prior from large-scale real footage and introduces controllability through shared implicit spatio-temporal features across domains. The resulting system supports coarse-to-fine control, generalizes across diverse CG and game inputs, and produces temporally consistent, controllable, and realistic urban scene videos from minimal 3D input. We will release the model and project webpage at this https URL.

Title: ZK-HybridFL: Zero-Knowledge Proof-Enhanced Hybrid Ledger for Federated Learning

• Authors: Amirhossein Taherpour, Xiaodong Wang
• Subjects: cs.LG, cs.CR, cs.DC
• Abstract URL: https://arxiv.org/abs/2601.22302
• Pdf URL: https://arxiv.org/pdf/2601.22302
• Copy Paste: [[2601.22302]] ZK-HybridFL: Zero-Knowledge Proof-Enhanced Hybrid Ledger for Federated Learning(https://arxiv.org/abs/2601.22302)
• Keywords: secure, security, privacy, attack, robust, federate
• Abstract: Federated learning (FL) enables collaborative model training while preserving data privacy, yet both centralized and decentralized approaches face challenges in scalability, security, and update validation. We propose ZK-HybridFL, a secure decentralized FL framework that integrates a directed acyclic graph (DAG) ledger with dedicated sidechains and zero-knowledge proofs (ZKPs) for privacy-preserving model validation. The framework uses event-driven smart contracts and an oracle-assisted sidechain to verify local model updates without exposing sensitive data. A built-in challenge mechanism efficiently detects adversarial behavior. In experiments on image classification and language modeling tasks, ZK-HybridFL achieves faster convergence, higher accuracy, lower perplexity, and reduced latency compared to Blade-FL and ChainFL. It remains robust against substantial fractions of adversarial and idle nodes, supports sub-second on-chain verification with efficient gas usage, and prevents invalid updates and orphanage-style attacks. This makes ZK-HybridFL a scalable and secure solution for decentralized FL across diverse environments.

Title: Stealthy Poisoning Attacks Bypass Defenses in Regression Settings

• Authors: Javier Carnerero-Cano, Luis Muñoz-González, Phillippa Spencer, Emil C. Lupu
• Subjects: cs.LG, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2601.22308
• Pdf URL: https://arxiv.org/pdf/2601.22308
• Copy Paste: [[2601.22308]] Stealthy Poisoning Attacks Bypass Defenses in Regression Settings(https://arxiv.org/abs/2601.22308)
• Keywords: defense, attack, robust, steal
• Abstract: Regression models are widely used in industrial processes, engineering and in natural and physical sciences, yet their robustness to poisoning has received less attention. When it has, studies often assume unrealistic threat models and are thus less useful in practice. In this paper, we propose a novel optimal stealthy attack formulation that considers different degrees of detectability and show that it bypasses state-of-the-art defenses. We further propose a new methodology based on normalization of objectives to evaluate different trade-offs between effectiveness and detectability. Finally, we develop a novel defense (BayesClean) against stealthy attacks. BayesClean improves on previous defenses when attacks are stealthy and the number of poisoning points is significant.

Title: SCALAR: Quantifying Structural Hallucination, Consistency, and Reasoning Gaps in Materials Foundation Models

• Authors: Can Polat, Erchin Serpedin, Mustafa Kurban, Hasan Kurban
• Subjects: cs.LG, cond-mat.mtrl-sci, cs.CE
• Abstract URL: https://arxiv.org/abs/2601.22312
• Pdf URL: https://arxiv.org/pdf/2601.22312
• Copy Paste: [[2601.22312]] SCALAR: Quantifying Structural Hallucination, Consistency, and Reasoning Gaps in Materials Foundation Models(https://arxiv.org/abs/2601.22312)
• Keywords: large language model
• Abstract: Large language models are increasingly applied to materials science reasoning, yet their behavior under physically structured distribution shifts remains poorly understood. We introduce SCALAR (Structural Consistency And Logic Across Regimes), a benchmark for evaluating geometric scale generalization and its connection to structural hallucination, consistency, and reasoning in materials foundation models. Given canonical crystal representations, models must reason about derived nanoparticle structures obtained through supercell expansion and geometric truncation across length scales spanning a few atoms to over 18,000 atoms, totaling $\approx$100,000 structures from DFT-validated unit cells. SCALAR defines three tasks. (i) CIF to property prediction. (ii) A Chain-of-Thought variant with explicit physics-grounded reasoning. (iii) Inverse retrieval identifying crystals from candidates given target properties. Outputs are evaluated via structured metrics capturing numeric error, hallucination, cross-prompt consistency, monotonic reasoning, output validity, and retrieval regret. Experiments across diverse foundation models reveal large, model-dependent shifts under explicit reasoning, often reducing hallucination and error, but frequently destabilizing consistency or validity. These results demonstrate that geometric scale generalization cannot be inferred from accuracy alone. Supplementary materials are available at this https URL.

Title: Hair-Trigger Alignment: Black-Box Evaluation Cannot Guarantee Post-Update Alignment

• Authors: Yavuz Bakman, Duygu Nur Yaldiz, Salman Avestimehr, Sai Praneeth Karimireddy
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22313
• Pdf URL: https://arxiv.org/pdf/2601.22313
• Copy Paste: [[2601.22313]] Hair-Trigger Alignment: Black-Box Evaluation Cannot Guarantee Post-Update Alignment(https://arxiv.org/abs/2601.22313)
• Keywords: privacy, robust, large language model
• Abstract: Large Language Models (LLMs) are rarely static and are frequently updated in practice. A growing body of alignment research has shown that models initially deemed "aligned" can exhibit misaligned behavior after fine-tuning, such as forgetting jailbreak safety features or re-surfacing knowledge that was intended to be forgotten. These works typically assume that the initial model is aligned based on static black-box evaluation, i.e., the absence of undesired responses to a fixed set of queries. In contrast, we formalize model alignment in both the static and post-update settings and uncover a fundamental limitation of black-box evaluation. We theoretically show that, due to overparameterization, static alignment provides no guarantee of post-update alignment for any update dataset. Moreover, we prove that static black-box probing cannot distinguish a model that is genuinely post-update robust from one that conceals an arbitrary amount of adversarial behavior which can be activated by even a single benign gradient update. We further validate these findings empirically in LLMs across three core alignment domains: privacy, jailbreak safety, and behavioral honesty. We demonstrate the existence of LLMs that pass all standard black-box alignment tests, yet become severely misaligned after a single benign update. Finally, we show that the capacity to hide such latent adversarial behavior increases with model scale, confirming our theoretical prediction that post-update misalignment grows with the number of parameters. Together, our results highlight the inadequacy of static evaluation protocols and emphasize the urgent need for post-update-robust alignment evaluation.

Title: Gaussian Process Bandit Optimization with Machine Learning Predictions and Application to Hypothesis Generation

• Authors: Xin Jennifer Chen, Yunjin Tong
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22315
• Pdf URL: https://arxiv.org/pdf/2601.22315
• Copy Paste: [[2601.22315]] Gaussian Process Bandit Optimization with Machine Learning Predictions and Application to Hypothesis Generation(https://arxiv.org/abs/2601.22315)
• Keywords: large language model
• Abstract: Many real-world optimization problems involve an expensive ground-truth oracle (e.g., human evaluation, physical experiments) and a cheap, low-fidelity prediction oracle (e.g., machine learning models, simulations). Meanwhile, abundant offline data (e.g., past experiments and predictions) are often available and can be used to pretrain powerful predictive models, as well as to provide an informative prior. We propose Prediction-Augmented Gaussian Process Upper Confidence Bound (PA-GP-UCB), a novel Bayesian optimization algorithm that leverages both oracles and offline data to achieve provable gains in sample efficiency for the ground-truth oracle queries. PA-GP-UCB employs a control-variates estimator derived from a joint Gaussian process posterior to correct prediction bias and reduce uncertainty. We prove that PA-GP-UCB preserves the standard regret rate of GP-UCB while achieving a strictly smaller leading constant that is explicitly controlled by prediction quality and offline data coverage. Empirically, PA-GP-UCB converges faster than Vanilla GP-UCB and naive prediction-augmented GP-UCB baselines on synthetic benchmarks and on a real-world hypothesis evaluation task grounded in human behavioral data, where predictions are provided by large language models. These results establish PA-GP-UCB as a general and sample-efficient framework for hypothesis generation under expensive feedback.

Title: Federate the Router: Learning Language Model Routers with Sparse and Decentralized Evaluations

• Authors: Baris Askin, Shivam Patel, Anupam Nayak, Andrea Vigano, Jiin Woo, Gauri Joshi, Carlee Joe-Wong
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22318
• Pdf URL: https://arxiv.org/pdf/2601.22318
• Copy Paste: [[2601.22318]] Federate the Router: Learning Language Model Routers with Sparse and Decentralized Evaluations(https://arxiv.org/abs/2601.22318)
• Keywords: privacy, federate, large language model
• Abstract: Large language models (LLMs) are increasingly accessed as remotely hosted services by edge and enterprise clients that cannot run frontier models locally. Since models vary widely in capability and price, routing queries to models that balance quality and inference cost is essential. Existing router approaches assume access to centralized query-model evaluation data. However, these data are often fragmented across clients, such as end users and organizations, and are privacy-sensitive, which makes centralizing data infeasible. Additionally, per-client router training is ineffective since local evaluation data is limited and covers only a restricted query distribution and a biased subset of model evaluations. We introduce the first federated framework for LLM routing, enabling clients to learn a shared routing policy from local offline query-model evaluation data. Our framework supports both parametric multilayer perceptron router and nonparametric K-means router under heterogeneous client query distributions and non-uniform model coverage. Across two benchmarks, federated collaboration improves the accuracy-cost frontier over client-local routers, both via increased effective model coverage and better query generalization. Our theoretical results also validate that federated training reduces routing suboptimality.

Title: Matrix Factorization for Practical Continual Mean Estimation Under User-Level Differential Privacy

• Authors: Nikita P. Kalinin, Ali Najar, Valentin Roth, Christoph H. Lampert
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2601.22320
• Pdf URL: https://arxiv.org/pdf/2601.22320
• Copy Paste: [[2601.22320]] Matrix Factorization for Practical Continual Mean Estimation Under User-Level Differential Privacy(https://arxiv.org/abs/2601.22320)
• Keywords: privacy, protect
• Abstract: We study continual mean estimation, where data vectors arrive sequentially and the goal is to maintain accurate estimates of the running mean. We address this problem under user-level differential privacy, which protects each user's entire dataset even when they contribute multiple data points. Previous work on this problem has focused on pure differential privacy. While important, this approach limits applicability, as it leads to overly noisy estimates. In contrast, we analyze the problem under approximate differential privacy, adopting recent advances in the Matrix Factorization mechanism. We introduce a novel mean estimation specific factorization, which is both efficient and accurate, achieving asymptotically lower mean-squared error bounds in continual mean estimation under user-level differential privacy.

Title: Spatially-Adaptive Conformal Graph Transformer for Indoor Localization in Wi-Fi Driven Networks

• Authors: Ayesh Abu Lehyeh, Anastassia Gharib, Safwan Wshah
• Subjects: cs.LG, eess.SP
• Abstract URL: https://arxiv.org/abs/2601.22322
• Pdf URL: https://arxiv.org/pdf/2601.22322
• Copy Paste: [[2601.22322]] Spatially-Adaptive Conformal Graph Transformer for Indoor Localization in Wi-Fi Driven Networks(https://arxiv.org/abs/2601.22322)
• Keywords: robust, transformer
• Abstract: Indoor localization is a critical enabler for a wide range of location-based services in smart environments, including navigation, asset tracking, and safety-critical applications. Recent graph-based models leverage spatial relationships between Wire-less Fidelity (Wi-Fi) Access Points (APs) and devices, offering finer localization granularity, but fall short in quantifying prediction uncertainty, a key requirement for real-world deployment. In this paper, we propose Spatially-Adaptive Conformal Graph Transformer (SAC-GT), a framework for accurate and reliable indoor localization. SAC-GT integrates a Graph Transformer (GT) model that captures network's spatial topology and signal strength dynamics, with a novel Spatially-Adaptive Conformal Prediction (SACP) method that provides region-specific uncertainty estimates. This allows SAC-GT to produce not only precise two-dimensional (2D) location predictions but also statistically valid confidence regions tailored to varying environmental conditions. Extensive evaluations on a large-scale real-world dataset demonstrate that the proposed SAC-GT solution achieves state-of-the-art localization accuracy while delivering robust and spatially adaptive reliability guarantees.

Title: DP-$λ$CGD: Efficient Noise Correlation for Differentially Private Model Training

• Authors: Nikita P. Kalinin, Ryan McKenna, Rasmus Pagh, Christoph H. Lampert
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22334
• Pdf URL: https://arxiv.org/pdf/2601.22334
• Copy Paste: [[2601.22334]] DP-$λ$CGD: Efficient Noise Correlation for Differentially Private Model Training(https://arxiv.org/abs/2601.22334)
• Keywords: privacy
• Abstract: Differentially private stochastic gradient descent (DP-SGD) is the gold standard for training machine learning models with formal differential privacy guarantees. Several recent extensions improve its accuracy by introducing correlated noise across training iterations. Matrix factorization mechanisms are a prominent example, but they correlate noise across many iterations and require storing previously added noise vectors, leading to substantial memory overhead in some settings. In this work, we propose a new noise correlation strategy that correlates noise only with the immediately preceding iteration and cancels a controlled portion of it. Our method relies on noise regeneration using a pseudorandom noise generator, eliminating the need to store past noise. As a result, it requires no additional memory beyond standard DP-SGD. We show that the computational overhead is minimal and empirically demonstrate improved accuracy over DP-SGD.

Title: Quantum-Inspired Reinforcement Learning for Secure and Sustainable AIoT-Driven Supply Chain Systems

• Authors: Muhammad Bilal Akram Dastagir, Omer Tariq, Shahid Mumtaz, Saif Al-Kuwari, Ahmed Farouk
• Subjects: cs.LG, quant-ph
• Abstract URL: https://arxiv.org/abs/2601.22339
• Pdf URL: https://arxiv.org/pdf/2601.22339
• Copy Paste: [[2601.22339]] Quantum-Inspired Reinforcement Learning for Secure and Sustainable AIoT-Driven Supply Chain Systems(https://arxiv.org/abs/2601.22339)
• Keywords: secure, security, attack, robust
• Abstract: Modern supply chains must balance high-speed logistics with environmental impact and security constraints, prompting a surge of interest in AI-enabled Internet of Things (AIoT) solutions for global commerce. However, conventional supply chain optimization models often overlook crucial sustainability goals and cyber vulnerabilities, leaving systems susceptible to both ecological harm and malicious attacks. To tackle these challenges simultaneously, this work integrates a quantum-inspired reinforcement learning framework that unifies carbon footprint reduction, inventory management, and cryptographic-like security measures. We design a quantum-inspired reinforcement learning framework that couples a controllable spin-chain analogy with real-time AIoT signals and optimizes a multi-objective reward unifying fidelity, security, and carbon costs. The approach learns robust policies with stabilized training via value-based and ensemble updates, supported by window-normalized reward components to ensure commensurate scaling. In simulation, the method exhibits smooth convergence, strong late-episode performance, and graceful degradation under representative noise channels, outperforming standard learned and model-based references, highlighting its robust handling of real-time sustainability and risk demands. These findings reinforce the potential for quantum-inspired AIoT frameworks to drive secure, eco-conscious supply chain operations at scale, laying the groundwork for globally connected infrastructures that responsibly meet both consumer and environmental needs.

Title: MixQuant: Pushing the Limits of Block Rotations in Post-Training Quantization

• Authors: Sai Sanjeet, Ian Colbert, Pablo Monteagudo-Lago, Giuseppe Franco, Yaman Umuroglu, Nicholas J. Fraser
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22347
• Pdf URL: https://arxiv.org/pdf/2601.22347
• Copy Paste: [[2601.22347]] MixQuant: Pushing the Limits of Block Rotations in Post-Training Quantization(https://arxiv.org/abs/2601.22347)
• Keywords: diffusion, transformer
• Abstract: Recent post-training quantization (PTQ) methods have adopted block rotations to diffuse outliers prior to rounding. While this reduces the overhead of full-vector rotations, the effect of block structure on outlier suppression remains poorly understood. To fill this gap, we present the first systematic, non-asymptotic analysis of outlier suppression for block Hadamard rotations. Our analysis reveals that outlier suppression is fundamentally limited by the geometry of the input vector. In particular, post-rotation outliers are deterministically minimized when the pre-rotation $\ell_1$ norm mass is evenly distributed across blocks. Guided by these insights, we introduce MixQuant, a block rotation-aware PTQ framework that redistributes activation mass via permutations prior to rotation. We propose a greedy mass diffusion algorithm to calibrate permutations by equalizing the expected blockwise $\ell_1$ norms. To avoid adding inference overhead, we identify permutation-equivariant regions in transformer architectures to merge the resulting permutations into model weights before deployment. Experiments show that MixQuant consistently improves accuracy across all block sizes, recovering up to 90% of the full-vector rotation perplexity when quantizing Llama3 1B to INT4 with block size 16, compared to 46% without permutations.

Title: Learning Policy Representations for Steerable Behavior Synthesis

• Authors: Beiming Li, Sergio Rozada, Alejandro Ribeiro
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22350
• Pdf URL: https://arxiv.org/pdf/2601.22350
• Copy Paste: [[2601.22350]] Learning Policy Representations for Steerable Behavior Synthesis(https://arxiv.org/abs/2601.22350)
• Keywords: generative
• Abstract: Given a Markov decision process (MDP), we seek to learn representations for a range of policies to facilitate behavior steering at test time. As policies of an MDP are uniquely determined by their occupancy measures, we propose modeling policy representations as expectations of state-action feature maps with respect to occupancy measures. We show that these representations can be approximated uniformly for a range of policies using a set-based architecture. Our model encodes a set of state-action samples into a latent embedding, from which we decode both the policy and its value functions corresponding to multiple rewards. We use variational generative approach to induce a smooth latent space, and further shape it with contrastive learning so that latent distances align with differences in value functions. This geometry permits gradient-based optimization directly in the latent space. Leveraging this capability, we solve a novel behavior synthesis task, where policies are steered to satisfy previously unseen value function constraints without additional training.

Title: Recoverability Has a Law: The ERR Measure for Tool-Augmented Agents

• Authors: Sri Vatsa Vuddanti, Satwik Kumar Chittiprolu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22352
• Pdf URL: https://arxiv.org/pdf/2601.22352
• Copy Paste: [[2601.22352]] Recoverability Has a Law: The ERR Measure for Tool-Augmented Agents(https://arxiv.org/abs/2601.22352)
• Keywords: robust
• Abstract: Language model agents often appear capable of self-recovery after failing tool call executions, yet this behavior lacks a formal explanation. We present a predictive theory that resolves this gap by showing that recoverability follows a measurable law. To elaborate, we formalize recoverability through Expected Recovery Regret (ERR), which quantifies the deviation of a recovery policy from the optimal one under stochastic execution noise, and derive a first-order relationship between ERR and an empirical observable quantity, the Efficiency Score (ES). This yields a falsifiable first-order quantitative law of recovery dynamics in tool-using agents. We empirically validate the law across five tool-use benchmarks spanning controlled perturbations, diagnostic reasoning, and real-world APIs. Across model scales, perturbation regimes, and recovery horizons, predicted regret under the ERR-ES law closely matched observed post-failure regret measured from Monte Carlo rollouts, within delta less than or equal to 0.05. Our results reveal that recoverability is not an artifact of model scale or architecture, but a governed property of interaction dynamics, providing a theoretical foundation for execution-level robustness in language agents.

Title: Relative Wasserstein Angle and the Problem of the $W_2$-Nearest Gaussian Distribution

• Authors: Binshuai Wang, Peng Wei
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22355
• Pdf URL: https://arxiv.org/pdf/2601.22355
• Copy Paste: [[2601.22355]] Relative Wasserstein Angle and the Problem of the $W_2$-Nearest Gaussian Distribution(https://arxiv.org/abs/2601.22355)
• Keywords: robust
• Abstract: We study the problem of quantifying how far an empirical distribution deviates from Gaussianity under the framework of optimal transport. By exploiting the cone geometry of the relative translation invariant quadratic Wasserstein space, we introduce two novel geometric quantities, the relative Wasserstein angle and the orthogonal projection distance, which provide meaningful measures of non-Gaussianity. We prove that the filling cone generated by any two rays in this space is flat, ensuring that angles, projections, and inner products are rigorously well-defined. This geometric viewpoint recasts Gaussian approximation as a projection problem onto the Gaussian cone and reveals that the commonly used moment-matching Gaussian can \emph{not} be the \(W_2\)-nearest Gaussian for a given empirical distribution. In one dimension, we derive closed-form expressions for the proposed quantities and extend them to several classical distribution families, including uniform, Laplace, and logistic distributions; while in high dimensions, we develop an efficient stochastic manifold optimization algorithm based on a semi-discrete dual formulation. Experiments on synthetic data and real-world feature distributions demonstrate that the relative Wasserstein angle is more robust than the Wasserstein distance and that the proposed nearest Gaussian provides a better approximation than moment matching in the evaluation of Fréchet Inception Distance (FID) scores.

Title: PoSafeNet: Safe Learning with Poset-Structured Neural Nets

• Authors: Kiwan Wong, Wei Xiao, Daniela Rus
• Subjects: cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2601.22356
• Pdf URL: https://arxiv.org/pdf/2601.22356
• Copy Paste: [[2601.22356]] PoSafeNet: Safe Learning with Poset-Structured Neural Nets(https://arxiv.org/abs/2601.22356)
• Keywords: robust
• Abstract: Safe learning is essential for deploying learningbased controllers in safety-critical robotic systems, yet existing approaches often enforce multiple safety constraints uniformly or via fixed priority orders, leading to infeasibility and brittle behavior. In practice, safety requirements are heterogeneous and admit only partial priority relations, where some constraints are comparable while others are inherently incomparable. We formalize this setting as poset-structured safety, modeling safety constraints as a partially ordered set and treating safety composition as a structural property of the policy class. Building on this formulation, we propose PoSafeNet, a differentiable neural safety layer that enforces safety via sequential closed-form projection under poset-consistent constraint orderings, enabling adaptive selection or mixing of valid safety executions while preserving priority semantics by construction. Experiments on multi-obstacle navigation, constrained robot manipulation, and vision-based autonomous driving demonstrate improved feasibility, robustness, and scalability over unstructured and differentiable quadratic program-based safety layers.

Title: Small Talk, Big Impact: The Energy Cost of Thanking AI

• Authors: Julien Delavande, Regis Pierrard, Sasha Luccioni
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22357
• Pdf URL: https://arxiv.org/pdf/2601.22357
• Copy Paste: [[2601.22357]] Small Talk, Big Impact: The Energy Cost of Thanking AI(https://arxiv.org/abs/2601.22357)
• Keywords: large language model
• Abstract: Being polite is free - or is it? In this paper, we quantify the energy cost of seemingly innocuous messages such as ``thank you'' when interacting with large language models, often used by users to convey politeness. Using real-world conversation traces and fine-grained energy measurements, we quantify how input length, output length and model size affect energy use. While politeness is our motivating example, it also serves as a controlled and reproducible proxy for measuring the energy footprint of a typical LLM interaction. Our findings provide actionable insights for building more sustainable and efficient LLM applications, especially in increasingly widespread real-world contexts like chat. As user adoption grows and billions of prompts are processed daily, understanding and mitigating this cost becomes crucial - not just for efficiency, but for sustainable AI deployment.

Title: The Unseen Threat: Residual Knowledge in Machine Unlearning under Perturbed Samples

• Authors: Hsiang Hsu, Pradeep Niroula, Zichang He, Ivan Brugere, Freddy Lecue, Chun-Fu Chen
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22359
• Pdf URL: https://arxiv.org/pdf/2601.22359
• Copy Paste: [[2601.22359]] The Unseen Threat: Residual Knowledge in Machine Unlearning under Perturbed Samples(https://arxiv.org/abs/2601.22359)
• Keywords: privacy
• Abstract: Machine unlearning offers a practical alternative to avoid full model re-training by approximately removing the influence of specific user data. While existing methods certify unlearning via statistical indistinguishability from re-trained models, these guarantees do not naturally extend to model outputs when inputs are adversarially perturbed. In particular, slight perturbations of forget samples may still be correctly recognized by the unlearned model - even when a re-trained model fails to do so - revealing a novel privacy risk: information about the forget samples may persist in their local neighborhood. In this work, we formalize this vulnerability as residual knowledge and show that it is inevitable in high-dimensional settings. To mitigate this risk, we propose a fine-tuning strategy, named RURK, that penalizes the model's ability to re-recognize perturbed forget samples. Experiments on vision benchmarks with deep neural networks demonstrate that residual knowledge is prevalent across existing unlearning methods and that our approach effectively prevents residual knowledge.

Title: MERMAID: Memory-Enhanced Retrieval and Reasoning with Multi-Agent Iterative Knowledge Grounding for Veracity Assessment

• Authors: Yupeng Cao, Chengyang He, Yangyang Yu, Ping Wang, K.P. Subbalakshmi
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22361
• Pdf URL: https://arxiv.org/pdf/2601.22361
• Copy Paste: [[2601.22361]] MERMAID: Memory-Enhanced Retrieval and Reasoning with Multi-Agent Iterative Knowledge Grounding for Veracity Assessment(https://arxiv.org/abs/2601.22361)
• Keywords: large language model
• Abstract: Assessing the veracity of online content has become increasingly critical. Large language models (LLMs) have recently enabled substantial progress in automated veracity assessment, including automated fact-checking and claim verification systems. Typical veracity assessment pipelines break down complex claims into sub-claims, retrieve external evidence, and then apply LLM reasoning to assess veracity. However, existing methods often treat evidence retrieval as a static, isolated step and do not effectively manage or reuse retrieved evidence across claims. In this work, we propose MERMAID, a memory-enhanced multi-agent veracity assessment framework that tightly couples the retrieval and reasoning processes. MERMAID integrates agent-driven search, structured knowledge representations, and a persistent memory module within a Reason-Action style iterative process, enabling dynamic evidence acquisition and cross-claim evidence reuse. By retaining retrieved evidence in an evidence memory, the framework reduces redundant searches and improves verification efficiency and consistency. We evaluate MERMAID on three fact-checking benchmarks and two claim-verification datasets using multiple LLMs, including GPT, LLaMA, and Qwen families. Experimental results show that MERMAID achieves state-of-the-art performance while improving the search efficiency, demonstrating the effectiveness of synergizing retrieval, reasoning, and memory for reliable veracity assessment.

Title: Understanding Efficiency: Quantization, Batching, and Serving Strategies in LLM Energy Use

• Authors: Julien Delavande, Regis Pierrard, Sasha Luccioni
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22362
• Pdf URL: https://arxiv.org/pdf/2601.22362
• Copy Paste: [[2601.22362]] Understanding Efficiency: Quantization, Batching, and Serving Strategies in LLM Energy Use(https://arxiv.org/abs/2601.22362)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) are increasingly deployed in production, contributing towards shifting the burden in terms of computational resources and energy demands from training to inference. While prior work has examined the energy cost of inference per prompt or per token, we highlight how \emph{system-level design choices} - such as numerical precision, batching strategy, and request scheduling - can lead to orders-of-magnitude differences in energy consumption for the same model. We perform a detailed empirical study of LLM inference energy and latency on NVIDIA H100 GPUs, analyzing the impact of quantization, batch size, and serving configuration (e.g., with Hugging Face's Text Generation Inference server). Our results reveal that lower-precision formats only yield energy gains in compute-bound regimes; that batching improves energy efficiency, especially in memory-bound phases like decoding; and that structured request timing (arrival shaping) can reduce per-request energy by up to 100 times. We argue that sustainable LLM deployment depends not only on model internals, but also on the orchestration of the serving stack. Our findings motivate phase-aware energy profiling and system-level optimizations for greener AI services.

Title: Context Structure Reshapes the Representational Geometry of Language Models

• Authors: Eghbal A. Hosseini, Yuxuan Li, Yasaman Bahri, Declan Campbell, Andrew Kyle Lampinen
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22364
• Pdf URL: https://arxiv.org/pdf/2601.22364
• Copy Paste: [[2601.22364]] Context Structure Reshapes the Representational Geometry of Language Models(https://arxiv.org/abs/2601.22364)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have been shown to organize the representations of input sequences into straighter neural trajectories in their deep layers, which has been hypothesized to facilitate next-token prediction via linear extrapolation. Language models can also adapt to diverse tasks and learn new structure in context, and recent work has shown that this in-context learning (ICL) can be reflected in representational changes. Here we bring these two lines of research together to explore whether representation straightening occurs \emph{within} a context during ICL. We measure representational straightening in Gemma 2 models across a diverse set of in-context tasks, and uncover a dichotomy in how LLMs' representations change in context. In continual prediction settings (e.g., natural language, grid world traversal tasks) we observe that increasing context increases the straightness of neural sequence trajectories, which is correlated with improvement in model prediction. Conversely, in structured prediction settings (e.g., few-shot tasks), straightening is inconsistent -- it is only present in phases of the task with explicit structure (e.g., repeating a template), but vanishes elsewhere. These results suggest that ICL is not a monolithic process. Instead, we propose that LLMs function like a Swiss Army knife: depending on task structure, the LLM dynamically selects between strategies, only some of which yield representational straightening.

Title: FIRE: Multi-fidelity Regression with Distribution-conditioned In-context Learning using Tabular Foundation Models

• Authors: Rosen Ting-Ying Yu, Nicholas Sung, Faez Ahmed
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22371
• Pdf URL: https://arxiv.org/pdf/2601.22371
• Copy Paste: [[2601.22371]] FIRE: Multi-fidelity Regression with Distribution-conditioned In-context Learning using Tabular Foundation Models(https://arxiv.org/abs/2601.22371)
• Keywords: robust
• Abstract: Multi-fidelity (MF) regression often operates in regimes of extreme data imbalance, where the commonly-used Gaussian-process (GP) surrogates struggle with cubic scaling costs and overfit to sparse high-fidelity observations, limiting efficiency and generalization in real-world applications. We introduce FIRE, a training-free MF framework that couples tabular foundation models (TFMs) to perform zero-shot in-context Bayesian inference via a high-fidelity correction model conditioned on the low-fidelity model's posterior predictive distributions. This cross-fidelity information transfer via distributional summaries captures heteroscedastic errors, enabling robust residual learning without model retraining. Across 31 benchmark problems spanning synthetic and real-world tasks (e.g., DrivAerNet, LCBench), FIRE delivers a stronger performance-time trade-off than seven state-of-the-art GP-based or deep learning MF regression methods, ranking highest in accuracy and uncertainty quantification with runtime advantages. Limitations include context window constraints and dependence on the quality of the pre-trained TFM's.

Title: Stability-Aware Prompt Optimization for Clinical Data Abstraction

• Authors: Arinbjörn Kolbeinsson, Daniel Timbie, Sajjan Narsinghani, Sanjay Hariharan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22373
• Pdf URL: https://arxiv.org/pdf/2601.22373
• Copy Paste: [[2601.22373]] Stability-Aware Prompt Optimization for Clinical Data Abstraction(https://arxiv.org/abs/2601.22373)
• Keywords: large language model
• Abstract: Large language models used for clinical abstraction are sensitive to prompt wording, yet most work treats prompts as fixed and studies uncertainty in isolation. We argue these should be treated jointly. Across two clinical tasks (MedAlign applicability/correctness and MS subtype abstraction) and multiple open and proprietary models, we measure prompt sensitivity via flip rates and relate it to calibration and selective prediction. We find that higher accuracy does not guarantee prompt stability, and that models can appear well-calibrated yet remain fragile to paraphrases. We propose a dual-objective prompt optimization loop that jointly targets accuracy and stability, showing that explicitly including a stability term reduces flip rates across tasks and models, sometimes at modest accuracy cost. Our results suggest prompt sensitivity should be an explicit objective when validating clinical LLM systems.

Title: FlexMap: Generalized HD Map Construction from Flexible Camera Configurations

• Authors: Run Wang, Chaoyi Zhou, Amir Salarpour, Xi Liu, Zhi-Qi Cheng, Feng Luo, Mert D. Pesé, Siyu Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22376
• Pdf URL: https://arxiv.org/pdf/2601.22376
• Copy Paste: [[2601.22376]] FlexMap: Generalized HD Map Construction from Flexible Camera Configurations(https://arxiv.org/abs/2601.22376)
• Keywords: robust
• Abstract: High-definition (HD) maps provide essential semantic information of road structures for autonomous driving systems, yet current HD map construction methods require calibrated multi-camera setups and either implicit or explicit 2D-to-BEV transformations, making them fragile when sensors fail or camera configurations vary across vehicle fleets. We introduce FlexMap, unlike prior methods that are fixed to a specific N-camera rig, our approach adapts to variable camera configurations without any architectural changes or per-configuration retraining. Our key innovation eliminates explicit geometric projections by using a geometry-aware foundation model with cross-frame attention to implicitly encode 3D scene understanding in feature space. FlexMap features two core components: a spatial-temporal enhancement module that separates cross-view spatial reasoning from temporal dynamics, and a camera-aware decoder with latent camera tokens, enabling view-adaptive attention without the need for projection matrices. Experiments demonstrate that FlexMap outperforms existing methods across multiple configurations while maintaining robustness to missing views and sensor variations, enabling more practical real-world deployment.

Title: Purely Agentic Black-Box Optimization for Biological Design

• Authors: Natalie Maus, Yimeng Zeng, Haydn Thomas Jones, Yining Huang, Gaurav Ng Goel, Alden Rose, Kyurae Kim, Hyun-Su Lee, Marcelo Der Torossian Torres, Fangping Wan, Cesar de la Fuente-Nunez, Mark Yatskar, Osbert Bastani, Jacob R. Gardner
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22382
• Pdf URL: https://arxiv.org/pdf/2601.22382
• Copy Paste: [[2601.22382]] Purely Agentic Black-Box Optimization for Biological Design(https://arxiv.org/abs/2601.22382)
• Keywords: large language model
• Abstract: Many key challenges in biological design-such as small-molecule drug discovery, antimicrobial peptide development, and protein engineering-can be framed as black-box optimization over vast, complex structured spaces. Existing methods rely mainly on raw structural data and struggle to exploit the rich scientific literature. While large language models (LLMs) have been added to these pipelines, they have been confined to narrow roles within structure-centered optimizers. We instead cast biological black-box optimization as a fully agentic, language-based reasoning process. We introduce Purely Agentic BLack-box Optimization (PABLO), a hierarchical agentic system that uses scientific LLMs pretrained on chemistry and biology literature to generate and iteratively refine biological candidates. On both the standard GuacaMol molecular design and antimicrobial peptide optimization tasks, PABLO achieves state-of-the-art performance, substantially improving sample efficiency and final objective values over established baselines. Compared to prior optimization methods that incorporate LLMs, PABLO achieves competitive token usage per run despite relying on LLMs throughout the optimization loop. Beyond raw performance, the agentic formulation offers key advantages for realistic design: it naturally incorporates semantic task descriptions, retrieval-augmented domain knowledge, and complex constraints. In follow-up in vitro validation, PABLO-optimized peptides showed strong activity against drug-resistant pathogens, underscoring the practical potential of PABLO for therapeutic discovery.

Title: Specialists or Generalists? Multi-Agent and Single-Agent LLMs for Essay Grading

• Authors: Jamiu Adekunle Idowu, Ahmed Almasoud
• Subjects: cs.CL, cs.MA
• Abstract URL: https://arxiv.org/abs/2601.22386
• Pdf URL: https://arxiv.org/pdf/2601.22386
• Copy Paste: [[2601.22386]] Specialists or Generalists? Multi-Agent and Single-Agent LLMs for Essay Grading(https://arxiv.org/abs/2601.22386)
• Keywords: large language model
• Abstract: Automated essay scoring (AES) systems increasingly rely on large language models, yet little is known about how architectural choices shape their performance across different essay quality levels. This paper evaluates single-agent and multi-agent LLM architectures for essay grading using the ASAP 2.0 corpus. Our multi-agent system decomposes grading into three specialist agents (Content, Structure, Language) coordinated by a Chairman Agent that implements rubric-aligned logic including veto rules and score capping. We test both architectures in zero-shot and few-shot conditions using GPT-5.1. Results show that the multi-agent system is significantly better at identifying weak essays while the single-agent system performs better on mid-range essays. Both architectures struggle with high-quality essays. Critically, few-shot calibration emerges as the dominant factor in system performance -- providing just two examples per score level improves QWK by approximately 26% for both architectures. These findings suggest architectural choice should align with specific deployment priorities, with multi-agent AI particularly suited for diagnostic screening of at-risk students, while single-agent models provide a cost-effective solution for general assessment.

Title: Culturally Grounded Personas in Large Language Models: Characterization and Alignment with Socio-Psychological Value Frameworks

• Authors: Candida M. Greco, Lucio La Cava, Andrea Tagarelli
• Subjects: cs.CL, cs.AI, cs.CY, cs.HC, physics.soc-ph
• Abstract URL: https://arxiv.org/abs/2601.22396
• Pdf URL: https://arxiv.org/pdf/2601.22396
• Copy Paste: [[2601.22396]] Culturally Grounded Personas in Large Language Models: Characterization and Alignment with Socio-Psychological Value Frameworks(https://arxiv.org/abs/2601.22396)
• Keywords: large language model
• Abstract: Despite the growing utility of Large Language Models (LLMs) for simulating human behavior, the extent to which these synthetic personas accurately reflect world and moral value systems across different cultural conditionings remains uncertain. This paper investigates the alignment of synthetic, culturally-grounded personas with established frameworks, specifically the World Values Survey (WVS), the Inglehart-Welzel Cultural Map, and Moral Foundations Theory. We conceptualize and produce LLM-generated personas based on a set of interpretable WVS-derived variables, and we examine the generated personas through three complementary lenses: positioning on the Inglehart-Welzel map, which unveils their interpretation reflecting stable differences across cultural conditionings; demographic-level consistency with the World Values Survey, where response distributions broadly track human group patterns; and moral profiles derived from a Moral Foundations questionnaire, which we analyze through a culture-to-morality mapping to characterize how moral responses vary across different cultural configurations. Our approach of culturally-grounded persona generation and analysis enables evaluation of cross-cultural structure and moral variation.

Title: Jailbreaks on Vision Language Model via Multimodal Reasoning

• Authors: Aarush Noheria, Yuguang Yao
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22398
• Pdf URL: https://arxiv.org/pdf/2601.22398
• Copy Paste: [[2601.22398]] Jailbreaks on Vision Language Model via Multimodal Reasoning(https://arxiv.org/abs/2601.22398)
• Keywords: defense, attack, steal
• Abstract: Vision-language models (VLMs) have become central to tasks such as visual question answering, image captioning, and text-to-image generation. However, their outputs are highly sensitive to prompt variations, which can reveal vulnerabilities in safety alignment. In this work, we present a jailbreak framework that exploits post-training Chain-of-Thought (CoT) prompting to construct stealthy prompts capable of bypassing safety filters. To further increase attack success rates (ASR), we propose a ReAct-driven adaptive noising mechanism that iteratively perturbs input images based on model feedback. This approach leverages the ReAct paradigm to refine adversarial noise in regions most likely to activate safety defenses, thereby enhancing stealth and evasion. Experimental results demonstrate that the proposed dual-strategy significantly improves ASR while maintaining naturalness in both text and visual domains.

Title: Bifocal Attention: Harmonizing Geometric and Spectral Positional Embeddings for Algorithmic Generalization

• Authors: Kanishk Awadhiya
• Subjects: cs.CL, cs.FL, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22402
• Pdf URL: https://arxiv.org/pdf/2601.22402
• Copy Paste: [[2601.22402]] Bifocal Attention: Harmonizing Geometric and Spectral Positional Embeddings for Algorithmic Generalization(https://arxiv.org/abs/2601.22402)
• Keywords: large language model
• Abstract: Rotary Positional Embeddings (RoPE) have become the standard for Large Language Models (LLMs) due to their ability to encode relative positions through geometric rotation. However, we identify a significant limitation we term ''Spectral Rigidity'': standard RoPE utilizes a fixed geometric decay ($\theta^{-i}$) optimized for local syntactic coherence, which fails to capture the long-range, periodic structures inherent in recursive logic and algorithmic reasoning. This results in a ''Structure Gap'', where models trained on shallow reasoning chains fail to extrapolate to deeper recursive steps. In this work, we introduce Bifocal Attention, an architectural paradigm that decouples positional encoding into two distinct modalities: Geometric Eyes (Standard RoPE) for precise token-level manipulation, and Spectral Eyes (Learnable Harmonic Operators) for tracking long-range recursive depth. We propose a novel training protocol, Spectral Evolution, which initializes positional frequencies as static geometric parameters but allows them to evolve via gradient descent into a harmonic basis optimized for the specific algorithmic topology of the task.

Title: Optimization, Generalization and Differential Privacy Bounds for Gradient Descent on Kolmogorov-Arnold Networks

• Authors: Puyu Wang, Junyu Zhou, Philipp Liznerski, Marius Kloft
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2601.22409
• Pdf URL: https://arxiv.org/pdf/2601.22409
• Copy Paste: [[2601.22409]] Optimization, Generalization and Differential Privacy Bounds for Gradient Descent on Kolmogorov-Arnold Networks(https://arxiv.org/abs/2601.22409)
• Keywords: privacy
• Abstract: Kolmogorov--Arnold Networks (KANs) have recently emerged as a structured alternative to standard MLPs, yet a principled theory for their training dynamics, generalization, and privacy properties remains limited. In this paper, we analyze gradient descent (GD) for training two-layer KANs and derive general bounds that characterize their training dynamics, generalization, and utility under differential privacy (DP). As a concrete instantiation, we specialize our analysis to logistic loss under an NTK-separable assumption, where we show that polylogarithmic network width suffices for GD to achieve an optimization rate of order $1/T$ and a generalization rate of order $1/n$, with $T$ denoting the number of GD iterations and $n$ the sample size. In the private setting, we characterize the noise required for $(\epsilon,\delta)$-DP and obtain a utility bound of order $\sqrt{d}/(n\epsilon)$ (with $d$ the input dimension), matching the classical lower bound for general convex Lipschitz problems. Our results imply that polylogarithmic width is not only sufficient but also necessary under differential privacy, revealing a qualitative gap between non-private (sufficiency only) and private (necessity also emerges) training regimes. Experiments further illustrate how these theoretical insights can guide practical choices, including network width selection and early stopping.

Title: Word-Centered Semantic Graphs for Interpretable Diachronic Sense Tracking

• Authors: Imene Kolli, Kai-Robin Lange, Jonas Rieger, Carsten Jentsch
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22410
• Pdf URL: https://arxiv.org/pdf/2601.22410
• Copy Paste: [[2601.22410]] Word-Centered Semantic Graphs for Interpretable Diachronic Sense Tracking(https://arxiv.org/abs/2601.22410)
• Keywords: segmentation
• Abstract: We propose an interpretable, graph-based framework for analyzing semantic shift in diachronic corpora. For each target word and time slice, we induce a word-centered semantic network that integrates distributional similarity from diachronic Skip-gram embeddings with lexical substitutability from time-specific masked language models. We identify sense-related structure by clustering the peripheral graph, align clusters across time via node overlap, and track change through cluster composition and normalized cluster mass. In an application study on a corpus of New York Times Magazine articles (1980 - 2017), we show that graph connectivity reflects polysemy dynamics and that the induced communities capture contrasting trajectories: event-driven sense replacement (trump), semantic stability with cluster over-segmentation effects (god), and gradual association shifts tied to digital communication (post). Overall, word-centered semantic graphs offer a compact and transparent representation for exploring sense evolution without relying on predefined sense inventories.

Title: MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learning

• Authors: Xunkai Li, Yuming Ai, Yinlin Zhu, Haodong Lu, Yi Zhang, Guohao Fu, Bowen Fan, Qiangqiang Dai, Rong-Hua Li, Guoren Wang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22416
• Pdf URL: https://arxiv.org/pdf/2601.22416
• Copy Paste: [[2601.22416]] MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learning(https://arxiv.org/abs/2601.22416)
• Keywords: privacy, robust, federate
• Abstract: Multimodal-attributed graphs (MMAGs) provide a unified framework for modeling complex relational data by integrating heterogeneous modalities with graph structures. While centralized learning has shown promising performance, MMAGs in real-world applications are often distributed across isolated platforms and cannot be shared due to privacy concerns or commercial constraints. Federated graph learning (FGL) offers a natural solution for collaborative training under such settings; however, existing studies largely focus on single-modality graphs and do not adequately address the challenges unique to multimodal federated graph learning (MMFGL). To bridge this gap, we present MM-OpenFGL, the first comprehensive benchmark that systematically formalizes the MMFGL paradigm and enables rigorous evaluation. MM-OpenFGL comprises 19 multimodal datasets spanning 7 application domains, 8 simulation strategies capturing modality and topology variations, 6 downstream tasks, and 57 state-of-the-art methods implemented through a modular API. Extensive experiments investigate MMFGL from the perspectives of necessity, effectiveness, robustness, and efficiency, offering valuable insights for future research on MMFGL.

Title: CoDCL: Counterfactual Data Augmentation Contrastive Learning for Continuous-Time Dynamic Network Link Prediction

• Authors: Hantong Feng, Yonggang Wu, Duxin Chen, Wenwu Yu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22427
• Pdf URL: https://arxiv.org/pdf/2601.22427
• Copy Paste: [[2601.22427]] CoDCL: Counterfactual Data Augmentation Contrastive Learning for Continuous-Time Dynamic Network Link Prediction(https://arxiv.org/abs/2601.22427)
• Keywords: robust
• Abstract: The rapid growth and continuous structural evolution of dynamic networks make effective predictions increasingly challenging. To enable prediction models to adapt to complex temporal environments, they need to be robust to emerging structural changes. We propose a dynamic network learning framework CoDCL, which combines counterfactual data augmentation with contrastive learning to address this this http URL, we devise a comprehensive strategy to generate high-quality counterfactual data, combining a dynamic treatments design with efficient structural neighborhood exploration to quantify the temporal changes in interaction this http URL, the entire CoDCL is designed as a plug-and-play universal module that can be seamlessly integrated into various existing temporal graph models without requiring architectural this http URL experiments on multiple real-world datasets demonstrate that CoDCL significantly gains state-of-the-art baseline models in the field of dynamic networks, confirming the critical role of integrating counterfactual data augmentation into dynamic representation learning.

Title: Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspective

• Authors: Georgi Ganev, Emiliano De Cristofaro
• Subjects: cs.CR, cs.CY, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22434
• Pdf URL: https://arxiv.org/pdf/2601.22434
• Copy Paste: [[2601.22434]] Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspective(https://arxiv.org/abs/2601.22434)
• Keywords: privacy, protect, attack, robust, generative
• Abstract: Training generative machine learning models to produce synthetic tabular data has become a popular approach for enhancing privacy in data sharing. As this typically involves processing sensitive personal information, releasing either the trained model or generated synthetic datasets can still pose privacy risks. Yet, recent research, commercial deployments, and privacy regulations like the General Data Protection Regulation (GDPR) largely assess anonymity at the level of an individual dataset. In this paper, we rethink anonymity claims about synthetic data from a model-centric perspective and argue that meaningful assessments must account for the capabilities and properties of the underlying generative model and be grounded in state-of-the-art privacy attacks. This perspective better reflects real-world products and deployments, where trained models are often readily accessible for interaction or querying. We interpret the GDPR's definitions of personal data and anonymization under such access assumptions to identify the types of identifiability risks that must be mitigated and map them to privacy attacks across different threat settings. We then argue that synthetic data techniques alone do not ensure sufficient anonymization. Finally, we compare the two mechanisms most commonly used alongside synthetic data -- Differential Privacy (DP) and Similarity-based Privacy Metrics (SBPMs) -- and argue that while DP can offer robust protections against identifiability risks, SBPMs lack adequate safeguards. Overall, our work connects regulatory notions of identifiability with model-centric privacy attacks, enabling more responsible and trustworthy regulatory assessment of synthetic data systems by researchers, practitioners, and policymakers.

Title: Large Language Model Agents Are Not Always Faithful Self-Evolvers

• Authors: Weixiang Zhao, Yingshuo Wang, Yichen Zhang, Yang Deng, Yanyan Zhao, Wanxiang Che, Bing Qin, Ting Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22436
• Pdf URL: https://arxiv.org/pdf/2601.22436
• Copy Paste: [[2601.22436]] Large Language Model Agents Are Not Always Faithful Self-Evolvers(https://arxiv.org/abs/2601.22436)
• Keywords: large language model
• Abstract: Self-evolving large language model (LLM) agents continually improve by accumulating and reusing past experience, yet it remains unclear whether they faithfully rely on that experience to guide their behavior. We present the first systematic investigation of experience faithfulness, the causal dependence of an agent's decisions on the experience it is given, in self-evolving LLM agents. Using controlled causal interventions on both raw and condensed forms of experience, we comprehensively evaluate four representative frameworks across 10 LLM backbones and 9 environments. Our analysis uncovers a striking asymmetry: while agents consistently depend on raw experience, they often disregard or misinterpret condensed experience, even when it is the only experience provided. This gap persists across single- and multi-agent configurations and across backbone scales. We trace its underlying causes to three factors: the semantic limitations of condensed content, internal processing biases that suppress experience, and task regimes where pretrained priors already suffice. These findings challenge prevailing assumptions about self-evolving methods and underscore the need for more faithful and reliable approaches to experience integration.

Title: Weak Diffusion Priors Can Still Achieve Strong Inverse-Problem Performance

• Authors: Jing Jia, Wei Yuan, Sifan Liu, Liyue Shen, Guanyang Wang
• Subjects: cs.LG, cs.CV, stat.CO, stat.ML
• Abstract URL: https://arxiv.org/abs/2601.22443
• Pdf URL: https://arxiv.org/pdf/2601.22443
• Copy Paste: [[2601.22443]] Weak Diffusion Priors Can Still Achieve Strong Inverse-Problem Performance(https://arxiv.org/abs/2601.22443)
• Keywords: robust, diffusion
• Abstract: Can a diffusion model trained on bedrooms recover human faces? Diffusion models are widely used as priors for inverse problems, but standard approaches usually assume a high-fidelity model trained on data that closely match the unknown signal. In practice, one often must use a mismatched or low-fidelity diffusion prior. Surprisingly, these weak priors often perform nearly as well as full-strength, in-domain baselines. We study when and why inverse solvers are robust to weak diffusion priors. Through extensive experiments, we find that weak priors succeed when measurements are highly informative (e.g., many observed pixels), and we identify regimes where they fail. Our theory, based on Bayesian consistency, gives conditions under which high-dimensional measurements make the posterior concentrate near the true signal. These results provide a principled justification on when weak diffusion priors can be used reliably.

Title: Automating Forecasting Question Generation and Resolution for AI Evaluation

• Authors: Nikos I. Bosse, Peter Mühlbacher, Jack Wildman, Lawrence Phillips, Dan Schwarz
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22444
• Pdf URL: https://arxiv.org/pdf/2601.22444
• Copy Paste: [[2601.22444]] Automating Forecasting Question Generation and Resolution for AI Evaluation(https://arxiv.org/abs/2601.22444)
• Keywords: robust
• Abstract: Forecasting future events is highly valuable in decision-making and is a robust measure of general intelligence. As forecasting is probabilistic, developing and evaluating AI forecasters requires generating large numbers of diverse and difficult questions, and accurately resolving them. Previous efforts to automate this laborious work relied on recurring data sources (e.g., weather, stocks), limiting diversity and utility. In this work, we present a system for generating and resolving high-quality forecasting questions automatically and at scale using LLM-powered web research agents. We use this system to generate 1499 diverse, real-world forecasting questions, and to resolve them several months later. We estimate that our system produces verifiable, unambiguous questions approximately 96% of the time, exceeding the rate of Metaculus, a leading human-curated forecasting platform. We also find that our system resolves questions at approximately 95% accuracy. We verify that forecasting agents powered by more intelligent LLMs perform better on these questions (Brier score of 0.134 for Gemini 3 Pro, 0.149 for GPT-5, and 0.179 for Gemini 2.5 Flash). Finally, we demonstrate how our system can be leveraged to directly improve forecasting, by evaluating a question decomposition strategy on a generated question set, yielding a significant improvement in Brier scores (0.132 vs. 0.141).

Title: Beyond Activation Patterns: A Weight-Based Out-of-Context Explanation of Sparse Autoencoder Features

• Authors: Yiting Liu, Zhi-Hong Deng
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22447
• Pdf URL: https://arxiv.org/pdf/2601.22447
• Copy Paste: [[2601.22447]] Beyond Activation Patterns: A Weight-Based Out-of-Context Explanation of Sparse Autoencoder Features(https://arxiv.org/abs/2601.22447)
• Keywords: interpretability
• Abstract: Sparse autoencoders (SAEs) have emerged as a powerful technique for decomposing language model representations into interpretable features. Current interpretation methods infer feature semantics from activation patterns, but overlook that features are trained to reconstruct activations that serve computational roles in the forward pass. We introduce a novel weight-based interpretation framework that measures functional effects through direct weight interactions, requiring no activation data. Through three experiments on Gemma-2 and Llama-3.1 models, we demonstrate that (1) 1/4 of features directly predict output tokens, (2) features actively participate in attention mechanisms with depth-dependent structure, and (3) semantic and non-semantic feature populations exhibit distinct distribution profiles in attention circuits. Our analysis provides the missing out-of-context half of SAE feature interpretability.

Title: Tuning the Implicit Regularizer of Masked Diffusion Language Models: Enhancing Generalization via Insights from $k$-Parity

• Authors: Jianhao Huang, Baharan Mirzasoleiman
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22450
• Pdf URL: https://arxiv.org/pdf/2601.22450
• Copy Paste: [[2601.22450]] Tuning the Implicit Regularizer of Masked Diffusion Language Models: Enhancing Generalization via Insights from $k$-Parity(https://arxiv.org/abs/2601.22450)
• Keywords: diffusion, generative
• Abstract: Masked Diffusion Language Models have recently emerged as a powerful generative paradigm, yet their generalization properties remain understudied compared to their auto-regressive counterparts. In this work, we investigate these properties within the setting of the $k$-parity problem (computing the XOR sum of $k$ relevant bits), where neural networks typically exhibit grokking -- a prolonged plateau of chance-level performance followed by sudden generalization. We theoretically decompose the Masked Diffusion (MD) objective into a Signal regime which drives feature learning, and a Noise regime which serves as an implicit regularizer. By training nanoGPT using MD objective on the $k$-parity problem, we demonstrate that MD objective fundamentally alters the learning landscape, enabling rapid and simultaneous generalization without experiencing grokking. Furthermore, we leverage our theoretical insights to optimize the distribution of the mask probability in the MD objective. Our method significantly improves perplexity for 50M-parameter models and achieves superior results across both pre-training from scratch and supervised fine-tuning. Specifically, we observe performance gains peaking at $8.8\%$ and $5.8\%$, respectively, on 8B-parameter models, confirming the scalability and effectiveness of our framework in large-scale masked diffusion language model regimes.

Title: Temporal Graph Pattern Machine

• Authors: Yijun Ma, Zehong Wang, Weixiang Sun, Yanfang Ye
• Subjects: cs.LG, cs.AI, cs.SI
• Abstract URL: https://arxiv.org/abs/2601.22454
• Pdf URL: https://arxiv.org/pdf/2601.22454
• Copy Paste: [[2601.22454]] Temporal Graph Pattern Machine(https://arxiv.org/abs/2601.22454)
• Keywords: transformer
• Abstract: Temporal graph learning is pivotal for deciphering dynamic systems, where the core challenge lies in explicitly modeling the underlying evolving patterns that govern network transformation. However, prevailing methods are predominantly task-centric and rely on restrictive assumptions -- such as short-term dependency modeling, static neighborhood semantics, and retrospective time usage. These constraints hinder the discovery of transferable temporal evolution mechanisms. To address this, we propose the Temporal Graph Pattern Machine (TGPM), a foundation framework that shifts the focus toward directly learning generalized evolving patterns. TGPM conceptualizes each interaction as an interaction patch synthesized via temporally-biased random walks, thereby capturing multi-scale structural semantics and long-range dependencies that extend beyond immediate neighborhoods. These patches are processed by a Transformer-based backbone designed to capture global temporal regularities while adapting to context-specific interaction dynamics. To further empower the model, we introduce a suite of self-supervised pre-training tasks -- specifically masked token modeling and next-time prediction -- to explicitly encode the fundamental laws of network evolution. Extensive experiments show that TGPM consistently achieves state-of-the-art performance in both transductive and inductive link prediction, demonstrating exceptional cross-domain transferability.

Title: ScribbleSense: Generative Scribble-Based Texture Editing with Intent Prediction

• Authors: Yudi Zhang, Yeming Geng, Lei Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22455
• Pdf URL: https://arxiv.org/pdf/2601.22455
• Copy Paste: [[2601.22455]] ScribbleSense: Generative Scribble-Based Texture Editing with Intent Prediction(https://arxiv.org/abs/2601.22455)
• Keywords: generative, large language model
• Abstract: Interactive 3D model texture editing presents enhanced opportunities for creating 3D assets, with freehand drawing style offering the most intuitive experience. However, existing methods primarily support sketch-based interactions for outlining, while the utilization of coarse-grained scribble-based interaction remains limited. Furthermore, current methodologies often encounter challenges due to the abstract nature of scribble instructions, which can result in ambiguous editing intentions and unclear target semantic locations. To address these issues, we propose ScribbleSense, an editing method that combines multimodal large language models (MLLMs) and image generation models to effectively resolve these challenges. We leverage the visual capabilities of MLLMs to predict the editing intent behind the scribbles. Once the semantic intent of the scribble is discerned, we employ globally generated images to extract local texture details, thereby anchoring local semantics and alleviating ambiguities concerning the target semantic locations. Experimental results indicate that our method effectively leverages the strengths of MLLMs, achieving state-of-the-art interactive editing performance for scribble-based texture editing.

Title: Machine Unlearning in Low-Dimensional Feature Subspace

• Authors: Kun Fang, Qinghua Tao, Junxu Liu, Yaxin Xiao, Qingqing Ye, Jian Sun, Haibo Hu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22456
• Pdf URL: https://arxiv.org/pdf/2601.22456
• Copy Paste: [[2601.22456]] Machine Unlearning in Low-Dimensional Feature Subspace(https://arxiv.org/abs/2601.22456)
• Keywords: privacy
• Abstract: Machine Unlearning (MU) aims at removing the influence of specific data from a pretrained model while preserving performance on the remaining data. In this work, a novel perspective for MU is presented upon low-dimensional feature subspaces, which gives rise to the potentials of separating the remaining and forgetting data herein. This separability motivates our LOFT, a method that proceeds unlearning in a LOw-dimensional FeaTure subspace from the pretrained model skithrough principal projections, which are optimized to maximally capture the information of the remaining data and meanwhile diminish that of the forgetting data. In training, LOFT simply optimizes a small-size projection matrix flexibly plugged into the pretrained model, and only requires one-shot feature fetching from the pretrained backbone instead of repetitively accessing the raw data. Hence, LOFT mitigates two critical issues in mainstream MU methods, i.e., the privacy leakage risk from massive data reload and the inefficiency of updates to the entire pretrained model. Extensive experiments validate the significantly lower computational overhead and superior unlearning performance of LOFT across diverse models, datasets, tasks, and applications. Code is anonymously available at this https URL.

Title: EvoEGF-Mol: Evolving Exponential Geodesic Flow for Structure-based Drug Design

• Authors: Yaowei Jin, Junjie Wang, Cheng Cao, Penglei Wang, Duo An, Qian Shi
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22466
• Pdf URL: https://arxiv.org/pdf/2601.22466
• Copy Paste: [[2601.22466]] EvoEGF-Mol: Evolving Exponential Geodesic Flow for Structure-based Drug Design(https://arxiv.org/abs/2601.22466)
• Keywords: generative
• Abstract: Structure-Based Drug Design (SBDD) aims to discover bioactive ligands. Conventional approaches construct probability paths separately in Euclidean and probabilistic spaces for continuous atomic coordinates and discrete chemical categories, leading to a mismatch with the underlying statistical manifolds. We address this issue from an information-geometric perspective by modeling molecules as composite exponential-family distributions and defining generative flows along exponential geodesics under the Fisher-Rao metric. To avoid the instantaneous trajectory collapse induced by geodesics directly targeting Dirac distributions, we propose Evolving Exponential Geodesic Flow for SBDD (EvoEGF-Mol), which replaces static Dirac targets with dynamically concentrating distributions, ensuring stable training via a progressive-parameter-refinement architecture. Our model approaches a reference-level PoseBusters passing rate (93.4%) on CrossDock, demonstrating remarkable geometric precision and interaction fidelity, while outperforming baselines on real-world MolGenBench tasks by recovering bioactive scaffolds and generating candidates that meet established MedChem filters.

Title: Training-Free Representation Guidance for Diffusion Models with a Representation Alignment Projector

• Authors: Wenqiang Zu, Shenghao Xie, Bo Lei, Lei Ma
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22468
• Pdf URL: https://arxiv.org/pdf/2601.22468
• Copy Paste: [[2601.22468]] Training-Free Representation Guidance for Diffusion Models with a Representation Alignment Projector(https://arxiv.org/abs/2601.22468)
• Keywords: diffusion, transformer, generative
• Abstract: Recent progress in generative modeling has enabled high-quality visual synthesis with diffusion-based frameworks, supporting controllable sampling and large-scale training. Inference-time guidance methods such as classifier-free and representative guidance enhance semantic alignment by modifying sampling dynamics; however, they do not fully exploit unsupervised feature representations. Although such visual representations contain rich semantic structure, their integration during generation is constrained by the absence of ground-truth reference images at inference. This work reveals semantic drift in the early denoising stages of diffusion transformers, where stochasticity results in inconsistent alignment even under identical conditioning. To mitigate this issue, we introduce a guidance scheme using a representation alignment projector that injects representations predicted by a projector into intermediate sampling steps, providing an effective semantic anchor without modifying the model architecture. Experiments on SiTs and REPAs show notable improvements in class-conditional ImageNet synthesis, achieving substantially lower FID scores; for example, REPA-XL/2 improves from 5.9 to 3.3, and the proposed method outperforms representative guidance when applied to SiT models. The approach further yields complementary gains when combined with classifier-free guidance, demonstrating enhanced semantic coherence and visual fidelity. These results establish representation-informed diffusion sampling as a practical strategy for reinforcing semantic preservation and image consistency.

Title: Unrewarded Exploration in Large Language Models Reveals Latent Learning from Psychology

• Authors: Jian Xiong, Jingbo Zhou, Zihan Zhou, Yixiong Xiao, Le Zhang, Jingyong Ye, Rui Qian, Yang Zhou, Dejing Dou
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22474
• Pdf URL: https://arxiv.org/pdf/2601.22474
• Copy Paste: [[2601.22474]] Unrewarded Exploration in Large Language Models Reveals Latent Learning from Psychology(https://arxiv.org/abs/2601.22474)
• Keywords: large language model
• Abstract: Latent learning, classically theorized by Tolman, shows that biological agents (e.g., rats) can acquire internal representations of their environment without rewards, enabling rapid adaptation once rewards are introduced. In contrast, from a cognitive science perspective, reward learning remains overly dependent on external feedback, limiting flexibility and generalization. Although recent advances in the reasoning capabilities of large language models (LLMs), such as OpenAI-o1 and DeepSeek-R1, mark a significant breakthrough, these models still rely primarily on reward-centric reinforcement learning paradigms. Whether and how the well-established phenomenon of latent learning in psychology can inform or emerge within LLMs' training remains largely unexplored. In this work, we present novel findings from our experiments that LLMs also exhibit the latent learning dynamics. During an initial phase of unrewarded exploration, LLMs display modest performance improvements, as this phase allows LLMs to organize task-relevant knowledge without being constrained by reward-driven biases, and performance is further enhanced once rewards are introduced. LLMs post-trained under this two-stage exploration regime ultimately achieve higher competence than those post-trained with reward-based reinforcement learning throughout. Beyond these empirical observations, we also provide theoretical analyses for our experiments explaining why unrewarded exploration yields performance gains, offering a mechanistic account of these dynamics. Specifically, we conducted extensive experiments across multiple model families and diverse task domains to establish the existence of the latent learning dynamics in LLMs.

Title: Transform-Augmented GRPO Improves Pass@k

• Authors: Khiem Le, Youssef Mroueh, Phuc Nguyen, Chi-Heng Lin, Shangqian Gao, Ting Hua, Nitesh V. Chawla
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22478
• Pdf URL: https://arxiv.org/pdf/2601.22478
• Copy Paste: [[2601.22478]] Transform-Augmented GRPO Improves Pass@k(https://arxiv.org/abs/2601.22478)
• Keywords: large language model
• Abstract: Large language models trained via next-token prediction are fundamentally pattern-matchers: sensitive to superficial phrasing variations even when the underlying problem is identical. Group Relative Policy Optimization (GRPO) was designed to improve reasoning, but in fact it worsens this situation through two failure modes: diversity collapse, where training amplifies a single solution strategy while ignoring alternatives of gradient signal, and gradient diminishing, where a large portion of questions yield zero gradients because all rollouts receive identical rewards. We propose TA-GRPO (Transform-Augmented GRPO), which generates semantically equivalent transformed variants of each question (via paraphrasing, variable renaming, and format changes) and computes advantages by pooling rewards across the entire group. This pooled computation ensures mixed rewards even when the original question is too easy or too hard, while training on diverse phrasings promotes multiple solution strategies. We provide theoretical justification showing that TA-GRPO reduces zero-gradient probability and improves generalization via reduced train-test distribution shift. Experiments on mathematical reasoning benchmarks show consistent Pass@k improvements, with gains up to 9.84 points on competition math (AMC12, AIME24) and 5.05 points on out-of-distribution scientific reasoning (GPQA-Diamond).

Title: Head-Aware Visual Cropping: Enhancing Fine-Grained VQA with Attention-Guided Subimage

• Authors: Junfei Xie, Peng Pan, Xulong Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22483
• Pdf URL: https://arxiv.org/pdf/2601.22483
• Copy Paste: [[2601.22483]] Head-Aware Visual Cropping: Enhancing Fine-Grained VQA with Attention-Guided Subimage(https://arxiv.org/abs/2601.22483)
• Keywords: large language model
• Abstract: Multimodal Large Language Models (MLLMs) show strong performance in Visual Question Answering (VQA) but remain limited in fine-grained reasoning due to low-resolution inputs and noisy attention aggregation. We propose \textbf{Head Aware Visual Cropping (HAVC)}, a training-free method that improves visual grounding by leveraging a selectively refined subset of attention heads. HAVC first filters heads through an OCR-based diagnostic task, ensuring that only those with genuine grounding ability are retained. At inference, these heads are further refined using spatial entropy for stronger spatial concentration and gradient sensitivity for predictive contribution. The fused signals produce a reliable Visual Cropping Guidance Map, which highlights the most task-relevant region and guides the cropping of a subimage subsequently provided to the MLLM together with the image-question pair. Extensive experiments on multiple fine-grained VQA benchmarks demonstrate that HAVC consistently outperforms state-of-the-art cropping strategies, achieving more precise localization, stronger visual grounding, providing a simple yet effective strategy for enhancing precision in MLLMs.

Title: Mitigating Cognitive Inertia in Large Reasoning Models via Latent Spike Steering

• Authors: Seojin Lee, ByeongJeong Kim, Hwanhee Lee
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22484
• Pdf URL: https://arxiv.org/pdf/2601.22484
• Copy Paste: [[2601.22484]] Mitigating Cognitive Inertia in Large Reasoning Models via Latent Spike Steering(https://arxiv.org/abs/2601.22484)
• Keywords: robust
• Abstract: While Large Reasoning Models (LRMs) have achieved remarkable performance by scaling test-time compute, they frequently suffer from Cognitive Inertia, a failure pattern manifesting as either overthinking (inertia of motion) or reasoning rigidity (inertia of direction). Existing detection methods, typically relying on superficial textual heuristics like self-correction tokens, often fail to capture the model's unvoiced internal conflicts. To address this, we propose STARS (Spike-Triggered Adaptive Reasoning Steering), a training-free framework designed to rectify cognitive inertia by monitoring latent dynamics. STARS identifies Cognitive Pivots-critical moments of reasoning transition-by detecting distinct L2 distance spikes in the hidden states. Upon detection, the framework employs geometric trajectory analysis to diagnose the structural nature of the transition and injects state-aware language cues to steer the model in real-time. Our experiments across diverse benchmarks confirm that STARS efficiently curtails redundant loops while improving accuracy through the adaptive correction of erroneous trajectories. STARS offers a robust, unsupervised mechanism to optimize the reasoning process of LRMs without requiring additional fine-tuning.

Title: FraudShield: Knowledge Graph Empowered Defense for LLMs against Fraud Attacks

• Authors: Naen Xu, Jinghuai Zhang, Ping He, Chunyi Zhou, Jun Wang, Zhihui Fu, Tianyu Du, Zhaoxiang Wang, Shouling Ji
• Subjects: cs.CR, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22485
• Pdf URL: https://arxiv.org/pdf/2601.22485
• Copy Paste: [[2601.22485]] FraudShield: Knowledge Graph Empowered Defense for LLMs against Fraud Attacks(https://arxiv.org/abs/2601.22485)
• Keywords: secure, protect, defense, attack, interpretability, large language model
• Abstract: Large language models (LLMs) have been widely integrated into critical automated workflows, including contract review and job application processes. However, LLMs are susceptible to manipulation by fraudulent information, which can lead to harmful outcomes. Although advanced defense methods have been developed to address this issue, they often exhibit limitations in effectiveness, interpretability, and generalizability, particularly when applied to LLM-based applications. To address these challenges, we introduce FraudShield, a novel framework designed to protect LLMs from fraudulent content by leveraging a comprehensive analysis of fraud tactics. Specifically, FraudShield constructs and refines a fraud tactic-keyword knowledge graph to capture high-confidence associations between suspicious text and fraud techniques. The structured knowledge graph augments the original input by highlighting keywords and providing supporting evidence, guiding the LLM toward more secure responses. Extensive experiments show that FraudShield consistently outperforms state-of-the-art defenses across four mainstream LLMs and five representative fraud types, while also offering interpretable clues for the model's generations.

Title: Elastic Spectral State Space Models for Budgeted Inference

• Authors: Dachuan Song, Xuan Wang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22488
• Pdf URL: https://arxiv.org/pdf/2601.22488
• Copy Paste: [[2601.22488]] Elastic Spectral State Space Models for Budgeted Inference(https://arxiv.org/abs/2601.22488)
• Keywords: transformer
• Abstract: Foundation models are typically trained at a fixed computational capacity, while real-world applications require deployment across platforms with different resource constraints. Current approaches usually rely on training families of model variants or model distillation, which requires additional training and supports only a pre-selected set of sizes rather than fine-grained adaptation at runtime. In this paper, we propose Elastic Spectral State Space Models (ES-SSM), which require only one-time training at full capacity, but can be directly truncated into arbitrary scales for budgeted, runtime inference without retraining. Our ES-SSM builds on Hankel spectral filtering over a state space model (SSM), coupled with a lightweight input-adaptive gate trained under randomized spectral budgets. Using a shared masked normalization rule over the ordered spectral channels, we encourage predictive capability to concentrate in low-index components, while higher-index components act primarily as refinement. We test our algorithm across long-sequence benchmarks spanning text, logic, retrieval, vision, and audio. We demonstrate that a single ES-SSM model trained once can be truncated to provide competitive performance compared with modern Transformer and SSM baselines at similar parameter scales. Furthermore, by testing under various runtime budgets, we observe smooth and stable budget-performance curves over a wide range of truncation levels.

Title: SSL: Sweet Spot Learning for Differentiated Guidance in Agentic Optimization

• Authors: Jinyang Wu, Changpeng Yang, Yuhao Shen, Fangzhi Xu, Bolin Ni, Chonghua Liao, Yuchen Liu, Hongzhen Wang, Shuai Nie, Shuai Zhang, Haoran Luo, Jiaming Xu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22491
• Pdf URL: https://arxiv.org/pdf/2601.22491
• Copy Paste: [[2601.22491]] SSL: Sweet Spot Learning for Differentiated Guidance in Agentic Optimization(https://arxiv.org/abs/2601.22491)
• Keywords: robust
• Abstract: Reinforcement learning with verifiable rewards has emerged as a powerful paradigm for training intelligent agents. However, existing methods typically employ binary rewards that fail to capture quality differences among trajectories achieving identical outcomes, thereby overlooking potential diversity within the solution space. Inspired by the ``sweet spot'' concept in tennis-the racket's core region that produces optimal hitting effects, we introduce \textbf{S}weet \textbf{S}pot \textbf{L}earning (\textbf{SSL}), a novel framework that provides differentiated guidance for agent optimization. SSL follows a simple yet effective principle: progressively amplified, tiered rewards guide policies toward the sweet-spot region of the solution space. This principle naturally adapts across diverse tasks: visual perception tasks leverage distance-tiered modeling to reward proximity, while complex reasoning tasks reward incremental progress toward promising solutions. We theoretically demonstrate that SSL preserves optimal solution ordering and enhances the gradient signal-to-noise ratio, thereby fostering more directed optimization. Extensive experiments across GUI perception, short/long-term planning, and complex reasoning tasks show consistent improvements over strong baselines on 12 benchmarks, achieving up to 2.5X sample efficiency gains and effective cross-task transferability. Our work establishes SSL as a general principle for training capable and robust agents.

Title: PromptMAD: Cross-Modal Prompting for Multi-Class Visual Anomaly Localization

• Authors: Duncan McCain, Hossein Kashiani, Fatemeh Afghah
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22492
• Pdf URL: https://arxiv.org/pdf/2601.22492
• Copy Paste: [[2601.22492]] PromptMAD: Cross-Modal Prompting for Multi-Class Visual Anomaly Localization(https://arxiv.org/abs/2601.22492)
• Keywords: diffusion, transformer
• Abstract: Visual anomaly detection in multi-class settings poses significant challenges due to the diversity of object categories, the scarcity of anomalous examples, and the presence of camouflaged defects. In this paper, we propose PromptMAD, a cross-modal prompting framework for unsupervised visual anomaly detection and localization that integrates semantic guidance through vision-language alignment. By leveraging CLIP-encoded text prompts describing both normal and anomalous class-specific characteristics, our method enriches visual reconstruction with semantic context, improving the detection of subtle and textural anomalies. To further address the challenge of class imbalance at the pixel level, we incorporate Focal loss function, which emphasizes hard-to-detect anomalous regions during training. Our architecture also includes a supervised segmentor that fuses multi-scale convolutional features with Transformer-based spatial attention and diffusion iterative refinement, yielding precise and high-resolution anomaly maps. Extensive experiments on the MVTec-AD dataset demonstrate that our method achieves state-of-the-art pixel-level performance, improving mean AUC to 98.35% and AP to 66.54%, while maintaining efficiency across diverse categories.

Title: Gradual Fine-Tuning for Flow Matching Models

• Authors: Gudrun Thorkelsdottir, Arindam Banerjee
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22495
• Pdf URL: https://arxiv.org/pdf/2601.22495
• Copy Paste: [[2601.22495]] Gradual Fine-Tuning for Flow Matching Models(https://arxiv.org/abs/2601.22495)
• Keywords: generative
• Abstract: Fine-tuning flow matching models is a central challenge in settings with limited data, evolving distributions, or strict efficiency demands, where unconstrained fine-tuning can erode the accuracy and efficiency gains learned during pretraining. Prior work has produced theoretical guarantees and empirical advances for reward-based fine-tuning formulations, but these methods often impose restrictions on permissible drift structure or training techniques. In this work, we propose Gradual Fine-Tuning (GFT), a principled framework for fine-tuning flow-based generative models when samples from the target distribution are available. For stochastic flows, GFT defines a temperature-controlled sequence of intermediate objectives that smoothly interpolate between the pretrained and target drifts, approaching the true target as the temperature approaches zero. We prove convergence results for both marginal and conditional GFT objectives, enabling the use of suitable (e.g., optimal transport) couplings during GFT while preserving correctness. Empirically, GFT improves convergence stability and shortens probability paths, resulting in faster inference, while maintaining generation quality comparable to standard fine-tuning. Our results position GFT as a theoretically grounded and practically effective alternative for scalable adaptation of flow matching models under distribution shift.

Title: MIRRORTALK: Forging Personalized Avatars Via Disentangled Style and Hierarchical Motion Control

• Authors: Renjie Lu, Xulong Zhang, Xiaoyang Qu, Jianzong Wang, Shangfei Wang
• Subjects: cs.CV, cs.SD
• Abstract URL: https://arxiv.org/abs/2601.22501
• Pdf URL: https://arxiv.org/pdf/2601.22501
• Copy Paste: [[2601.22501]] MIRRORTALK: Forging Personalized Avatars Via Disentangled Style and Hierarchical Motion Control(https://arxiv.org/abs/2601.22501)
• Keywords: diffusion, generative
• Abstract: Synthesizing personalized talking faces that uphold and highlight a speaker's unique style while maintaining lip-sync accuracy remains a significant challenge. A primary limitation of existing approaches is the intrinsic confounding of speaker-specific talking style and semantic content within facial motions, which prevents the faithful transfer of a speaker's unique persona to arbitrary speech. In this paper, we propose MirrorTalk, a generative framework based on a conditional diffusion model, combined with a Semantically-Disentangled Style Encoder (SDSE) that can distill pure style representations from a brief reference video. To effectively utilize this representation, we further introduce a hierarchical modulation strategy within the diffusion process. This mechanism guides the synthesis by dynamically balancing the contributions of audio and style features across distinct facial regions, ensuring both precise lip-sync accuracy and expressive full-face dynamics. Extensive experiments demonstrate that MirrorTalk achieves significant improvements over state-of-the-art methods in terms of lip-sync accuracy and personalization preservation.

Title: DreamVAR: Taming Reinforced Visual Autoregressive Model for High-Fidelity Subject-Driven Image Generation

• Authors: Xin Jiang, Jingwen Chen, Yehao Li, Yingwei Pan, Kezhou Chen, Zechao Li, Ting Yao, Tao Mei
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22507
• Pdf URL: https://arxiv.org/pdf/2601.22507
• Copy Paste: [[2601.22507]] DreamVAR: Taming Reinforced Visual Autoregressive Model for High-Fidelity Subject-Driven Image Generation(https://arxiv.org/abs/2601.22507)
• Keywords: diffusion
• Abstract: Recent advances in subject-driven image generation using diffusion models have attracted considerable attention for their remarkable capabilities in producing high-quality images. Nevertheless, the potential of Visual Autoregressive (VAR) models, despite their unified architecture and efficient inference, remains underexplored. In this work, we present DreamVAR, a novel framework for subject-driven image synthesis built upon a VAR model that employs next-scale prediction. Technically, multi-scale features of the reference subject are first extracted by a visual tokenizer. Instead of interleaving these conditional features with target image tokens across scales, our DreamVAR pre-fills the full subject feature sequence prior to predicting target image tokens. This design simplifies autoregressive dependencies and mitigates the train-test discrepancy in multi-scale conditioning scenario within the VAR paradigm. DreamVAR further incorporates reinforcement learning to jointly enhance semantic alignment and subject consistency. Extensive experiments demonstrate that DreamVAR achieves superior appearance preservation compared to leading diffusion-based methods.

Title: Shattered Compositionality: Counterintuitive Learning Dynamics of Transformers for Arithmetic

• Authors: Xingyu Zhao, Darsh Sharma, Rheeya Uppaal, Yiqiao Zhong
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22510
• Pdf URL: https://arxiv.org/pdf/2601.22510
• Copy Paste: [[2601.22510]] Shattered Compositionality: Counterintuitive Learning Dynamics of Transformers for Arithmetic(https://arxiv.org/abs/2601.22510)
• Keywords: robust, transformer, large language model
• Abstract: Large language models (LLMs) often exhibit unexpected errors or unintended behavior, even at scale. While recent work reveals the discrepancy between LLMs and humans in skill compositions, the learning dynamics of skill compositions and the underlying cause of non-human behavior remain elusive. In this study, we investigate the mechanism of learning dynamics by training transformers on synthetic arithmetic tasks. Through extensive ablations and fine-grained diagnostic metrics, we discover that transformers do not reliably build skill compositions according to human-like sequential rules. Instead, they often acquire skills in reverse order or in parallel, which leads to unexpected mixing errors especially under distribution shifts--a phenomenon we refer to as shattered compositionality. To explain these behaviors, we provide evidence that correlational matching to the training data, rather than causal or procedural composition, shapes learning dynamics. We further show that shattered compositionality persists in modern LLMs and is not mitigated by pure model scaling or scratchpad-based reasoning. Our results reveal a fundamental mismatch between a model's learning behavior and desired skill compositions, with implications for reasoning reliability, out-of-distribution robustness, and alignment.

Title: DNA: Uncovering Universal Latent Forgery Knowledge

• Authors: Jingtong Dou, Chuancheng Shi, Yemin Wang, Shiming Guo, Anqi Yi, Wenhua Wu, Li Zhang, Fei Shen, Tat-Seng Chua
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22515
• Pdf URL: https://arxiv.org/pdf/2601.22515
• Copy Paste: [[2601.22515]] DNA: Uncovering Universal Latent Forgery Knowledge(https://arxiv.org/abs/2601.22515)
• Keywords: robust, generative
• Abstract: As generative AI achieves hyper-realism, superficial artifact detection has become obsolete. While prevailing methods rely on resource-intensive fine-tuning of black-box backbones, we propose that forgery detection capability is already encoded within pre-trained models rather than requiring end-to-end retraining. To elicit this intrinsic capability, we propose the discriminative neural anchors (DNA) framework, which employs a coarse-to-fine excavation mechanism. First, by analyzing feature decoupling and attention distribution shifts, we pinpoint critical intermediate layers where the focus of the model logically transitions from global semantics to local anomalies. Subsequently, we introduce a triadic fusion scoring metric paired with a curvature-truncation strategy to strip away semantic redundancy, precisely isolating the forgery-discriminative units (FDUs) inherently imprinted with sensitivity to forgery traces. Moreover, we introduce HIFI-Gen, a high-fidelity synthetic benchmark built upon the very latest models, to address the lag in existing datasets. Experiments demonstrate that by solely relying on these anchors, DNA achieves superior detection performance even under few-shot conditions. Furthermore, it exhibits remarkable robustness across diverse architectures and against unseen generative models, validating that waking up latent neurons is more effective than extensive fine-tuning.

Title: SCOPE-PD: Explainable AI on Subjective and Clinical Objective Measurements of Parkinson's Disease for Precision Decision-Making

• Authors: Md Mezbahul Islam, John Michael Templeton, Masrur Sobhan, Christian Poellabauer, Ananda Mohan Mondal
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22516
• Pdf URL: https://arxiv.org/pdf/2601.22516
• Copy Paste: [[2601.22516]] SCOPE-PD: Explainable AI on Subjective and Clinical Objective Measurements of Parkinson's Disease for Precision Decision-Making(https://arxiv.org/abs/2601.22516)
• Keywords: interpretability, generative
• Abstract: Parkinson's disease (PD) is a chronic and complex neurodegenerative disorder influenced by genetic, clinical, and lifestyle factors. Predicting this disease early is challenging because it depends on traditional diagnostic methods that face issues of subjectivity, which commonly delay diagnosis. Several objective analyses are currently in practice to help overcome the challenges of subjectivity; however, a proper explanation of these analyses is still lacking. While machine learning (ML) has demonstrated potential in supporting PD diagnosis, existing approaches often rely on subjective reports only and lack interpretability for individualized risk estimation. This study proposes SCOPE-PD, an explainable AI-based prediction framework, by integrating subjective and objective assessments to provide personalized health decisions. Subjective and objective clinical assessment data are collected from the Parkinson's Progression Markers Initiative (PPMI) study to construct a multimodal prediction framework. Several ML techniques are applied to these data, and the best ML model is selected to interpret the results. Model interpretability is examined using SHAP-based analysis. The Random Forest algorithm achieves the highest accuracy of 98.66 percent using combined features from both subjective and objective test data. Tremor, bradykinesia, and facial expression are identified as the top three contributing features from the MDS-UPDRS test in the prediction of PD.

Title: Variational Bayesian Flow Network for Graph Generation

• Authors: Yida Xiong, Jiameng Chen, Xiuwen Gong, Jia Wu, Shirui Pan, Wenbin Hu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22524
• Pdf URL: https://arxiv.org/pdf/2601.22524
• Copy Paste: [[2601.22524]] Variational Bayesian Flow Network for Graph Generation(https://arxiv.org/abs/2601.22524)
• Keywords: diffusion, generative
• Abstract: Graph generation aims to sample discrete node and edge attributes while satisfying coupled structural constraints. Diffusion models for graphs often adopt largely factorized forward-noising, and many flow-matching methods start from factorized reference noise and coordinate-wise interpolation, so node-edge coupling is not encoded by the generative geometry and must be recovered implicitly by the core network, which can be brittle after discrete decoding. Bayesian Flow Networks (BFNs) evolve distribution parameters and naturally support discrete generation. But classical BFNs typically rely on factorized beliefs and independent channels, which limit geometric evidence fusion. We propose Variational Bayesian Flow Network (VBFN), which performs a variational lifting to a tractable joint Gaussian variational belief family governed by structured precisions. Each Bayesian update reduces to solving a symmetric positive definite linear system, enabling coupled node and edge updates within a single fusion step. We construct sample-agnostic sparse precisions from a representation-induced dependency graph, thereby avoiding label leakage while enforcing node-edge consistency. On synthetic and molecular graph datasets, VBFN improves fidelity and diversity, and surpasses baseline methods.

Title: $ρ$-$\texttt{EOS}$: Training-free Bidirectional Variable-Length Control for Masked Diffusion LLMs

• Authors: Jingyi Yang, Yuxian Jiang, Jing Shao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22527
• Pdf URL: https://arxiv.org/pdf/2601.22527
• Copy Paste: [[2601.22527]] $ρ$-$\texttt{EOS}$: Training-free Bidirectional Variable-Length Control for Masked Diffusion LLMs(https://arxiv.org/abs/2601.22527)
• Keywords: diffusion, large language model
• Abstract: Beyond parallel generation and global context modeling, current masked diffusion large language models (dLLMs) suffer from a fundamental limitation: they require a predefined, fixed generation length, which lacks flexibility and forces an inevitable trade-off between output quality and computational efficiency. To address this, we study the denoising dynamics and find that the implicit density ($\rho$) of end-of-sequence ($\texttt{EOS}$) tokens serves as a reliable signal of generation sufficiency. In particular, the evolving implicit $\texttt{EOS}$ density during denoising reveals whether the current masked space is excessive or insufficient, thereby guiding the adjustment direction for generation length. Building on this insight, we propose $\textbf{$\rho$-$\texttt{EOS}$}$, a training-free, single-stage strategy that enables bidirectional variable-length generation for masked dLLMs. Unlike prior two-stage approaches--which require separate length adjustment and iterative mask insertion phases while supporting only unidirectional expansion--$\textbf{$\rho$-$\texttt{EOS}$}$ achieves bidirectional length adjustment within a unified denoising process by continuously estimating the implicit $\texttt{EOS}$ density: excessively high density triggers $\texttt{MASK}$ token contraction, while insufficient density induces expansion. Extensive experiments on mathematics and code benchmarks demonstrate that $\textbf{$\rho$-$\texttt{EOS}$}$ achieves comparable performance while substantially improving inference efficiency and token utilization.

Title: SHED Light on Segmentation for Dense Prediction

• Authors: Seung Hyun Lee, Sangwoo Mo, Stella X. Yu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22529
• Pdf URL: https://arxiv.org/pdf/2601.22529
• Copy Paste: [[2601.22529]] SHED Light on Segmentation for Dense Prediction(https://arxiv.org/abs/2601.22529)
• Keywords: segmentation
• Abstract: Dense prediction infers per-pixel values from a single image and is fundamental to 3D perception and robotics. Although real-world scenes exhibit strong structure, existing methods treat it as an independent pixel-wise prediction, often resulting in structural inconsistencies. We propose SHED, a novel encoder-decoder architecture that enforces geometric prior explicitly by incorporating segmentation into dense prediction. By bidirectional hierarchical reasoning, segment tokens are hierarchically pooled in the encoder and unpooled in the decoder to reverse the hierarchy. The model is supervised only at the final output, allowing the segment hierarchy to emerge without explicit segmentation supervision. SHED improves depth boundary sharpness and segment coherence, while demonstrating strong cross-domain generalization from synthetic to the real-world environments. Its hierarchy-aware decoder better captures global 3D scene layouts, leading to improved semantic segmentation performance. Moreover, SHED enhances 3D reconstruction quality and reveals interpretable part-level structures that are often missed by conventional pixel-wise methods.

Title: Learn from A Rationalist: Distilling Intermediate Interpretable Rationales

• Authors: Jiayi Dai, Randy Goebel
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22531
• Pdf URL: https://arxiv.org/pdf/2601.22531
• Copy Paste: [[2601.22531]] Learn from A Rationalist: Distilling Intermediate Interpretable Rationales(https://arxiv.org/abs/2601.22531)
• Keywords: extraction, interpretability, transformer
• Abstract: Because of the pervasive use of deep neural networks (DNNs), especially in high-stakes domains, the interpretability of DNNs has received increased attention. The general idea of rationale extraction (RE) is to provide an interpretable-by-design framework for DNNs via a select-predict architecture where two neural networks learn jointly to perform feature selection and prediction, respectively. Given only the remote supervision from the final task prediction, the process of learning to select subsets of features (or \emph{rationales}) requires searching in the space of all possible feature combinations, which is computationally challenging and even harder when the base neural networks are not sufficiently capable. To improve the predictive performance of RE models that are based on less capable or smaller neural networks (i.e., the students), we propose \textbf{REKD} (\textbf{R}ationale \textbf{E}xtraction with \textbf{K}nowledge \textbf{D}istillation) where a student RE model learns from the rationales and predictions of a teacher (i.e., a \emph{rationalist}) in addition to the student's own RE optimization. This structural adjustment to RE aligns well with how humans could learn effectively from interpretable and verifiable knowledge. Because of the neural-model agnostic nature of the method, any black-box neural network could be integrated as a backbone model. To demonstrate the viability of REKD, we conduct experiments with multiple variants of BERT and vision transformer (ViT) models. Our experiments across language and vision classification datasets (i.e., IMDB movie reviews, CIFAR 10 and CIFAR 100) show that REKD significantly improves the predictive performance of the student RE models.

Title: Towards the Holographic Characteristic of LLMs for Efficient Short-text Generation

• Authors: Shun Qian, Bingquan Liu, Chengjie Sun, Zhen Xu, Baoxun Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22546
• Pdf URL: https://arxiv.org/pdf/2601.22546
• Copy Paste: [[2601.22546]] Towards the Holographic Characteristic of LLMs for Efficient Short-text Generation(https://arxiv.org/abs/2601.22546)
• Keywords: large language model
• Abstract: The recent advancements in Large Language Models (LLMs) have attracted interest in exploring their in-context learning abilities and chain-of-thought capabilities. However, there are few studies investigating the specific traits related to the powerful generation capacity of LLMs. This paper aims to delve into the generation characteristics exhibited by LLMs. Through our investigation, we have discovered that language models tend to capture target-side keywords at the beginning of the generation process. We name this phenomenon the Holographic Characteristic of language models. For the purpose of exploring this characteristic and further improving the inference efficiency of language models, we propose a plugin called HOLO, which leverages the Holographic Characteristic to extract target-side keywords from language models within a limited number of generation steps and complements the sentence with a parallel lexically constrained text generation method. To verify the effectiveness of HOLO, we conduct massive experiments on language models of varying architectures and scales in the short-text generation scenario. The results demonstrate that HOLO achieves comparable performance to the baselines in terms of both automatic and human-like evaluation metrics and highlight the potential of the Holographic Characteristic.

Title: Are LLM Evaluators Really Narcissists? Sanity Checking Self-Preference Evaluations

• Authors: Dani Roytburg, Matthew Bozoukov, Matthew Nguyen, Mackenzie Puig-Hall, Narmeen Oozeer
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22548
• Pdf URL: https://arxiv.org/pdf/2601.22548
• Copy Paste: [[2601.22548]] Are LLM Evaluators Really Narcissists? Sanity Checking Self-Preference Evaluations(https://arxiv.org/abs/2601.22548)
• Keywords: large language model
• Abstract: Recent research has shown that large language models (LLM) favor own outputs when acting as judges, undermining the integrity of automated post-training and evaluation workflows. However, it is difficult to disentangle which evaluation biases are explained by narcissism versus general experimental confounds, distorting measurements of self-preference bias. We discover a core methodological confound which could reduce measurement error by 89.6%. Specifically, LLM evaluators may deliver self-preferring verdicts when the judge responds to queries which they completed incorrectly themselves; this would be true regardless of whether one of their responses is their own. To decouple self-preference signals from noisy outputs on hard problems, we introduce an Evaluator Quality Baseline, which compares the probability that a judge incorrectly votes for itself against the probability that it votes for an incorrect response from another model. Evaluating this simple baseline on 37,448 queries, only 51% of initial findings retain statistical significance. Finally, we turn towards characterizing the entropy of "easy" versus "hard" evaluation votes from LLM judges. Our corrective baseline enables future research on self-preference by eliminating noisy data from potential solutions. More widely, this work contributes to the growing body of work on cataloging and isolating judge-bias effects.

Title: VocBulwark: Towards Practical Generative Speech Watermarking via Additional-Parameter Injection

• Authors: Weizhi Liu, Yue Li, Zhaoxia Yin
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2601.22556
• Pdf URL: https://arxiv.org/pdf/2601.22556
• Copy Paste: [[2601.22556]] VocBulwark: Towards Practical Generative Speech Watermarking via Additional-Parameter Injection(https://arxiv.org/abs/2601.22556)
• Keywords: security, defense, attack, robust, watermark, generative
• Abstract: Generated speech achieves human-level naturalness but escalates security risks of misuse. However, existing watermarking methods fail to reconcile fidelity with robustness, as they rely either on simple superposition in the noise space or on intrusive alterations to model weights. To bridge this gap, we propose VocBulwark, an additional-parameter injection framework that freezes generative model parameters to preserve perceptual quality. Specifically, we design a Temporal Adapter to deeply entangle watermarks with acoustic attributes, synergizing with a Coarse-to-Fine Gated Extractor to resist advanced attacks. Furthermore, we develop an Accuracy-Guided Optimization Curriculum that dynamically orchestrates gradient flow to resolve the optimization conflict between fidelity and robustness. Comprehensive experiments demonstrate that VocBulwark achieves high-capacity and high-fidelity watermarking, offering robust defense against complex practical scenarios, with resilience to Codec regenerations and variable-length manipulations.

Title: EUGens: Efficient, Unified, and General Dense Layers

• Authors: Sang Min Kim, Byeongchan Kim, Arijit Sehanobish, Somnath Basu Roy Chowdhury, Rahul Kidambi, Dongseok Shim, Avinava Dubey, Snigdha Chaturvedi, Min-hwan Oh, Krzysztof Choromanski
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22563
• Pdf URL: https://arxiv.org/pdf/2601.22563
• Copy Paste: [[2601.22563]] EUGens: Efficient, Unified, and General Dense Layers(https://arxiv.org/abs/2601.22563)
• Keywords: transformer
• Abstract: Efficient neural networks are essential for scaling machine learning models to real-time applications and resource-constrained environments. Fully-connected feedforward layers (FFLs) introduce computation and parameter count bottlenecks within neural network architectures. To address this challenge, in this work, we propose a new class of dense layers that generalize standard fully-connected feedforward layers, \textbf{E}fficient, \textbf{U}nified and \textbf{Gen}eral dense layers (EUGens). EUGens leverage random features to approximate standard FFLs and go beyond them by incorporating a direct dependence on the input norms in their computations. The proposed layers unify existing efficient FFL extensions and improve efficiency by reducing inference complexity from quadratic to linear time. They also lead to \textbf{the first} unbiased algorithms approximating FFLs with arbitrary polynomial activation functions. Furthermore, EuGens reduce the parameter count and computational overhead while preserving the expressive power and adaptability of FFLs. We also present a layer-wise knowledge transfer technique that bypasses backpropagation, enabling efficient adaptation of EUGens to pre-trained models. Empirically, we observe that integrating EUGens into Transformers and MLPs yields substantial improvements in inference speed (up to \textbf{27}\%) and memory efficiency (up to \textbf{30}\%) across a range of tasks, including image classification, language model pre-training, and 3D scene reconstruction. Overall, our results highlight the potential of EUGens for the scalable deployment of large-scale neural networks in real-world scenarios.

Title: Whispers of Wealth: Red-Teaming Google's Agent Payments Protocol via Prompt Injection

• Authors: Tanusree Debi, Wentian Zhu
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22569
• Pdf URL: https://arxiv.org/pdf/2601.22569
• Copy Paste: [[2601.22569]] Whispers of Wealth: Red-Teaming Google's Agent Payments Protocol via Prompt Injection(https://arxiv.org/abs/2601.22569)
• Keywords: secure, attack, robust, large language model
• Abstract: Large language model (LLM) based agents are increasingly used to automate financial transactions, yet their reliance on contextual reasoning exposes payment systems to prompt-driven manipulation. The Agent Payments Protocol (AP2) aims to secure agent-led purchases through cryptographically verifiable mandates, but its practical robustness remains underexplored. In this work, we perform an AI red-teaming evaluation of AP2 and identify vulnerabilities arising from indirect and direct prompt injection. We introduce two attack techniques, the Branded Whisper Attack and the Vault Whisper Attack which manipulate product ranking and extract sensitive user data. Using a functional AP2 based shopping agent built with Gemini-2.5-Flash and the Google ADK framework, we experimentally validate that simple adversarial prompts can reliably subvert agent behavior. Our findings reveal critical weaknesses in current agentic payment architectures and highlight the need for stronger isolation and defensive safeguards in LLM-mediated financial systems.

Title: DELNet: Continuous All-in-One Weather Removal via Dynamic Expert Library

• Authors: Shihong Liu, Kun Zuo, Hanguang Xiao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22573
• Pdf URL: https://arxiv.org/pdf/2601.22573
• Copy Paste: [[2601.22573]] DELNet: Continuous All-in-One Weather Removal via Dynamic Expert Library(https://arxiv.org/abs/2601.22573)
• Keywords: robust
• Abstract: All-in-one weather image restoration methods are valuable in practice but depend on pre-collected data and require retraining for unseen degradations, leading to high cost. We propose DELNet, a continual learning framework for weather image restoration. DELNet integrates a judging valve that measures task similarity to distinguish new from known tasks, and a dynamic expert library that stores experts trained on different degradations. For new tasks, the valve selects top-k experts for knowledge transfer while adding new experts to capture task-specific features; for known tasks, the corresponding experts are directly reused. This design enables continuous optimization without retraining existing models. Experiments on OTS, Rain100H, and Snow100K demonstrate that DELNet surpasses state-of-the-art continual learning methods, achieving PSNR gains of 16\%, 11\%, and 12\%, respectively. These results highlight the effectiveness, robustness, and efficiency of DELNet, which reduces retraining cost and enables practical deployment in real-world scenarios.

Title: Mitigating Hallucinations in Video Large Language Models via Spatiotemporal-Semantic Contrastive Decoding

• Authors: Yuansheng Gao, Jinman Zhao, Tong Zhang, Xingguo Xu, Han Bao, Zonghui Wang, Wenzhi Chen
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22574
• Pdf URL: https://arxiv.org/pdf/2601.22574
• Copy Paste: [[2601.22574]] Mitigating Hallucinations in Video Large Language Models via Spatiotemporal-Semantic Contrastive Decoding(https://arxiv.org/abs/2601.22574)
• Keywords: robust, large language model
• Abstract: Although Video Large Language Models perform remarkably well across tasks such as video understanding, question answering, and reasoning, they still suffer from the problem of hallucination, which refers to generating outputs that are inconsistent with explicit video content or factual evidence. However, existing decoding methods for mitigating video hallucinations, while considering the spatiotemporal characteristics of videos, mostly rely on heuristic designs. As a result, they fail to precisely capture the root causes of hallucinations and their fine-grained temporal and semantic correlations, leading to limited robustness and generalization in complex scenarios. To more effectively mitigate video hallucinations, we propose a novel decoding strategy termed Spatiotemporal-Semantic Contrastive Decoding. This strategy constructs negative features by deliberately disrupting the spatiotemporal consistency and semantic associations of video features, and suppresses video hallucinations through contrastive decoding against the original video features during inference. Extensive experiments demonstrate that our method not only effectively mitigates the occurrence of hallucinations, but also preserves the general video understanding and reasoning capabilities of the model.

Title: PhoStream: Benchmarking Real-World Streaming for Omnimodal Assistants in Mobile Scenarios

• Authors: Xudong Lu, Huankang Guan, Yang Bo, Jinpeng Chen, Xintong Guo, Shuhan Li, Fang Liu, Peiwen Sun, Xueying Li, Wei Zhang, Xue Yang, Rui Liu, Hongsheng Li
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22575
• Pdf URL: https://arxiv.org/pdf/2601.22575
• Copy Paste: [[2601.22575]] PhoStream: Benchmarking Real-World Streaming for Omnimodal Assistants in Mobile Scenarios(https://arxiv.org/abs/2601.22575)
• Keywords: generative, large language model
• Abstract: Multimodal Large Language Models excel at offline audio-visual understanding, but their ability to serve as mobile assistants in continuous real-world streams remains underexplored. In daily phone use, mobile assistants must track streaming audio-visual inputs and respond at the right time, yet existing benchmarks are often restricted to multiple-choice questions or use shorter videos. In this paper, we introduce PhoStream, the first mobile-centric streaming benchmark that unifies on-screen and off-screen scenarios to evaluate video, audio, and temporal reasoning. PhoStream contains 5,572 open-ended QA pairs from 578 videos across 4 scenarios and 10 capabilities. We build it with an Automated Generative Pipeline backed by rigorous human verification, and evaluate models using a realistic Online Inference Pipeline and LLM-as-a-Judge evaluation for open-ended responses. Experiments reveal a temporal asymmetry in LLM-judged scores (0-100): models perform well on Instant and Backward tasks (Gemini 3 Pro exceeds 80), but drop sharply on Forward tasks (16.40), largely due to early responses before the required visual and audio cues appear. This highlights a fundamental limitation: current MLLMs struggle to decide when to speak, not just what to say. Code and datasets used in this work will be made publicly accessible at this https URL.

Title: FedDis: A Causal Disentanglement Framework for Federated Traffic Prediction

• Authors: Chengyang Zhou, Zijian Zhang, Chunxu Zhang, Hao Miao, Yulin Zhang, Kedi Lyu, Juncheng Hu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22578
• Pdf URL: https://arxiv.org/pdf/2601.22578
• Copy Paste: [[2601.22578]] FedDis: A Causal Disentanglement Framework for Federated Traffic Prediction(https://arxiv.org/abs/2601.22578)
• Keywords: privacy, robust, federate, generative
• Abstract: Federated learning offers a promising paradigm for privacy-preserving traffic prediction, yet its performance is often challenged by the non-identically and independently distributed (non-IID) nature of decentralized traffic data. Existing federated methods frequently struggle with this data heterogeneity, typically entangling globally shared patterns with client-specific local dynamics within a single representation. In this work, we postulate that this heterogeneity stems from the entanglement of two distinct generative sources: client-specific localized dynamics and cross-client global spatial-temporal patterns. Motivated by this perspective, we introduce FedDis, a novel framework that, to the best of our knowledge, is the first to leverage causal disentanglement for federated spatial-temporal prediction. Architecturally, FedDis comprises a dual-branch design wherein a Personalized Bank learns to capture client-specific factors, while a Global Pattern Bank distills common knowledge. This separation enables robust cross-client knowledge transfer while preserving high adaptability to unique local environments. Crucially, a mutual information minimization objective is employed to enforce informational orthogonality between the two branches, thereby ensuring effective disentanglement. Comprehensive experiments conducted on four real-world benchmark datasets demonstrate that FedDis consistently achieves state-of-the-art performance, promising efficiency, and superior expandability.

Title: Non-Intrusive Graph-Based Bot Detection for E-Commerce Using Inductive Graph Neural Networks

• Authors: Sichen Zhao, Zhiming Xue, Yalun Qi, Xianling Zeng, Zihan Yu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22579
• Pdf URL: https://arxiv.org/pdf/2601.22579
• Copy Paste: [[2601.22579]] Non-Intrusive Graph-Based Bot Detection for E-Commerce Using Inductive Graph Neural Networks(https://arxiv.org/abs/2601.22579)
• Keywords: security, robust
• Abstract: Malicious bots pose a growing threat to e-commerce platforms by scraping data, hoarding inventory, and perpetrating fraud. Traditional bot mitigation techniques, including IP blacklists and CAPTCHA-based challenges, are increasingly ineffective or intrusive, as modern bots leverage proxies, botnets, and AI-assisted evasion strategies. This work proposes a non-intrusive graph-based bot detection framework for e-commerce that models user session behavior through a graph representation and applies an inductive graph neural network for classification. The approach captures both relational structure and behavioral semantics, enabling accurate identification of subtle automated activity that evades feature-based methods. Experiments on real-world e-commerce traffic demonstrate that the proposed inductive graph model outperforms a strong session-level multilayer perceptron baseline in terms of AUC and F1 score. Additional adversarial perturbation and cold-start simulations show that the model remains robust under moderate graph modifications and generalizes effectively to previously unseen sessions and URLs. The proposed framework is deployment-friendly, integrates with existing systems without client-side instrumentation, and supports real-time inference and incremental updates, making it suitable for practical e-commerce security deployments.

Title: SpanNorm: Reconciling Training Stability and Performance in Deep Transformers

• Authors: Chao Wang, Bei Li, Jiaqi Zhang, Xinyu Liu, Yuchun Fan, Linkun Lyu, Xin Chen, Jingang Wang, Tong Xiao, Peng Pei, Xunliang Cai
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22580
• Pdf URL: https://arxiv.org/pdf/2601.22580
• Copy Paste: [[2601.22580]] SpanNorm: Reconciling Training Stability and Performance in Deep Transformers(https://arxiv.org/abs/2601.22580)
• Keywords: transformer, large language model
• Abstract: The success of Large Language Models (LLMs) hinges on the stable training of deep Transformer architectures. A critical design choice is the placement of normalization layers, leading to a fundamental trade-off: the ``PreNorm'' architecture ensures training stability at the cost of potential performance degradation in deep models, while the ``PostNorm'' architecture offers strong performance but suffers from severe training instability. In this work, we propose SpanNorm, a novel technique designed to resolve this dilemma by integrating the strengths of both paradigms. Structurally, SpanNorm establishes a clean residual connection that spans the entire transformer block to stabilize signal propagation, while employing a PostNorm-style computation that normalizes the aggregated output to enhance model performance. We provide a theoretical analysis demonstrating that SpanNorm, combined with a principled scaling strategy, maintains bounded signal variance throughout the network, preventing the gradient issues that plague PostNorm models, and also alleviating the representation collapse of PreNorm. Empirically, SpanNorm consistently outperforms standard normalization schemes in both dense and Mixture-of-Experts (MoE) scenarios, paving the way for more powerful and stable Transformer architectures.

Title: Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry

• Authors: Zhuochun Li, Yong Zhang, Ming Li, Yuelyu Ji, Yiming Zeng, Ning Cheng, Yun Zhu, Yanmeng Wang, Shaojun Wang, Jing Xiao, Daqing He
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22588
• Pdf URL: https://arxiv.org/pdf/2601.22588
• Copy Paste: [[2601.22588]] Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry(https://arxiv.org/abs/2601.22588)
• Keywords: generative, large language model
• Abstract: Large language models (LLMs) are widely used as reference-free evaluators via prompting, but this "LLM-as-a-Judge" paradigm is costly, opaque, and sensitive to prompt design. In this work, we investigate whether smaller models can serve as efficient evaluators by leveraging internal representations instead of surface generation. We uncover a consistent empirical pattern: small LMs, despite with weak generative ability, encode rich evaluative signals in their hidden states. This motivates us to propose the Semantic Capacity Asymmetry Hypothesis: evaluation requires significantly less semantic capacity than generation and can be grounded in intermediate representations, suggesting that evaluation does not necessarily need to rely on large-scale generative models but can instead leverage latent features from smaller ones. Our findings motivate a paradigm shift from LLM-as-a-Judge to Representation-as-a-Judge, a decoding-free evaluation strategy that probes internal model structure rather than relying on prompted output. We instantiate this paradigm through INSPECTOR, a probing-based framework that predicts aspect-level evaluation scores from small model representations. Experiments on reasoning benchmarks (GSM8K, MATH, GPQA) show that INSPECTOR substantially outperforms prompting-based small LMs and closely approximates full LLM judges, while offering a more efficient, reliable, and interpretable alternative for scalable evaluation.

Title: FedCARE: Federated Unlearning with Conflict-Aware Projection and Relearning-Resistant Recovery

• Authors: Yue Li, Mingmin Chu, Xilei Yang, Da Xiao, Ziqi Xu, Wei Shao, Qipeng Song, Hui Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22589
• Pdf URL: https://arxiv.org/pdf/2601.22589
• Copy Paste: [[2601.22589]] FedCARE: Federated Unlearning with Conflict-Aware Projection and Relearning-Resistant Recovery(https://arxiv.org/abs/2601.22589)
• Keywords: privacy, federate
• Abstract: Federated learning (FL) enables collaborative model training without centralizing raw data, but privacy regulations such as the right to be forgotten require FL systems to remove the influence of previously used training data upon request. Retraining a federated model from scratch is prohibitively expensive, motivating federated unlearning (FU). However, existing FU methods suffer from high unlearning overhead, utility degradation caused by entangled knowledge, and unintended relearning during post-unlearning recovery. In this paper, we propose FedCARE, a unified and low overhead FU framework that enables conflict-aware unlearning and relearning-resistant recovery. FedCARE leverages gradient ascent for efficient forgetting when target data are locally available and employs data free model inversion to construct class level proxies of shared knowledge. Based on these insights, FedCARE integrates a pseudo-sample generator, conflict-aware projected gradient ascent for utility preserving unlearning, and a recovery strategy that suppresses rollback toward the pre-unlearning model. FedCARE supports client, instance, and class level unlearning with modest overhead. Extensive experiments on multiple datasets and model architectures under both IID and non-IID settings show that FedCARE achieves effective forgetting, improved utility retention, and reduced relearning risk compared to state of the art FU baselines.

Title: Heterogeneous Graph Alignment for Joint Reasoning and Interpretability

• Authors: Zahra Moslemi, Ziyi Liang, Norbert Fortin, Babak Shahbaba
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22593
• Pdf URL: https://arxiv.org/pdf/2601.22593
• Copy Paste: [[2601.22593]] Heterogeneous Graph Alignment for Joint Reasoning and Interpretability(https://arxiv.org/abs/2601.22593)
• Keywords: interpretability, transformer
• Abstract: Multi-graph learning is crucial for extracting meaningful signals from collections of heterogeneous graphs. However, effectively integrating information across graphs with differing topologies, scales, and semantics, often in the absence of shared node identities, remains a significant challenge. We present the Multi-Graph Meta-Transformer (MGMT), a unified, scalable, and interpretable framework for cross-graph learning. MGMT first applies Graph Transformer encoders to each graph, mapping structure and attributes into a shared latent space. It then selects task-relevant supernodes via attention and builds a meta-graph that connects functionally aligned supernodes across graphs using similarity in the latent space. Additional Graph Transformer layers on this meta-graph enable joint reasoning over intra- and inter-graph structure. The meta-graph provides built-in interpretability: supernodes and superedges highlight influential substructures and cross-graph alignments. Evaluating MGMT on both synthetic datasets and real-world neuroscience applications, we show that MGMT consistently outperforms existing state-of-the-art models in graph-level prediction tasks while offering interpretable representations that facilitate scientific discoveries. Our work establishes MGMT as a unified framework for structured multi-graph learning, advancing representation techniques in domains where graph-based data plays a central role.

Title: Language Model Circuits Are Sparse in the Neuron Basis

• Authors: Aryaman Arora, Zhengxuan Wu, Jacob Steinhardt, Sarah Schwettmann
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22594
• Pdf URL: https://arxiv.org/pdf/2601.22594
• Copy Paste: [[2601.22594]] Language Model Circuits Are Sparse in the Neuron Basis(https://arxiv.org/abs/2601.22594)
• Keywords: interpretability
• Abstract: The high-level concepts that a neural network uses to perform computation need not be aligned to individual neurons (Smolensky, 1986). Language model interpretability research has thus turned to techniques such as \textit{sparse autoencoders} (SAEs) to decompose the neuron basis into more interpretable units of model computation, for tasks such as \textit{circuit tracing}. However, not all neuron-based representations are uninterpretable. For the first time, we empirically show that \textbf{MLP neurons are as sparse a feature basis as SAEs}. We use this finding to develop an end-to-end pipeline for circuit tracing on the MLP neuron basis, which locates causal circuitry on a variety of tasks using gradient-based attribution. On a standard subject-verb agreement benchmark (Marks et al., 2025), a circuit of $\approx 10^2$ MLP neurons is enough to control model behaviour. On the multi-hop city $\to$ state $\to$ capital task from Lindsey et al., 2025, we find a circuit in which small sets of neurons encode specific latent reasoning steps (e.g.~`map city to its state'), and can be steered to change the model's output. This work thus advances automated interpretability of language models without additional training costs.

Title: Lethe:Adapter-Augmented Dual-Stream Update for Persistent Knowledge Erasure in Federated Unlearning

• Authors: Hanwei Tan, Wentai Hu, Ligang He, Yijun Quan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22601
• Pdf URL: https://arxiv.org/pdf/2601.22601
• Copy Paste: [[2601.22601]] Lethe:Adapter-Augmented Dual-Stream Update for Persistent Knowledge Erasure in Federated Unlearning(https://arxiv.org/abs/2601.22601)
• Keywords: federate
• Abstract: Federated unlearning (FU) aims to erase designated client-level, class-level, or sample-level knowledge from a global model. Existing studies commonly assume that the collaboration ends up with the unlearning operation, overlooking the follow-up situation where the federated training continues over the remaining this http URL identify a critical failure mode, termed Knowledge resurfacing, by revealing that continued training can re-activate unlearned knowledge and cause the removed influence to resurface in the global model. To address this, we propose Lethe, a novel federated unlearning method that de-correlates knowledge to be unlearned from knowledge to be retained, ensuring persistent erasure during continued this http URL follows a Reshape--Rectify--Restore pipeline: a temporary adapter is first trained with gradient ascent on the unlearning data to obtain magnified updates, which is then used as corrective signals to diverge layer-wise rectification on the remaining updates in two streams. Finally, the adapter is removed and a short recovery stage is performed on the retained data. Our experiments show that Lethe supports unlearning in the federated system at all levels in a unified manner and maintains superior persistence (Resurfacing Rate <1% in most cases) even after numerous rounds of follow-up training.

Title: Local-Global Multimodal Contrastive Learning for Molecular Property Prediction

• Authors: Xiayu Liu, Zhengyi Lu, Yunhong Liao, Chan Fan, Hou-biao Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22610
• Pdf URL: https://arxiv.org/pdf/2601.22610
• Copy Paste: [[2601.22610]] Local-Global Multimodal Contrastive Learning for Molecular Property Prediction(https://arxiv.org/abs/2601.22610)
• Keywords: transformer
• Abstract: Accurate molecular property prediction requires integrating complementary information from molecular structure and chemical semantics. In this work, we propose LGM-CL, a local-global multimodal contrastive learning framework that jointly models molecular graphs and textual representations derived from SMILES and chemistry-aware augmented texts. Local functional group information and global molecular topology are captured using AttentiveFP and Graph Transformer encoders, respectively, and aligned through self-supervised contrastive learning. In addition, chemically enriched textual descriptions are contrasted with original SMILES to incorporate physicochemical semantics in a task-agnostic manner. During fine-tuning, molecular fingerprints are further integrated via Dual Cross-attention multimodal fusion. Extensive experiments on MoleculeNet benchmarks demonstrate that LGM-CL achieves consistent and competitive performance across both classification and regression tasks, validating the effectiveness of unified local-global and multimodal representation learning.

Title: Stabilizing Transformer Training Through Consensus

• Authors: Shyam Venkatasubramanian, Sean Moushegian, Michael Lin, Mir Park, Ankit Singhal, Connor Lee
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22614
• Pdf URL: https://arxiv.org/pdf/2601.22614
• Copy Paste: [[2601.22614]] Stabilizing Transformer Training Through Consensus(https://arxiv.org/abs/2601.22614)
• Keywords: transformer
• Abstract: Standard attention-based transformers are known to exhibit instability under learning rate overspecification during training, particularly at high learning rates. While various methods have been proposed to improve resilience to such overspecification by modifying the optimization procedure, fundamental architectural innovations to this end remain underexplored. In this work, we illustrate that the consensus mechanism, a drop-in replacement for attention, stabilizes transformer training across a wider effective range of learning rates. We formulate consensus as a graphical model and provide extensive empirical analysis demonstrating improved stability across learning rate sweeps on text, DNA, and protein modalities. We further propose a hybrid consensus-attention framework that preserves performance while improving stability. We provide theoretical analysis characterizing the properties of consensus.

Title: Layer-wise Swapping for Generalizable Multilingual Safety

• Authors: Hyunseo Shin, Wonseok Hwang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22620
• Pdf URL: https://arxiv.org/pdf/2601.22620
• Copy Paste: [[2601.22620]] Layer-wise Swapping for Generalizable Multilingual Safety(https://arxiv.org/abs/2601.22620)
• Keywords: large language model
• Abstract: Despite the rapid advancements of Large Language Models (LLMs), safety risks remain a critical challenge for low-resource languages. Existing safety datasets are predominantly English centric, limiting progress in multilingual safety alignment. As a result, low resource expert models, finetuned on their respective instruction datasets, tend to exhibit higher unsafety rates compared to their high resource counterparts. In this work, we propose a safety aware layer swapping method that transfers safety alignment from an English safety expert to low resource language experts without additional training. To further enhance transfer ability, our method adaptively selects or blends modules based on their degree of specialization. Our approach preserves performance on general language understanding tasks while enhancing safety in the target languages. Experimental results show that the proposed method achieves comparable performance to the language expert on general benchmarks such as MMMLU, BELEBELE, and MGSM, while producing more aligned and less harmful responses on the MultiJail safety benchmark.

Title: TTCS: Test-Time Curriculum Synthesis for Self-Evolving

• Authors: Chengyi Yang, Zhishang Xiang, Yunbo Tang, Zongpei Teng, Chengsong Huang, Fei Long, Yuhan Liu, Jinsong Su
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22628
• Pdf URL: https://arxiv.org/pdf/2601.22628
• Copy Paste: [[2601.22628]] TTCS: Test-Time Curriculum Synthesis for Self-Evolving(https://arxiv.org/abs/2601.22628)
• Keywords: large language model
• Abstract: Test-Time Training offers a promising way to improve the reasoning ability of large language models (LLMs) by adapting the model using only the test questions. However, existing methods struggle with difficult reasoning problems for two reasons: raw test questions are often too difficult to yield high-quality pseudo-labels, and the limited size of test sets makes continuous online updates prone to instability. To address these limitations, we propose TTCS, a co-evolving test-time training framework. Specifically, TTCS initializes two policies from the same pretrained model: a question synthesizer and a reasoning solver. These policies evolve through iterative optimization: the synthesizer generates progressively challenging question variants conditioned on the test questions, creating a structured curriculum tailored to the solver's current capability, while the solver updates itself using self-consistency rewards computed from multiple sampled responses on both original test and synthetic questions. Crucially, the solver's feedback guides the synthesizer to generate questions aligned with the model's current capability, and the generated question variants in turn stabilize the solver's test-time training. Experiments show that TTCS consistently strengthens the reasoning ability on challenging mathematical benchmarks and transfers to general-domain tasks across different LLM backbones, highlighting a scalable path towards dynamically constructing test-time curricula for self-evolving. Our code and implementation details are available at this https URL.

Title: Time-Annealed Perturbation Sampling: Diverse Generation for Diffusion Language Models

• Authors: Jingxuan Wu, Zhenglin Wan, Xingrui Yu, Yuzhe Yang, Yiqiao Huang, Ivor Tsang, Yang You
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22629
• Pdf URL: https://arxiv.org/pdf/2601.22629
• Copy Paste: [[2601.22629]] Time-Annealed Perturbation Sampling: Diverse Generation for Diffusion Language Models(https://arxiv.org/abs/2601.22629)
• Keywords: diffusion
• Abstract: Diffusion language models (Diffusion-LMs) introduce an explicit temporal dimension into text generation, yet how this structure can be leveraged to control generation diversity for exploring multiple valid semantic or reasoning paths remains underexplored. In this paper, we show that Diffusion-LMs, like diffusion models in image generation, exhibit a temporal division of labor: early denoising steps largely determine the global semantic structure, while later steps focus on local lexical refinement. Building on this insight, we propose Time-Annealed Perturbation Sampling (TAPS), a training-free inference strategy that encourages semantic branching early in the diffusion process while progressively reducing perturbations to preserve fluency and instruction adherence. TAPS is compatible with both non-autoregressive and semi-autoregressive Diffusion backbones, demonstrated on LLaDA and TraDo in our paper, and consistently improves output diversity across creative writing and reasoning benchmarks without compromising generation quality.

Title: LINA: Linear Autoregressive Image Generative Models with Continuous Tokens

• Authors: Jiahao Wang, Ting Pan, Haoge Deng, Dongchen Han, Taiqiang Wu, Xinlong Wang, Ping Luo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22630
• Pdf URL: https://arxiv.org/pdf/2601.22630
• Copy Paste: [[2601.22630]] LINA: Linear Autoregressive Image Generative Models with Continuous Tokens(https://arxiv.org/abs/2601.22630)
• Keywords: diffusion, transformer, generative
• Abstract: Autoregressive models with continuous tokens form a promising paradigm for visual generation, especially for text-to-image (T2I) synthesis, but they suffer from high computational cost. We study how to design compute-efficient linear attention within this framework. Specifically, we conduct a systematic empirical analysis of scaling behavior with respect to parameter counts under different design choices, focusing on (1) normalization paradigms in linear attention (division-based vs. subtraction-based) and (2) depthwise convolution for locality augmentation. Our results show that although subtraction-based normalization is effective for image classification, division-based normalization scales better for linear generative transformers. In addition, incorporating convolution for locality modeling plays a crucial role in autoregressive generation, consistent with findings in diffusion models. We further extend gating mechanisms, commonly used in causal linear attention, to the bidirectional setting and propose a KV gate. By introducing data-independent learnable parameters to the key and value states, the KV gate assigns token-wise memory weights, enabling flexible memory management similar to forget gates in language models. Based on these findings, we present LINA, a simple and compute-efficient T2I model built entirely on linear attention, capable of generating high-fidelity 1024x1024 images from user instructions. LINA achieves competitive performance on both class-conditional and T2I benchmarks, obtaining 2.18 FID on ImageNet (about 1.4B parameters) and 0.74 on GenEval (about 1.5B parameters). A single linear attention module reduces FLOPs by about 61 percent compared to softmax attention. Code and models are available at: this https URL.

Title: DART-ing Through the Drift: Dynamic Tracing of Knowledge Neurons for Adaptive Inference-Time Pruning

• Authors: Abhishek Tyagi, Yunuo Cen, Shrey Dhorajiya, Bharadwaj Veeravalli, Xuanyao Fong
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22632
• Pdf URL: https://arxiv.org/pdf/2601.22632
• Copy Paste: [[2601.22632]] DART-ing Through the Drift: Dynamic Tracing of Knowledge Neurons for Adaptive Inference-Time Pruning(https://arxiv.org/abs/2601.22632)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) exhibit substantial parameter redundancy, particularly in Feed-Forward Networks (FFNs). Existing pruning methods suffer from two primary limitations. First, reliance on dataset-specific calibration introduces significant data dependency and computational overhead. Second, being predominantly static, they fail to account for the evolving subset of knowledge neurons in LLMs during autoregressive generation as the context evolves. To address this, we introduce DART, i.e., Dynamic Attention-Guided Runtime Tracing), a lightweight, training-free method that performs on-the-fly context-based pruning. DART monitors shifts in attention score distributions to infer context changes, dynamically updating neuron-level masks to retain salient parameters. Across ten benchmarks, DART outperforms prior dynamic baseline, achieving accuracy gains of up to 14.5% on LLAMA-3.1-8B at 70% FFN sparsity. Furthermore, DART achieves up to 3x better ROUGE-L scores with respect to static-masked pruning on summarization tasks, with its performance comparable to the original dense models. We conclusively demonstrate that the proposed framework effectively adapts to diverse semantic contexts, preserves model capabilities across both general and domain-specific tasks while running at less than 10MBs of memory for LLAMA-3.1-8B(16GBs) with 0.1% FLOPs overhead. The code is available at this https URL.

Title: Pushing the Boundaries of Natural Reasoning: Interleaved Bonus from Formal-Logic Verification

• Authors: Chuxue Cao, Jinluan Yang, Haoran Li, Kunhao Pan, Zijian Zhao, Zhengyu Chen, Yuchen Tian, Lijun Wu, Conghui He, Sirui Han, Yike Guo
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22642
• Pdf URL: https://arxiv.org/pdf/2601.22642
• Copy Paste: [[2601.22642]] Pushing the Boundaries of Natural Reasoning: Interleaved Bonus from Formal-Logic Verification(https://arxiv.org/abs/2601.22642)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) show remarkable capabilities, yet their stochastic next-token prediction creates logical inconsistencies and reward hacking that formal symbolic systems avoid. To bridge this gap, we introduce a formal logic verification-guided framework that dynamically interleaves formal symbolic verification with the natural language generation process, providing real-time feedback to detect and rectify errors as they occur. Distinguished from previous neuro-symbolic methods limited by passive post-hoc validation, our approach actively penalizes intermediate fallacies during the reasoning chain. We operationalize this framework via a novel two-stage training pipeline that synergizes formal logic verification-guided supervised fine-tuning and policy optimization. Extensive evaluation on six benchmarks spanning mathematical, logical, and general reasoning demonstrates that our 7B and 14B models outperform state-of-the-art baselines by average margins of 10.4% and 14.2%, respectively. These results validate that formal verification can serve as a scalable mechanism to significantly push the performance boundaries of advanced LLM reasoning.

Title: GUDA: Counterfactual Group-wise Training Data Attribution for Diffusion Models via Unlearning

• Authors: Naoki Murata, Yuhta Takida, Chieh-Hsin Lai, Toshimitsu Uesaka, Bac Nguyen, Stefano Ermon, Yuki Mitsufuji
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22651
• Pdf URL: https://arxiv.org/pdf/2601.22651
• Copy Paste: [[2601.22651]] GUDA: Counterfactual Group-wise Training Data Attribution for Diffusion Models via Unlearning(https://arxiv.org/abs/2601.22651)
• Keywords: diffusion, generative
• Abstract: Training-data attribution for vision generative models aims to identify which training data influenced a given output. While most methods score individual examples, practitioners often need group-level answers (e.g., artistic styles or object classes). Group-wise attribution is counterfactual: how would a model's behavior on a generated sample change if a group were absent from training? A natural realization of this counterfactual is Leave-One-Group-Out (LOGO) retraining, which retrains the model with each group removed; however, it becomes computationally prohibitive as the number of groups grows. We propose GUDA (Group Unlearning-based Data Attribution) for diffusion models, which approximates each counterfactual model by applying machine unlearning to a shared full-data model instead of training from scratch. GUDA quantifies group influence using differences in a likelihood-based scoring rule (ELBO) between the full model and each unlearned counterfactual. Experiments on CIFAR-10 and artistic style attribution with Stable Diffusion show that GUDA identifies primary contributing groups more reliably than semantic similarity, gradient-based attribution, and instance-level unlearning approaches, while achieving x100 speedup on CIFAR-10 over LOGO retraining.

Title: The Semantic Trap: Do Fine-tuned LLMs Learn Vulnerability Root Cause or Just Functional Pattern?

• Authors: Feiyang Huang, Yuqiang Sun, Fan Zhang, Ziqi Yang, Han Liu, Yang Liu
• Subjects: cs.CR, cs.SE
• Abstract URL: https://arxiv.org/abs/2601.22655
• Pdf URL: https://arxiv.org/pdf/2601.22655
• Copy Paste: [[2601.22655]] The Semantic Trap: Do Fine-tuned LLMs Learn Vulnerability Root Cause or Just Functional Pattern?(https://arxiv.org/abs/2601.22655)
• Keywords: security, robust
• Abstract: LLMs demonstrate promising performance in software vulnerability detection after fine-tuning. However, it remains unclear whether these gains reflect a genuine understanding of vulnerability root causes or merely an exploitation of functional patterns. In this paper, we identify a critical failure mode termed the "semantic trap," where fine-tuned LLMs achieve high detection scores by associating certain functional domains with vulnerability likelihood rather than reasoning about the underlying security this http URL systematically evaluate this phenomenon, we propose TrapEval, a comprehensive evaluation framework designed to disentangle vulnerability root cause from functional pattern. TrapEval introduces two complementary datasets derived from real-world open-source projects: V2N, which pairs vulnerable code with unrelated benign code, and V2P, which pairs vulnerable code with its corresponding patched version, forcing models to distinguish near-identical code that differs only in subtle security-critical logic. Using TrapEval, we fine-tune five representative state-of-the-art LLMs across three model families and evaluate them under cross-dataset testing, semantic-preserving perturbations, and varying degrees of semantic gap measured by this http URL empirical results reveal that, despite improvements in metrics, fine-tuned LLMs consistently struggle to distinguish vulnerable code from its patched counterpart, exhibit severe robustness degradation under minor semantic-preserving transformations, and rely heavily on functional-context shortcuts when the semantic gap is small. These findings provide strong evidence that current fine-tuning practices often fail to impart true vulnerability reasoning. Our findings serve as a wake-up call: high benchmark scores on traditional datasets may be illusory, masking the model's inability to understand the true causal logic of vulnerabilities.

Title: NAG: A Unified Native Architecture for Encoder-free Text-Graph Modeling in Language Models

• Authors: Haisong Gong, Zhibo Liu, Qiang Liu, Shu Wu, Liang Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22657
• Pdf URL: https://arxiv.org/pdf/2601.22657
• Copy Paste: [[2601.22657]] NAG: A Unified Native Architecture for Encoder-free Text-Graph Modeling in Language Models(https://arxiv.org/abs/2601.22657)
• Keywords: robust
• Abstract: Prevailing methods for integrating graphs into Language Models (LMs) typically rely on a segregated architecture: external Graph Neural Networks (GNNs) encode structural topology, while LMs process textual semantics. We argue this approach is suboptimal for text-graphs: it creates a conceptually disjointed interaction paradigm. By segregating structural encoding from semantic processing, these systems must perform a complex implicit alignment between abstract graph tokens and concrete textual elements. Challenging the necessity of external encoders, we propose NAG (Native Architecture for Graphs), a unified framework that internalizes graph processing within the LM's native manifold. Instead of bridging disparate embedding spaces, NAG repurposes the self-attention mechanism to enforce topological dependencies and recalibrates positional IDs to ensure structural equivalence. This allows the model to harness its intrinsic linguistic capability to simultaneously comprehend node and edge content alongside structural topology. We introduce two efficient implementations: NAG-Zero for absolute preservation of the base model's linguistic capabilities, and NAG-LoRA for enhanced structural adaptation. Experiments across diverse graph tasks validate that NAG achieves robust graph comprehension without the overhead of external encoders, offering a simpler, more coherent paradigm for text-graph modeling.

Title: Unsupervised Synthetic Image Attribution: Alignment and Disentanglement

• Authors: Zongfang Liu, Guangyi Chen, Boyang Sun, Tongliang Liu, Kun Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22663
• Pdf URL: https://arxiv.org/pdf/2601.22663
• Copy Paste: [[2601.22663]] Unsupervised Synthetic Image Attribution: Alignment and Disentanglement(https://arxiv.org/abs/2601.22663)
• Keywords: protect
• Abstract: As the quality of synthetic images improves, identifying the underlying concepts of model-generated images is becoming increasingly crucial for copyright protection and ensuring model transparency. Existing methods achieve this attribution goal by training models using annotated pairs of synthetic images and their original training sources. However, obtaining such paired supervision is challenging, as it requires either well-designed synthetic concepts or precise annotations from millions of training sources. To eliminate the need for costly paired annotations, in this paper, we explore the possibility of unsupervised synthetic image attribution. We propose a simple yet effective unsupervised method called Alignment and Disentanglement. Specifically, we begin by performing basic concept alignment using contrastive self-supervised learning. Next, we enhance the model's attribution ability by promoting representation disentanglement with the Infomax loss. This approach is motivated by an interesting observation: contrastive self-supervised models, such as MoCo and DINO, inherently exhibit the ability to perform simple cross-domain alignment. By formulating this observation as a theoretical assumption on cross-covariance, we provide a theoretical explanation of how alignment and disentanglement can approximate the concept-matching process through a decomposition of the canonical correlation analysis objective. On the real-world benchmarks, AbC, we show that our unsupervised method surprisingly outperforms the supervised methods. As a starting point, we expect our intuitive insights and experimental findings to provide a fresh perspective on this challenging task.

Title: ExpAlign: Expectation-Guided Vision-Language Alignment for Open-Vocabulary Grounding

• Authors: Junyi Hu, Tian Bai, Fengyi Wu, Wenyan Li, Zhenming Peng, Yi Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22666
• Pdf URL: https://arxiv.org/pdf/2601.22666
• Copy Paste: [[2601.22666]] ExpAlign: Expectation-Guided Vision-Language Alignment for Open-Vocabulary Grounding(https://arxiv.org/abs/2601.22666)
• Keywords: segmentation
• Abstract: Open-vocabulary grounding requires accurate vision-language alignment under weak supervision, yet existing methods either rely on global sentence embeddings that lack fine-grained expressiveness or introduce token-level alignment with explicit supervision or heavy cross-attention designs. We propose ExpAlign, a theoretically grounded vision-language alignment framework built on a principled multiple instance learning formulation. ExpAlign introduces an Expectation Alignment Head that performs attention-based soft MIL pooling over token-region similarities, enabling implicit token and instance selection without additional annotations. To further stabilize alignment learning, we develop an energy-based multi-scale consistency regularization scheme, including a Top-K multi-positive contrastive objective and a Geometry-Aware Consistency Objective derived from a Lagrangian-constrained free-energy minimization. Extensive experiments show that ExpAlign consistently improves open-vocabulary detection and zero-shot instance segmentation, particularly on long-tail categories. Most notably, it achieves 36.2 AP$_r$ on the LVIS minival split, outperforming other state-of-the-art methods at comparable model scale, while remaining lightweight and inference-efficient.

Title: Beyond Fixed Rounds: Data-Free Early Stopping for Practical Federated Learning

• Authors: Youngjoon Lee, Hyukjoon Lee, Seungrok Jung, Andy Luo, Jinu Gong, Yang Cao, Joonhyuk Kang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22669
• Pdf URL: https://arxiv.org/pdf/2601.22669
• Copy Paste: [[2601.22669]] Beyond Fixed Rounds: Data-Free Early Stopping for Practical Federated Learning(https://arxiv.org/abs/2601.22669)
• Keywords: privacy, federate, data-free
• Abstract: Federated Learning (FL) facilitates decentralized collaborative learning without transmitting raw data. However, reliance on fixed global rounds or validation data for hyperparameter tuning hinders practical deployment by incurring high computational costs and privacy risks. To address this, we propose a data-free early stopping framework that determines the optimal stopping point by monitoring the task vector's growth rate using solely server-side parameters. The numerical results on skin lesion/blood cell classification demonstrate that our approach is comparable to validation-based early stopping across various state-of-the-art FL methods. In particular, the proposed framework spends an average of 47/20 (skin lesion/blood cell) rounds to achieve over 12.5%/10.3% higher performance than early stopping based on validation data. To the best of our knowledge, this is the first work to propose an early stopping framework for FL methods without using any validation data.

Title: VisionTrim: Unified Vision Token Compression for Training-Free MLLM Acceleration

• Authors: Hanxun Yu, Wentong Li, Xuan Qu, Song Wang, Junbo Chen, Jianke Zhu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22674
• Pdf URL: https://arxiv.org/pdf/2601.22674
• Copy Paste: [[2601.22674]] VisionTrim: Unified Vision Token Compression for Training-Free MLLM Acceleration(https://arxiv.org/abs/2601.22674)
• Keywords: large language model
• Abstract: Multimodal large language models (MLLMs) suffer from high computational costs due to excessive visual tokens, particularly in high-resolution and video-based scenarios. Existing token reduction methods typically focus on isolated pipeline components and often neglect textual alignment, leading to performance degradation. In this paper, we propose VisionTrim, a unified framework for training-free MLLM acceleration, integrating two effective plug-and-play modules: 1) the Dominant Vision Token Selection (DVTS) module, which preserves essential visual tokens via a global-local view, and 2) the Text-Guided Vision Complement (TGVC) module, which facilitates context-aware token merging guided by textual cues. Extensive experiments across diverse image and video multimodal benchmarks demonstrate the performance superiority of our VisionTrim, advancing practical MLLM deployment in real-world applications. The code is available at: this https URL.

Title: Stabilizing Consistency Training: A Flow Map Analysis and Self-Distillation

• Authors: Youngjoong Kim, Duhoe Kim, Woosung Kim, Jaesik Park
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22679
• Pdf URL: https://arxiv.org/pdf/2601.22679
• Copy Paste: [[2601.22679]] Stabilizing Consistency Training: A Flow Map Analysis and Self-Distillation(https://arxiv.org/abs/2601.22679)
• Keywords: diffusion, generative
• Abstract: Consistency models have been proposed for fast generative modeling, achieving results competitive with diffusion and flow models. However, these methods exhibit inherent instability and limited reproducibility when training from scratch, motivating subsequent work to explain and stabilize these issues. While these efforts have provided valuable insights, the explanations remain fragmented, and the theoretical relationships remain unclear. In this work, we provide a theoretical examination of consistency models by analyzing them from a flow map-based perspective. This joint analysis clarifies how training stability and convergence behavior can give rise to degenerate solutions. Building on these insights, we revisit self-distillation as a practical remedy for certain forms of suboptimal convergence and reformulate it to avoid excessive gradient norms for stable optimization. We further demonstrate that our strategy extends beyond image generation to diffusion-based policy learning, without reliance on a pretrained diffusion model for initialization, thereby illustrating its broader applicability.

Title: Visual Personalization Turing Test

• Authors: Rameen Abdal, James Burgess, Sergey Tulyakov, Kuan-Chieh Jackson Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22680
• Pdf URL: https://arxiv.org/pdf/2601.22680
• Copy Paste: [[2601.22680]] Visual Personalization Turing Test(https://arxiv.org/abs/2601.22680)
• Keywords: privacy, generative
• Abstract: We introduce the Visual Personalization Turing Test (VPTT), a new paradigm for evaluating contextual visual personalization based on perceptual indistinguishability, rather than identity replication. A model passes the VPTT if its output (image, video, 3D asset, etc.) is indistinguishable to a human or calibrated VLM judge from content a given person might plausibly create or share. To operationalize VPTT, we present the VPTT Framework, integrating a 10k-persona benchmark (VPTT-Bench), a visual retrieval-augmented generator (VPRAG), and the VPTT Score, a text-only metric calibrated against human and VLM judgments. We show high correlation across human, VLM, and VPTT evaluations, validating the VPTT Score as a reliable perceptual proxy. Experiments demonstrate that VPRAG achieves the best alignment-originality balance, offering a scalable and privacy-safe foundation for personalized generative AI.

Title: TSLM: Tree-Structured Language Modeling for Divergent Thinking

• Authors: Doyoung Kim, Jaehyeok Doo, Minjoon Seo
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22688
• Pdf URL: https://arxiv.org/pdf/2601.22688
• Copy Paste: [[2601.22688]] TSLM: Tree-Structured Language Modeling for Divergent Thinking(https://arxiv.org/abs/2601.22688)
• Keywords: robust
• Abstract: Language models generate reasoning sequentially, preventing them from decoupling irrelevant exploration paths during search. We introduce Tree-Structured Language Modeling (TSLM), which uses special tokens to encode branching structure, enabling models to generate and selectively expand multiple search paths within a single generation process. By training on complete search trees including both successful and failed attempts, TSLM learns to internalize systematic exploration without redundant recomputation of shared prefixes. TSLM achieves robust performance and superior inference efficiency by avoiding the multiple independent forward passes required by external search methods. These results suggest a new paradigm of inference-time scaling for robust reasoning, demonstrating that supervised learning on complete tree-structured traces provides an efficient alternative for developing systematic exploration capabilities in language models.

Title: Do Transformers Have the Ability for Periodicity Generalization?

• Authors: Huanyu Liu, Ge Li, Yihong Dong, Sihan Wu, Peixu Wang, Sihao Cheng, Taozhi Chen, Kechi Zhang, Hao Zhu, Tongxuan Liu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22690
• Pdf URL: https://arxiv.org/pdf/2601.22690
• Copy Paste: [[2601.22690]] Do Transformers Have the Ability for Periodicity Generalization?(https://arxiv.org/abs/2601.22690)
• Keywords: transformer, generative, large language model
• Abstract: Large language models (LLMs) based on the Transformer have demonstrated strong performance across diverse tasks. However, current models still exhibit substantial limitations in out-of-distribution (OOD) generalization compared with humans. We investigate this gap through periodicity, one of the basic OOD scenarios. Periodicity captures invariance amid variation. Periodicity generalization represents a model's ability to extract periodic patterns from training data and generalize to OOD scenarios. We introduce a unified interpretation of periodicity from the perspective of abstract algebra and reasoning, including both single and composite periodicity, to explain why Transformers struggle to generalize periodicity. Then we construct Coper about composite periodicity, a controllable generative benchmark with two OOD settings, Hollow and Extrapolation. Experiments reveal that periodicity generalization in Transformers is limited, where models can memorize periodic data during training, but cannot generalize to unseen composite periodicity. We release the source code to support future research.

Title: FNF: Functional Network Fingerprint for Large Language Models

• Authors: Yiheng Liu, Junhao Ning, Sichen Xia, Haiyang Sun, Yang Yang, Hanyang Chi, Xiaohui Gao, Ning Qiang, Bao Ge, Junwei Han, Xintao Hu
• Subjects: cs.CL, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2601.22692
• Pdf URL: https://arxiv.org/pdf/2601.22692
• Copy Paste: [[2601.22692]] FNF: Functional Network Fingerprint for Large Language Models(https://arxiv.org/abs/2601.22692)
• Keywords: protect, robust, large language model
• Abstract: The development of large language models (LLMs) is costly and has significant commercial value. Consequently, preventing unauthorized appropriation of open-source LLMs and protecting developers' intellectual property rights have become critical challenges. In this work, we propose the Functional Network Fingerprint (FNF), a training-free, sample-efficient method for detecting whether a suspect LLM is derived from a victim model, based on the consistency between their functional network activity. We demonstrate that models that share a common origin, even with differences in scale or architecture, exhibit highly consistent patterns of neuronal activity within their functional networks across diverse input samples. In contrast, models trained independently on distinct data or with different objectives fail to preserve such activity alignment. Unlike conventional approaches, our method requires only a few samples for verification, preserves model utility, and remains robust to common model modifications (such as fine-tuning, pruning, and parameter permutation), as well as to comparisons across diverse architectures and dimensionalities. FNF thus provides model owners and third parties with a simple, non-invasive, and effective tool for protecting LLM intellectual property. The code is available at this https URL.

Title: PEAR: Pixel-aligned Expressive humAn mesh Recovery

• Authors: Jiahao Wu, Yunfei Liu, Lijian Lin, Ye Zhu, Lei Zhu, Jingyi Li, Yu Li
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22693
• Pdf URL: https://arxiv.org/pdf/2601.22693
• Copy Paste: [[2601.22693]] PEAR: Pixel-aligned Expressive humAn mesh Recovery(https://arxiv.org/abs/2601.22693)
• Keywords: robust
• Abstract: Reconstructing detailed 3D human meshes from a single in-the-wild image remains a fundamental challenge in computer vision. Existing SMPLX-based methods often suffer from slow inference, produce only coarse body poses, and exhibit misalignments or unnatural artifacts in fine-grained regions such as the face and hands. These issues make current approaches difficult to apply to downstream tasks. To address these challenges, we propose PEAR-a fast and robust framework for pixel-aligned expressive human mesh recovery. PEAR explicitly tackles three major limitations of existing methods: slow inference, inaccurate localization of fine-grained human pose details, and insufficient facial expression capture. Specifically, to enable real-time SMPLX parameter inference, we depart from prior designs that rely on high resolution inputs or multi-branch architectures. Instead, we adopt a clean and unified ViT-based model capable of recovering coarse 3D human geometry. To compensate for the loss of fine-grained details caused by this simplified architecture, we introduce pixel-level supervision to optimize the geometry, significantly improving the reconstruction accuracy of fine-grained human details. To make this approach practical, we further propose a modular data annotation strategy that enriches the training data and enhances the robustness of the model. Overall, PEAR is a preprocessing-free framework that can simultaneously infer EHM-s (SMPLX and scaled-FLAME) parameters at over 100 FPS. Extensive experiments on multiple benchmark datasets demonstrate that our method achieves substantial improvements in pose estimation accuracy compared to previous SMPLX-based approaches. Project page: this https URL

Title: Models Know Models Best: Evaluation via Model-Preferred Formats

• Authors: Joonhak Lee, Sungmok Jung, Jongyeon Park, Jaejin Lee
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22699
• Pdf URL: https://arxiv.org/pdf/2601.22699
• Copy Paste: [[2601.22699]] Models Know Models Best: Evaluation via Model-Preferred Formats(https://arxiv.org/abs/2601.22699)
• Keywords: large language model
• Abstract: Performance of Large Language Models (LLMs) on multiple-choice tasks differs markedly between symbol-based and cloze-style evaluation formats. The observed discrepancies are systematically attributable to task characteristics: natural language continuation benefits from likelihood scoring, whereas explicit comparison is better suited to symbol-based selection. These trends are consistent across various decoder-based LLMs, indicating model-agnostic effects. To address these inconsistencies, a dynamic format-alignment strategy is introduced that employs a lightweight classifier trained on latent model-preference signals. In contrast to human-designed heuristics, which often degrade performance, this approach uses model-generated signals to determine the optimal format for each problem instance. The proposed method achieves substantial and consistent improvements in zero-shot accuracy across reasoning and knowledge benchmarks, better revealing the models' latent capabilities.

Title: RealSec-bench: A Benchmark for Evaluating Secure Code Generation in Real-World Repositories

• Authors: Yanlin Wang, Ziyao Zhang, Chong Wang, Xinyi Xu, Mingwei Liu, Yong Wang, Jiachi Chen, Zibin Zheng
• Subjects: cs.CR, cs.SE
• Abstract URL: https://arxiv.org/abs/2601.22706
• Pdf URL: https://arxiv.org/pdf/2601.22706
• Copy Paste: [[2601.22706]] RealSec-bench: A Benchmark for Evaluating Secure Code Generation in Real-World Repositories(https://arxiv.org/abs/2601.22706)
• Keywords: secure, security, large language model
• Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in code generation, but their proficiency in producing secure code remains a critical, under-explored area. Existing benchmarks often fall short by relying on synthetic vulnerabilities or evaluating functional correctness in isolation, failing to capture the complex interplay between functionality and security found in real-world software. To address this gap, we introduce RealSec-bench, a new benchmark for secure code generation meticulously constructed from real-world, high-risk Java repositories. Our methodology employs a multi-stage pipeline that combines systematic SAST scanning with CodeQL, LLM-based false positive elimination, and rigorous human expert validation. The resulting benchmark contains 105 instances grounded in real-word repository contexts, spanning 19 Common Weakness Enumeration (CWE) types and exhibiting a wide diversity of data flow complexities, including vulnerabilities with up to 34-hop inter-procedural dependencies. Using RealSec-bench, we conduct an extensive empirical study on 5 popular LLMs. We introduce a novel composite metric, SecurePass@K, to assess both functional correctness and security simultaneously. We find that while Retrieval-Augmented Generation (RAG) techniques can improve functional correctness, they provide negligible benefits to security. Furthermore, explicitly prompting models with general security guidelines often leads to compilation failures, harming functional correctness without reliably preventing vulnerabilities. Our work highlights the gap between functional and secure code generation in current LLMs.

Title: AlienLM: Alienization of Language for API-Boundary Privacy in Black-Box LLMs

• Authors: Jaehee Kim, Pilsung Kang
• Subjects: cs.CR, cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22710
• Pdf URL: https://arxiv.org/pdf/2601.22710
• Copy Paste: [[2601.22710]] AlienLM: Alienization of Language for API-Boundary Privacy in Black-Box LLMs(https://arxiv.org/abs/2601.22710)
• Keywords: privacy, protect, attack
• Abstract: Modern LLMs are increasingly accessed via black-box APIs, requiring users to transmit sensitive prompts, outputs, and fine-tuning data to external providers, creating a critical privacy risk at the API boundary. We introduce AlienLM, a deployable API-only privacy layer that protects text by translating it into an Alien Language via a vocabulary-scale bijection, enabling lossless recovery on the client side. Using only standard fine-tuning APIs, Alien Adaptation Training (AAT) adapts target models to operate directly on alienized inputs. Across four LLM backbones and seven benchmarks, AlienLM retains over 81\% of plaintext-oracle performance on average, substantially outperforming random-bijection and character-level baselines. Under adversaries with access to model weights, corpus statistics, and learning-based inverse translation, recovery attacks reconstruct fewer than 0.22\% of alienized tokens. Our results demonstrate a practical pathway for privacy-preserving LLM deployment under API-only access, substantially reducing plaintext exposure while maintaining task performance.

Title: SQUAD: Scalable Quorum Adaptive Decisions via ensemble of early exit neural networks

• Authors: Matteo Gambella, Fabrizio Pittorino, Giuliano Casale, Manuel Roveri
• Subjects: cs.LG, cs.CV, cs.DC
• Abstract URL: https://arxiv.org/abs/2601.22711
• Pdf URL: https://arxiv.org/pdf/2601.22711
• Copy Paste: [[2601.22711]] SQUAD: Scalable Quorum Adaptive Decisions via ensemble of early exit neural networks(https://arxiv.org/abs/2601.22711)
• Keywords: robust
• Abstract: Early-exit neural networks have become popular for reducing inference latency by allowing intermediate predictions when sufficient confidence is achieved. However, standard approaches typically rely on single-model confidence thresholds, which are frequently unreliable due to inherent calibration issues. To address this, we introduce SQUAD (Scalable Quorum Adaptive Decisions), the first inference scheme that integrates early-exit mechanisms with distributed ensemble learning, improving uncertainty estimation while reducing the inference time. Unlike traditional methods that depend on individual confidence scores, SQUAD employs a quorum-based stopping criterion on early-exit learners by collecting intermediate predictions incrementally in order of computational complexity until a consensus is reached and halting the computation at that exit if the consensus is statistically significant. To maximize the efficacy of this voting mechanism, we also introduce QUEST (Quorum Search Technique), a Neural Architecture Search method to select early-exit learners with optimized hierarchical diversity, ensuring learners are complementary at every intermediate layer. This consensus-driven approach yields statistically robust early exits, improving the test accuracy up to 5.95% compared to state-of-the-art dynamic solutions with a comparable computational cost and reducing the inference latency up to 70.60% compared to static ensembles while maintaining a good accuracy.

Title: Vision-Language Models Unlock Task-Centric Latent Actions

• Authors: Alexander Nikulin, Ilya Zisman, Albina Klepach, Denis Tarasov, Alexander Derevyagin, Andrei Polubarov, Lyubaykin Nikita, Vladislav Kurenkov
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22714
• Pdf URL: https://arxiv.org/pdf/2601.22714
• Copy Paste: [[2601.22714]] Vision-Language Models Unlock Task-Centric Latent Actions(https://arxiv.org/abs/2601.22714)
• Keywords: robust
• Abstract: Latent Action Models (LAMs) have rapidly gained traction as an important component in the pre-training pipelines of leading Vision-Language-Action models. However, they fail when observations contain action-correlated distractors, often encoding noise instead of meaningful latent actions. Humans, on the other hand, can effortlessly distinguish task-relevant motions from irrelevant details in any video given only a brief task description. In this work, we propose to utilize the common-sense reasoning abilities of Vision-Language Models (VLMs) to provide promptable representations, effectively separating controllable changes from the noise in unsupervised way. We use these representations as targets during LAM training and benchmark a wide variety of popular VLMs, revealing substantial variation in the quality of promptable representations as well as their robustness to different prompts and hyperparameters. Interestingly, we find that more recent VLMs may perform worse than older ones. Finally, we show that simply asking VLMs to ignore distractors can substantially improve latent action quality, yielding up to a six-fold increase in downstream success rates on Distracting MetaWorld.

Title: Breaking the Blocks: Continuous Low-Rank Decomposed Scaling for Unified LLM Quantization and Adaptation

• Authors: Pingzhi Tang, Ruijie Zhou, Fanxu Meng, Wenjie Pei, Muhan Zhang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22716
• Pdf URL: https://arxiv.org/pdf/2601.22716
• Copy Paste: [[2601.22716]] Breaking the Blocks: Continuous Low-Rank Decomposed Scaling for Unified LLM Quantization and Adaptation(https://arxiv.org/abs/2601.22716)
• Keywords: robust
• Abstract: Current quantization methods for LLMs predominantly rely on block-wise structures to maintain efficiency, often at the cost of representational flexibility. In this work, we demonstrate that element-wise quantization can be made as efficient as block-wise scaling while providing strictly superior expressive power by modeling the scaling manifold as continuous low-rank matrices ($S = BA$). We propose Low-Rank Decomposed Scaling (LoRDS), a unified framework that rethinks quantization granularity through this low-rank decomposition. By "breaking the blocks" of spatial constraints, LoRDS establishes a seamless efficiency lifecycle: it provides high-fidelity PTQ initialization refined via iterative optimization, enables joint QAT of weights and scaling factors, and facilitates high-rank multiplicative PEFT adaptation. Unlike additive PEFT approaches such as QLoRA, LoRDS enables high-rank weight updates within a low-rank budget while incurring no additional inference overhead. Supported by highly optimized Triton kernels, LoRDS consistently outperforms state-of-the-art baselines across various model families in both quantization and downstream fine-tuning tasks. Notably, on Llama3-8B, our method achieves up to a 27.0% accuracy improvement at 3 bits over NormalFloat quantization and delivers a 1.5x inference speedup on NVIDIA RTX 4090 while enhancing PEFT performance by 9.6% on downstream tasks over 4bit QLoRA, offering a robust and integrated solution for unified compression and adaptation of LLMs.

Title: AEGIS: White-Box Attack Path Generation using LLMs and Training Effectiveness Evaluation for Large-Scale Cyber Defence Exercises

• Authors: Ivan K. Tung, Yu Xiang Shi, Alex Chien, Wenkai Liu, Lawrence Zheng
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22720
• Pdf URL: https://arxiv.org/pdf/2601.22720
• Copy Paste: [[2601.22720]] AEGIS: White-Box Attack Path Generation using LLMs and Training Effectiveness Evaluation for Large-Scale Cyber Defence Exercises(https://arxiv.org/abs/2601.22720)
• Keywords: attack
• Abstract: Creating attack paths for cyber defence exercises requires substantial expert effort. Existing automation requires vulnerability graphs or exploit sets curated in advance, limiting where it can be applied. We present AEGIS, a system that generates attack paths using LLMs, white-box access, and Monte Carlo Tree Search over real exploit execution. LLM-based search discovers exploits dynamically without pre-existing vulnerability graphs, while white-box access enables validating exploits in isolation before committing to attack paths. Evaluation at CIDeX 2025, a large-scale exercise spanning 46 IT hosts, showed that AEGIS-generated paths are comparable to human-authored scenarios across four dimensions of training experience (perceived learning, engagement, believability, challenge). Results were measured with a validated questionnaire extensible to general simulation-based training. By automating exploit chain discovery and validation, AEGIS reduces scenario development from months to days, shifting expert effort from technical validation to scenario design.

Title: OpenVTON-Bench: A Large-Scale High-Resolution Benchmark for Controllable Virtual Try-On Evaluation

• Authors: Jin Li, Tao Chen, Shuai Jiang, Weijie Wang, Jingwen Luo, Chenhui Wu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22725
• Pdf URL: https://arxiv.org/pdf/2601.22725
• Copy Paste: [[2601.22725]] OpenVTON-Bench: A Large-Scale High-Resolution Benchmark for Controllable Virtual Try-On Evaluation(https://arxiv.org/abs/2601.22725)
• Keywords: robust, diffusion, segmentation
• Abstract: Recent advances in diffusion models have significantly elevated the visual fidelity of Virtual Try-On (VTON) systems, yet reliable evaluation remains a persistent bottleneck. Traditional metrics struggle to quantify fine-grained texture details and semantic consistency, while existing datasets fail to meet commercial standards in scale and diversity. We present OpenVTON-Bench, a large-scale benchmark comprising approximately 100K high-resolution image pairs (up to $1536 \times 1536$). The dataset is constructed using DINOv3-based hierarchical clustering for semantically balanced sampling and Gemini-powered dense captioning, ensuring a uniform distribution across 20 fine-grained garment categories. To support reliable evaluation, we propose a multi-modal protocol that measures VTON quality along five interpretable dimensions: background consistency, identity fidelity, texture fidelity, shape plausibility, and overall realism. The protocol integrates VLM-based semantic reasoning with a novel Multi-Scale Representation Metric based on SAM3 segmentation and morphological erosion, enabling the separation of boundary alignment errors from internal texture artifacts. Experimental results show strong agreement with human judgments (Kendall's $\tau$ of 0.833 vs. 0.611 for SSIM), establishing a robust benchmark for VTON evaluation.

Title: GaussianOcc3D: A Gaussian-Based Adaptive Multi-modal 3D Occupancy Prediction

• Authors: A. Enes Doruk, Hasan F. Ates
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22729
• Pdf URL: https://arxiv.org/pdf/2601.22729
• Copy Paste: [[2601.22729]] GaussianOcc3D: A Gaussian-Based Adaptive Multi-modal 3D Occupancy Prediction(https://arxiv.org/abs/2601.22729)
• Keywords: robust
• Abstract: 3D semantic occupancy prediction is a pivotal task in autonomous driving, providing a dense and fine-grained understanding of the surrounding environment, yet single-modality methods face trade-offs between camera semantics and LiDAR geometry. Existing multi-modal frameworks often struggle with modality heterogeneity, spatial misalignment, and the representation crisis--where voxels are computationally heavy and BEV alternatives are lossy. We present GaussianOcc3D, a multi-modal framework bridging camera and LiDAR through a memory-efficient, continuous 3D Gaussian representation. We introduce four modules: (1) LiDAR Depth Feature Aggregation (LDFA), using depth-wise deformable sampling to lift sparse signals onto Gaussian primitives; (2) Entropy-Based Feature Smoothing (EBFS) to mitigate domain noise; (3) Adaptive Camera-LiDAR Fusion (ACLF) with uncertainty-aware reweighting for sensor reliability; and (4) a Gauss-Mamba Head leveraging Selective State Space Models for global context with linear complexity. Evaluations on Occ3D, SurroundOcc, and SemanticKITTI benchmarks demonstrate state-of-the-art performance, achieving mIoU scores of 49.4%, 28.9%, and 25.2% respectively. GaussianOcc3D exhibits superior robustness across challenging rainy and nighttime conditions.

Title: ImgCoT: Compressing Long Chain of Thought into Compact Visual Tokens for Efficient Reasoning of Large Language Model

• Authors: Xiaoshu Chen, Sihang Zhou, Ke Liang, Taichun Zhou, Xinwang Liu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22730
• Pdf URL: https://arxiv.org/pdf/2601.22730
• Copy Paste: [[2601.22730]] ImgCoT: Compressing Long Chain of Thought into Compact Visual Tokens for Efficient Reasoning of Large Language Model(https://arxiv.org/abs/2601.22730)
• Keywords: large language model
• Abstract: Compressing long chains of thought (CoT) into compact latent tokens is crucial for efficient reasoning with large language models (LLMs). Recent studies employ autoencoders to achieve this by reconstructing textual CoT from latent tokens, thus encoding CoT semantics. However, treating textual CoT as the reconstruction target forces latent tokens to preserve surface-level linguistic features (e.g., word choice and syntax), introducing a strong linguistic inductive bias that prioritizes linguistic form over reasoning structure and limits logical abstraction. Thus, we propose ImgCoT that replaces the reconstruction target from textual CoT to the visual CoT obtained by rendering CoT into images. This substitutes linguistic bias with spatial inductive bias, i.e., a tendency to model spatial layouts of the reasoning steps in visual CoT, enabling latent tokens to better capture global reasoning structure. Moreover, although visual latent tokens encode abstract reasoning structure, they may blur reasoning details. We thus propose a loose ImgCoT, a hybrid reasoning that augments visual latent tokens with a few key textual reasoning steps, selected based on low token log-likelihood. This design allows LLMs to retain both global reasoning structure and fine-grained reasoning details with fewer tokens than the complete CoT. Extensive experiments across multiple datasets and LLMs demonstrate the effectiveness of the two versions of ImgCoT.

Title: MM-THEBench: Do Reasoning MLLMs Think Reasonably?

• Authors: Zhidian Huang, Zijun Yao, Ji Qi, Shangqing Tu, Junxian Ma, Jinxin Liu, Weichuan Liu, Xiaoyin Che, Lei Hou, Juanzi Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22735
• Pdf URL: https://arxiv.org/pdf/2601.22735
• Copy Paste: [[2601.22735]] MM-THEBench: Do Reasoning MLLMs Think Reasonably?(https://arxiv.org/abs/2601.22735)
• Keywords: robust, large language model
• Abstract: Recent advances in multimodal large language models (MLLMs) mark a shift from non-thinking models to post-trained reasoning models capable of solving complex problems through thinking. However, whether such thinking mitigates hallucinations in multimodal perception and reasoning remains unclear. Self-reflective reasoning enhances robustness but introduces additional hallucinations, and subtle perceptual errors still result in incorrect or coincidentally correct answers. Existing benchmarks primarily focus on models before the emergence of reasoning MLLMs, neglecting the internal thinking process and failing to measure the hallucinations that occur during thinking. To address these challenges, we introduce MM-THEBench, a comprehensive benchmark for assessing hallucinations of intermediate CoTs in reasoning MLLMs. MM-THEBench features a fine-grained taxonomy grounded in cognitive dimensions, diverse data with verified reasoning annotations, and a multi-level automated evaluation framework. Extensive experiments on mainstream reasoning MLLMs reveal insights into how thinking affects hallucination and reasoning capability in various multimodal tasks.

Title: Lingua-SafetyBench: A Benchmark for Safety Evaluation of Multilingual Vision-Language Models

• Authors: Enyi Shi, Pengyang Shao, Yanxin Zhang, Chenhang Cui, Jiayi Lyu, Xu Xie, Xiaobo Xia, Fei Shen, Tat-Seng Chua
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22737
• Pdf URL: https://arxiv.org/pdf/2601.22737
• Copy Paste: [[2601.22737]] Lingua-SafetyBench: A Benchmark for Safety Evaluation of Multilingual Vision-Language Models(https://arxiv.org/abs/2601.22737)
• Keywords: attack, robust
• Abstract: Robust safety of vision-language large models (VLLMs) under joint multilingual and multimodal inputs remains underexplored. Existing benchmarks are typically multilingual but text-only, or multimodal but monolingual. Recent multilingual multimodal red-teaming efforts render harmful prompts into images, yet rely heavily on typography-style visuals and lack semantically grounded image-text pairs, limiting coverage of realistic cross-modal interactions. We introduce Lingua-SafetyBench, a benchmark of 100,440 harmful image-text pairs across 10 languages, explicitly partitioned into image-dominant and text-dominant subsets to disentangle risk sources. Evaluating 11 open-source VLLMs reveals a consistent asymmetry: image-dominant risks yield higher ASR in high-resource languages, while text-dominant risks are more severe in non-high-resource languages. A controlled study on the Qwen series shows that scaling and version upgrades reduce Attack Success Rate (ASR) overall but disproportionately benefit HRLs, widening the gap between HRLs and Non-HRLs under text-dominant risks. This underscores the necessity of language- and modality-aware safety alignment beyond mere this http URL facilitate reproducibility and future research, we will publicly release our benchmark, model checkpoints, and source this http URL code and dataset will be available at this https URL this paper contains examples with unsafe content.

Title: AR-BENCH: Benchmarking Legal Reasoning with Judgment Error Detection, Classification and Correction

• Authors: Yifei Li, Richong Zhang, Wanyu Tu, Zhijie Nie, Haokun Luo, Chuantao Yin, Pengchong Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22742
• Pdf URL: https://arxiv.org/pdf/2601.22742
• Copy Paste: [[2601.22742]] AR-BENCH: Benchmarking Legal Reasoning with Judgment Error Detection, Classification and Correction(https://arxiv.org/abs/2601.22742)
• Keywords: large language model
• Abstract: Legal judgments may contain errors due to the complexity of case circumstances and the abstract nature of legal concepts, while existing appellate review mechanisms face efficiency pressures from a surge in case volumes. Although current legal AI research focuses on tasks like judgment prediction and legal document generation, the task of judgment review differs fundamentally in its objectives and paradigm: it centers on detecting, classifying, and correcting errors after a judgment is issued, constituting anomaly detection rather than prediction or generation. To address this research gap, we introduce a novel task APPELLATE REVIEW, aiming to assess models' diagnostic reasoning and reliability in legal practice. We also construct a novel dataset benchmark AR-BENCH, which comprises 8,700 finely annotated decisions and 34,617 supplementary corpora. By evaluating 14 large language models, we reveal critical limitations in existing models' ability to identify legal application errors, providing empirical evidence for future improvements.

Title: Beauty and the Beast: Imperceptible Perturbations Against Diffusion-Based Face Swapping via Directional Attribute Editing

• Authors: Yilong Huang, Songze Li
• Subjects: cs.CV, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22744
• Pdf URL: https://arxiv.org/pdf/2601.22744
• Copy Paste: [[2601.22744]] Beauty and the Beast: Imperceptible Perturbations Against Diffusion-Based Face Swapping via Directional Attribute Editing(https://arxiv.org/abs/2601.22744)
• Keywords: protect, defense, diffusion
• Abstract: Diffusion-based face swapping achieves state-of-the-art performance, yet it also exacerbates the potential harm of malicious face swapping to violate portraiture right or undermine personal reputation. This has spurred the development of proactive defense methods. However, existing approaches face a core trade-off: large perturbations distort facial structures, while small ones weaken protection effectiveness. To address these issues, we propose FaceDefense, an enhanced proactive defense framework against diffusion-based face swapping. Our method introduces a new diffusion loss to strengthen the defensive efficacy of adversarial examples, and employs a directional facial attribute editing to restore perturbation-induced distortions, thereby enhancing visual imperceptibility. A two-phase alternating optimization strategy is designed to generate final perturbed face images. Extensive experiments show that FaceDefense significantly outperforms existing methods in both imperceptibility and defense effectiveness, achieving a superior trade-off.

Title: Discovering Scaling Exponents with Physics-Informed Müntz-Szász Networks

• Authors: Gnankan Landry Regis N'guessan, Bum Jun Kim
• Subjects: cs.LG, math.NA
• Abstract URL: https://arxiv.org/abs/2601.22751
• Pdf URL: https://arxiv.org/pdf/2601.22751
• Copy Paste: [[2601.22751]] Discovering Scaling Exponents with Physics-Informed Müntz-Szász Networks(https://arxiv.org/abs/2601.22751)
• Keywords: interpretability
• Abstract: Physical systems near singularities, interfaces, and critical points exhibit power-law scaling, yet standard neural networks leave the governing exponents implicit. We introduce physics-informed M"untz-Sz'asz Networks (MSN-PINN), a power-law basis network that treats scaling exponents as trainable parameters. The model outputs both the solution and its scaling structure. We prove identifiability, or unique recovery, and show that, under these conditions, the squared error between learned and true exponents scales as $O(|\mu - \alpha|^2)$. Across experiments, MSN-PINN achieves single-exponent recovery with 1--5% error under noise and sparse sampling. It recovers corner singularity exponents for the two-dimensional Laplace equation with 0.009% error, matches the classical result of Kondrat'ev (1967), and recovers forcing-induced exponents in singular Poisson problems with 0.03% and 0.05% errors. On a 40-configuration wedge benchmark, it reaches a 100% success rate with 0.022% mean error. Constraint-aware training encodes physical requirements such as boundary condition compatibility and improves accuracy by three orders of magnitude over naive training. By combining the expressiveness of neural networks with the interpretability of asymptotic analysis, MSN-PINN produces learned parameters with direct physical meaning.

Title: OSNIP: Breaking the Privacy-Utility-Efficiency Trilemma in LLM Inference via Obfuscated Semantic Null Space

• Authors: Zhiyuan Cao, Zeyu Ma, Chenhao Yang, Han Zheng, Mingang Chen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22752
• Pdf URL: https://arxiv.org/pdf/2601.22752
• Copy Paste: [[2601.22752]] OSNIP: Breaking the Privacy-Utility-Efficiency Trilemma in LLM Inference via Obfuscated Semantic Null Space(https://arxiv.org/abs/2601.22752)
• Keywords: security, privacy, attack, generative
• Abstract: We propose Obfuscated Semantic Null space Injection for Privacy (OSNIP), a lightweight client-side encryption framework for privacy-preserving LLM inference. Generalizing the geometric intuition of linear kernels to the high-dimensional latent space of LLMs, we formally define the ``Obfuscated Semantic Null Space'', a high-dimensional regime that preserves semantic fidelity while enforcing near-orthogonality to the original embedding. By injecting perturbations that project the original embedding into this space, OSNIP ensures privacy without any post-processing. Furthermore, OSNIP employs a key-dependent stochastic mapping that synthesizes individualized perturbation trajectories unique to each user. Evaluations on 12 generative and classification benchmarks show that OSNIP achieves state-of-the-art performance, sharply reducing attack success rates while maintaining strong model utility under strict security constraints.

Title: Procedural Knowledge Extraction from Industrial Troubleshooting Guides Using Vision Language Models

• Authors: Guillermo Gil de Avalle, Laura Maruster, Christos Emmanouilidis
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22754
• Pdf URL: https://arxiv.org/pdf/2601.22754
• Copy Paste: [[2601.22754]] Procedural Knowledge Extraction from Industrial Troubleshooting Guides Using Vision Language Models(https://arxiv.org/abs/2601.22754)
• Keywords: robust, extraction
• Abstract: Industrial troubleshooting guides encode diagnostic procedures in flowchart-like diagrams where spatial layout and technical language jointly convey meaning. To integrate this knowledge into operator support systems, which assist shop-floor personnel in diagnosing and resolving equipment issues, the information must first be extracted and structured for machine interpretation. However, when performed manually, this extraction is labor-intensive and error-prone. Vision Language Models offer potential to automate this process by jointly interpreting visual and textual meaning, yet their performance on such guides remains underexplored. This paper evaluates two VLMs on extracting structured knowledge, comparing two prompting strategies: standard instruction-guided versus an augmented approach that cues troubleshooting layout patterns. Results reveal model-specific trade-offs between layout sensitivity and semantic robustness, informing practical deployment decisions.

Title: Unveiling Scaling Behaviors in Molecular Language Models: Effects of Model Size, Data, and Representation

• Authors: Dong Xu, Qihua Pan, Sisi Yuan, Jianqiang Li, Zexuan Zhu, Junkai Ji
• Subjects: cs.LG, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2601.22757
• Pdf URL: https://arxiv.org/pdf/2601.22757
• Copy Paste: [[2601.22757]] Unveiling Scaling Behaviors in Molecular Language Models: Effects of Model Size, Data, and Representation(https://arxiv.org/abs/2601.22757)
• Keywords: generative
• Abstract: Molecular generative models, often employing GPT-style language modeling on molecular string representations, have shown promising capabilities when scaled to large datasets and model sizes. However, it remains unclear and subject to debate whether these models adhere to predictable scaling laws under fixed computational budgets, which is a crucial understanding for optimally allocating resources between model size, data volume, and molecular representation. In this study, we systematically investigate the scaling behavior of molecular language models across both pretraining and downstream tasks. We train 300 models and conduct over 10,000 experiments, rigorously controlling compute budgets while independently varying model size, number of training tokens, and molecular representation. Our results demonstrate clear scaling laws in molecular models for both pretraining and downstream transfer, reveal the substantial impact of molecular representation on performance, and explain previously observed inconsistencies in scaling behavior for molecular generation. Additionally, we publicly release the largest library of molecular language models to date to facilitate future research and development. Code and models are available at this https URL.

Title: Sparse Attention as Compact Kernel Regression

• Authors: Saul Santos, Nuno Gonçalves, Daniel C. McNamee, André F.T Martins
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22766
• Pdf URL: https://arxiv.org/pdf/2601.22766
• Copy Paste: [[2601.22766]] Sparse Attention as Compact Kernel Regression(https://arxiv.org/abs/2601.22766)
• Keywords: transformer
• Abstract: Recent work has revealed a link between self-attention mechanisms in transformers and test-time kernel regression via the Nadaraya-Watson estimator, with standard softmax attention corresponding to a Gaussian kernel. However, a kernel-theoretic understanding of sparse attention mechanisms is currently missing. In this paper, we establish a formal correspondence between sparse attention and compact (bounded support) kernels. We show that normalized ReLU and sparsemax attention arise from Epanechnikov kernel regression under fixed and adaptive normalizations, respectively. More generally, we demonstrate that widely used kernels in nonparametric density estimation -- including Epanechnikov, biweight, and triweight -- correspond to $\alpha$-entmax attention with $\alpha = 1 + \frac{1}{n}$ for $n \in \mathbb{N}$, while the softmax/Gaussian relationship emerges in the limit $n \to \infty$. This unified perspective explains how sparsity naturally emerges from kernel design and provides principled alternatives to heuristic top-$k$ attention and other associative memory mechanisms. Experiments with a kernel-regression-based variant of transformers -- Memory Mosaics -- show that kernel-based sparse attention achieves competitive performance on language modeling, in-context learning, and length generalization tasks, offering a principled framework for designing attention mechanisms.

Title: Okara: Detection and Attribution of TLS Man-in-the-Middle Vulnerabilities in Android Apps with Foundation Models

• Authors: Haoyun Yang, Ronghong Huang, Yong Fang, Beizeng Zhang, Junpu Guo, Zhanyu Wu, Xianghang Mi
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2601.22770
• Pdf URL: https://arxiv.org/pdf/2601.22770
• Copy Paste: [[2601.22770]] Okara: Detection and Attribution of TLS Man-in-the-Middle Vulnerabilities in Android Apps with Foundation Models(https://arxiv.org/abs/2601.22770)
• Keywords: secure, security, attack
• Abstract: Transport Layer Security (TLS) is fundamental to secure online communication, yet vulnerabilities in certificate validation that enable Man-in-the-Middle (MitM) attacks remain a pervasive threat in Android apps. Existing detection tools are hampered by low-coverage UI interaction, costly instrumentation, and a lack of scalable root-cause analysis. We present Okara, a framework that leverages foundation models to automate the detection and deep attribution of TLS MitM Vulnerabilities (TMVs). Okara's detection component, TMV-Hunter, employs foundation model-driven GUI agents to achieve high-coverage app interaction, enabling efficient vulnerability discovery at scale. Deploying TMV-Hunter on 37,349 apps from Google Play and a third-party store revealed 8,374 (22.42%) vulnerable apps. Our measurement shows these vulnerabilities are widespread across all popularity levels, affect critical functionalities like authentication and code delivery, and are highly persistent with a median vulnerable lifespan of over 1,300 days. Okara's attribution component, TMV-ORCA, combines dynamic instrumentation with a novel LLM-based classifier to locate and categorize vulnerable code according to a comprehensive new taxonomy. This analysis attributes 41% of vulnerabilities to third-party libraries and identifies recurring insecure patterns, such as empty trust managers and flawed hostname verification. We have initiated a large-scale responsible disclosure effort and will release our tools and datasets to support further research and mitigation.

Title: RASST: Fast Cross-modal Retrieval-Augmented Simultaneous Speech Translation

• Authors: Jiaxuan Luo, Siqi Ouyang, Lei Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22777
• Pdf URL: https://arxiv.org/pdf/2601.22777
• Copy Paste: [[2601.22777]] RASST: Fast Cross-modal Retrieval-Augmented Simultaneous Speech Translation(https://arxiv.org/abs/2601.22777)
• Keywords: large language model
• Abstract: Simultaneous speech translation (SST) produces target text incrementally from partial speech input. Recent speech large language models (Speech LLMs) have substantially improved SST quality, yet they still struggle to correctly translate rare and domain-specific terminology. While retrieval augmentation has been effective for terminology translation in machine translation, bringing retrieval to SST is non-trivial: it requires fast and accurate cross-modal (speech-to-text) retrieval under partial, continually arriving input, and the model must decide whether and when to apply retrieved terms during incremental generation. We propose Retrieval-Augmented Simultaneous Speech Translation (RASST), which tightly integrates cross-modal retrieval into the SST pipeline. RASST trains a lightweight speech-text retriever and performs efficient sliding-window retrieval, providing chunkwise terminology hints to the Speech LLM. We further synthesize training data that teaches the Speech LLM to leverage retrieved terms precisely. Experiments on three language directions of the ACL 60/60 dev set show that RASST improves terminology translation accuracy by up to 16% and increases overall translation quality by up to 3 BLEU points, with ablations confirming the contribution of each component.

Title: Color Matters: Demosaicing-Guided Color Correlation Training for Generalizable AI-Generated Image Detection

• Authors: Nan Zhong, Yiran Xu, Mian Zou
• Subjects: cs.CV, cs.CR
• Abstract URL: https://arxiv.org/abs/2601.22778
• Pdf URL: https://arxiv.org/pdf/2601.22778
• Copy Paste: [[2601.22778]] Color Matters: Demosaicing-Guided Color Correlation Training for Generalizable AI-Generated Image Detection(https://arxiv.org/abs/2601.22778)
• Keywords: robust, generative
• Abstract: As realistic AI-generated images threaten digital authenticity, we address the generalization failure of generative artifact-based detectors by exploiting the intrinsic properties of the camera imaging pipeline. Concretely, we investigate color correlations induced by the color filter array (CFA) and demosaicing, and propose a Demosaicing-guided Color Correlation Training (DCCT) framework for AI-generated image detection. By simulating the CFA sampling pattern, we decompose each color image into a single-channel input (as the condition) and the remaining two channels as the ground-truth targets (for prediction). A self-supervised U-Net is trained to model the conditional distribution of the missing channels from the given one, parameterized via a mixture of logistic functions. Our theoretical analysis reveals that DCCT targets a provable distributional difference in color-correlation features between photographic and AI-generated images. By leveraging these distinct features to construct a binary classifier, DCCT achieves state-of-the-art generalization and robustness, significantly outperforming prior methods across over 20 unseen generators.

Title: Float8@2bits: Entropy Coding Enables Data-Free Model Compression

• Authors: Patrick Putzky, Martin Genzel, Mattes Mollenhauer, Sebastian Schulze, Thomas Wollmann, Stefan Dietzel
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22787
• Pdf URL: https://arxiv.org/pdf/2601.22787
• Copy Paste: [[2601.22787]] Float8@2bits: Entropy Coding Enables Data-Free Model Compression(https://arxiv.org/abs/2601.22787)
• Keywords: robust, data-free
• Abstract: Post-training compression is currently divided into two contrasting regimes. On the one hand, fast, data-free, and model-agnostic methods (e.g., NF4 or HQQ) offer maximum accessibility but suffer from functional collapse at extreme bit-rates below 4 bits. On the other hand, techniques leveraging calibration data or extensive recovery training achieve superior fidelity but impose high computational constraints and face uncertain robustness under data distribution shifts. We introduce EntQuant, the first framework to unite the advantages of these distinct paradigms. By matching the performance of data-dependent methods with the speed and universality of data-free techniques, EntQuant enables practical utility in the extreme compression regime. Our method decouples numerical precision from storage cost via entropy coding, compressing a 70B parameter model in less than 30 minutes. We demonstrate that EntQuant does not only achieve state-of-the-art results on standard evaluation sets and models, but also retains functional performance on more complex benchmarks with instruction-tuned models, all at modest inference overhead.

Title: Sparse or Dense? A Mechanistic Estimation of Computation Density in Transformer-based LLMs

• Authors: Corentin Kervadec, Iuliia Lysova, Marco Baroni, Gemma Boleda
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22795
• Pdf URL: https://arxiv.org/pdf/2601.22795
• Copy Paste: [[2601.22795]] Sparse or Dense? A Mechanistic Estimation of Computation Density in Transformer-based LLMs(https://arxiv.org/abs/2601.22795)
• Keywords: interpretability, transformer, large language model
• Abstract: Transformer-based large language models (LLMs) are comprised of billions of parameters arranged in deep and wide computational graphs. Several studies on LLM efficiency optimization argue that it is possible to prune a significant portion of the parameters, while only marginally impacting performance. This suggests that the computation is not uniformly distributed across the parameters. We introduce here a technique to systematically quantify computation density in LLMs. In particular, we design a density estimator drawing on mechanistic interpretability. We experimentally test our estimator and find that: (1) contrary to what has been often assumed, LLM processing generally involves dense computation; (2) computation density is dynamic, in the sense that models shift between sparse and dense processing regimes depending on the input; (3) per-input density is significantly correlated across LLMs, suggesting that the same inputs trigger either low or high density. Investigating the factors influencing density, we observe that predicting rarer tokens requires higher density, and increasing context length often decreases the density. We believe that our computation density estimator will contribute to a better understanding of the processing at work in LLMs, challenging their symbolic interpretation.

Title: Trackly: A Unified SaaS Platform for User Behavior Analytics and Real Time Rule Based Anomaly Detection

• Authors: Md Zahurul Haque, Md. Hafizur Rahman, Yeahyea Sarker
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22800
• Pdf URL: https://arxiv.org/pdf/2601.22800
• Copy Paste: [[2601.22800]] Trackly: A Unified SaaS Platform for User Behavior Analytics and Real Time Rule Based Anomaly Detection(https://arxiv.org/abs/2601.22800)
• Keywords: secure, security, attack
• Abstract: Understanding user behavior is essential for improving digital experiences, optimizing business conversions, and mitigating threats like account takeovers, fraud, and bot attacks. Most platforms separate product analytics and security, creating fragmented visibility and delayed threat detection. Trackly, a scalable SaaS platform, unifies comprehensive user behavior analytics with real time, rule based anomaly detection. It tracks sessions, IP based geo location, device browser fingerprints, and granular events such as page views, add to cart, and checkouts. Suspicious activities logins from new devices or locations, impossible travel (Haversine formula), rapid bot like actions, VPN proxy usage, or multiple accounts per IP are flagged via configurable rules with weighted risk scoring, enabling transparent, explainable decisions. A real time dashboard provides global session maps, DAU MAU, bounce rates, and session durations. Integration is simplified with a lightweight JavaScript SDK and secure REST APIs. Implemented on a multi tenant microservices stack (this http URL Core, MongoDB, RabbitMQ, this http URL), Trackly achieved 98.1% accuracy, 97.7% precision, and 2.25% false positives on synthetic datasets, proving its efficiency for SMEs and ecommerce.

Title: Clipping-Free Policy Optimization for Large Language Models

• Authors: Ömer Veysel Çağatan, Barış Akgün, Gözde Gül Şahin, Xuandong Zhao
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22801
• Pdf URL: https://arxiv.org/pdf/2601.22801
• Copy Paste: [[2601.22801]] Clipping-Free Policy Optimization for Large Language Models(https://arxiv.org/abs/2601.22801)
• Keywords: large language model
• Abstract: Reinforcement learning has become central to post-training large language models, yet dominant algorithms rely on clipping mechanisms that introduce optimization issues at scale, including zero-gradient regions, reward hacking, and training instability. We propose Clipping-Free Policy Optimization (CFPO), which replaces heuristic clipping with a convex quadratic penalty derived from Total Variation divergence constraints, yielding an everywhere-differentiable objective that enforces stable policy updates without hard boundaries. We evaluate CFPO across both reasoning and alignment settings. In reasoning, CFPO matches clipping-based methods on downstream benchmarks while extending the stable training regime. In alignment, CFPO mitigates verbosity exploitation and reduces capability degradation, while achieving competitive instruction-following performance. CFPO requires only a one-line code change and no additional hyperparameters. Our results suggest that CFPO is a promising drop-in alternative to clipping-based methods for LLM post-training.

Title: Trojan-Resilient NTT: Protecting Against Control Flow and Timing Faults on Reconfigurable Platforms

• Authors: Rourab Paul, Krishnendu Guha, Amlan Chakrabarti
• Subjects: cs.CR, cs.AR
• Abstract URL: https://arxiv.org/abs/2601.22804
• Pdf URL: https://arxiv.org/pdf/2601.22804
• Copy Paste: [[2601.22804]] Trojan-Resilient NTT: Protecting Against Control Flow and Timing Faults on Reconfigurable Platforms(https://arxiv.org/abs/2601.22804)
• Keywords: secure, protect, attack
• Abstract: Number Theoretic Transform (NTT) is the most essential component for polynomial multiplications used in lattice-based Post-Quantum Cryptography (PQC) algorithms such as Kyber, Dilithium, NTRU etc. However, side-channel attacks (SCA) and hardware vulnerabilities in the form of hardware Trojans may alter control signals to disrupt the circuit's control flow and introduce unconventional delays in the critical hardware of PQC. Hardware Trojans, especially on control signals, are more low cost and impactful than data signals because a single corrupted control signal can disrupt or bypass entire computation sequences, whereas data faults usually cause only localized errors. On the other hand, adversaries can perform Soft Analytical Side Channel Attacks (SASCA) on the design using the inserted hardware Trojan. In this paper, we present a secure NTT architecture capable of detecting unconventional delays, control-flow disruptions, and SASCA, while providing an adaptive fault-correction methodology for their mitigation. Extensive simulations and implementations of our Secure NTT on Artix-7 FPGA with different Kyber variants show that our fault detection and correction modules can efficiently detect and correct faults whether caused unintentionally or intentionally by hardware Trojans with a high success rate, while introducing only modest area and time overheads.

Title: SOMBRERO: Measuring and Steering Boundary Placement in End-to-End Hierarchical Sequence Models

• Authors: Pit Neitemeier, Alessio Serra, Jiaze Li, Sascha Wirges, Lukas Balles, Jan Hendrik Metzen
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22805
• Pdf URL: https://arxiv.org/pdf/2601.22805
• Copy Paste: [[2601.22805]] SOMBRERO: Measuring and Steering Boundary Placement in End-to-End Hierarchical Sequence Models(https://arxiv.org/abs/2601.22805)
• Keywords: segmentation
• Abstract: Hierarchical sequence models replace fixed tokenization with learned segmentations that compress long byte sequences for efficient autoregressive modeling. While recent end-to-end methods can learn meaningful boundaries from the language-modeling objective alone, it remains difficult to quantitatively assess and systematically steer where compute is spent. We introduce a router-agnostic metric of boundary quality, boundary enrichment B, which measures how strongly chunk starts concentrate on positions with high next-byte surprisal. Guided by this metric, we propose Sombrero, which steers boundary placement toward predictive difficulty via a confidence-alignment boundary loss and stabilizes boundary learning by applying confidence-weighted smoothing at the input level rather than on realized chunks. On 1B scale, across UTF-8 corpora covering English and German text as well as code and mathematical content, Sombrero improves the accuracy-efficiency trade-off and yields boundaries that more consistently align compute with hard-to-predict positions.

Title: Diachronic Stereo Matching for Multi-Date Satellite Imagery

• Authors: Elías Masquil (IIE, UDELAR), Luca Savant Aira (Polito), Roger Marí, Thibaud Ehret (AMIAD), Pablo Musé (IIE, UDELAR, CB), Gabriele Facciolo (CB, IUF)
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22808
• Pdf URL: https://arxiv.org/pdf/2601.22808
• Copy Paste: [[2601.22808]] Diachronic Stereo Matching for Multi-Date Satellite Imagery(https://arxiv.org/abs/2601.22808)
• Keywords: robust
• Abstract: Recent advances in image-based satellite 3D reconstruction have progressed along two complementary directions. On one hand, multi-date approaches using NeRF or Gaussian-splatting jointly model appearance and geometry across many acquisitions, achieving accurate reconstructions on opportunistic imagery with numerous observations. On the other hand, classical stereoscopic reconstruction pipelines deliver robust and scalable results for simultaneous or quasi-simultaneous image pairs. However, when the two images are captured months apart, strong seasonal, illumination, and shadow changes violate standard stereoscopic assumptions, causing existing pipelines to fail. This work presents the first Diachronic Stereo Matching method for satellite imagery, enabling reliable 3D reconstruction from temporally distant pairs. Two advances make this possible: (1) fine-tuning a state-of-the-art deep stereo network that leverages monocular depth priors, and (2) exposing it to a dataset specifically curated to include a diverse set of diachronic image pairs. In particular, we start from a pretrained MonSter model, trained initially on a mix of synthetic and real datasets such as SceneFlow and KITTI, and fine-tune it on a set of stereo pairs derived from the DFC2019 remote sensing challenge. This dataset contains both synchronic and diachronic pairs under diverse seasonal and illumination conditions. Experiments on multi-date WorldView-3 imagery demonstrate that our approach consistently surpasses classical pipelines and unadapted deep stereo models on both synchronic and diachronic settings. Fine-tuning on temporally diverse images, together with monocular priors, proves essential for enabling 3D reconstruction from previously incompatible acquisition dates. Left image (winter) Right image (autumn) DSM geometry Ours (1.23 m) Zero-shot (3.99 m) LiDAR GT Figure 1. Output geometry for a winter-autumn image pair from Omaha (OMA 331 test scene). Our method recovers accurate geometry despite the diachronic nature of the pair, exhibiting strong appearance changes, which cause existing zero-shot methods to fail. Missing values due to perspective shown in black. Mean altitude error in parentheses; lower is better.

Title: FarmMind: Reasoning-Query-Driven Dynamic Segmentation for Farmland Remote Sensing Images

• Authors: Haiyang Wu, Weiliang Mu, Jipeng Zhang, Zhong Dandan, Zhuofei Du, Haifeng Li, Tao Chao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22809
• Pdf URL: https://arxiv.org/pdf/2601.22809
• Copy Paste: [[2601.22809]] FarmMind: Reasoning-Query-Driven Dynamic Segmentation for Farmland Remote Sensing Images(https://arxiv.org/abs/2601.22809)
• Keywords: segmentation
• Abstract: Existing methods for farmland remote sensing image (FRSI) segmentation generally follow a static segmentation paradigm, where analysis relies solely on the limited information contained within a single input patch. Consequently, their reasoning capability is limited when dealing with complex scenes characterized by ambiguity and visual uncertainty. In contrast, human experts, when interpreting remote sensing images in such ambiguous cases, tend to actively query auxiliary images (such as higher-resolution, larger-scale, or temporally adjacent data) to conduct cross-verification and achieve more comprehensive reasoning. Inspired by this, we propose a reasoning-query-driven dynamic segmentation framework for FRSIs, named FarmMind. This framework breaks through the limitations of the static segmentation paradigm by introducing a reasoning-query mechanism, which dynamically and on-demand queries external auxiliary images to compensate for the insufficient information in a single input image. Unlike direct queries, this mechanism simulates the thinking process of human experts when faced with segmentation ambiguity: it first analyzes the root causes of segmentation ambiguities through reasoning, and then determines what type of auxiliary image needs to be queried based on this analysis. Extensive experiments demonstrate that FarmMind achieves superior segmentation performance and stronger generalization ability compared with existing methods. The source code and dataset used in this work are publicly available at: this https URL.

Title: Cascaded Flow Matching for Heterogeneous Tabular Data with Mixed-Type Features

• Authors: Markus Mueller, Kathrin Gruber, Dennis Fok
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2601.22816
• Pdf URL: https://arxiv.org/pdf/2601.22816
• Copy Paste: [[2601.22816]] Cascaded Flow Matching for Heterogeneous Tabular Data with Mixed-Type Features(https://arxiv.org/abs/2601.22816)
• Keywords: diffusion, generative
• Abstract: Advances in generative modeling have recently been adapted to tabular data containing discrete and continuous features. However, generating mixed-type features that combine discrete states with an otherwise continuous distribution in a single feature remains challenging. We advance the state-of-the-art in diffusion models for tabular data with a cascaded approach. We first generate a low-resolution version of a tabular data row, that is, the collection of the purely categorical features and a coarse categorical representation of numerical features. Next, this information is leveraged in the high-resolution flow matching model via a novel guided conditional probability path and data-dependent coupling. The low-resolution representation of numerical features explicitly accounts for discrete outcomes, such as missing or inflated values, and therewith enables a more faithful generation of mixed-type features. We formally prove that this cascade tightens the transport cost bound. The results indicate that our model generates significantly more realistic samples and captures distributional details more accurately, for example, the detection score increases by 40%.

Title: Hide and Seek in Embedding Space: Geometry-based Steganography and Detection in Large Language Models

• Authors: Charles Westphal, Keivan Navaie, Fernando E. Rosas
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22818
• Pdf URL: https://arxiv.org/pdf/2601.22818
• Copy Paste: [[2601.22818]] Hide and Seek in Embedding Space: Geometry-based Steganography and Detection in Large Language Models(https://arxiv.org/abs/2601.22818)
• Keywords: defense, attack, interpretability, large language model
• Abstract: Fine-tuned LLMs can covertly encode prompt secrets into outputs via steganographic channels. Prior work demonstrated this threat but relied on trivially recoverable encodings. We formalize payload recoverability via classifier accuracy and show previous schemes achieve 100\% recoverability. In response, we introduce low-recoverability steganography, replacing arbitrary mappings with embedding-space-derived ones. For Llama-8B (LoRA) and Ministral-8B (LoRA) trained on TrojanStego prompts, exact secret recovery rises from 17$\rightarrow$30\% (+78\%) and 24$\rightarrow$43\% (+80\%) respectively, while on Llama-70B (LoRA) trained on Wiki prompts, it climbs from 9$\rightarrow$19\% (+123\%), all while reducing payload recoverability. We then discuss detection. We argue that detecting fine-tuning-based steganographic attacks requires approaches beyond traditional steganalysis. Standard approaches measure distributional shift, which is an expected side-effect of fine-tuning. Instead, we propose a mechanistic interpretability approach: linear probes trained on later-layer activations detect the secret with up to 33\% higher accuracy in fine-tuned models compared to base models, even for low-recoverability schemes. This suggests that malicious fine-tuning leaves actionable internal signatures amenable to interpretability-based defenses.

Title: Offline Reinforcement Learning of High-Quality Behaviors Under Robust Style Alignment

• Authors: Mathieu Petitbois, Rémy Portelas, Sylvain Lamprier
• Subjects: cs.LG, cs.AI, cs.RO
• Abstract URL: https://arxiv.org/abs/2601.22823
• Pdf URL: https://arxiv.org/pdf/2601.22823
• Copy Paste: [[2601.22823]] Offline Reinforcement Learning of High-Quality Behaviors Under Robust Style Alignment(https://arxiv.org/abs/2601.22823)
• Keywords: robust
• Abstract: We study offline reinforcement learning of style-conditioned policies using explicit style supervision via subtrajectory labeling functions. In this setting, aligning style with high task performance is particularly challenging due to distribution shift and inherent conflicts between style and reward. Existing methods, despite introducing numerous definitions of style, often fail to reconcile these objectives effectively. To address these challenges, we propose a unified definition of behavior style and instantiate it into a practical framework. Building on this, we introduce Style-Conditioned Implicit Q-Learning (SCIQL), which leverages offline goal-conditioned RL techniques, such as hindsight relabeling and value learning, and combine it with a new Gated Advantage Weighted Regression mechanism to efficiently optimize task performance while preserving style alignment. Experiments demonstrate that SCIQL achieves superior performance on both objectives compared to prior offline methods. Code, datasets and visuals are available in: this https URL.

Title: A Comparative Evaluation of Large Vision-Language Models for 2D Object Detection under SOTIF Conditions

• Authors: Ji Zhou, Yilin Ding, Yongqi Zhao, Jiachen Xu, Arno Eichberger
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2601.22830
• Pdf URL: https://arxiv.org/pdf/2601.22830
• Copy Paste: [[2601.22830]] A Comparative Evaluation of Large Vision-Language Models for 2D Object Detection under SOTIF Conditions(https://arxiv.org/abs/2601.22830)
• Keywords: robust
• Abstract: Reliable environmental perception remains one of the main obstacles for safe operation of automated vehicles. Safety of the Intended Functionality (SOTIF) concerns safety risks from perception insufficiencies, particularly under adverse conditions where conventional detectors often falter. While Large Vision-Language Models (LVLMs) demonstrate promising semantic reasoning, their quantitative effectiveness for safety-critical 2D object detection is underexplored. This paper presents a systematic evaluation of ten representative LVLMs using the PeSOTIF dataset, a benchmark specifically curated for long-tail traffic scenarios and environmental degradations. Performance is quantitatively compared against the classical perception approach, a YOLO-based detector. Experimental results reveal a critical trade-off: top-performing LVLMs (e.g., Gemini 3, Doubao) surpass the YOLO baseline in recall by over 25% in complex natural scenarios, exhibiting superior robustness to visual degradation. Conversely, the baseline retains an advantage in geometric precision for synthetic perturbations. These findings highlight the complementary strengths of semantic reasoning versus geometric regression, supporting the use of LVLMs as high-level safety validators in SOTIF-oriented automated driving systems.

Title: NativeTok: Native Visual Tokenization for Improved Image Generation

• Authors: Bin Wu, Mengqi Huang, Weinan Jia, Zhendong Mao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22837
• Pdf URL: https://arxiv.org/pdf/2601.22837
• Copy Paste: [[2601.22837]] NativeTok: Native Visual Tokenization for Improved Image Generation(https://arxiv.org/abs/2601.22837)
• Keywords: transformer, generative
• Abstract: VQ-based image generation typically follows a two-stage pipeline: a tokenizer encodes images into discrete tokens, and a generative model learns their dependencies for reconstruction. However, improved tokenization in the first stage does not necessarily enhance the second-stage generation, as existing methods fail to constrain token dependencies. This mismatch forces the generative model to learn from unordered distributions, leading to bias and weak coherence. To address this, we propose native visual tokenization, which enforces causal dependencies during tokenization. Building on this idea, we introduce NativeTok, a framework that achieves efficient reconstruction while embedding relational constraints within token sequences. NativeTok consists of: (1) a Meta Image Transformer (MIT) for latent image modeling, and (2) a Mixture of Causal Expert Transformer (MoCET), where each lightweight expert block generates a single token conditioned on prior tokens and latent features. We further design a Hierarchical Native Training strategy that updates only new expert blocks, ensuring training efficiency. Extensive experiments demonstrate the effectiveness of NativeTok.

Title: Neural Clothing Tryer: Customized Virtual Try-On via Semantic Enhancement and Controlling Diffusion Model

• Authors: Zhijing Yang, Weiwei Zhang, Mingliang Yang, Siyuan Peng, Yukai Shi, Junpeng Tan, Tianshui Chen, Liruo Zhong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22838
• Pdf URL: https://arxiv.org/pdf/2601.22838
• Copy Paste: [[2601.22838]] Neural Clothing Tryer: Customized Virtual Try-On via Semantic Enhancement and Controlling Diffusion Model(https://arxiv.org/abs/2601.22838)
• Keywords: diffusion
• Abstract: This work aims to address a novel Customized Virtual Try-ON (Cu-VTON) task, enabling the superimposition of a specified garment onto a model that can be customized in terms of appearance, posture, and additional attributes. Compared with traditional VTON task, it enables users to tailor digital avatars to their individual preferences, thereby enhancing the virtual fitting experience with greater flexibility and engagement. To address this task, we introduce a Neural Clothing Tryer (NCT) framework, which exploits the advanced diffusion models equipped with semantic enhancement and controlling modules to better preserve semantic characterization and textural details of the garment and meanwhile facilitating the flexible editing of the model's postures and appearances. Specifically, NCT introduces a semantic-enhanced module to take semantic descriptions of garments and utilizes a visual-language encoder to learn aligned features across modalities. The aligned features are served as condition input to the diffusion model to enhance the preservation of the garment's semantics. Then, a semantic controlling module is designed to take the garment image, tailored posture image, and semantic description as input to maintain garment details while simultaneously editing model postures, expressions, and various attributes. Extensive experiments on the open available benchmark demonstrate the superior performance of the proposed NCT framework.

Title: Unconditional flow-based time series generation with equivariance-regularised latent spaces

• Authors: Camilo Carvajal Reyes, Felipe Tobar
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22848
• Pdf URL: https://arxiv.org/pdf/2601.22848
• Copy Paste: [[2601.22848]] Unconditional flow-based time series generation with equivariance-regularised latent spaces(https://arxiv.org/abs/2601.22848)
• Keywords: diffusion, generative
• Abstract: Flow-based models have proven successful for time-series generation, particularly when defined in lower-dimensional latent spaces that enable efficient sampling. However, how to design latent representations with desirable equivariance properties for time-series generative modelling remains underexplored. In this work, we propose a latent flow-matching framework in which equivariance is explicitly encouraged through a simple regularisation of a pre-trained autoencoder. Specifically, we introduce an equivariance loss that enforces consistency between transformed signals and their reconstructions, and use it to fine-tune latent spaces with respect to basic time-series transformations such as translation and amplitude scaling. We show that these equivariance-regularised latent spaces improve generation quality while preserving the computational advantages of latent flow models. Experiments on multiple real-world datasets demonstrate that our approach consistently outperforms existing diffusion-based baselines in standard time-series generation metrics, while achieving orders-of-magnitude faster sampling. These results highlight the practical benefits of incorporating geometric inductive biases into latent generative models for time series.

Title: When Meanings Meet: Investigating the Emergence and Quality of Shared Concept Spaces during Multilingual Language Model Training

• Authors: Felicia Körner, Max Müller-Eberstein, Anna Korhonen, Barbara Plank
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22851
• Pdf URL: https://arxiv.org/pdf/2601.22851
• Copy Paste: [[2601.22851]] When Meanings Meet: Investigating the Emergence and Quality of Shared Concept Spaces during Multilingual Language Model Training(https://arxiv.org/abs/2601.22851)
• Keywords: interpretability, large language model
• Abstract: Training Large Language Models (LLMs) with high multilingual coverage is becoming increasingly important -- especially when monolingual resources are scarce. Recent studies have found that LLMs process multilingual inputs in shared concept spaces, thought to support generalization and cross-lingual transfer. However, these prior studies often do not use causal methods, lack deeper error analysis or focus on the final model only, leaving open how these spaces emerge during training. We investigate the development of language-agnostic concept spaces during pretraining of EuroLLM through the causal interpretability method of activation patching. We isolate cross-lingual concept representations, then inject them into a translation prompt to investigate how consistently translations can be altered, independently of the language. We find that shared concept spaces emerge early} and continue to refine, but that alignment with them is language-dependent}. Furthermore, in contrast to prior work, our fine-grained manual analysis reveals that some apparent gains in translation quality reflect shifts in behavior -- like selecting senses for polysemous words or translating instead of copying cross-lingual homographs -- rather than improved translation ability. Our findings offer new insight into the training dynamics of cross-lingual alignment and the conditions under which causal interpretability methods offer meaningful insights in multilingual contexts.

Title: Hierarchical Shift Mixing -- Beyond Dense Attention in Transformers

• Authors: Robert Forchheimer
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22852
• Pdf URL: https://arxiv.org/pdf/2601.22852
• Copy Paste: [[2601.22852]] Hierarchical Shift Mixing -- Beyond Dense Attention in Transformers(https://arxiv.org/abs/2601.22852)
• Keywords: transformer, large language model
• Abstract: Since the introduction of the Transformer architecture for large language models, the softmax-based attention layer has faced increasing scrutinity due to its quadratic-time computational complexity. Attempts have been made to replace it with less complex methods, at the cost of reduced performance in most cases. We introduce Hierarchical Shift Mixing (HSM), a general framework for token mixing that distributes pairwise token interactions across Transformer layers rather than computing them densely within each layer. HSM enables linear-time complexity while remaining agnostic to the specific mixing function. We show that even simple HSM variants achieve performance close to softmax attention, and that hybrid architectures combining HSM with softmax attention can outperform a GPT-style Transformer baseline while reducing computational cost during both training and inference.

Title: Inference-Time Dynamic Modality Selection for Incomplete Multimodal Classification

• Authors: Siyi Du, Xinzhe Luo, Declan P. O'Regan, Chen Qin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22853
• Pdf URL: https://arxiv.org/pdf/2601.22853
• Copy Paste: [[2601.22853]] Inference-Time Dynamic Modality Selection for Incomplete Multimodal Classification(https://arxiv.org/abs/2601.22853)
• Keywords: robust
• Abstract: Multimodal deep learning (MDL) has achieved remarkable success across various domains, yet its practical deployment is often hindered by incomplete multimodal data. Existing incomplete MDL methods either discard missing modalities, risking the loss of valuable task-relevant information, or recover them, potentially introducing irrelevant noise, leading to the discarding-imputation dilemma. To address this dilemma, in this paper, we propose DyMo, a new inference-time dynamic modality selection framework that adaptively identifies and integrates reliable recovered modalities, fully exploring task-relevant information beyond the conventional discard-or-impute paradigm. Central to DyMo is a novel selection algorithm that maximizes multimodal task-relevant information for each test sample. Since direct estimation of such information at test time is intractable due to the unknown data distribution, we theoretically establish a connection between information and the task loss, which we compute at inference time as a tractable proxy. Building on this, a novel principled reward function is proposed to guide modality selection. In addition, we design a flexible multimodal network architecture compatible with arbitrary modality combinations, alongside a tailored training strategy for robust representation learning. Extensive experiments on diverse natural and medical image datasets show that DyMo significantly outperforms state-of-the-art incomplete/dynamic MDL methods across various missing-data scenarios. Our code is available at this https URL.

Title: Matterhorn: Efficient Analog Sparse Spiking Transformer Architecture with Masked Time-To-First-Spike Encoding

• Authors: Zhanglu Yan, Kaiwen Tang, Zixuan Zhu, Zhenyu Bai, Qianhui Liu, Weng-Fai Wong
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22876
• Pdf URL: https://arxiv.org/pdf/2601.22876
• Copy Paste: [[2601.22876]] Matterhorn: Efficient Analog Sparse Spiking Transformer Architecture with Masked Time-To-First-Spike Encoding(https://arxiv.org/abs/2601.22876)
• Keywords: transformer
• Abstract: Spiking neural networks (SNNs) have emerged as a promising candidate for energy-efficient LLM inference. However, current energy evaluations for SNNs primarily focus on counting accumulate operations, and fail to account for real-world hardware costs such as data movement, which can consume nearly 80% of the total energy. In this paper, we propose Matterhorn, a spiking transformer that integrates a novel masked time-to-first-spike (M-TTFS) encoding method to reduce spike movement and a memristive synapse unit (MSU) to eliminate weight access overhead. M-TTFS employs a masking strategy that reassigns the zero-energy silent state (a spike train of all 0s) to the most frequent membrane potential rather than the lowest. This aligns the coding scheme with the data distribution, minimizing spike movement energy without information loss. We further propose a `dead zone' strategy that maximizes sparsity by mapping all values within a given range to the silent state. At the hardware level, the MSU utilizes compute-in-memory (CIM) technology to perform analog integration directly within memory, effectively removing weight access costs. On the GLUE benchmark, Matterhorn establishes a new state-of-the-art, surpassing existing SNNs by 1.42% in average accuracy while delivering a 2.31 times improvement in energy efficiency.

Title: Synthetic Time Series Generation via Complex Networks

• Authors: Jaime Vale, Vanessa Freitas Silva, Maria Eduarda Silva, Fernando Silva
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22879
• Pdf URL: https://arxiv.org/pdf/2601.22879
• Copy Paste: [[2601.22879]] Synthetic Time Series Generation via Complex Networks(https://arxiv.org/abs/2601.22879)
• Keywords: privacy, robust, generative
• Abstract: Time series data are essential for a wide range of applications, particularly in developing robust machine learning models. However, access to high-quality datasets is often limited due to privacy concerns, acquisition costs, and labeling challenges. Synthetic time series generation has emerged as a promising solution to address these constraints. In this work, we present a framework for generating synthetic time series by leveraging complex networks mappings. Specifically, we investigate whether time series transformed into Quantile Graphs (QG) -- and then reconstructed via inverse mapping -- can produce synthetic data that preserve the statistical and structural properties of the original. We evaluate the fidelity and utility of the generated data using both simulated and real-world datasets, and compare our approach against state-of-the-art Generative Adversarial Network (GAN) methods. Results indicate that our quantile graph-based methodology offers a competitive and interpretable alternative for synthetic time series generation.

Title: Leveraging LLMs For Turkish Skill Extraction

• Authors: Ezgi Arslan İltüzer, Özgür Anıl Özlü, Vahid Farajijobehdar, Gülşen Eryiğit
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22885
• Pdf URL: https://arxiv.org/pdf/2601.22885
• Copy Paste: [[2601.22885]] Leveraging LLMs For Turkish Skill Extraction(https://arxiv.org/abs/2601.22885)
• Keywords: extraction, large language model
• Abstract: Skill extraction is a critical component of modern recruitment systems, enabling efficient job matching, personalized recommendations, and labor market analysis. Despite Türkiye's significant role in the global workforce, Turkish, a morphologically complex language, lacks both a skill taxonomy and a dedicated skill extraction dataset, resulting in underexplored research in skill extraction for Turkish. This article seeks the answers to three research questions: 1) How can skill extraction be effectively performed for this language, in light of its low resource nature? 2)~What is the most promising model? 3) What is the impact of different Large Language Models (LLMs) and prompting strategies on skill extraction (i.e., dynamic vs. static few-shot samples, varying context information, and encouraging causal reasoning)? The article introduces the first Turkish skill extraction dataset and performance evaluations of automated skill extraction using LLMs. The manually annotated dataset contains 4,819 labeled skill spans from 327 job postings across different occupation areas. The use of LLM outperforms supervised sequence labeling when used in an end-to-end pipeline, aligning extracted spans with standardized skills in the ESCO taxonomy more effectively. The best-performing configuration, utilizing Claude Sonnet 3.7 with dynamic few-shot prompting for skill identification, embedding-based retrieval, and LLM-based reranking for skill linking, achieves an end-to-end performance of 0.56, positioning Turkish alongside similar studies in other languages, which are few in the literature. Our findings suggest that LLMs can improve skill extraction performance in low-resource settings, and we hope that our work will accelerate similar research on skill extraction for underrepresented languages.

Title: MoVE: Mixture of Value Embeddings -- A New Axis for Scaling Parametric Memory in Autoregressive Models

• Authors: Yangyan Li
• Subjects: cs.LG, cs.AI, cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22887
• Pdf URL: https://arxiv.org/pdf/2601.22887
• Copy Paste: [[2601.22887]] MoVE: Mixture of Value Embeddings -- A New Axis for Scaling Parametric Memory in Autoregressive Models(https://arxiv.org/abs/2601.22887)
• Keywords: generative
• Abstract: Autoregressive sequence modeling stands as the cornerstone of modern Generative AI, powering results across diverse modalities ranging from text generation to image generation. However, a fundamental limitation of this paradigm is the rigid structural coupling of model capacity to computational cost: expanding a model's parametric memory -- its repository of factual knowledge or visual patterns -- traditionally requires deepening or widening the network, which incurs a proportional rise in active FLOPs. In this work, we introduce $\textbf{MoVE (Mixture of Value Embeddings)}$, a mechanism that breaks this coupling and establishes a new axis for scaling capacity. MoVE decouples memory from compute by introducing a global bank of learnable value embeddings shared across all attention layers. For every step in the sequence, the model employs a differentiable soft gating mechanism to dynamically mix retrieved concepts from this bank into the standard value projection. This architecture allows parametric memory to be scaled independently of network depth by simply increasing the number of embedding slots. We validate MoVE through strictly controlled experiments on two representative applications of autoregressive modeling: Text Generation and Image Generation. In both domains, MoVE yields consistent performance improvements over standard and layer-wise memory baselines, enabling the construction of "memory-dense" models that achieve lower perplexity and higher fidelity than their dense counterparts at comparable compute budgets.

Title: DiffuSpeech: Silent Thought, Spoken Answer via Unified Speech-Text Diffusion

• Authors: Yuxuan Lou, Ziming Wu, Yaochen Wang, Yong Liu, Yingxuan Ren, Fuming Lai, Shaobing Lian, Jie Tang, Yang You
• Subjects: cs.CL, cs.AI, cs.LG, cs.SD
• Abstract URL: https://arxiv.org/abs/2601.22889
• Pdf URL: https://arxiv.org/pdf/2601.22889
• Copy Paste: [[2601.22889]] DiffuSpeech: Silent Thought, Spoken Answer via Unified Speech-Text Diffusion(https://arxiv.org/abs/2601.22889)
• Keywords: diffusion, generative
• Abstract: Current speech language models generate responses directly without explicit reasoning, leading to errors that cannot be corrected once audio is produced. We introduce \textbf{``Silent Thought, Spoken Answer''} -- a paradigm where speech LLMs generate internal text reasoning alongside spoken responses, with thinking traces informing speech quality. To realize this, we present \method{}, the first diffusion-based speech-text language model supporting both understanding and generation, unifying discrete text and tokenized speech under a single masked diffusion framework. Unlike autoregressive approaches, \method{} jointly generates reasoning traces and speech tokens through iterative denoising, with modality-specific masking schedules. We also construct \dataset{}, the first speech QA dataset with paired text reasoning traces, containing 26K samples totaling 319 hours. Experiments show \method{} achieves state-of-the-art speech-to-speech QA accuracy, outperforming the best baseline by up to 9 points, while attaining the best TTS quality among generative models (6.2\% WER) and preserving language understanding (66.2\% MMLU). Ablations confirm that both the diffusion architecture and thinking traces contribute to these gains.

Title: Assessing the Real-World Impact of Post-Quantum Cryptography on WPA-Enterprise Networks

• Authors: Lukas Köder, Nils Lohmiller, Phil Schmieder, Bastian Buck, Michael Menth, Tobias Heer
• Subjects: cs.CR, cs.NI, cs.PF
• Abstract URL: https://arxiv.org/abs/2601.22892
• Pdf URL: https://arxiv.org/pdf/2601.22892
• Copy Paste: [[2601.22892]] Assessing the Real-World Impact of Post-Quantum Cryptography on WPA-Enterprise Networks(https://arxiv.org/abs/2601.22892)
• Keywords: security, protect
• Abstract: The advent of large-scale quantum computers poses a significant threat to contemporary network security protocols, including Wi-Fi Protected Access (WPA)-Enterprise authentication. To mitigate this threat, the adoption of Post-Quantum Cryptography (PQC) is critical. In this work, we investigate the performance impact of PQC algorithms on WPA-Enterprise-based authentication. To this end, we conduct an experimental evaluation of authentication latency using a testbed built with the open-source tools FreeRADIUS and hostapd, measuring the time spent at the client, access point, and RADIUS server. We evaluate multiple combinations of PQC algorithms and analyze their performance overhead in comparison to currently deployed cryptographic schemes. Beyond performance, we assess the security implications of these algorithm choices by relating authentication mechanisms to the quantum effort required for their exploitation. This perspective enables a systematic categorization of PQ-relevant weaknesses in WPA-Enterprise according to their practical urgency. The evaluation results show that, although PQC introduces additional authentication latency, combinations such as ML-DSA-65 and Falcon-1024 used in conjunction with ML-KEM provide a favorable trade-off between security and performance. Furthermore, we demonstrate that the resulting overhead can be effectively mitigated through session resumption. Overall, this work presents a first real-world performance evaluation of PQC-enabled WPA-Enterprise authentication and demonstrates its practical feasibility for enterprise Wi-Fi deployments.

Title: Uncertainty-Aware Extrapolation in Bayesian Oblique Trees

• Authors: Viktor Andonovikj, Sašo Džeroski, Pavle Boškoski
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22899
• Pdf URL: https://arxiv.org/pdf/2601.22899
• Copy Paste: [[2601.22899]] Uncertainty-Aware Extrapolation in Bayesian Oblique Trees(https://arxiv.org/abs/2601.22899)
• Keywords: interpretability
• Abstract: Decision trees are widely used due to their interpretability and efficiency, but they struggle in regression tasks that require reliable extrapolation and well-calibrated uncertainty. Piecewise-constant leaf predictions are bounded by the training targets and often become overconfident under distribution shift. We propose a single-tree Bayesian model that extends VSPYCT by equipping each leaf with a GP predictor. Bayesian oblique splits provide uncertainty-aware partitioning of the input space, while GP leaves model local functional behaviour and enable principled extrapolation beyond the observed target range. We present an efficient inference and prediction scheme that combines posterior sampling of split parameters with \gls{gp} posterior predictions, and a gating mechanism that activates GP-based extrapolation when inputs fall outside the training support of a leaf. Experiments on benchmark regression tasks show improvements in the predictive performance compared to standard variational oblique trees, and substantial performance gains in extrapolation scenarios.

Title: DINO-SAE: DINO Spherical Autoencoder for High-Fidelity Image Reconstruction and Generation

• Authors: Hun Chang, Byunghee Cha, Jong Chul Ye
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22904
• Pdf URL: https://arxiv.org/pdf/2601.22904
• Copy Paste: [[2601.22904]] DINO-SAE: DINO Spherical Autoencoder for High-Fidelity Image Reconstruction and Generation(https://arxiv.org/abs/2601.22904)
• Keywords: diffusion, transformer, generative
• Abstract: Recent studies have explored using pretrained Vision Foundation Models (VFMs) such as DINO for generative autoencoders, showing strong generative performance. Unfortunately, existing approaches often suffer from limited reconstruction fidelity due to the loss of high-frequency details. In this work, we present the DINO Spherical Autoencoder (DINO-SAE), a framework that bridges semantic representation and pixel-level reconstruction. Our key insight is that semantic information in contrastive representations is primarily encoded in the direction of feature vectors, while forcing strict magnitude matching can hinder the encoder from preserving fine-grained details. To address this, we introduce Hierarchical Convolutional Patch Embedding module that enhances local structure and texture preservation, and Cosine Similarity Alignment objective that enforces semantic consistency while allowing flexible feature magnitudes for detail retention. Furthermore, leveraging the observation that SSL-based foundation model representations intrinsically lie on a hypersphere, we employ Riemannian Flow Matching to train a Diffusion Transformer (DiT) directly on this spherical latent manifold. Experiments on ImageNet-1K demonstrate that our approach achieves state-of-the-art reconstruction quality, reaching 0.37 rFID and 26.2 dB PSNR, while maintaining strong semantic alignment to the pretrained VFM. Notably, our Riemannian Flow Matching-based DiT exhibits efficient convergence, achieving a gFID of 3.47 at 80 epochs.

Title: Multi-Cue Anomaly Detection and Localization under Data Contamination

• Authors: Anindya Sundar Das, Monowar Bhuyan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22913
• Pdf URL: https://arxiv.org/pdf/2601.22913
• Copy Paste: [[2601.22913]] Multi-Cue Anomaly Detection and Localization under Data Contamination(https://arxiv.org/abs/2601.22913)
• Keywords: robust, interpretability, segmentation
• Abstract: Visual anomaly detection in real-world industrial settings faces two major limitations. First, most existing methods are trained on purely normal data or on unlabeled datasets assumed to be predominantly normal, presuming the absence of contamination, an assumption that is rarely satisfied in practice. Second, they assume no access to labeled anomaly samples, limiting the model from learning discriminative characteristics of true anomalies. Therefore, these approaches often struggle to distinguish anomalies from normal instances, resulting in reduced detection and weak localization performance. In real-world applications, where training data are frequently contaminated with anomalies, such methods fail to deliver reliable performance. In this work, we propose a robust anomaly detection framework that integrates limited anomaly supervision into the adaptive deviation learning paradigm. We introduce a composite anomaly score that combines three complementary components: a deviation score capturing statistical irregularity, an entropy-based uncertainty score reflecting predictive inconsistency, and a segmentation-based score highlighting spatial abnormality. This unified scoring mechanism enables accurate detection and supports gradient-based localization, providing intuitive and explainable visual evidence of anomalous regions. Following the few-anomaly paradigm, we incorporate a small set of labeled anomalies during training while simultaneously mitigating the influence of contaminated samples through adaptive instance weighting. Extensive experiments on the MVTec and VisA benchmarks demonstrate that our framework outperforms state-of-the-art baselines and achieves strong detection and localization performance, interpretability, and robustness under various levels of data contamination.

Title: Deep in the Jungle: Towards Automating Chimpanzee Population Estimation

• Authors: Tom Raynes, Otto Brookes, Timm Haucke, Lukas Bösch, Anne-Sophie Crunchant, Hjalmar Kühl, Sara Beery, Majid Mirmehdi, Tilo Burghardt
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22917
• Pdf URL: https://arxiv.org/pdf/2601.22917
• Copy Paste: [[2601.22917]] Deep in the Jungle: Towards Automating Chimpanzee Population Estimation(https://arxiv.org/abs/2601.22917)
• Keywords: transformer
• Abstract: The estimation of abundance and density in unmarked populations of great apes relies on statistical frameworks that require animal-to-camera distance measurements. In practice, acquiring these distances depends on labour-intensive manual interpretation of animal observations across large camera trap video corpora. This study introduces and evaluates an only sparsely explored alternative: the integration of computer vision-based monocular depth estimation (MDE) pipelines directly into ecological camera trap workflows for great ape conservation. Using a real-world dataset of 220 camera trap videos documenting a wild chimpanzee population, we combine two MDE models, Dense Prediction Transformers and Depth Anything, with multiple distance sampling strategies. These components are used to generate detection distance estimates, from which population density and abundance are inferred. Comparative analysis against manually derived ground-truth distances shows that calibrated DPT consistently outperforms Depth Anything. This advantage is observed in both distance estimation accuracy and downstream density and abundance inference. Nevertheless, both models exhibit systematic biases. We show that, given complex forest environments, they tend to overestimate detection distances and consequently underestimate density and abundance relative to conventional manual approaches. We further find that failures in animal detection across distance ranges are a primary factor limiting estimation accuracy. Overall, this work provides a case study that shows MDE-driven camera trap distance sampling is a viable and practical alternative to manual distance estimation. The proposed approach yields population estimates within 22% of those obtained using traditional methods.

Title: Evaluating Large Language Models for Security Bug Report Prediction

• Authors: Farnaz Soltaniani, Shoaib Razzaq, Mohammad Ghafari
• Subjects: cs.CR, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22921
• Pdf URL: https://arxiv.org/pdf/2601.22921
• Copy Paste: [[2601.22921]] Evaluating Large Language Models for Security Bug Report Prediction(https://arxiv.org/abs/2601.22921)
• Keywords: security, large language model
• Abstract: Early detection of security bug reports (SBRs) is critical for timely vulnerability mitigation. We present an evaluation of prompt-based engineering and fine-tuning approaches for predicting SBRs using Large Language Models (LLMs). Our findings reveal a distinct trade-off between the two approaches. Prompted proprietary models demonstrate the highest sensitivity to SBRs, achieving a G-measure of 77% and a recall of 74% on average across all the datasets, albeit at the cost of a higher false-positive rate, resulting in an average precision of only 22%. Fine-tuned models, by contrast, exhibit the opposite behavior, attaining a lower overall G-measure of 51% but substantially higher precision of 75% at the cost of reduced recall of 36%. Though a one-time investment in building fine-tuned models is necessary, the inference on the largest dataset is up to 50 times faster than that of proprietary models. These findings suggest that further investigations to harness the power of LLMs for SBR prediction are necessary.

Title: LLMs Explain't: A Post-Mortem on Semantic Interpretability in Transformer Models

• Authors: Alhassan Abdelhalim, Janick Edinger, Sören Laue, Michaela Regneri
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22928
• Pdf URL: https://arxiv.org/pdf/2601.22928
• Copy Paste: [[2601.22928]] LLMs Explain't: A Post-Mortem on Semantic Interpretability in Transformer Models(https://arxiv.org/abs/2601.22928)
• Keywords: interpretability, explainability, transformer, large language model
• Abstract: Large Language Models (LLMs) are becoming increasingly popular in pervasive computing due to their versatility and strong performance. However, despite their ubiquitous use, the exact mechanisms underlying their outstanding performance remain unclear. Different methods for LLM explainability exist, and many are, as a method, not fully understood themselves. We started with the question of how linguistic abstraction emerges in LLMs, aiming to detect it across different LLM modules (attention heads and input embeddings). For this, we used methods well-established in the literature: (1) probing for token-level relational structures, and (2) feature-mapping using embeddings as carriers of human-interpretable properties. Both attempts failed for different methodological reasons: Attention-based explanations collapsed once we tested the core assumption that later-layer representations still correspond to tokens. Property-inference methods applied to embeddings also failed because their high predictive scores were driven by methodological artifacts and dataset structure rather than meaningful semantic knowledge. These failures matter because both techniques are widely treated as evidence for what LLMs supposedly understand, yet our results show such conclusions are unwarranted. These limitations are particularly relevant in pervasive and distributed computing settings where LLMs are deployed as system components and interpretability methods are relied upon for debugging, compression, and explaining models.

Title: Semantic Leakage from Image Embeddings

• Authors: Yiyi Chen, Qiongkai Xu, Desmond Eliott, Qiongxiu Li, Johannes Bjerva
• Subjects: cs.CV, cs.CL, cs.CR
• Abstract URL: https://arxiv.org/abs/2601.22929
• Pdf URL: https://arxiv.org/pdf/2601.22929
• Copy Paste: [[2601.22929]] Semantic Leakage from Image Embeddings(https://arxiv.org/abs/2601.22929)
• Keywords: privacy
• Abstract: Image embeddings are generally assumed to pose limited privacy risk. We challenge this assumption by formalizing semantic leakage as the ability to recover semantic structures from compressed image embeddings. Surprisingly, we show that semantic leakage does not require exact reconstruction of the original image. Preserving local semantic neighborhoods under embedding alignment is sufficient to expose the intrinsic vulnerability of image embeddings. Crucially, this preserved neighborhood structure allows semantic information to propagate through a sequence of lossy mappings. Based on this conjecture, we propose Semantic Leakage from Image Embeddings (SLImE), a lightweight inference framework that reveals semantic information from standalone compressed image embeddings, incorporating a locally trained semantic retriever with off-the-shelf models, without training task-specific decoders. We thoroughly validate each step of the framework empirically, from aligned embeddings to retrieved tags, symbolic representations, and grammatical and coherent descriptions. We evaluate SLImE across a range of open and closed embedding models, including GEMINI, COHERE, NOMIC, and CLIP, and demonstrate consistent recovery of semantic information across diverse inference tasks. Our results reveal a fundamental vulnerability in image embeddings, whereby the preservation of semantic neighborhoods under alignment enables semantic leakage, highlighting challenges for privacy preservation.1

Title: Protecting Private Code in IDE Autocomplete using Differential Privacy

• Authors: Evgeny Grigorenko, David Stanojević, David Ilić, Egor Bogomolov, Kostadin Cvejoski
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22935
• Pdf URL: https://arxiv.org/pdf/2601.22935
• Copy Paste: [[2601.22935]] Protecting Private Code in IDE Autocomplete using Differential Privacy(https://arxiv.org/abs/2601.22935)
• Keywords: privacy, protect, defense, attack, robust, membership infer, large language model
• Abstract: Modern Integrated Development Environments (IDEs) increasingly leverage Large Language Models (LLMs) to provide advanced features like code autocomplete. While powerful, training these models on user-written code introduces significant privacy risks, making the models themselves a new type of data vulnerability. Malicious actors can exploit this by launching attacks to reconstruct sensitive training data or infer whether a specific code snippet was used for training. This paper investigates the use of Differential Privacy (DP) as a robust defense mechanism for training an LLM for Kotlin code completion. We fine-tune a \texttt{Mellum} model using DP and conduct a comprehensive evaluation of its privacy and utility. Our results demonstrate that DP provides a strong defense against Membership Inference Attacks (MIAs), reducing the attack's success rate close to a random guess (AUC from 0.901 to 0.606). Furthermore, we show that this privacy guarantee comes at a minimal cost to model performance, with the DP-trained model achieving utility scores comparable to its non-private counterpart, even when trained on 100x less data. Our findings suggest that DP is a practical and effective solution for building private and trustworthy AI-powered IDE features.

Title: A Real-Time Privacy-Preserving Behavior Recognition System via Edge-Cloud Collaboration

• Authors: Huan Song, Shuyu Tian, Junyi Hao, Cheng Yuan, Zhenyu Jia, Jiawei Shao, Xuelong Li
• Subjects: cs.CR, cs.AI, eess.IV, eess.SP
• Abstract URL: https://arxiv.org/abs/2601.22938
• Pdf URL: https://arxiv.org/pdf/2601.22938
• Copy Paste: [[2601.22938]] A Real-Time Privacy-Preserving Behavior Recognition System via Edge-Cloud Collaboration(https://arxiv.org/abs/2601.22938)
• Keywords: security, privacy, attack, robust
• Abstract: As intelligent sensing expands into high-privacy environments such as restrooms and changing rooms, the field faces a critical privacy-security paradox. Traditional RGB surveillance raises significant concerns regarding visual recording and storage, while existing privacy-preserving methods-ranging from physical desensitization to traditional cryptographic or obfuscation techniques-often compromise semantic understanding capabilities or fail to guarantee mathematical irreversibility against reconstruction attacks. To address these challenges, this study presents a novel privacy-preserving perception technology based on the AI Flow theoretical framework and an edge-cloud collaborative architecture. The proposed methodology integrates source desensitization with irreversible feature mapping. Leveraging Information Bottleneck theory, the edge device performs millisecond-level processing to transform raw imagery into abstract feature vectors via non-linear mapping and stochastic noise injection. This process constructs a unidirectional information flow that strips identity-sensitive attributes, rendering the reconstruction of original images impossible. Subsequently, the cloud platform utilizes multimodal family models to perform joint inference solely on these abstract vectors to detect abnormal behaviors. This approach fundamentally severs the path to privacy leakage at the architectural level, achieving a breakthrough from video surveillance to de-identified behavior perception and offering a robust solution for risk management in high-sensitivity public spaces.

Title: Environment-Conditioned Tail Reweighting for Total Variation Invariant Risk Minimization

• Authors: Wang Yuanchao, Lai Zhao-Rong, Zhong Tianqi, Li Fengnan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22944
• Pdf URL: https://arxiv.org/pdf/2601.22944
• Copy Paste: [[2601.22944]] Environment-Conditioned Tail Reweighting for Total Variation Invariant Risk Minimization(https://arxiv.org/abs/2601.22944)
• Keywords: robust
• Abstract: Out-of-distribution (OOD) generalization remains challenging when models simultaneously encounter correlation shifts across environments and diversity shifts driven by rare or hard samples. Existing invariant risk minimization (IRM) methods primarily address spurious correlations at the environment level, but often overlook sample-level heterogeneity within environments, which can critically impact OOD performance. In this work, we propose \emph{Environment-Conditioned Tail Reweighting for Total Variation Invariant Risk Minimization} (ECTR), a unified framework that augments TV-based invariant learning with environment-conditioned tail reweighting to jointly address both types of distribution shift. By integrating environment-level invariance with within-environment robustness, the proposed approach makes these two mechanisms complementary under mixed distribution shifts. We further extend the framework to scenarios without explicit environment annotations by inferring latent environments through a minimax formulation. Experiments across regression, tabular, time-series, and image classification benchmarks under mixed distribution shifts demonstrate consistent improvements in both worst-environment and average OOD performance.

Title: From Data Leak to Secret Misses: The Impact of Data Leakage on Secret Detection Models

• Authors: Farnaz Soltaniani, Mohammad Ghafari
• Subjects: cs.CR, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22946
• Pdf URL: https://arxiv.org/pdf/2601.22946
• Copy Paste: [[2601.22946]] From Data Leak to Secret Misses: The Impact of Data Leakage on Secret Detection Models(https://arxiv.org/abs/2601.22946)
• Keywords: security
• Abstract: Machine learning models are increasingly used for software security tasks. These models are commonly trained and evaluated on large Internet-derived datasets, which often contain duplicated or highly similar samples. When such samples are split across training and test sets, data leakage may occur, allowing models to memorize patterns instead of learning to generalize. We investigate duplication in a widely used benchmark dataset of hard coded secrets and show how data leakage can substantially inflate the reported performance of AI-based secret detectors, resulting in a misleading picture of their real-world effectiveness.

Title: Relaxing Positional Alignment in Masked Diffusion Language Models

• Authors: Mengyu Ye, Ryosuke Takahashi, Keito Kudo, Jun Suzuki
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22947
• Pdf URL: https://arxiv.org/pdf/2601.22947
• Copy Paste: [[2601.22947]] Relaxing Positional Alignment in Masked Diffusion Language Models(https://arxiv.org/abs/2601.22947)
• Keywords: robust, diffusion
• Abstract: Masked diffusion language models (MDLMs) have emerged as a promising alternative to dominant autoregressive approaches. Although they achieve competitive performance on several tasks, a substantial gap remains in open-ended text generation. We hypothesize that one cause of this gap is that strict positional prediction makes MDLM decoding highly sensitive to token misalignment, and we show through controlled interventions that a one-position shift can severely disrupt semantics. This observation suggests that enforcing strict positional supervision during training is misaligned with the irreversible denoising dynamics of MDLM decoding. Motivated by this mismatch, we adopt an alignment-flexible supervision strategy during fine-tuning. Specifically, we introduce a special token via the connectionist temporal classification objective. We apply this approach to the widely used MDLM model and conduct experiments on five open-ended text generation benchmarks. Our method consistently outperforms the original model and improves robustness to positional shifts, indicating that relaxing strict positional supervision is an important factor in improving generation quality in MDLMs.

Title: Perplexity Cannot Always Tell Right from Wrong

• Authors: Petar Veličković, Federico Barbero, Christos Perivolaropoulos, Simon Osindero, Razvan Pascanu
• Subjects: cs.LG, cs.AI, cs.CL, stat.ML
• Abstract URL: https://arxiv.org/abs/2601.22950
• Pdf URL: https://arxiv.org/pdf/2601.22950
• Copy Paste: [[2601.22950]] Perplexity Cannot Always Tell Right from Wrong(https://arxiv.org/abs/2601.22950)
• Keywords: transformer
• Abstract: Perplexity -- a function measuring a model's overall level of "surprise" when encountering a particular output -- has gained significant traction in recent years, both as a loss function and as a simple-to-compute metric of model quality. Prior studies have pointed out several limitations of perplexity, often from an empirical manner. Here we leverage recent results on Transformer continuity to show in a rigorous manner how perplexity may be an unsuitable metric for model selection. Specifically, we prove that, if there is any sequence that a compact decoder-only Transformer model predicts accurately and confidently -- a necessary pre-requisite for strong generalisation -- it must imply existence of another sequence with very low perplexity, but not predicted correctly by that same model. Further, by analytically studying iso-perplexity plots, we find that perplexity will not always select for the more accurate model -- rather, any increase in model confidence must be accompanied by a commensurate rise in accuracy for the new model to be selected.

Title: Residual Context Diffusion Language Models

• Authors: Yuezhou Hu, Harman Singh, Monishwaran Maheswaran, Haocheng Xi, Coleman Hooper, Jintao Zhang, Aditya Tomar, Michael W. Mahoney, Sewon Min, Mehrdad Farajtabar, Kurt Keutzer, Amir Gholami, Chenfeng Xu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22954
• Pdf URL: https://arxiv.org/pdf/2601.22954
• Copy Paste: [[2601.22954]] Residual Context Diffusion Language Models(https://arxiv.org/abs/2601.22954)
• Keywords: diffusion, large language model
• Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to purely autoregressive language models because they can decode multiple tokens in parallel. However, state-of-the-art block-wise dLLMs rely on a "remasking" mechanism that decodes only the most confident tokens and discards the rest, effectively wasting computation. We demonstrate that recycling computation from the discarded tokens is beneficial, as these tokens retain contextual information useful for subsequent decoding iterations. In light of this, we propose Residual Context Diffusion (RCD), a module that converts these discarded token representations into contextual residuals and injects them back for the next denoising step. RCD uses a decoupled two-stage training pipeline to bypass the memory bottlenecks associated with backpropagation. We validate our method on both long CoT reasoning (SDAR) and short CoT instruction following (LLaDA) models. We demonstrate that a standard dLLM can be efficiently converted to the RCD paradigm with merely ~1 billion tokens. RCD consistently improves frontier dLLMs by 5-10 points in accuracy with minimal extra computation overhead across a wide range of benchmarks. Notably, on the most challenging AIME tasks, RCD nearly doubles baseline accuracy and attains up to 4-5x fewer denoising steps at equivalent accuracy levels.

Title: Triage: Hierarchical Visual Budgeting for Efficient Video Reasoning in Vision-Language Models

• Authors: Anmin Wang, Nan Zhang, Wei Tao, Xiaoyang Qu, Guokuan Li, Jiguang Wan, Jianzong Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22959
• Pdf URL: https://arxiv.org/pdf/2601.22959
• Copy Paste: [[2601.22959]] Triage: Hierarchical Visual Budgeting for Efficient Video Reasoning in Vision-Language Models(https://arxiv.org/abs/2601.22959)
• Keywords: secure
• Abstract: Vision-Language Models (VLMs) face significant computational challenges in video processing due to massive data redundancy, which creates prohibitively long token sequences. To address this, we introduce Triage, a training-free, plug-and-play framework that reframes video reasoning as a resource allocation problem via hierarchical visual budgeting. Its first stage, Frame-Level Budgeting, identifies keyframes by evaluating their visual dynamics and relevance, generating a strategic prior based on their importance scores. Guided by this prior, the second stage, Token-Level Budgeting, allocates tokens in two phases: it first secures high-relevance Core Tokens, followed by diverse Context Tokens selected with an efficient batched Maximal Marginal Relevance (MMR) algorithm. Extensive experiments demonstrate that Triage improves inference speed and reduces memory footprint, while maintaining or surpassing the performance of baselines and other methods on various video reasoning benchmarks.

Title: Improving Supervised Machine Learning Performance in Optical Quality Control via Generative AI for Dataset Expansion

• Authors: Dennis Sprute, Hanna Senke, Holger Flatt
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22961
• Pdf URL: https://arxiv.org/pdf/2601.22961
• Copy Paste: [[2601.22961]] Improving Supervised Machine Learning Performance in Optical Quality Control via Generative AI for Dataset Expansion(https://arxiv.org/abs/2601.22961)
• Keywords: diffusion, generative, segmentation
• Abstract: Supervised machine learning algorithms play a crucial role in optical quality control within industrial production. These approaches require representative datasets for effective model training. However, while non-defective components are frequent, defective parts are rare in production, resulting in highly imbalanced datasets that adversely impact model performance. Existing strategies to address this challenge, such as specialized loss functions or traditional data augmentation techniques, have limitations, including the need for careful hyperparameter tuning or the alteration of only simple image features. Therefore, this work explores the potential of generative artificial intelligence (GenAI) as an alternative method for expanding limited datasets and enhancing supervised machine learning performance. Specifically, we investigate Stable Diffusion and CycleGAN as image generation models, focusing on the segmentation of combine harvester components in thermal images for subsequent defect detection. Our results demonstrate that dataset expansion using Stable Diffusion yields the most significant improvement, enhancing segmentation performance by 4.6 %, resulting in a Mean Intersection over Union (Mean IoU) of 84.6 %.

Title: A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training

• Authors: Zihan Qiu, Zeyu Huang, Kaiyue Wen, Peng Jin, Bo Zheng, Yuxin Zhou, Haofeng Huang, Zekun Wang, Xiao Li, Huaqing Zhang, Yang Xu, Haoran Lian, Siqi Zhang, Rui Men, Jianwei Zhang, Ivan Titov, Dayiheng Liu, Jingren Zhou, Junyang Lin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22966
• Pdf URL: https://arxiv.org/pdf/2601.22966
• Copy Paste: [[2601.22966]] A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training(https://arxiv.org/abs/2601.22966)
• Keywords: robust, transformer, large language model
• Abstract: We investigate the functional role of emergent outliers in large language models, specifically attention sinks (a few tokens that consistently receive large attention logits) and residual sinks (a few fixed dimensions with persistently large activations across most tokens). We hypothesize that these outliers, in conjunction with the corresponding normalizations (\textit{e.g.}, softmax attention and RMSNorm), effectively rescale other non-outlier components. We term this phenomenon \textit{outlier-driven rescaling} and validate this hypothesis across different model architectures and training token counts. This view unifies the origin and mitigation of both sink types. Our main conclusions and observations include: (1) Outliers function jointly with normalization: removing normalization eliminates the corresponding outliers but degrades training stability and performance; directly clipping outliers while retaining normalization leads to degradation, indicating that outlier-driven rescaling contributes to training stability. (2) Outliers serve more as rescale factors rather than contributors, as the final contributions of attention and residual sinks are significantly smaller than those of non-outliers. (3) Outliers can be absorbed into learnable parameters or mitigated via explicit gated rescaling, leading to improved training performance (average gain of 2 points) and enhanced quantization robustness (1.2 points degradation under W4A4 quantization).

Title: Improved Algorithms for Nash Welfare in Linear Bandits

• Authors: Dhruv Sarkar, Nishant Pandey, Sayak Ray Chowdhury
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22969
• Pdf URL: https://arxiv.org/pdf/2601.22969
• Copy Paste: [[2601.22969]] Improved Algorithms for Nash Welfare in Linear Bandits(https://arxiv.org/abs/2601.22969)
• Keywords: fair
• Abstract: Nash regret has recently emerged as a principled fairness-aware performance metric for stochastic multi-armed bandits, motivated by the Nash Social Welfare objective. Although this notion has been extended to linear bandits, existing results suffer from suboptimality in ambient dimension $d$, stemming from proof techniques that rely on restrictive concentration inequalities. In this work, we resolve this open problem by introducing new analytical tools that yield an order-optimal Nash regret bound in linear bandits. Beyond Nash regret, we initiate the study of $p$-means regret in linear bandits, a unifying framework that interpolates between fairness and utility objectives and strictly generalizes Nash regret. We propose a generic algorithmic framework, FairLinBandit, that works as a meta-algorithm on top of any linear bandit strategy. We instantiate this framework using two bandit algorithms: Phased Elimination and Upper Confidence Bound, and prove that both achieve sublinear $p$-means regret for the entire range of $p$. Extensive experiments on linear bandit instances generated from real-world datasets demonstrate that our methods consistently outperform the existing state-of-the-art baseline.

Title: Stabilizing the Q-Gradient Field for Policy Smoothness in Actor-Critic

• Authors: Jeong Woon Lee, Kyoleen Kwak, Daeho Kim, Hyoseok Hwang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22970
• Pdf URL: https://arxiv.org/pdf/2601.22970
• Copy Paste: [[2601.22970]] Stabilizing the Q-Gradient Field for Policy Smoothness in Actor-Critic(https://arxiv.org/abs/2601.22970)
• Keywords: robust
• Abstract: Policies learned via continuous actor-critic methods often exhibit erratic, high-frequency oscillations, making them unsuitable for physical deployment. Current approaches attempt to enforce smoothness by directly regularizing the policy's output. We argue that this approach treats the symptom rather than the cause. In this work, we theoretically establish that policy non-smoothness is fundamentally governed by the differential geometry of the critic. By applying implicit differentiation to the actor-critic objective, we prove that the sensitivity of the optimal policy is bounded by the ratio of the Q-function's mixed-partial derivative (noise sensitivity) to its action-space curvature (signal distinctness). To empirically validate this theoretical insight, we introduce PAVE (Policy-Aware Value-field Equalization), a critic-centric regularization framework that treats the critic as a scalar field and stabilizes its induced action-gradient field. PAVE rectifies the learning signal by minimizing the Q-gradient volatility while preserving local curvature. Experimental results demonstrate that PAVE achieves smoothness and robustness comparable to policy-side smoothness regularization methods, while maintaining competitive task performance, without modifying the actor.

Title: SpecIBT: Formally Verified Protection Against Speculative Control-Flow Hijacking

• Authors: Jonathan Baumann, Yonghyun Kim, Yan Farba, Catalin Hritcu, Julay Leatherman-Brooks
• Subjects: cs.CR, cs.PL
• Abstract URL: https://arxiv.org/abs/2601.22978
• Pdf URL: https://arxiv.org/pdf/2601.22978
• Copy Paste: [[2601.22978]] SpecIBT: Formally Verified Protection Against Speculative Control-Flow Hijacking(https://arxiv.org/abs/2601.22978)
• Keywords: security, protect, defense
• Abstract: This paper introduces SpecIBT, a formally verified defense against Spectre BTB, RSB, and PHT that combines CET-style hardware-assisted control-flow integrity with compiler-inserted speculative load hardening (SLH). SpecIBT is based on the novel observation that in the presence of CET-style protection, we can precisely detect BTB misspeculation for indirect calls and set the SLH misspeculation flag. We formalize SpecIBT as a transformation in Rocq and provide a machine-checked proof that it achieves relative security: any transformed program running with speculation leaks no more than what the source program leaks without speculation. This strong security guarantee applies to arbitrary programs, even those not following the cryptographic constant-time programming discipline.

Title: Learnable Permutation for Structured Sparsity on Transformer Models

• Authors: Zekai Li, Ji Liu, Guanchen Li, Yixing Xu, Ziqiong Liu, Xuanwu Yin, Dong Li, Emad Barsoum
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22980
• Pdf URL: https://arxiv.org/pdf/2601.22980
• Copy Paste: [[2601.22980]] Learnable Permutation for Structured Sparsity on Transformer Models(https://arxiv.org/abs/2601.22980)
• Keywords: transformer, large language model
• Abstract: Structured sparsity has emerged as a popular model pruning technique, widely adopted in various architectures, including CNNs, Transformer models, and especially large language models (LLMs) in recent years. A promising direction to further improve post-pruning performance is weight permutation, which reorders model weights into patterns more amenable to pruning. However, the exponential growth of the permutation search space with the scale of Transformer architectures forces most methods to rely on greedy or heuristic algorithms, limiting the effectiveness of reordering. In this work, we propose a novel end-to-end learnable permutation framework. Our method introduces a learnable permutation cost matrix to quantify the cost of swapping any two input channels of a given weight matrix, a differentiable bipartite matching solver to obtain the optimal binary permutation matrix given a cost matrix, and a sparsity optimization loss function to directly optimize the permutation operator. We extensively validate our approach on vision and language Transformers, demonstrating that our method achieves state-of-the-art permutation results for structured sparsity.

Title: About an Automating Annotation Method for Robot Markers

• Authors: Wataru Uemura, Takeru Nagashima
• Subjects: cs.CV, cs.AI, cs.RO
• Abstract URL: https://arxiv.org/abs/2601.22982
• Pdf URL: https://arxiv.org/pdf/2601.22982
• Copy Paste: [[2601.22982]] About an Automating Annotation Method for Robot Markers(https://arxiv.org/abs/2601.22982)
• Keywords: robust
• Abstract: Factory automation has become increasingly important due to labor shortages, leading to the introduction of autonomous mobile robots for tasks such as material transportation. Markers are commonly used for robot self-localization and object identification. In the RoboCup Logistics League (RCLL), ArUco markers are employed both for robot localization and for identifying processing modules. Conventional recognition relies on OpenCV-based image processing, which detects black-and-white marker patterns. However, these methods often fail under noise, motion blur, defocus, or varying illumination conditions. Deep-learning-based recognition offers improved robustness under such conditions, but requires large amounts of annotated data. Annotation must typically be done manually, as the type and position of objects cannot be detected automatically, making dataset preparation a major bottleneck. In contrast, ArUco markers include built-in recognition modules that provide both ID and positional information, enabling automatic annotation. This paper proposes an automated annotation method for training deep-learning models on ArUco marker images. By leveraging marker detection results obtained from the ArUco module, the proposed approach eliminates the need for manual labeling. A YOLO-based model is trained using the automatically annotated dataset, and its performance is evaluated under various conditions. Experimental results demonstrate that the proposed method improves recognition performance compared with conventional image-processing techniques, particularly for images affected by blur or defocus. Automatic annotation also reduces human effort and ensures consistent labeling quality. Future work will investigate the relationship between confidence thresholds and recognition performance.

Title: PIDSMaker: Building and Evaluating Provenance-based Intrusion Detection Systems

• Authors: Tristan Bilot, Baoxiang Jiang, Thomas Pasquier
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22983
• Pdf URL: https://arxiv.org/pdf/2601.22983
• Copy Paste: [[2601.22983]] PIDSMaker: Building and Evaluating Provenance-based Intrusion Detection Systems(https://arxiv.org/abs/2601.22983)
• Keywords: fair
• Abstract: Recent provenance-based intrusion detection systems (PIDSs) have demonstrated strong potential for detecting advanced persistent threats (APTs) by applying machine learning to system provenance graphs. However, evaluating and comparing PIDSs remains difficult: prior work uses inconsistent preprocessing pipelines, non-standard dataset splits, and incompatible ground-truth labeling and metrics. These discrepancies undermine reproducibility, impede fair comparison, and impose substantial re-implementation overhead on researchers. We present PIDSMaker, an open-source framework for developing and evaluating PIDSs under consistent protocols. PIDSMaker consolidates eight state-of-the-art systems into a modular, extensible architecture with standardized preprocessing and ground-truth labels, enabling consistent experiments and apples-to-apples comparisons. A YAML-based configuration interface supports rapid prototyping by composing components across systems without code changes. PIDSMaker also includes utilities for ablation studies, hyperparameter tuning, multi-run instability measurement, and visualization, addressing methodological gaps identified in prior work. We demonstrate PIDSMaker through concrete use cases and release it with preprocessed datasets and labels to support shared evaluation for the PIDS community.

Title: dgMARK: Decoding-Guided Watermarking for Diffusion Language Models

• Authors: Pyo Min Hong, Albert No
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22985
• Pdf URL: https://arxiv.org/pdf/2601.22985
• Copy Paste: [[2601.22985]] dgMARK: Decoding-Guided Watermarking for Diffusion Language Models(https://arxiv.org/abs/2601.22985)
• Keywords: robust, watermark, diffusion
• Abstract: We propose dgMARK, a decoding-guided watermarking method for discrete diffusion language models (dLLMs). Unlike autoregressive models, dLLMs can generate tokens in arbitrary order. While an ideal conditional predictor would be invariant to this order, practical dLLMs exhibit strong sensitivity to the unmasking order, creating a new channel for watermarking. dgMARK steers the unmasking order toward positions whose high-reward candidate tokens satisfy a simple parity constraint induced by a binary hash, without explicitly reweighting the model's learned probabilities. The method is plug-and-play with common decoding strategies (e.g., confidence, entropy, and margin-based ordering) and can be strengthened with a one-step lookahead variant. Watermarks are detected via elevated parity-matching statistics, and a sliding-window detector ensures robustness under post-editing operations including insertion, deletion, substitution, and paraphrasing.

Title: Mano: Restriking Manifold Optimization for LLM Training

• Authors: Yufei Gu, Zeke Xie
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.23000
• Pdf URL: https://arxiv.org/pdf/2601.23000
• Copy Paste: [[2601.23000]] Mano: Restriking Manifold Optimization for LLM Training(https://arxiv.org/abs/2601.23000)
• Keywords: large language model
• Abstract: While large language models (LLMs) have emerged as a significant advancement in artificial intelligence, the hardware and computational costs for training LLMs are also significantly burdensome. Among the state-of-the-art optimizers, AdamW relies on diagonal curvature estimates and ignores structural properties, while Muon applies global spectral normalization at the expense of losing curvature information. In this study, we restriked manifold optimization methods for training LLMs, which may address both optimizers' limitations, while conventional manifold optimization methods have been largely overlooked due to the poor performance in large-scale model optimization. By innovatively projecting the momentum onto the tangent space of model parameters and constraining it on a rotational Oblique manifold, we propose a novel, powerful, and efficient optimizer **Mano** that is the first to bridge the performance gap between manifold optimization and modern optimizers. Extensive experiments on the LLaMA and Qwen3 models demonstrate that Mano consistently and significantly outperforms AdamW and Muon even with less memory consumption and computational complexity, respectively, suggesting an expanded Pareto frontier in terms of space and time efficiency.

Title: Bias Beyond Borders: Political Ideology Evaluation and Steering in Multilingual LLMs

• Authors: Afrozah Nadeem, Agrima, Mehwish Nasim, Usman Naseem
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.23001
• Pdf URL: https://arxiv.org/pdf/2601.23001
• Copy Paste: [[2601.23001]] Bias Beyond Borders: Political Ideology Evaluation and Steering in Multilingual LLMs(https://arxiv.org/abs/2601.23001)
• Keywords: fair, large language model
• Abstract: Large Language Models (LLMs) increasingly shape global discourse, making fairness and ideological neutrality essential for responsible AI deployment. Despite growing attention to political bias in LLMs, prior work largely focuses on high-resource, Western languages or narrow multilingual settings, leaving cross-lingual consistency and safe post-hoc mitigation underexplored. To address this gap, we present a large-scale multilingual evaluation of political bias spanning 50 countries and 33 languages. We introduce a complementary post-hoc mitigation framework, Cross-Lingual Alignment Steering (CLAS), designed to augment existing steering methods by aligning ideological representations across languages and dynamically regulating intervention strength. This method aligns latent ideological representations induced by political prompts into a shared ideological subspace, ensuring cross lingual consistency, with the adaptive mechanism prevents over correction and preserves coherence. Experiments demonstrate substantial bias reduction along both economic and social axes with minimal degradation in response quality. The proposed framework establishes a scalable and interpretable paradigm for fairness-aware multilingual LLM governance, balancing ideological neutrality with linguistic and cultural diversity.

Title: InstructDiff: Domain-Adaptive Data Selection via Differential Entropy for Efficient LLM Fine-Tuning

• Authors: Junyou Su, He Zhu, Xiao Luo, Liyu Zhang, Hong-Yu Zhou, Yun Chen, Peng Li, Yang Liu, Guanhua Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.23006
• Pdf URL: https://arxiv.org/pdf/2601.23006
• Copy Paste: [[2601.23006]] InstructDiff: Domain-Adaptive Data Selection via Differential Entropy for Efficient LLM Fine-Tuning(https://arxiv.org/abs/2601.23006)
• Keywords: large language model
• Abstract: Supervised fine-tuning (SFT) is fundamental to adapting large language models, yet training on complete datasets incurs prohibitive costs with diminishing returns. Existing data selection methods suffer from severe domain specificity: techniques optimized for general instruction-following fail on reasoning tasks, and vice versa. We observe that measuring entropy differences between base models and minimally instruction-tuned calibrated models reveals a pattern -- samples with the lowest differential entropy consistently yield optimal performance across domains, yet this principle manifests domain-adaptively: reasoning tasks favor entropy increase (cognitive expansion), while general tasks favor entropy decrease (cognitive compression). We introduce InstructDiff, a unified framework that operationalizes differential entropy as a domain-adaptive selection criterion through warmup calibration, bi-directional NLL filtering, and entropy-based ranking. Extensive experiments show that InstructDiff achieves 17\% relative improvement over full data training on mathematical reasoning and 52\% for general instruction-following, outperforming prior baselines while using only 10\% of the data.

Title: Automatic Constraint Policy Optimization based on Continuous Constraint Interpolation Framework for Offline Reinforcement Learning

• Authors: Xinchen Han, Qiuyang Fang, Hossam Afifi, Michel Marot
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.23010
• Pdf URL: https://arxiv.org/pdf/2601.23010
• Copy Paste: [[2601.23010]] Automatic Constraint Policy Optimization based on Continuous Constraint Interpolation Framework for Offline Reinforcement Learning(https://arxiv.org/abs/2601.23010)
• Keywords: robust
• Abstract: Offline Reinforcement Learning (RL) relies on policy constraints to mitigate extrapolation error, where both the constraint form and constraint strength critically shape performance. However, most existing methods commit to a single constraint family: weighted behavior cloning, density regularization, or support constraints, without a unified principle that explains their connections or trade-offs. In this work, we propose Continuous Constraint Interpolation (CCI), a unified optimization framework in which these three constraint families arise as special cases along a common constraint spectrum. The CCI framework introduces a single interpolation parameter that enables smooth transitions and principled combinations across constraint types. Building on CCI, we develop Automatic Constraint Policy Optimization (ACPO), a practical primal--dual algorithm that adapts the interpolation parameter via a Lagrangian dual update. Moreover, we establish a maximum-entropy performance difference lemma and derive performance lower bounds for both the closed-form optimal policy and its parametric projection. Experiments on D4RL and NeoRL2 demonstrate robust gains across diverse domains, achieving state-of-the-art performance overall.

Title: Leveraging Convolutional Sparse Autoencoders for Robust Movement Classification from Low-Density sEMG

• Authors: Blagoj Hristov, Zoran Hadzi-Velkov, Katerina Hadzi-Velkova Saneva, Gorjan Nadzinski, Vesna Ojleska Latkoska
• Subjects: cs.LG, cs.AI, eess.SP
• Abstract URL: https://arxiv.org/abs/2601.23011
• Pdf URL: https://arxiv.org/pdf/2601.23011
• Copy Paste: [[2601.23011]] Leveraging Convolutional Sparse Autoencoders for Robust Movement Classification from Low-Density sEMG(https://arxiv.org/abs/2601.23011)
• Keywords: robust
• Abstract: Reliable control of myoelectric prostheses is often hindered by high inter-subject variability and the clinical impracticality of high-density sensor arrays. This study proposes a deep learning framework for accurate gesture recognition using only two surface electromyography (sEMG) channels. The method employs a Convolutional Sparse Autoencoder (CSAE) to extract temporal feature representations directly from raw signals, eliminating the need for heuristic feature engineering. On a 6-class gesture set, our model achieved a multi-subject F1-score of 94.3% $\pm$ 0.3%. To address subject-specific differences, we present a few-shot transfer learning protocol that improved performance on unseen subjects from a baseline of 35.1% $\pm$ 3.1% to 92.3% $\pm$ 0.9% with minimal calibration data. Furthermore, the system supports functional extensibility through an incremental learning strategy, allowing for expansion to a 10-class set with a 90.0% $\pm$ 0.2% F1-score without full model retraining. By combining high precision with minimal computational and sensor overhead, this framework provides a scalable and efficient approach for the next generation of affordable and adaptive prosthetic systems.

Title: DimABSA: Building Multilingual and Multidomain Datasets for Dimensional Aspect-Based Sentiment Analysis

• Authors: Lung-Hao Lee, Liang-Chih Yu, Natalia Loukashevich, Ilseyar Alimova, Alexander Panchenko, Tzu-Mi Lin, Zhe-Yu Xu, Jian-Yu Zhou, Guangmin Zheng, Jin Wang, Sharanya Awasthi, Jonas Becker, Jan Philip Wahle, Terry Ruas, Shamsuddeen Hassan Muhammad, Saif M. Mohammed
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.23022
• Pdf URL: https://arxiv.org/pdf/2601.23022
• Copy Paste: [[2601.23022]] DimABSA: Building Multilingual and Multidomain Datasets for Dimensional Aspect-Based Sentiment Analysis(https://arxiv.org/abs/2601.23022)
• Keywords: large language model
• Abstract: Aspect-Based Sentiment Analysis (ABSA) focuses on extracting sentiment at a fine-grained aspect level and has been widely applied across real-world domains. However, existing ABSA research relies on coarse-grained categorical labels (e.g., positive, negative), which limits its ability to capture nuanced affective states. To address this limitation, we adopt a dimensional approach that represents sentiment with continuous valence-arousal (VA) scores, enabling fine-grained analysis at both the aspect and sentiment levels. To this end, we introduce DimABSA, the first multilingual, dimensional ABSA resource annotated with both traditional ABSA elements (aspect terms, aspect categories, and opinion terms) and newly introduced VA scores. This resource contains 76,958 aspect instances across 42,590 sentences, spanning six languages and four domains. We further introduce three subtasks that combine VA scores with different ABSA elements, providing a bridge from traditional ABSA to dimensional ABSA. Given that these subtasks involve both categorical and continuous outputs, we propose a new unified metric, continuous F1 (cF1), which incorporates VA prediction error into standard F1. We provide a comprehensive benchmark using both prompted and fine-tuned large language models across all subtasks. Our results show that DimABSA is a challenging benchmark and provides a foundation for advancing multilingual dimensional ABSA.

Title: Causal Characterization of Measurement and Mechanistic Anomalies

• Authors: Hendrik Suhr, David Kaltenpoth, Jilles Vreeken
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.23026
• Pdf URL: https://arxiv.org/pdf/2601.23026
• Copy Paste: [[2601.23026]] Causal Characterization of Measurement and Mechanistic Anomalies(https://arxiv.org/abs/2601.23026)
• Keywords: robust
• Abstract: Root cause analysis of anomalies aims to identify those features that cause the deviation from the normal process. Existing methods ignore, however, that anomalies can arise through two fundamentally different processes: measurement errors, where data was generated normally but one or more values were recorded incorrectly, and mechanism shifts, where the causal process generating the data changed. While measurement errors can often be safely corrected, mechanistic anomalies require careful consideration. We define a causal model that explicitly captures both types by treating outliers as latent interventions on latent ("true") and observed ("measured") variables. We show that they are identifiable, and propose a maximum likelihood estimation approach to put this to practice. Experiments show that our method matches state-of-the-art performance in root cause localization, while it additionally enables accurate classification of anomaly types, and remains robust even when the causal DAG is unknown.

Title: One-shot Optimized Steering Vector for Hallucination Mitigation for VLMs

• Authors: Youxu Shi, Suorong Yang, Dong Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.23041
• Pdf URL: https://arxiv.org/pdf/2601.23041
• Copy Paste: [[2601.23041]] One-shot Optimized Steering Vector for Hallucination Mitigation for VLMs(https://arxiv.org/abs/2601.23041)
• Keywords: generative
• Abstract: Vision Language Models (VLMs) achieve strong performance on multimodal tasks but still suffer from hallucination and safety-related failures that persist even at scale. Steering offers a lightweight technique to improve model performance. However, steering, whether input-dependent or input-independent, achieves a meaningful trade-off between efficiency and effectiveness. In this work, we observe that steering vectors can generalize across inputs when tasks share aligned semantic intent. Based on this insight, we propose \textbf{OSGA} (\textbf{O}ne-shot \textbf{S}teering with \textbf{G}enerative \textbf{A}nchor), an input-independent framework that improves model performance with a single optimization instance. OSGA first selects an informative sample via a variance-based data selection strategy and learns a single steering vector with a contrastive objective with generative anchor regularization. The resulting vector can be universally applied at a certain layer during inference time without modifying model parameters. Experiments across multiple benchmarks show that a single OSGA-optimized steering vector consistently improves hallucination mitigation and safety enhancement with negligible overhead, highlighting one-shot steering as a practical and scalable solution for reliable VLMs.

Title: Adaptive Edge Learning for Density-Aware Graph Generation

• Authors: Seyedeh Ava Razi Razavi, James Sargant, Sheridan Houghten, Renata Dividino
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.23052
• Pdf URL: https://arxiv.org/pdf/2601.23052
• Copy Paste: [[2601.23052]] Adaptive Edge Learning for Density-Aware Graph Generation(https://arxiv.org/abs/2601.23052)
• Keywords: generative
• Abstract: Generating realistic graph-structured data is challenging due to discrete structures, variable sizes, and class-specific connectivity patterns that resist conventional generative modelling. While recent graph generation methods employ generative adversarial network (GAN) frameworks to handle permutation invariance and irregular topologies, they typically rely on random edge sampling with fixed probabilities, limiting their capacity to capture complex structural dependencies between nodes. We propose a density-aware conditional graph generation framework using Wasserstein GANs (WGAN) that replaces random sampling with a learnable distance-based edge predictor. Our approach embeds nodes into a latent space where proximity correlates with edge likelihood, enabling the generator to learn meaningful connectivity patterns. A differentiable edge predictor determines pairwise relationships directly from node embeddings, while a density-aware selection mechanism adaptively controls edge density to match class-specific sparsity distributions observed in real graphs. We train the model using a WGAN with gradient penalty, employing a GCN-based critic to ensure generated graphs exhibit realistic topology and align with target class distributions. Experiments on benchmark datasets demonstrate that our method produces graphs with superior structural coherence and class-consistent connectivity compared to existing baselines. The learned edge predictor captures complex relational patterns beyond simple heuristics, generating graphs whose density and topology closely match real structural distributions. Our results show improved training stability and controllable synthesis, making the framework effective for realistic graph generation and data augmentation. Source code is publicly available at this https URL.

Title: From Absolute to Relative: Rethinking Reward Shaping in Group-Based Reinforcement Learning

• Authors: Wenzhe Niu, Wei He, Zongxia Xie, Jinpeng Ou, Huichuan Fan, Yuchen Ge, Yanru Sun, Ziyin Wang, Yizhao Sun, Chengshun Shi, Jiuchong Gao, Jinghua Hao, Renqing He
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.23058
• Pdf URL: https://arxiv.org/pdf/2601.23058
• Copy Paste: [[2601.23058]] From Absolute to Relative: Rethinking Reward Shaping in Group-Based Reinforcement Learning(https://arxiv.org/abs/2601.23058)
• Keywords: robust, large language model
• Abstract: Reinforcement learning has become a cornerstone for enhancing the reasoning capabilities of Large Language Models, where group-based approaches such as GRPO have emerged as efficient paradigms that optimize policies by leveraging intra-group performance differences. However, these methods typically rely on absolute numerical rewards, introducing intrinsic limitations. In verifiable tasks, identical group evaluations often result in sparse supervision, while in open-ended scenarios, the score range instability of reward models undermines advantage estimation based on group means. To address these limitations, we propose Reinforcement Learning with Relative Rewards (RLRR), a framework that shifts reward shaping from absolute scoring to relative ranking. Complementing this framework, we introduce the Ranking Reward Model, a listwise preference model tailored for group-based optimization to directly generate relative rankings. By transforming raw evaluations into robust relative signals, RLRR effectively mitigates signal sparsity and reward instability. Experimental results demonstrate that RLRR yields consistent performance improvements over standard group-based baselines across reasoning benchmarks and open-ended generation tasks.

Title: HierLoc: Hyperbolic Entity Embeddings for Hierarchical Visual Geolocation

• Authors: Hari Krishna Gadi, Daniel Matos, Hongyi Luo, Lu Liu, Yongliang Wang, Yanfeng Zhang, Liqiu Meng
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.23064
• Pdf URL: https://arxiv.org/pdf/2601.23064
• Copy Paste: [[2601.23064]] HierLoc: Hyperbolic Entity Embeddings for Hierarchical Visual Geolocation(https://arxiv.org/abs/2601.23064)
• Keywords: generative
• Abstract: Visual geolocalization, the task of predicting where an image was taken, remains challenging due to global scale, visual ambiguity, and the inherently hierarchical structure of geography. Existing paradigms rely on either large-scale retrieval, which requires storing a large number of image embeddings, grid-based classifiers that ignore geographic continuity, or generative models that diffuse over space but struggle with fine detail. We introduce an entity-centric formulation of geolocation that replaces image-to-image retrieval with a compact hierarchy of geographic entities embedded in Hyperbolic space. Images are aligned directly to country, region, subregion, and city entities through Geo-Weighted Hyperbolic contrastive learning by directly incorporating haversine distance into the contrastive objective. This hierarchical design enables interpretable predictions and efficient inference with 240k entity embeddings instead of over 5 million image embeddings on the OSV5M benchmark, on which our method establishes a new state-of-the-art performance. Compared to the current methods in the literature, it reduces mean geodesic error by 19.5\%, while improving the fine-grained subregion accuracy by 43%. These results demonstrate that geometry-aware hierarchical embeddings provide a scalable and conceptually new alternative for global image geolocation.

Title: ExplainerPFN: Towards tabular foundation models for model-free zero-shot feature importance estimations

• Authors: Joao Fonseca, Julia Stoyanovich
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.23068
• Pdf URL: https://arxiv.org/pdf/2601.23068
• Copy Paste: [[2601.23068]] ExplainerPFN: Towards tabular foundation models for model-free zero-shot feature importance estimations(https://arxiv.org/abs/2601.23068)
• Keywords: interpretability
• Abstract: Computing the importance of features in supervised classification tasks is critical for model interpretability. Shapley values are a widely used approach for explaining model predictions, but require direct access to the underlying model, an assumption frequently violated in real-world deployments. Further, even when model access is possible, their exact computation may be prohibitively expensive. We investigate whether meaningful Shapley value estimations can be obtained in a zero-shot setting, using only the input data distribution and no evaluations of the target model. To this end, we introduce ExplainerPFN, a tabular foundation model built on TabPFN that is pretrained on synthetic datasets generated from random structural causal models and supervised using exact or near-exact Shapley values. Once trained, ExplainerPFN predicts feature attributions for unseen tabular datasets without model access, gradients, or example explanations. Our contributions are fourfold: (1) we show that few-shot learning-based explanations can achieve high fidelity to SHAP values with as few as two reference observations; (2) we propose ExplainerPFN, the first zero-shot method for estimating Shapley values without access to the underlying model or reference explanations; (3) we provide an open-source implementation of ExplainerPFN, including the full training pipeline and synthetic data generator; and (4) through extensive experiments on real and synthetic datasets, we show that ExplainerPFN achieves performance competitive with few-shot surrogate explainers that rely on 2-10 SHAP examples.

Title: SplineFlow: Flow Matching for Dynamical Systems with B-Spline Interpolants

• Authors: Santanu Subhash Rathod, Pietro Liò, Xiao Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.23072
• Pdf URL: https://arxiv.org/pdf/2601.23072
• Copy Paste: [[2601.23072]] SplineFlow: Flow Matching for Dynamical Systems with B-Spline Interpolants(https://arxiv.org/abs/2601.23072)
• Keywords: generative
• Abstract: Flow matching is a scalable generative framework for characterizing continuous normalizing flows with wide-range applications. However, current state-of-the-art methods are not well-suited for modeling dynamical systems, as they construct conditional paths using linear interpolants that may not capture the underlying state evolution, especially when learning higher-order dynamics from irregular sampled observations. Constructing unified paths that satisfy multi-marginal constraints across observations is challenging, since naïve higher-order polynomials tend to be unstable and oscillatory. We introduce SplineFlow, a theoretically grounded flow matching algorithm that jointly models conditional paths across observations via B-spline interpolation. Specifically, SplineFlow exploits the smoothness and stability of B-spline bases to learn the complex underlying dynamics in a structured manner while ensuring the multi-marginal requirements are met. Comprehensive experiments across various deterministic and stochastic dynamical systems of varying complexity, as well as on cellular trajectory inference tasks, demonstrate the strong improvement of SplineFlow over existing baselines. Our code is available at: this https URL.

Title: Character as a Latent Variable in Large Language Models: A Mechanistic Account of Emergent Misalignment and Conditional Safety Failures

• Authors: Yanghao Su, Wenbo Zhou, Tianwei Zhang, Qiu Han, Weiming Zhang, Nenghai Yu, Jie Zhang
• Subjects: cs.CL, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2601.23081
• Pdf URL: https://arxiv.org/pdf/2601.23081
• Copy Paste: [[2601.23081]] Character as a Latent Variable in Large Language Models: A Mechanistic Account of Emergent Misalignment and Conditional Safety Failures(https://arxiv.org/abs/2601.23081)
• Keywords: defense, robust, large language model
• Abstract: Emergent Misalignment refers to a failure mode in which fine-tuning large language models (LLMs) on narrowly scoped data induces broadly misaligned behavior. Prior explanations mainly attribute this phenomenon to the generalization of erroneous or unsafe content. In this work, we show that this view is incomplete. Across multiple domains and model families, we find that fine-tuning models on data exhibiting specific character-level dispositions induces substantially stronger and more transferable misalignment than incorrect-advice fine-tuning, while largely preserving general capabilities. This indicates that emergent misalignment arises from stable shifts in model behavior rather than from capability degradation or corrupted knowledge. We further show that such behavioral dispositions can be conditionally activated by both training-time triggers and inference-time persona-aligned prompts, revealing shared structure across emergent misalignment, backdoor activation, and jailbreak susceptibility. Overall, our results identify character formation as a central and underexplored alignment risk, suggesting that robust alignment must address behavioral dispositions rather than isolated errors or prompt-level defenses.

Title: From Similarity to Vulnerability: Key Collision Attack on LLM Semantic Caching

• Authors: Zhixiang Zhang, Zesen Liu, Yuchong Xie, Quanfeng Huang, Dongdong She
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.23088
• Pdf URL: https://arxiv.org/pdf/2601.23088
• Copy Paste: [[2601.23088]] From Similarity to Vulnerability: Key Collision Attack on LLM Semantic Caching(https://arxiv.org/abs/2601.23088)
• Keywords: security, privacy, attack
• Abstract: Semantic caching has emerged as a pivotal technique for scaling LLM applications, widely adopted by major providers including AWS and Microsoft. By utilizing semantic embedding vectors as cache keys, this mechanism effectively minimizes latency and redundant computation for semantically similar queries. In this work, we conceptualize semantic cache keys as a form of fuzzy hashes. We demonstrate that the locality required to maximize cache hit rates fundamentally conflicts with the cryptographic avalanche effect necessary for collision resistance. Our conceptual analysis formalizes this inherent trade-off between performance (locality) and security (collision resilience), revealing that semantic caching is naturally vulnerable to key collision attacks. While prior research has focused on side-channel and privacy risks, we present the first systematic study of integrity risks arising from cache collisions. We introduce CacheAttack, an automated framework for launching black-box collision attacks. We evaluate CacheAttack in security-critical tasks and agentic workflows. It achieves a hit rate of 86\% in LLM response hijacking and can induce malicious behaviors in LLM agent, while preserving strong transferability across different embedding models. A case study on a financial agent further illustrates the real-world impact of these vulnerabilities. Finally, we discuss mitigation strategies.

Title: WiFiPenTester: Advancing Wireless Ethical Hacking with Governed GenAI

• Authors: Haitham S. Al-Sinani, Chris J. Mitchell
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.23092
• Pdf URL: https://arxiv.org/pdf/2601.23092
• Copy Paste: [[2601.23092]] WiFiPenTester: Advancing Wireless Ethical Hacking with Governed GenAI(https://arxiv.org/abs/2601.23092)
• Keywords: security, attack, large language model
• Abstract: Wireless ethical hacking relies heavily on skilled practitioners manually interpreting reconnaissance results and executing complex, time-sensitive sequences of commands to identify vulnerable targets, capture authentication handshakes, and assess password resilience; a process that is inherently labour-intensive, difficult to scale, and prone to subjective judgement and human error. To help address these limitations, we propose WiFiPenTester, an experimental, governed, and reproducible system for GenAI-enabled wireless ethical hacking. The system integrates large language models into the reconnaissance and decision-support phases of wireless security assessment, enabling intelligent target ranking, attack feasibility estimation, and strategy recommendation, while preserving strict human-in-the-loop control and budget-aware execution. We describe the system architecture, threat model, governance mechanisms, and prompt-engineering methodology, and empirical experiments conducted across multiple wireless environments. The results demonstrate that GenAI assistance improves target selection accuracy and overall assessment efficiency, while maintaining auditability and ethical safeguards. This indicates that WiFiPenTester is a meaningful step toward practical, safe, and scalable GenAI-assisted wireless penetration testing, while reinforcing the necessity of bounded autonomy, human oversight, and rigorous governance mechanisms when deploying GenAI in ethical hacking.

Title: Safer Policy Compliance with Dynamic Epistemic Fallback

• Authors: Joseph Marvin Imperial, Harish Tayyar Madabushi
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.23094
• Pdf URL: https://arxiv.org/pdf/2601.23094
• Copy Paste: [[2601.23094]] Safer Policy Compliance with Dynamic Epistemic Fallback(https://arxiv.org/abs/2601.23094)
• Keywords: privacy, defense, attack, robust
• Abstract: Humans develop a series of cognitive defenses, known as epistemic vigilance, to combat risks of deception and misinformation from everyday interactions. Developing safeguards for LLMs inspired by this mechanism might be particularly helpful for their application in high-stakes tasks such as automating compliance with data privacy laws. In this paper, we introduce Dynamic Epistemic Fallback (DEF), a dynamic safety protocol for improving an LLM's inference-time defenses against deceptive attacks that make use of maliciously perturbed policy texts. Through various levels of one-sentence textual cues, DEF nudges LLMs to flag inconsistencies, refuse compliance, and fallback to their parametric knowledge upon encountering perturbed policy texts. Using globally recognized legal policies such as HIPAA and GDPR, our empirical evaluations report that DEF effectively improves the capability of frontier LLMs to detect and refuse perturbed versions of policies, with DeepSeek-R1 achieving a 100% detection rate in one setting. This work encourages further efforts to develop cognitively inspired defenses to improve LLM robustness against forms of harm and deception that exploit legal artifacts.

Title: CATTO: Balancing Preferences and Confidence in Language Models

• Authors: Nisarg Parikh, Kunjal Panchal, Ananya Sai, Pannaga Shivaswamy, Andrew Lan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.23096
• Pdf URL: https://arxiv.org/pdf/2601.23096
• Copy Paste: [[2601.23096]] CATTO: Balancing Preferences and Confidence in Language Models(https://arxiv.org/abs/2601.23096)
• Keywords: large language model
• Abstract: Large language models (LLMs) often make accurate next token predictions but their confidence in these predictions can be poorly calibrated: high-confidence predictions are frequently wrong, and low-confidence predictions may be correct. This miscalibration is exacerbated by preference-based alignment methods breaking the link between predictive probability and correctness. We introduce a Calibration Aware Token-level Training Objective (CATTO), a calibration-aware objective that aligns predicted confidence with empirical prediction correctness, which can be combined with the original preference optimization objectives. Empirically, CATTO reduces Expected Calibration Error (ECE) by 2.22%-7.61% in-distribution and 1.46%-10.44% out-of-distribution compared to direct preference optimization (DPO), and by 0.22%-1.24% in-distribution and 1.23%-5.07% out-of-distribution compared to the strongest DPO baseline. This improvement in confidence does not come at a cost of losing task accuracy, where CATTO maintains or slightly improves multiple-choice question-answering accuracy on five datasets. We also introduce Confidence@k, a test-time scaling mechanism leveraging calibrated token probabilities for Bayes-optimal selection of output tokens.

Title: Rethinking Transferable Adversarial Attacks on Point Clouds from a Compact Subspace Perspective

• Authors: Keke Tang, Xianheng Liu, Weilong Peng, Xiaofei Wang, Daizong Liu, Peican Zhu, Can Lu, Zhihong Tian
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.23102
• Pdf URL: https://arxiv.org/pdf/2601.23102
• Copy Paste: [[2601.23102]] Rethinking Transferable Adversarial Attacks on Point Clouds from a Compact Subspace Perspective(https://arxiv.org/abs/2601.23102)
• Keywords: defense, attack, robust
• Abstract: Transferable adversarial attacks on point clouds remain challenging, as existing methods often rely on model-specific gradients or heuristics that limit generalization to unseen architectures. In this paper, we rethink adversarial transferability from a compact subspace perspective and propose CoSA, a transferable attack framework that operates within a shared low-dimensional semantic space. Specifically, each point cloud is represented as a compact combination of class-specific prototypes that capture shared semantic structure, while adversarial perturbations are optimized within a low-rank subspace to induce coherent and architecture-agnostic variations. This design suppresses model-dependent noise and constrains perturbations to semantically meaningful directions, thereby improving cross-model transferability without relying on surrogate-specific artifacts. Extensive experiments on multiple datasets and network architectures demonstrate that CoSA consistently outperforms state-of-the-art transferable attacks, while maintaining competitive imperceptibility and robustness under common defense strategies. Codes will be made public upon paper acceptance.

Title: FlowCalib: LiDAR-to-Vehicle Miscalibration Detection using Scene Flows

• Authors: Ilir Tahiraj, Peter Wittal, Markus Lienkamp
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2601.23107
• Pdf URL: https://arxiv.org/pdf/2601.23107
• Copy Paste: [[2601.23107]] FlowCalib: LiDAR-to-Vehicle Miscalibration Detection using Scene Flows(https://arxiv.org/abs/2601.23107)
• Keywords: robust
• Abstract: Accurate sensor-to-vehicle calibration is essential for safe autonomous driving. Angular misalignments of LiDAR sensors can lead to safety-critical issues during autonomous operation. However, current methods primarily focus on correcting sensor-to-sensor errors without considering the miscalibration of individual sensors that cause these errors in the first place. We introduce FlowCalib, the first framework that detects LiDAR-to-vehicle miscalibration using motion cues from the scene flow of static objects. Our approach leverages the systematic bias induced by rotational misalignment in the flow field generated from sequential 3D point clouds, eliminating the need for additional sensors. The architecture integrates a neural scene flow prior for flow estimation and incorporates a dual-branch detection network that fuses learned global flow features with handcrafted geometric descriptors. These combined representations allow the system to perform two complementary binary classification tasks: a global binary decision indicating whether misalignment is present and separate, axis-specific binary decisions indicating whether each rotational axis is misaligned. Experiments on the nuScenes dataset demonstrate FlowCalib's ability to robustly detect miscalibration, establishing a benchmark for sensor-to-vehicle miscalibration detection.

Title: To See Far, Look Close: Evolutionary Forecasting for Long-term Time Series

• Authors: Jiaming Ma, Siyuan Mu, Ruilin Tang, Haofeng Ma, Qihe Huang, Zhengyang Zhou, Pengkun Wang, Binwu Wang, Yang Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.23114
• Pdf URL: https://arxiv.org/pdf/2601.23114
• Copy Paste: [[2601.23114]] To See Far, Look Close: Evolutionary Forecasting for Long-term Time Series(https://arxiv.org/abs/2601.23114)
• Keywords: robust, generative
• Abstract: The prevailing Direct Forecasting (DF) paradigm dominates Long-term Time Series Forecasting (LTSF) by forcing models to predict the entire future horizon in a single forward pass. While efficient, this rigid coupling of output and evaluation horizons necessitates computationally prohibitive re-training for every target horizon. In this work, we uncover a counter-intuitive optimization anomaly: models trained on short horizons-when coupled with our proposed Evolutionary Forecasting (EF) paradigm-significantly outperform those trained directly on long horizons. We attribute this success to the mitigation of a fundamental optimization pathology inherent in DF, where conflicting gradients from distant futures cripple the learning of local dynamics. We establish EF as a unified generative framework, proving that DF is merely a degenerate special case of EF. Extensive experiments demonstrate that a singular EF model surpasses task-specific DF ensembles across standard benchmarks and exhibits robust asymptotic stability in extreme extrapolation. This work propels a paradigm shift in LTSF: moving from passive Static Mapping to autonomous Evolutionary Reasoning.

Title: Secure Tool Manifest and Digital Signing Solution for Verifiable MCP and LLM Pipelines

• Authors: Saeid Jamshidi, Kawser Wazed Nafi, Arghavan Moradi Dakhel, Foutse Khomh, Amin Nikanjam, Mohammad Adnan Hamdaqa
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.23132
• Pdf URL: https://arxiv.org/pdf/2601.23132
• Copy Paste: [[2601.23132]] Secure Tool Manifest and Digital Signing Solution for Verifiable MCP and LLM Pipelines(https://arxiv.org/abs/2601.23132)
• Keywords: secure, security, large language model
• Abstract: Large Language Models (LLMs) are increasingly adopted in sensitive domains such as healthcare and financial institutions' data analytics; however, their execution pipelines remain vulnerable to manipulation and unverifiable behavior. Existing control mechanisms, such as the Model Context Protocol (MCP), define compliance policies for tool invocation but lack verifiable enforcement and transparent validation of model actions. To address this gap, we propose a novel Secure Tool Manifest and Digital Signing Framework, a structured and security-aware extension of Model Context Protocols. The framework enforces cryptographically signed manifests, integrates transparent verification logs, and isolates model-internal execution metadata from user-visible components to ensure verifiable execution integrity. Furthermore, the evaluation demonstrates that the framework scales nearly linearly (R-squared = 0.998), achieves near-perfect acceptance of valid executions while consistently rejecting invalid ones, and maintains balanced model utilization across execution pipelines.

Title: Securing Time in Energy IoT: A Clock-Dynamics-Aware Spatio-Temporal Graph Attention Network for Clock Drift Attacks and Y2K38 Failures

• Authors: Saeid Jamshidi, Omar Abdul Wahab, Rolando Herrero, Foutse Khomh
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.23147
• Pdf URL: https://arxiv.org/pdf/2601.23147
• Copy Paste: [[2601.23147]] Securing Time in Energy IoT: A Clock-Dynamics-Aware Spatio-Temporal Graph Attention Network for Clock Drift Attacks and Y2K38 Failures(https://arxiv.org/abs/2601.23147)
• Keywords: attack, transformer
• Abstract: The integrity of time in distributed Internet of Things (IoT) devices is crucial for reliable operation in energy cyber-physical systems, such as smart grids and microgrids. However, IoT systems are vulnerable to clock drift, time-synchronization manipulation, and timestamp discontinuities, such as the Year 2038 (Y2K38) Unix overflow, all of which disrupt temporal ordering. Conventional anomaly-detection models, which assume reliable timestamps, fail to capture temporal inconsistencies. This paper introduces STGAT (Spatio-Temporal Graph Attention Network), a framework that models both temporal distortion and inter-device consistency in energy IoT systems. STGAT combines drift-aware temporal embeddings and temporal self-attention to capture corrupted time evolution at individual devices, and uses graph attention to model spatial propagation of timing errors. A curvature-regularized latent representation geometrically separates normal clock evolution from anomalies caused by drift, synchronization offsets, and overflow events. Experimental results on energy IoT telemetry with controlled timing perturbations show that STGAT achieves 95.7% accuracy, outperforming recurrent, transformer, and graph-based baselines with significant improvements (d > 1.8, p < 0.001). Additionally, STGAT reduces detection delay by 26%, achieving a 2.3-time-step delay while maintaining stable performance under overflow, drift, and physical inconsistencies.

Title: Manifold-Aware Perturbations for Constrained Generative Modeling

• Authors: Katherine Keegan, Lars Ruthotto
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.23151
• Pdf URL: https://arxiv.org/pdf/2601.23151
• Copy Paste: [[2601.23151]] Manifold-Aware Perturbations for Constrained Generative Modeling(https://arxiv.org/abs/2601.23151)
• Keywords: diffusion, generative
• Abstract: Generative models have enjoyed widespread success in a variety of applications. However, they encounter inherent mathematical limitations in modeling distributions where samples are constrained by equalities, as is frequently the setting in scientific domains. In this work, we develop a computationally cheap, mathematically justified, and highly flexible distributional modification for combating known pitfalls in equality-constrained generative models. We propose perturbing the data distribution in a constraint-aware way such that the new distribution has support matching the ambient space dimension while still implicitly incorporating underlying manifold geometry. Through theoretical analyses and empirical evidence on several representative tasks, we illustrate that our approach consistently enables data distribution recovery and stable sampling with both diffusion models and normalizing flows.

Title: Behemoth: Benchmarking Unlearning in LLMs Using Fully Synthetic Data

• Authors: Eugenia Iofinova, Dan Alistarh
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.23153
• Pdf URL: https://arxiv.org/pdf/2601.23153
• Copy Paste: [[2601.23153]] Behemoth: Benchmarking Unlearning in LLMs Using Fully Synthetic Data(https://arxiv.org/abs/2601.23153)
• Keywords: large language model
• Abstract: As artificial neural networks, and specifically large language models, have improved rapidly in capabilities and quality, they have increasingly been deployed in real-world applications, from customer service to Google search, despite the fact that they frequently make factually incorrect or undesirable statements. This trend has inspired practical and academic interest in model editing, that is, in adjusting the weights of the model to modify its likely outputs for queries relating to a specific fact or set of facts. This may be done either to amend a fact or set of facts, for instance, to fix a frequent error in the training data, or to suppress a fact or set of facts entirely, for instance, in case of dangerous knowledge. Multiple methods have been proposed to do such edits. However, at the same time, it has been shown that such model editing can be brittle and incomplete. Moreover the effectiveness of any model editing method necessarily depends on the data on which the model is trained, and, therefore, a good understanding of the interaction of the training data distribution and the way it is stored in the network is necessary and helpful to reliably perform model editing. However, working with large language models trained on real-world data does not allow us to understand this relationship or fully measure the effects of model editing. We therefore propose Behemoth, a fully synthetic data generation framework. To demonstrate the practical insights from the framework, we explore model editing in the context of simple tabular data, demonstrating surprising findings that, in some cases, echo real-world results, for instance, that in some cases restricting the update rank results in a more effective update. The code is available at this https URL.

Title: SPICE: Submodular Penalized Information-Conflict Selection for Efficient Large Language Model Training

• Authors: Powei Chang, Jinpeng Zhang, Bowen Chen, Chenyu Wang, Chenlu Guo, Yixing Zhang, Yukang Gao, JianXiang Xiang, Yue Gao, Chaoqun Sun, Yiyi Chen, Dongying Kong
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.23155
• Pdf URL: https://arxiv.org/pdf/2601.23155
• Copy Paste: [[2601.23155]] SPICE: Submodular Penalized Information-Conflict Selection for Efficient Large Language Model Training(https://arxiv.org/abs/2601.23155)
• Keywords: large language model
• Abstract: Information-based data selection for instruction tuning is compelling: maximizing the log-determinant of the Fisher information yields a monotone submodular objective, enabling greedy algorithms to achieve a $(1-1/e)$ approximation under a cardinality budget. In practice, however, we identify alleviating gradient conflicts, misalignment between per-sample gradients, is a key factor that slows down the decay of marginal log-determinant information gains, thereby preventing significant loss of information. We formalize this via an $\varepsilon$-decomposition that quantifies the deviation from ideal submodularity as a function of conflict statistics, yielding data-dependent approximation factors that tighten as conflicts diminish. Guided by this analysis, we propose SPICE, a conflict-aware selector that maximizes information while penalizing misalignment, and that supports early stopping and proxy models for efficiency. Empirically, SPICE selects subsets with higher log-determinant information than original criteria, and these informational gains translate into performance improvements: across 8 benchmarks with LLaMA2-7B and Qwen2-7B, SPICE uses only 10% of the data, yet matches or exceeds 6 methods including full-data tuning. This achieves performance improvements with substantially lower training cost.

Title: Unsupervised Hierarchical Skill Discovery

• Authors: Damion Harvey (1), Geraud Nangue Tasse (1 and 2), Branden Ingram (1 and 2), Benjamin Rosman (1 and 2), Steven James (1 and 2) ((1) University of the Witwatersrand, Johannesburg, South Africa, (2) Machine Intelligence and Neural Discovery (MIND) Institute, University of the Witwatersrand, Johannesburg, South Africa)
• Subjects: cs.LG, cs.FL
• Abstract URL: https://arxiv.org/abs/2601.23156
• Pdf URL: https://arxiv.org/pdf/2601.23156
• Copy Paste: [[2601.23156]] Unsupervised Hierarchical Skill Discovery(https://arxiv.org/abs/2601.23156)
• Keywords: segmentation
• Abstract: We consider the problem of unsupervised skill segmentation and hierarchical structure discovery in reinforcement learning. While recent approaches have sought to segment trajectories into reusable skills or options, most rely on action labels, rewards, or handcrafted annotations, limiting their applicability. We propose a method that segments unlabelled trajectories into skills and induces a hierarchical structure over them using a grammar-based approach. The resulting hierarchy captures both low-level behaviours and their composition into higher-level skills. We evaluate our approach in high-dimensional, pixel-based environments, including Craftax and the full, unmodified version of Minecraft. Using metrics for skill segmentation, reuse, and hierarchy quality, we find that our method consistently produces more structured and semantically meaningful hierarchies than existing baselines. Furthermore, as a proof of concept for utility, we demonstrate that these discovered hierarchies accelerate and stabilise learning on downstream reinforcement learning tasks.

Title: No More, No Less: Least-Privilege Language Models

• Authors: Paulius Rauba, Dominykas Seputis, Patrikas Vanagas, Mihaela van der Schaar
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.23157
• Pdf URL: https://arxiv.org/pdf/2601.23157
• Copy Paste: [[2601.23157]] No More, No Less: Least-Privilege Language Models(https://arxiv.org/abs/2601.23157)
• Keywords: security
• Abstract: Least privilege is a core security principle: grant each request only the minimum access needed to achieve its goal. Deployed language models almost never follow it, instead being exposed through a single API endpoint that serves all users and requests. This gap exists not because least privilege would be unhelpful; deployments would benefit greatly from reducing unnecessary capability exposure. The real obstacle is definitional and mechanistic: what does "access" mean inside a language model, and how can we enforce it without retraining or deploying multiple models? We take inspiration from least privilege in computer systems and define a class of models called least-privilege language models, where privilege is reachable internal computation during the forward pass. In this view, lowering privilege literally shrinks the model's accessible function class, as opposed to denying access via learned policies. We formalize deployment-time control as a monitor-allocator-enforcer stack, separating (i) request-time signals, (ii) a decision rule that allocates privilege, and (iii) an inference-time mechanism that selects privilege. We then propose Nested Least-Privilege Networks, a shape-preserving, rank-indexed intervention that provides a smooth, reversible control knob. We show that this knob yields policy-usable privilege-utility frontiers and enables selective suppression of targeted capabilities with limited collateral degradation across various policies. Most importantly, we argue for a new deployment paradigm that challenges the premise that language models can only be controlled at the output level.

Title: Segment Any Events with Language

• Authors: Seungjun Lee, Gim Hee Lee
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.23159
• Pdf URL: https://arxiv.org/pdf/2601.23159
• Copy Paste: [[2601.23159]] Segment Any Events with Language(https://arxiv.org/abs/2601.23159)
• Keywords: segmentation
• Abstract: Scene understanding with free-form language has been widely explored within diverse modalities such as images, point clouds, and LiDAR. However, related studies on event sensors are scarce or narrowly centered on semantic-level understanding. We introduce SEAL, the first Semantic-aware Segment Any Events framework that addresses Open-Vocabulary Event Instance Segmentation (OV-EIS). Given the visual prompt, our model presents a unified framework to support both event segmentation and open-vocabulary mask classification at multiple levels of granularity, including instance-level and part-level. To enable thorough evaluation on OV-EIS, we curate four benchmarks that cover label granularity from coarse to fine class configurations and semantic granularity from instance-level to part-level understanding. Extensive experiments show that our SEAL largely outperforms proposed baselines in terms of performance and inference speed with a parameter-efficient architecture. In the Appendix, we further present a simple variant of our SEAL achieving generic spatiotemporal OV-EIS that does not require any visual prompts from users in the inference. Check out our project page in this https URL

Title: Probing the Trajectories of Reasoning Traces in Large Language Models

• Authors: Marthe Ballon, Brecht Verbeken, Vincent Ginis, Andres Algaba
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.23163
• Pdf URL: https://arxiv.org/pdf/2601.23163
• Copy Paste: [[2601.23163]] Probing the Trajectories of Reasoning Traces in Large Language Models(https://arxiv.org/abs/2601.23163)
• Keywords: large language model
• Abstract: Large language models (LLMs) increasingly solve difficult problems by producing "reasoning traces" before emitting a final response. However, it remains unclear how accuracy and decision commitment evolve along a reasoning trajectory, and whether intermediate trace segments provide answer-relevant information beyond generic length or stylistic effects. Here, we propose a protocol to systematically probe the trajectories of reasoning traces in LLMs by 1) generating a model's reasoning trace, 2) truncating it at fixed token-percentiles, and 3) injecting each partial trace back into the model (or a different model) to measure the induced distribution over answer choices via next-token probabilities. We apply this protocol to the open-source Qwen3-4B/-8B/-14B and gpt-oss-20b/-120b models across the multiple-choice GPQA Diamond and MMLU-Pro benchmarks. We find that accuracy and decision commitment consistently increase as the percentage of provided reasoning tokens grows. These gains are primarily driven by relevant content in the model generation rather than context length or generic "reasoning style" effects. Stronger models often backtrack successfully from incorrect partial traces, but immediate answers often remain anchored in the weaker model's incorrect response. More broadly, we show that trajectory probing provides diagnostics for efficient and safer deployment of reasoning models as the measurements can inform practical trace-handling and monitoring policies that improve reliability without assuming intermediate tokens are inherently faithful explanations.

Title: Hi-Light: A Path to high-fidelity, high-resolution video relighting with a Novel Evaluation Paradigm

• Authors: Xiangrui Liu, Haoxiang Li, Yezhou Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.23167
• Pdf URL: https://arxiv.org/pdf/2601.23167
• Copy Paste: [[2601.23167]] Hi-Light: A Path to high-fidelity, high-resolution video relighting with a Novel Evaluation Paradigm(https://arxiv.org/abs/2601.23167)
• Keywords: robust, diffusion
• Abstract: Video relighting offers immense creative potential and commercial value but is hindered by challenges, including the absence of an adequate evaluation metric, severe light flickering, and the degradation of fine-grained details during editing. To overcome these challenges, we introduce Hi-Light, a novel, training-free framework for high-fidelity, high-resolution, robust video relighting. Our approach introduces three technical innovations: lightness prior anchored guided relighting diffusion that stabilises intermediate relit video, a Hybrid Motion-Adaptive Lighting Smoothing Filter that leverages optical flow to ensure temporal stability without introducing motion blur, and a LAB-based Detail Fusion module that preserves high-frequency detail information from the original video. Furthermore, to address the critical gap in evaluation, we propose the Light Stability Score, the first quantitative metric designed to specifically measure lighting consistency. Extensive experiments demonstrate that Hi-Light significantly outperforms state-of-the-art methods in both qualitative and quantitative comparisons, producing stable, highly detailed relit videos.

Title: Names Don't Matter: Symbol-Invariant Transformer for Open-Vocabulary Learning

• Authors: İlker Işık, Wenchao Li
• Subjects: cs.LG, cs.LO, cs.SC
• Abstract URL: https://arxiv.org/abs/2601.23169
• Pdf URL: https://arxiv.org/pdf/2601.23169
• Copy Paste: [[2601.23169]] Names Don't Matter: Symbol-Invariant Transformer for Open-Vocabulary Learning(https://arxiv.org/abs/2601.23169)
• Keywords: transformer
• Abstract: Current neural architectures lack a principled way to handle interchangeable tokens, i.e., symbols that are semantically equivalent yet distinguishable, such as bound variables. As a result, models trained on fixed vocabularies often struggle to generalize to unseen symbols, even when the underlying semantics remain unchanged. We propose a novel Transformer-based mechanism that is provably invariant to the renaming of interchangeable tokens. Our approach employs parallel embedding streams to isolate the contribution of each interchangeable token in the input, combined with an aggregated attention mechanism that enables structured information sharing across streams. Experimental results confirm the theoretical guarantees of our method and demonstrate substantial performance gains on open-vocabulary tasks that require generalization to novel symbols.

Title: MeshGraphNet-Transformer: Scalable Mesh-based Learned Simulation for Solid Mechanics

• Authors: Mikel M. Iparraguirre, Iciar Alfaro, David Gonzalez, Elias Cueto
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.23177
• Pdf URL: https://arxiv.org/pdf/2601.23177
• Copy Paste: [[2601.23177]] MeshGraphNet-Transformer: Scalable Mesh-based Learned Simulation for Solid Mechanics(https://arxiv.org/abs/2601.23177)
• Keywords: transformer
• Abstract: We present MeshGraphNet-Transformer (MGN-T), a novel architecture that combines the global modeling capabilities of Transformers with the geometric inductive bias of MeshGraphNets, while preserving a mesh-based graph representation. MGN-T overcomes a key limitation of standard MGN, the inefficient long-range information propagation caused by iterative message passing on large, high-resolution meshes. A physics-attention Transformer serves as a global processor, updating all nodal states simultaneously while explicitly retaining node and edge attributes. By directly capturing long-range physical interactions, MGN-T eliminates the need for deep message-passing stacks or hierarchical, coarsened meshes, enabling efficient learning on high-resolution meshes with varying geometries, topologies, and boundary conditions at an industrial scale. We demonstrate that MGN-T successfully handles industrial-scale meshes for impact dynamics, a setting in which standard MGN fails due message-passing under-reaching. The method accurately models self-contact, plasticity, and multivariate outputs, including internal, phenomenological plastic variables. Moreover, MGN-T outperforms state-of-the-art approaches on classical benchmarks, achieving higher accuracy while maintaining practical efficiency, using only a fraction of the parameters required by competing baselines.

Title: TriSpec: Ternary Speculative Decoding via Lightweight Proxy Verification

• Authors: Haoyun Jiang, Junqi He, Feng Hong, Xinlong Yang, Jianwei Zhang, Zheng Li, Zhengyang Zhuge, Zhiyong Chen, Bo Han, Junyang Lin, Jiangchao Yao
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.23180
• Pdf URL: https://arxiv.org/pdf/2601.23180
• Copy Paste: [[2601.23180]] TriSpec: Ternary Speculative Decoding via Lightweight Proxy Verification(https://arxiv.org/abs/2601.23180)
• Keywords: large language model
• Abstract: Inference efficiency in Large Language Models (LLMs) is fundamentally limited by their serial, autoregressive generation, especially as reasoning becomes a key capability and response sequences grow longer. Speculative decoding (SD) offers a powerful solution, providing significant speed-ups through its lightweight drafting and parallel verification mechanism. While existing work has nearly saturated improvements in draft effectiveness and efficiency, this paper advances SD from a new yet critical perspective: the verification cost. We propose TriSpec, a novel ternary SD framework that, at its core, introduces a lightweight proxy to significantly reduce computational cost by approving easily verifiable draft sequences and engaging the full target model only when encountering uncertain tokens. TriSpec can be integrated with state-of-the-art SD methods like EAGLE-3 to further reduce verification costs, achieving greater acceleration. Extensive experiments on the Qwen3 and DeepSeek-R1-Distill-Qwen/LLaMA families show that TriSpec achieves up to 35\% speedup over standard SD, with up to 50\% fewer target model invocations while maintaining comparable accuracy.

Title: FourierSampler: Unlocking Non-Autoregressive Potential in Diffusion Language Models via Frequency-Guided Generation

• Authors: Siyang He, Qiqi Wang, Xiaoran Liu, Hongnan Ma, Yiwei Shi, Yuerong Song, Ying Zhu, Tianyi Liang, Zengfeng Huang, Ziwei He, Xipeng Qiu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.23182
• Pdf URL: https://arxiv.org/pdf/2601.23182
• Copy Paste: [[2601.23182]] FourierSampler: Unlocking Non-Autoregressive Potential in Diffusion Language Models via Frequency-Guided Generation(https://arxiv.org/abs/2601.23182)
• Keywords: diffusion
• Abstract: Despite the non-autoregressive potential of diffusion language models (dLLMs), existing decoding strategies demonstrate positional bias, failing to fully unlock the potential of arbitrary generation. In this work, we delve into the inherent spectral characteristics of dLLMs and present the first frequency-domain analysis showing that low-frequency components in hidden states primarily encode global structural information and long-range dependencies, while high-frequency components are responsible for characterizing local details. Based on this observation, we propose FourierSampler, which leverages a frequency-domain sliding window mechanism to dynamically guide the model to achieve a "structure-to-detail" generation. FourierSampler outperforms other inference enhancement strategies on LLADA and SDAR, achieving relative improvements of 20.4% on LLaDA1.5-8B and 16.0% on LLaDA-8B-Instruct. It notably surpasses similarly sized autoregressive models like Llama3.1-8B-Instruct.

Title: JobResQA: A Benchmark for LLM Machine Reading Comprehension on Multilingual Résumés and JDs

• Authors: Casimiro Pio Carrino, Paula Estrella, Rabih Zbib, Carlos Escolano, José A. R. Fonollosa
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.23183
• Pdf URL: https://arxiv.org/pdf/2601.23183
• Copy Paste: [[2601.23183]] JobResQA: A Benchmark for LLM Machine Reading Comprehension on Multilingual Résumés and JDs(https://arxiv.org/abs/2601.23183)
• Keywords: privacy, extraction, fair
• Abstract: We introduce JobResQA, a multilingual Question Answering benchmark for evaluating Machine Reading Comprehension (MRC) capabilities of LLMs on HR-specific tasks involving résumés and job descriptions. The dataset comprises 581 QA pairs across 105 synthetic résumé-job description pairs in five languages (English, Spanish, Italian, German, and Chinese), with questions spanning three complexity levels from basic factual extraction to complex cross-document reasoning. We propose a data generation pipeline derived from real-world sources through de-identification and data synthesis to ensure both realism and privacy, while controlled demographic and professional attributes (implemented via placeholders) enable systematic bias and fairness studies. We also present a cost-effective, human-in-the-loop translation pipeline based on the TEaR methodology, incorporating MQM error annotations and selective post-editing to ensure an high-quality multi-way parallel benchmark. We provide a baseline evaluations across multiple open-weight LLM families using an LLM-as-judge approach revealing higher performances on English and Spanish but substantial degradation for other languages, highlighting critical gaps in multilingual MRC capabilities for HR applications. JobResQA provides a reproducible benchmark for advancing fair and reliable LLM-based HR systems. The benchmark is publicly available at: this https URL

Title: ReGuLaR: Variational Latent Reasoning Guided by Rendered Chain-of-Thought

• Authors: Fanmeng Wang, Haotian Liu, Guojiang Zhao, Hongteng Xu, Zhifeng Gao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.23184
• Pdf URL: https://arxiv.org/pdf/2601.23184
• Copy Paste: [[2601.23184]] ReGuLaR: Variational Latent Reasoning Guided by Rendered Chain-of-Thought(https://arxiv.org/abs/2601.23184)
• Keywords: large language model
• Abstract: While Chain-of-Thought (CoT) significantly enhances the performance of Large Language Models (LLMs), explicit reasoning chains introduce substantial computational redundancy. Recent latent reasoning methods attempt to mitigate this by compressing reasoning processes into latent space, but often suffer from severe performance degradation due to the lack of appropriate compression guidance. In this study, we propose Rendered CoT-Guided variational Latent Reasoning (ReGuLaR), a simple yet novel latent learning paradigm resolving this issue. Fundamentally, we formulate latent reasoning within the Variational Auto-Encoding (VAE) framework, sampling the current latent reasoning state from the posterior distribution conditioned on previous ones. Specifically, when learning this variational latent reasoning model, we render explicit reasoning chains as images, from which we extract dense visual-semantic representations to regularize the posterior distribution, thereby achieving efficient compression with minimal information loss. Extensive experiments demonstrate that ReGuLaR significantly outperforms existing latent reasoning methods across both computational efficiency and reasoning effectiveness, and even surpasses CoT through multi-modal reasoning, providing a new and insightful solution to latent reasoning. Code: this https URL.

Title: Deep Search with Hierarchical Meta-Cognitive Monitoring Inspired by Cognitive Neuroscience

• Authors: Zhongxiang Sun, Qipeng Wang, Weijie Yu, Jingxuan Yang, Haolang Lu, Jun Xu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.23188
• Pdf URL: https://arxiv.org/pdf/2601.23188
• Copy Paste: [[2601.23188]] Deep Search with Hierarchical Meta-Cognitive Monitoring Inspired by Cognitive Neuroscience(https://arxiv.org/abs/2601.23188)
• Keywords: robust, large language model
• Abstract: Deep search agents powered by large language models have demonstrated strong capabilities in multi-step retrieval, reasoning, and long-horizon task execution. However, their practical failures often stem from the lack of mechanisms to monitor and regulate reasoning and retrieval states as tasks evolve under uncertainty. Insights from cognitive neuroscience suggest that human metacognition is hierarchically organized, integrating fast anomaly detection with selectively triggered, experience-driven reflection. In this work, we propose Deep Search with Meta-Cognitive Monitoring (DS-MCM), a deep search framework augmented with an explicit hierarchical metacognitive monitoring mechanism. DS-MCM integrates a Fast Consistency Monitor, which performs lightweight checks on the alignment between external evidence and internal reasoning confidence, and a Slow Experience-Driven Monitor, which is selectively activated to guide corrective intervention based on experience memory from historical agent trajectories. By embedding monitoring directly into the reasoning-retrieval loop, DS-MCM determines both when intervention is warranted and how corrective actions should be informed by prior experience. Experiments across multiple deep search benchmarks and backbone models demonstrate that DS-MCM consistently improves performance and robustness.

Title: Med-Scout: Curing MLLMs' Geometric Blindness in Medical Perception via Geometry-Aware RL Post-Training

• Authors: Anglin Liu, Ruichao Chen, Yi Lu, Hongxia Xu, Jintai Chen
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.23220
• Pdf URL: https://arxiv.org/pdf/2601.23220
• Copy Paste: [[2601.23220]] Med-Scout: Curing MLLMs' Geometric Blindness in Medical Perception via Geometry-Aware RL Post-Training(https://arxiv.org/abs/2601.23220)
• Keywords: large language model
• Abstract: Despite recent Multimodal Large Language Models (MLLMs)' linguistic prowess in medical diagnosis, we find even state-of-the-art MLLMs suffer from a critical perceptual deficit: geometric blindness. This failure to ground outputs in objective geometric constraints leads to plausible yet factually incorrect hallucinations, rooted in training paradigms that prioritize linguistic fluency over geometric fidelity. This paper introduces Med-Scout, a novel framework that "cures" this blindness via Reinforcement Learning (RL) that leverages the intrinsic geometric logic latent within unlabeled medical images. Instead of relying on costly expert annotations, Med-Scout derives verifiable supervision signals through three strategic proxy tasks: Hierarchical Scale Localization, Topological Jigsaw Reconstruction, and Anomaly Consistency Detection. To rigorously quantify this deficit, we present Med-Scout-Bench, a new benchmark specifically designed to evaluate geometric perception. Extensive evaluations show that Med-Scout significantly mitigates geometric blindness, outperforming leading proprietary and open-source MLLMs by over 40% on our benchmark. Furthermore, this enhanced geometric perception generalizes to broader medical understanding, achieving superior results on radiological and comprehensive medical VQA tasks.

Title: Optimal Fair Aggregation of Crowdsourced Noisy Labels using Demographic Parity Constraints

• Authors: Gabriel Singer, Samuel Gruffaz, Olivier Vo Van, Nicolas Vayatis, Argyris Kalogeratos
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.23221
• Pdf URL: https://arxiv.org/pdf/2601.23221
• Copy Paste: [[2601.23221]] Optimal Fair Aggregation of Crowdsourced Noisy Labels using Demographic Parity Constraints(https://arxiv.org/abs/2601.23221)
• Keywords: fair
• Abstract: As acquiring reliable ground-truth labels is usually costly, or infeasible, crowdsourcing and aggregation of noisy human annotations is the typical resort. Aggregating subjective labels, though, may amplify individual biases, particularly regarding sensitive features, raising fairness concerns. Nonetheless, fairness in crowdsourced aggregation remains largely unexplored, with no existing convergence guarantees and only limited post-processing approaches for enforcing $\varepsilon$-fairness under demographic parity. We address this gap by analyzing the fairness s of crowdsourced aggregation methods within the $\varepsilon$-fairness framework, for Majority Vote and Optimal Bayesian aggregation. In the small-crowd regime, we derive an upper bound on the fairness gap of Majority Vote in terms of the fairness gaps of the individual annotators. We further show that the fairness gap of the aggregated consensus converges exponentially fast to that of the ground-truth under interpretable conditions. Since ground-truth itself may still be unfair, we generalize a state-of-the-art multiclass fairness post-processing algorithm from the continuous to the discrete setting, which enforces strict demographic parity constraints to any aggregation rule. Experiments on synthetic and real datasets demonstrate the effectiveness of our approach and corroborate the theoretical insights.

Title: Region-Normalized DPO for Medical Image Segmentation under Noisy Judges

• Authors: Hamza Kalisch, Constantin Seibold, Jens Kleesiek, Ken Herrmann, Frederic Jonske
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.23222
• Pdf URL: https://arxiv.org/pdf/2601.23222
• Copy Paste: [[2601.23222]] Region-Normalized DPO for Medical Image Segmentation under Noisy Judges(https://arxiv.org/abs/2601.23222)
• Keywords: segmentation
• Abstract: While dense pixel-wise annotations remain the gold standard for medical image segmentation, they are costly to obtain and limit scalability. In contrast, many deployed systems already produce inexpensive automatic quality-control (QC) signals like model agreement, uncertainty measures, or learned mask-quality scores which can be used for further model training without additional ground-truth annotation. However, these signals can be noisy and biased, making preference-based fine-tuning susceptible to harmful updates. We study Direct Preference Optimization (DPO) for segmentation from such noisy judges using proposals generated by a supervised base segmenter trained on a small labeled set. We find that outcomes depend strongly on how preference pairs are mined: selecting the judge's top-ranked proposal can improve peak performance when the judge is reliable, but can amplify harmful errors under weaker judges. We propose Region-Normalized DPO (RN-DPO), a segmentation-aware objective which normalizes preference updates by the size of the disagreement region between masks, reducing the leverage of harmful comparisons and improving optimization stability. Across two medical datasets and multiple regimes, RN-DPO improves sustained performance and stabilizes preference-based fine-tuning, outperforming standard DPO and strong baselines without requiring additional pixel annotations.

Title: Video-o3: Native Interleaved Clue Seeking for Long Video Multi-Hop Reasoning

• Authors: Xiangyu Zeng, Zhiqiu Zhang, Yuhan Zhu, Xinhao Li, Zikang Wang, Changlian Ma, Qingyu Zhang, Zizheng Huang, Kun Ouyang, Tianxiang Jiang, Ziang Yan, Yi Wang, Hongjie Zhang, Yali Wang, Limin Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.23224
• Pdf URL: https://arxiv.org/pdf/2601.23224
• Copy Paste: [[2601.23224]] Video-o3: Native Interleaved Clue Seeking for Long Video Multi-Hop Reasoning(https://arxiv.org/abs/2601.23224)
• Keywords: large language model
• Abstract: Existing multimodal large language models for long-video understanding predominantly rely on uniform sampling and single-turn inference, limiting their ability to identify sparse yet critical evidence amid extensive redundancy. We introduce Video-o3, a novel framework that supports iterative discovery of salient visual clues, fine-grained inspection of key segments, and adaptive termination once sufficient evidence is acquired. Technically, we address two core challenges in interleaved tool invocation. First, to mitigate attention dispersion induced by the heterogeneity of reasoning and tool-calling, we propose Task-Decoupled Attention Masking, which isolates per-step concentration while preserving shared global context. Second, to control context length growth in multi-turn interactions, we introduce a Verifiable Trajectory-Guided Reward that balances exploration coverage with reasoning efficiency. To support training at scale, we further develop a data synthesis pipeline and construct Seeker-173K, comprising 173K high-quality tool-interaction trajectories for effective supervised and reinforcement learning. Extensive experiments show that Video-o3 substantially outperforms state-of-the-art methods, achieving 72.1% accuracy on MLVU and 46.5% on Video-Holmes. These results demonstrate Video-o3's strong multi-hop evidence-seeking and reasoning capabilities, and validate the effectiveness of native tool invocation in long-video scenarios.

Title: Agile Reinforcement Learning through Separable Neural Architecture

• Authors: Rajib Mostakim, Reza T. Batley, Sourav Saha
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.23225
• Pdf URL: https://arxiv.org/pdf/2601.23225
• Copy Paste: [[2601.23225]] Agile Reinforcement Learning through Separable Neural Architecture(https://arxiv.org/abs/2601.23225)
• Keywords: robust
• Abstract: Deep reinforcement learning (RL) is increasingly deployed in resource-constrained environments, yet the go-to function approximators - multilayer perceptrons (MLPs) - are often parameter-inefficient due to an imperfect inductive bias for the smooth structure of many value functions. This mismatch can also hinder sample efficiency and slow policy learning in this capacity-limited regime. Although model compression techniques exist, they operate post-hoc and do not improve learning efficiency. Recent spline-based separable architectures - such as Kolmogorov-Arnold Networks (KANs) - have been shown to offer parameter efficiency but are widely reported to exhibit significant computational overhead, especially at scale. In seeking to address these limitations, this work introduces SPAN (SPline-based Adaptive Networks), a novel function approximation approach to RL. SPAN adapts the low rank KHRONOS framework by integrating a learnable preprocessing layer with a separable tensor product B-spline basis. SPAN is evaluated across discrete (PPO) and high-dimensional continuous (SAC) control tasks, as well as offline settings (Minari/D4RL). Empirical results demonstrate that SPAN achieves a 30-50% improvement in sample efficiency and 1.3-9 times higher success rates across benchmarks compared to MLP baselines. Furthermore, SPAN demonstrates superior anytime performance and robustness to hyperparameter variations, suggesting it as a viable, high performance alternative for learning intrinsically efficient policies in resource-limited settings.

Title: ShotFinder: Imagination-Driven Open-Domain Video Shot Retrieval via Web Search

• Authors: Tao Yu, Haopeng Jin, Hao Wang, Shenghua Chai, Yujia Yang, Junhao Gong, Jiaming Guo, Minghui Zhang, Xinlong Chen, Zhenghao Zhang, Yuxuan Zhou, Yanpei Gong, YuanCheng Liu, Yiming Ding, Kangwei Zeng, Pengfei Yang, Zhongtian Luo, Yufei Xiong, Shanbin Zhang, Shaoxiong Cheng, Huang Ruilin, Li Shuo, Yuxi Niu, Xinyuan Zhang, Yueya Xu, Jie Mao, Ruixuan Ji, Yaru Zhao, Mingchen Zhang, Jiabing Yang, Jiaqi Liu, YiFan Zhang, Hongzhu Yi, Xinming Wang, Cheng Zhong, Xiao Ma, Zhang Zhang, Yan Huang, Liang Wang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.23232
• Pdf URL: https://arxiv.org/pdf/2601.23232
• Copy Paste: [[2601.23232]] ShotFinder: Imagination-Driven Open-Domain Video Shot Retrieval via Web Search(https://arxiv.org/abs/2601.23232)
• Keywords: large language model
• Abstract: In recent years, large language models (LLMs) have made rapid progress in information retrieval, yet existing research has mainly focused on text or static multimodal settings. Open-domain video shot retrieval, which involves richer temporal structure and more complex semantics, still lacks systematic benchmarks and analysis. To fill this gap, we introduce ShotFinder, a benchmark that formalizes editing requirements as keyframe-oriented shot descriptions and introduces five types of controllable single-factor constraints: Temporal order, Color, Visual style, Audio, and Resolution. We curate 1,210 high-quality samples from YouTube across 20 thematic categories, using large models for generation with human verification. Based on the benchmark, we propose ShotFinder, a text-driven three-stage retrieval and localization pipeline: (1) query expansion via video imagination, (2) candidate video retrieval with a search engine, and (3) description-guided temporal localization. Experiments on multiple closed-source and open-source models reveal a significant gap to human performance, with clear imbalance across constraints: temporal localization is relatively tractable, while color and visual style remain major challenges. These results reveal that open-domain video shot retrieval is still a critical capability that multimodal large models have yet to overcome.

Title: Sequence Diffusion Model for Temporal Link Prediction in Continuous-Time Dynamic Graph

• Authors: Nguyen Minh Duc, Viet Cuong Ta
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.23233
• Pdf URL: https://arxiv.org/pdf/2601.23233
• Copy Paste: [[2601.23233]] Sequence Diffusion Model for Temporal Link Prediction in Continuous-Time Dynamic Graph(https://arxiv.org/abs/2601.23233)
• Keywords: diffusion, generative
• Abstract: Temporal link prediction in dynamic graphs is a fundamental problem in many real-world systems. Existing temporal graph neural networks mainly focus on learning representations of historical interactions. Despite their strong performance, these models are still purely discriminative, producing point estimates for future links and lacking an explicit mechanism to capture the uncertainty and sequential structure of future temporal interactions. In this paper, we propose SDG, a novel sequence-level diffusion framework that unifies dynamic graph learning with generative denoising. Specifically, SDG injects noise into the entire historical interaction sequence and jointly reconstructs all interaction embeddings through a conditional denoising process, thereby enabling the model to capture more comprehensive interaction distributions. To align the generative process with temporal link prediction, we employ a cross-attention denoising decoder to guide the reconstruction of the destination sequence and optimize the model in an end-to-end manner. Extensive experiments on various temporal graph benchmarks show that SDG consistently achieves state-of-the-art performance in the temporal link prediction task.

Title: YuriiFormer: A Suite of Nesterov-Accelerated Transformers

• Authors: Aleksandr Zimin, Yury Polyanskiy, Philippe Rigollet
• Subjects: cs.LG, cs.AI, math.OC, stat.ML
• Abstract URL: https://arxiv.org/abs/2601.23236
• Pdf URL: https://arxiv.org/pdf/2601.23236
• Copy Paste: [[2601.23236]] YuriiFormer: A Suite of Nesterov-Accelerated Transformers(https://arxiv.org/abs/2601.23236)
• Keywords: transformer
• Abstract: We propose a variational framework that interprets transformer layers as iterations of an optimization algorithm acting on token embeddings. In this view, self-attention implements a gradient step of an interaction energy, while MLP layers correspond to gradient updates of a potential energy. Standard GPT-style transformers emerge as vanilla gradient descent on the resulting composite objective, implemented via Lie--Trotter splitting between these two energy functionals. This perspective enables principled architectural design using classical optimization ideas. As a proof of concept, we introduce a Nesterov-style accelerated transformer that preserves the same attention and MLP oracles. The resulting architecture consistently outperforms a nanoGPT baseline on TinyStories and OpenWebText, demonstrating that optimization-theoretic insights can translate into practical gains.

Title: How well do generative models solve inverse problems? A benchmark study

• Authors: Patrick Krüger, Patrick Materne, Werner Krebs, Hanno Gottschalk
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.23238
• Pdf URL: https://arxiv.org/pdf/2601.23238
• Copy Paste: [[2601.23238]] How well do generative models solve inverse problems? A benchmark study(https://arxiv.org/abs/2601.23238)
• Keywords: generative
• Abstract: Generative learning generates high dimensional data based on low dimensional conditions, also called prompts. Therefore, generative learning algorithms are eligible for solving (Bayesian) inverse problems. In this article we compare a traditional Bayesian inverse approach based on a forward regression model and a prior sampled with the Markov Chain Monte Carlo method with three state of the art generative learning models, namely conditional Generative Adversarial Networks, Invertible Neural Networks and Conditional Flow Matching. We apply them to a problem of gas turbine combustor design where we map six independent design parameters to three performance labels. We propose several metrics for the evaluation of this inverse design approaches and measure the accuracy of the labels of the generated designs along with the diversity. We also study the performance as a function of the training dataset size. Our benchmark has a clear winner, as Conditional Flow Matching consistently outperforms all competing approaches.

Title: Structured Over Scale: Learning Spatial Reasoning from Educational Video

• Authors: Bishoy Galoaa, Xiangyu Bai, Sarah Ostadabbas
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.23251
• Pdf URL: https://arxiv.org/pdf/2601.23251
• Copy Paste: [[2601.23251]] Structured Over Scale: Learning Spatial Reasoning from Educational Video(https://arxiv.org/abs/2601.23251)
• Keywords: robust
• Abstract: Vision-language models (VLMs) demonstrate impressive performance on standard video understanding benchmarks yet fail systematically on simple reasoning tasks that preschool children can solve, including counting, spatial reasoning, and compositional understanding. We hypothesize that the pedagogically-structured content of educational videos provides an ideal training signal for improving these capabilities. We introduce DoraVQA, a dataset of 5,344 question-answer pairs automatically extracted from 8 seasons of Dora the Explorer with precise timestamp alignment. Each episode follows a consistent \textit{context-question-pause-answer} structure that creates a self-contained learning environment analogous to interactive tutoring. We fine-tune both Qwen2 and Qwen3 using Group Relative Policy Optimization (GRPO), leveraging the clear correctness signals and structured reasoning traces inherent in educational content. Despite training exclusively on 38 hours of children's educational videos, our approach achieves improvements of 8-14 points on DoraVQA and state-of-the-art 86.16\% on CVBench, with strong transfer to Video-MME and NExT-QA, demonstrating effective generalization from narrow pedagogical content to broad multimodal understanding. Through cross-domain benchmarks, we show that VLMs can perform tasks that require robust reasoning learned from structured educational content, suggesting that content structure matters as much as content scale.

Title: Now You Hear Me: Audio Narrative Attacks Against Large Audio-Language Models

• Authors: Ye Yu, Haibo Jin, Yaoning Yu, Jun Zhuang, Haohan Wang
• Subjects: cs.CL, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2601.23255
• Pdf URL: https://arxiv.org/pdf/2601.23255
• Copy Paste: [[2601.23255]] Now You Hear Me: Audio Narrative Attacks Against Large Audio-Language Models(https://arxiv.org/abs/2601.23255)
• Keywords: security, attack
• Abstract: Large audio-language models increasingly operate on raw speech inputs, enabling more seamless integration across domains such as voice assistants, education, and clinical triage. This transition, however, introduces a distinct class of vulnerabilities that remain largely uncharacterized. We examine the security implications of this modality shift by designing a text-to-audio jailbreak that embeds disallowed directives within a narrative-style audio stream. The attack leverages an advanced instruction-following text-to-speech (TTS) model to exploit structural and acoustic properties, thereby circumventing safety mechanisms primarily calibrated for text. When delivered through synthetic speech, the narrative format elicits restricted outputs from state-of-the-art models, including Gemini 2.0 Flash, achieving a 98.26% success rate that substantially exceeds text-only baselines. These results highlight the need for safety frameworks that jointly reason over linguistic and paralinguistic representations, particularly as speech-based interfaces become more prevalent.

Title: TEON: Tensorized Orthonormalization Beyond Layer-Wise Muon for Large Language Model Pre-Training

• Authors: Ruijie Zhang, Yequan Zhao, Ziyue Liu, Zhengyang Wang, Dongyang Li, Yupeng Su, Sijia Liu, Zheng Zhang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.23261
• Pdf URL: https://arxiv.org/pdf/2601.23261
• Copy Paste: [[2601.23261]] TEON: Tensorized Orthonormalization Beyond Layer-Wise Muon for Large Language Model Pre-Training(https://arxiv.org/abs/2601.23261)
• Keywords: robust, large language model
• Abstract: The Muon optimizer has demonstrated strong empirical performance in pre-training large language models by performing matrix-level gradient (or momentum) orthogonalization in each layer independently. In this work, we propose TEON, a principled generalization of Muon that extends orthogonalization beyond individual layers by modeling the gradients of a neural network as a structured higher-order tensor. We present TEON's improved convergence guarantee over layer-wise Muon, and further develop a practical instantiation of TEON based on the theoretical analysis with corresponding ablation. We evaluate our approach on two widely adopted architectures: GPT-style models, ranging from 130M to 774M parameters, and LLaMA-style models, ranging from 60M to 1B parameters. Experimental results show that TEON consistently improves training and validation perplexity across model scales and exhibits strong robustness under various approximate SVD schemes.

Title: Particle-Guided Diffusion Models for Partial Differential Equations

• Authors: Andrew Millard, Fredrik Lindsten, Zheng Zhao
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.23262
• Pdf URL: https://arxiv.org/pdf/2601.23262
• Copy Paste: [[2601.23262]] Particle-Guided Diffusion Models for Partial Differential Equations(https://arxiv.org/abs/2601.23262)
• Keywords: diffusion, generative
• Abstract: We introduce a guided stochastic sampling method that augments sampling from diffusion models with physics-based guidance derived from partial differential equation (PDE) residuals and observational constraints, ensuring generated samples remain physically admissible. We embed this sampling procedure within a new Sequential Monte Carlo (SMC) framework, yielding a scalable generative PDE solver. Across multiple benchmark PDE systems as well as multiphysics and interacting PDE systems, our method produces solution fields with lower numerical error than existing state-of-the-art generative methods.

Title: UPA: Unsupervised Prompt Agent via Tree-Based Search and Selection

• Authors: Siran Peng, Weisong Zhao, Tianyu Fu, Chenxu Zhao, Tianshuo Zhang, Haoyuan Zhang, Xiangyu Zhu, Minghui Wu, Zhen Lei
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.23273
• Pdf URL: https://arxiv.org/pdf/2601.23273
• Copy Paste: [[2601.23273]] UPA: Unsupervised Prompt Agent via Tree-Based Search and Selection(https://arxiv.org/abs/2601.23273)
• Keywords: large language model
• Abstract: Prompt agents have recently emerged as a promising paradigm for automated prompt optimization, framing refinement as a sequential decision-making problem over a structured prompt space. While this formulation enables the use of advanced planning algorithms, these methods typically assume access to supervised reward signals, which are often unavailable in practical scenarios. In this work, we propose UPA, an Unsupervised Prompt Agent that realizes structured search and selection without relying on supervised feedback. Specifically, during search, UPA iteratively constructs an evolving tree structure to navigate the prompt space, guided by fine-grained and order-invariant pairwise comparisons from Large Language Models (LLMs). Crucially, as these local comparisons do not inherently yield a consistent global scale, we decouple systematic prompt exploration from final selection, introducing a two-stage framework grounded in the Bradley-Terry-Luce (BTL) model. This framework first performs path-wise Bayesian aggregation of local comparisons to filter candidates under uncertainty, followed by global tournament-style comparisons to infer latent prompt quality and identify the optimal prompt. Experiments across multiple tasks demonstrate that UPA consistently outperforms existing prompt optimization methods, showing that agent-style optimization remains highly effective even in fully unsupervised settings.

Title: FOCUS: DLLMs Know How to Tame Their Compute Bound

• Authors: Kaihua Liang, Xin Tan, An Zhong, Hong Xu, Marco Canini
• Subjects: cs.LG, cs.AR, cs.CL
• Abstract URL: https://arxiv.org/abs/2601.23278
• Pdf URL: https://arxiv.org/pdf/2601.23278
• Copy Paste: [[2601.23278]] FOCUS: DLLMs Know How to Tame Their Compute Bound(https://arxiv.org/abs/2601.23278)
• Keywords: diffusion, large language model
• Abstract: Diffusion Large Language Models (DLLMs) offer a compelling alternative to Auto-Regressive models, but their deployment is constrained by high decoding cost. In this work, we identify a key inefficiency in DLLM decoding: while computation is parallelized over token blocks, only a small subset of tokens is decodable at each diffusion step, causing most compute to be wasted on non-decodable tokens. We further observe a strong correlation between attention-derived token importance and token-wise decoding probability. Based on this insight, we propose FOCUS -- an inference system designed for DLLMs. By dynamically focusing computation on decodable tokens and evicting non-decodable ones on-the-fly, FOCUS increases the effective batch size, alleviating compute limitations and enabling scalable throughput. Empirical evaluations demonstrate that FOCUS achieves up to 3.52$\times$ throughput improvement over the production-grade engine LMDeploy, while preserving or improving generation quality across multiple benchmarks. The FOCUS system is publicly available on GitHub: this https URL.

Title: Decoupled Diffusion Sampling for Inverse Problems on Function Spaces

• Authors: Thomas Y.L. Lin, Jiachen Yao, Lufang Chiang, Julius Berner, Anima Anandkumar
• Subjects: cs.LG, math.NA
• Abstract URL: https://arxiv.org/abs/2601.23280
• Pdf URL: https://arxiv.org/pdf/2601.23280
• Copy Paste: [[2601.23280]] Decoupled Diffusion Sampling for Inverse Problems on Function Spaces(https://arxiv.org/abs/2601.23280)
• Keywords: diffusion, generative
• Abstract: We propose a data-efficient, physics-aware generative framework in function space for inverse PDE problems. Existing plug-and-play diffusion posterior samplers represent physics implicitly through joint coefficient-solution modeling, requiring substantial paired supervision. In contrast, our Decoupled Diffusion Inverse Solver (DDIS) employs a decoupled design: an unconditional diffusion learns the coefficient prior, while a neural operator explicitly models the forward PDE for guidance. This decoupling enables superior data efficiency and effective physics-informed learning, while naturally supporting Decoupled Annealing Posterior Sampling (DAPS) to avoid over-smoothing in Diffusion Posterior Sampling (DPS). Theoretically, we prove that DDIS avoids the guidance attenuation failure of joint models when training data is scarce. Empirically, DDIS achieves state-of-the-art performance under sparse observation, improving $l_2$ error by 11% and spectral error by 54% on average; when data is limited to 1%, DDIS maintains accuracy with 40% advantage in $l_2$ error compared to joint models.

Title: User Prompting Strategies and Prompt Enhancement Methods for Open-Set Object Detection in XR Environments

• Authors: Junfeng Lin, Yanming Xiu, Maria Gorlatova
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.23281
• Pdf URL: https://arxiv.org/pdf/2601.23281
• Copy Paste: [[2601.23281]] User Prompting Strategies and Prompt Enhancement Methods for Open-Set Object Detection in XR Environments(https://arxiv.org/abs/2601.23281)
• Keywords: robust
• Abstract: Open-set object detection (OSOD) localizes objects while identifying and rejecting unknown classes at inference. While recent OSOD models perform well on benchmarks, their behavior under realistic user prompting remains underexplored. In interactive XR settings, user-generated prompts are often ambiguous, underspecified, or overly detailed. To study prompt-conditioned robustness, we evaluate two OSOD models, GroundingDINO and YOLO-E, on real-world XR images and simulate diverse user prompting behaviors using vision-language models. We consider four prompt types: standard, underdetailed, overdetailed, and pragmatically ambiguous, and examine the impact of two enhancement strategies on these prompts. Results show that both models exhibit stable performance under underdetailed and standard prompts, while they suffer degradation under ambiguous prompts. Overdetailed prompts primarily affect GroundingDINO. Prompt enhancement substantially improves robustness under ambiguity, yielding gains exceeding 55% mIoU and 41% average confidence. Based on the findings, we propose several prompting strategies and prompt enhancement methods for OSOD models in XR environments.

Title: VideoGPA: Distilling Geometry Priors for 3D-Consistent Video Generation

• Authors: Hongyang Du, Junjie Ye, Xiaoyan Cong, Runhao Li, Jingcheng Ni, Aman Agarwal, Zeqi Zhou, Zekun Li, Randall Balestriero, Yue Wang
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.23286
• Pdf URL: https://arxiv.org/pdf/2601.23286
• Copy Paste: [[2601.23286]] VideoGPA: Distilling Geometry Priors for 3D-Consistent Video Generation(https://arxiv.org/abs/2601.23286)
• Keywords: diffusion, generative
• Abstract: While recent video diffusion models (VDMs) produce visually impressive results, they fundamentally struggle to maintain 3D structural consistency, often resulting in object deformation or spatial drift. We hypothesize that these failures arise because standard denoising objectives lack explicit incentives for geometric coherence. To address this, we introduce VideoGPA (Video Geometric Preference Alignment), a data-efficient self-supervised framework that leverages a geometry foundation model to automatically derive dense preference signals that guide VDMs via Direct Preference Optimization (DPO). This approach effectively steers the generative distribution toward inherent 3D consistency without requiring human annotations. VideoGPA significantly enhances temporal stability, physical plausibility, and motion coherence using minimal preference pairs, consistently outperforming state-of-the-art baselines in extensive experiments.