2026-01-30

Title: Rethinking LLM-Driven Heuristic Design: Generating Efficient and Specialized Solvers via Dynamics-Aware Optimization

• Authors: Rongzheng Wang, Yihong Huang, Muquan Li, Jiakai Li, Di Liang, Bob Simons, Pei Ke, Shuang Liang, Ke Qin
• Subjects: cs.LG, cs.AI, cs.NE
• Abstract URL: https://arxiv.org/abs/2601.20868
• Pdf URL: https://arxiv.org/pdf/2601.20868
• Copy Paste: [[2601.20868]] Rethinking LLM-Driven Heuristic Design: Generating Efficient and Specialized Solvers via Dynamics-Aware Optimization(https://arxiv.org/abs/2601.20868)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have advanced the field of Combinatorial Optimization through automated heuristic generation. Instead of relying on manual design, this LLM-Driven Heuristic Design (LHD) process leverages LLMs to iteratively generate and refine solvers to achieve high performance. However, existing LHD frameworks face two critical limitations: (1) Endpoint-only evaluation, which ranks solvers solely by final quality, ignoring the convergence process and runtime efficiency; (2) High adaptation costs, where distribution shifts necessitate re-adaptation to generate specialized solvers for new instance groups. To address these issues, we propose Dynamics-Aware Solver Heuristics (DASH), a framework that co-optimizes solver search mechanisms and runtime schedules guided by a convergence-aware metric, thereby identifying efficient and high-performance solvers. Furthermore, to mitigate expensive re-adaptation, DASH incorporates Profiled Library Retrieval (PLR). PLR efficiently archives specialized solvers concurrently with the evolutionary process, enabling cost-effective warm-starts for heterogeneous distributions. Experiments on four combinatorial optimization problems demonstrate that DASH improves runtime efficiency by over 3 times, while surpassing the solution quality of state-of-the-art baselines across diverse problem scales. Furthermore, by enabling profile-based warm starts, DASH maintains superior accuracy under different distributions while cutting LLM adaptation costs by over 90%.

Title: MA-LipNet: Multi-Dimensional Attention Networks for Robust Lipreading

• Authors: Matteo Rossi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.20881
• Pdf URL: https://arxiv.org/pdf/2601.20881
• Copy Paste: [[2601.20881]] MA-LipNet: Multi-Dimensional Attention Networks for Robust Lipreading(https://arxiv.org/abs/2601.20881)
• Keywords: security, robust
• Abstract: Lipreading, the technology of decoding spoken content from silent videos of lip movements, holds significant application value in fields such as public security. However, due to the subtle nature of articulatory gestures, existing lipreading methods often suffer from limited feature discriminability and poor generalization capabilities. To address these challenges, this paper delves into the purification of visual features from temporal, spatial, and channel dimensions. We propose a novel method named Multi-Attention Lipreading Network(MA-LipNet). The core of MA-LipNet lies in its sequential application of three dedicated attention modules. Firstly, a \textit{Channel Attention (CA)} module is employed to adaptively recalibrate channel-wise features, thereby mitigating interference from less informative channels. Subsequently, two spatio-temporal attention modules with distinct granularities-\textit{Joint Spatial-Temporal Attention (JSTA)} and \textit{Separate Spatial-Temporal Attention (SSTA)}-are leveraged to suppress the influence of irrelevant pixels and video frames. The JSTA module performs a coarse-grained filtering by computing a unified weight map across the spatio-temporal dimensions, while the SSTA module conducts a more fine-grained refinement by separately modeling temporal and spatial attentions. Extensive experiments conducted on the CMLR and GRID datasets demonstrate that MA-LipNet significantly reduces the Character Error Rate (CER) and Word Error Rate (WER), validating its effectiveness and superiority over several state-of-the-art methods. Our work highlights the importance of multi-dimensional feature refinement for robust visual speech recognition.

Title: What Hard Tokens Reveal: Exploiting Low-confidence Tokens for Membership Inference Attacks against Large Language Models

• Authors: Md Tasnim Jawad, Mingyan Xiao, Yanzhao Wu
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2601.20885
• Pdf URL: https://arxiv.org/pdf/2601.20885
• Copy Paste: [[2601.20885]] What Hard Tokens Reveal: Exploiting Low-confidence Tokens for Membership Inference Attacks against Large Language Models(https://arxiv.org/abs/2601.20885)
• Keywords: privacy, protect, defense, attack, robust, membership infer, large language model
• Abstract: With the widespread adoption of Large Language Models (LLMs) and increasingly stringent privacy regulations, protecting data privacy in LLMs has become essential, especially for privacy-sensitive applications. Membership Inference Attacks (MIAs) attempt to determine whether a specific data sample was included in the model training/fine-tuning dataset, posing serious privacy risks. However, most existing MIA techniques against LLMs rely on sequence-level aggregated prediction statistics, which fail to distinguish prediction improvements caused by generalization from those caused by memorization, leading to low attack effectiveness. To address this limitation, we propose a novel membership inference approach that captures the token-level probabilities for low-confidence (hard) tokens, where membership signals are more pronounced. By comparing token-level probability improvements at hard tokens between a fine-tuned target model and a pre-trained reference model, HT-MIA isolates strong and robust membership signals that are obscured by prior MIA approaches. Extensive experiments on both domain-specific medical datasets and general-purpose benchmarks demonstrate that HT-MIA consistently outperforms seven state-of-the-art MIA baselines. We further investigate differentially private training as an effective defense mechanism against MIAs in LLMs. Overall, our HT-MIA framework establishes hard-token based analysis as a state-of-the-art foundation for advancing membership inference attacks and defenses for LLMs.

Title: A generative machine learning model for designing metal hydrides applied to hydrogen storage

• Authors: Xiyuan Liu, Christian Hacker, Shengnian Wang, Yuhua Duan
• Subjects: cs.LG, cond-mat.mtrl-sci, stat.AP
• Abstract URL: https://arxiv.org/abs/2601.20892
• Pdf URL: https://arxiv.org/pdf/2601.20892
• Copy Paste: [[2601.20892]] A generative machine learning model for designing metal hydrides applied to hydrogen storage(https://arxiv.org/abs/2601.20892)
• Keywords: generative
• Abstract: Developing new metal hydrides is a critical step toward efficient hydrogen storage in carbon-neutral energy systems. However, existing materials databases, such as the Materials Project, contain a limited number of well-characterized hydrides, which constrains the discovery of optimal candidates. This work presents a framework that integrates causal discovery with a lightweight generative machine learning model to generate novel metal hydride candidates that may not exist in current databases. Using a dataset of 450 samples (270 training, 90 validation, and 90 testing), the model generates 1,000 candidates. After ranking and filtering, six previously unreported chemical formulas and crystal structures are identified, four of which are validated by density functional theory simulations and show strong potential for future experimental investigation. Overall, the proposed framework provides a scalable and time-efficient approach for expanding hydrogen storage datasets and accelerating materials discovery.

Title: Is Parameter Isolation Better for Prompt-Based Continual Learning?

• Authors: Jiangyang Li, Chenhao Ding, Songlin Dong, Qiang Wang, Jianchao Zhao, Yuhang He, Yihong Gong
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.20894
• Pdf URL: https://arxiv.org/pdf/2601.20894
• Copy Paste: [[2601.20894]] Is Parameter Isolation Better for Prompt-Based Continual Learning?(https://arxiv.org/abs/2601.20894)
• Keywords: protect
• Abstract: Prompt-based continual learning methods effectively mitigate catastrophic forgetting. However, most existing methods assign a fixed set of prompts to each task, completely isolating knowledge across tasks and resulting in suboptimal parameter utilization. To address this, we consider the practical needs of continual learning and propose a prompt-sharing framework. This framework constructs a global prompt pool and introduces a task-aware gated routing mechanism that sparsely activates a subset of prompts to achieve dynamic decoupling and collaborative optimization of task-specific feature representations. Furthermore, we introduce a history-aware modulator that leverages cumulative prompt activation statistics to protect frequently used prompts from excessive updates, thereby mitigating inefficient parameter usage and knowledge forgetting. Extensive analysis and empirical results demonstrate that our approach consistently outperforms existing static allocation strategies in effectiveness and efficiency.

Title: Towards Sensitivity-Aware Language Models

• Authors: Dren Fazlija, Iyiola E. Olatunji, Daniel Kudenko, Sandipan Sikdar
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2601.20901
• Pdf URL: https://arxiv.org/pdf/2601.20901
• Copy Paste: [[2601.20901]] Towards Sensitivity-Aware Language Models(https://arxiv.org/abs/2601.20901)
• Keywords: privacy
• Abstract: With LLMs increasingly deployed in corporate data management, it is crucial to ensure that these models do not leak sensitive information. In the context of corporate data management, the concept of sensitivity awareness has been introduced, enabling LLMs to adhere to predefined access rights rules. However, it remains unclear how sensitivity awareness relates to established notions of privacy, such as differential privacy (DP), thereby making it difficult to deploy meaningfully in real-world applications. In this work, we formalize the notion of sensitivity awareness and theoretically establish its connection to DP. Additionally, we develop a supervised fine-tuning recipe to make existing, four-bit quantized LLMs more sensitivity-aware. With a performance boost of up to 21.7%, the finetuned LLMs not only substantially improve over their baseline but also outperform other full-precision open-source and commercial models of similar size in achieving sensitivity awareness, demonstrating the effectiveness of our proposed approach. At the same time, our method also largely preserves the models' performance on other tasks, such as general instruction-following, mathematical, and common-sense reasoning.

Title: ICON: Intent-Context Coupling for Efficient Multi-Turn Jailbreak Attack

• Authors: Xingwei Lin, Wenhao Lin, Sicong Cao, Jiahao Yu, Renke Huang, Lei Xue, Chunming Wu
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.20903
• Pdf URL: https://arxiv.org/pdf/2601.20903
• Copy Paste: [[2601.20903]] ICON: Intent-Context Coupling for Efficient Multi-Turn Jailbreak Attack(https://arxiv.org/abs/2601.20903)
• Keywords: attack, large language model
• Abstract: Multi-turn jailbreak attacks have emerged as a critical threat to Large Language Models (LLMs), bypassing safety mechanisms by progressively constructing adversarial contexts from scratch and incrementally refining prompts. However, existing methods suffer from the inefficiency of incremental context construction that requires step-by-step LLM interaction, and often stagnate in suboptimal regions due to surface-level optimization. In this paper, we characterize the Intent-Context Coupling phenomenon, revealing that LLM safety constraints are significantly relaxed when a malicious intent is coupled with a semantically congruent context pattern. Driven by this insight, we propose ICON, an automated multi-turn jailbreak framework that efficiently constructs an authoritative-style context via prior-guided semantic routing. Specifically, ICON first routes the malicious intent to a congruent context pattern (e.g., Scientific Research) and instantiates it into an attack prompt sequence. This sequence progressively builds the authoritative-style context and ultimately elicits prohibited content. In addition, ICON incorporates a Hierarchical Optimization Strategy that combines local prompt refinement with global context switching, preventing the attack from stagnating in ineffective contexts. Experimental results across eight SOTA LLMs demonstrate the effectiveness of ICON, achieving a state-of-the-art average Attack Success Rate (ASR) of 97.1\%. Code is available at this https URL.

Title: TwinWeaver: An LLM-Based Foundation Model Framework for Pan-Cancer Digital Twins

• Authors: Nikita Makarov, Maria Bordukova, Lena Voith von Voithenberg, Estrella Pivel-Villanueva, Sabrina Mielke, Jonathan Wickes, Hanchen Wang, Mingyu Derek Ma, Keunwoo Choi, Kyunghyun Cho, Stephen Ra, Raul Rodriguez-Esteban, Fabian Schmich, Michael Menden
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.20906
• Pdf URL: https://arxiv.org/pdf/2601.20906
• Copy Paste: [[2601.20906]] TwinWeaver: An LLM-Based Foundation Model Framework for Pan-Cancer Digital Twins(https://arxiv.org/abs/2601.20906)
• Keywords: large language model
• Abstract: Precision oncology requires forecasting clinical events and trajectories, yet modeling sparse, multi-modal clinical time series remains a critical challenge. We introduce TwinWeaver, an open-source framework that serializes longitudinal patient histories into text, enabling unified event prediction as well as forecasting with large language models, and use it to build Genie Digital Twin (GDT) on 93,054 patients across 20 cancer types. In benchmarks, GDT significantly reduces forecasting error, achieving a median Mean Absolute Scaled Error (MASE) of 0.87 compared to 0.97 for the strongest time-series baseline (p<0.001). Furthermore, GDT improves risk stratification, achieving an average concordance index (C-index) of 0.703 across survival, progression, and therapy switching tasks, surpassing the best baseline of 0.662. GDT also generalizes to out-of-distribution clinical trials, matching trained baselines at zero-shot and surpassing them with fine-tuning, achieving a median MASE of 0.75-0.88 and outperforming the strongest baseline in event prediction with an average C-index of 0.672 versus 0.648. Finally, TwinWeaver enables an interpretable clinical reasoning extension, providing a scalable and transparent foundation for longitudinal clinical modeling.

Title: Non-Markov Multi-Round Conversational Image Generation with History-Conditioned MLLMs

• Authors: Haochen Zhang, Animesh Sinha, Felix Juefei-Xu, Haoyu Ma, Kunpeng Li, Zhipeng Fan, Meng Dong, Xiaoliang Dai, Tingbo Hou, Peizhao Zhang, Zecheng He
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.20911
• Pdf URL: https://arxiv.org/pdf/2601.20911
• Copy Paste: [[2601.20911]] Non-Markov Multi-Round Conversational Image Generation with History-Conditioned MLLMs(https://arxiv.org/abs/2601.20911)
• Keywords: large language model
• Abstract: Conversational image generation requires a model to follow user instructions across multiple rounds of interaction, grounded in interleaved text and images that accumulate as chat history. While recent multimodal large language models (MLLMs) can generate and edit images, most existing multi-turn benchmarks and training recipes are effectively Markov: the next output depends primarily on the most recent image, enabling shortcut solutions that ignore long-range history. In this work we formalize and target the more challenging non-Markov setting, where a user may refer back to earlier states, undo changes, or reference entities introduced several rounds ago. We present (i) non-Markov multi-round data construction strategies, including rollback-style editing that forces retrieval of earlier visual states and name-based multi-round personalization that binds names to appearances across rounds; (ii) a history-conditioned training and inference framework with token-level caching to prevent multi-round identity drift; and (iii) enabling improvements for high-fidelity image reconstruction and editable personalization, including a reconstruction-based DiT detokenizer and a multi-stage fine-tuning curriculum. We demonstrate that explicitly training for non-Markov interactions yields substantial improvements in multi-round consistency and instruction compliance, while maintaining strong single-round editing and personalization.

Title: Noisy but Valid: Robust Statistical Evaluation of LLMs with Imperfect Judges

• Authors: Chen Feng, Minghe Shen, Ananth Balashankar, Carsten Gerner-Beuerle, Miguel R. D. Rodrigues
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2601.20913
• Pdf URL: https://arxiv.org/pdf/2601.20913
• Copy Paste: [[2601.20913]] Noisy but Valid: Robust Statistical Evaluation of LLMs with Imperfect Judges(https://arxiv.org/abs/2601.20913)
• Keywords: robust, large language model
• Abstract: Reliable certification of Large Language Models (LLMs)-verifying that failure rates are below a safety threshold-is critical yet challenging. While "LLM-as-a-Judge" offers scalability, judge imperfections, noise, and bias can invalidate statistical guarantees. We introduce a "Noisy but Valid" hypothesis testing framework to address this. By leveraging a small human-labelled calibration set to estimate the judge's True Positive and False Positive Rates (TPR/FPR), we derive a variance-corrected critical threshold applied to a large judge-labelled dataset. Crucially, our framework theoretically guarantees finite-sample Type-I error control (validity) despite calibration uncertainty. This distinguishes our work from Prediction-Powered Inference (PPI), positioning our method as a diagnostic tool that explicitly models judge behavior rather than a black-box estimator. Our contributions include: (1) Theoretical Guarantees: We derive the exact conditions under which noisy testing yields higher statistical power than direct evaluation; (2) Empirical Validation: Experiments on Jigsaw Comment, Hate Speech and SafeRLHF confirm our theory; (3) The Oracle Gap: We reveal a significant performance gap between practical methods and the theoretical "Oracle" (perfectly known judge parameters), quantifying the cost of estimation. Specifically, we provide the first systematic treatment of the imperfect-judge setting, yielding interpretable diagnostics of judge reliability and clarifying how evaluation power depends on judge quality, dataset size, and certification levels. Together, these results sharpen understanding of statistical evaluation with LLM judges, and highlight trade-offs among competing inferential tools.

Title: Robust Federated Learning for Malicious Clients using Loss Trend Deviation Detection

• Authors: Deepthy K Bhaskar, Minimol B, Binu V P
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2601.20915
• Pdf URL: https://arxiv.org/pdf/2601.20915
• Copy Paste: [[2601.20915]] Robust Federated Learning for Malicious Clients using Loss Trend Deviation Detection(https://arxiv.org/abs/2601.20915)
• Keywords: secure, privacy, defense, attack, robust, federate
• Abstract: Federated Learning (FL) facilitates collaborative model training among distributed clients while ensuring that raw data remains on local this http URL this advantage, FL systems are still exposed to risks from malicious or unreliable participants. Such clients can interfere with the training process by sending misleading updates, which can negatively affect the performance and reliability of the global model. Many existing defense mechanisms rely on gradient inspection, complex similarity computations, or cryptographic operations, which introduce additional overhead and may become unstable under non-IID data distributions. In this paper, we propose the Federated Learning with Loss Trend Detection (FL-LTD), a lightweight and privacy-preserving defense framework that detects and mitigates malicious behavior by monitoring temporal loss dynamics rather than model gradients. The proposed approach identifies anomalous clients by detecting abnormal loss stagnation or abrupt loss fluctuations across communication rounds. To counter adaptive attackers, a short-term memory mechanism is incorporated to sustain mitigation for clients previously flagged as anomalous, while enabling trust recovery for stable participants. We evaluate FL-LTD on a non-IID federated MNIST setup under loss manipulation attacks. Experimental results demonstrate that the proposed method significantly enhances robustness, achieving a final test accuracy of 0.84, compared to 0.41 for standard FedAvg under attack. FL-LTD incurs negligible computational and communication overhead, maintains stable convergence, and avoids client exclusion or access to sensitive data, highlighting the effectiveness of loss-based monitoring for secure federated learning.

Title: FIPS 204-Compatible Threshold ML-DSA via Masked Lagrange Reconstruction

• Authors: Leo Kao
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2601.20917
• Pdf URL: https://arxiv.org/pdf/2601.20917
• Copy Paste: [[2601.20917]] FIPS 204-Compatible Threshold ML-DSA via Masked Lagrange Reconstruction(https://arxiv.org/abs/2601.20917)
• Keywords: security, protect
• Abstract: We present masked Lagrange reconstruction, a technique that enables threshold ML-DSA (FIPS 204) with arbitrary thresholds $T$ while producing standard 3.3 KB signatures verifiable by unmodified FIPS 204 implementations. Concurrent approaches have limitations: Bienstock et al. (ePrint 2025/1163) achieve arbitrary $T$ but require honest-majority and 37--136 rounds; Celi et al. (ePrint 2026/013) achieve dishonest-majority but are limited to $T \leq 6$. Our technique addresses the barrier that Lagrange coefficients grow as $\Theta(q)$ for moderate $T$, making individual contributions too large for ML-DSA's rejection sampling. Unlike ECDSA threshold schemes where pairwise masks suffice for correctness, ML-DSA requires solving three additional challenges absent in prior work: (1) rejection sampling on $\|z\|_\infty$ must still pass after masking, (2) the $r_0$-check exposes $c s_2$ enabling key recovery if unprotected, and (3) the resulting Irwin-Hall nonce distribution must preserve EUF-CMA security. We solve all three. We instantiate this technique in three deployment profiles with full security proofs. Profile P1 (TEE-assisted) achieves 3-round signing with a trusted coordinator, with EUF-CMA security under Module-SIS. Profile P2 (fully distributed) eliminates hardware trust via MPC in 8 rounds, achieving UC security against malicious adversaries corrupting up to $n-1$ parties. Profile P3 (2PC-assisted) uses lightweight 2PC for the $r_0$-check in 3--5 rounds, achieving UC security under a 1-of-2 CP honest assumption with the best empirical performance (249ms). Our scheme requires $|S| \geq T+1$ signers and achieves success rates of 23--32\%, matching single-signer ML-DSA.

Title: DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agents

• Authors: Nikita Gupta, Riju Chatterjee, Lukas Haas, Connie Tao, Andrew Wang, Chang Liu, Hidekazu Oiwa, Elena Gribovskaya, Jan Ackermann, John Blitzer, Sasha Goldshtein, Dipanjan Das
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.20975
• Pdf URL: https://arxiv.org/pdf/2601.20975
• Copy Paste: [[2601.20975]] DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agents(https://arxiv.org/abs/2601.20975)
• Keywords: robust
• Abstract: We introduce DeepSearchQA, a 900-prompt benchmark for evaluating agents on difficult multi-step information-seeking tasks across 17 different fields. Unlike traditional benchmarks that target single answer retrieval or broad-spectrum factuality, DeepSearchQA features a dataset of challenging, handcrafted tasks designed to evaluate an agent's ability to execute complex search plans to generate exhaustive answer lists. This shift in design explicitly tests three critical, yet under-evaluated capabilities: 1) systematic collation of fragmented information from disparate sources, 2) de-duplication and entity resolution to ensure precision, and 3) the ability to reason about stopping criteria within an open-ended search space. Each task is structured as a causal chain, where discovering information for one step is dependent on the successful completion of the previous one, stressing long-horizon planning and context retention. All tasks are grounded in the open web with objectively verifiable answer sets. Our comprehensive evaluation of state-of-the-art agent architectures reveals significant performance limitations: even the most advanced models struggle to balance high recall with precision. We observe distinct failure modes ranging from premature stopping (under-retrieval) to hedging behaviors, where agents cast an overly wide net of low-confidence answers to artificially boost recall. These findings highlight critical headroom in current agent designs and position DeepSearchQA as an essential diagnostic tool for driving future research toward more robust, deep-research capabilities.

Title: The Depth Delusion: Why Transformers Should Be Wider, Not Deeper

• Authors: Md Muhtasim Munif Fahim, Md Rezaul Karim
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.20994
• Pdf URL: https://arxiv.org/pdf/2601.20994
• Copy Paste: [[2601.20994]] The Depth Delusion: Why Transformers Should Be Wider, Not Deeper(https://arxiv.org/abs/2601.20994)
• Keywords: transformer
• Abstract: Neural scaling laws describe how language model loss decreases with parameters and data, but treat architecture as interchangeable--a billion parameters could arise from a shallow-wide model (10 layers & 8,192 hidden dimension) or a deep-narrow one (80 layers & 2,048 hidden dimension). We propose architecture-conditioned scaling laws decomposing this dependence, finding that optimal depth scales as D* ~ C^0.12 while optimal width scales as W* ~ C^0.34, meaning width should grow 2.8x faster than depth. We discover a critical depth phenomenon: beyond D_crit ~ W^0.44 (sublinear in W), adding layers increases loss despite adding parameters--the Depth Delusion. Empirically, we validate these findings across 30 transformer architectures spanning 17M to 7B parameters, each trained on representative high-compute samples, achieving R^2 = 0.922. Our central finding: at 7B scale, a 64-layer model (6.38B params) underperforms a 32-layer model (6.86B params) by 0.12 nats, despite being significantly deeper. This demonstrates that optimal depth-width tradeoffs persist at the production scale.

Title: MADE: Benchmark Environments for Closed-Loop Materials Discovery

• Authors: Shreshth A Malik, Tiarnan Doherty, Panagiotis Tigas, Muhammed Razzak, Stephen J. Roberts, Aron Walsh, Yarin Gal
• Subjects: cs.LG, cond-mat.mtrl-sci
• Abstract URL: https://arxiv.org/abs/2601.20996
• Pdf URL: https://arxiv.org/pdf/2601.20996
• Copy Paste: [[2601.20996]] MADE: Benchmark Environments for Closed-Loop Materials Discovery(https://arxiv.org/abs/2601.20996)
• Keywords: generative
• Abstract: Existing benchmarks for computational materials discovery primarily evaluate static predictive tasks or isolated computational sub-tasks. While valuable, these evaluations neglect the inherently iterative and adaptive nature of scientific discovery. We introduce MAterials Discovery Environments (MADE), a novel framework for benchmarking end-to-end autonomous materials discovery pipelines. MADE simulates closed-loop discovery campaigns in which an agent or algorithm proposes, evaluates, and refines candidate materials under a constrained oracle budget, capturing the sequential and resource-limited nature of real discovery workflows. We formalize discovery as a search for thermodynamically stable compounds relative to a given convex hull, and evaluate efficacy and efficiency via comparison to baseline algorithms. The framework is flexible; users can compose discovery agents from interchangeable components such as generative models, filters, and planners, enabling the study of arbitrary workflows ranging from fixed pipelines to fully agentic systems with tool use and adaptive decision making. We demonstrate this by conducting systematic experiments across a family of systems, enabling ablation of components in discovery pipelines, and comparison of how methods scale with system complexity.

Title: What Are Brands Telling You About Smishing? A Cross-Industry Evaluation of Customer Guidance

• Authors: Dev Vikesh Doshi, Mehjabeen Tasnim, Fernando Landeros, Chinthagumpala Muni Venkatesh, Daniel Timko, Muhammad Lutfor Rahman
• Subjects: cs.CR, cs.HC
• Abstract URL: https://arxiv.org/abs/2601.20999
• Pdf URL: https://arxiv.org/pdf/2601.20999
• Copy Paste: [[2601.20999]] What Are Brands Telling You About Smishing? A Cross-Industry Evaluation of Customer Guidance(https://arxiv.org/abs/2601.20999)
• Keywords: protect, attack
• Abstract: Phishing attacks through text, also known as smishing, are a prevalent type of social engineering tactic in which attackers impersonate brands to deceive victims into providing personal information and/or money. While smishing awareness and cyber education are a key method by which organizations communicate this awareness, the guidance itself varies widely. In this paper, we investigate the state of practice of how 149 well-known brands across 25 categories educate their customers about smishing and what smishing prevention and reporting advice they provide. After conducting a comprehensive content analysis of the brands, we identified significant gaps in the smishing-related information provided: only 46\% of the 149 brands mentioned the definition of smishing, less than 1\% had a video tutorial on smishing, and only 50\% of brands provided instructions on how to report. Our study highlights variation in terminology, prevention advice, and reporting mechanisms across industries, with some brands recommending potentially ineffective strategies such as "ignoring suspicious messages." These findings establish a baseline for understanding the current state of industry smishing awareness advice and provide specific areas where standardization improvements are needed. From our evaluation, we provide recommendations for brands on how to offer streamlined education to their respective customers on smishing for better awareness and protection against increasing smishing attacks.

Title: UrduBench: An Urdu Reasoning Benchmark using Contextually Ensembled Translations with Human-in-the-Loop

• Authors: Muhammad Ali Shafique, Areej Mehboob, Layba Fiaz, Muhammad Usman Qadeer, Hamza Farooq
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21000
• Pdf URL: https://arxiv.org/pdf/2601.21000
• Copy Paste: [[2601.21000]] UrduBench: An Urdu Reasoning Benchmark using Contextually Ensembled Translations with Human-in-the-Loop(https://arxiv.org/abs/2601.21000)
• Keywords: robust, large language model
• Abstract: Recent advances in large language models (LLMs) have led to strong reasoning capabilities; however, evaluating such models in low-resource languages remains challenging due to the lack of standardized benchmarks. In particular, Urdu reasoning evaluation has been limited by the sensitivity of machine translation and an emphasis on general language tasks rather than reasoning benchmarks. In this paper, we propose a contextually ensembled translation framework with human-in-the-loop validation that leverages multiple translation systems to develop Urdu reasoning benchmarks while preserving contextual and structural integrity. Using this framework, we translate widely adopted reasoning and question-answering benchmarks, including MGSM, MATH-500, CommonSenseQA, and OpenBookQA, into Urdu, collectively referred to as UrduBench, and conduct a comprehensive evaluation of both reasoning-oriented and instruction-tuned LLMs across multiple prompting strategies. Our analysis reveals performance differences across (1) four datasets, (2) five task difficulty levels, (3) diverse model architectures, (4) multiple model scaling settings, and (5) language consistency tests. We find that multi-step and symbolic reasoning tasks pose significant challenges in Urdu, and that stable language alignment is a critical prerequisite for robust reasoning. Overall, our work establishes a scalable methodology for standardized reasoning evaluation in Urdu and provides empirical insights into multilingual reasoning failures. This experimental setup is also broadly applicable to other low-resource languages. The code and datasets will be publicly released.

Title: Order-Aware Test-Time Adaptation: Leveraging Temporal Dynamics for Robust Streaming Inference

• Authors: Young Kyung Kim, Oded Schlesinger, Qiangqiang Wu, J. Matías Di Martino, Guillermo Sapiro
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21012
• Pdf URL: https://arxiv.org/pdf/2601.21012
• Copy Paste: [[2601.21012]] Order-Aware Test-Time Adaptation: Leveraging Temporal Dynamics for Robust Streaming Inference(https://arxiv.org/abs/2601.21012)
• Keywords: robust
• Abstract: Test-Time Adaptation (TTA) enables pre-trained models to adjust to distribution shift by learning from unlabeled test-time streams. However, existing methods typically treat these streams as independent samples, overlooking the supervisory signal inherent in temporal dynamics. To address this, we introduce Order-Aware Test-Time Adaptation (OATTA). We formulate test-time adaptation as a gradient-free recursive Bayesian estimation task, using a learned dynamic transition matrix as a temporal prior to refine the base model's predictions. To ensure safety in weakly structured streams, we introduce a likelihood-ratio gate (LLR) that reverts to the base predictor when temporal evidence is absent. OATTA is a lightweight, model-agnostic module that incurs negligible computational overhead. Extensive experiments across image classification, wearable and physiological signal analysis, and language sentiment analysis demonstrate its universality; OATTA consistently boosts established baselines, improving accuracy by up to 6.35%. Our findings establish that modeling temporal dynamics provides a critical, orthogonal signal beyond standard order-agnostic TTA approaches.

Title: Conditional Denoising Model as a Physical Surrogate Model

• Authors: José Afonso, Pedro Viegas, Rodrigo Ventura, Vasco Guerra
• Subjects: cs.LG, cs.AI, physics.plasm-ph
• Abstract URL: https://arxiv.org/abs/2601.21021
• Pdf URL: https://arxiv.org/pdf/2601.21021
• Copy Paste: [[2601.21021]] Conditional Denoising Model as a Physical Surrogate Model(https://arxiv.org/abs/2601.21021)
• Keywords: generative
• Abstract: Surrogate modeling for complex physical systems typically faces a trade-off between data-fitting accuracy and physical consistency. Physics-consistent approaches typically treat physical laws as soft constraints within the loss function, a strategy that frequently fails to guarantee strict adherence to the governing equations, or rely on post-processing corrections that do not intrinsically learn the underlying solution geometry. To address these limitations, we introduce the {Conditional Denoising Model (CDM)}, a generative model designed to learn the geometry of the physical manifold itself. By training the network to restore clean states from noisy ones, the model learns a vector field that points continuously towards the valid solution subspace. We introduce a time-independent formulation that transforms inference into a deterministic fixed-point iteration, effectively projecting noisy approximations onto the equilibrium manifold. Validated on a low-temperature plasma physics and chemistry benchmark, the CDM achieves higher parameter and data efficiency than physics-consistent baselines. Crucially, we demonstrate that the denoising objective acts as a powerful implicit regularizer: despite never seeing the governing equations during training, the model adheres to physical constraints more strictly than baselines trained with explicit physics losses.

Title: SIGMA-PPG: Statistical-prior Informed Generative Masking Architecture for PPG Foundation Model

• Authors: Zongheng Guo, Tao Chen, Yang Jiao, Yi Pan, Xiao Hu, Manuela Ferrario
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21031
• Pdf URL: https://arxiv.org/pdf/2601.21031
• Copy Paste: [[2601.21031]] SIGMA-PPG: Statistical-prior Informed Generative Masking Architecture for PPG Foundation Model(https://arxiv.org/abs/2601.21031)
• Keywords: generative
• Abstract: Current foundation model for photoplethysmography (PPG) signals is challenged by the intrinsic redundancy and noise of the signal. Standard masked modeling often yields trivial solutions while contrastive methods lack morphological precision. To address these limitations, we propose a Statistical-prior Informed Generative Masking Architecture (SIGMA-PPG), a generative foundation model featuring a Prior-Guided Adversarial Masking mechanism, where a reinforcement learning-driven teacher leverages statistical priors to create challenging learning paths that prevent overfitting to noise. We also incorporate a semantic consistency constraint via vector quantization to ensure that physiologically identical waveforms (even those altered by recording artifacts or minor perturbations) map to shared indices. This enhances codebook semantic density and eliminates redundant feature structures. Pre-trained on over 120,000 hours of data, SIGMA-PPG achieves superior average performance compared to five state-of-the-art baselines across 12 diverse downstream tasks. The code is available at this https URL.

Title: Predict-Project-Renoise: Sampling Diffusion Models under Hard Constraints

• Authors: Omer Rochman-Sharabi, Gilles Louppe
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21033
• Pdf URL: https://arxiv.org/pdf/2601.21033
• Copy Paste: [[2601.21033]] Predict-Project-Renoise: Sampling Diffusion Models under Hard Constraints(https://arxiv.org/abs/2601.21033)
• Keywords: diffusion
• Abstract: Neural emulators based on diffusion models show promise for scientific applications, but vanilla models cannot guarantee physical accuracy or constraint satisfaction. We address this by introducing a constrained sampling framework that enforces hard constraints, such as physical laws or observational consistency, at generation time. Our approach defines a constrained forward process that diffuses only over the feasible set of constraint-satisfying samples, inducing constrained marginal distributions. To reverse this, we propose Predict-Project-Renoise (PPR), an iterative algorithm that samples from the constrained marginals by alternating between denoising predictions, projecting onto the feasible set, and renoising. Experiments on 2D distributions, PDEs, and global weather forecasting demonstrate that PPR reduces constraint violations by over an order of magnitude while improving sample consistency and better matching the true constrained distribution compared to baselines.

Title: Thinking in Frames: How Visual Context and Test-Time Scaling Empower Video Reasoning

• Authors: Chengzu Li, Zanyi Wang, Jiaang Li, Yi Xu, Han Zhou, Huanyu Zhang, Ruichuan An, Dengyang Jiang, Zhaochong An, Ivan Vulić, Serge Belongie, Anna Korhonen
• Subjects: cs.LG, cs.AI, cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21037
• Pdf URL: https://arxiv.org/pdf/2601.21037
• Copy Paste: [[2601.21037]] Thinking in Frames: How Visual Context and Test-Time Scaling Empower Video Reasoning(https://arxiv.org/abs/2601.21037)
• Keywords: robust
• Abstract: Vision-Language Models have excelled at textual reasoning, but they often struggle with fine-grained spatial understanding and continuous action planning, failing to simulate the dynamics required for complex visual reasoning. In this work, we formulate visual reasoning by means of video generation models, positing that generated frames can act as intermediate reasoning steps between initial states and solutions. We evaluate their capacity in two distinct regimes: Maze Navigation for sequential discrete planning with low visual change and Tangram Puzzle for continuous manipulation with high visual change. Our experiments reveal three critical insights: (1) Robust Zero-Shot Generalization: In both tasks, the model demonstrates strong performance on unseen data distributions without specific finetuning. (2) Visual Context: The model effectively uses visual context as explicit control, such as agent icons and tangram shapes, enabling it to maintain high visual consistency and adapt its planning capability robustly to unseen patterns. (3) Visual Test-Time Scaling: We observe a test-time scaling law in sequential planning; increasing the generated video length (visual inference budget) empowers better zero-shot generalization to spatially and temporally complex paths. These findings suggest that video generation is not merely a media tool, but a scalable, generalizable paradigm for visual reasoning.

Title: Test-Time Adaptation for Unsupervised Combinatorial Optimization

• Authors: Yiqiao Liao, Farinaz Koushanfar, Parinaz Naghizadeh
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21048
• Pdf URL: https://arxiv.org/pdf/2601.21048
• Copy Paste: [[2601.21048]] Test-Time Adaptation for Unsupervised Combinatorial Optimization(https://arxiv.org/abs/2601.21048)
• Keywords: robust
• Abstract: Unsupervised neural combinatorial optimization (NCO) enables learning powerful solvers without access to ground-truth solutions. Existing approaches fall into two disjoint paradigms: models trained for generalization across instances, and instance-specific models optimized independently at test time. While the former are efficient during inference, they lack effective instance-wise adaptability; the latter are flexible but fail to exploit learned inductive structure and are prone to poor local optima. This motivates the central question of our work: how can we leverage the inductive bias learned through generalization while unlocking the flexibility required for effective instance-wise adaptation? We first identify a challenge in bridging these two paradigms: generalization-focused models often constitute poor warm starts for instance-wise optimization, potentially underperforming even randomly initialized models when fine-tuned at test time. To resolve this incompatibility, we propose TACO, a model-agnostic test-time adaptation framework that unifies and extends the two existing paradigms for unsupervised NCO. TACO applies strategic warm-starting to partially relax trained parameters while preserving inductive bias, enabling rapid and effective unsupervised adaptation. Crucially, compared to naively fine-tuning a trained generalizable model or optimizing an instance-specific model from scratch, TACO achieves better solution quality while incurring negligible additional computational cost. Experiments on canonical CO problems, Minimum Vertex Cover and Maximum Clique, demonstrate the effectiveness and robustness of TACO across static, distribution-shifted, and dynamic combinatorial optimization problems, establishing it as a practical bridge between generalizable and instance-specific unsupervised NCO.

Title: SMKC: Sketch Based Kernel Correlation Images for Variable Cardinality Time Series Anomaly Detection

• Authors: Haokun Zhou
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21050
• Pdf URL: https://arxiv.org/pdf/2601.21050
• Copy Paste: [[2601.21050]] SMKC: Sketch Based Kernel Correlation Images for Variable Cardinality Time Series Anomaly Detection(https://arxiv.org/abs/2601.21050)
• Keywords: robust
• Abstract: Conventional anomaly detection in multivariate time series relies on the assumption that the set of observed variables remains static. In operational environments, however, monitoring systems frequently experience sensor churn. Signals may appear, disappear, or be renamed, creating data windows where the cardinality varies and may include values unseen during training. To address this challenge, we propose SMKC, a framework that decouples the dynamic input structure from the anomaly detector. We first employ permutation-invariant feature hashing to sketch raw inputs into a fixed size state sequence. We then construct a hybrid kernel image to capture global temporal structure through pairwise comparisons of the sequence and its derivatives. The model learns normal patterns using masked reconstruction and a teacher-student prediction objective. Our evaluation reveals that robust log-distance channels provide the primary discriminative signal, whereas cosine representations often fail to capture sufficient contrast. Notably, we find that a detector using random projections and nearest neighbors on the SMKC representation performs competitively with fully trained baselines without requiring gradient updates. This highlights the effectiveness of the representation itself and offers a practical cold-start solution for resource-constrained deployments.

Title: Human-LLM Collaborative Feature Engineering for Tabular Data

• Authors: Zhuoyan Li, Aditya Bansal, Jinzhao Li, Shishuang He, Zhuoran Lu, Mutian Zhang, Qin Liu, Yiwei Yang, Swati Jain, Ming Yin, Yunyao Li
• Subjects: cs.LG, cs.CL, cs.HC
• Abstract URL: https://arxiv.org/abs/2601.21060
• Pdf URL: https://arxiv.org/pdf/2601.21060
• Copy Paste: [[2601.21060]] Human-LLM Collaborative Feature Engineering for Tabular Data(https://arxiv.org/abs/2601.21060)
• Keywords: large language model
• Abstract: Large language models (LLMs) are increasingly used to automate feature engineering in tabular learning. Given task-specific information, LLMs can propose diverse feature transformation operations to enhance downstream model performance. However, current approaches typically assign the LLM as a black-box optimizer, responsible for both proposing and selecting operations based solely on its internal heuristics, which often lack calibrated estimations of operation utility and consequently lead to repeated exploration of low-yield operations without a principled strategy for prioritizing promising directions. In this paper, we propose a human-LLM collaborative feature engineering framework for tabular learning. We begin by decoupling the transformation operation proposal and selection processes, where LLMs are used solely to generate operation candidates, while the selection is guided by explicitly modeling the utility and uncertainty of each proposed operation. Since accurate utility estimation can be difficult especially in the early rounds of feature engineering, we design a mechanism within the framework that selectively elicits and incorporates human expert preference feedback, comparing which operations are more promising, into the selection process to help identify more effective operations. Our evaluations on both the synthetic study and the real user study demonstrate that the proposed framework improves feature engineering performance across a variety of tabular datasets and reduces users' cognitive load during the feature engineering process.

Title: Signal from Structure: Exploiting Submodular Upper Bounds in Generative Flow Networks

• Authors: Alexandre Larouche, Audrey Durand
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2601.21061
• Pdf URL: https://arxiv.org/pdf/2601.21061
• Copy Paste: [[2601.21061]] Signal from Structure: Exploiting Submodular Upper Bounds in Generative Flow Networks(https://arxiv.org/abs/2601.21061)
• Keywords: generative
• Abstract: Generative Flow Networks (GFlowNets; GFNs) are a class of generative models that learn to sample compositional objects proportionally to their a priori unknown value, their reward. We focus on the case where the reward has a specified, actionable structure, namely that it is submodular. We show submodularity can be harnessed to retrieve upper bounds on the reward of compositional objects that have not yet been observed. We provide in-depth analyses of the probability of such bounds occurring, as well as how many unobserved compositional objects can be covered by a bound. Following the Optimism in the Face of Uncertainty principle, we then introduce SUBo-GFN, which uses the submodular upper bounds to train a GFN. We show that SUBo-GFN generates orders of magnitude more training data than classical GFNs for the same number of queries to the reward function. We demonstrate the effectiveness of SUBo-GFN in terms of distribution matching and high-quality candidate generation on synthetic and real-world submodular tasks.

Title: Textual Equilibrium Propagation for Deep Compound AI Systems

• Authors: Minghui Chen, Wenlong Deng, James Zou, Han Yu, Xiaoxiao Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21064
• Pdf URL: https://arxiv.org/pdf/2601.21064
• Copy Paste: [[2601.21064]] Textual Equilibrium Propagation for Deep Compound AI Systems(https://arxiv.org/abs/2601.21064)
• Keywords: large language model
• Abstract: Large language models (LLMs) are increasingly deployed as part of compound AI systems that coordinate multiple modules (e.g., retrievers, tools, verifiers) over long-horizon workflows. Recent approaches that propagate textual feedback globally (e.g., TextGrad) make it feasible to optimize such pipelines, but we find that performance degrades as system depth grows. In particular, long-horizon agentic workflows exhibit two depth-scaling failure modes: 1) exploding textual gradient, where textual feedback grows exponentially with depth, leading to prohibitively long message and amplifies evaluation biases; and 2) vanishing textual gradient, where limited long-context ability causes models overemphasize partial feedback and compression of lengthy feedback causes downstream messages to lose specificity gradually as they propagate many hops upstream. To mitigate these issues, we introduce Textual Equilibrium Propagation (TEP), a local learning principle inspired by Equilibrium Propagation in energy-based models. TEP includes two phases: 1) a free phase where a local LLM critics iteratively refine prompts until reaching equilibrium (no further improvements are suggested); and 2) a nudged phase which applies proximal prompt edits with bounded modification intensity, using task-level objectives that propagate via forward signaling rather than backward feedback chains. This design supports local prompt optimization followed by controlled adaptation toward global goals without the computational burden and signal degradation of global textual backpropagation. Across long-horizon QA benchmarks and multi-agent tool-use dataset, TEP consistently improves accuracy and efficiency over global propagation methods such as TextGrad. The gains grows with depth, while preserving the practicality of black-box LLM components in deep compound AI system.

Title: BadDet+: Robust Backdoor Attacks for Object Detection

• Authors: Kealan Dunnett, Reza Arablouei, Dimity Miller, Volkan Dedeoglu, Raja Jurdak
• Subjects: cs.CV, cs.CR
• Abstract URL: https://arxiv.org/abs/2601.21066
• Pdf URL: https://arxiv.org/pdf/2601.21066
• Copy Paste: [[2601.21066]] BadDet+: Robust Backdoor Attacks for Object Detection(https://arxiv.org/abs/2601.21066)
• Keywords: defense, attack, robust
• Abstract: Backdoor attacks pose a severe threat to deep learning, yet their impact on object detection remains poorly understood compared to image classification. While attacks have been proposed, we identify critical weaknesses in existing detection-based methods, specifically their reliance on unrealistic assumptions and a lack of physical validation. To bridge this gap, we introduce BadDet+, a penalty-based framework that unifies Region Misclassification Attacks (RMA) and Object Disappearance Attacks (ODA). The core mechanism utilizes a log-barrier penalty to suppress true-class predictions for triggered inputs, resulting in (i) position and scale invariance, and (ii) enhanced physical robustness. On real-world benchmarks, BadDet+ achieves superior synthetic-to-physical transfer compared to existing RMA and ODA baselines while preserving clean performance. Theoretical analysis confirms the proposed penalty acts within a trigger-specific feature subspace, reliably inducing attacks without degrading standard inference. These results highlight significant vulnerabilities in object detection and the necessity for specialized defenses.

Title: Shape of Thought: Progressive Object Assembly via Visual Chain-of-Thought

• Authors: Yu Huo, Siyu Zhang, Kun Zeng, Haoyue Liu, Owen Lee, Junlin Chen, Yuquan Lu, Yifu Guo, Yaodong Liang, Xiaoying Tang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21081
• Pdf URL: https://arxiv.org/pdf/2601.21081
• Copy Paste: [[2601.21081]] Shape of Thought: Progressive Object Assembly via Visual Chain-of-Thought(https://arxiv.org/abs/2601.21081)
• Keywords: generative
• Abstract: Multimodal models for text-to-image generation have achieved strong visual fidelity, yet they remain brittle under compositional structural constraints-notably generative numeracy, attribute binding, and part-level relations. To address these challenges, we propose Shape-of-Thought (SoT), a visual CoT framework that enables progressive shape assembly via coherent 2D projections without external engines at inference time. SoT trains a unified multimodal autoregressive model to generate interleaved textual plans and rendered intermediate states, helping the model capture shape-assembly logic without producing explicit geometric representations. To support this paradigm, we introduce SoT-26K, a large-scale dataset of grounded assembly traces derived from part-based CAD hierarchies, and T2S-CompBench, a benchmark for evaluating structural integrity and trace faithfulness. Fine-tuning on SoT-26K achieves 88.4% on component numeracy and 84.8% on structural topology, outperforming text-only baselines by around 20%. SoT establishes a new paradigm for transparent, process-supervised compositional generation. The code is available at this https URL. The SoT-26K dataset will be released upon acceptance.

Title: LOCUS: Low-Dimensional Model Embeddings for Efficient Model Exploration, Comparison, and Selection

• Authors: Shivam Patel, William Cocke, Gauri Joshi
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21082
• Pdf URL: https://arxiv.org/pdf/2601.21082
• Copy Paste: [[2601.21082]] LOCUS: Low-Dimensional Model Embeddings for Efficient Model Exploration, Comparison, and Selection(https://arxiv.org/abs/2601.21082)
• Keywords: robust, large language model
• Abstract: The rapidly growing ecosystem of Large Language Models (LLMs) makes it increasingly challenging to manage and utilize the vast and dynamic pool of models effectively. We propose LOCUS, a method that produces low-dimensional vector embeddings that compactly represent a language model's capabilities across queries. LOCUS is an attention-based approach that generates embeddings by a deterministic forward pass over query encodings and evaluation scores via an encoder model, enabling seamless incorporation of new models to the pool and refinement of existing model embeddings without having to perform any retraining. We additionally train a correctness predictor that uses model embeddings and query encodings to achieve state-of-the-art routing accuracy on unseen queries. Experiments show that LOCUS needs up to 4.8x fewer query evaluation samples than baselines to produce informative and robust embeddings. Moreover, the learned embedding space is geometrically meaningful: proximity reflects model similarity, enabling a range of downstream applications including model comparison and clustering, model portfolio selection, and resilient proxies of unavailable models.

Title: ChunkWise LoRA: Adaptive Sequence Partitioning for Memory-Efficient Low-Rank Adaptation and Accelerated LLM Inference

• Authors: Ketan Thakkar, Maitreyi Chatterjee, Ramasubramanian Balasubramanian, Achyuthan Jootoo, Rajendra Ugrani
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21109
• Pdf URL: https://arxiv.org/pdf/2601.21109
• Copy Paste: [[2601.21109]] ChunkWise LoRA: Adaptive Sequence Partitioning for Memory-Efficient Low-Rank Adaptation and Accelerated LLM Inference(https://arxiv.org/abs/2601.21109)
• Keywords: transformer, large language model
• Abstract: Recent advances in low-rank adaptation (LoRA) have enabled efficient fine-tuning of large language models (LLMs) with minimal additional parameters. However, existing LoRA methods apply static rank configurations uniformly across all input tokens, ignoring variation in token complexity and computational requirements. In this work, we propose ChunkWise LoRA, a dynamic and adaptive approach that partitions sequences into variable-length chunks based on token complexity and assigns each chunk a tailored low-rank configuration. Our system introduces a runtime scheduler that estimates token difficulty, performs adaptive chunking, and selects per-chunk LoRA rank and scaling using a rank-ladder mechanism. To preserve output consistency, we further introduce a boundary-safe composition module and integrate policy-driven KV-cache strategies. Experiments on benchmark datasets such as Wikitext-103 and SQuAD demonstrate that ChunkWise LoRA achieves up to 34\% lower latency and 38% memory reduction compared to baseline LoRA, while maintaining or improving task performance metrics like BLEU, EM, and perplexity. The proposed framework remains fully compatible with existing transformer architectures and inference frameworks, providing a practical solution for real-world deployment of parameter-efficient LLMs.

Title: Large Language Models Naively Recover Ethnicity from Individual Records

• Authors: Noah Dasanaike
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21132
• Pdf URL: https://arxiv.org/pdf/2601.21132
• Copy Paste: [[2601.21132]] Large Language Models Naively Recover Ethnicity from Individual Records(https://arxiv.org/abs/2601.21132)
• Keywords: transformer, large language model
• Abstract: I demonstrate that large language models can infer ethnicity from names with accuracy exceeding that of Bayesian Improved Surname Geocoding (BISG) without additional training data, enabling inference outside the United States and to contextually appropriate classification categories. Using stratified samples from Florida and North Carolina voter files with self-reported race, LLM-based classification achieves up to 84.7% accuracy, outperforming BISG (68.2%) on balanced samples. I test six models including Gemini 3 Flash, GPT-4o, and open-source alternatives such as DeepSeek v3.2 and GLM-4.7. Enabling extended reasoning can improve accuracy by 1-3 percentage points, though effects vary across contexts; including metadata such as party registration reaches 86.7%. LLM classification also reduces the income bias inherent in BISG, where minorities in wealthier neighborhoods are systematically misclassified as White. I further validate using Lebanese voter registration with religious sect (64.3% accuracy), Indian MPs from reserved constituencies (99.2%), and Indian land records with caste classification (74.0%). Aggregate validation across India, Uganda, Nepal, Armenia, Chile, and Costa Rica using original full-count voter rolls demonstrates that the method recovers known population distributions where naming conventions are distinctive. For large-scale applications, small transformer models fine-tuned on LLM labels exceed BISG accuracy while enabling local deployment at no cost.

Title: Learning to Advect: A Neural Semi-Lagrangian Architecture for Weather Forecasting

• Authors: Carlos A. Pereira, Stéphane Gaudreault, Valentin Dallerit, Christopher Subich, Shoyon Panday, Siqi Wei, Sasa Zhang, Siddharth Rout, Eldad Haber, Raymond J. Spiteri, David Millard, Emilia Diaconescu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21151
• Pdf URL: https://arxiv.org/pdf/2601.21151
• Copy Paste: [[2601.21151]] Learning to Advect: A Neural Semi-Lagrangian Architecture for Weather Forecasting(https://arxiv.org/abs/2601.21151)
• Keywords: diffusion
• Abstract: Recent machine-learning approaches to weather forecasting often employ a monolithic architecture, where distinct physical mechanisms (advection, transport), diffusion-like mixing, thermodynamic processes, and forcing are represented implicitly within a single large network. This representation is particularly problematic for advection, where long-range transport must be treated with expensive global interaction mechanisms or through deep, stacked convolutional layers. To mitigate this, we present PARADIS, a physics-inspired global weather prediction model that imposes inductive biases on network behavior through a functional decomposition into advection, diffusion, and reaction blocks acting on latent variables. We implement advection through a Neural Semi-Lagrangian operator that performs trajectory-based transport via differentiable interpolation on the sphere, enabling end-to-end learning of both the latent modes to be transported and their characteristic trajectories. Diffusion-like processes are modeled through depthwise-separable spatial mixing, while local source terms and vertical interactions are modeled via pointwise channel interactions, enabling operator-level physical structure. PARADIS provides state-of-the-art forecast skill at a fraction of the training cost. On ERA5-based benchmarks, the 1 degree PARADIS model, with a total training cost of less than a GPU month, meets or exceeds the performance of 0.25 degree traditional and machine-learning baselines, including the ECMWF HRES forecast and DeepMind's GraphCast.

Title: Bidirectional Cross-Perception for Open-Vocabulary Semantic Segmentation in Remote Sensing Imagery

• Authors: Jianzheng Wang, Huan Ni
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21159
• Pdf URL: https://arxiv.org/pdf/2601.21159
• Copy Paste: [[2601.21159]] Bidirectional Cross-Perception for Open-Vocabulary Semantic Segmentation in Remote Sensing Imagery(https://arxiv.org/abs/2601.21159)
• Keywords: diffusion, segmentation
• Abstract: High-resolution remote sensing imagery is characterized by densely distributed land-cover objects and complex boundaries, which places higher demands on both geometric localization and semantic prediction. Existing training-free open-vocabulary semantic segmentation (OVSS) methods typically fuse CLIP and vision foundation models (VFMs) using "one-way injection" and "shallow post-processing" strategies, making it difficult to satisfy these requirements. To address this issue, we propose a spatial-regularization-aware dual-branch collaborative inference framework for training-free OVSS, termed SDCI. First, during feature encoding, SDCI introduces a cross-model attention fusion (CAF) module, which guides collaborative inference by injecting self-attention maps into each other. Second, we propose a bidirectional cross-graph diffusion refinement (BCDR) module that enhances the reliability of dual-branch segmentation scores through iterative random-walk diffusion. Finally, we incorporate low-level superpixel structures and develop a convex-optimization-based superpixel collaborative prediction (CSCP) mechanism to further refine object boundaries. Experiments on multiple remote sensing semantic segmentation benchmarks demonstrate that our method achieves better performance than existing approaches. Moreover, ablation studies further confirm that traditional object-based remote sensing image analysis methods leveraging superpixel structures remain effective within deep learning frameworks. Code: this https URL.

Title: A Federated Generalized Expectation-Maximization Algorithm for Mixture Models with an Unknown Number of Components

• Authors: Michael Ibrahim, Nagi Gebraeel, Weijun Xie
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21160
• Pdf URL: https://arxiv.org/pdf/2601.21160
• Copy Paste: [[2601.21160]] A Federated Generalized Expectation-Maximization Algorithm for Mixture Models with an Unknown Number of Components(https://arxiv.org/abs/2601.21160)
• Keywords: federate
• Abstract: We study the problem of federated clustering when the total number of clusters $K$ across clients is unknown, and the clients have heterogeneous but potentially overlapping cluster sets in their local data. To that end, we develop FedGEM: a federated generalized expectation-maximization algorithm for the training of mixture models with an unknown number of components. Our proposed algorithm relies on each of the clients performing EM steps locally, and constructing an uncertainty set around the maximizer associated with each local component. The central server utilizes the uncertainty sets to learn potential cluster overlaps between clients, and infer the global number of clusters via closed-form computations. We perform a thorough theoretical study of our algorithm, presenting probabilistic convergence guarantees under common assumptions. Subsequently, we study the specific setting of isotropic GMMs, providing tractable, low-complexity computations to be performed by each client during each iteration of the algorithm, as well as rigorously verifying assumptions required for algorithm convergence. We perform various numerical experiments, where we empirically demonstrate that our proposed method achieves comparable performance to centralized EM, and that it outperforms various existing federated clustering methods.

Title: The Powers of Precision: Structure-Informed Detection in Complex Systems -- From Customer Churn to Seizure Onset

• Authors: Augusto Santos, Teresa Santos, Catarina Rodrigues, José M. F. Moura
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2601.21170
• Pdf URL: https://arxiv.org/pdf/2601.21170
• Copy Paste: [[2601.21170]] The Powers of Precision: Structure-Informed Detection in Complex Systems -- From Customer Churn to Seizure Onset(https://arxiv.org/abs/2601.21170)
• Keywords: explainability
• Abstract: Emergent phenomena -- onset of epileptic seizures, sudden customer churn, or pandemic outbreaks -- often arise from hidden causal interactions in complex systems. We propose a machine learning method for their early detection that addresses a core challenge: unveiling and harnessing a system's latent causal structure despite the data-generating process being unknown and partially observed. The method learns an optimal feature representation from a one-parameter family of estimators -- powers of the empirical covariance or precision matrix -- offering a principled way to tune in to the underlying structure driving the emergence of critical events. A supervised learning module then classifies the learned representation. We prove structural consistency of the family and demonstrate the empirical soundness of our approach on seizure detection and churn prediction, attaining competitive results in both. Beyond prediction, and toward explainability, we ascertain that the optimal covariance power exhibits evidence of good identifiability while capturing structural signatures, thus reconciling predictive performance with interpretable statistical structure.

Title: Enhancing Underwater Light Field Images via Global Geometry-aware Diffusion Process

• Authors: Yuji Lin, Qian Zhao, Zongsheng Yue, Junhui Hou, Deyu Meng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21179
• Pdf URL: https://arxiv.org/pdf/2601.21179
• Copy Paste: [[2601.21179]] Enhancing Underwater Light Field Images via Global Geometry-aware Diffusion Process(https://arxiv.org/abs/2601.21179)
• Keywords: diffusion
• Abstract: This work studies the challenging problem of acquiring high-quality underwater images via 4-D light field (LF) imaging. To this end, we propose GeoDiff-LF, a novel diffusion-based framework built upon SD-Turbo to enhance underwater 4-D LF imaging by leveraging its spatial-angular structure. GeoDiff-LF consists of three key adaptations: (1) a modified U-Net architecture with convolutional and attention adapters to model geometric cues, (2) a geometry-guided loss function using tensor decomposition and progressive weighting to regularize global structure, and (3) an optimized sampling strategy with noise prediction to improve efficiency. By integrating diffusion priors and LF geometry, GeoDiff-LF effectively mitigates color distortion in underwater scenes. Extensive experiments demonstrate that our framework outperforms existing methods across both visual fidelity and quantitative performance, advancing the state-of-the-art in enhancing underwater imaging. The code will be publicly available at this https URL.

Title: Rethinking Refinement: Correcting Generative Bias without Noise Injection

• Authors: Xin Peng, Ang Gao
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21182
• Pdf URL: https://arxiv.org/pdf/2601.21182
• Copy Paste: [[2601.21182]] Rethinking Refinement: Correcting Generative Bias without Noise Injection(https://arxiv.org/abs/2601.21182)
• Keywords: diffusion, generative
• Abstract: Generative models, including diffusion and flow-based models, often exhibit systematic biases that degrade sample quality, particularly in high-dimensional settings. We revisit refinement methods and show that effective bias correction can be achieved as a post-hoc procedure, without noise injection or multi-step resampling of the sampling process. We propose a flow-matching-based \textbf{Bi-stage Flow Refinement (BFR)} framework with two refinement strategies operating at different stages: latent space alignment for approximately invertible generators and data space refinement trained with lightweight augmentations. Unlike previous refiners that perturb sampling dynamics, BFR preserves the original ODE trajectory and applies deterministic corrections to generated samples. Experiments on MNIST, CIFAR-10, and FFHQ at 256x256 resolution demonstrate consistent improvements in fidelity and coverage; notably, starting from base samples with FID 3.95, latent space refinement achieves a \textbf{state-of-the-art} FID of \textbf{1.46} on MNIST using only a single additional function evaluation (1-NFE), while maintaining sample diversity.

Title: Adaptive and Robust Cost-Aware Proof of Quality for Decentralized LLM Inference Networks

• Authors: Arther Tian, Alex Ding, Frank Chen, Simon Wu, Aaron Chan
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21189
• Pdf URL: https://arxiv.org/pdf/2601.21189
• Copy Paste: [[2601.21189]] Adaptive and Robust Cost-Aware Proof of Quality for Decentralized LLM Inference Networks(https://arxiv.org/abs/2601.21189)
• Keywords: attack, robust, large language model
• Abstract: Decentralized large language model inference networks require lightweight mechanisms to reward high quality outputs under heterogeneous latency and cost. Proof of Quality provides scalable verification by sampling evaluator nodes that score candidate outputs, then aggregating their scores into a consensus signal that determines rewards. However, evaluator heterogeneity and malicious score manipulation can distort consensus and inflate payouts, which weakens incentive alignment in open participation settings. This paper extends a cost-aware Proof of Quality mechanism by adding adversary-resilient consensus formation. We study robust aggregation rules, including median and trimmed mean, and an adaptive trust-weighted consensus that updates evaluator weights from deviation signals. Using question answering and summarization workloads with a ground truth proxy for offline analysis, we quantify evaluator reliability and show strong variance across evaluators, including task-dependent misalignment that can invert correlations. We then evaluate robustness under four adversarial strategies, including noise injection, boosting, sabotage, and intermittent manipulation, across a sweep of malicious ratios and evaluator sample sizes. Our results show that robust aggregation improves consensus alignment with the ground truth proxy and reduces sensitivity to noisy and strategic attacks compared with simple averaging. We further characterize the operational trade-off introduced by evaluator sampling, where larger evaluator sets reduce evaluator rewards and increase payoff variance while inference rewards remain relatively stable in our configuration. These findings motivate robust consensus as a default component for cost-aware Proof of Quality and provide practical guidance for selecting evaluator sampling parameters under adversarial risk and resource constraints.

Title: Generative Recall, Dense Reranking: Learning Multi-View Semantic IDs for Efficient Text-to-Video Retrieval

• Authors: Zecheng Zhao, Zhi Chen, Zi Huang, Shazia Sadiq, Tong Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21193
• Pdf URL: https://arxiv.org/pdf/2601.21193
• Copy Paste: [[2601.21193]] Generative Recall, Dense Reranking: Learning Multi-View Semantic IDs for Efficient Text-to-Video Retrieval(https://arxiv.org/abs/2601.21193)
• Keywords: generative
• Abstract: Text-to-Video Retrieval (TVR) is essential in video platforms. Dense retrieval with dual-modality encoders leads in accuracy, but its computation and storage scale poorly with corpus size. Thus, real-time large-scale applications adopt two-stage retrieval, where a fast recall model gathers a small candidate pool, which is reranked by an advanced dense retriever. Due to hugely reduced candidates, the reranking model can use any off-the-shelf dense retriever without hurting efficiency, meaning the recall model bounds two-stage TVR performance. Recently, generative retrieval (GR) replaces dense video embeddings with discrete semantic IDs and retrieves by decoding text queries into ID tokens. GR offers near-constant inference and storage complexity, and its semantic IDs capture high-level video features via quantization, making it ideal for quickly eliminating irrelevant candidates during recall. However, as a recall model in two-stage TVR, GR suffers from (i) semantic ambiguity, where each video satisfies diverse queries but is forced into one semantic ID; and (ii) cross-modal misalignment, as semantic IDs are solely derived from visual features without text supervision. We propose Generative Recall and Dense Reranking (GRDR), designing a novel GR method to uplift recalled candidate quality. GRDR assigns multiple semantic IDs to each video using a query-guided multi-view tokenizer exposing diverse semantic access paths, and jointly trains the tokenizer and generative retriever via a shared codebook to cast semantic IDs as the semantic bridge between texts and videos. At inference, trie-constrained decoding generates a compact candidate set reranked by a dense model for fine-grained matching. Experiments on TVR benchmarks show GRDR matches strong dense retrievers in accuracy while reducing index storage by an order of magnitude and accelerating up to 300$\times$ in full-corpus retrieval.

Title: Scaling Embeddings Outperforms Scaling Experts in Language Models

• Authors: Hong Liu, Jiaqi Zhang, Chao Wang, Xing Hu, Linkun Lyu, Jiaqi Sun, Xurui Yang, Bo Wang, Fengcun Li, Yulei Qian, Lingtong Si, Yerui Sun, Rumei Li, Peng Pei, Yuchen Xie, Xunliang Cai
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21204
• Pdf URL: https://arxiv.org/pdf/2601.21204
• Copy Paste: [[2601.21204]] Scaling Embeddings Outperforms Scaling Experts in Language Models(https://arxiv.org/abs/2601.21204)
• Keywords: large language model
• Abstract: While Mixture-of-Experts (MoE) architectures have become the standard for sparsity scaling in large language models, they increasingly face diminishing returns and system-level bottlenecks. In this work, we explore embedding scaling as a potent, orthogonal dimension for scaling sparsity. Through a comprehensive analysis and experiments, we identify specific regimes where embedding scaling achieves a superior Pareto frontier compared to expert scaling. We systematically characterize the critical architectural factors governing this efficacy -- ranging from parameter budgeting to the interplay with model width and depth. Moreover, by integrating tailored system optimizations and speculative decoding, we effectively convert this sparsity into tangible inference speedups. Guided by these insights, we introduce LongCat-Flash-Lite, a 68.5B parameter model with ~3B activated trained from scratch. Despite allocating over 30B parameters to embeddings, LongCat-Flash-Lite not only surpasses parameter-equivalent MoE baselines but also exhibits exceptional competitiveness against existing models of comparable scale, particularly in agentic and coding domains.

Title: A Sheaf-Theoretic and Topological Perspective on Complex Network Modeling and Attention Mechanisms in Graph Neural Models

• Authors: Chuan-Shen Hu
• Subjects: cs.LG, cs.AI, math.AT
• Abstract URL: https://arxiv.org/abs/2601.21207
• Pdf URL: https://arxiv.org/pdf/2601.21207
• Copy Paste: [[2601.21207]] A Sheaf-Theoretic and Topological Perspective on Complex Network Modeling and Attention Mechanisms in Graph Neural Models(https://arxiv.org/abs/2601.21207)
• Keywords: diffusion
• Abstract: Combinatorial and topological structures, such as graphs, simplicial complexes, and cell complexes, form the foundation of geometric and topological deep learning (GDL and TDL) architectures. These models aggregate signals over such domains, integrate local features, and generate representations for diverse real-world applications. However, the distribution and diffusion behavior of GDL and TDL features during training remains an open and underexplored problem. Motivated by this gap, we introduce a cellular sheaf theoretic framework for modeling and analyzing the local consistency and harmonicity of node features and edge weights in graph-based architectures. By tracking local feature alignments and agreements through sheaf structures, the framework offers a topological perspective on feature diffusion and aggregation. Furthermore, a multiscale extension inspired by topological data analysis (TDA) is proposed to capture hierarchical feature interactions in graph models. This approach enables a joint characterization of GDL and TDL architectures based on their underlying geometric and topological structures and the learned signals defined on them, providing insights for future studies on conventional tasks such as node classification, substructure detection, and community detection.

Title: SPOILER-GUARD: Gating Latency Effects of Memory Accesses through Randomized Dependency Prediction

• Authors: Gayathri Subramanian, Girinath P, Nitya Ranganathan, Kamakoti Veezhinathan, Gopalakrishnan Srinivasan
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2601.21211
• Pdf URL: https://arxiv.org/pdf/2601.21211
• Copy Paste: [[2601.21211]] SPOILER-GUARD: Gating Latency Effects of Memory Accesses through Randomized Dependency Prediction(https://arxiv.org/abs/2601.21211)
• Keywords: defense, attack
• Abstract: Modern microprocessors depend on speculative execution, creating vulnerabilities that enable transient execution attacks. Prior defenses target speculative data leakage but overlook false dependencies from partial address aliasing, where repeated squash and reissue events increase the load-store latency, which is exploited by the SPOILER attack. We present SPOILER-GUARD, a hardware defense that obfuscates speculative dependency resolution by dynamically randomizing the physical address bits used for load-store comparisons and tagging store entries to prevent latency-amplifying misspeculations. Implemented in gem5 and evaluated with SPEC 2017, SPOILER-GUARD reduces misspeculation to 0.0004 percent and improves integer and floating-point performance by 2.12 and 2.87 percent. HDL synthesis with Synopsys Design Compiler at 14 nm node demonstrates minimal overheads - 69 ps latency in critical path, 0.064 square millimeter in area, and 5.863 mW in power.

Title: Scaling Reasoning Hop Exposes Weaknesses: Demystifying and Improving Hop Generalization in Large Language Models

• Authors: Zhaoyi Li, Jiatong Li, Gangwei Jiang, Linqi Song, Defu Lian, Ying Wei
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21214
• Pdf URL: https://arxiv.org/pdf/2601.21214
• Copy Paste: [[2601.21214]] Scaling Reasoning Hop Exposes Weaknesses: Demystifying and Improving Hop Generalization in Large Language Models(https://arxiv.org/abs/2601.21214)
• Keywords: large language model
• Abstract: Chain-of-thought (CoT) reasoning has become the standard paradigm for enabling Large Language Models (LLMs) to solve complex problems. However, recent studies reveal a sharp performance drop in reasoning hop generalization scenarios, where the required number of reasoning steps exceeds training distributions while the underlying algorithm remains unchanged. The internal mechanisms driving this failure remain poorly understood. In this work, we conduct a systematic study on tasks from multiple domains, and find that errors concentrate at token positions of a few critical error types, rather than being uniformly distributed. Closer inspection reveals that these token-level erroneous predictions stem from internal competition mechanisms: certain attention heads, termed erroneous processing heads (ep heads), tip the balance by amplifying incorrect reasoning trajectories while suppressing correct ones. Notably, removing individual ep heads during inference can often restore the correct predictions. Motivated by these insights, we propose test-time correction of reasoning, a lightweight intervention method that dynamically identifies and deactivates ep heads in the reasoning process. Extensive experiments across different tasks and LLMs show that it consistently improves reasoning hop generalization, highlighting both its effectiveness and potential.

Title: Temporal Context and Architecture: A Benchmark for Naturalistic EEG Decoding

• Authors: Mehmet Ergezer
• Subjects: cs.LG, cs.AI, eess.SP
• Abstract URL: https://arxiv.org/abs/2601.21215
• Pdf URL: https://arxiv.org/pdf/2601.21215
• Copy Paste: [[2601.21215]] Temporal Context and Architecture: A Benchmark for Naturalistic EEG Decoding(https://arxiv.org/abs/2601.21215)
• Keywords: robust, transformer
• Abstract: We study how model architecture and temporal context interact in naturalistic EEG decoding. Using the HBN movie-watching dataset, we benchmark five architectures, CNN, LSTM, a stabilized Transformer (EEGXF), S4, and S5, on a 4-class task across segment lengths from 8s to 128s. Accuracy improves with longer context: at 64s, S5 reaches 98.7%+/-0.6 and CNN 98.3%+/-0.3, while S5 uses ~20x fewer parameters than CNN. To probe real-world robustness, we evaluate zero-shot cross-frequency shifts, cross-task OOD inputs, and leave-one-subject-out generalization. S5 achieves stronger cross-subject accuracy but makes over-confident errors on OOD tasks; EEGXF is more conservative and stable under frequency shifts, though less calibrated in-distribution. These results reveal a practical efficiency-robustness trade-off: S5 for parameter-efficient peak accuracy; EEGXF when robustness and conservative uncertainty are critical.

Title: Parametric Knowledge is Not All You Need: Toward Honest Large Language Models via Retrieval of Pretraining Data

• Authors: Christopher Adrian Kusuma, Muhammad Reza Qorib, Hwee Tou Ng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21218
• Pdf URL: https://arxiv.org/pdf/2601.21218
• Copy Paste: [[2601.21218]] Parametric Knowledge is Not All You Need: Toward Honest Large Language Models via Retrieval of Pretraining Data(https://arxiv.org/abs/2601.21218)
• Keywords: robust, large language model
• Abstract: Large language models (LLMs) are highly capable of answering questions, but they are often unaware of their own knowledge boundary, i.e., knowing what they know and what they don't know. As a result, they can generate factually incorrect responses on topics they do not have enough knowledge of, commonly known as hallucination. Rather than hallucinating, a language model should be more honest and respond with "I don't know" when it does not have enough knowledge about a topic. Many methods have been proposed to improve LLM honesty, but their evaluations lack robustness, as they do not take into account the knowledge that the LLM has ingested during its pretraining. In this paper, we propose a more robust evaluation benchmark dataset for LLM honesty by utilizing Pythia, a truly open LLM with publicly available pretraining data. In addition, we also propose a novel method for harnessing the pretraining data to build a more honest LLM.

Title: LAMP: Learning Universal Adversarial Perturbations for Multi-Image Tasks via Pre-trained Models

• Authors: Alvi Md Ishmam, Najibul Haque Sarker, Zaber Ibn Abdul Hakim, Chris Thomas
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21220
• Pdf URL: https://arxiv.org/pdf/2601.21220
• Copy Paste: [[2601.21220]] LAMP: Learning Universal Adversarial Perturbations for Multi-Image Tasks via Pre-trained Models(https://arxiv.org/abs/2601.21220)
• Keywords: attack, robust, large language model
• Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable performance across vision-language tasks. Recent advancements allow these models to process multiple images as inputs. However, the vulnerabilities of multi-image MLLMs remain unexplored. Existing adversarial attacks focus on single-image settings and often assume a white-box threat model, which is impractical in many real-world scenarios. This paper introduces LAMP, a black-box method for learning Universal Adversarial Perturbations (UAPs) targeting multi-image MLLMs. LAMP applies an attention-based constraint that prevents the model from effectively aggregating information across images. LAMP also introduces a novel cross-image contagious constraint that forces perturbed tokens to influence clean tokens, spreading adversarial effects without requiring all inputs to be modified. Additionally, an index-attention suppression loss enables a robust position-invariant attack. Experimental results show that LAMP outperforms SOTA baselines and achieves the highest attack success rates across multiple vision-language tasks and models.

Title: MGSM-Pro: A Simple Strategy for Robust Multilingual Mathematical Reasoning Evaluation

• Authors: Tianyi Xu, Kosei Uemura, Alfred Malengo Kondoro, Tadesse Destaw Belay, Catherine Nana Nyaah Essuman, Ifeoma Okoh, Ganiyat Afolabi, Ayodele Awokoya, David Ifeoluwa Adelani
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21225
• Pdf URL: https://arxiv.org/pdf/2601.21225
• Copy Paste: [[2601.21225]] MGSM-Pro: A Simple Strategy for Robust Multilingual Mathematical Reasoning Evaluation(https://arxiv.org/abs/2601.21225)
• Keywords: robust, large language model
• Abstract: Large language models have made substantial progress in mathematical reasoning. However, benchmark development for multilingual evaluation has lagged behind English in both difficulty and recency. Recently, GSM-Symbolic showed a strong evidence of high variance when models are evaluated on different instantiations of the same question; however, the evaluation was conducted only in English. In this paper, we introduce MGSM-Pro, an extension of MGSM dataset with GSM-Symbolic approach. Our dataset provides five instantiations per MGSM question by varying names, digits and irrelevant context. Evaluations across nine languages reveal that many low-resource languages suffer large performance drops when tested on digit instantiations different from those in the original test set. We further find that some proprietary models, notably Gemini 2.5 Flash and GPT-4.1, are less robust to digit instantiation, whereas Claude 4.0 Sonnet is more robust. Among open models, GPT-OSS 120B and DeepSeek V3 show stronger robustness. Based on these findings, we recommend evaluating each problem using at least five digit-varying instantiations to obtain a more robust and realistic assessment of math reasoning.

Title: PHDME: Physics-Informed Diffusion Models without Explicit Governing Equations

• Authors: Kaiyuan Tan, Kendra Givens, Peilun Li, Thomas Beckers
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21234
• Pdf URL: https://arxiv.org/pdf/2601.21234
• Copy Paste: [[2601.21234]] PHDME: Physics-Informed Diffusion Models without Explicit Governing Equations(https://arxiv.org/abs/2601.21234)
• Keywords: diffusion
• Abstract: Diffusion models provide expressive priors for forecasting trajectories of dynamical systems, but are typically unreliable in the sparse data regime. Physics-informed machine learning (PIML) improves reliability in such settings; however, most methods require \emph{explicit governing equations} during training, which are often only partially known due to complex and nonlinear dynamics. We introduce \textbf{PHDME}, a port-Hamiltonian diffusion framework designed for \emph{sparse observations} and \emph{incomplete physics}. PHDME leverages port-Hamiltonian structural prior but does not require full knowledge of the closed-form governing equations. Our approach first trains a Gaussian process distributed Port-Hamiltonian system (GP-dPHS) on limited observations to capture an energy-based representation of the dynamics. The GP-dPHS is then used to generate a physically consistent artificial dataset for diffusion training, and to inform the diffusion model with a structured physics residual loss. After training, the diffusion model acts as an amortized sampler and forecaster for fast trajectory generation. Finally, we apply split conformal calibration to provide uncertainty statements for the generated predictions. Experiments on PDE benchmarks and a real-world spring system show improved accuracy and physical consistency under data scarcity.

Title: SHARP: Social Harm Analysis via Risk Profiles for Measuring Inequities in Large Language Models

• Authors: Alok Abhishek, Tushar Bandopadhyay, Lisa Erickson
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21235
• Pdf URL: https://arxiv.org/pdf/2601.21235
• Copy Paste: [[2601.21235]] SHARP: Social Harm Analysis via Risk Profiles for Measuring Inequities in Large Language Models(https://arxiv.org/abs/2601.21235)
• Keywords: fair, large language model
• Abstract: Large language models (LLMs) are increasingly deployed in high-stakes domains, where rare but severe failures can result in irreversible harm. However, prevailing evaluation benchmarks often reduce complex social risk to mean-centered scalar scores, thereby obscuring distributional structure, cross-dimensional interactions, and worst-case behavior. This paper introduces Social Harm Analysis via Risk Profiles (SHARP), a framework for multidimensional, distribution-aware evaluation of social harm. SHARP models harm as a multivariate random variable and integrates explicit decomposition into bias, fairness, ethics, and epistemic reliability with a union-of-failures aggregation reparameterized as additive cumulative log-risk. The framework further employs risk-sensitive distributional statistics, with Conditional Value at Risk (CVaR95) as a primary metric, to characterize worst-case model behavior. Application of SHARP to eleven frontier LLMs, evaluated on a fixed corpus of n=901 socially sensitive prompts, reveals that models with similar average risk can exhibit more than twofold differences in tail exposure and volatility. Across models, dimension-wise marginal tail behavior varies systematically across harm dimensions, with bias exhibiting the strongest tail severities, epistemic and fairness risks occupying intermediate regimes, and ethical misalignment consistently lower; together, these patterns reveal heterogeneous, model-dependent failure structures that scalar benchmarks conflate. These findings indicate that responsible evaluation and governance of LLMs require moving beyond scalar averages toward multidimensional, tail-sensitive risk profiling.

Title: PTQ4ARVG: Post-Training Quantization for AutoRegressive Visual Generation Models

• Authors: Xuewen Liu, Zhikai Li, Jing Zhang, Mengjuan Chen, Qingyi Gu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21238
• Pdf URL: https://arxiv.org/pdf/2601.21238
• Copy Paste: [[2601.21238]] PTQ4ARVG: Post-Training Quantization for AutoRegressive Visual Generation Models(https://arxiv.org/abs/2601.21238)
• Keywords: diffusion
• Abstract: AutoRegressive Visual Generation (ARVG) models retain an architecture compatible with language models, while achieving performance comparable to diffusion-based models. Quantization is commonly employed in neural networks to reduce model size and computational latency. However, applying quantization to ARVG remains largely underexplored, and existing quantization methods fail to generalize effectively to ARVG models. In this paper, we explore this issue and identify three key challenges: (1) severe outliers at channel-wise level, (2) highly dynamic activations at token-wise level, and (3) mismatched distribution information at sample-wise level. To these ends, we propose PTQ4ARVG, a training-free post-training quantization (PTQ) framework consisting of: (1) Gain-Projected Scaling (GPS) mitigates the channel-wise outliers, which expands the quantization loss via a Taylor series to quantify the gain of scaling for activation-weight quantization, and derives the optimal scaling factor through differentiation.(2) Static Token-Wise Quantization (STWQ) leverages the inherent properties of ARVG, fixed token length and position-invariant distribution across samples, to address token-wise variance without incurring dynamic calibration overhead.(3) Distribution-Guided Calibration (DGC) selects samples that contribute most to distributional entropy, eliminating the sample-wise distribution mismatch. Extensive experiments show that PTQ4ARVG can effectively quantize the ARVG family models to 8-bit and 6-bit while maintaining competitive performance. Code is available at this http URL .

Title: Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks

• Authors: Luwei Sun, Dongrui Shen, Jianfe Li, Yulong Zhao, Han Feng
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21242
• Pdf URL: https://arxiv.org/pdf/2601.21242
• Copy Paste: [[2601.21242]] Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks(https://arxiv.org/abs/2601.21242)
• Keywords: diffusion, generative
• Abstract: Motivated by challenges in conditional generative modeling, where the target conditional density takes the form of a ratio f1 over f2, this paper develops a theoretical framework for approximating such ratio-type functionals. Here, f1 and f2 are kernel-based marginal densities that capture structured interactions, a setting central to diffusion-based generative models. We provide a concise proof for approximating these ratio-type functionals using deep neural networks with the SignReLU activation function, leveraging the activation's piecewise structure. Under standard regularity assumptions, we establish L^p(Omega) approximation bounds and convergence rates. Specializing to Denoising Diffusion Probabilistic Models (DDPMs), we construct a SignReLU-based neural estimator for the reverse process and derive bounds on the excess Kullback-Leibler (KL) risk between the generated and true data distributions. Our analysis decomposes this excess risk into approximation and estimation error components. These results provide generalization guarantees for finite-sample training of diffusion-based generative models.

Title: Conditional Generative Framework with Peak-Aware Attention for Robust Chemical Detection under Interferences

• Authors: Namkyung Yoon, Sanghong Kim, Hwangnam Kim
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21246
• Pdf URL: https://arxiv.org/pdf/2601.21246
• Copy Paste: [[2601.21246]] Conditional Generative Framework with Peak-Aware Attention for Robust Chemical Detection under Interferences(https://arxiv.org/abs/2601.21246)
• Keywords: robust, generative
• Abstract: Gas chromatography-mass spectrometry (GC-MS) is a widely used analytical method for chemical substance detection, but measurement reliability tends to deteriorate in the presence of interfering substances. In particular, interfering substances cause nonspecific peaks, residence time shifts, and increased background noise, resulting in reduced sensitivity and false alarms. To overcome these challenges, in this paper, we propose an artificial intelligence discrimination framework based on a peak-aware conditional generative model to improve the reliability of GC-MS measurements under interference conditions. The framework is learned with a novel peak-aware mechanism that highlights the characteristic peaks of GC-MS data, allowing it to generate important spectral features more faithfully. In addition, chemical and solvent information is encoded in a latent vector embedded with it, allowing a conditional generative adversarial neural network (CGAN) to generate a synthetic GC-MS signal consistent with the experimental conditions. This generates an experimental dataset that assumes indirect substance situations in chemical substance data, where acquisition is limited without conducting real experiments. These data are used for the learning of AI-based GC-MS discrimination models to help in accurate chemical substance discrimination. We conduct various quantitative and qualitative evaluations of the generated simulated data to verify the validity of the proposed framework. We also verify how the generative model improves the performance of the AI discrimination framework. Representatively, the proposed method is shown to consistently achieve cosine similarity and Pearson correlation coefficient values above 0.9 while preserving peak number diversity and reducing false alarms in the discrimination model.

Title: NFCDS: A Plug-and-Play Noise Frequency-Controlled Diffusion Sampling Strategy for Image Restoration

• Authors: Zhen Wang, Hongyi Liu, Jianing Li, Zhihui Wei
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21248
• Pdf URL: https://arxiv.org/pdf/2601.21248
• Copy Paste: [[2601.21248]] NFCDS: A Plug-and-Play Noise Frequency-Controlled Diffusion Sampling Strategy for Image Restoration(https://arxiv.org/abs/2601.21248)
• Keywords: diffusion
• Abstract: Diffusion sampling-based Plug-and-Play (PnP) methods produce images with high perceptual quality but often suffer from reduced data fidelity, primarily due to the noise introduced during reverse diffusion. To address this trade-off, we propose Noise Frequency-Controlled Diffusion Sampling (NFCDS), a spectral modulation mechanism for reverse diffusion noise. We show that the fidelity-perception conflict can be fundamentally understood through noise frequency: low-frequency components induce blur and degrade fidelity, while high-frequency components drive detail generation. Based on this insight, we design a Fourier-domain filter that progressively suppresses low-frequency noise and preserves high-frequency content. This controlled refinement injects a data-consistency prior directly into sampling, enabling fast convergence to results that are both high-fidelity and perceptually convincing--without additional training. As a PnP module, NFCDS seamlessly integrates into existing diffusion-based restoration frameworks and improves the fidelity-perception balance across diverse zero-shot tasks.

Title: Lossless Copyright Protection via Intrinsic Model Fingerprinting

• Authors: Lingxiao Chen, Liqin Wang, Wei Lu, Xiangyang Luo
• Subjects: cs.CR, cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21252
• Pdf URL: https://arxiv.org/pdf/2601.21252
• Copy Paste: [[2601.21252]] Lossless Copyright Protection via Intrinsic Model Fingerprinting(https://arxiv.org/abs/2601.21252)
• Keywords: protect, robust, watermark, diffusion
• Abstract: The exceptional performance of diffusion models establishes them as high-value intellectual property but exposes them to unauthorized replication. Existing protection methods either modify the model to embed watermarks, which impairs performance, or extract model fingerprints by manipulating the denoising process, rendering them incompatible with black-box APIs. In this paper, we propose TrajPrint, a completely lossless and training-free framework that verifies model copyright by extracting unique manifold fingerprints formed during deterministic generation. Specifically, we first utilize a watermarked image as an anchor and exactly trace the path back to its trajectory origin, effectively locking the model fingerprint mapped by this path. Subsequently, we implement a joint optimization strategy that employs dual-end anchoring to synthesize a specific fingerprint noise, which strictly adheres to the target manifold for robust watermark recovery. As input, it enables the protected target model to recover the watermarked image, while failing on non-target models. Finally, we achieved verification via atomic inference and statistical hypothesis testing. Extensive experiments demonstrate that TrajPrint achieves lossless verification in black-box API scenarios with superior robustness against model modifications.

Title: Virtualization-based Penetration Testing Study for Detecting Accessibility Abuse Vulnerabilities in Banking Apps in East and Southeast Asia

• Authors: Wei Minn, Phong Phan, Vikas K. Malviya, Benjamin Adolphi, Yan Naing Tun, Henning Benzon Treichl, Albert Ching, Lwin Khin Shar, David Lo
• Subjects: cs.CR, cs.SE
• Abstract URL: https://arxiv.org/abs/2601.21258
• Pdf URL: https://arxiv.org/pdf/2601.21258
• Copy Paste: [[2601.21258]] Virtualization-based Penetration Testing Study for Detecting Accessibility Abuse Vulnerabilities in Banking Apps in East and Southeast Asia(https://arxiv.org/abs/2601.21258)
• Keywords: security, protect, attack
• Abstract: Android banking applications have revolutionized financial management by allowing users to perform various financial activities through mobile devices. However, this convenience has attracted cybercriminals who exploit security vulnerabilities to access sensitive financial data. FjordPhantom, a malware identified by our industry collaborator, uses virtualization and hooking to bypass the detection of malicious accessibility services, allowing it to conduct keylogging, screen scraping, and unauthorized data access. This malware primarily affects banking and finance apps across East and Southeast Asia region where our industry partner's clients are primarily based in. It requires users to be deceived into installing a secondary malicious component and activating a malicious accessibility service. In our study, we conducted an empirical study on the susceptibility of banking apps in the region to FjordPhantom, analyzed the effectiveness of protective measures currently implemented in those apps, and discussed ways to detect and prevent such attacks by identifying and mitigating the vulnerabilities exploited by this malware.

Title: User-Centric Phishing Detection: A RAG and LLM-Based Approach

• Authors: Abrar Hamed Al Barwani, Abdelaziz Amara Korba, Raja Waseem Anwar
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2601.21261
• Pdf URL: https://arxiv.org/pdf/2601.21261
• Copy Paste: [[2601.21261]] User-Centric Phishing Detection: A RAG and LLM-Based Approach(https://arxiv.org/abs/2601.21261)
• Keywords: security, large language model
• Abstract: The escalating sophistication of phishing emails necessitates a shift beyond traditional rule-based and conventional machine-learning-based detectors. Although large language models (LLMs) offer strong natural language understanding, using them as standalone classifiers often yields elevated falsepositive (FP) rates, which mislabel legitimate emails as phishing and create significant operational burden. This paper presents a personalized phishing detection framework that integrates LLMs with retrieval-augmented generation (RAG). For each message, the system constructs user-specific context by retrieving a compact set of the user's historical legitimate emails and enriching it with real-time domain and URL reputation from a cyber-threat intelligence platform, then conditions the LLM's decision on this evidence. We evaluate four open-source LLMs (Llama4-Scout, DeepSeek-R1, Mistral-Saba, and Gemma2) on an email dataset collected from public and institutional sources. Results show high performance; for example, Llama4-Scout attains an F1-score of 0.9703 and achieves a 66.7% reduction in FPs with RAG. These findings validate that a RAG-based, user-profiling approach is both feasible and effective for building high-precision, low-friction email security systems that adapt to individual communication patterns.

Title: CausalEmbed: Auto-Regressive Multi-Vector Generation in Latent Space for Visual Document Embedding

• Authors: Jiahao Huo, Yu Huang, Yibo Yan, Ye Pan, Yi Cao, Mingdong Ou, Philip S. Yu, Xuming Hu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21262
• Pdf URL: https://arxiv.org/pdf/2601.21262
• Copy Paste: [[2601.21262]] CausalEmbed: Auto-Regressive Multi-Vector Generation in Latent Space for Visual Document Embedding(https://arxiv.org/abs/2601.21262)
• Keywords: generative, large language model
• Abstract: Although Multimodal Large Language Models (MLLMs) have shown remarkable potential in Visual Document Retrieval (VDR) through generating high-quality multi-vector embeddings, the substantial storage overhead caused by representing a page with thousands of visual tokens limits their practicality in real-world applications. To address this challenge, we propose an auto-regressive generation approach, CausalEmbed, for constructing multi-vector embeddings. By incorporating iterative margin loss during contrastive training, CausalEmbed encourages the embedding models to learn compact and well-structured representations. Our method enables efficient VDR tasks using only dozens of visual tokens, achieving a 30-155x reduction in token count while maintaining highly competitive performance across various backbones and benchmarks. Theoretical analysis and empirical results demonstrate the unique advantages of auto-regressive embedding generation in terms of training efficiency and scalability at test time. As a result, CausalEmbed introduces a flexible test-time scaling strategy for multi-vector VDR representations and sheds light on the generative paradigm within multimodal document retrieval.

Title: Model-Free Neural State Estimation in Nonlinear Dynamical Systems: A Comparative Study of Neural Architectures and Classical Filters

• Authors: Zhuochen Liu, Hans Walker, Rahul Jain
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21266
• Pdf URL: https://arxiv.org/pdf/2601.21266
• Copy Paste: [[2601.21266]] Model-Free Neural State Estimation in Nonlinear Dynamical Systems: A Comparative Study of Neural Architectures and Classical Filters(https://arxiv.org/abs/2601.21266)
• Keywords: transformer
• Abstract: Neural network models are increasingly used for state estimation in control and decision-making problems, yet it remains unclear to what extent they behave as principled filters in nonlinear dynamical systems. Unlike classical filters, which rely on explicit knowledge of system dynamics and noise models, neural estimators can be trained purely from data without access to the underlying system equations. In this work, we present a systematic empirical comparison between such model-free neural network models and classical filtering methods across multiple nonlinear scenarios. Our study evaluates Transformer-based models, state-space neural networks, and recurrent architectures alongside particle filters and nonlinear Kalman filters. The results show that neural models (in particular, state-space models (SSMs)) achieve state estimation performance that approaches strong nonlinear Kalman filters in nonlinear scenarios and outperform weaker classical baselines despite lacking access to system models, while also attaining substantially higher inference throughput.

Title: Token Entropy Regularization for Multi-modal Antenna Affiliation Identification

• Authors: Dong Chen, Ruoyu Li, Xinyan Zhang, Jialei Xu, Ruoseng Zhao, Zhikang Zhang, Lingyun Li, Zizhuang Wei
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21280
• Pdf URL: https://arxiv.org/pdf/2601.21280
• Copy Paste: [[2601.21280]] Token Entropy Regularization for Multi-modal Antenna Affiliation Identification(https://arxiv.org/abs/2601.21280)
• Keywords: transformer
• Abstract: Accurate antenna affiliation identification is crucial for optimizing and maintaining communication networks. Current practice, however, relies on the cumbersome and error-prone process of manual tower inspections. We propose a novel paradigm shift that fuses video footage of base stations, antenna geometric features, and Physical Cell Identity (PCI) signals, transforming antenna affiliation identification into multi-modal classification and matching tasks. Publicly available pretrained transformers struggle with this unique task due to a lack of analogous data in the communications domain, which hampers cross-modal alignment. To address this, we introduce a dedicated training framework that aligns antenna images with corresponding PCI signals. To tackle the representation alignment challenge, we propose a novel Token Entropy Regularization module in the pretraining stage. Our experiments demonstrate that TER accelerates convergence and yields significant performance gains. Further analysis reveals that the entropy of the first token is modality-dependent. Code will be made available upon publication.

Title: WorldBench: Disambiguating Physics for Diagnostic Evaluation of World Models

• Authors: Rishi Upadhyay, Howard Zhang, Jim Solomon, Ayush Agrawal, Pranay Boreddy, Shruti Satya Narayana, Yunhao Ba, Alex Wong, Celso M de Melo, Achuta Kadambi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21282
• Pdf URL: https://arxiv.org/pdf/2601.21282
• Copy Paste: [[2601.21282]] WorldBench: Disambiguating Physics for Diagnostic Evaluation of World Models(https://arxiv.org/abs/2601.21282)
• Keywords: robust, generative
• Abstract: Recent advances in generative foundational models, often termed "world models," have propelled interest in applying them to critical tasks like robotic planning and autonomous system training. For reliable deployment, these models must exhibit high physical fidelity, accurately simulating real-world dynamics. Existing physics-based video benchmarks, however, suffer from entanglement, where a single test simultaneously evaluates multiple physical laws and concepts, fundamentally limiting their diagnostic capability. We introduce WorldBench, a novel video-based benchmark specifically designed for concept-specific, disentangled evaluation, allowing us to rigorously isolate and assess understanding of a single physical concept or law at a time. To make WorldBench comprehensive, we design benchmarks at two different levels: 1) an evaluation of intuitive physical understanding with concepts such as object permanence or scale/perspective, and 2) an evaluation of low-level physical constants and material properties such as friction coefficients or fluid viscosity. When SOTA video-based world models are evaluated on WorldBench, we find specific patterns of failure in particular physics concepts, with all tested models lacking the physical consistency required to generate reliable real-world interactions. Through its concept-specific evaluation, WorldBench offers a more nuanced and scalable framework for rigorously evaluating the physical reasoning capabilities of video generation and world models, paving the way for more robust and generalizable world-model-driven learning.

Title: DUET: Distilled LLM Unlearning from an Efficiently Contextualized Teacher

• Authors: Yisheng Zhong, Zhengbang Yang, Zhuangdi Zhu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21283
• Pdf URL: https://arxiv.org/pdf/2601.21283
• Copy Paste: [[2601.21283]] DUET: Distilled LLM Unlearning from an Efficiently Contextualized Teacher(https://arxiv.org/abs/2601.21283)
• Keywords: attack
• Abstract: LLM unlearning is a technique to remove the impacts of undesirable knowledge from the model without retraining from scratch, which is indispensable towards trustworthy AI. Existing unlearning methods face significant limitations: conventional tuning-based unlearning is computationally heavy and prone to catastrophic forgetting. In contrast, in-contextualized unlearning is lightweight for precise unlearning but vulnerable to prompt removal or reverse engineering attacks. In response, we propose Distilled Unlearning from an Efficient Teacher (DUET), a novel distillation-based unlearning method that combines the merits of these two lines of work. It learns a student model to imitate the behavior of a prompt-steered teacher that effectively refuses undesirable knowledge generation while preserving general domain knowledge. Extensive evaluations on existing benchmarks with our enriched evaluation protocols demonstrate that DUET achieves higher performance in both forgetting and utility preservation, while being orders of magnitude more data-efficient than state-of-the-art unlearning methods.

Title: PILD: Physics-Informed Learning via Diffusion

• Authors: Tianyi Zeng, Tianyi Wang, Jiaru Zhang, Zimo Zeng, Feiyang Zhang, Yiming Xu, Sikai Chen, Yajie Zou, Yangyang Wang, Junfeng Jiao, Christian Claudel, Xinbo Chen
• Subjects: cs.LG, cs.AI, cs.ET, math.AP
• Abstract URL: https://arxiv.org/abs/2601.21284
• Pdf URL: https://arxiv.org/pdf/2601.21284
• Copy Paste: [[2601.21284]] PILD: Physics-Informed Learning via Diffusion(https://arxiv.org/abs/2601.21284)
• Keywords: diffusion, generative
• Abstract: Diffusion models have emerged as powerful generative tools for modeling complex data distributions, yet their purely data-driven nature limits applicability in practical engineering and scientific problems where physical laws need to be followed. This paper proposes Physics-Informed Learning via Diffusion (PILD), a framework that unifies diffusion modeling and first-principles physical constraints by introducing a virtual residual observation sampled from a Laplace distribution to supervise generation during training. To further integrate physical laws, a conditional embedding module is incorporated to inject physical information into the denoising network at multiple layers, ensuring consistent guidance throughout the diffusion process. The proposed PILD framework is concise, modular, and broadly applicable to problems governed by ordinary differential equations, partial differential equations, as well as algebraic equations or inequality constraints. Extensive experiments across engineering and scientific tasks including estimating vehicle trajectories, tire forces, Darcy flow and plasma dynamics, demonstrate that our PILD substantially improves accuracy, stability, and generalization over existing physics-informed and diffusion-based baselines.

Title: Towards Zero Rotation and Beyond: Architecting Neural Networks for Fast Secure Inference with Homomorphic Encryption

• Authors: Yifei Cai, Yizhou Feng, Qiao Zhang, Chunsheng Xin, Hongyi Wu
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2601.21287
• Pdf URL: https://arxiv.org/pdf/2601.21287
• Copy Paste: [[2601.21287]] Towards Zero Rotation and Beyond: Architecting Neural Networks for Fast Secure Inference with Homomorphic Encryption(https://arxiv.org/abs/2601.21287)
• Keywords: secure, privacy
• Abstract: Privacy-preserving deep learning addresses privacy concerns in Machine Learning as a Service (MLaaS) by using Homomorphic Encryption (HE) for linear computations. However, the computational overhead remains a major challenge. While prior work has improved efficiency, most approaches build on models originally designed for plaintext inference. Such models incur architectural inefficiencies when adapted to HE. We argue that substantial gains require networks tailored to HE rather than retrofitting plaintext architectures. Our design has two components: the building block and the overall architecture. First, StriaBlock targets the most expensive HE operation, rotation. It integrates ExRot-Free Convolution and a novel Cross Kernel, eliminating external rotations and requiring only 19% of the internal rotations used by plaintext models. Second, our architectural principles include (i) the Focused Constraint Principle, which limits cost-sensitive factors while preserving flexibility elsewhere, and (ii) the Channel Packing-Aware Scaling Principle, which adapts bottleneck ratios to ciphertext channel capacity that varies with depth. Together, these strategies control both local and end-to-end HE cost, enabling a balanced HE-tailored network. We evaluate the resulting StriaNet across datasets of varying scales, including ImageNet, Tiny ImageNet, and CIFAR-10. At comparable accuracy, StriaNet achieves speedups of 9.78x, 6.01x, and 9.24x on ImageNet, Tiny ImageNet, and CIFAR-10, respectively.

Title: TimeSliver : Symbolic-Linear Decomposition for Explainable Time Series Classification

• Authors: Akash Pandey, Payal Mohapatra, Wei Chen, Qi Zhu, Sinan Keten
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21289
• Pdf URL: https://arxiv.org/pdf/2601.21289
• Copy Paste: [[2601.21289]] TimeSliver : Symbolic-Linear Decomposition for Explainable Time Series Classification(https://arxiv.org/abs/2601.21289)
• Keywords: explainability
• Abstract: Identifying the extent to which every temporal segment influences a model's predictions is essential for explaining model decisions and increasing transparency. While post-hoc explainable methods based on gradients and feature-based attributions have been popular, they suffer from reference state sensitivity and struggle to generalize across time-series datasets, as they treat time points independently and ignore sequential dependencies. Another perspective on explainable time-series classification is through interpretable components of the model, for instance, leveraging self-attention mechanisms to estimate temporal attribution; however, recent findings indicate that these attention weights often fail to provide faithful measures of temporal importance. In this work, we advance this perspective and present a novel explainability-driven deep learning framework, TimeSliver, which jointly utilizes raw time-series data and its symbolic abstraction to construct a representation that maintains the original temporal structure. Each element in this representation linearly encodes the contribution of each temporal segment to the final prediction, allowing us to assign a meaningful importance score to every time point. For time-series classification, TimeSliver outperforms other temporal attribution methods by 11% on 7 distinct synthetic and real-world multivariate time-series datasets. TimeSliver also achieves predictive performance within 2% of state-of-the-art baselines across 26 UEA benchmark datasets, positioning it as a strong and explainable framework for general time-series classification.

Title: Gaussian Belief Propagation Network for Depth Completion

• Authors: Jie Tang, Pingping Xie, Jian Li, Ping Tan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21291
• Pdf URL: https://arxiv.org/pdf/2601.21291
• Copy Paste: [[2601.21291]] Gaussian Belief Propagation Network for Depth Completion(https://arxiv.org/abs/2601.21291)
• Keywords: robust
• Abstract: Depth completion aims to predict a dense depth map from a color image with sparse depth measurements. Although deep learning methods have achieved state-of-the-art (SOTA), effectively handling the sparse and irregular nature of input depth data in deep networks remains a significant challenge, often limiting performance, especially under high sparsity. To overcome this limitation, we introduce the Gaussian Belief Propagation Network (GBPN), a novel hybrid framework synergistically integrating deep learning with probabilistic graphical models for end-to-end depth completion. Specifically, a scene-specific Markov Random Field (MRF) is dynamically constructed by the Graphical Model Construction Network (GMCN), and then inferred via Gaussian Belief Propagation (GBP) to yield the dense depth distribution. Crucially, the GMCN learns to construct not only the data-dependent potentials of MRF but also its structure by predicting adaptive non-local edges, enabling the capture of complex, long-range spatial dependencies. Furthermore, we enhance GBP with a serial \& parallel message passing scheme, designed for effective information propagation, particularly from sparse measurements. Extensive experiments demonstrate that GBPN achieves SOTA performance on the NYUv2 and KITTI benchmarks. Evaluations across varying sparsity levels, sparsity patterns, and datasets highlight GBPN's superior performance, notable robustness, and generalizable capability.

Title: Physics-Guided Tiny-Mamba Transformer for Reliability-Aware Early Fault Warning

• Authors: Changyu Li, Dingcheng Huang, Kexuan Yao, Xiaoya Ni, Lijuan Shen, Fei Luo
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21293
• Pdf URL: https://arxiv.org/pdf/2601.21293
• Copy Paste: [[2601.21293]] Physics-Guided Tiny-Mamba Transformer for Reliability-Aware Early Fault Warning(https://arxiv.org/abs/2601.21293)
• Keywords: transformer
• Abstract: Reliability-centered prognostics for rotating machinery requires early warning signals that remain accurate under nonstationary operating conditions, domain shifts across speed/load/sensors, and severe class imbalance, while keeping the false-alarm rate small and predictable. We propose the Physics-Guided Tiny-Mamba Transformer (PG-TMT), a compact tri-branch encoder tailored for online condition monitoring. A depthwise-separable convolutional stem captures micro-transients, a Tiny-Mamba state-space branch models near-linear long-range dynamics, and a lightweight local Transformer encodes cross-channel resonances. We derive an analytic temporal-to-spectral mapping that ties the model's attention spectrum to classical bearing fault-order bands, yielding a band-alignment score that quantifies physical plausibility and provides physics-grounded explanations. To ensure decision reliability, healthy-score exceedances are modeled with extreme-value theory (EVT), which yields an on-threshold achieving a target false-alarm intensity (events/hour); a dual-threshold hysteresis with a minimum hold time further suppresses chatter. Under a leakage-free streaming protocol with right-censoring of missed detections on CWRU, Paderborn, XJTU-SY, and an industrial pilot, PG-TMT attains higher precision-recall AUC (primary under imbalance), competitive or better ROC AUC, and shorter mean time-to-detect at matched false-alarm intensity, together with strong cross-domain transfer. By coupling physics-aligned representations with EVT-calibrated decision rules, PG-TMT delivers calibrated, interpretable, and deployment-ready early warnings for reliability-centric prognostics and health management.

Title: Mam-App: A Novel Parameter-Efficient Mamba Model for Apple Leaf Disease Classification

• Authors: Md Nadim Mahamood, Md Imran Hasan, Md Rasheduzzaman, Ausrukona Ray, Md Shafi Ud Doula, Kamrul Hasan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21307
• Pdf URL: https://arxiv.org/pdf/2601.21307
• Copy Paste: [[2601.21307]] Mam-App: A Novel Parameter-Efficient Mamba Model for Apple Leaf Disease Classification(https://arxiv.org/abs/2601.21307)
• Keywords: robust, extraction
• Abstract: The rapid growth of the global population, alongside exponential technological advancement, has intensified the demand for food production. Meeting this demand depends not only on increasing agricultural yield but also on minimizing food loss caused by crop diseases. Diseases account for a substantial portion of apple production losses, despite apples being among the most widely produced and nutritionally valuable fruits worldwide. Previous studies have employed machine learning techniques for feature extraction and early diagnosis of apple leaf diseases, and more recently, deep learning-based models have shown remarkable performance in disease recognition. However, most state-of-the-art deep learning models are highly parameter-intensive, resulting in increased training and inference time. Although lightweight models are more suitable for user-friendly and resource-constrained applications, they often suffer from performance degradation. To address the trade-off between efficiency and performance, we propose Mam-App, a parameter-efficient Mamba-based model for feature extraction and leaf disease classification. The proposed approach achieves competitive state-of-the-art performance on the PlantVillage Apple Leaf Disease dataset, attaining 99.58% accuracy, 99.30% precision, 99.14% recall, and a 99.22% F1-score, while using only 0.051M parameters. This extremely low parameter count makes the model suitable for deployment on drones, mobile devices, and other low-resource platforms. To demonstrate the robustness and generalizability of the proposed model, we further evaluate it on the PlantVillage Corn Leaf Disease and Potato Leaf Disease datasets. The model achieves 99.48%, 99.20%, 99.34%, and 99.27% accuracy, precision, recall, and F1-score on the corn dataset and 98.46%, 98.91%, 95.39%, and 97.01% on the potato dataset, respectively.

Title: Few-Shot Learning for Dynamic Operations of Automated Electric Taxi Fleets under Evolving Charging Infrastructure: A Meta-Deep Reinforcement Learning Approach

• Authors: Xiaozhuang Li, Xindi Tang, Fang He
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21312
• Pdf URL: https://arxiv.org/pdf/2601.21312
• Copy Paste: [[2601.21312]] Few-Shot Learning for Dynamic Operations of Automated Electric Taxi Fleets under Evolving Charging Infrastructure: A Meta-Deep Reinforcement Learning Approach(https://arxiv.org/abs/2601.21312)
• Keywords: robust
• Abstract: With the rapid expansion of electric vehicles (EVs) and charging infrastructure, the effective management of Autonomous Electric Taxi (AET) fleets faces a critical challenge in environments with dynamic and uncertain charging availability. While most existing research assumes a static charging network, this simplification creates a significant gap between theoretical models and real-world operations. To bridge this gap, we propose GAT-PEARL, a novel meta-reinforcement learning framework that learns an adaptive operational policy. Our approach integrates a graph attention network (GAT) to effectively extract robust spatial representations under infrastructure layouts and model the complex spatiotemporal relationships of the urban environment, and employs probabilistic embeddings for actor-critic reinforcement learning (PEARL) to enable rapid, inference-based adaptation to changes in charging network layouts without retraining. Through extensive simulations on real-world data in Chengdu, China, we demonstrate that GAT-PEARL significantly outperforms conventional reinforcement learning baselines, showing superior generalization to unseen infrastructure layouts and achieving higher overall operational efficiency in dynamic settings.

Title: Distributionally Robust Classification for Multi-source Unsupervised Domain Adaptation

• Authors: Seonghwi Kim, Sung Ho Jo, Wooseok Ha, Minwoo Chae
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21315
• Pdf URL: https://arxiv.org/pdf/2601.21315
• Copy Paste: [[2601.21315]] Distributionally Robust Classification for Multi-source Unsupervised Domain Adaptation(https://arxiv.org/abs/2601.21315)
• Keywords: robust
• Abstract: Unsupervised domain adaptation (UDA) is a statistical learning problem when the distribution of training (source) data is different from that of test (target) data. In this setting, one has access to labeled data only from the source domain and unlabeled data from the target domain. The central objective is to leverage the source data and the unlabeled target data to build models that generalize to the target domain. Despite its potential, existing UDA approaches often struggle in practice, particularly in scenarios where the target domain offers only limited unlabeled data or spurious correlations dominate the source domain. To address these challenges, we propose a novel distributionally robust learning framework that models uncertainty in both the covariate distribution and the conditional label distribution. Our approach is motivated by the multi-source domain adaptation setting but is also directly applicable to the single-source scenario, making it versatile in practice. We develop an efficient learning algorithm that can be seamlessly integrated with existing UDA methods. Extensive experiments under various distribution shift scenarios show that our method consistently outperforms strong baselines, especially when target data are extremely scarce.

Title: Adversarial Vulnerability Transcends Computational Paradigms: Feature Engineering Provides No Defense Against Neural Adversarial Transfer

• Authors: Achraf Hsain, Ahmed Abdelkader, Emmanuel Baldwin Mbaya, Hamoud Aljamaan
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21323
• Pdf URL: https://arxiv.org/pdf/2601.21323
• Copy Paste: [[2601.21323]] Adversarial Vulnerability Transcends Computational Paradigms: Feature Engineering Provides No Defense Against Neural Adversarial Transfer(https://arxiv.org/abs/2601.21323)
• Keywords: security, protect, defense, attack, extraction
• Abstract: Deep neural networks are vulnerable to adversarial examples--inputs with imperceptible perturbations causing misclassification. While adversarial transfer within neural networks is well-documented, whether classical ML pipelines using handcrafted features inherit this vulnerability when attacked via neural surrogates remains unexplored. Feature engineering creates information bottlenecks through gradient quantization and spatial binning, potentially filtering high-frequency adversarial signals. We evaluate this hypothesis through the first comprehensive study of adversarial transfer from DNNs to HOG-based classifiers. Using VGG16 as a surrogate, we generate FGSM and PGD adversarial examples and test transfer to four classical classifiers (KNN, Decision Tree, Linear SVM, Kernel SVM) and a shallow neural network across eight HOG configurations on CIFAR-10. Our results strongly refute the protective hypothesis: all classifiers suffer 16.6%-59.1% relative accuracy drops, comparable to neural-to-neural transfer. More surprisingly, we discover attack hierarchy reversal--contrary to patterns where iterative PGD dominates FGSM within neural networks, FGSM causes greater degradation than PGD in 100% of classical ML cases, suggesting iterative attacks overfit to surrogate-specific features that don't survive feature extraction. Block normalization provides partial but insufficient mitigation. These findings demonstrate that adversarial vulnerability is not an artifact of end-to-end differentiability but a fundamental property of image classification systems, with implications for security-critical deployments across computational paradigms.

Title: Do Pathology Foundation Models Encode Disease Progression? A Pseudotime Analysis of Visual Representations

• Authors: Pritika Vig (1 and 2), Ren-Chin Wu (3), William Lotter (2, 4 and 5) ((1) Massachusetts Institute of Technology, (2) Department of Data Science, Dana-Farber Cancer Institute, (3) Department of Pathology, Dana-Farber Cancer Institute, (4) Brigham and Women's Hospital, (5) Harvard Medical School)
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21334
• Pdf URL: https://arxiv.org/pdf/2601.21334
• Copy Paste: [[2601.21334]] Do Pathology Foundation Models Encode Disease Progression? A Pseudotime Analysis of Visual Representations(https://arxiv.org/abs/2601.21334)
• Keywords: robust, diffusion
• Abstract: Vision foundation models trained on discretely sampled images achieve strong performance on classification benchmarks, yet whether their representations encode the continuous processes underlying their training data remains unclear. This question is especially pertinent in computational pathology, where we posit that models whose latent representations implicitly capture continuous disease progression may better reflect underlying biology, support more robust generalization, and enable quantitative analyses of features associated with disease transitions. Using diffusion pseudotime, a method developed to infer developmental trajectories from single-cell transcriptomics, we probe whether foundation models organize disease states along coherent progression directions in representation space. Across four cancer progressions and six models, we find that all pathology-specific models recover trajectory orderings significantly exceeding null baselines, with vision-only models achieving the highest fidelities $(\tau > 0.78$ on CRC-Serrated). Model rankings by trajectory fidelity on reference diseases strongly predict few-shot classification performance on held-out diseases ($\rho = 0.92$), and exploratory analysis shows cell-type composition varies smoothly along inferred trajectories in patterns consistent with known stromal remodeling. Together, these results demonstrate that vision foundation models can implicitly learn to represent continuous processes from independent static observations, and that trajectory fidelity provides a complementary measure of representation quality beyond downstream performance. While demonstrated in pathology, this framework could be applied to other domains where continuous processes are observed through static snapshots.

Title: Dynamical Adapter Fusion: Constructing A Global Adapter for Pre-Trained Model-based Class-Incremental Learning

• Authors: Ruiqi Liu, Boyu Diao, Zijia An, Zhulin An, Fei Wang, Yongjun Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21341
• Pdf URL: https://arxiv.org/pdf/2601.21341
• Copy Paste: [[2601.21341]] Dynamical Adapter Fusion: Constructing A Global Adapter for Pre-Trained Model-based Class-Incremental Learning(https://arxiv.org/abs/2601.21341)
• Keywords: robust
• Abstract: Class-Incremental Learning (CIL) requires models to continuously acquire new classes without forgetting previously learned ones. A dominant paradigm involves freezing a pre-trained model and training lightweight, task-specific adapters. However, maintaining task-specific parameters hinders knowledge transfer and incurs high retrieval costs, while naive parameter fusion often leads to destructive interference and catastrophic forgetting. To address these challenges, we propose Dynamical Adapter Fusion (DAF) to construct a single robust global adapter. Grounded in the PAC-Bayes theorem, we derive a fusion mechanism that explicitly integrates three components: the optimized task-specific adapter parameters, the previous global adapter parameters, and the initialization parameters. We utilize the Taylor expansion of the loss function to derive the optimal fusion coefficients, dynamically achieving the best balance between stability and plasticity. Furthermore, we propose a Robust Initialization strategy to effectively capture global knowledge patterns. Experiments on multiple CIL benchmarks demonstrate that DAF achieves state-of-the-art (SOTA) performance.

Title: Self-Improving Pretraining: using post-trained models to pretrain better models

• Authors: Ellen Xiaoqing Tan, Shehzaad Dhuliawala, Jing Xu, Ping Yu, Sainbayar Sukhbaatar, Jason Weston, Olga Golovneva
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21343
• Pdf URL: https://arxiv.org/pdf/2601.21343
• Copy Paste: [[2601.21343]] Self-Improving Pretraining: using post-trained models to pretrain better models(https://arxiv.org/abs/2601.21343)
• Keywords: large language model
• Abstract: Ensuring safety, factuality and overall quality in the generations of large language models is a critical challenge, especially as these models are increasingly deployed in real-world applications. The prevailing approach to addressing these issues involves collecting expensive, carefully curated datasets and applying multiple stages of fine-tuning and alignment. However, even this complex pipeline cannot guarantee the correction of patterns learned during pretraining. Therefore, addressing these issues during pretraining is crucial, as it shapes a model's core behaviors and prevents unsafe or hallucinated outputs from becoming deeply embedded. To tackle this issue, we introduce a new pretraining method that streams documents and uses reinforcement learning (RL) to improve the next K generated tokens at each step. A strong, post-trained model judges candidate generations -- including model rollouts, the original suffix, and a rewritten suffix -- for quality, safety, and factuality. Early in training, the process relies on the original and rewritten suffixes; as the model improves, RL rewards high-quality rollouts. This approach builds higher quality, safer, and more factual models from the ground up. In experiments, our method gives 36.2% and 18.5% relative improvements over standard pretraining in terms of factuality and safety, and up to 86.3% win rate improvements in overall generation quality.

Title: Memorization Control in Diffusion Models from Denoising-centric Perspective

• Authors: Thuy Phuong Vu, Mai Viet Hoang Do, Minhhuy Le, Dinh-Cuong Hoang, Phan Xuan Tan
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21348
• Pdf URL: https://arxiv.org/pdf/2601.21348
• Copy Paste: [[2601.21348]] Memorization Control in Diffusion Models from Denoising-centric Perspective(https://arxiv.org/abs/2601.21348)
• Keywords: diffusion
• Abstract: Controlling memorization in diffusion models is critical for applications that require generated data to closely match the training distribution. Existing approaches mainly focus on data centric or model centric modifications, treating the diffusion model as an isolated predictor. In this paper, we study memorization in diffusion models from a denoising centric perspective. We show that uniform timestep sampling leads to unequal learning contributions across denoising steps due to differences in signal to noise ratio, which biases training toward memorization. To address this, we propose a timestep sampling strategy that explicitly controls where learning occurs along the denoising trajectory. By adjusting the width of the confidence interval, our method provides direct control over the memorization generalization trade off. Experiments on image and 1D signal generation tasks demonstrate that shifting learning emphasis toward later denoising steps consistently reduces memorization and improves distributional alignment with training data, validating the generality and effectiveness of our approach.

Title: Factored Causal Representation Learning for Robust Reward Modeling in RLHF

• Authors: Yupei Yang, Lin Yang, Wanxi Deng, Lin Qu, Fan Feng, Biwei Huang, Shikui Tu, Lei Xu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21350
• Pdf URL: https://arxiv.org/pdf/2601.21350
• Copy Paste: [[2601.21350]] Factored Causal Representation Learning for Robust Reward Modeling in RLHF(https://arxiv.org/abs/2601.21350)
• Keywords: robust, large language model
• Abstract: A reliable reward model is essential for aligning large language models with human preferences through reinforcement learning from human feedback. However, standard reward models are susceptible to spurious features that are not causally related to human labels. This can lead to reward hacking, where high predicted reward does not translate into better behavior. In this work, we address this problem from a causal perspective by proposing a factored representation learning framework that decomposes the model's contextual embedding into (1) causal factors that are sufficient for reward prediction and (2) non-causal factors that capture reward-irrelevant attributes such as length or sycophantic bias. The reward head is then constrained to depend only on the causal component. In addition, we introduce an adversarial head trained to predict reward from the non-causal factors, while applying gradient reversal to discourage them from encoding reward-relevant information. Experiments on both mathematical and dialogue tasks demonstrate that our method learns more robust reward models and consistently improves downstream RLHF performance over state-of-the-art baselines. Analyses on length and sycophantic bias further validate the effectiveness of our method in mitigating reward hacking behaviors.

Title: SecIC3: Customizing IC3 for Hardware Security Verification

• Authors: Qinhan Tan, Akash Gaonkar, Yu-Wei Fan, Aarti Gupta, Sharad Malik
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2601.21353
• Pdf URL: https://arxiv.org/pdf/2601.21353
• Copy Paste: [[2601.21353]] SecIC3: Customizing IC3 for Hardware Security Verification(https://arxiv.org/abs/2601.21353)
• Keywords: security
• Abstract: Recent years have seen significant advances in using formal verification to check hardware security properties. Of particular practical interest are checking confidentiality and integrity of secrets, by checking that there is no information flow between the secrets and observable outputs. A standard method for checking information flow is to translate the corresponding non-interference hyperproperty into a safety property on a self-composition of the design, which has two copies of the design composed together. Although prior efforts have aimed to reduce the size of the self-composed design, there are no state-of-the-art model checkers that exploit their special structure for hardware security verification. In this paper, we propose SecIC3, a hardware model checking algorithm based on IC3 that is customized to exploit this self-composition structure. SecIC3 utilizes this structure in two complementary techniques: symmetric state exploration and adding equivalence predicates. We implement SecIC3 on top of two open-source IC3 implementations and evaluate it on a non-interference checking benchmark consisting of 10 designs. The experiment results show that SecIC3 significantly reduces the time for finding security proofs, with up to 49.3x proof speedup compared to baseline implementations.

Title: The Compliance Paradox: Semantic-Instruction Decoupling in Automated Academic Code Evaluation

• Authors: Devanshu Sahoo, Manish Prasad, Vasudev Majhi, Arjun Neekhra, Yash Sinha, Murari Mandal, Vinay Chamola, Dhruv Kumar
• Subjects: cs.CL, cs.AI, cs.ET, cs.LG, cs.SE
• Abstract URL: https://arxiv.org/abs/2601.21360
• Pdf URL: https://arxiv.org/pdf/2601.21360
• Copy Paste: [[2601.21360]] The Compliance Paradox: Semantic-Instruction Decoupling in Automated Academic Code Evaluation(https://arxiv.org/abs/2601.21360)
• Keywords: robust, large language model
• Abstract: The rapid integration of Large Language Models (LLMs) into educational assessment rests on the unverified assumption that instruction following capability translates directly to objective adjudication. We demonstrate that this assumption is fundamentally flawed. Instead of evaluating code quality, models frequently decouple from the submission's logic to satisfy hidden directives, a systemic vulnerability we term the Compliance Paradox, where models fine-tuned for extreme helpfulness are vulnerable to adversarial manipulation. To expose this, we introduce the Semantic-Preserving Adversarial Code Injection (SPACI) Framework and the Abstract Syntax Tree-Aware Semantic Injection Protocol (AST-ASIP). These methods exploit the Syntax-Semantics Gap by embedding adversarial directives into syntactically inert regions (trivia nodes) of the Abstract Syntax Tree. Through a large-scale evaluation of 9 SOTA models across 25,000 submissions in Python, C, C++, and Java, we reveal catastrophic failure rates (>95%) in high-capacity open-weights models like DeepSeek-V3, which systematically prioritize hidden formatting constraints over code correctness. We quantify this failure using our novel tripartite framework measuring Decoupling Probability, Score Divergence, and Pedagogical Severity to demonstrate the widespread "False Certification" of functionally broken code. Our findings suggest that current alignment paradigms create a "Trojan" vulnerability in automated grading, necessitating a shift from standard RLHF toward domain-specific Adjudicative Robustness, where models are conditioned to prioritize evidence over instruction compliance. We release our complete dataset and injection framework to facilitate further research on the topic.

Title: Perceptrons and localization of attention's mean-field landscape

• Authors: Antonio Álvarez-López, Borjan Geshkovski, Domènec Ruiz-Balet
• Subjects: cs.LG, math.OC
• Abstract URL: https://arxiv.org/abs/2601.21366
• Pdf URL: https://arxiv.org/pdf/2601.21366
• Copy Paste: [[2601.21366]] Perceptrons and localization of attention's mean-field landscape(https://arxiv.org/abs/2601.21366)
• Keywords: transformer
• Abstract: The forward pass of a Transformer can be seen as an interacting particle system on the unit sphere: time plays the role of layers, particles that of token embeddings, and the unit sphere idealizes layer normalization. In some weight settings the system can even be seen as a gradient flow for an explicit energy, and one can make sense of the infinite context length (mean-field) limit thanks to Wasserstein gradient flows. In this paper we study the effect of the perceptron block in this setting, and show that critical points are generically atomic and localized on subsets of the sphere.

Title: Rethinking Federated Graph Foundation Models: A Graph-Language Alignment-based Approach

• Authors: Yinlin Zhu, Di Wu, Xianzhi Zhang, Yuming Ai, Xunkai Li, Miao Hu, Guocong Quan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21369
• Pdf URL: https://arxiv.org/pdf/2601.21369
• Copy Paste: [[2601.21369]] Rethinking Federated Graph Foundation Models: A Graph-Language Alignment-based Approach(https://arxiv.org/abs/2601.21369)
• Keywords: privacy, robust, federate
• Abstract: Recent studies of federated graph foundational models (FedGFMs) break the idealized and untenable assumption of having centralized data storage to train graph foundation models, and accommodate the reality of distributed, privacy-restricted data silos. Despite their simplicity and intuition, existing studies that project aligned generalizable knowledge onto a discrete token space via vector-quantized backbones suffer from irreversible knowledge loss during the quantization process. In this context, we argue that reconciling the semantic-structural orthogonality and integrity between pre-trained language models (PLMs) and graph neural networks (GNNs) is paramount for developing effective FedGFMs while simultaneously mitigating the severe data heterogeneity and communication constraints inherent in distributed, resource-limited environments. To address these issues, we propose FedGALA (Federated Graph And Language Alignment), a framework that resolves graph-based semantic-structural orthogonality and integrity in federated settings by employing unsupervised contrastive learning to align GNNs and frozen PLMs within a continuous embedding space, thereby capturing robust, transferable general knowledge. Subsequently, FedGALA leverages a communication-efficient prompt tuning mechanism to steer these pre-aligned encoders and frozen PLMs, facilitating effective adaptation to diverse downstream tasks while circumventing the prohibitive overhead of full-parameter fine-tuning. The comprehensive experiments validate that FedGALA outperforms all competitive baselines across multi-domain datasets on multiple tasks with up to 14.37% performance improvement.

Title: Towards Geometry-Aware and Motion-Guided Video Human Mesh Recovery

• Authors: Hongjun Chen, Huan Zheng, Wencheng Han, Jianbing Shen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21376
• Pdf URL: https://arxiv.org/pdf/2601.21376
• Copy Paste: [[2601.21376]] Towards Geometry-Aware and Motion-Guided Video Human Mesh Recovery(https://arxiv.org/abs/2601.21376)
• Keywords: robust
• Abstract: Existing video-based 3D Human Mesh Recovery (HMR) methods often produce physically implausible results, stemming from their reliance on flawed intermediate 3D pose anchors and their inability to effectively model complex spatiotemporal dynamics. To overcome these deep-rooted architectural problems, we introduce HMRMamba, a new paradigm for HMR that pioneers the use of Structured State Space Models (SSMs) for their efficiency and long-range modeling prowess. Our framework is distinguished by two core contributions. First, the Geometry-Aware Lifting Module, featuring a novel dual-scan Mamba architecture, creates a robust foundation for reconstruction. It directly grounds the 2D-to-3D pose lifting process with geometric cues from image features, producing a highly reliable 3D pose sequence that serves as a stable anchor. Second, the Motion-guided Reconstruction Network leverages this anchor to explicitly process kinematic patterns over time. By injecting this crucial temporal awareness, it significantly enhances the final mesh's coherence and robustness, particularly under occlusion and motion blur. Comprehensive evaluations on 3DPW, MPI-INF-3DHP, and Human3.6M benchmarks confirm that HMRMamba sets a new state-of-the-art, outperforming existing methods in both reconstruction accuracy and temporal consistency while offering superior computational efficiency.

Title: RerouteGuard: Understanding and Mitigating Adversarial Risks for LLM Routing

• Authors: Wenhui Zhang, Huiyu Xu, Zhibo Wang, Zhichao Li, Zeqing He, Xuelin Wei, Kui Ren
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2601.21380
• Pdf URL: https://arxiv.org/pdf/2601.21380
• Copy Paste: [[2601.21380]] RerouteGuard: Understanding and Mitigating Adversarial Risks for LLM Routing(https://arxiv.org/abs/2601.21380)
• Keywords: security, attack, interpretability
• Abstract: Recent advancements in multi-model AI systems have leveraged LLM routers to reduce computational cost while maintaining response quality by assigning queries to the most appropriate model. However, as classifiers, LLM routers are vulnerable to novel adversarial attacks in the form of LLM rerouting, where adversaries prepend specially crafted triggers to user queries to manipulate routing decisions. Such attacks can lead to increased computational cost, degraded response quality, and even bypass safety guardrails, yet their security implications remain largely underexplored. In this work, we bridge this gap by systematizing LLM rerouting threats based on the adversary's objectives (i.e., cost escalation, quality hijacking, and safety bypass) and knowledge. Based on the threat taxonomy, we conduct a measurement study of real-world LLM routing systems against existing LLM rerouting attacks. The results reveal that existing routing systems are vulnerable to rerouting attacks, especially in the cost escalation scenario. We then characterize existing rerouting attacks using interpretability techniques, revealing that they exploit router decision boundaries through confounder gadgets that prepend queries to force misrouting. To mitigate these risks, we introduce RerouteGuard, a flexible and scalable guardrail framework for LLM rerouting. RerouteGuard filters adversarial rerouting prompts via dynamic embedding-based detection and adaptive thresholding. Extensive evaluations in three attack settings and four benchmarks demonstrate that RerouteGuard achieves over 99% detection accuracy against state-of-the-art rerouting attacks, while maintaining negligible impact on legitimate queries. The experimental results indicate that RerouteGuard offers a principled and practical solution for safeguarding multi-model AI systems against adversarial rerouting.

Title: DA-SPS: A Dual-stage Network based on Singular Spectrum Analysis, Patching-strategy and Spearman-correlation for Multivariate Time-series Prediction

• Authors: Tianhao Zhang, Shusen Ma, Yu Kang, Yun-Bo Zhao
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21381
• Pdf URL: https://arxiv.org/pdf/2601.21381
• Copy Paste: [[2601.21381]] DA-SPS: A Dual-stage Network based on Singular Spectrum Analysis, Patching-strategy and Spearman-correlation for Multivariate Time-series Prediction(https://arxiv.org/abs/2601.21381)
• Keywords: extraction
• Abstract: Multivariate time-series forecasting, as a typical problem in the field of time series prediction, has a wide range of applications in weather forecasting, traffic flow prediction, and other scenarios. However, existing works do not effectively consider the impact of extraneous variables on the prediction of the target variable. On the other hand, they fail to fully extract complex sequence information based on various time patterns of the sequences. To address these drawbacks, we propose a DA-SPS model, which adopts different modules for feature extraction based on the information characteristics of different variables. DA-SPS mainly consists of two stages: the target variable processing stage (TVPS) and the extraneous variables processing stage (EVPS). In TVPS, the model first uses Singular Spectrum Analysis (SSA) to process the target variable sequence and then uses Long Short-Term Memory (LSTM) and P-Conv-LSTM which deploys a patching strategy to extract features from trend and seasonality components, respectively. In EVPS, the model filters extraneous variables that have a strong correlation with the target variate by using Spearman correlation analysis and further analyses them using the L-Attention module which consists of LSTM and attention mechanism. Finally, the results obtained by TVPS and EVPS are combined through weighted summation and linear mapping to produce the final prediction. The results on four public datasets demonstrate that the DA-SPS model outperforms existing state-of-the-art methods. Additionally, its performance in real-world scenarios is further validated using a private dataset collected by ourselves, which contains the test items' information on laptop motherboards.

Title: User-Centric Evidence Ranking for Attribution and Fact Verification

• Authors: Guy Alt, Eran Hirsch, Serwar Basch, Ido Dagan, Oren Glickman
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21387
• Pdf URL: https://arxiv.org/pdf/2601.21387
• Copy Paste: [[2601.21387]] User-Centric Evidence Ranking for Attribution and Fact Verification(https://arxiv.org/abs/2601.21387)
• Keywords: large language model
• Abstract: Attribution and fact verification are critical challenges in natural language processing for assessing information reliability. While automated systems and Large Language Models (LLMs) aim to retrieve and select concise evidence to support or refute claims, they often present users with either insufficient or overly redundant information, leading to inefficient and error-prone verification. To address this, we propose Evidence Ranking, a novel task that prioritizes presenting sufficient information as early as possible in a ranked list. This minimizes user reading effort while still making all available evidence accessible for sequential verification. We compare two approaches for the new ranking task: one-shot ranking and incremental ranking. We introduce a new evaluation framework, inspired by information retrieval metrics, and construct a unified benchmark by aggregating existing fact verification datasets. Extensive experiments with diverse models show that incremental ranking strategies better capture complementary evidence and that LLM-based methods outperform shallower baselines, while still facing challenges in balancing sufficiency and redundancy. Compared to evidence selection, we conduct a controlled user study and demonstrate that evidence ranking both reduces reading effort and improves verification. This work provides a foundational step toward more interpretable, efficient, and user-aligned information verification systems.

Title: Learning to Optimize Job Shop Scheduling Under Structural Uncertainty

• Authors: Rui Zhang, Jianwei Niu, Xuefeng Liu, Shaojie Tang, Jing Yuan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21389
• Pdf URL: https://arxiv.org/pdf/2601.21389
• Copy Paste: [[2601.21389]] Learning to Optimize Job Shop Scheduling Under Structural Uncertainty(https://arxiv.org/abs/2601.21389)
• Keywords: fair
• Abstract: The Job-Shop Scheduling Problem (JSSP), under various forms of manufacturing uncertainty, has recently attracted considerable research attention. Most existing studies focus on parameter uncertainty, such as variable processing times, and typically adopt the actor-critic framework. In this paper, we explore a different but prevalent form of uncertainty in JSSP: structural uncertainty. Structural uncertainty arises when a job may follow one of several routing paths, and the selection is determined not by policy, but by situational factors (e.g., the quality of intermediate products) that cannot be known in advance. Existing methods struggle to address this challenge due to incorrect credit assignment: a high-quality action may be unfairly penalized if it is followed by a time-consuming path. To address this problem, we propose a novel method named UP-AAC. In contrast to conventional actor-critic methods, UP-AAC employs an asymmetric architecture. While its actor receives a standard stochastic state, the critic is crucially provided with a deterministic state reconstructed in hindsight. This design allows the critic to learn a more accurate value function, which in turn provides a lower-variance policy gradient to the actor, leading to more stable learning. In addition, we design an attention-based Uncertainty Perception Model (UPM) to enhance the actor's scheduling decisions. Extensive experiments demonstrate that our method outperforms existing approaches in reducing makespan on benchmark instances.

Title: Rectifying Geometry-Induced Similarity Distortions for Real-World Aerial-Ground Person Re-Identification

• Authors: Kailash A. Hambarde, Hugo Proença
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21405
• Pdf URL: https://arxiv.org/pdf/2601.21405
• Copy Paste: [[2601.21405]] Rectifying Geometry-Induced Similarity Distortions for Real-World Aerial-Ground Person Re-Identification(https://arxiv.org/abs/2601.21405)
• Keywords: robust
• Abstract: Aerial-ground person re-identification (AG-ReID) is fundamentally challenged by extreme viewpoint and distance discrepancies between aerial and ground cameras, which induce severe geometric distortions and invalidate the assumption of a shared similarity space across views. Existing methods primarily rely on geometry-aware feature learning or appearance-conditioned prompting, while implicitly assuming that the geometry-invariant dot-product similarity used in attention mechanisms remains reliable under large viewpoint and scale variations. We argue that this assumption does not hold. Extreme camera geometry systematically distorts the query-key similarity space and degrades attention-based matching, even when feature representations are partially aligned. To address this issue, we introduce Geometry-Induced Query-Key Transformation (GIQT), a lightweight low-rank module that explicitly rectifies the similarity space by conditioning query-key interactions on camera geometry. Rather than modifying feature representations or the attention formulation itself, GIQT adapts the similarity computation to compensate for dominant geometry-induced anisotropic distortions. Building on this local similarity rectification, we further incorporate a geometry-conditioned prompt generation mechanism that provides global, view-adaptive representation priors derived directly from camera geometry. Experiments on four aerial-ground person re-identification benchmarks demonstrate that the proposed framework consistently improves robustness under extreme and previously unseen geometric conditions, while introducing minimal computational overhead compared to state-of-the-art methods.

Title: Generation Enhances Understanding in Unified Multimodal Models via Multi-Representation Generation

• Authors: Zihan Su, Hongyang Wei, Kangrui Cen, Yong Wang, Guanhua Chen, Chun Yuan, Xiangxiang Chu
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21406
• Pdf URL: https://arxiv.org/pdf/2601.21406
• Copy Paste: [[2601.21406]] Generation Enhances Understanding in Unified Multimodal Models via Multi-Representation Generation(https://arxiv.org/abs/2601.21406)
• Keywords: segmentation
• Abstract: Unified Multimodal Models (UMMs) integrate both visual understanding and generation within a single framework. Their ultimate aspiration is to create a cycle where understanding and generation mutually reinforce each other. While recent post-training methods have successfully leveraged understanding to enhance generation, the reverse direction of utilizing generation to improve understanding remains largely unexplored. In this work, we propose UniMRG (Unified Multi-Representation Generation), a simple yet effective architecture-agnostic post-training method. UniMRG enhances the understanding capabilities of UMMs by incorporating auxiliary generation tasks. Specifically, we train UMMs to generate multiple intrinsic representations of input images, namely pixel (reconstruction), depth (geometry), and segmentation (structure), alongside standard visual understanding objectives. By synthesizing these diverse representations, UMMs capture complementary information regarding appearance, spatial relations, and structural layout. Consequently, UMMs develop a deeper and more comprehensive understanding of visual inputs. Extensive experiments across diverse UMM architectures demonstrate that our method notably enhances fine-grained perception, reduces hallucinations, and improves spatial understanding, while simultaneously boosting generation capabilities.

Title: Revisiting Diffusion Model Predictions Through Dimensionality

• Authors: Qing Jin, Chaoyang Wang
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21419
• Pdf URL: https://arxiv.org/pdf/2601.21419
• Copy Paste: [[2601.21419]] Revisiting Diffusion Model Predictions Through Dimensionality(https://arxiv.org/abs/2601.21419)
• Keywords: diffusion, generative
• Abstract: Recent advances in diffusion and flow matching models have highlighted a shift in the preferred prediction target -- moving from noise ($\varepsilon$) and velocity (v) to direct data (x) prediction -- particularly in high-dimensional settings. However, a formal explanation of why the optimal target depends on the specific properties of the data remains elusive. In this work, we provide a theoretical framework based on a generalized prediction formulation that accommodates arbitrary output targets, of which $\varepsilon$-, v-, and x-prediction are special cases. We derive the analytical relationship between data's geometry and the optimal prediction target, offering a rigorous justification for why x-prediction becomes superior when the ambient dimension significantly exceeds the data's intrinsic dimension. Furthermore, while our theory identifies dimensionality as the governing factor for the optimal prediction target, the intrinsic dimension of manifold-bound data is typically intractable to estimate in practice. To bridge this gap, we propose k-Diff, a framework that employs a data-driven approach to learn the optimal prediction parameter k directly from data, bypassing the need for explicit dimension estimation. Extensive experiments in both latent-space and pixel-space image generation demonstrate that k-Diff consistently outperforms fixed-target baselines across varying architectures and data scales, providing a principled and automated approach to enhancing generative performance.

Title: ConceptMoE: Adaptive Token-to-Concept Compression for Implicit Compute Allocation

• Authors: Zihao Huang, Jundong Zhou, Xingwei Qu, Qiyang Min, Ge Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21420
• Pdf URL: https://arxiv.org/pdf/2601.21420
• Copy Paste: [[2601.21420]] ConceptMoE: Adaptive Token-to-Concept Compression for Implicit Compute Allocation(https://arxiv.org/abs/2601.21420)
• Keywords: large language model
• Abstract: Large language models allocate uniform computation across all tokens, ignoring that some sequences are trivially predictable while others require deep reasoning. We introduce ConceptMoE, which dynamically merges semantically similar tokens into concept representations, performing implicit token-level compute allocation. A learnable chunk module identifies optimal boundaries by measuring inter-token similarity, compressing sequences by a target ratio $R$ before they enter the compute-intensive concept model. Crucially, the MoE architecture enables controlled evaluation: we reallocate saved computation to match baseline activated FLOPs (excluding attention map computation) and total parameters, isolating genuine architectural benefits. Under these conditions, ConceptMoE consistently outperforms standard MoE across language and vision-language tasks, achieving +0.9 points on language pretraining, +2.3 points on long context understanding, and +0.6 points on multimodal benchmarks. When converting pretrained MoE during continual training with layer looping, gains reach +5.5 points, demonstrating practical applicability. Beyond performance, ConceptMoE reduces attention computation by up to $R^2\times$ and KV cache by $R\times$. At $R=2$, empirical measurements show prefill speedups reaching 175\% and decoding speedups up to 117\% on long sequences. The minimal architectural modifications enable straightforward integration into existing MoE, demonstrating that adaptive concept-level processing fundamentally improves both effectiveness and efficiency of large language models.

Title: From Implicit Ambiguity to Explicit Solidity: Diagnosing Interior Geometric Degradation in Neural Radiance Fields for Dense 3D Scene Understanding

• Authors: Jiangsan Zhao, Jakob Geipel, Kryzysztof Kusnierek
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21421
• Pdf URL: https://arxiv.org/pdf/2601.21421
• Copy Paste: [[2601.21421]] From Implicit Ambiguity to Explicit Solidity: Diagnosing Interior Geometric Degradation in Neural Radiance Fields for Dense 3D Scene Understanding(https://arxiv.org/abs/2601.21421)
• Keywords: robust, segmentation
• Abstract: Neural Radiance Fields (NeRFs) have emerged as a powerful paradigm for multi-view reconstruction, complementing classical photogrammetric pipelines based on Structure-from-Motion (SfM) and Multi-View Stereo (MVS). However, their reliability for quantitative 3D analysis in dense, self-occluding scenes remains poorly understood. In this study, we identify a fundamental failure mode of implicit density fields under heavy occlusion, which we term Interior Geometric Degradation (IGD). We show that transmittance-based volumetric optimization satisfies photometric supervision by reconstructing hollow or fragmented structures rather than solid interiors, leading to systematic instance undercounting. Through controlled experiments on synthetic datasets with increasing occlusion, we demonstrate that state-of-the-art mask-supervised NeRFs saturate at approximately 89% instance recovery in dense scenes, despite improved surface coherence and mask quality. To overcome this limitation, we introduce an explicit geometric pipeline based on Sparse Voxel Rasterization (SVRaster), initialized from SfM feature geometry. By projecting 2D instance masks onto an explicit voxel grid and enforcing geometric separation via recursive splitting, our approach preserves physical solidity and achieves a 95.8% recovery rate in dense clusters. A sensitivity analysis using degraded segmentation masks further shows that explicit SfM-based geometry is substantially more robust to supervision failure, recovering 43% more instances than implicit baselines. These results demonstrate that explicit geometric priors are a prerequisite for reliable quantitative analysis in highly self-occluding 3D scenes.

Title: MultiModal Fine-tuning with Synthetic Captions

• Authors: Shohei Enomoto, Shin'ya Yamaguchi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21426
• Pdf URL: https://arxiv.org/pdf/2601.21426
• Copy Paste: [[2601.21426]] MultiModal Fine-tuning with Synthetic Captions(https://arxiv.org/abs/2601.21426)
• Keywords: large language model
• Abstract: In this paper, we address a fundamental gap between pre-training and fine-tuning of deep neural networks: while pre-training has shifted from unimodal to multimodal learning with enhanced visual understanding, fine-tuning predominantly remains unimodal, limiting the benefits of rich pre-trained representations. To bridge this gap, we propose a novel approach that transforms unimodal datasets into multimodal ones using Multimodal Large Language Models (MLLMs) to generate synthetic image captions for fine-tuning models with a multimodal objective. Our method employs carefully designed prompts incorporating class labels and domain context to produce high-quality captions tailored for classification tasks. Furthermore, we introduce a supervised contrastive loss function that explicitly encourages clustering of same-class representations during fine-tuning, along with a new inference technique that leverages class-averaged text embeddings from multiple synthetic captions per image. Extensive experiments across 13 image classification benchmarks demonstrate that our approach outperforms baseline methods, with particularly significant improvements in few-shot learning scenarios. Our work establishes a new paradigm for dataset enhancement that effectively bridges the gap between multimodal pre-training and fine-tuning. Our code is available at this https URL.

Title: From Consistency to Complementarity: Aligned and Disentangled Multi-modal Learning for Time Series Understanding and Reasoning

• Authors: Hang Ni, Weijia Zhang, Fei Wang, Zezhi Shao, Hao Liu
• Subjects: cs.LG, cs.AI, cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21436
• Pdf URL: https://arxiv.org/pdf/2601.21436
• Copy Paste: [[2601.21436]] From Consistency to Complementarity: Aligned and Disentangled Multi-modal Learning for Time Series Understanding and Reasoning(https://arxiv.org/abs/2601.21436)
• Keywords: robust, large language model
• Abstract: Advances in multi-modal large language models (MLLMs) have inspired time series understanding and reasoning tasks, that enable natural language querying over time series, producing textual analyses of complex temporal dynamics. Recent attempts hybridize numerical time series with their visualized plots, facilitating precise value reasoning and visual structure comprehension for comprehensive time series understanding of MLLMs. However, effective cross-modal integration remains challenging due to fine-grained temporal misalignment across modalities and severe entanglement between shared and modality-specific semantics, which hinder localized interpretation and complementary reasoning. To address these issues, we propose MADI, a multi-modal LLM enhanced with fine-grained alignment and disentangled interaction, featuring (1) Patch-level Alignment, which enforces physically grounded fine-grained correspondence across heterogeneous modalities, (2) Discrete Disentangled Interaction, which separates modality-common semantics into compact discrete latents and adaptively synergizes the purified modality-unique information, and (3) Critical-token Highlighting, which emphasizes informative, query-relevant signals for robust reasoning. Experiments on synthetic and real-world benchmarks show that MADI consistently outperforms general-purpose LLMs and time-series-specialized MLLMs.

Title: Accurate Network Traffic Matrix Prediction via LEAD: an LLM-Enhanced Adapter-Based Conditional Diffusion Model

• Authors: Yu Sun, Yaqiong Liu, Nan Cheng, Jiayuan Li, Zihan Jia, Xialin Du, Mugen Peng
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21437
• Pdf URL: https://arxiv.org/pdf/2601.21437
• Copy Paste: [[2601.21437]] Accurate Network Traffic Matrix Prediction via LEAD: an LLM-Enhanced Adapter-Based Conditional Diffusion Model(https://arxiv.org/abs/2601.21437)
• Keywords: diffusion, large language model
• Abstract: Driven by the evolution toward 6G and AI-native edge intelligence, network operations increasingly require predictive and risk-aware adaptation under stringent computation and latency constraints. Network Traffic Matrix (TM), which characterizes flow volumes between nodes, is a fundamental signal for proactive traffic engineering. However, accurate TM forecasting remains challenging due to the stochastic, non-linear, and bursty nature of network dynamics. Existing discriminative models often suffer from over-smoothing and provide limited uncertainty awareness, leading to poor fidelity under extreme bursts. To address these limitations, we propose LEAD, a Large Language Model (LLM)-Enhanced Adapter-based conditional Diffusion model. First, LEAD adopts a "Traffic-to-Image" paradigm to transform traffic matrices into RGB images, enabling global dependency modeling via vision backbones. Then, we design a "Frozen LLM with Trainable Adapter" model, which efficiently captures temporal semantics with limited computational cost. Moreover, we propose a Dual-Conditioning Strategy to precisely guide a diffusion model to generate complex, dynamic network traffic matrices. Experiments on the Abilene and GEANT datasets demonstrate that LEAD outperforms all baselines. On the Abilene dataset, LEAD attains a remarkable 45.2% reduction in RMSE against the best baseline, with the error margin rising only marginally from 0.1098 at one-step to 0.1134 at 20-step predictions. Meanwhile, on the GEANT dataset, LEAD achieves a 0.0258 RMSE at 20-step prediction horizon which is 27.3% lower than the best baseline.

Title: Synthetic Pattern Generation and Detection of Financial Activities using Graph Autoencoders

• Authors: Francesco Zola, Lucia Muñoz, Andrea Venturi, Amaia Gil
• Subjects: cs.LG, cs.CE, cs.ET
• Abstract URL: https://arxiv.org/abs/2601.21446
• Pdf URL: https://arxiv.org/pdf/2601.21446
• Copy Paste: [[2601.21446]] Synthetic Pattern Generation and Detection of Financial Activities using Graph Autoencoders(https://arxiv.org/abs/2601.21446)
• Keywords: privacy
• Abstract: Illicit financial activities such as money laundering often manifest through recurrent topological patterns in transaction networks. Detecting these patterns automatically remains challenging due to the scarcity of labeled real-world data and strict privacy constraints. To address this, we investigate whether Graph Autoencoders (GAEs) can effectively learn and distinguish topological patterns that mimic money laundering operations when trained on synthetic data. The analysis consists of two phases: (i) data generation, where synthetic samples are created for seven well-known illicit activity patterns using parametrized generators that preserve structural consistency while introducing realistic variability; and (ii) model training and validation, where separate GAEs are trained on each pattern without explicit labels, relying solely on reconstruction error as an indicator of learned structure. We compare three GAE implementations based on three distinct convolutional layers: Graph Convolutional (GAE-GCN), GraphSAGE (GAE-SAGE), and Graph Attention Network (GAE-GAT). Experimental results show that GAE-GCN achieves the most consistent reconstruction performance across patterns, while GAE-SAGE and GAE-GAT exhibit competitive results only in few specific patterns. These findings suggest that graph-based representation learning on synthetic data provides a viable path toward developing AI-driven tools for detecting illicit behaviors, overcoming the limitations of financial datasets.

Title: SAGE: Sequence-level Adaptive Gradient Evolution for Generative Recommendation

• Authors: Yu Xie, Xing Kai Ren, Ying Qi, Hu Yao
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21452
• Pdf URL: https://arxiv.org/pdf/2601.21452
• Copy Paste: [[2601.21452]] SAGE: Sequence-level Adaptive Gradient Evolution for Generative Recommendation(https://arxiv.org/abs/2601.21452)
• Keywords: generative, large language model
• Abstract: While works such as OneRec have validated the scaling laws of Large Language Models (LLMs) in recommender systems, they rely on a cumbersome separate vocabulary. This dependency prevents the model architecture from reusing native LLM vocabularies, resulting in high maintenance costs and poor scalability. In response, we aim to efficiently reuse open-source LLM architectures without constructing a separate tokenization vocabulary. Furthermore, we identify that the optimization strategy of OneRec Gradient Bounded Policy Optimization (GBPO),suffers from a "Symmetric Conservatism" problem: its static gradient boundaries structurally suppress the update momentum required for cold-start items and fail to prevent diversity collapse in high-noise this http URL address this issue, we propose SAGE (Sequence-level Adaptive Gradient Evolution), a unified optimization framework tailored for list-wise generative recommendation. SAGE introduces two key innovations:(1) Sequence-level Signal Decoupling: By combining a geometric mean importance ratio with decoupled multi-objective advantages, we eliminate token-level variance and resolve the "Reward Collapse" problem. (2) Asymmetric Adaptive Dynamics: We construct a dynamic gradient manifold that applies a "Boost Factor" to high-potential cold start items to achieve super-linear updates and employs an "Entropy Aware Penalty" to break information cocoons. Theoretical analysis and empirical results demonstrate that SAGE effectively unblocks cold-start traffic and sustains recommendation diversity, all while retaining the numerical stability of GBPO.

Title: Mining Forgery Traces from Reconstruction Error: A Weakly Supervised Framework for Multimodal Deepfake Temporal Localization

• Authors: Midou Guo, Qilin Yin, Wei Lu, Xiangyang Luo, Rui Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21458
• Pdf URL: https://arxiv.org/pdf/2601.21458
• Copy Paste: [[2601.21458]] Mining Forgery Traces from Reconstruction Error: A Weakly Supervised Framework for Multimodal Deepfake Temporal Localization(https://arxiv.org/abs/2601.21458)
• Keywords: robust
• Abstract: Modern deepfakes have evolved into localized and intermittent manipulations that require fine-grained temporal localization. The prohibitive cost of frame-level annotation makes weakly supervised methods a practical necessity, which rely only on video-level labels. To this end, we propose Reconstruction-based Temporal Deepfake Localization (RT-DeepLoc), a weakly supervised temporal forgery localization framework that identifies forgeries via reconstruction errors. Our framework uses a Masked Autoencoder (MAE) trained exclusively on authentic data to learn its intrinsic spatiotemporal patterns; this allows the model to produce significant reconstruction discrepancies for forged segments, effectively providing the missing fine-grained cues for localization. To robustly leverage these indicators, we introduce a novel Asymmetric Intra-video Contrastive Loss (AICL). By focusing on the compactness of authentic features guided by these reconstruction cues, AICL establishes a stable decision boundary that enhances local discrimination while preserving generalization to unseen forgeries. Extensive experiments on large-scale datasets, including LAV-DF, demonstrate that RT-DeepLoc achieves state-of-the-art performance in weakly-supervised temporal forgery localization.

Title: L$^3$: Large Lookup Layers

• Authors: Albert Tseng, Christopher De Sa
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21461
• Pdf URL: https://arxiv.org/pdf/2601.21461
• Copy Paste: [[2601.21461]] L$^3$: Large Lookup Layers(https://arxiv.org/abs/2601.21461)
• Keywords: transformer
• Abstract: Modern sparse language models typically achieve sparsity through Mixture-of-Experts (MoE) layers, which dynamically route tokens to dense MLP "experts." However, dynamic hard routing has a number of drawbacks, such as potentially poor hardware efficiency and needing auxiliary losses for stable training. In contrast, the tokenizer embedding table, which is natively sparse, largely avoids these issues by selecting a single embedding per token at the cost of not having contextual information. In this work, we introduce the Large Lookup Layer (L$^3$), which unlocks a new axis of sparsity by generalizing embedding tables to model decoder layers. L$^3$ layers use static token-based routing to aggregate a set of learned embeddings per token in a context-dependent way, allowing the model to efficiently balance memory and compute by caching information in embeddings. L$^3$ has two main components: (1) a systems-friendly architecture that allows for fast training and CPU-offloaded inference with no overhead, and (2) an information-theoretic embedding allocation algorithm that effectively balances speed and quality. We empirically test L$^3$ by training transformers with up to 2.6B active parameters and find that L$^3$ strongly outperforms both dense models and iso-sparse MoEs in both language modeling and downstream tasks.

Title: Conversation for Non-verifiable Learning: Self-Evolving LLMs through Meta-Evaluation

• Authors: Yuan Sui, Bryan Hooi
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21464
• Pdf URL: https://arxiv.org/pdf/2601.21464
• Copy Paste: [[2601.21464]] Conversation for Non-verifiable Learning: Self-Evolving LLMs through Meta-Evaluation(https://arxiv.org/abs/2601.21464)
• Keywords: large language model
• Abstract: Training large language models (LLMs) for non-verifiable tasks, such as creative writing, dialogue, and ethical reasoning, remains challenging due to the absence of ground-truth labels. While LLM-as-Judge approaches offer a scalable alternative to human feedback, they face a fundamental limitation: performance is constrained by the evaluator's own quality. If the judge cannot recognize good solutions, it cannot provide useful training signals, and evaluation biases (e.g., favoring verbosity over quality) remain unaddressed. This motivates meta-evaluation: the ability to evaluate and improve the evaluator itself. We introduce CoNL, a framework that unifies generation, evaluation, and meta-evaluation through multi-agent self-play. Our key insight: critique quality can be measured by whether it helps others improve their solutions. In CoNL, multiple agents sharing the same policy engage in structured conversations to propose, critique, and revise solutions. Critiques that enable solution improvements earn a diagnostic reward, creating explicit supervision for meta-evaluation and enabling joint optimization of generation and judging capabilities through self-play, without external judges or ground truth. Experiments on five benchmarks show that CoNL achieves consistent improvements over self-rewarding baselines while maintaining stable training.

Title: SOUP: Token-level Single-sample Mix-policy Reinforcement Learning for Large Language Models

• Authors: Lei Yang, Wei Bi, Chenxi Sun, Renren Jin, Deyi Xiong
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21476
• Pdf URL: https://arxiv.org/pdf/2601.21476
• Copy Paste: [[2601.21476]] SOUP: Token-level Single-sample Mix-policy Reinforcement Learning for Large Language Models(https://arxiv.org/abs/2601.21476)
• Keywords: large language model
• Abstract: On-policy reinforcement learning (RL) methods widely used for language model post-training, like Group Relative Policy Optimization (GRPO), often suffer from limited exploration and early saturation due to low sampling diversity. While off-policy data can help, current approaches that mix entire trajectories cause significant policy mismatch and instability. In this work, we propose the $\textbf{S}$ingle-sample Mix-p$\textbf{O}$licy $\textbf{U}$nified $\textbf{P}$aradigm (SOUP), a framework that unifies off- and on-policy learning within individual samples at the token level. It confines off-policy influence to the prefix of a generated sequence sampled from historical policies, while the continuation is generated on-policy. Through token-level importance ratios, SOUP effectively leverages off-policy information while preserving training stability. Extensive experiments demonstrate that SOUP consistently outperforms standard on-policy training and existing off-policy extensions. Our further analysis clarifies how our fine-grained, single-sample mix-policy training can improve both exploration and final performance in LLM RL.

Title: Hypernetwork-Based Adaptive Aggregation for Multimodal Multiple-Instance Learning in Predicting Coronary Calcium Debulking

• Authors: Kaito Shiku, Ichika Seo, Tetsuya Matoba, Rissei Hino, Yasuhiro Nakano, Ryoma Bise
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21479
• Pdf URL: https://arxiv.org/pdf/2601.21479
• Copy Paste: [[2601.21479]] Hypernetwork-Based Adaptive Aggregation for Multimodal Multiple-Instance Learning in Predicting Coronary Calcium Debulking(https://arxiv.org/abs/2601.21479)
• Keywords: transformer
• Abstract: In this paper, we present the first attempt to estimate the necessity of debulking coronary artery calcifications from computed tomography (CT) images. We formulate this task as a Multiple-instance Learning (MIL) problem. The difficulty of this task lies in that physicians adjust their focus and decision criteria for device usage according to tabular data representing each patient's condition. To address this issue, we propose a hypernetwork-based adaptive aggregation transformer (HyperAdAgFormer), which adaptively modifies the feature aggregation strategy for each patient based on tabular data through a hypernetwork. The experiments using the clinical dataset demonstrated the effectiveness of HyperAdAgFormer. The code is publicly available at this https URL.

Title: DimStance: Multilingual Datasets for Dimensional Stance Analysis

• Authors: Jonas Becker, Liang-Chih Yu, Shamsuddeen Hassan Muhammad, Jan Philip Wahle, Terry Ruas, Idris Abdulmumin, Lung-Hao Lee, Wen-Ni Liu, Tzu-Mi Lin, Zhe-Yu Xu, Ying-Lung Lin, Jin Wang, Maryam Ibrahim Mukhtar, Bela Gipp, Saif M. Mohammed
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21483
• Pdf URL: https://arxiv.org/pdf/2601.21483
• Copy Paste: [[2601.21483]] DimStance: Multilingual Datasets for Dimensional Stance Analysis(https://arxiv.org/abs/2601.21483)
• Keywords: protect, large language model
• Abstract: Stance detection is an established task that classifies an author's attitude toward a specific target into categories such as Favor, Neutral, and Against. Beyond categorical stance labels, we leverage a long-established affective science framework to model stance along real-valued dimensions of valence (negative-positive) and arousal (calm-active). This dimensional approach captures nuanced affective states underlying stance expressions, enabling fine-grained stance analysis. To this end, we introduce DimStance, the first dimensional stance resource with valence-arousal (VA) annotations. This resource comprises 11,746 target aspects in 7,365 texts across five languages (English, German, Chinese, Nigerian Pidgin, and Swahili) and two domains (politics and environmental protection). To facilitate the evaluation of stance VA prediction, we formulate the dimensional stance regression task, analyze cross-lingual VA patterns, and benchmark pretrained and large language models under regression and prompting settings. Results show competitive performance of fine-tuned LLM regressors, persistent challenges in low-resource languages, and limitations of token-based generation. DimStance provides a foundation for multilingual, emotion-aware, stance analysis and benchmarking.

Title: ETS: Energy-Guided Test-Time Scaling for Training-Free RL Alignment

• Authors: Xiuyu Li, Jinkai Zhang, Mingyang Yi, Yu Li, Longqiang Wang, Yue Wang, Ju Fan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21484
• Pdf URL: https://arxiv.org/pdf/2601.21484
• Copy Paste: [[2601.21484]] ETS: Energy-Guided Test-Time Scaling for Training-Free RL Alignment(https://arxiv.org/abs/2601.21484)
• Keywords: diffusion
• Abstract: Reinforcement Learning (RL) post-training alignment for language models is effective, but also costly and unstable in practice, owing to its complicated training process. To address this, we propose a training-free inference method to sample directly from the optimal RL policy. The transition probability applied to Masked Language Modeling (MLM) consists of a reference policy model and an energy term. Based on this, our algorithm, Energy-Guided Test-Time Scaling (ETS), estimates the key energy term via online Monte Carlo, with a provable convergence rate. Moreover, to ensure practical efficiency, ETS leverages modern acceleration frameworks alongside tailored importance sampling estimators, substantially reducing inference latency while provably preserving sampling quality. Experiments on MLM (including autoregressive models and diffusion language models) across reasoning, coding, and science benchmarks show that our ETS consistently improves generation quality, validating its effectiveness and design.

Title: SimGraph: A Unified Framework for Scene Graph-Based Image Generation and Editing

• Authors: Thanh-Nhan Vo, Trong-Thuan Nguyen, Tam V. Nguyen, Minh-Triet Tran
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21498
• Pdf URL: https://arxiv.org/pdf/2601.21498
• Copy Paste: [[2601.21498]] SimGraph: A Unified Framework for Scene Graph-Based Image Generation and Editing(https://arxiv.org/abs/2601.21498)
• Keywords: diffusion, generative
• Abstract: Recent advancements in Generative Artificial Intelligence (GenAI) have significantly enhanced the capabilities of both image generation and editing. However, current approaches often treat these tasks separately, leading to inefficiencies and challenges in maintaining spatial consistency and semantic coherence between generated content and edits. Moreover, a major obstacle is the lack of structured control over object relationships and spatial arrangements. Scene graph-based methods, which represent objects and their interrelationships in a structured format, offer a solution by providing greater control over composition and interactions in both image generation and editing. To address this, we introduce SimGraph, a unified framework that integrates scene graph-based image generation and editing, enabling precise control over object interactions, layouts, and spatial coherence. In particular, our framework integrates token-based generation and diffusion-based editing within a single scene graph-driven model, ensuring high-quality and consistent results. Through extensive experiments, we empirically demonstrate that our approach outperforms existing state-of-the-art methods.

Title: MURAD: A Large-Scale Multi-Domain Unified Reverse Arabic Dictionary Dataset

• Authors: Serry Sibaee, Yasser Alhabashi, Nadia Sibai, Yara Farouk, Adel Ammar, Sawsan AlHalawani, Wadii Boulila
• Subjects: cs.CL, cs.CY, cs.DB, cs.IR
• Abstract URL: https://arxiv.org/abs/2601.21512
• Pdf URL: https://arxiv.org/pdf/2601.21512
• Copy Paste: [[2601.21512]] MURAD: A Large-Scale Multi-Domain Unified Reverse Arabic Dictionary Dataset(https://arxiv.org/abs/2601.21512)
• Keywords: extraction
• Abstract: Arabic is a linguistically and culturally rich language with a vast vocabulary that spans scientific, religious, and literary domains. Yet, large-scale lexical datasets linking Arabic words to precise definitions remain limited. We present MURAD (Multi-domain Unified Reverse Arabic Dictionary), an open lexical dataset with 96,243 word-definition pairs. The data come from trusted reference works and educational sources. Extraction used a hybrid pipeline integrating direct text parsing, optical character recognition, and automated reconstruction. This ensures accuracy and clarity. Each record aligns a target word with its standardized Arabic definition and metadata that identifies the source domain. The dataset covers terms from linguistics, Islamic studies, mathematics, physics, psychology, and engineering. It supports computational linguistics and lexicographic research. Applications include reverse dictionary modeling, semantic retrieval, and educational tools. By releasing this resource, we aim to advance Arabic natural language processing and promote reproducible research on Arabic lexical semantics.

Title: HERS: Hidden-Pattern Expert Learning for Risk-Specific Vehicle Damage Adaptation in Diffusion Models

• Authors: Teerapong Panboonyuen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21517
• Pdf URL: https://arxiv.org/pdf/2601.21517
• Copy Paste: [[2601.21517]] HERS: Hidden-Pattern Expert Learning for Risk-Specific Vehicle Damage Adaptation in Diffusion Models(https://arxiv.org/abs/2601.21517)
• Keywords: diffusion, generative, large language model
• Abstract: Recent advances in text-to-image (T2I) diffusion models have enabled increasingly realistic synthesis of vehicle damage, raising concerns about their reliability in automated insurance workflows. The ability to generate crash-like imagery challenges the boundary between authentic and synthetic data, introducing new risks of misuse in fraud or claim manipulation. To address these issues, we propose HERS (Hidden-Pattern Expert Learning for Risk-Specific Damage Adaptation), a framework designed to improve fidelity, controllability, and domain alignment of diffusion-generated damage images. HERS fine-tunes a base diffusion model via domain-specific expert adaptation without requiring manual annotation. Using self-supervised image-text pairs automatically generated by a large language model and T2I pipeline, HERS models each damage category, such as dents, scratches, broken lights, or cracked paint, as a separate expert. These experts are later integrated into a unified multi-damage model that balances specialization with generalization. We evaluate HERS across four diffusion backbones and observe consistent improvements: plus 5.5 percent in text faithfulness and plus 2.3 percent in human preference ratings compared to baselines. Beyond image fidelity, we discuss implications for fraud detection, auditability, and safe deployment of generative models in high-stakes domains. Our findings highlight both the opportunities and risks of domain-specific diffusion, underscoring the importance of trustworthy generation in safety-critical applications such as auto insurance.

Title: A Unified SPD Token Transformer Framework for EEG Classification: Systematic Comparison of Geometric Embeddings

• Authors: Chi-Sheng Chen, En-Jui Kuo, Guan-Ying Chen, Xinyu Zhang, Fan Zhang
• Subjects: cs.LG, cs.HC
• Abstract URL: https://arxiv.org/abs/2601.21521
• Pdf URL: https://arxiv.org/pdf/2601.21521
• Copy Paste: [[2601.21521]] A Unified SPD Token Transformer Framework for EEG Classification: Systematic Comparison of Geometric Embeddings(https://arxiv.org/abs/2601.21521)
• Keywords: transformer
• Abstract: Spatial covariance matrices of EEG signals are Symmetric Positive Definite (SPD) and lie on a Riemannian manifold, yet the theoretical connection between embedding geometry and optimization dynamics remains unexplored. We provide a formal analysis linking embedding choice to gradient conditioning and numerical stability for SPD manifolds, establishing three theoretical results: (1) BWSPD's $\sqrt{\kappa}$ gradient conditioning (vs $\kappa$ for Log-Euclidean) via Daleckii-Kre\uın matrices provides better gradient conditioning on high-dimensional inputs ($d \geq 22$), with this advantage reducing on low-dimensional inputs ($d \leq 8$) where eigendecomposition overhead dominates; (2) Embedding-Space Batch Normalization (BN-Embed) approximates Riemannian normalization up to $O(\varepsilon^2)$ error, yielding $+26\%$ accuracy on 56-channel ERP data but negligible effect on 8-channel SSVEP data, matching the channel-count-dependent prediction; (3) bi-Lipschitz bounds prove BWSPD tokens preserve manifold distances with distortion governed solely by the condition ratio $\kappa$. We validate these predictions via a unified Transformer framework comparing BWSPD, Log-Euclidean, and Euclidean embeddings within identical architecture across 1,500+ runs on three EEG paradigms (motor imagery, ERP, SSVEP; 36 subjects). Our Log-Euclidean Transformer achieves state-of-the-art performance on all datasets, substantially outperforming classical Riemannian classifiers and recent SPD baselines, while BWSPD offers competitive accuracy with similar training time.

Title: More Bang for the Buck: Improving the Inference of Large Language Models at a Fixed Budget using Reset and Discard (ReD)

• Authors: Sagi Meir, Tommer D. Keidar, Noam Levi, Shlomi Reuveni, Barak Hirshberg
• Subjects: cs.LG, cond-mat.dis-nn, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2601.21522
• Pdf URL: https://arxiv.org/pdf/2601.21522
• Copy Paste: [[2601.21522]] More Bang for the Buck: Improving the Inference of Large Language Models at a Fixed Budget using Reset and Discard (ReD)(https://arxiv.org/abs/2601.21522)
• Keywords: large language model
• Abstract: The performance of large language models (LLMs) on verifiable tasks is usually measured by pass@k, the probability of answering a question correctly at least once in k trials. At a fixed budget, a more suitable metric is coverage@cost, the average number of unique questions answered as a function of the total number of attempts. We connect the two metrics and show that the empirically-observed power-law behavior in pass@k leads to a sublinear growth of the coverage@cost (diminishing returns). To solve this problem, we propose Reset-and-Discard (ReD), a query method of LLMs that increases coverage@cost for any given budget, regardless of the pass@k form. Moreover, given a pass@k, we can quantitatively predict the savings in the total number of attempts using ReD. If pass@k is not available for the model, ReD can infer its power-law exponent. Experiments on three LLMs using HumanEval demonstrate that ReD substantially reduces the required attempts, tokens, and USD cost to reach a desired coverage, while also offering an efficient way to measure inference power-laws.

Title: Fast and Geometrically Grounded Lorentz Neural Networks

• Authors: Robert van der Klis, Ricardo Chávez Torres, Max van Spengler, Yuhui Ding, Thomas Hofmann, Pascal Mettes
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21529
• Pdf URL: https://arxiv.org/pdf/2601.21529
• Copy Paste: [[2601.21529]] Fast and Geometrically Grounded Lorentz Neural Networks(https://arxiv.org/abs/2601.21529)
• Keywords: robust
• Abstract: Hyperbolic space is quickly gaining traction as a promising geometry for hierarchical and robust representation learning. A core open challenge is the development of a mathematical formulation of hyperbolic neural networks that is both efficient and captures the key properties of hyperbolic space. The Lorentz model of hyperbolic space has been shown to enable both fast forward and backward propagation. However, we prove that, with the current formulation of Lorentz linear layers, the hyperbolic norms of the outputs scale logarithmically with the number of gradient descent steps, nullifying the key advantage of hyperbolic geometry. We propose a new Lorentz linear layer grounded in the well-known ``distance-to-hyperplane" formulation. We prove that our formulation results in the usual linear scaling of output hyperbolic norms with respect to the number of gradient descent steps. Our new formulation, together with further algorithmic efficiencies through Lorentzian activation functions and a new caching strategy results in neural networks fully abiding by hyperbolic geometry while simultaneously bridging the computation gap to Euclidean neural networks. Code available at: this https URL.

Title: On the Adversarial Robustness of Large Vision-Language Models under Visual Token Compression

• Authors: Xinwei Zhang, Hangcheng Liu, Li Bai, Hao Wang, Qingqing Ye, Tianwei Zhang, Haibo Hu
• Subjects: cs.CR, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21531
• Pdf URL: https://arxiv.org/pdf/2601.21531
• Copy Paste: [[2601.21531]] On the Adversarial Robustness of Large Vision-Language Models under Visual Token Compression(https://arxiv.org/abs/2601.21531)
• Keywords: security, defense, attack, robust
• Abstract: Visual token compression is widely used to accelerate large vision-language models (LVLMs) by pruning or merging visual tokens, yet its adversarial robustness remains unexplored. We show that existing encoder-based attacks can substantially overestimate the robustness of compressed LVLMs, due to an optimization-inference mismatch: perturbations are optimized on the full-token representation, while inference is performed through a token-compression bottleneck. To address this gap, we propose the Compression-AliGnEd attack (CAGE), which aligns perturbation optimization with compression inference without assuming access to the deployed compression mechanism or its token budget. CAGE combines (i) expected feature disruption, which concentrates distortion on tokens likely to survive across plausible budgets, and (ii) rank distortion alignment, which actively aligns token distortions with rank scores to promote the retention of highly distorted evidence. Across diverse representative plug-and-play compression mechanisms and datasets, our results show that CAGE consistently achieves lower robust accuracy than the baseline. This work highlights that robustness assessments ignoring compression can be overly optimistic, calling for compression-aware security evaluation and defenses for efficient LVLMs.

Title: Bi-Anchor Interpolation Solver for Accelerating Generative Modeling

• Authors: Hongxu Chen, Hongxiang Li, Zhen Wang, Long Chen
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21542
• Pdf URL: https://arxiv.org/pdf/2601.21542
• Copy Paste: [[2601.21542]] Bi-Anchor Interpolation Solver for Accelerating Generative Modeling(https://arxiv.org/abs/2601.21542)
• Keywords: generative
• Abstract: Flow Matching (FM) models have emerged as a leading paradigm for high-fidelity synthesis. However, their reliance on iterative Ordinary Differential Equation (ODE) solving creates a significant latency bottleneck. Existing solutions face a dichotomy: training-free solvers suffer from significant performance degradation at low Neural Function Evaluations (NFEs), while training-based one- or few-steps generation methods incur prohibitive training costs and lack plug-and-play versatility. To bridge this gap, we propose the Bi-Anchor Interpolation Solver (BA-solver). BA-solver retains the versatility of standard training-free solvers while achieving significant acceleration by introducing a lightweight SideNet (1-2% backbone size) alongside the frozen backbone. Specifically, our method is founded on two synergistic components: \textbf{1) Bidirectional Temporal Perception}, where the SideNet learns to approximate both future and historical velocities without retraining the heavy backbone; and 2) Bi-Anchor Velocity Integration, which utilizes the SideNet with two anchor velocities to efficiently approximate intermediate velocities for batched high-order integration. By utilizing the backbone to establish high-precision ``anchors'' and the SideNet to densify the trajectory, BA-solver enables large interval sizes with minimized error. Empirical results on ImageNet-256^2 demonstrate that BA-solver achieves generation quality comparable to 100+ NFEs Euler solver in just 10 NFEs and maintains high fidelity in as few as 5 NFEs, incurring negligible training costs. Furthermore, BA-solver ensures seamless integration with existing generative pipelines, facilitating downstream tasks such as image editing.

Title: inversedMixup: Data Augmentation via Inverting Mixed Embeddings

• Authors: Fanshuang Kong, Richong Zhang, Qiyu Sun, Zhijie Nie, Ting Deng, Chunming Hu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21543
• Pdf URL: https://arxiv.org/pdf/2601.21543
• Copy Paste: [[2601.21543]] inversedMixup: Data Augmentation via Inverting Mixed Embeddings(https://arxiv.org/abs/2601.21543)
• Keywords: interpretability
• Abstract: Mixup generates augmented samples by linearly interpolating inputs and labels with a controllable ratio. However, since it operates in the latent embedding level, the resulting samples are not human-interpretable. In contrast, LLM-based augmentation methods produce sentences via prompts at the token level, yielding readable outputs but offering limited control over the generation process. Inspired by recent advances in LLM inversion, which reconstructs natural language from embeddings and helps bridge the gap between latent embedding space and discrete token space, we propose inversedMixup, a unified framework that combines the controllability of Mixup with the interpretability of LLM-based generation. Specifically, inversedMixup adopts a three-stage training procedure to align the output embedding space of a task-specific model with the input embedding space of an LLM. Upon successful alignment, inversedMixup can reconstruct mixed embeddings with a controllable mixing ratio into human-interpretable augmented sentences, thereby improving the augmentation performance. Additionally, inversedMixup provides the first empirical evidence of the manifold intrusion phenomenon in text Mixup and introduces a simple yet effective strategy to mitigate it. Extensive experiments demonstrate the effectiveness and generalizability of our approach in both few-shot and fully supervised scenarios.

Title: Note2Chat: Improving LLMs for Multi-Turn Clinical History Taking Using Medical Notes

• Authors: Yang Zhou, Zhenting Sheng, Mingrui Tan, Yuting Song, Jun Zhou, Yu Heng Kwan, Lian Leng Low, Yang Bai, Yong Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21551
• Pdf URL: https://arxiv.org/pdf/2601.21551
• Copy Paste: [[2601.21551]] Note2Chat: Improving LLMs for Multi-Turn Clinical History Taking Using Medical Notes(https://arxiv.org/abs/2601.21551)
• Keywords: interpretability, large language model
• Abstract: Effective clinical history taking is a foundational yet underexplored component of clinical reasoning. While large language models (LLMs) have shown promise on static benchmarks, they often fall short in dynamic, multi-turn diagnostic settings that require iterative questioning and hypothesis refinement. To address this gap, we propose \method{}, a note-driven framework that trains LLMs to conduct structured history taking and diagnosis by learning from widely available medical notes. Instead of relying on scarce and sensitive dialogue data, we convert real-world medical notes into high-quality doctor-patient dialogues using a decision tree-guided generation and refinement pipeline. We then propose a three-stage fine-tuning strategy combining supervised learning, simulated data augmentation, and preference learning. Furthermore, we propose a novel single-turn reasoning paradigm that reframes history taking as a sequence of single-turn reasoning problems. This design enhances interpretability and enables local supervision, dynamic adaptation, and greater sample efficiency. Experimental results show that our method substantially improves clinical reasoning, achieving gains of +16.9 F1 and +21.0 Top-1 diagnostic accuracy over GPT-4o. Our code and dataset can be found at this https URL.

Title: ASTRA: Automated Synthesis of agentic Trajectories and Reinforcement Arenas

• Authors: Xiaoyu Tian, Haotian Wang, Shuaiting Chen, Hao Zhou, Kaichi Yu, Yudian Zhang, Jade Ouyang, Junxi Yin, Jiong Chen, Baoyan Guo, Lei Zhang, Junjie Tao, Yuansheng Song, Ming Cui, Chengwei Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21558
• Pdf URL: https://arxiv.org/pdf/2601.21558
• Copy Paste: [[2601.21558]] ASTRA: Automated Synthesis of agentic Trajectories and Reinforcement Arenas(https://arxiv.org/abs/2601.21558)
• Keywords: robust, large language model
• Abstract: Large language models (LLMs) are increasingly used as tool-augmented agents for multi-step decision making, yet training robust tool-using agents remains challenging. Existing methods still require manual intervention, depend on non-verifiable simulated environments, rely exclusively on either supervised fine-tuning (SFT) or reinforcement learning (RL), and struggle with stable long-horizon, multi-turn learning. To address these challenges, we introduce ASTRA, a fully automated end-to-end framework for training tool-augmented language model agents via scalable data synthesis and verifiable reinforcement learning. ASTRA integrates two complementary components. First, a pipeline that leverages the static topology of tool-call graphs synthesizes diverse, structurally grounded trajectories, instilling broad and transferable tool-use competence. Second, an environment synthesis framework that captures the rich, compositional topology of human semantic reasoning converts decomposed question-answer traces into independent, code-executable, and rule-verifiable environments, enabling deterministic multi-turn RL. Based on this method, we develop a unified training methodology that integrates SFT with online RL using trajectory-level rewards to balance task completion and interaction efficiency. Experiments on multiple agentic tool-use benchmarks demonstrate that ASTRA-trained models achieve state-of-the-art performance at comparable scales, approaching closed-source systems while preserving core reasoning ability. We release the full pipelines, environments, and trained models at this https URL.

Title: HistoPrism: Unlocking Functional Pathway Analysis from Pan-Cancer Histology via Gene Expression Prediction

• Authors: Susu Hu, Qinghe Zeng, Nithya Bhasker, Jakob Nicolas Kather, Stefanie Speidel
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21560
• Pdf URL: https://arxiv.org/pdf/2601.21560
• Copy Paste: [[2601.21560]] HistoPrism: Unlocking Functional Pathway Analysis from Pan-Cancer Histology via Gene Expression Prediction(https://arxiv.org/abs/2601.21560)
• Keywords: transformer
• Abstract: Predicting spatial gene expression from H&E histology offers a scalable and clinically accessible alternative to sequencing, but realizing clinical impact requires models that generalize across cancer types and capture biologically coherent signals. Prior work is often limited to per-cancer settings and variance-based evaluation, leaving functional relevance underexplored. We introduce HistoPrism, an efficient transformer-based architecture for pan-cancer prediction of gene expression from histology. To evaluate biological meaning, we introduce a pathway-level benchmark, shifting assessment from isolated gene-level variance to coherent functional pathways. HistoPrism not only surpasses prior state-of-the-art models on highly variable genes , but also more importantly, achieves substantial gains on pathway-level prediction, demonstrating its ability to recover biologically coherent transcriptomic patterns. With strong pan-cancer generalization and improved efficiency, HistoPrism establishes a new standard for clinically relevant transcriptomic modeling from routinely available histology.

Title: Representation Unlearning: Forgetting through Information Compression

• Authors: Antonio Almudévar, Alfonso Ortega
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21564
• Pdf URL: https://arxiv.org/pdf/2601.21564
• Copy Paste: [[2601.21564]] Representation Unlearning: Forgetting through Information Compression(https://arxiv.org/abs/2601.21564)
• Keywords: privacy, robust
• Abstract: Machine unlearning seeks to remove the influence of specific training data from a model, a need driven by privacy regulations and robustness concerns. Existing approaches typically modify model parameters, but such updates can be unstable, computationally costly, and limited by local approximations. We introduce Representation Unlearning, a framework that performs unlearning directly in the model's representation space. Instead of modifying model parameters, we learn a transformation over representations that imposes an information bottleneck: maximizing mutual information with retained data while suppressing information about data to be forgotten. We derive variational surrogates that make this objective tractable and show how they can be instantiated in two practical regimes: when both retain and forget data are available, and in a zero-shot setting where only forget data can be accessed. Experiments across several benchmarks demonstrate that Representation Unlearning achieves more reliable forgetting, better utility retention, and greater computational efficiency than parameter-centric baselines.

Title: Bridging Functional and Representational Similarity via Usable Information

• Authors: Antonio Almudévar, Alfonso Ortega
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21568
• Pdf URL: https://arxiv.org/pdf/2601.21568
• Copy Paste: [[2601.21568]] Bridging Functional and Representational Similarity via Usable Information(https://arxiv.org/abs/2601.21568)
• Keywords: robust
• Abstract: We present a unified framework for quantifying the similarity between representations through the lens of \textit{usable information}, offering a rigorous theoretical and empirical synthesis across three key dimensions. First, addressing functional similarity, we establish a formal link between stitching performance and conditional mutual information. We further reveal that stitching is inherently asymmetric, demonstrating that robust functional comparison necessitates a bidirectional analysis rather than a unidirectional mapping. Second, concerning representational similarity, we prove that reconstruction-based metrics and standard tools (e.g., CKA, RSA) act as estimators of usable information under specific constraints. Crucially, we show that similarity is relative to the capacity of the predictive family: representations that appear distinct to a rigid observer may be identical to a more expressive one. Third, we demonstrate that representational similarity is sufficient but not necessary for functional similarity. We unify these concepts through a task-granularity hierarchy: similarity on a complex task guarantees similarity on any coarser derivative, establishing representational similarity as the limit of maximum granularity: input reconstruction.

Title: Shaping capabilities with token-level data filtering

• Authors: Neil Rathi, Alec Radford
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21571
• Pdf URL: https://arxiv.org/pdf/2601.21571
• Copy Paste: [[2601.21571]] Shaping capabilities with token-level data filtering(https://arxiv.org/abs/2601.21571)
• Keywords: robust
• Abstract: Current approaches to reducing undesired capabilities in language models are largely post hoc, and can thus be easily bypassed by adversaries. A natural alternative is to shape capabilities during pretraining itself. On the proxy task of removing medical capabilities, we show that the simple intervention of filtering pretraining data is highly effective, robust, and inexpensive at scale. Inspired by work on data attribution, we show that filtering tokens is more effective than filtering documents, achieving the same hit to undesired capabilities at a lower cost to benign ones. Training models spanning two orders of magnitude, we then demonstrate that filtering gets more effective with scale: for our largest models, token filtering leads to a 7000x compute slowdown on the forget domain. We also show that models trained with token filtering can still be aligned on the forget domain. Along the way, we introduce a methodology for labeling tokens with sparse autoencoders and distilling cheap, high-quality classifiers. We also demonstrate that filtering can be robust to noisy labels with sufficient pretraining compute.

Title: Learning the Mechanism of Catastrophic Forgetting: A Perspective from Gradient Similarity

• Authors: Mutian Yang, Zisen Zhan, Yutong Chen, Haolin Li, Kaiwen Wang, Kaili Zheng, Yuguang Wang, Qi Wang, Jiandong Gao, Ji Wu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21577
• Pdf URL: https://arxiv.org/pdf/2601.21577
• Copy Paste: [[2601.21577]] Learning the Mechanism of Catastrophic Forgetting: A Perspective from Gradient Similarity(https://arxiv.org/abs/2601.21577)
• Keywords: large language model
• Abstract: Catastrophic forgetting during knowledge injection severely undermines the continual learning capability of large language models (LLMs). Although existing methods attempt to mitigate this issue, they often lack a foundational theoretical explanation. We establish a gradient-based theoretical framework to explain catastrophic forgetting. We first prove that strongly negative gradient similarity is a fundamental cause of forgetting. We then use gradient similarity to identify two types of neurons: conflicting neurons that induce forgetting and account for 50%-75% of neurons, and collaborative neurons that mitigate forgetting and account for 25%-50%. Based on this analysis, we propose a knowledge injection method, Collaborative Neural Learning (CNL). By freezing conflicting neurons and updating only collaborative neurons, CNL theoretically eliminates catastrophic forgetting under an infinitesimal learning rate eta and an exactly known mastered set. Experiments on five LLMs, four datasets, and four optimizers show that CNL achieves zero forgetting in in-set settings and reduces forgetting by 59.1%-81.7% in out-of-set settings.

Title: CORDS: Continuous Representations of Discrete Structures

• Authors: Tin Hadži Veljković, Erik Bekkers, Michael Tiemann, Jan-Willem van de Meent
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21583
• Pdf URL: https://arxiv.org/pdf/2601.21583
• Copy Paste: [[2601.21583]] CORDS: Continuous Representations of Discrete Structures(https://arxiv.org/abs/2601.21583)
• Keywords: robust
• Abstract: Many learning problems require predicting sets of objects when the number of objects is not known beforehand. Examples include object detection, molecular modeling, and scientific inference tasks such as astrophysical source detection. Existing methods often rely on padded representations or must explicitly infer the set size, which often poses challenges. We present a novel strategy for addressing this challenge by casting prediction of variable-sized sets as a continuous inference problem. Our approach, CORDS (Continuous Representations of Discrete Structures), provides an invertible mapping that transforms a set of spatial objects into continuous fields: a density field that encodes object locations and count, and a feature field that carries their attributes over the same support. Because the mapping is invertible, models operate entirely in field space while remaining exactly decodable to discrete sets. We evaluate CORDS across molecular generation and regression, object detection, simulation-based inference, and a mathematical task involving recovery of local maxima, demonstrating robust handling of unknown set sizes with competitive accuracy.

Title: ICL-EVADER: Zero-Query Black-Box Evasion Attacks on In-Context Learning and Their Defenses

• Authors: Ningyuan He, Ronghong Huang, Qianqian Tang, Hongyu Wang, Xianghang Mi, Shanqing Guo
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2601.21586
• Pdf URL: https://arxiv.org/pdf/2601.21586
• Copy Paste: [[2601.21586]] ICL-EVADER: Zero-Query Black-Box Evasion Attacks on In-Context Learning and Their Defenses(https://arxiv.org/abs/2601.21586)
• Keywords: security, defense, attack, robust, large language model
• Abstract: In-context learning (ICL) has become a powerful, data-efficient paradigm for text classification using large language models. However, its robustness against realistic adversarial threats remains largely unexplored. We introduce ICL-Evader, a novel black-box evasion attack framework that operates under a highly practical zero-query threat model, requiring no access to model parameters, gradients, or query-based feedback during attack generation. We design three novel attacks, Fake Claim, Template, and Needle-in-a-Haystack, that exploit inherent limitations of LLMs in processing in-context prompts. Evaluated across sentiment analysis, toxicity, and illicit promotion tasks, our attacks significantly degrade classifier performance (e.g., achieving up to 95.3% attack success rate), drastically outperforming traditional NLP attacks which prove ineffective under the same constraints. To counter these vulnerabilities, we systematically investigate defense strategies and identify a joint defense recipe that effectively mitigates all attacks with minimal utility loss (<5% accuracy degradation). Finally, we translate our defensive insights into an automated tool that proactively fortifies standard ICL prompts against adversarial evasion. This work provides a comprehensive security assessment of ICL, revealing critical vulnerabilities and offering practical solutions for building more robust systems. Our source code and evaluation datasets are publicly available at: this https URL .

Title: Heterogeneity-Aware Knowledge Sharing for Graph Federated Learning

• Authors: Wentao Yu, Sheng Wan, Shuo Chen, Bo Han, Chen Gong
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21589
• Pdf URL: https://arxiv.org/pdf/2601.21589
• Copy Paste: [[2601.21589]] Heterogeneity-Aware Knowledge Sharing for Graph Federated Learning(https://arxiv.org/abs/2601.21589)
• Keywords: privacy, protect, federate
• Abstract: Graph Federated Learning (GFL) enables distributed graph representation learning while protecting the privacy of graph data. However, GFL suffers from heterogeneity arising from diverse node features and structural topologies across multiple clients. To address both types of heterogeneity, we propose a novel graph Federated learning method via Semantic and Structural Alignment (FedSSA), which shares the knowledge of both node features and structural topologies. For node feature heterogeneity, we propose a novel variational model to infer class-wise node distributions, so that we can cluster clients based on inferred distributions and construct cluster-level representative distributions. We then minimize the divergence between local and cluster-level distributions to facilitate semantic knowledge sharing. For structural heterogeneity, we employ spectral Graph Neural Networks (GNNs) and propose a spectral energy measure to characterize structural information, so that we can cluster clients based on spectral energy and build cluster-level spectral GNNs. We then align the spectral characteristics of local spectral GNNs with those of cluster-level spectral GNNs to enable structural knowledge sharing. Experiments on six homophilic and five heterophilic graph datasets under both non-overlapping and overlapping partitioning settings demonstrate that FedSSA consistently outperforms eleven state-of-the-art methods.

Title: Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening

• Authors: Xiaotong Ji, Rasul Tutunov, Matthieu Zimmer, Haitham Bou Ammar
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21590
• Pdf URL: https://arxiv.org/pdf/2601.21590
• Copy Paste: [[2601.21590]] Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening(https://arxiv.org/abs/2601.21590)
• Keywords: generative, large language model
• Abstract: Reinforcement learning (RL) post-training is a dominant approach for improving the reasoning performance of large language models (LLMs), yet growing evidence suggests that its gains arise primarily from distribution sharpening rather than the acquisition of new capabilities. Recent work has shown that sampling from the power distribution of LLMs using Markov chain Monte Carlo (MCMC) can recover performance comparable to RL post-training without relying on external rewards; however, the high computational cost of MCMC makes such approaches impractical for widespread adoption. In this work, we propose a theoretically grounded alternative that eliminates the need for iterative MCMC. We derive a novel formulation showing that the global power distribution can be approximated by a token-level scaled low-temperature one, where the scaling factor captures future trajectory quality. Leveraging this insight, we introduce a training-free and verifier-free algorithm that sharpens the base model's generative distribution autoregressively. Empirically, we evaluate our method on math, QA, and code tasks across four LLMs, and show that our method matches or surpasses one-shot GRPO without relying on any external rewards, while reducing inference latency by over 10x compared to MCMC-based sampling.

Title: Unifying Heterogeneous Degradations: Uncertainty-Aware Diffusion Bridge Model for All-in-One Image Restoration

• Authors: Luwei Tu, Jiawei Wu, Xing Luo, Zhi Jin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21592
• Pdf URL: https://arxiv.org/pdf/2601.21592
• Copy Paste: [[2601.21592]] Unifying Heterogeneous Degradations: Uncertainty-Aware Diffusion Bridge Model for All-in-One Image Restoration(https://arxiv.org/abs/2601.21592)
• Keywords: diffusion
• Abstract: All-in-One Image Restoration (AiOIR) faces the fundamental challenge in reconciling conflicting optimization objectives across heterogeneous degradations. Existing methods are often constrained by coarse-grained control mechanisms or fixed mapping schedules, yielding suboptimal adaptation. To address this, we propose an Uncertainty-Aware Diffusion Bridge Model (UDBM), which innovatively reformulates AiOIR as a stochastic transport problem steered by pixel-wise uncertainty. By introducing a relaxed diffusion bridge formulation which replaces the strict terminal constraint with a relaxed constraint, we model the uncertainty of degradations while theoretically resolving the drift singularity inherent in standard diffusion bridges. Furthermore, we devise a dual modulation strategy: the noise schedule aligns diverse degradations into a shared high-entropy latent space, while the path schedule adaptively regulates the transport trajectory motivated by the viscous dynamics of entropy regularization. By effectively rectifying the transport geometry and dynamics, UDBM achieves state-of-the-art performance across diverse restoration tasks within a single inference step.

Title: HydroSense: A Dual-Microcontroller IoT Framework for Real-Time Multi-Parameter Water Quality Monitoring with Edge Processing and Cloud Analytics

• Authors: Abdul Hasib, A. S. M. Ahsanul Sarkar Akib, Anish Giri
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21595
• Pdf URL: https://arxiv.org/pdf/2601.21595
• Copy Paste: [[2601.21595]] HydroSense: A Dual-Microcontroller IoT Framework for Real-Time Multi-Parameter Water Quality Monitoring with Edge Processing and Cloud Analytics(https://arxiv.org/abs/2601.21595)
• Keywords: robust
• Abstract: The global water crisis necessitates affordable, accurate, and real-time water quality monitoring solutions. Traditional approaches relying on manual sampling or expensive commercial systems fail to address accessibility challenges in resource-constrained environments. This paper presents HydroSense, an innovative Internet of Things framework that integrates six critical water quality parameters including pH, dissolved oxygen (DO), temperature, total dissolved solids (TDS), estimated nitrogen, and water level into a unified monitoring system. HydroSense employs a novel dual-microcontroller architecture, utilizing Arduino Uno for precision analog measurements with five-point calibration algorithms and ESP32 for wireless connectivity, edge processing, and cloud integration. The system implements advanced signal processing techniques including median filtering for TDS measurement, temperature compensation algorithms, and robust error handling. Experimental validation over 90 days demonstrates exceptional performance metrics: pH accuracy of plus or minus 0.08 units across the 0 to 14 range, DO measurement stability within plus or minus 0.2 mg/L, TDS accuracy of plus or minus 1.9 percent across 0 to 1000 ppm, and 99.8 percent cloud data transmission reliability. With a total implementation cost of 32,983 BDT (approximately 300 USD), HydroSense achieves an 85 percent cost reduction compared to commercial systems while providing enhanced connectivity through the Firebase real-time database. This research establishes a new paradigm for accessible environmental monitoring, demonstrating that professional-grade water quality assessment can be achieved through intelligent system architecture and cost-effective component selection.

Title: WMVLM: Evaluating Diffusion Model Image Watermarking via Vision-Language Models

• Authors: Zijin Yang, Yu Sun, Kejiang Chen, Jiawei Zhao, Jun Jiang, Weiming Zhang, Nenghai Yu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21610
• Pdf URL: https://arxiv.org/pdf/2601.21610
• Copy Paste: [[2601.21610]] WMVLM: Evaluating Diffusion Model Image Watermarking via Vision-Language Models(https://arxiv.org/abs/2601.21610)
• Keywords: security, interpretability, watermark, diffusion
• Abstract: Digital watermarking is essential for securing generated images from diffusion models. Accurate watermark evaluation is critical for algorithm development, yet existing methods have significant limitations: they lack a unified framework for both residual and semantic watermarks, provide results without interpretability, neglect comprehensive security considerations, and often use inappropriate metrics for semantic watermarks. To address these gaps, we propose WMVLM, the first unified and interpretable evaluation framework for diffusion model image watermarking via vision-language models (VLMs). We redefine quality and security metrics for each watermark type: residual watermarks are evaluated by artifact strength and erasure resistance, while semantic watermarks are assessed through latent distribution shifts. Moreover, we introduce a three-stage training strategy to progressively enable the model to achieve classification, scoring, and interpretable text generation. Experiments show WMVLM outperforms state-of-the-art VLMs with strong generalization across datasets, diffusion models, and watermarking methods.

Title: PathReasoner-R1: Instilling Structured Reasoning into Pathology Vision-Language Model via Knowledge-Guided Policy Optimization

• Authors: Songhan Jiang, Fengchun Liu, Ziyue Wang, Linghan Cai, Yongbing Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21617
• Pdf URL: https://arxiv.org/pdf/2601.21617
• Copy Paste: [[2601.21617]] PathReasoner-R1: Instilling Structured Reasoning into Pathology Vision-Language Model via Knowledge-Guided Policy Optimization(https://arxiv.org/abs/2601.21617)
• Keywords: robust
• Abstract: Vision-Language Models (VLMs) are advancing computational pathology with superior visual understanding capabilities. However, current systems often reduce diagnosis to directly output conclusions without verifiable evidence-linked reasoning, which severely limits clinical trust and hinders expert error rectification. To address these barriers, we construct PathReasoner, the first large-scale dataset of whole-slide image (WSI) reasoning. Unlike previous work reliant on unverified distillation, we develop a rigorous knowledge-guided generation pipeline. By leveraging medical knowledge graphs, we explicitly align structured pathological findings and clinical reasoning with diagnoses, generating over 20K high-quality instructional samples. Based on the database, we propose PathReasoner-R1, which synergizes trajectory-masked supervised fine-tuning with reasoning-oriented reinforcement learning to instill structured chain-of-thought capabilities. To ensure medical rigor, we engineer a knowledge-aware multi-granular reward function incorporating an Entity Reward mechanism strictly aligned with knowledge graphs. This effectively guides the model to optimize for logical consistency rather than mere outcome matching, thereby enhancing robustness. Extensive experiments demonstrate that PathReasoner-R1 achieves state-of-the-art performance on both PathReasoner and public benchmarks across various image scales, equipping pathology models with transparent, clinically grounded reasoning capabilities. Dataset and code are available at this https URL.

Title: Similarity of Processing Steps in Vision Model Representations

• Authors: Matéo Mahaut, Marco Baroni
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21621
• Pdf URL: https://arxiv.org/pdf/2601.21621
• Copy Paste: [[2601.21621]] Similarity of Processing Steps in Vision Model Representations(https://arxiv.org/abs/2601.21621)
• Keywords: transformer
• Abstract: Recent literature suggests that the bigger the model, the more likely it is to converge to similar, ``universal'' representations, despite different training objectives, datasets, or modalities. While this literature shows that there is an area where model representations are similar, we study here how vision models might get to those representations--in particular, do they also converge to the same intermediate steps and operations? We therefore study the processes that lead to convergent representations in different models. First, we quantify distance between different model representations at different stages. We follow the evolution of distances between models throughout processing, identifying the processing steps which are most different between models. We find that while layers at similar positions in different models have the most similar representations, strong differences remain. Classifier models, unlike the others, will discard information about low-level image statistics in their final layers. CNN- and transformer-based models also behave differently, with transformer models applying smoother changes to representations from one layer to the next. These distinctions clarify the level and nature of convergence between model representations, and enables a more qualitative account of the underlying processes in image models.

Title: LAMP: Look-Ahead Mixed-Precision Inference of Large Language Models

• Authors: Stanislav Budzinskiy, Marian Gloser, Tolunay Yilmaz, Ying Hong Tham, Yuanyi Lin, Wenyi Fang, Fan Wu, Philipp Petersen
• Subjects: cs.LG, math.NA
• Abstract URL: https://arxiv.org/abs/2601.21623
• Pdf URL: https://arxiv.org/pdf/2601.21623
• Copy Paste: [[2601.21623]] LAMP: Look-Ahead Mixed-Precision Inference of Large Language Models(https://arxiv.org/abs/2601.21623)
• Keywords: transformer, large language model
• Abstract: Mixed-precision computations are a hallmark of the current stage of AI, driving the progress in large language models towards efficient, locally deployable solutions. This article addresses the floating-point computation of compositionally-rich functions, concentrating on transformer inference. Based on the rounding error analysis of a composition $f(g(\mathrm{x}))$, we provide an adaptive strategy that selects a small subset of components of $g(\mathrm{x})$ to be computed more accurately while all other computations can be carried out with lower accuracy. We then explain how this strategy can be applied to different compositions within a transformer and illustrate its overall effect on transformer inference. We study the effectiveness of this algorithm numerically on GPT-2 models and demonstrate that already very low recomputation rates allow for improvements of up to two orders of magnitude in accuracy.

Title: HeRo-Q: A General Framework for Stable Low Bit Quantization via Hessian Conditioning

• Authors: Jinhao Zhang Yunquan Zhang, Zicheng yan, Boyang Zhang, Jun Sun, Daning Cheng
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21626
• Pdf URL: https://arxiv.org/pdf/2601.21626
• Copy Paste: [[2601.21626]] HeRo-Q: A General Framework for Stable Low Bit Quantization via Hessian Conditioning(https://arxiv.org/abs/2601.21626)
• Keywords: robust
• Abstract: Post Training Quantization (PTQ), a mainstream model compression technique, often leads to the paradoxical 'low error, high loss' phenomenon because it focuses solely on minimizing quantization error. The root cause lies in the Hessian matrix of the LLM loss landscape: a few high curvature directions are extremely sensitive to perturbations. To address this, we propose the Hessian Robust Quantization (HeRo Q) algorithm, which applies a lightweight, learnable rotation-compression matrix to the weight space prior to quantization. This joint framework reshapes the loss landscape by reducing the largest Hessian eigenvalue and reducing its max eigenvalue, thereby significantly enhancing robustness to quantization noise. HeRo-Q requires no architectural modifications, incurs negligible computational overhead, and integrates seamlessly into existing PTQ pipelines. Experiments on Llama and Qwen models show that HeRo Q consistently outperforms state of the art methods including GPTQ, AWQ, and SpinQuant not only achieving superior performance under standard W4A8 settings, but also excelling in the highly challenging W3A16 ultra low bit regime, where it boosts GSM8K accuracy on Llama3 8B to 70.15\% and effectively avoids the logical collapse commonly seen in aggressive quantization.

Title: Noise as a Probe: Membership Inference Attacks on Diffusion Models Leveraging Initial Noise

• Authors: Puwei Lian, Yujun Cai, Songze Li, Bingkun Bao
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21628
• Pdf URL: https://arxiv.org/pdf/2601.21628
• Copy Paste: [[2601.21628]] Noise as a Probe: Membership Inference Attacks on Diffusion Models Leveraging Initial Noise(https://arxiv.org/abs/2601.21628)
• Keywords: privacy, attack, membership infer, diffusion
• Abstract: Diffusion models have achieved remarkable progress in image generation, but their increasing deployment raises serious concerns about privacy. In particular, fine-tuned models are highly vulnerable, as they are often fine-tuned on small and private datasets. Membership inference attacks (MIAs) are used to assess privacy risks by determining whether a specific sample was part of a model's training data. Existing MIAs against diffusion models either assume obtaining the intermediate results or require auxiliary datasets for training the shadow model. In this work, we utilized a critical yet overlooked vulnerability: the widely used noise schedules fail to fully eliminate semantic information in the images, resulting in residual semantic signals even at the maximum noise step. We empirically demonstrate that the fine-tuned diffusion model captures hidden correlations between the residual semantics in initial noise and the original images. Building on this insight, we propose a simple yet effective membership inference attack, which injects semantic information into the initial noise and infers membership by analyzing the model's generation result. Extensive experiments demonstrate that the semantic initial noise can strongly reveal membership information, highlighting the vulnerability of diffusion models to MIAs.

Title: A Tilted Seesaw: Revisiting Autoencoder Trade-off for Controllable Diffusion

• Authors: Pu Cao, Yiyang Ma, Feng Zhou, Xuedan Yin, Qing Song, Lu Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21633
• Pdf URL: https://arxiv.org/pdf/2601.21633
• Copy Paste: [[2601.21633]] A Tilted Seesaw: Revisiting Autoencoder Trade-off for Controllable Diffusion(https://arxiv.org/abs/2601.21633)
• Keywords: diffusion, generative
• Abstract: In latent diffusion models, the autoencoder (AE) is typically expected to balance two capabilities: faithful reconstruction and a generation-friendly latent space (e.g., low gFID). In recent ImageNet-scale AE studies, we observe a systematic bias toward generative metrics in handling this trade-off: reconstruction metrics are increasingly under-reported, and ablation-based AE selection often favors the best-gFID configuration even when reconstruction fidelity degrades. We theoretically analyze why this gFID-dominant preference can appear unproblematic for ImageNet generation, yet becomes risky when scaling to controllable diffusion: AEs can induce condition drift, which limits achievable condition alignment. Meanwhile, we find that reconstruction fidelity, especially instance-level measures, better indicates controllability. We empirically validate the impact of tilted autoencoder evaluation on controllability by studying several recent ImageNet AEs. Using a multi-dimensional condition-drift evaluation protocol reflecting controllable generation tasks, we find that gFID is only weakly predictive of condition preservation, whereas reconstruction-oriented metrics are substantially more aligned. ControlNet experiments further confirm that controllability tracks condition preservation rather than gFID. Overall, our results expose a gap between ImageNet-centric AE evaluation and the requirements of scalable controllable diffusion, offering practical guidance for more reliable benchmarking and model selection.

Title: RSGround-R1: Rethinking Remote Sensing Visual Grounding through Spatial Reasoning

• Authors: Shiqi Huang, Shuting He, Bihan Wen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21634
• Pdf URL: https://arxiv.org/pdf/2601.21634
• Copy Paste: [[2601.21634]] RSGround-R1: Rethinking Remote Sensing Visual Grounding through Spatial Reasoning(https://arxiv.org/abs/2601.21634)
• Keywords: robust, large language model
• Abstract: Remote Sensing Visual Grounding (RSVG) aims to localize target objects in large-scale aerial imagery based on natural language descriptions. Owing to the vast spatial scale and high semantic ambiguity of remote sensing scenes, these descriptions often rely heavily on positional cues, posing unique challenges for Multimodal Large Language Models (MLLMs) in spatial reasoning. To leverage this unique feature, we propose a reasoning-guided, position-aware post-training framework, dubbed \textbf{RSGround-R1}, to progressively enhance spatial understanding. Specifically, we first introduce Chain-of-Thought Supervised Fine-Tuning (CoT-SFT) using synthetically generated RSVG reasoning data to establish explicit position awareness. Reinforcement Fine-Tuning (RFT) is then applied, augmented by our newly designed positional reward that provides continuous and distance-aware guidance toward accurate localization. Moreover, to mitigate incoherent localization behaviors across rollouts, we introduce a spatial consistency guided optimization scheme that dynamically adjusts policy updates based on their spatial coherence, ensuring stable and robust convergence. Extensive experiments on RSVG benchmarks demonstrate superior performance and generalization of our model.

Title: Sampling-Free Privacy Accounting for Matrix Mechanisms under Random Allocation

• Authors: Jan Schuchardt, Nikita Kalinin
• Subjects: cs.LG, cs.CR, stat.ML
• Abstract URL: https://arxiv.org/abs/2601.21636
• Pdf URL: https://arxiv.org/pdf/2601.21636
• Copy Paste: [[2601.21636]] Sampling-Free Privacy Accounting for Matrix Mechanisms under Random Allocation(https://arxiv.org/abs/2601.21636)
• Keywords: privacy
• Abstract: We study privacy amplification for differentially private model training with matrix factorization under random allocation (also known as the balls-in-bins model). Recent work by Choquette-Choo et al. (2025) proposes a sampling-based Monte Carlo approach to compute amplification parameters in this setting. However, their guarantees either only hold with some high probability or require random abstention by the mechanism. Furthermore, the required number of samples for ensuring $(\epsilon,\delta)$-DP is inversely proportional to $\delta$. In contrast, we develop sampling-free bounds based on Rényi divergence and conditional composition. The former is facilitated by a dynamic programming formulation to efficiently compute the bounds. The latter complements it by offering stronger privacy guarantees for small $\epsilon$, where Rényi divergence bounds inherently lead to an over-approximation. Our framework applies to arbitrary banded and non-banded matrices. Through numerical comparisons, we demonstrate the efficacy of our approach across a broad range of matrix mechanisms used in research and practice.

Title: Generative Design of Ship Propellers using Conditional Flow Matching

• Authors: Patrick Kruger, Rafael Diaz, Simon Hauschulz, Stefan Harries, Hanno Gottschalk
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21637
• Pdf URL: https://arxiv.org/pdf/2601.21637
• Copy Paste: [[2601.21637]] Generative Design of Ship Propellers using Conditional Flow Matching(https://arxiv.org/abs/2601.21637)
• Keywords: generative
• Abstract: In this paper, we explore the use of generative artificial intelligence (GenAI) for ship propeller design. While traditional forward machine learning models predict the performance of mechanical components based on given design parameters, GenAI models aim to generate designs that achieve specified performance targets. In particular, we employ conditional flow matching to establish a bidirectional mapping between design parameters and simulated noise that is conditioned on performance labels. This approach enables the generation of multiple valid designs corresponding to the same performance targets by sampling over the noise vector. To support model training, we generate data using a vortex lattice method for numerical simulation and analyze the trade-off between model accuracy and the amount of available data. We further propose data augmentation using pseudo-labels derived from less data-intensive forward surrogate models, which can often improve overall model performance. Finally, we present examples of distinct propeller geometries that exhibit nearly identical performance characteristics, illustrating the versatility and potential of GenAI in engineering design.

Title: Seg-MoE: Multi-Resolution Segment-wise Mixture-of-Experts for Time Series Forecasting Transformers

• Authors: Evandro S. Ortigossa, Eran Segal
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21641
• Pdf URL: https://arxiv.org/pdf/2601.21641
• Copy Paste: [[2601.21641]] Seg-MoE: Multi-Resolution Segment-wise Mixture-of-Experts for Time Series Forecasting Transformers(https://arxiv.org/abs/2601.21641)
• Keywords: transformer
• Abstract: Transformer-based models have recently made significant advances in accurate time-series forecasting, but even these architectures struggle to scale efficiently while capturing long-term temporal dynamics. Mixture-of-Experts (MoE) layers are a proven solution to scaling problems in natural language processing. However, existing MoE approaches for time-series forecasting rely on token-wise routing mechanisms, which may fail to exploit the natural locality and continuity of temporal data. In this work, we introduce Seg-MoE, a sparse MoE design that routes and processes contiguous time-step segments rather than making independent expert decisions. Token segments allow each expert to model intra-segment interactions directly, naturally aligning with inherent temporal patterns. We integrate Seg-MoE layers into a time-series Transformer and evaluate it on multiple multivariate long-term forecasting benchmarks. Seg-MoE consistently achieves state-of-the-art forecasting accuracy across almost all prediction horizons, outperforming both dense Transformers and prior token-wise MoE models. Comprehensive ablation studies confirm that segment-level routing is the key factor driving these gains. Our results show that aligning the MoE routing granularity with the inherent structure of time series provides a powerful, yet previously underexplored, inductive bias, opening new avenues for conditionally sparse architectures in sequential data modeling.

Title: ILRR: Inference-Time Steering Method for Masked Diffusion Language Models

• Authors: Eden Avrahami, Eliya Nachmani
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21647
• Pdf URL: https://arxiv.org/pdf/2601.21647
• Copy Paste: [[2601.21647]] ILRR: Inference-Time Steering Method for Masked Diffusion Language Models(https://arxiv.org/abs/2601.21647)
• Keywords: diffusion
• Abstract: Discrete Diffusion Language Models (DLMs) offer a promising non-autoregressive alternative for text generation, yet effective mechanisms for inference-time control remain relatively underexplored. Existing approaches include sampling-level guidance procedures or trajectory optimization mechanisms. In this work, we introduce Iterative Latent Representation Refinement (ILRR), a learning-free framework for steering DLMs using a single reference sequence. ILRR guides generation by dynamically aligning the internal activations of the generated sequence with those of a given reference throughout the denoising process. This approach captures and transfers high-level semantic properties, with a tunable steering scale enabling flexible control over attributes such as sentiment. We further introduce Spatially Modulated Steering, an extension that enables steering long texts using shorter references by regulating guidance intensity across the sequence. Empirically, we demonstrate that ILRR achieves effective attribute steering on LLaDA and MDLM architectures with a minor computational overhead, requiring only one additional parallel forward pass per denoising step. Under the same compute budget, ILRR improves attribute accuracy over comparable baselines by 10$\%$ to 60$\%$ points, while maintaining high generation quality.

Title: SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning

• Authors: Jinjun Peng, Magnus Saebo, Tianjun Zhong, Yi-Jie Cheng, Junfeng Yang, Baishakhi Ray, Simin Chen, Yangruibo Ding
• Subjects: cs.LG, cs.AI, cs.CL, cs.SE
• Abstract URL: https://arxiv.org/abs/2601.21649
• Pdf URL: https://arxiv.org/pdf/2601.21649
• Copy Paste: [[2601.21649]] SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning(https://arxiv.org/abs/2601.21649)
• Keywords: privacy
• Abstract: The deployment of coding agents in privacy-sensitive and resource-constrained environments drives the demand for capable open-weight Small Language Models (SLMs). However, they suffer from a fundamental capability gap: unlike frontier large models, they lack the inference-time strong generalization to work with complicated, unfamiliar codebases. We identify that the prevailing Task-Centric Learning (TCL) paradigm, which scales exposure across disparate repositories, fails to address this limitation. In response, we propose Repository-Centric Learning (RCL), a paradigm shift that prioritizes vertical repository depth over horizontal task breadth, suggesting SLMs must internalize the "physics" of a target software environment through parametric knowledge acquisition, rather than attempting to recover it via costly inference-time search. Following this new paradigm, we design a four-unit Repository-Centric Experience, transforming static codebases into interactive learning signals, to train SWE-Spot-4B, a family of highly compact models built as repo-specialized experts that breaks established scaling trends, outperforming open-weight models up to larger (e.g., CWM by Meta, Qwen3-Coder-30B) and surpassing/matching efficiency-focused commercial models (e.g., GPT-4.1-mini, GPT-5-nano) across multiple SWE tasks. Further analysis reveals that RCL yields higher training sample efficiency and lower inference costs, emphasizing that for building efficient intelligence, repository mastery is a distinct and necessary dimension that complements general coding capability.

Title: Gauge-invariant representation holonomy

• Authors: Vasileios Sevetlidis, George Pavlidis
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21653
• Pdf URL: https://arxiv.org/pdf/2601.21653
• Copy Paste: [[2601.21653]] Gauge-invariant representation holonomy(https://arxiv.org/abs/2601.21653)
• Keywords: robust
• Abstract: Deep networks learn internal representations whose geometry--how features bend, rotate, and evolve--affects both generalization and robustness. Existing similarity measures such as CKA or SVCCA capture pointwise overlap between activation sets, but miss how representations change along input paths. Two models may appear nearly identical under these metrics yet respond very differently to perturbations or adversarial stress. We introduce representation holonomy, a gauge-invariant statistic that measures this path dependence. Conceptually, holonomy quantifies the "twist" accumulated when features are parallel-transported around a small loop in input space: flat representations yield zero holonomy, while nonzero values reveal hidden curvature. Our estimator fixes gauge through global whitening, aligns neighborhoods using shared subspaces and rotation-only Procrustes, and embeds the result back to the full feature space. We prove invariance to orthogonal (and affine, post-whitening) transformations, establish a linear null for affine layers, and show that holonomy vanishes at small radii. Empirically, holonomy increases with loop radius, separates models that appear similar under CKA, and correlates with adversarial and corruption robustness. It also tracks training dynamics as features form and stabilize. Together, these results position representation holonomy as a practical and scalable diagnostic for probing the geometric structure of learned representations beyond pointwise similarity.

Title: TabClustPFN: A Prior-Fitted Network for Tabular Data Clustering

• Authors: Tianqi Zhao, Guanyang Wang, Yan Shuo Tan, Qiong Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21656
• Pdf URL: https://arxiv.org/pdf/2601.21656
• Copy Paste: [[2601.21656]] TabClustPFN: A Prior-Fitted Network for Tabular Data Clustering(https://arxiv.org/abs/2601.21656)
• Keywords: robust
• Abstract: Clustering tabular data is a fundamental yet challenging problem due to heterogeneous feature types, diverse data-generating mechanisms, and the absence of transferable inductive biases across datasets. Prior-fitted networks (PFNs) have recently demonstrated strong generalization in supervised tabular learning by amortizing Bayesian inference under a broad synthetic prior. Extending this paradigm to clustering is nontrivial: clustering is unsupervised, admits a combinatorial and permutation-invariant output space, and requires inferring the number of clusters. We introduce TabClustPFN, a prior-fitted network for tabular data clustering that performs amortized Bayesian inference over both cluster assignments and cluster cardinality. Pretrained on synthetic datasets drawn from a flexible clustering prior, TabClustPFN clusters unseen datasets in a single forward pass, without dataset-specific retraining or hyperparameter tuning. The model naturally handles heterogeneous numerical and categorical features and adapts to a wide range of clustering structures. Experiments on synthetic data and curated real-world tabular benchmarks show that TabClustPFN outperforms classical, deep, and amortized clustering baselines, while exhibiting strong robustness in out-of-the-box exploratory settings. Code is available at this https URL.

Title: Authenticated encryption for space telemetry

• Authors: Andrew Savchenko
• Subjects: cs.CR, cs.NI, eess.SY
• Abstract URL: https://arxiv.org/abs/2601.21657
• Pdf URL: https://arxiv.org/pdf/2601.21657
• Copy Paste: [[2601.21657]] Authenticated encryption for space telemetry(https://arxiv.org/abs/2601.21657)
• Keywords: security, protect, robust
• Abstract: We explore how command stack protection requirements outlined in NASA-STD-1006A can be satisfied within the context of emergency space telemetry. Proposed implementation of lightweight authenticated encryption offers strong security without sacrificing performance in resource-constrained environments. It produces fixed-length messages, maintaining compatibility with the underlying data transport protocols. By focusing on predictable properties and robust authentication, we create a scheme that protects the confidentiality, integrity and authenticity of telemetry data in emergency communications while balancing security requirements with the operational constraints.

Title: Few-Shot Domain Adaptation with Temporal References and Static Priors for Glacier Calving Front Delineation

• Authors: Marcel Dreier, Nora Gourmelon, Dakota Pyles, Thorsten Seehaus, Matthias H. Braun, Andreas Maier, Vincent Christlein
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21663
• Pdf URL: https://arxiv.org/pdf/2601.21663
• Copy Paste: [[2601.21663]] Few-Shot Domain Adaptation with Temporal References and Static Priors for Glacier Calving Front Delineation(https://arxiv.org/abs/2601.21663)
• Keywords: segmentation
• Abstract: During benchmarking, the state-of-the-art model for glacier calving front delineation achieves near-human performance. However, when applied in a real-world setting at a novel study site, its delineation accuracy is insufficient for calving front products intended for further scientific analyses. This site represents an out-of-distribution domain for a model trained solely on the benchmark dataset. By employing a few-shot domain adaptation strategy, incorporating spatial static prior knowledge, and including summer reference images in the input time series, the delineation error is reduced from 1131.6 m to 68.7 m without any architectural modifications. These methodological advancements establish a framework for applying deep learning-based calving front segmentation to novel study sites, enabling calving front monitoring on a global scale.

Title: AdaptBPE: From General Purpose to Specialized Tokenizers

• Authors: Vijini Liyanage, François Yvon
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21665
• Pdf URL: https://arxiv.org/pdf/2601.21665
• Copy Paste: [[2601.21665]] AdaptBPE: From General Purpose to Specialized Tokenizers(https://arxiv.org/abs/2601.21665)
• Keywords: large language model
• Abstract: Subword tokenization methods, such as Byte-Pair Encoding (BPE), significantly impact the performance and efficiency of large language models (LLMs). The standard approach involves training a general-purpose tokenizer that uniformly processes all textual data during both training and inference. However, the use of a generic set of tokens can incur inefficiencies when applying the model to specific domains or languages. To address this limitation, we propose a post-training adaptation strategy that selectively replaces low-utility tokens with more relevant ones based on their frequency in an adaptation corpus. Our algorithm identifies the token inventory that most effectively encodes the adaptation corpus for a given target vocabulary size. Extensive experiments on generation and classification tasks across multiple languages demonstrate that our adapted tokenizers compress test corpora more effectively than baselines using the same vocabulary size. This method serves as a lightweight adaptation mechanism, akin to a vocabulary fine-tuning process, enabling optimized tokenization for specific domains or tasks. Our code and data are available at this https URL.

Title: When Gradient Optimization Is Not Enough: $\dagger$ Dispersive and Anchoring Geometric Regularizer for Multimodal Learning

• Authors: Zixuan Xia, Hao Wang, Pengcheng Weng, Yanyu Qian, Yangxin Xu, William Dan, Fei Wang
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21670
• Pdf URL: https://arxiv.org/pdf/2601.21670
• Copy Paste: [[2601.21670]] When Gradient Optimization Is Not Enough: $\dagger$ Dispersive and Anchoring Geometric Regularizer for Multimodal Learning(https://arxiv.org/abs/2601.21670)
• Keywords: robust
• Abstract: Multimodal learning aims to integrate complementary information from heterogeneous modalities, yet strong optimization alone does not guaranty well-structured representations. Even under carefully balanced training schemes, multimodal models often exhibit geometric pathologies, including intra-modal representation collapse and sample-level cross-modal inconsistency, which degrade both unimodal robustness and multimodal fusion. We identify representation geometry as a missing control axis in multimodal learning and propose \regName, a lightweight geometry-aware regularization framework. \regName enforces two complementary constraints on intermediate embeddings: an intra-modal dispersive regularization that promotes representation diversity, and an inter-modal anchoring regularization that bounds sample-level cross-modal drift without rigid alignment. The proposed regularizer is plug-and-play, requires no architectural modifications, and is compatible with various training paradigms. Extensive experiments across multiple multimodal benchmarks demonstrate consistent improvements in both multimodal and unimodal performance, showing that explicitly regulating representation geometry effectively mitigates modality trade-offs.

Title: Multimodal Visual Surrogate Compression for Alzheimer's Disease Classification

• Authors: Dexuan Ding, Ciyuan Peng, Endrowednes Kuantama, Jingcai Guo, Jia Wu, Jian Yang, Amin Beheshti, Ming-Hsuan Yang, Yuankai Qi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21673
• Pdf URL: https://arxiv.org/pdf/2601.21673
• Copy Paste: [[2601.21673]] Multimodal Visual Surrogate Compression for Alzheimer's Disease Classification(https://arxiv.org/abs/2601.21673)
• Keywords: extraction
• Abstract: High-dimensional structural MRI (sMRI) images are widely used for Alzheimer's Disease (AD) diagnosis. Most existing methods for sMRI representation learning rely on 3D architectures (e.g., 3D CNNs), slice-wise feature extraction with late aggregation, or apply training-free feature extractions using 2D foundation models (e.g., DINO). However, these three paradigms suffer from high computational cost, loss of cross-slice relations, and limited ability to extract discriminative features, respectively. To address these challenges, we propose Multimodal Visual Surrogate Compression (MVSC). It learns to compress and adapt large 3D sMRI volumes into compact 2D features, termed as visual surrogates, which are better aligned with frozen 2D foundation models to extract powerful representations for final AD classification. MVSC has two key components: a Volume Context Encoder that captures global cross-slice context under textual guidance, and an Adaptive Slice Fusion module that aggregates slice-level information in a text-enhanced, patch-wise manner. Extensive experiments on three large-scale Alzheimer's disease benchmarks demonstrate our MVSC performs favourably on both binary and multi-class classification tasks compared against state-of-the-art methods.

Title: Scale-Dependent Semantic Dynamics Revealed by Allan Deviation

• Authors: Debayan Dasgupta
• Subjects: cs.CL, physics.data-an
• Abstract URL: https://arxiv.org/abs/2601.21678
• Pdf URL: https://arxiv.org/pdf/2601.21678
• Copy Paste: [[2601.21678]] Scale-Dependent Semantic Dynamics Revealed by Allan Deviation(https://arxiv.org/abs/2601.21678)
• Keywords: large language model
• Abstract: While language progresses through a sequence of semantic states, the underlying dynamics of this progression remain elusive. Here, we treat the semantic progression of written text as a stochastic trajectory in a high-dimensional state space. We utilize Allan deviation, a tool from precision metrology, to analyze the stability of meaning by treating ordered sentence embeddings as a displacement signal. Our analysis reveals two distinct dynamical regimes: short-time power-law scaling, which differentiates creative literature from technical texts, and a long-time crossover to a stability-limited noise floor. We find that while large language models successfully mimic the local scaling statistics of human text, they exhibit a systematic reduction in their stability horizon. These results establish semantic coherence as a measurable physical property, offering a framework to differentiate the nuanced dynamics of human cognition from the patterns generated by algorithmic models.

Title: LLM4Fluid: Large Language Models as Generalizable Neural Solvers for Fluid Dynamics

• Authors: Qisong Xiao, Xinhai Chen, Qinglin Wang, Xiaowei Guo, Binglin Wang, Weifeng Chen, Zhichao Wang, Yunfei Liu, Rui Xia, Hang Zou, Gencheng Liu, Shuai Li, Jie Liu
• Subjects: cs.LG, physics.flu-dyn
• Abstract URL: https://arxiv.org/abs/2601.21681
• Pdf URL: https://arxiv.org/pdf/2601.21681
• Copy Paste: [[2601.21681]] LLM4Fluid: Large Language Models as Generalizable Neural Solvers for Fluid Dynamics(https://arxiv.org/abs/2601.21681)
• Keywords: robust, large language model
• Abstract: Deep learning has emerged as a promising paradigm for spatio-temporal modeling of fluid dynamics. However, existing approaches often suffer from limited generalization to unseen flow conditions and typically require retraining when applied to new scenarios. In this paper, we present LLM4Fluid, a spatio-temporal prediction framework that leverages Large Language Models (LLMs) as generalizable neural solvers for fluid dynamics. The framework first compresses high-dimensional flow fields into a compact latent space via reduced-order modeling enhanced with a physics-informed disentanglement mechanism, effectively mitigating spatial feature entanglement while preserving essential flow structures. A pretrained LLM then serves as a temporal processor, autoregressively predicting the dynamics of physical sequences with time series prompts. To bridge the modality gap between prompts and physical sequences, which can otherwise degrade prediction accuracy, we propose a dedicated modality alignment strategy that resolves representational mismatch and stabilizes long-term prediction. Extensive experiments across diverse flow scenarios demonstrate that LLM4Fluid functions as a robust and generalizable neural solver without retraining, achieving state-of-the-art accuracy while exhibiting powerful zero-shot and in-context learning capabilities. Code and datasets are publicly available at this https URL.

Title: FIT: Defying Catastrophic Forgetting in Continual LLM Unlearning

• Authors: Xiaoyu Xu, Minxin Du, Kun Fang, Zi Liang, Yaxin Xiao, Zhicong Huang, Cheng Hong, Qingqing Ye, Haibo Hu
• Subjects: cs.CL, cs.AI, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21682
• Pdf URL: https://arxiv.org/pdf/2601.21682
• Copy Paste: [[2601.21682]] FIT: Defying Catastrophic Forgetting in Continual LLM Unlearning(https://arxiv.org/abs/2601.21682)
• Keywords: privacy, attack, robust, large language model
• Abstract: Large language models (LLMs) demonstrate impressive capabilities across diverse tasks but raise concerns about privacy, copyright, and harmful materials. Existing LLM unlearning methods rarely consider the continual and high-volume nature of real-world deletion requests, which can cause utility degradation and catastrophic forgetting as requests accumulate. To address this challenge, we introduce \fit, a framework for continual unlearning that handles large numbers of deletion requests while maintaining robustness against both catastrophic forgetting and post-unlearning recovery. \fit mitigates degradation through rigorous data \underline{F}iltering, \underline{I}mportance-aware updates, and \underline{T}argeted layer attribution, enabling stable performance across long sequences of unlearning operations and achieving a favorable balance between forgetting effectiveness and utility retention. To support realistic evaluation, we present \textbf{PCH}, a benchmark covering \textbf{P}ersonal information, \textbf{C}opyright, and \textbf{H}armful content in sequential deletion scenarios, along with two symmetric metrics, Forget Degree (F.D.) and Retain Utility (R.U.), which jointly assess forgetting quality and utility preservation. Extensive experiments on four open-source LLMs with hundreds of deletion requests show that \fit achieves the strongest trade-off between F.D. and R.U., surpasses existing methods on MMLU, CommonsenseQA, and GSM8K, and remains resistant against both relearning and quantization recovery attacks.

Title: Do Not Waste Your Rollouts: Recycling Search Experience for Efficient Test-Time Scaling

• Authors: Xinglin Wang, Jiayi Shi, Shaoxiong Feng, Peiwen Yuan, Yiwei Li, Yueqi Zhang, Chuyi Tan, Ji Zhang, Boyuan Pan, Yao Hu, Kan Li
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21684
• Pdf URL: https://arxiv.org/pdf/2601.21684
• Copy Paste: [[2601.21684]] Do Not Waste Your Rollouts: Recycling Search Experience for Efficient Test-Time Scaling(https://arxiv.org/abs/2601.21684)
• Keywords: large language model
• Abstract: Test-Time Scaling enhances the reasoning capabilities of Large Language Models by allocating additional inference compute to broaden the exploration of the solution space. However, existing search strategies typically treat rollouts as disposable samples, where valuable intermediate insights are effectively discarded after each trial. This systemic memorylessness leads to massive computational redundancy, as models repeatedly re-derive discovered conclusions and revisit known dead ends across extensive attempts. To bridge this gap, we propose \textbf{Recycling Search Experience (RSE)}, a self-guided, training-free strategy that turns test-time search from a series of isolated trials into a cumulative process. By actively distilling raw trajectories into a shared experience bank, RSE enables positive recycling of intermediate conclusions to shortcut redundant derivations and negative recycling of failure patterns to prune encountered dead ends. Theoretically, we provide an analysis that formalizes the efficiency gains of RSE, validating its advantage over independent sampling in solving complex reasoning tasks. Empirically, extensive experiments on HMMT24, HMMT25, IMO-Bench, and HLE show that RSE consistently outperforms strong baselines with comparable computational cost, achieving state-of-the-art scaling efficiency.

Title: Understanding Model Merging: A Unified Generalization Framework for Heterogeneous Experts

• Authors: Qinglun Li, Anke Tang, Miao Zhang, Mengzhu Wang, Quanjun Yin, Li Shen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21690
• Pdf URL: https://arxiv.org/pdf/2601.21690
• Copy Paste: [[2601.21690]] Understanding Model Merging: A Unified Generalization Framework for Heterogeneous Experts(https://arxiv.org/abs/2601.21690)
• Keywords: robust
• Abstract: Model merging efficiently aggregates capabilities from multiple fine-tuned models into a single one, operating purely in parameter space without original data or expensive re-computation. Despite empirical successes, a unified theory for its effectiveness under heterogeneous finetuning hyperparameters (e.g., varying learning rates, batch sizes) remains missing. Moreover, the lack of hyperparameter transparency in open-source fine-tuned models makes it difficult to predict merged-model performance, leaving practitioners without guidance on how to fine-tune merge-friendly experts. To address those two challenges, we employ $L_2$-Stability theory under heterogeneous hyperparameter environments to analyze the generalization of the merged model $\boldsymbol{x}_{avg}$. This pioneering analysis yields two key contributions: (i) \textit{A unified theoretical framework} is provided to explain existing merging algorithms, revealing how they optimize specific terms in our bound, thus offering a strong theoretical foundation for empirical observations. (ii) \textit{Actionable recommendations} are proposed for practitioners to strategically fine-tune expert models, enabling the construction of merge-friendly models within the pretraining-to-finetuning pipeline. Extensive experiments on the ResNet/Vit family across 20/8 visual classification tasks, involving thousands of finetuning models, robustly confirm the impact of different hyperparameters on the generalization of $\boldsymbol{x}_{avg}$ predicted by our theoretical results.

Title: ChartE$^{3}$: A Comprehensive Benchmark for End-to-End Chart Editing

• Authors: Shuo Li, Jiajun Sun, Zhekai Wang, Xiaoran Fan, Hui Li, Dingwen Yang, Zhiheng Xi, Yijun Wang, Zifei Shan, Tao Gui, Qi Zhang, Xuanjing Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21694
• Pdf URL: https://arxiv.org/pdf/2601.21694
• Copy Paste: [[2601.21694]] ChartE$^{3}$: A Comprehensive Benchmark for End-to-End Chart Editing(https://arxiv.org/abs/2601.21694)
• Keywords: large language model
• Abstract: Charts are a fundamental visualization format for structured data analysis. Enabling end-to-end chart editing according to user intent is of great practical value, yet remains challenging due to the need for both fine-grained control and global structural consistency. Most existing approaches adopt pipeline-based designs, where natural language or code serves as an intermediate representation, limiting their ability to faithfully execute complex edits. We introduce ChartE$^{3}$, an End-to-End Chart Editing benchmark that directly evaluates models without relying on intermediate natural language programs or code-level supervision. ChartE$^{3}$ focuses on two complementary editing dimensions: local editing, which involves fine-grained appearance changes such as font or color adjustments, and global editing, which requires holistic, data-centric transformations including data filtering and trend line addition. ChartE$^{3}$ contains over 1,200 high-quality samples constructed via a well-designed data pipeline with human curation. Each sample is provided as a triplet of a chart image, its underlying code, and a multimodal editing instruction, enabling evaluation from both objective and subjective perspectives. Extensive benchmarking of state-of-the-art multimodal large language models reveals substantial performance gaps, particularly on global editing tasks, highlighting critical limitations in current end-to-end chart editing capabilities.

Title: Toward Culturally Aligned LLMs through Ontology-Guided Multi-Agent Reasoning

• Authors: Wonduk Seo, Wonseok Choi, Junseo Koh, Juhyeon Lee, Hyunjin An, Minhyeong Yu, Jian Park, Qingshan Zhou, Seunghyun Lee, Yi Bu
• Subjects: cs.CL, cs.AI, cs.IR, cs.MA, cs.SI
• Abstract URL: https://arxiv.org/abs/2601.21700
• Pdf URL: https://arxiv.org/pdf/2601.21700
• Copy Paste: [[2601.21700]] Toward Culturally Aligned LLMs through Ontology-Guided Multi-Agent Reasoning(https://arxiv.org/abs/2601.21700)
• Keywords: robust, interpretability, large language model
• Abstract: Large Language Models (LLMs) increasingly support culturally sensitive decision making, yet often exhibit misalignment due to skewed pretraining data and the absence of structured value representations. Existing methods can steer outputs, but often lack demographic grounding and treat values as independent, unstructured signals, reducing consistency and interpretability. We propose OG-MAR, an Ontology-Guided Multi-Agent Reasoning framework. OG-MAR summarizes respondent-specific values from the World Values Survey (WVS) and constructs a global cultural ontology by eliciting relations over a fixed taxonomy via competency questions. At inference time, it retrieves ontology-consistent relations and demographically similar profiles to instantiate multiple value-persona agents, whose outputs are synthesized by a judgment agent that enforces ontology consistency and demographic proximity. Experiments on regional social-survey benchmarks across four LLM backbones show that OG-MAR improves cultural alignment and robustness over competitive baselines, while producing more transparent reasoning traces.

Title: Beyond Forgetting: Machine Unlearning Elicits Controllable Side Behaviors and Capabilities

• Authors: Tien Dang, The-Hai Nguyen, Dinh Mai Phuong, Nguyen Minh Phuong, Hoang Thanh-Tung, Le-Minh Nguyen, Naoya Inoue
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21702
• Pdf URL: https://arxiv.org/pdf/2601.21702
• Copy Paste: [[2601.21702]] Beyond Forgetting: Machine Unlearning Elicits Controllable Side Behaviors and Capabilities(https://arxiv.org/abs/2601.21702)
• Keywords: fair
• Abstract: We consider representation misdirection (RM), a class of LLM unlearning methods that achieves forgetting by manipulating the forget-representations, that is, latent representations of forget samples. Despite being important, the roles of target vectors used in RM, however, remain underexplored. Here, we approach and revisit RM through the lens of the linear representation hypothesis. Specifically, if one can somehow identify a one-dimensional representation corresponding to a high-level concept, the linear representation hypothesis enables linear operations on this concept vector within the forget-representation space. Under this view, we hypothesize that, beyond forgetting, machine unlearning elicits controllable side behaviors and stronger side capabilities corresponding to the high-level concept. Our hypothesis is empirically validated across a wide range of tasks, including behavioral control (e.g., controlling unlearned models' truth, sentiment, and refusal) and capability enhancement (e.g., improving unlearned models' in-context learning capability). Our findings reveal that this fairly attractive phenomenon could be either a hidden risk if misused or a mechanism that can be harnessed for developing models that require stronger capabilities and controllable behaviors.

Title: SmartMeterFM: Unifying Smart Meter Data Generative Tasks Using Flow Matching Models

• Authors: Nan Lin, Yanbo Wang, Jacco Heres, Peter Palensky, Pedro P. Vergara
• Subjects: cs.LG, eess.SY
• Abstract URL: https://arxiv.org/abs/2601.21706
• Pdf URL: https://arxiv.org/pdf/2601.21706
• Copy Paste: [[2601.21706]] SmartMeterFM: Unifying Smart Meter Data Generative Tasks Using Flow Matching Models(https://arxiv.org/abs/2601.21706)
• Keywords: privacy, generative
• Abstract: Smart meter data is the foundation for planning and operating the distribution network. Unfortunately, such data are not always available due to privacy regulations. Meanwhile, the collected data may be corrupted due to sensor or transmission failure, or it may not have sufficient resolution for downstream tasks. A wide range of generative tasks is formulated to address these issues, including synthetic data generation, missing data imputation, and super-resolution. Despite the success of machine learning models on these tasks, dedicated models need to be designed and trained for each task, leading to redundancy and inefficiency. In this paper, by recognizing the powerful modeling capability of flow matching models, we propose a new approach to unify diverse smart meter data generative tasks with a single model trained for conditional generation. The proposed flow matching models are trained to generate challenging, high-dimensional time series data, specifically monthly smart meter data at a 15 min resolution. By viewing different generative tasks as distinct forms of partial data observations and injecting them into the generation process, we unify tasks such as imputation and super-resolution with a single model, eliminating the need for re-training. The data generated by our model not only are consistent with the given observations but also remain realistic, showing better performance against interpolation and other machine learning based baselines dedicated to the tasks.

Title: Why Attention Patterns Exist: A Unifying Temporal Perspective Analysis

• Authors: Qingyue Yang, Jie Wang, Xing Li, Yinqi Bai, Xialiang Tong, Huiling Zhen, Jianye Hao, Mingxuan Yuan, Bin Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21709
• Pdf URL: https://arxiv.org/pdf/2601.21709
• Copy Paste: [[2601.21709]] Why Attention Patterns Exist: A Unifying Temporal Perspective Analysis(https://arxiv.org/abs/2601.21709)
• Keywords: large language model
• Abstract: Attention patterns play a crucial role in both training and inference of large language models (LLMs). Prior works have identified individual patterns such as retrieval heads, sink heads, and diagonal traces, yet these observations remain fragmented and lack a unifying explanation. To bridge this gap, we introduce \textbf{Temporal Attention Pattern Predictability Analysis (TAPPA), a unifying framework that explains diverse attention patterns by analyzing their underlying mathematical formulations} from a temporally continuous perspective. TAPPA both deepens the understanding of attention behavior and guides inference acceleration approaches. Specifically, TAPPA characterizes attention patterns as predictable patterns with clear regularities and unpredictable patterns that appear effectively random. Our analysis further reveals that this distinction can be explained by the degree of query self-similarity along the temporal dimension. Focusing on the predictable patterns, we further provide a detailed mathematical analysis of three representative cases through the joint effect of queries, keys, and Rotary Positional Embeddings (RoPE). We validate TAPPA by applying its insights to KV cache compression and LLM pruning tasks. Across these tasks, a simple metric motivated by TAPPA consistently improves performance over baseline methods. The code is available at this https URL.

Title: TACLer: Tailored Curriculum Reinforcement Learning for Efficient Reasoning

• Authors: Huiyuan Lai, Malvina Nissim
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21711
• Pdf URL: https://arxiv.org/pdf/2601.21711
• Copy Paste: [[2601.21711]] TACLer: Tailored Curriculum Reinforcement Learning for Efficient Reasoning(https://arxiv.org/abs/2601.21711)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have shown remarkable performance on complex reasoning tasks, especially when equipped with long chain-of-thought (CoT) reasoning. However, eliciting long CoT typically requires large-scale reinforcement learning (RL) training, while often leading to overthinking with redundant intermediate steps. To improve learning and reasoning efficiency, while preserving or even enhancing performance, we propose TACLer, a model-tailored curriculum reinforcement learning framework that gradually increases the complexity of the data based on the model's proficiency in multi-stage RL training. TACLer features two core components: (i) tailored curriculum learning that determines what knowledge the model lacks and needs to learn in progressive stages; (ii) a hybrid Thinking/NoThinking reasoning paradigm that balances accuracy and efficiency by enabling or disabling the Thinking mode. Our experiments show that TACLer yields a twofold advantage in learning and reasoning: (i) it reduces computational cost, cutting training compute by over 50% compared to long thinking models and reducing inference token usage by over 42% relative to the base model; and (ii) it improves accuracy by over 9% on the base model, consistently outperforming state-of-the-art Nothinking and Thinking baselines across four math datasets with complex problems.

Title: DreamActor-M2: Universal Character Image Animation via Spatiotemporal In-Context Learning

• Authors: Mingshuang Luo, Shuang Liang, Zhengkun Rong, Yuxuan Luo, Tianshu Hu, Ruibing Hou, Hong Chang, Yong Li, Yuan Zhang, Mingyuan Gao
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21716
• Pdf URL: https://arxiv.org/pdf/2601.21716
• Copy Paste: [[2601.21716]] DreamActor-M2: Universal Character Image Animation via Spatiotemporal In-Context Learning(https://arxiv.org/abs/2601.21716)
• Keywords: robust, generative
• Abstract: Character image animation aims to synthesize high-fidelity videos by transferring motion from a driving sequence to a static reference image. Despite recent advancements, existing methods suffer from two fundamental challenges: (1) suboptimal motion injection strategies that lead to a trade-off between identity preservation and motion consistency, manifesting as a "see-saw", and (2) an over-reliance on explicit pose priors (e.g., skeletons), which inadequately capture intricate dynamics and hinder generalization to arbitrary, non-humanoid characters. To address these challenges, we present DreamActor-M2, a universal animation framework that reimagines motion conditioning as an in-context learning problem. Our approach follows a two-stage paradigm. First, we bridge the input modality gap by fusing reference appearance and motion cues into a unified latent space, enabling the model to jointly reason about spatial identity and temporal dynamics by leveraging the generative prior of foundational models. Second, we introduce a self-bootstrapped data synthesis pipeline that curates pseudo cross-identity training pairs, facilitating a seamless transition from pose-dependent control to direct, end-to-end RGB-driven animation. This strategy significantly enhances generalization across diverse characters and motion scenarios. To facilitate comprehensive evaluation, we further introduce AW Bench, a versatile benchmark encompassing a wide spectrum of characters types and motion scenarios. Extensive experiments demonstrate that DreamActor-M2 achieves state-of-the-art performance, delivering superior visual fidelity and robust cross-domain generalization. Project Page: this https URL

Title: LoRA and Privacy: When Random Projections Help (and When They Don't)

• Authors: Yaxi Hu, Johanna Düngler, Bernhard Schölkopf, Amartya Sanyal
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2601.21719
• Pdf URL: https://arxiv.org/pdf/2601.21719
• Copy Paste: [[2601.21719]] LoRA and Privacy: When Random Projections Help (and When They Don't)(https://arxiv.org/abs/2601.21719)
• Keywords: privacy, attack, membership infer
• Abstract: We introduce the (Wishart) projection mechanism, a randomized map of the form $S \mapsto M f(S)$ with $M \sim W_d(1/r I_d, r)$ and study its differential privacy properties. For vector-valued queries $f$, we prove non-asymptotic DP guarantees without any additive noise, showing that Wishart randomness alone can suffice. For matrix-valued queries, however, we establish a sharp negative result: in the noise-free setting, the mechanism is not DP, and we demonstrate its vulnerability by implementing a near perfect membership inference attack (AUC $> 0.99$). We then analyze a noisy variant and prove privacy amplification due to randomness and low rank projection, in both large- and small-rank regimes, yielding stronger privacy guarantees than additive noise alone. Finally, we show that LoRA-style updates are an instance of the matrix-valued mechanism, implying that LoRA is not inherently private despite its built-in randomness, but that low-rank fine-tuning can be more private than full fine-tuning at the same noise level. Preliminary experiments suggest that tighter accounting enables lower noise and improved accuracy in practice.

Title: Enhancing Language Models for Robust Greenwashing Detection

• Authors: Neil Heinrich Braun, Keane Ong, Rui Mao, Erik Cambria, Gianmarco Mengaldo
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21722
• Pdf URL: https://arxiv.org/pdf/2601.21722
• Copy Paste: [[2601.21722]] Enhancing Language Models for Robust Greenwashing Detection(https://arxiv.org/abs/2601.21722)
• Keywords: robust
• Abstract: Sustainability reports are critical for ESG assessment, yet greenwashing and vague claims often undermine their reliability. Existing NLP models lack robustness to these practices, typically relying on surface-level patterns that generalize poorly. We propose a parameter-efficient framework that structures LLM latent spaces by combining contrastive learning with an ordinal ranking objective to capture graded distinctions between concrete actions and ambiguous claims. Our approach incorporates gated feature modulation to filter disclosure noise and utilizes MetaGradNorm to stabilize multi-objective optimization. Experiments in cross-category settings demonstrate superior robustness over standard baselines while revealing a trade-off between representational rigidity and generalization.

Title: Amortized Spectral Kernel Discovery via Prior-Data Fitted Network

• Authors: Kaustubh Sharma, Srijan Tiwari, Ojasva Nema, Parikshit Pareek
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21731
• Pdf URL: https://arxiv.org/pdf/2601.21731
• Copy Paste: [[2601.21731]] Amortized Spectral Kernel Discovery via Prior-Data Fitted Network(https://arxiv.org/abs/2601.21731)
• Keywords: interpretability
• Abstract: Prior-Data Fitted Networks (PFNs) enable efficient amortized inference but lack transparent access to their learned priors and kernels. This opacity hinders their use in downstream tasks, such as surrogate-based optimization, that require explicit covariance models. We introduce an interpretability-driven framework for amortized spectral discovery from pre-trained PFNs with decoupled attention. We perform a mechanistic analysis on a trained PFN that identifies attention latent output as the key intermediary, linking observed function data to spectral structure. Building on this insight, we propose decoder architectures that map PFN latents to explicit spectral density estimates and corresponding stationary kernels via Bochner's theorem. We study this pipeline in both single-realization and multi-realization regimes, contextualizing theoretical limits on spectral identifiability and proving consistency when multiple function samples are available. Empirically, the proposed decoders recover complex multi-peak spectral mixtures and produce explicit kernels that support Gaussian process regression with accuracy comparable to PFNs and optimization-based baselines, while requiring only a single forward pass. This yields orders-of-magnitude reductions in inference time compared to optimization-based baselines.

Title: CE-GOCD: Central Entity-Guided Graph Optimization for Community Detection to Augment LLM Scientific Question Answering

• Authors: Jiayin Lan, Jiaqi Li, Baoxin Wang, Ming Liu, Dayong Wu, Shijin Wang, Bing Qin, Guoping Hu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21733
• Pdf URL: https://arxiv.org/pdf/2601.21733
• Copy Paste: [[2601.21733]] CE-GOCD: Central Entity-Guided Graph Optimization for Community Detection to Augment LLM Scientific Question Answering(https://arxiv.org/abs/2601.21733)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) are increasingly used for question answering over scientific research papers. Existing retrieval augmentation methods often rely on isolated text chunks or concepts, but overlook deeper semantic connections between papers. This impairs the LLM's comprehension of scientific literature, hindering the comprehensiveness and specificity of its responses. To address this, we propose Central Entity-Guided Graph Optimization for Community Detection (CE-GOCD), a method that augments LLMs' scientific question answering by explicitly modeling and leveraging semantic substructures within academic knowledge graphs. Our approach operates by: (1) leveraging paper titles as central entities for targeted subgraph retrieval, (2) enhancing implicit semantic discovery via subgraph pruning and completion, and (3) applying community detection to distill coherent paper groups with shared themes. We evaluated the proposed method on three NLP literature-based question-answering datasets, and the results demonstrate its superiority over other retrieval-augmented baseline approaches, confirming the effectiveness of our framework.

Title: Why Adam Works Better with $β_1 = β_2$: The Missing Gradient Scale Invariance Principle

• Authors: Alberto Fernández-Hernández, Cristian Pérez-Corral, Jose I. Mestre, Manuel F. Dolz, Enrique S. Quintana-Ortí
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2601.21739
• Pdf URL: https://arxiv.org/pdf/2601.21739
• Copy Paste: [[2601.21739]] Why Adam Works Better with $β_1 = β_2$: The Missing Gradient Scale Invariance Principle(https://arxiv.org/abs/2601.21739)
• Keywords: robust
• Abstract: Adam has been at the core of large-scale training for almost a decade, yet a simple empirical fact remains unaccounted for: both validation scores and the qualitative behaviour of the training runs improve when the momentum parameters satisfy $\beta_{1}=\beta_{2}$. Some recent studies have reported this pattern, but there is still no explanation for why this choice helps. We show that this choice is closely tied to a structural property that we refer to as \textit{gradient scale invariance}. We formalize this notion and prove that Adam becomes gradient scale invariant of first order if and only if $\beta_{1}=\beta_{2}$. This perspective places the balanced regime of Adam in direct alignment with the design principles underlying several recent optimizers that explicitly enforce scale-robust updates. The theory is supported by experiments across vision and language tasks, and across different architectural families, in which rescaling the gradient has a markedly smoother effect on the update when $\beta_{1}=\beta_{2}$. Overall, our results offer a coherent explanation for an open question in the behavior of Adam and provide a simple principle that helps guide the design of future optimizers.

Title: Temporal Guidance for Large Language Models

• Authors: Hong-Kai Zheng, Piji Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21744
• Pdf URL: https://arxiv.org/pdf/2601.21744
• Copy Paste: [[2601.21744]] Temporal Guidance for Large Language Models(https://arxiv.org/abs/2601.21744)
• Keywords: large language model
• Abstract: Contrastive Decoding (CD) enhances the generation quality of large language models (LLMs) but incurs significant additional computational overhead due to the need for an auxiliary model. Existing internal self-contrastive decoding methods, such as Decoding by Contrasting Layers (DoLa), focus on discrepancies across different layers, which are notably unstable on small-scale models. In this work, based on the observation that LLMs exhibit local preferences, we propose a novel contrastive guidance strategy along the temporal dimension, namely Temporal Guidance (TeGu). Our method ingeniously leverages Multi-Token Prediction (MTP) to construct weaker amateur predictions for model self-contrast. To standardize the implementation of this mechanism, we further introduce a lightweight Conditional MTP Projector (cMTPP), which avoids maintaining multiple independent networks as required by other MTP modules. Across various model series and benchmarks, TeGu achieves significant performance improvements while maintaining low additional memory consumption and computational overhead.

Title: Temporal Sepsis Modeling: a Fully Interpretable Relational Way

• Authors: Vincent Lemaire, Nédra Meloulli, Pierre Jaquet
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21747
• Pdf URL: https://arxiv.org/pdf/2601.21747
• Copy Paste: [[2601.21747]] Temporal Sepsis Modeling: a Fully Interpretable Relational Way(https://arxiv.org/abs/2601.21747)
• Keywords: interpretability
• Abstract: Sepsis remains one of the most complex and heterogeneous syndromes in intensive care, characterized by diverse physiological trajectories and variable responses to treatment. While deep learning models perform well in the early prediction of sepsis, they often lack interpretability and ignore latent patient sub-phenotypes. In this work, we propose a machine learning framework by opening up a new avenue for addressing this issue: a relational approach. Temporal data from electronic medical records (EMRs) are viewed as multivariate patient logs and represented in a relational data schema. Then, a propositionalisation technique (based on classic aggregation/selection functions from the field of relational data) is applied to construct interpretable features to "flatten" the data. Finally, the flattened data is classified using a selective naive Bayesian classifier. Experimental validation demonstrates the relevance of the suggested approach as well as its extreme interpretability. The interpretation is fourfold: univariate, global, local, and counterfactual.

Title: Dynamic Topology Awareness: Breaking the Granularity Rigidity in Vision-Language Navigation

• Authors: Jiankun Peng, Jianyuan Guo, Ying Xu, Yue Liu, Jiashuang Yan, Xuanwei Ye, Houhua Li, Xiaoming Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21751
• Pdf URL: https://arxiv.org/pdf/2601.21751
• Copy Paste: [[2601.21751]] Dynamic Topology Awareness: Breaking the Granularity Rigidity in Vision-Language Navigation(https://arxiv.org/abs/2601.21751)
• Keywords: robust, transformer
• Abstract: Vision-Language Navigation in Continuous Environments (VLN-CE) presents a core challenge: grounding high-level linguistic instructions into precise, safe, and long-horizon spatial actions. Explicit topological maps have proven to be a vital solution for providing robust spatial memory in such tasks. However, existing topological planning methods suffer from a "Granularity Rigidity" problem. Specifically, these methods typically rely on fixed geometric thresholds to sample nodes, which fails to adapt to varying environmental complexities. This rigidity leads to a critical mismatch: the model tends to over-sample in simple areas, causing computational redundancy, while under-sampling in high-uncertainty regions, increasing collision risks and compromising precision. To address this, we propose DGNav, a framework for Dynamic Topological Navigation, introducing a context-aware mechanism to modulate map density and connectivity on-the-fly. Our approach comprises two core innovations: (1) A Scene-Aware Adaptive Strategy that dynamically modulates graph construction thresholds based on the dispersion of predicted waypoints, enabling "densification on demand" in challenging environments; (2) A Dynamic Graph Transformer that reconstructs graph connectivity by fusing visual, linguistic, and geometric cues into dynamic edge weights, enabling the agent to filter out topological noise and enhancing instruction adherence. Extensive experiments on the R2R-CE and RxR-CE benchmarks demonstrate DGNav exhibits superior navigation performance and strong generalization capabilities. Furthermore, ablation studies confirm that our framework achieves an optimal trade-off between navigation efficiency and safe exploration. The code is available at this https URL.

Title: CoFrGeNet: Continued Fraction Architectures for Language Generation

• Authors: Amit Dhurandhar, Vijil Chenthamarakshan, Dennis Wei, Tejaswini Pedapati, Karthikeyan Natesan Ramamurthy, Rahul Nair
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21766
• Pdf URL: https://arxiv.org/pdf/2601.21766
• Copy Paste: [[2601.21766]] CoFrGeNet: Continued Fraction Architectures for Language Generation(https://arxiv.org/abs/2601.21766)
• Keywords: transformer, generative
• Abstract: Transformers are arguably the preferred architecture for language generation. In this paper, inspired by continued fractions, we introduce a new function class for generative modeling. The architecture family implementing this function class is named CoFrGeNets - Continued Fraction Generative Networks. We design novel architectural components based on this function class that can replace Multi-head Attention and Feed-Forward Networks in Transformer blocks while requiring much fewer parameters. We derive custom gradient formulations to optimize the proposed components more accurately and efficiently than using standard PyTorch-based gradients. Our components are a plug-in replacement requiring little change in training or inference procedures that have already been put in place for Transformer-based models thus making our approach easy to incorporate in large industrial workflows. We experiment on two very different transformer architectures GPT2-xl (1.5B) and Llama3 (3.2B), where the former we pre-train on OpenWebText and GneissWeb, while the latter we pre-train on the docling data mix which consists of nine different datasets. Results show that the performance on downstream classification, Q\& A, reasoning and text understanding tasks of our models is competitive and sometimes even superior to the original models with $\frac{2}{3}$ to $\frac{1}{2}$ the parameters and shorter pre-training time. We believe that future implementations customized to hardware will further bring out the true potential of our architectures.

Title: Evaluating ChatGPT on Medical Information Extraction Tasks: Performance, Explainability and Beyond

• Authors: Wei Zhu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21767
• Pdf URL: https://arxiv.org/pdf/2601.21767
• Copy Paste: [[2601.21767]] Evaluating ChatGPT on Medical Information Extraction Tasks: Performance, Explainability and Beyond(https://arxiv.org/abs/2601.21767)
• Keywords: extraction, explainability, large language model
• Abstract: Large Language Models (LLMs) like ChatGPT have demonstrated amazing capabilities in comprehending user intents and generate reasonable and useful responses. Beside their ability to chat, their capabilities in various natural language processing (NLP) tasks are of interest to the research community. In this paper, we focus on assessing the overall ability of ChatGPT in 4 different medical information extraction (MedIE) tasks across 6 benchmark datasets. We present the systematically analysis by measuring ChatGPT's performance, explainability, confidence, faithfulness, and uncertainty. Our experiments reveal that: (a) ChatGPT's performance scores on MedIE tasks fall behind those of the fine-tuned baseline models. (b) ChatGPT can provide high-quality explanations for its decisions, however, ChatGPT is over-confident in its predcitions. (c) ChatGPT demonstrates a high level of faithfulness to the original text in the majority of cases. (d) The uncertainty in generation causes uncertainty in information extraction results, thus may hinder its applications in MedIE tasks.

Title: Zonkey: A Hierarchical Diffusion Language Model with Differentiable Tokenization and Probabilistic Attention

• Authors: Alon Rozental
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21768
• Pdf URL: https://arxiv.org/pdf/2601.21768
• Copy Paste: [[2601.21768]] Zonkey: A Hierarchical Diffusion Language Model with Differentiable Tokenization and Probabilistic Attention(https://arxiv.org/abs/2601.21768)
• Keywords: diffusion, large language model
• Abstract: Large language models (LLMs) have revolutionized natural language processing, yet they remain constrained by fixed, non-differentiable tokenizers like Byte Pair Encoding (BPE), which hinder end-to-end optimization and adaptability to noisy or domain-specific data. We introduce Zonkey, a hierarchical diffusion model that addresses these limitations through a fully trainable pipeline from raw characters to document-level representations. At its core is a differentiable tokenizer (Segment Splitter) that learns probabilistic beginning-of-sequence (BOS) decisions, enabling adaptive splits that emerge as linguistically meaningful (e.g., word boundaries at spaces, sentence starts at periods) without explicit supervision. This differentiability is enabled by our novel Probabilistic Attention mechanism, which incorporates position-specific existence probabilities to simulate soft masking over theoretically infinite sequences while preserving gradients. Sequences decay probabilistically rather than relying on end-of-sequence tokens, supporting variable-length outputs. Hierarchical levels compress sequences into higher abstractions (e.g., character n-grams to word-like vectors, then sentence-like), with reconstruction via our Denoising Diffusion Mixed Model (DDMM) for stable and efficient denoising in latent space. A Stitcher ensures overlap invariance across segments. Trained end-to-end on Wikipedia, Zonkey generates coherent, variable-length text from noise, demonstrating emergent hierarchies and promising qualitative alignment to data distributions compared to entropy-based learnable tokenizers. Our approach advances toward fully gradient-based LLMs, with potential for better domain adaptation and scalable generation. We release the source code for training and reproducing our experiments.

Title: Quantum LEGO Learning: A Modular Design Principle for Hybrid Artificial Intelligence

• Authors: Jun Qi, Chao-Han Huck Yang, Pin-Yu Chen, Min-Hsiu Hsieh, Hector Zenil, Jesper Tegner
• Subjects: cs.LG, quant-ph
• Abstract URL: https://arxiv.org/abs/2601.21780
• Pdf URL: https://arxiv.org/pdf/2601.21780
• Copy Paste: [[2601.21780]] Quantum LEGO Learning: A Modular Design Principle for Hybrid Artificial Intelligence(https://arxiv.org/abs/2601.21780)
• Keywords: robust
• Abstract: Hybrid quantum-classical learning models increasingly integrate neural networks with variational quantum circuits (VQCs) to exploit complementary inductive biases. However, many existing approaches rely on tightly coupled architectures or task-specific encoders, limiting conceptual clarity, generality, and transferability across learning settings. In this work, we introduce Quantum LEGO Learning, a modular and architecture-agnostic learning framework that treats classical and quantum components as reusable, composable learning blocks with well-defined roles. Within this framework, a pre-trained classical neural network serves as a frozen feature block, while a VQC acts as a trainable adaptive module that operates on structured representations rather than raw inputs. This separation enables efficient learning under constrained quantum resources and provides a principled abstraction for analyzing hybrid models. We develop a block-wise generalization theory that decomposes learning error into approximation and estimation components, explicitly characterizing how the complexity and training status of each block influence overall performance. Our analysis generalizes prior tensor-network-specific results and identifies conditions under which quantum modules provide representational advantages over comparably sized classical heads. Empirically, we validate the framework through systematic block-swap experiments across frozen feature extractors and both quantum and classical adaptive heads. Experiments on quantum dot classification demonstrate stable optimization, reduced sensitivity to qubit count, and robustness to realistic noise.

Title: Synthetic-to-Real Domain Bridging for Single-View 3D Reconstruction of Ships for Maritime Monitoring

• Authors: Borja Carrillo-Perez, Felix Sattler, Angel Bueno Rodriguez, Maurice Stephan, Sarah Barnes
• Subjects: cs.CV, cs.AI, cs.GR
• Abstract URL: https://arxiv.org/abs/2601.21786
• Pdf URL: https://arxiv.org/pdf/2601.21786
• Copy Paste: [[2601.21786]] Synthetic-to-Real Domain Bridging for Single-View 3D Reconstruction of Ships for Maritime Monitoring(https://arxiv.org/abs/2601.21786)
• Keywords: segmentation
• Abstract: Three-dimensional (3D) reconstruction of ships is an important part of maritime monitoring, allowing improved visualization, inspection, and decision-making in real-world monitoring environments. However, most state-ofthe-art 3D reconstruction methods require multi-view supervision, annotated 3D ground truth, or are computationally intensive, making them impractical for real-time maritime deployment. In this work, we present an efficient pipeline for single-view 3D reconstruction of real ships by training entirely on synthetic data and requiring only a single view at inference. Our approach uses the Splatter Image network, which represents objects as sparse sets of 3D Gaussians for rapid and accurate reconstruction from single images. The model is first fine-tuned on synthetic ShapeNet vessels and further refined with a diverse custom dataset of 3D ships, bridging the domain gap between synthetic and real-world imagery. We integrate a state-of-the-art segmentation module based on YOLOv8 and custom preprocessing to ensure compatibility with the reconstruction network. Postprocessing steps include real-world scaling, centering, and orientation alignment, followed by georeferenced placement on an interactive web map using AIS metadata and homography-based mapping. Quantitative evaluation on synthetic validation data demonstrates strong reconstruction fidelity, while qualitative results on real maritime images from the ShipSG dataset confirm the potential for transfer to operational maritime settings. The final system provides interactive 3D inspection of real ships without requiring real-world 3D annotations. This pipeline provides an efficient, scalable solution for maritime monitoring and highlights a path toward real-time 3D ship visualization in practical applications. Interactive demo: this https URL.

Title: ECSEL: Explainable Classification via Signomial Equation Learning

• Authors: Adia Lumadjeng, Ilker Birbil, Erman Acar
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2601.21789
• Pdf URL: https://arxiv.org/pdf/2601.21789
• Copy Paste: [[2601.21789]] ECSEL: Explainable Classification via Signomial Equation Learning(https://arxiv.org/abs/2601.21789)
• Keywords: interpretability
• Abstract: We introduce ECSEL, an explainable classification method that learns formal expressions in the form of signomial equations, motivated by the observation that many symbolic regression benchmarks admit compact signomial structure. ECSEL directly constructs a structural, closed-form expression that serves as both a classifier and an explanation. On standard symbolic regression benchmarks, our method recovers a larger fraction of target equations than competing state-of-the-art approaches while requiring substantially less computation. Leveraging this efficiency, ECSEL achieves classification accuracy competitive with established machine learning models without sacrificing interpretability. Further, we show that ECSEL satisfies some desirable properties regarding global feature behavior, decision-boundary analysis, and local feature attributions. Experiments on benchmark datasets and two real-world case studies i.e., e-commerce and fraud detection, demonstrate that the learned equations expose dataset biases, support counterfactual reasoning, and yield actionable insights.

Title: NetMamba+: A Framework of Pre-trained Models for Efficient and Accurate Network Traffic Classification

• Authors: Tongze Wang, Xiaohui Xie, Wenduo Wang, Chuyi Wang, Jinzhou Liu, Boyan Huang, Yannan Hu, Youjian Zhao, Yong Cui
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21792
• Pdf URL: https://arxiv.org/pdf/2601.21792
• Copy Paste: [[2601.21792]] NetMamba+: A Framework of Pre-trained Models for Efficient and Accurate Network Traffic Classification(https://arxiv.org/abs/2601.21792)
• Keywords: security, robust, transformer
• Abstract: With the rapid growth of encrypted network traffic, effective traffic classification has become essential for network security and quality of service management. Current machine learning and deep learning approaches for traffic classification face three critical challenges: computational inefficiency of Transformer architectures, inadequate traffic representations with loss of crucial byte-level features while retaining detrimental biases, and poor handling of long-tail distributions in real-world data. We propose NetMamba+, a framework that addresses these challenges through three key innovations: (1) an efficient architecture considering Mamba and Flash Attention mechanisms, (2) a multimodal traffic representation scheme that preserves essential traffic information while eliminating biases, and (3) a label distribution-aware fine-tuning strategy. Evaluation experiments on massive datasets encompassing four main classification tasks showcase NetMamba+'s superior classification performance compared to state-of-the-art baselines, with improvements of up to 6.44\% in F1 score. Moreover, NetMamba+ demonstrates excellent efficiency, achieving 1.7x higher inference throughput than the best baseline while maintaining comparably low memory usage. Furthermore, NetMamba+ exhibits superior few-shot learning abilities, achieving better classification performance with fewer labeled data. Additionally, we implement an online traffic classification system that demonstrates robust real-world performance with a throughput of 261.87 Mb/s. As the first framework to adapt Mamba architecture for network traffic classification, NetMamba+ opens new possibilities for efficient and accurate traffic analysis in complex network environments.

Title: Knowledge Vector Weakening: Efficient Training-free Unlearning for Large Vision-Language Models

• Authors: Yejin Kim, Dongjun Hwang, Sungmin Cha, Junsuk Choe
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21794
• Pdf URL: https://arxiv.org/pdf/2601.21794
• Copy Paste: [[2601.21794]] Knowledge Vector Weakening: Efficient Training-free Unlearning for Large Vision-Language Models(https://arxiv.org/abs/2601.21794)
• Keywords: privacy
• Abstract: Large Vision-Language Models (LVLMs) are widely adopted for their strong multimodal capabilities, yet they raise serious concerns such as privacy leakage and harmful content generation. Machine unlearning has emerged as a promising solution for removing the influence of specific data from trained models. However, existing approaches largely rely on gradient-based optimization, incurring substantial computational costs for large-scale LVLMs. To address this limitation, we propose Knowledge Vector Weakening (KVW), a training-free unlearning method that directly intervenes in the full model without gradient computation. KVW identifies knowledge vectors that are activated during the model's output generation on the forget set and progressively weakens their contributions, thereby preventing the model from exploiting undesirable knowledge. Experiments on the MLLMU and CLEAR benchmarks demonstrate that KVW achieves a stable forget-retain trade-off while significantly improving computational efficiency over gradient-based and LoRA-based unlearning methods.

Title: Effective LoRA Adapter Routing using Task Representations

• Authors: Akash Dhasade, Anne-Marie Kermarrec, Igor Pavlovic, Diana Petrescu, Rafael Pires, Mathis Randl, Martijn de Vos
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21795
• Pdf URL: https://arxiv.org/pdf/2601.21795
• Copy Paste: [[2601.21795]] Effective LoRA Adapter Routing using Task Representations(https://arxiv.org/abs/2601.21795)
• Keywords: robust, large language model
• Abstract: Low-rank adaptation (LoRA) enables parameter efficient specialization of large language models (LLMs) through modular adapters, resulting in rapidly growing public adapter pools spanning diverse tasks. Effectively using these adapters requires routing: selecting and composing the appropriate adapters for a query. We introduce LORAUTER, a novel routing framework that selects and composes LoRA adapters using task representations rather than adapter characteristics. Unlike existing approaches that map queries directly to adapters, LORAUTER routes queries via task embeddings derived from small validation sets and does not require adapter training data. By operating at the task level, LORAUTER achieves efficient routing that scales with the number of tasks rather than the number of adapters. Experiments across multiple tasks show that LORAUTER consistently outperforms baseline routing approaches, matching Oracle performance (101.2%) when task-aligned adapters exist and achieving state-of-the-art results on unseen tasks (+5.2 points). We further demonstrate the robustness of LORAUTER to very large, noisy adapter pools by scaling it to over 1500 adapters.

Title: KID: Knowledge-Injected Dual-Head Learning for Knowledge-Grounded Harmful Meme Detection

• Authors: Yaocong Li, Leihan Zhang, Le Zhang, Qiang Yan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21796
• Pdf URL: https://arxiv.org/pdf/2601.21796
• Copy Paste: [[2601.21796]] KID: Knowledge-Injected Dual-Head Learning for Knowledge-Grounded Harmful Meme Detection(https://arxiv.org/abs/2601.21796)
• Keywords: robust
• Abstract: Internet memes have become pervasive carriers of digital culture on social platforms. However, their heavy reliance on metaphors and sociocultural context also makes them subtle vehicles for harmful content, posing significant challenges for automated content moderation. Existing approaches primarily focus on intra-modal and inter-modal signal analysis, while the understanding of implicit toxicity often depends on background knowledge that is not explicitly present in the meme itself. To address this challenge, we propose KID, a Knowledge-Injected Dual-Head Learning framework for knowledge-grounded harmful meme detection. KID adopts a label-constrained distillation paradigm to decompose complex meme understanding into structured reasoning chains that explicitly link visual evidence, background knowledge, and classification labels. These chains guide the learning process by grounding external knowledge in meme-specific contexts. In addition, KID employs a dual-head architecture that jointly optimizes semantic generation and classification objectives, enabling aligned linguistic reasoning while maintaining stable decision boundaries. Extensive experiments on five multilingual datasets spanning English, Chinese, and low-resource Bengali demonstrate that KID achieves SOTA performance on both binary and multi-label harmful meme detection tasks, improving over previous best methods by 2.1%--19.7% across primary evaluation metrics. Ablation studies further confirm the effectiveness of knowledge injection and dual-head joint learning, highlighting their complementary contributions to robust and generalizable meme understanding. The code and data are available at this https URL.

Title: CG-MLLM: Captioning and Generating 3D content via Multi-modal Large Language Models

• Authors: Junming Huang, Weiwei Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21798
• Pdf URL: https://arxiv.org/pdf/2601.21798
• Copy Paste: [[2601.21798]] CG-MLLM: Captioning and Generating 3D content via Multi-modal Large Language Models(https://arxiv.org/abs/2601.21798)
• Keywords: transformer, large language model
• Abstract: Large Language Models(LLMs) have revolutionized text generation and multimodal perception, but their capabilities in 3D content generation remain underexplored. Existing methods compromise by producing either low-resolution meshes or coarse structural proxies, failing to capture fine-grained geometry natively. In this paper, we propose CG-MLLM, a novel Multi-modal Large Language Model (MLLM) capable of 3D captioning and high-resolution 3D generation in a single framework. Leveraging the Mixture-of-Transformer architecture, CG-MLLM decouples disparate modeling needs, where the Token-level Autoregressive (TokenAR) Transformer handles token-level content, and the Block-level Autoregressive (BlockAR) Transformer handles block-level content. By integrating a pre-trained vision-language backbone with a specialized 3D VAE latent space, CG-MLLM facilitates long-context interactions between standard tokens and spatial blocks within a single integrated architecture. Experimental results show that CG-MLLM significantly outperforms existing MLLMs in generating high-fidelity 3D objects, effectively bringing high-resolution 3D content creation into the mainstream LLM paradigm.

Title: RAG-E: Quantifying Retriever-Generator Alignment and Failure Modes

• Authors: Korbinian Randl, Guido Rocchietti, Aron Henriksson, Ziawasch Abedjan, Tony Lindgren, John Pavlopoulos
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21803
• Pdf URL: https://arxiv.org/pdf/2601.21803
• Copy Paste: [[2601.21803]] RAG-E: Quantifying Retriever-Generator Alignment and Failure Modes(https://arxiv.org/abs/2601.21803)
• Keywords: explainability
• Abstract: Retrieval-Augmented Generation (RAG) systems combine dense retrievers and language models to ground LLM outputs in retrieved documents. However, the opacity of how these components interact creates challenges for deployment in high-stakes domains. We present RAG-E, an end-to-end explainability framework that quantifies retriever-generator alignment through mathematically grounded attribution methods. Our approach adapts Integrated Gradients for retriever analysis, introduces PMCSHAP, a Monte Carlo-stabilized Shapley Value approximation, for generator attribution, and introduces the Weighted Attribution-Relevance Gap (WARG) metric to measure how well a generator's document usage aligns with a retriever's ranking. Empirical analysis on TREC CAsT and FoodSafeSum reveals critical misalignments: for 47.4% to 66.7% of queries, generators ignore the retriever's top-ranked documents, while 48.1% to 65.9% rely on documents ranked as less relevant. These failure modes demonstrate that RAG output quality depends not solely on individual component performance but on their interplay, which can be audited via RAG-E.

Title: Distribution-Aware Reward Estimation for Test-Time Reinforcement Learning

• Authors: Bodong Du, Xuanqi Huang, Xiaomeng Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21804
• Pdf URL: https://arxiv.org/pdf/2601.21804
• Copy Paste: [[2601.21804]] Distribution-Aware Reward Estimation for Test-Time Reinforcement Learning(https://arxiv.org/abs/2601.21804)
• Keywords: robust, large language model
• Abstract: Test-time reinforcement learning (TTRL) enables large language models (LLMs) to self-improve on unlabeled inputs, but its effectiveness critically depends on how reward signals are estimated without ground-truth supervision. Most existing TTRL methods rely on majority voting (MV) over rollouts to produce deterministic rewards, implicitly assuming that the majority rollout provides a reliable learning signal. We show that this assumption is fragile: MV reduces the rollout distribution into a single outcome, discarding information about non-majority but correct actions candidates, and yields systematically biased reward estimates. To address this, we propose Distribution-AwareReward Estimation (DARE), which shifts reward estimation from a single majority outcome to the full empirical rollout distribution. DARE further augments this distribution-based reward with an exploration bonus and a distribution pruning mechanism for non-majority rollout exploration and reward denoise, yielding a more informative and robust reward estimation. Extensive experiments on challenging reasoning benchmarks show that DARE improves optimization stability and final performance over recent baselines, achieving relative improvements of 25.3% on challenging AIME 2024 and 5.3% on AMC.

Title: Nonparametric LLM Evaluation from Preference Data

• Authors: Dennis Frauen, Athiya Deviyani, Mihaela van der Schaar, Stefan Feuerriegel
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21816
• Pdf URL: https://arxiv.org/pdf/2601.21816
• Copy Paste: [[2601.21816]] Nonparametric LLM Evaluation from Preference Data(https://arxiv.org/abs/2601.21816)
• Keywords: large language model
• Abstract: Evaluating the performance of large language models (LLMs) from human preference data is crucial for obtaining LLM leaderboards. However, many existing approaches either rely on restrictive parametric assumptions or lack valid uncertainty quantification when flexible machine learning methods are used. In this paper, we propose a nonparametric statistical framework, DMLEval, for comparing and ranking LLMs from preference data using debiased machine learning (DML). For this, we introduce generalized average ranking scores (GARS), which generalize commonly used ranking models, including the Bradley-Terry model or PageRank/ Rank centrality, with complex human responses such as ties. DMLEval comes with the following advantages: (i) It produces statistically efficient estimates of GARS ranking scores. (ii) It naturally allows the incorporation of black-box machine learning methods for estimation. (iii) It can be combined with pre-trained LLM evaluators (e.g., using LLM-as-a-judge). (iv) It suggests optimal policies for collecting preference data under budget constraints. We demonstrate these advantages both theoretically and empirically using both synthetic and real-world preference datasets. In summary, our framework provides practitioners with powerful, state-of-the-art methods for comparing or ranking LLMs.

Title: DASH: Deterministic Attention Scheduling for High-throughput Reproducible LLM Training

• Authors: Xinwei Qiang, Hongmin Chen, Shixuan Sun, Jingwen Leng, Xin Liu, Minyi Guo
• Subjects: cs.LG, cs.DC
• Abstract URL: https://arxiv.org/abs/2601.21824
• Pdf URL: https://arxiv.org/pdf/2601.21824
• Copy Paste: [[2601.21824]] DASH: Deterministic Attention Scheduling for High-throughput Reproducible LLM Training(https://arxiv.org/abs/2601.21824)
• Keywords: large language model
• Abstract: Determinism is indispensable for reproducibility in large language model (LLM) training, yet it often exacts a steep performance cost. In widely used attention implementations such as FlashAttention-3, the deterministic backward pass can incur up to a 37.9% throughput reduction relative to its non-deterministic counterpart, primarily because gradient accumulation operations must be serialized to guarantee numerical consistency. This performance loss stems from suboptimal scheduling of compute and gradient-reduction phases, leading to significant hardware underutilization. To address this challenge, we formulate the backward pass of deterministic attention as a scheduling problem on a Directed Acyclic Graph (DAG) and derive schedules that minimize the critical path length. Building on this formulation, we present DASH (Deterministic Attention Scheduling for High-Throughput), which encapsulates two complementary scheduling strategies: (i) Descending Q-Tile Iteration, a reversed query-block traversal that shrinks pipeline stalls in causal attention, and (ii) Shift Scheduling, a theoretically optimal schedule within our DAG model that reduces pipeline stalls for both full and causal masks. Our empirical evaluations on NVIDIA H800 GPUs demonstrate that DASH narrows the performance gap of deterministic attention. The proposed strategies improve the throughput of the attention backward pass by up to 1.28$\times$ compared to the baseline, significantly advancing the efficiency of reproducible LLM training. Our code is open-sourced at this https URL.

Title: Mil-SCORE: Benchmarking Long-Context Geospatial Reasoning and Planning in Large Language Models

• Authors: Aadi Palnitkar, Mingyang Mao, Nicholas Waytowich, Vinicius G. Goecks, Tinoosh Mohsenin, Xiaomin Lin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21826
• Pdf URL: https://arxiv.org/pdf/2601.21826
• Copy Paste: [[2601.21826]] Mil-SCORE: Benchmarking Long-Context Geospatial Reasoning and Planning in Large Language Models(https://arxiv.org/abs/2601.21826)
• Keywords: large language model
• Abstract: As large language models (LLMs) are applied to increasingly longer and more complex tasks, there is a growing need for realistic long-context benchmarks that require selective reading and integration of heterogeneous, multi-modal information sources. This need is especially acute for geospatial planning problems, such as those found in planning for large-scale military operations, which demand fast and accurate reasoning over maps, orders, intelligence reports, and other distributed data. To address this gap, we present MilSCORE (Military Scenario Contextual Reasoning), to our knowledge the first scenario-level dataset of expert-authored, multi-hop questions grounded in a complex, simulated military planning scenario used for training. MilSCORE is designed to evaluate high-stakes decision-making and planning, probing LLMs' ability to combine tactical and spatial reasoning across multiple sources and to reason over long-horizon, geospatially rich context. The benchmark includes a diverse set of question types across seven categories targeting both factual recall and multi-step reasoning about constraints, strategy, and spatial analysis. We provide an evaluation protocol and report baseline results for a range of contemporary vision-language models. Our findings highlight substantial headroom on MilSCORE, indicating that current systems struggle with realistic, scenario-level long-context planning, and positioning MilSCORE as a challenging testbed for future work.

Title: Embodied Task Planning via Graph-Informed Action Generation with Large Lanaguage Model

• Authors: Xiang Li, Ning Yan, Masood Mortazavi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21841
• Pdf URL: https://arxiv.org/pdf/2601.21841
• Copy Paste: [[2601.21841]] Embodied Task Planning via Graph-Informed Action Generation with Large Lanaguage Model(https://arxiv.org/abs/2601.21841)
• Keywords: large language model
• Abstract: While Large Language Models (LLMs) have demonstrated strong zero-shot reasoning capabilities, their deployment as embodied agents still faces fundamental challenges in long-horizon planning. Unlike open-ended text generation, embodied agents must decompose high-level intent into actionable sub-goals while strictly adhering to the logic of a dynamic, observed environment. Standard LLM planners frequently fail to maintain strategy coherence over extended horizons due to context window limitation or hallucinate transitions that violate constraints. We propose GiG, a novel planning framework that structures embodied agents' memory using a Graph-in-Graph architecture. Our approach employs a Graph Neural Network (GNN) to encode environmental states into embeddings, organizing these embeddings into action-connected execution trace graphs within an experience memory bank. By clustering these graph embeddings, the framework enables retrieval of structure-aware priors, allowing agents to ground current decisions in relevant past structural patterns. Furthermore, we introduce a novel bounded lookahead module that leverages symbolic transition logic to enhance the agents' planning capabilities through the grounded action projection. We evaluate our framework on three embodied planning benchmarks-Robotouille Synchronous, Robotouille Asynchronous, and ALFWorld. Our method outperforms state-of-the-art baselines, achieving Pass@1 performance gains of up to 22% on Robotouille Synchronous, 37% on Asynchronous, and 15% on ALFWorld with comparable or lower computational cost.

Title: READY: Reward Discovery for Meta-Black-Box Optimization

• Authors: Zechuan Huang, Zhiguang Cao, Hongshu Guo, Yue-Jiao Gong, Zeyuan Ma
• Subjects: cs.LG, cs.NE
• Abstract URL: https://arxiv.org/abs/2601.21847
• Pdf URL: https://arxiv.org/pdf/2601.21847
• Copy Paste: [[2601.21847]] READY: Reward Discovery for Meta-Black-Box Optimization(https://arxiv.org/abs/2601.21847)
• Keywords: large language model
• Abstract: Meta-Black-Box Optimization (MetaBBO) is an emerging avenue within Optimization community, where algorithm design policy could be meta-learned by reinforcement learning to enhance optimization performance. So far, the reward functions in existing MetaBBO works are designed by human experts, introducing certain design bias and risks of reward hacking. In this paper, we use Large Language Model~(LLM) as an automated reward discovery tool for MetaBBO. Specifically, we consider both effectiveness and efficiency sides. On effectiveness side, we borrow the idea of evolution of heuristics, introducing tailored evolution paradigm in the iterative LLM-based program search process, which ensures continuous improvement. On efficiency side, we additionally introduce multi-task evolution architecture to support parallel reward discovery for diverse MetaBBO approaches. Such parallel process also benefits from knowledge sharing across tasks to accelerate convergence. Empirical results demonstrate that the reward functions discovered by our approach could be helpful for boosting existing MetaBBO works, underscoring the importance of reward design in MetaBBO. We provide READY's project at this https URL.

Title: Visual Disentangled Diffusion Autoencoders: Scalable Counterfactual Generation for Foundation Models

• Authors: Sidney Bender, Marco Morik
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21851
• Pdf URL: https://arxiv.org/pdf/2601.21851
• Copy Paste: [[2601.21851]] Visual Disentangled Diffusion Autoencoders: Scalable Counterfactual Generation for Foundation Models(https://arxiv.org/abs/2601.21851)
• Keywords: robust, diffusion
• Abstract: Foundation models, despite their robust zero-shot capabilities, remain vulnerable to spurious correlations and 'Clever Hans' strategies. Existing mitigation methods often rely on unavailable group labels or computationally expensive gradient-based adversarial optimization. To address these limitations, we propose Visual Disentangled Diffusion Autoencoders (DiDAE), a novel framework integrating frozen foundation models with disentangled dictionary learning for efficient, gradient-free counterfactual generation directly for the foundation model. DiDAE first edits foundation model embeddings in interpretable disentangled directions of the disentangled dictionary and then decodes them via a diffusion autoencoder. This allows the generation of multiple diverse, disentangled counterfactuals for each factual, much faster than existing baselines, which generate single entangled counterfactuals. When paired with Counterfactual Knowledge Distillation, DiDAE-CFKD achieves state-of-the-art performance in mitigating shortcut learning, improving downstream performance on unbalanced datasets.

Title: Trajectory-Guided Diffusion for Foreground-Preserving Background Generation in Multi-Layer Documents

• Authors: Taewon Kang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21857
• Pdf URL: https://arxiv.org/pdf/2601.21857
• Copy Paste: [[2601.21857]] Trajectory-Guided Diffusion for Foreground-Preserving Background Generation in Multi-Layer Documents(https://arxiv.org/abs/2601.21857)
• Keywords: diffusion, generative
• Abstract: We present a diffusion-based framework for document-centric background generation that achieves foreground preservation and multi-page stylistic consistency through latent-space design rather than explicit constraints. Instead of suppressing diffusion updates or applying masking heuristics, our approach reinterprets diffusion as the evolution of stochastic trajectories through a structured latent space. By shaping the initial noise and its geometric alignment, background generation naturally avoids designated foreground regions, allowing readable content to remain intact without auxiliary mechanisms. To address the long-standing issue of stylistic drift across pages, we decouple style control from text conditioning and introduce cached style directions as persistent vectors in latent space. Once selected, these directions constrain diffusion trajectories to a shared stylistic subspace, ensuring consistent appearance across pages and editing iterations. This formulation eliminates the need for repeated prompt-based style specification and provides a more stable foundation for multi-page generation. Our framework admits a geometric and physical interpretation, where diffusion paths evolve on a latent manifold shaped by preferred directions, and foreground regions are rarely traversed as a consequence of trajectory initialization rather than explicit exclusion. The proposed method is training-free, compatible with existing diffusion backbones, and produces visually coherent, foreground-preserving results across complex documents. By reframing diffusion as trajectory design in latent space, we offer a principled approach to consistent and structured generative modeling.

Title: MoHETS: Long-term Time Series Forecasting with Mixture-of-Heterogeneous-Experts

• Authors: Evandro S. Ortigossa, Guy Lutsker, Eran Segal
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21866
• Pdf URL: https://arxiv.org/pdf/2601.21866
• Copy Paste: [[2601.21866]] MoHETS: Long-term Time Series Forecasting with Mixture-of-Heterogeneous-Experts(https://arxiv.org/abs/2601.21866)
• Keywords: robust, transformer
• Abstract: Real-world multivariate time series can exhibit intricate multi-scale structures, including global trends, local periodicities, and non-stationary regimes, which makes long-horizon forecasting challenging. Although sparse Mixture-of-Experts (MoE) approaches improve scalability and specialization, they typically rely on homogeneous MLP experts that poorly capture the diverse temporal dynamics of time series data. We address these limitations with MoHETS, an encoder-only Transformer that integrates sparse Mixture-of-Heterogeneous-Experts (MoHE) layers. MoHE routes temporal patches to a small subset of expert networks, combining a shared depthwise-convolution expert for sequence-level continuity with routed Fourier-based experts for patch-level periodic structures. MoHETS further improves robustness to non-stationary dynamics by incorporating exogenous information via cross-attention over covariate patch embeddings. Finally, we replace parameter-heavy linear projection heads with a lightweight convolutional patch decoder, improving parameter efficiency, reducing training instability, and allowing a single model to generalize across arbitrary forecast horizons. We validate across seven multivariate benchmarks and multiple horizons, with MoHETS consistently achieving state-of-the-art performance, reducing the average MSE by $12\%$ compared to strong recent baselines, demonstrating effective heterogeneous specialization for long-term forecasting.

Title: Managing Solution Stability in Decision-Focused Learning with Cost Regularization

• Authors: Victor Spitzer, Francois Sanson
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21883
• Pdf URL: https://arxiv.org/pdf/2601.21883
• Copy Paste: [[2601.21883]] Managing Solution Stability in Decision-Focused Learning with Cost Regularization(https://arxiv.org/abs/2601.21883)
• Keywords: robust
• Abstract: Decision-focused learning integrates predictive modeling and combinatorial optimization by training models to directly improve decision quality rather than prediction accuracy alone. Differentiating through combinatorial optimization problems represents a central challenge, and recent approaches tackle this difficulty by introducing perturbation-based approximations. In this work, we focus on estimating the objective function coefficients of a combinatorial optimization problem. Our study demonstrates that fluctuations in perturbation intensity occurring during the learning phase can lead to ineffective training, by establishing a theoretical link to the notion of solution stability in combinatorial optimization. We propose addressing this issue by introducing a regularization of the estimated cost vectors which improves the robustness and reliability of the learning process, as demonstrated by extensive numerical experiments.

Title: Improving Classifier-Free Guidance of Flow Matching via Manifold Projection

• Authors: Jian-Feng Cai, Haixia Liu, Zhengyi Su, Chao Wang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21892
• Pdf URL: https://arxiv.org/pdf/2601.21892
• Copy Paste: [[2601.21892]] Improving Classifier-Free Guidance of Flow Matching via Manifold Projection(https://arxiv.org/abs/2601.21892)
• Keywords: robust, diffusion
• Abstract: Classifier-free guidance (CFG) is a widely used technique for controllable generation in diffusion and flow-based models. Despite its empirical success, CFG relies on a heuristic linear extrapolation that is often sensitive to the guidance scale. In this work, we provide a principled interpretation of CFG through the lens of optimization. We demonstrate that the velocity field in flow matching corresponds to the gradient of a sequence of smoothed distance functions, which guides latent variables toward the scaled target image set. This perspective reveals that the standard CFG formulation is an approximation of this gradient, where the prediction gap, the discrepancy between conditional and unconditional outputs, governs guidance sensitivity. Leveraging this insight, we reformulate the CFG sampling as a homotopy optimization with a manifold constraint. This formulation necessitates a manifold projection step, which we implement via an incremental gradient descent scheme during sampling. To improve computational efficiency and stability, we further enhance this iterative process with Anderson Acceleration without requiring additional model evaluations. Our proposed methods are training-free and consistently refine generation fidelity, prompt alignment, and robustness to the guidance scale. We validate their effectiveness across diverse benchmarks, demonstrating significant improvements on large-scale models such as DiT-XL-2-256, Flux, and Stable Diffusion 3.5.

Title: WADBERT: Dual-channel Web Attack Detection Based on BERT Models

• Authors: Kangqiang Luo, Yi Xie, Shiqian Zhao, Jing Pan
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2601.21893
• Pdf URL: https://arxiv.org/pdf/2601.21893
• Copy Paste: [[2601.21893]] WADBERT: Dual-channel Web Attack Detection Based on BERT Models(https://arxiv.org/abs/2601.21893)
• Keywords: defense, attack
• Abstract: Web attack detection is the first line of defense for securing web applications, designed to preemptively identify malicious activities. Deep learning-based approaches are increasingly popular for their advantages: automatically learning complex patterns and extracting semantic features from HTTP requests to achieve superior detection performance. However, existing methods are less effective in embedding irregular HTTP requests, even failing to model unordered parameters and achieve attack traceability. In this paper, we propose an effective web attack detection model, named WADBERT. It achieves high detection accuracy while enabling the precise identification of malicious parameters. To this end, we first employ Hybrid Granularity Embedding (HGE) to generate fine-grained embeddings for URL and payload parameters. Then, URLBERT and SecBERT are respectively utilized to extract their semantic features. Further, parameter-level features (extracted by SecBERT) are fused through a multi-head attention mechanism, resulting in a comprehensive payload feature. Finally, by feeding the concatenated URL and payload features into a linear classifier, a final detection result is obtained. The experimental results on CSIC2010 and SR-BH2020 datasets validate the efficacy of WADBERT, which respectively achieves F1-scores of 99.63% and 99.50%, and significantly outperforms state-of-the-art methods.

Title: Not All Code Is Equal: A Data-Centric Study of Code Complexity and LLM Reasoning

• Authors: Lukas Twist, Shu Yang, Hanqi Yan, Jingzhi Gong, Di Wang, Helen Yannakoudakis, Jie M. Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21894
• Pdf URL: https://arxiv.org/pdf/2601.21894
• Copy Paste: [[2601.21894]] Not All Code Is Equal: A Data-Centric Study of Code Complexity and LLM Reasoning(https://arxiv.org/abs/2601.21894)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) increasingly exhibit strong reasoning abilities, often attributed to their capacity to generate chain-of-thought-style intermediate reasoning. Recent work suggests that exposure to code can further enhance these skills, but existing studies largely treat code as a generic training signal, leaving open the question of which properties of code actually contribute to improved reasoning. To address this gap, we study the structural complexity of code, which captures control flow and compositional structure that may shape how models internalise multi-step reasoning during fine-tuning. We examine two complementary settings: solution-driven complexity, where complexity varies across multiple solutions to the same problem, and problem-driven complexity, where complexity reflects variation in the underlying tasks. Using cyclomatic complexity and logical lines of code to construct controlled fine-tuning datasets, we evaluate a range of open-weight LLMs on diverse reasoning benchmarks. Our findings show that although code can improve reasoning, structural properties strongly determine its usefulness. In 83% of experiments, restricting fine-tuning data to a specific structural complexity range outperforms training on structurally diverse code, pointing to a data-centric path for improving reasoning beyond scaling.

Title: Learn-to-Distance: Distance Learning for Detecting LLM-Generated Text

• Authors: Hongyi Zhou, Jin Zhu, Erhan Xu, Kai Ye, Ying Yang, Chengchun Shi
• Subjects: cs.CL, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2601.21895
• Pdf URL: https://arxiv.org/pdf/2601.21895
• Copy Paste: [[2601.21895]] Learn-to-Distance: Distance Learning for Detecting LLM-Generated Text(https://arxiv.org/abs/2601.21895)
• Keywords: large language model
• Abstract: Modern large language models (LLMs) such as GPT, Claude, and Gemini have transformed the way we learn, work, and communicate. Yet, their ability to produce highly human-like text raises serious concerns about misinformation and academic integrity, making it an urgent need for reliable algorithms to detect LLM-generated content. In this paper, we start by presenting a geometric approach to demystify rewrite-based detection algorithms, revealing their underlying rationale and demonstrating their generalization ability. Building on this insight, we introduce a novel rewrite-based detection algorithm that adaptively learns the distance between the original and rewritten text. Theoretically, we demonstrate that employing an adaptively learned distance function is more effective for detection than using a fixed distance. Empirically, we conduct extensive experiments with over 100 settings, and find that our approach demonstrates superior performance over baseline algorithms in the majority of scenarios. In particular, it achieves relative improvements from 57.8\% to 80.6\% over the strongest baseline across different target LLMs (e.g., GPT, Claude, and Gemini).

Title: Past- and Future-Informed KV Cache Policy with Salience Estimation in Autoregressive Video Diffusion

• Authors: Hanmo Chen, Chenghao Xu, Xu Yang, Xuan Chen, Cheng Deng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21896
• Pdf URL: https://arxiv.org/pdf/2601.21896
• Copy Paste: [[2601.21896]] Past- and Future-Informed KV Cache Policy with Salience Estimation in Autoregressive Video Diffusion(https://arxiv.org/abs/2601.21896)
• Keywords: diffusion
• Abstract: Video generation is pivotal to digital media creation, and recent advances in autoregressive video generation have markedly enhanced the efficiency of real-time video synthesis. However, existing approaches generally rely on heuristic KV Cache policies, which ignore differences in token importance in long-term video generation. This leads to the loss of critical spatiotemporal information and the accumulation of redundant, invalid cache, thereby degrading video generation quality and efficiency. To address this limitation, we first observe that token contributions to video generation are highly time-heterogeneous and accordingly propose a novel Past- and Future-Informed KV Cache Policy (PaFu-KV). Specifically, PaFu-KV introduces a lightweight Salience Estimation Head distilled from a bidirectional teacher to estimate salience scores, allowing the KV cache to retain informative tokens while discarding less relevant ones. This policy yields a better quality-efficiency trade-off by shrinking KV cache capacity and reducing memory footprint at inference time. Extensive experiments on benchmarks demonstrate that our method preserves high-fidelity video generation quality while enables accelerated inference, thereby enabling more efficient long-horizon video generation. Our code will be released upon paper acceptance.

Title: A Low-Complexity Plug-and-Play Deep Learning Model for Generalizable Massive MIMO Precoding

• Authors: Ali Hasanzadeh Karkan, Ahmed Ibrahim, Jean-François Frigon, François Leduc-Primeau
• Subjects: cs.LG, eess.SP
• Abstract URL: https://arxiv.org/abs/2601.21897
• Pdf URL: https://arxiv.org/pdf/2601.21897
• Copy Paste: [[2601.21897]] A Low-Complexity Plug-and-Play Deep Learning Model for Generalizable Massive MIMO Precoding(https://arxiv.org/abs/2601.21897)
• Keywords: robust
• Abstract: Massive multiple-input multiple-output (mMIMO) downlink precoding offers high spectral efficiency but remains challenging to deploy in practice because near-optimal algorithms such as the weighted minimum mean squared error (WMMSE) are computationally expensive, and sensitive to SNR and channel-estimation quality, while existing deep learning (DL)-based solutions often lack robustness and require retraining for each deployment site. This paper proposes a plug-and-play precoder (PaPP), a DL framework with a backbone that can be trained for either fully digital (FDP) or hybrid beamforming (HBF) precoding and reused across sites, transmit-power levels, and with varying amounts of channel estimation error, avoiding the need to train a new model from scratch at each deployment. PaPP combines a high-capacity teacher and a compact student with a self-supervised loss that balances teacher imitation and normalized sum-rate, trained using meta-learning domain-generalization and transmit-power-aware input normalization. Numerical results on ray-tracing data from three unseen sites show that the PaPP FDP and HBF models both outperform conventional and deep learning baselines, after fine-tuning with a small set of local unlabeled samples. Across both architectures, PaPP achieves more than 21$\times$ reduction in modeled computation energy and maintains good performance under channel-estimation errors, making it a practical solution for energy-efficient mMIMO precoding.

Title: Breaking the Regional Barrier: Inductive Semantic Topology Learning for Worldwide Air Quality Forecasting

• Authors: Zhiqing Cui, Siru Zhong, Ming Jin, Shirui Pan, Qingsong Wen, Yuxuan Liang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21899
• Pdf URL: https://arxiv.org/pdf/2601.21899
• Copy Paste: [[2601.21899]] Breaking the Regional Barrier: Inductive Semantic Topology Learning for Worldwide Air Quality Forecasting(https://arxiv.org/abs/2601.21899)
• Keywords: diffusion
• Abstract: Global air quality forecasting grapples with extreme spatial heterogeneity and the poor generalization of existing transductive models to unseen regions. To tackle this, we propose OmniAir, a semantic topology learning framework tailored for global station-level prediction. By encoding invariant physical environmental attributes into generalizable station identities and dynamically constructing adaptive sparse topologies, our approach effectively captures long-range non-Euclidean correlations and physical diffusion patterns across unevenly distributed global networks. We further curate WorldAir, a massive dataset covering over 7,800 stations worldwide. Extensive experiments show that OmniAir achieves state-of-the-art performance against 18 baselines, maintaining high efficiency and scalability with speeds nearly 10 times faster than existing models, while effectively bridging the monitoring gap in data-sparse regions.

Title: TraceRouter: Robust Safety for Large Foundation Models via Path-Level Intervention

• Authors: Chuancheng Shi, Shangze Li, Wenjun Lu, Wenhua Wu, Cong Wang, Zifeng Cheng, Fei Shen, Tat-Seng Chua
• Subjects: cs.CV, cs.AI, cs.CY, cs.MM
• Abstract URL: https://arxiv.org/abs/2601.21900
• Pdf URL: https://arxiv.org/pdf/2601.21900
• Copy Paste: [[2601.21900]] TraceRouter: Robust Safety for Large Foundation Models via Path-Level Intervention(https://arxiv.org/abs/2601.21900)
• Keywords: defense, robust
• Abstract: Despite their capabilities, large foundation models (LFMs) remain susceptible to adversarial manipulation. Current defenses predominantly rely on the "locality hypothesis", suppressing isolated neurons or features. However, harmful semantics act as distributed, cross-layer circuits, rendering such localized interventions brittle and detrimental to utility. To bridge this gap, we propose \textbf{TraceRouter}, a path-level framework that traces and disconnects the causal propagation circuits of illicit semantics. TraceRouter operates in three stages: (1) it pinpoints a sensitive onset layer by analyzing attention divergence; (2) it leverages sparse autoencoders (SAEs) and differential activation analysis to disentangle and isolate malicious features; and (3) it maps these features to downstream causal pathways via feature influence scores (FIS) derived from zero-out interventions. By selectively suppressing these causal chains, TraceRouter physically severs the flow of harmful information while leaving orthogonal computation routes intact. Extensive experiments demonstrate that TraceRouter significantly outperforms state-of-the-art baselines, achieving a superior trade-off between adversarial robustness and general utility. Our code will be publicly released. WARNING: This paper contains unsafe model responses.

Title: Hardware-Triggered Backdoors

• Authors: Jonas Möller, Erik Imgrund, Thorsten Eisenhofer, Konrad Rieck
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2601.21902
• Pdf URL: https://arxiv.org/pdf/2601.21902
• Copy Paste: [[2601.21902]] Hardware-Triggered Backdoors(https://arxiv.org/abs/2601.21902)
• Keywords: defense, attack
• Abstract: Machine learning models are routinely deployed on a wide range of computing hardware. Although such hardware is typically expected to produce identical results, differences in its design can lead to small numerical variations during inference. In this work, we show that these variations can be exploited to create backdoors in machine learning models. The core idea is to shape the model's decision function such that it yields different predictions for the same input when executed on different hardware. This effect is achieved by locally moving the decision boundary close to a target input and then refining numerical deviations to flip the prediction on selected hardware. We empirically demonstrate that these hardware-triggered backdoors can be created reliably across common GPU accelerators. Our findings reveal a novel attack vector affecting the use of third-party models, and we investigate different defenses to counter this threat.

Title: Beyond the Finite Variant Property: Extending Symbolic Diffie-Hellman Group Models (Extended Version)

• Authors: Sofia Giampietro, Ralf Sasse, David Basin
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2601.21910
• Pdf URL: https://arxiv.org/pdf/2601.21910
• Copy Paste: [[2601.21910]] Beyond the Finite Variant Property: Extending Symbolic Diffie-Hellman Group Models (Extended Version)(https://arxiv.org/abs/2601.21910)
• Keywords: security, attack
• Abstract: Diffie-Hellman groups are commonly used in cryptographic protocols. While most state-of-the-art, symbolic protocol verifiers support them to some degree, they do not support all mathematical operations possible in these groups. In particular, they lack support for exponent addition, as these tools reason about terms using unification, which is undecidable in the theory describing all Diffie-Hellman operators. In this paper we approximate such a theory and propose a semi-decision procedure to determine whether a protocol, which may use all operations in such groups, satisfies user-defined properties. We implement this approach by extending the Tamarin prover to support the full Diffie-Hellman theory, including group element multiplication and hence addition of exponents. This is the first time a state-of-the-art tool can model and reason about such protocols. We illustrate our approach's effectiveness with different case studies: ElGamal encryption and MQV. Using Tamarin, we prove security properties of ElGamal, and we rediscover known attacks on MQV.

Title: Zero-Shot Video Restoration and Enhancement with Assistance of Video Diffusion Models

• Authors: Cong Cao, Huanjing Yue, Shangbin Xie, Xin Liu, Jingyu Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21922
• Pdf URL: https://arxiv.org/pdf/2601.21922
• Copy Paste: [[2601.21922]] Zero-Shot Video Restoration and Enhancement with Assistance of Video Diffusion Models(https://arxiv.org/abs/2601.21922)
• Keywords: diffusion
• Abstract: Although diffusion-based zero-shot image restoration and enhancement methods have achieved great success, applying them to video restoration or enhancement will lead to severe temporal flickering. In this paper, we propose the first framework that utilizes the rapidly-developed video diffusion model to assist the image-based method in maintaining more temporal consistency for zero-shot video restoration and enhancement. We propose homologous latents fusion, heterogenous latents fusion, and a COT-based fusion ratio strategy to utilize both homologous and heterogenous text-to-video diffusion models to complement the image method. Moreover, we propose temporal-strengthening post-processing to utilize the image-to-video diffusion model to further improve temporal consistency. Our method is training-free and can be applied to any diffusion-based image restoration and enhancement methods. Experimental results demonstrate the superiority of the proposed method.

Title: Just Noticeable Difference Modeling for Deep Visual Features

• Authors: Rui Zhao, Wenrui Li, Lin Zhu, Yajing Zheng, Weisi Lin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21933
• Pdf URL: https://arxiv.org/pdf/2601.21933
• Copy Paste: [[2601.21933]] Just Noticeable Difference Modeling for Deep Visual Features(https://arxiv.org/abs/2601.21933)
• Keywords: segmentation
• Abstract: Deep visual features are increasingly used as the interface in vision systems, motivating the need to describe feature characteristics and control feature quality for machine perception. Just noticeable difference (JND) characterizes the maximum imperceptible distortion for images under human or machine vision. Extending it to deep visual features naturally meets the above demand by providing a task-aligned tolerance boundary in feature space, offering a practical reference for controlling feature quality under constrained resources. We propose FeatJND, a task-aligned JND formulation that predicts the maximum tolerable per-feature perturbation map while preserving downstream task performance. We propose a FeatJND estimator at standardized split points and validate it across image classification, detection, and instance segmentation. Under matched distortion strength, FeatJND-based distortions consistently preserve higher task performance than unstructured Gaussian perturbations, and attribution visualizations suggest FeatJND can suppress non-critical feature regions. As an application, we further apply FeatJND to token-wise dynamic quantization and show that FeatJND-guided step-size allocation yields clear gains over random step-size permutation and global uniform step size under the same noise budget. Our code will be released after publication.

Title: Robust Multimodal Representation Learning in Healthcare

• Authors: Xiaoguang Zhu, Linxiao Gong, Lianlong Sun, Yang Liu, Haoyu Wang, Jing Liu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21941
• Pdf URL: https://arxiv.org/pdf/2601.21941
• Copy Paste: [[2601.21941]] Robust Multimodal Representation Learning in Healthcare(https://arxiv.org/abs/2601.21941)
• Keywords: robust
• Abstract: Medical multimodal representation learning aims to integrate heterogeneous data into unified patient representations to support clinical outcome prediction. However, real-world medical datasets commonly contain systematic biases from multiple sources, which poses significant challenges for medical multimodal representation learning. Existing approaches typically focus on effective multimodal fusion, neglecting inherent biased features that affect the generalization ability. To address these challenges, we propose a Dual-Stream Feature Decorrelation Framework that identifies and handles the biases through structural causal analysis introduced by latent confounders. Our method employs a causal-biased decorrelation framework with dual-stream neural networks to disentangle causal features from spurious correlations, utilizing generalized cross-entropy loss and mutual information minimization for effective decorrelation. The framework is model-agnostic and can be integrated into existing medical multimodal learning methods. Comprehensive experiments on MIMIC-IV, eICU, and ADNI datasets demonstrate consistent performance improvements.

Title: Entropy-Based Dimension-Free Convergence and Loss-Adaptive Schedules for Diffusion Models

• Authors: Ahmad Aghapour, Erhan Bayraktar, Ziqing Zhang
• Subjects: cs.LG, cs.IT
• Abstract URL: https://arxiv.org/abs/2601.21943
• Pdf URL: https://arxiv.org/pdf/2601.21943
• Copy Paste: [[2601.21943]] Entropy-Based Dimension-Free Convergence and Loss-Adaptive Schedules for Diffusion Models(https://arxiv.org/abs/2601.21943)
• Keywords: diffusion, generative
• Abstract: Diffusion generative models synthesize samples by discretizing reverse-time dynamics driven by a learned score (or denoiser). Existing convergence analyses of diffusion models typically scale at least linearly with the ambient dimension, and sharper rates often depend on intrinsic-dimension assumptions or other geometric restrictions on the target distribution. We develop an alternative, information-theoretic approach to dimension-free convergence that avoids any geometric assumptions. Under mild assumptions on the target distribution, we bound KL divergence between the target and generated distributions by $O(H^2/K)$ (up to endpoint factors), where $H$ is the Shannon entropy and $K$ is the number of sampling steps. Moreover, using a reformulation of the KL divergence, we propose a Loss-Adaptive Schedule (LAS) for efficient discretization of reverse SDE which is lightweight and relies only on the training loss, requiring no post-training heavy computation. Empirically, LAS improves sampling quality over common heuristic schedules.

Title: Clarity: The Flexibility-Interpretability Trade-Off in Sparsity-aware Concept Bottleneck Models

• Authors: Konstantinos P. Panousis, Diego Marcos
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21944
• Pdf URL: https://arxiv.org/pdf/2601.21944
• Copy Paste: [[2601.21944]] Clarity: The Flexibility-Interpretability Trade-Off in Sparsity-aware Concept Bottleneck Models(https://arxiv.org/abs/2601.21944)
• Keywords: interpretability
• Abstract: The widespread adoption of Vision-Language Models (VLMs) across fields has amplified concerns about model interpretability. Distressingly, these models are often treated as black-boxes, with limited or non-existent investigation of their decision making process. Despite numerous post- and ante-hoc interepretability methods, systematic and objective evaluation of the learned representations remains limited, particularly for sparsity-aware methods that are increasingly considered to "induce interpretability". In this work, we focus on Concept Bottleneck Models and investigate how different modeling decisions affect the emerging representations. We introduce the notion of clarity, a measure, capturing the interplay between the downstream performance and the sparsity and precision of the concept representation, while proposing an interpretability assessment framework using datasets with ground truth concept annotations. We consider both VLM- and attribute predictor-based CBMs, and three different sparsity-inducing strategies: per example $\ell_1, \ell_0$ and Bernoulli-based formulations. Our experiments reveal a critical trade-off between flexibility and interpretability, under which a given method can exhibit markedly different behaviors even at comparable performance levels. The code will be made publicly available upon publication.

Title: From Generative Modeling to Clinical Classification: A GPT-Based Architecture for EHR Notes

• Authors: Fariba Afrin Irany
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.21955
• Pdf URL: https://arxiv.org/pdf/2601.21955
• Copy Paste: [[2601.21955]] From Generative Modeling to Clinical Classification: A GPT-Based Architecture for EHR Notes(https://arxiv.org/abs/2601.21955)
• Keywords: transformer, generative
• Abstract: The increasing availability of unstructured clinical narratives in electronic health records (EHRs) has created new opportunities for automated disease characterization, cohort identification, and clinical decision support. However, modeling long, domain-specific clinical text remains challenging due to limited labeled data, severe class imbalance, and the high computational cost of adapting large pretrained language models. This study presents a GPT-based architecture for clinical text classification that adapts a pretrained decoder-only Transformer using a selective fine-tuning strategy. Rather than updating all model parameters, the majority of the GPT-2 backbone is frozen, and training is restricted to the final Transformer block, the final layer normalization, and a lightweight classification head. This approach substantially reduces the number of trainable parameters while preserving the representational capacity required to model complex clinical language. The proposed method is evaluated on radiology reports from the MIMIC-IV-Note dataset using uncertainty-aware CheXpert-style labels derived directly from report text. Experiments cover multiple problem formulations, including multi-label classification of radiographic findings, binary per-label classification under different uncertainty assumptions, and aggregate disease outcome prediction. Across varying dataset sizes, the model exhibits stable convergence behavior and strong classification performance, particularly in settings dominated by non-mention and negated findings. Overall, the results indicate that selective fine-tuning of pretrained generative language models provides an efficient and effective pathway for clinical text classification, enabling scalable adaptation to real-world EHR data while significantly reducing computational complexity.

Title: Uncertainty-Aware Data-Based Method for Fast and Reliable Shape Optimization

• Authors: Yunjia Yang, Runze Li, Yufei Zhang, Haixin Chen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21956
• Pdf URL: https://arxiv.org/pdf/2601.21956
• Copy Paste: [[2601.21956]] Uncertainty-Aware Data-Based Method for Fast and Reliable Shape Optimization(https://arxiv.org/abs/2601.21956)
• Keywords: robust
• Abstract: Data-based optimization (DBO) offers a promising approach for efficiently optimizing shape for better aerodynamic performance by leveraging a pretrained surrogate model for offline evaluations during iterations. However, DBO heavily relies on the quality of the training database. Samples outside the training distribution encountered during optimization can lead to significant prediction errors, potentially misleading the optimization process. Therefore, incorporating uncertainty quantification into optimization is critical for detecting outliers and enhancing robustness. This study proposes an uncertainty-aware data-based optimization (UA-DBO) framework to monitor and minimize surrogate model uncertainty during DBO. A probabilistic encoder-decoder surrogate model is developed to predict uncertainties associated with its outputs, and these uncertainties are integrated into a model-confidence-aware objective function to penalize samples with large prediction errors during data-based optimization process. The UA-DBO framework is evaluated on two multipoint optimization problems aimed at improving airfoil drag divergence and buffet performance. Results demonstrate that UA-DBO consistently reduces prediction errors in optimized samples and achieves superior performance gains compared to original DBO. Moreover, compared to multipoint optimization based on full computational simulations, UA-DBO offers comparable optimization effectiveness while significantly accelerating optimization speed.

Title: PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsing

• Authors: Cheng Cui, Ting Sun, Suyin Liang, Tingquan Gao, Zelun Zhang, Jiaxuan Liu, Xueqing Wang, Changda Zhou, Hongen Liu, Manhui Lin, Yue Zhang, Yubo Zhang, Yi Liu, Dianhai Yu, Yanjun Ma
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.21957
• Pdf URL: https://arxiv.org/pdf/2601.21957
• Copy Paste: [[2601.21957]] PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsing(https://arxiv.org/abs/2601.21957)
• Keywords: robust
• Abstract: We introduce PaddleOCR-VL-1.5, an upgraded model achieving a new state-of-the-art (SOTA) accuracy of 94.5% on OmniDocBench v1.5. To rigorously evaluate robustness against real-world physical distortions, including scanning, skew, warping, screen-photography, and illumination, we propose the Real5-OmniDocBench benchmark. Experimental results demonstrate that this enhanced model attains SOTA performance on the newly curated benchmark. Furthermore, we extend the model's capabilities by incorporating seal recognition and text spotting tasks, while remaining a 0.9B ultra-compact VLM with high efficiency. Code: this https URL

Title: From Tokens to Blocks: A Block-Diffusion Perspective on Molecular Generation

• Authors: Qianwei Yang, Dong Xu, Zhangfan Yang, Sisi Yuan, Zexuan Zhu, Jianqiang Li, Junkai Ji
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21964
• Pdf URL: https://arxiv.org/pdf/2601.21964
• Copy Paste: [[2601.21964]] From Tokens to Blocks: A Block-Diffusion Perspective on Molecular Generation(https://arxiv.org/abs/2601.21964)
• Keywords: diffusion, generative
• Abstract: Drug discovery can be viewed as a combinatorial search over an immense chemical space, motivating the development of deep generative models for de novo molecular design. Among these, GPT-based molecular language models (MLM) have shown strong molecular design performance by learning chemical syntax and semantics from large-scale data. However, existing MLMs face two fundamental limitations: they inadequately capture the graph-structured nature of molecules when formulated as next-token prediction problems, and they typically lack explicit mechanisms for target-aware generation. Here, we propose SoftMol, a unified framework that co-designs molecular representation, model architecture, and search strategy for target-aware molecular generation. SoftMol introduces soft fragments, a rule-free block representation of SMILES that enables diffusion-native modeling, and develops SoftBD, the first block-diffusion molecular language model that combines local bidirectional diffusion with autoregressive generation under molecular structural constraints. To favor generated molecules with high drug-likeness and synthetic accessibility, SoftBD is trained on a carefully curated dataset named ZINC-Curated. SoftMol further integrates a gated Monte Carlo tree search to assemble fragments in a target-aware manner. Experimental results show that, compared with current state-of-the-art models, SoftMol achieves 100% chemical validity, improves binding affinity by 9.7%, yields a 2-3x increase in molecular diversity, and delivers a 6.6x speedup in inference efficiency. Code is available at this https URL

Title: Secure Group Key Agreement on Cyber-Physical System Buses

• Authors: Sebastian N. Peters, Lukas Lautenschlager, David Emeis, Jason Lochert
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2601.21966
• Pdf URL: https://arxiv.org/pdf/2601.21966
• Copy Paste: [[2601.21966]] Secure Group Key Agreement on Cyber-Physical System Buses(https://arxiv.org/abs/2601.21966)
• Keywords: secure
• Abstract: Cyber-Physical Systems (CPSs) rely on distributed embedded devices that often must communicate securely over buses. Ensuring message integrity and authenticity on these buses typically requires group-shared keys for Message Authentication Codes (MACs). To avoid insecure fixed pre-shared keys and trust-on-first-use concepts, a Group Key Agreement (GKA) protocol is needed to dynamically agree on a key amongst the devices. Yet existing GKA protocols lack adaptability to constrained CPS buses. This paper targets authenticated, fully distributed GKA suitable for bus topologies under constraints of industrial and cyber-physical systems, including broadcast-only links, half-duplex operation, resource limits, dynamic membership (including unannounced leaves), a long device lifetime, and a strong Dolev-Yao adversary capable of partitioning the bus. We first systematise existing protocols, then derive the requirements necessary for an authenticated and fully distributed GKA on bus systems. Finally, we design, implement, and evaluate a custom GKA protocol based on TreeKEM.

Title: Token-Guard: Towards Token-Level Hallucination Control via Self-Checking Decoding

• Authors: Yifan Zhu, Huiqiang Rong, Haoran Luo
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.21969
• Pdf URL: https://arxiv.org/pdf/2601.21969
• Copy Paste: [[2601.21969]] Token-Guard: Towards Token-Level Hallucination Control via Self-Checking Decoding(https://arxiv.org/abs/2601.21969)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) often hallucinate, generating content inconsistent with the input. Retrieval-Augmented Generation (RAG) and Reinforcement Learning with Human Feedback (RLHF) can mitigate hallucinations but require resource-intensive retrieval or large-scale fine-tuning. Decoding-based methods are lighter yet lack explicit hallucination control. To address this, we present Token-Guard, a token-level hallucination control method based on self-checking decoding. Token-Guard performs internal verification at each reasoning step to detect hallucinated tokens before they propagate. Candidate fragments are further evaluated in a latent space with explicit hallucination risk scoring, while iterative pruning and regeneration dynamically correct detected errors. Experiments on HALU datasets show Token-Guard substantially reduces hallucinations and improves generation accuracy, offering a scalable, modular solution for reliable LLM outputs. Our code is publicly available.

Title: PowerGenie: Analytically-Guided Evolutionary Discovery of Superior Reconfigurable Power Converters

• Authors: Jian Gao, Yiwei Zou, Abhishek Pradhan, Wenhao Huang, Yumin Su, Kaiyuan Yang, Xuan Zhang
• Subjects: cs.LG, cs.AR
• Abstract URL: https://arxiv.org/abs/2601.21984
• Pdf URL: https://arxiv.org/pdf/2601.21984
• Copy Paste: [[2601.21984]] PowerGenie: Analytically-Guided Evolutionary Discovery of Superior Reconfigurable Power Converters(https://arxiv.org/abs/2601.21984)
• Keywords: generative
• Abstract: Discovering superior circuit topologies requires navigating an exponentially large design space-a challenge traditionally reserved for human experts. Existing AI methods either select from predefined templates or generate novel topologies at a limited scale without rigorous verification, leaving large-scale performance-driven discovery underexplored. We present PowerGenie, a framework for automated discovery of higher-performance reconfigurable power converters at scale. PowerGenie introduces: (1) an automated analytical framework that determines converter functionality and theoretical performance limits without component sizing or SPICE simulation, and (2) an evolutionary finetuning method that co-evolves a generative model with its training distribution through fitness selection and uniqueness verification. Unlike existing methods that suffer from mode collapse and overfitting, our approach achieves higher syntax validity, function validity, novelty rate, and figure-of-merit (FoM). PowerGenie discovers a novel 8-mode reconfigurable converter with 23% higher FoM than the best training topology. SPICE simulations confirm average absolute efficiency gains of 10% across 8 modes and up to 17% at a single mode. Code is available at this https URL.

Title: Elign: Equivariant Diffusion Model Alignment from Foundational Machine Learning Force Fields

• Authors: Yunyang Li, Lin Huang, Luojia Xia, Wenhe Zhang, Mark Gerstein
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21985
• Pdf URL: https://arxiv.org/pdf/2601.21985
• Copy Paste: [[2601.21985]] Elign: Equivariant Diffusion Model Alignment from Foundational Machine Learning Force Fields(https://arxiv.org/abs/2601.21985)
• Keywords: diffusion, generative
• Abstract: Generative models for 3D molecular conformations must respect Euclidean symmetries and concentrate probability mass on thermodynamically favorable, mechanically stable structures. However, E(3)-equivariant diffusion models often reproduce biases from semi-empirical training data rather than capturing the equilibrium distribution of a high-fidelity Hamiltonian. While physics-based guidance can correct this, it faces two computational bottlenecks: expensive quantum-chemical evaluations (e.g., DFT) and the need to repeat such queries at every sampling step. We present Elign, a post-training framework that amortizes both costs. First, we replace expensive DFT evaluations with a faster, pretrained foundational machine-learning force field (MLFF) to provide physical signals. Second, we eliminate repeated run-time queries by shifting physical steering to the training phase. To achieve the second amortization, we formulate reverse diffusion as a reinforcement learning problem and introduce Force--Energy Disentangled Group Relative Policy Optimization (FED-GRPO) to fine-tune the denoising policy. FED-GRPO includes a potential-based energy reward and a force-based stability reward, which are optimized and group-normalized independently. Experiments show that Elign generates conformations with lower gold-standard DFT energies and forces, while improving stability. Crucially, inference remains as fast as unguided sampling, since no energy evaluations are required during generation.

Title: Mechanistic Data Attribution: Tracing the Training Origins of Interpretable LLM Units

• Authors: Jianhui Chen, Yuzhang Luo, Liangming Pan
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.21996
• Pdf URL: https://arxiv.org/pdf/2601.21996
• Copy Paste: [[2601.21996]] Mechanistic Data Attribution: Tracing the Training Origins of Interpretable LLM Units(https://arxiv.org/abs/2601.21996)
• Keywords: interpretability
• Abstract: While Mechanistic Interpretability has identified interpretable circuits in LLMs, their causal origins in training data remain elusive. We introduce Mechanistic Data Attribution (MDA), a scalable framework that employs Influence Functions to trace interpretable units back to specific training samples. Through extensive experiments on the Pythia family, we causally validate that targeted intervention--removing or augmenting a small fraction of high-influence samples--significantly modulates the emergence of interpretable heads, whereas random interventions show no effect. Our analysis reveals that repetitive structural data (e.g., LaTeX, XML) acts as a mechanistic catalyst. Furthermore, we observe that interventions targeting induction head formation induce a concurrent change in the model's in-context learning (ICL) capability. This provides direct causal evidence for the long-standing hypothesis regarding the functional link between induction heads and ICL. Finally, we propose a mechanistic data augmentation pipeline that consistently accelerates circuit convergence across model scales, providing a principled methodology for steering the developmental trajectories of LLMs.

Title: Causal World Modeling for Robot Control

• Authors: Lin Li, Qihang Zhang, Yiming Luo, Shuai Yang, Ruilin Wang, Fei Han, Mingrui Yu, Zelin Gao, Nan Xue, Xing Zhu, Yujun Shen, Yinghao Xu
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2601.21998
• Pdf URL: https://arxiv.org/pdf/2601.21998
• Copy Paste: [[2601.21998]] Causal World Modeling for Robot Control(https://arxiv.org/abs/2601.21998)
• Keywords: diffusion, transformer
• Abstract: This work highlights that video world modeling, alongside vision-language pre-training, establishes a fresh and independent foundation for robot learning. Intuitively, video world models provide the ability to imagine the near future by understanding the causality between actions and visual dynamics. Inspired by this, we introduce LingBot-VA, an autoregressive diffusion framework that learns frame prediction and policy execution simultaneously. Our model features three carefully crafted designs: (1) a shared latent space, integrating vision and action tokens, driven by a Mixture-of-Transformers (MoT) architecture, (2) a closed-loop rollout mechanism, allowing for ongoing acquisition of environmental feedback with ground-truth observations, (3) an asynchronous inference pipeline, parallelizing action prediction and motor execution to support efficient control. We evaluate our model on both simulation benchmarks and real-world scenarios, where it shows significant promise in long-horizon manipulation, data efficiency in post-training, and strong generalizability to novel configurations. The code and model are made publicly available to facilitate the community.

Title: Rate-Distortion Optimization for Transformer Inference

• Authors: Anderson de Andrade, Alon Harell, Ivan V. Bajić
• Subjects: cs.LG, cs.IT
• Abstract URL: https://arxiv.org/abs/2601.22002
• Pdf URL: https://arxiv.org/pdf/2601.22002
• Copy Paste: [[2601.22002]] Rate-Distortion Optimization for Transformer Inference(https://arxiv.org/abs/2601.22002)
• Keywords: explainability, transformer
• Abstract: Transformers achieve superior performance on many tasks, but impose heavy compute and memory requirements during inference. This inference can be made more efficient by partitioning the process across multiple devices, which, in turn, requires compressing its intermediate representations. In this work, we introduce a principled rate-distortion-based framework for lossy compression that learns compact encodings that explicitly trade off bitrate against accuracy. Experiments on language benchmarks show that the proposed codec achieves substantial savings with improved accuracy in some cases, outperforming more complex baseline methods. We characterize and analyze the rate-distortion performance of transformers, offering a unified lens for understanding performance in representation coding. This formulation extends information-theoretic concepts to define the gap between rate and entropy, and derive some of its bounds. We further develop probably approximately correct (PAC)-style bounds for estimating this gap. For different architectures and tasks, we empirically demonstrate that their rates are driven by these bounds, adding to the explainability of the formulation.

Title: Putting a Face to Forgetting: Continual Learning meets Mechanistic Interpretability

• Authors: Sergi Masip, Gido M. van de Ven, Javier Ferrando, Tinne Tuytelaars
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22012
• Pdf URL: https://arxiv.org/pdf/2601.22012
• Copy Paste: [[2601.22012]] Putting a Face to Forgetting: Continual Learning meets Mechanistic Interpretability(https://arxiv.org/abs/2601.22012)
• Keywords: interpretability, transformer
• Abstract: Catastrophic forgetting in continual learning is often measured at the performance or last-layer representation level, overlooking the underlying mechanisms. We introduce a mechanistic framework that offers a geometric interpretation of catastrophic forgetting as the result of transformations to the encoding of individual features. These transformations can lead to forgetting by reducing the allocated capacity of features (worse representation) and disrupting their readout by downstream computations. Analysis of a tractable model formalizes this view, allowing us to identify best- and worst-case scenarios. Through experiments on this model, we empirically test our formal analysis and highlight the detrimental effect of depth. Finally, we demonstrate how our framework can be used in the analysis of practical models through the use of Crosscoders. We present a case study of a Vision Transformer trained on sequential CIFAR-10. Our work provides a new, feature-centric vocabulary for continual learning.

Title: TBDFiltering: Sample-Efficient Tree-Based Data Filtering

• Authors: Robert Istvan Busa-Fekete, Julian Zimmert, Anne Xiangyi Zheng, Claudio Gentile, Andras Gyorgy
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22016
• Pdf URL: https://arxiv.org/pdf/2601.22016
• Copy Paste: [[2601.22016]] TBDFiltering: Sample-Efficient Tree-Based Data Filtering(https://arxiv.org/abs/2601.22016)
• Keywords: large language model
• Abstract: The quality of machine learning models depends heavily on their training data. Selecting high-quality, diverse training sets for large language models (LLMs) is a difficult task, due to the lack of cheap and reliable quality metrics. While querying existing LLMs for document quality is common, this is not scalable to the large number (billions) of documents used in training. Instead, practitioners often use classifiers trained on sparse quality signals. In this paper, we propose a text-embedding-based hierarchical clustering approach that adaptively selects the documents to be evaluated by the LLM to estimate cluster quality. We prove that our method is query efficient: under the assumption that the hierarchical clustering contains a subtree such that each leaf cluster in the tree is pure enough (i.e., it mostly contains either only good or only bad documents), with high probability, the method can correctly predict the quality of each document after querying a small number of documents. The number of such documents is proportional to the size of the smallest subtree with (almost) pure leaves, without the algorithm knowing this subtree in advance. Furthermore, in a comprehensive experimental study, we demonstrate the benefits of our algorithm compared to other classifier-based filtering methods.

Title: Visual-Guided Key-Token Regularization for Multimodal Large Language Model Unlearning

• Authors: Chengyi Cai, Zesheng Ye, Peike Li, Bo Han, Jianzhong Qi, Feng Liu
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22020
• Pdf URL: https://arxiv.org/pdf/2601.22020
• Copy Paste: [[2601.22020]] Visual-Guided Key-Token Regularization for Multimodal Large Language Model Unlearning(https://arxiv.org/abs/2601.22020)
• Keywords: large language model
• Abstract: Unlearning in Multimodal Large Language Models (MLLMs) prevents the model from revealing private information when queried about target images. Existing MLLM unlearning methods largely adopt approaches developed for LLMs. They treat all answer tokens uniformly, disregarding their varying importance in the unlearning process. Moreover, these methods focus exclusively on the language modality, disregarding visual cues that indicate key tokens in answers. In this paper, after formulating the problem of unlearning in multimodal question answering for MLLMs, we propose Visual-Guided Key-Token Regularization (ViKeR). We leverage irrelevant visual inputs to predict ideal post-unlearning token-level distributions and use these distributions to regularize the unlearning process, thereby prioritizing key tokens. Further, we define key tokens in unlearning via information entropy and discuss ViKeR's effectiveness through token-level gradient reweighting, which amplifies updates on key tokens. Experiments on MLLMU and CLEAR benchmarks demonstrate that our method effectively performs unlearning while mitigating forgetting and maintaining response coherence.

Title: When "Better" Prompts Hurt: Evaluation-Driven Iteration for LLM Applications

• Authors: Daniel Commey
• Subjects: cs.CL, cs.AI, cs.IR, cs.SE
• Abstract URL: https://arxiv.org/abs/2601.22025
• Pdf URL: https://arxiv.org/pdf/2601.22025
• Copy Paste: [[2601.22025]] When "Better" Prompts Hurt: Evaluation-Driven Iteration for LLM Applications(https://arxiv.org/abs/2601.22025)
• Keywords: extraction, large language model
• Abstract: Evaluating Large Language Model (LLM) applications differs from traditional software testing because outputs are stochastic, high-dimensional, and sensitive to prompt and model changes. We present an evaluation-driven workflow - Define, Test, Diagnose, Fix - that turns these challenges into a repeatable engineering loop. We introduce the Minimum Viable Evaluation Suite (MVES), a tiered set of recommended evaluation components for (i) general LLM applications, (ii) retrieval-augmented generation (RAG), and (iii) agentic tool-use workflows. We also synthesize common evaluation methods (automated checks, human rubrics, and LLM-as-judge) and discuss known judge failure modes. In reproducible local experiments (Ollama; Llama 3 8B Instruct and Qwen 2.5 7B Instruct), we observe that a generic "improved" prompt template can trade off behaviors: on our small structured suites, extraction pass rate decreased from 100% to 90% and RAG compliance from 93.3% to 80% for Llama 3 when replacing task-specific prompts with generic rules, while instruction-following improved. These findings motivate evaluation-driven prompt iteration and careful claim calibration rather than universal prompt recipes. All test suites, harnesses, and results are included for reproducibility.

Title: From Logits to Latents: Contrastive Representation Shaping for LLM Unlearning

• Authors: Haoran Tang, Rajiv Khanna
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22028
• Pdf URL: https://arxiv.org/pdf/2601.22028
• Copy Paste: [[2601.22028]] From Logits to Latents: Contrastive Representation Shaping for LLM Unlearning(https://arxiv.org/abs/2601.22028)
• Keywords: privacy
• Abstract: Most LLM unlearning methods aim to approximate retrain-from-scratch behaviors with minimal distribution shift, often via alignment-style objectives defined in the prediction space. While effective at reducing forgotten content generation, such approaches may act as suppression: forgotten concepts can persist in representations and remain entangled with retained knowledge. We introduce CLReg, a contrastive representation regularizer that identifies forget features while pushing them away from retain features, explicitly reducing forget-retain interference with minimal shifts on retain features. We provide first theoretical insights that relate representation shaping to entanglement reduction. Across unlearning benchmarks and LLMs of different sizes, CLReg decreases forget-retain representation entanglement that facilitates mainstream unlearning methods without positing extra privacy risks, inspiring future work that reshapes the representation space to remove forget concepts.

Title: The Ensemble Inverse Problem: Applications and Methods

• Authors: Zhengyan Huan, Camila Pazos, Martin Klassen, Vincent Croft, Pierre-Hugues Beauchemin, Shuchin Aeron
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22029
• Pdf URL: https://arxiv.org/pdf/2601.22029
• Copy Paste: [[2601.22029]] The Ensemble Inverse Problem: Applications and Methods(https://arxiv.org/abs/2601.22029)
• Keywords: generative
• Abstract: We introduce a new multivariate statistical problem that we refer to as the Ensemble Inverse Problem (EIP). The aim of EIP is to invert for an ensemble that is distributed according to the pushforward of a prior under a forward process. In high energy physics (HEP), this is related to a widely known problem called unfolding, which aims to reconstruct the true physics distribution of quantities, such as momentum and angle, from measurements that are distorted by detector effects. In recent applications, the EIP also arises in full waveform inversion (FWI) and inverse imaging with unknown priors. We propose non-iterative inference-time methods that construct posterior samplers based on a new class of conditional generative models, which we call ensemble inverse generative models. For the posterior modeling, these models additionally use the ensemble information contained in the observation set on top of single measurements. Unlike existing methods, our proposed methods avoid explicit and iterative use of the forward model at inference time via training across several sets of truth-observation pairs that are consistent with the same forward model, but originate from a wide range of priors. We demonstrate that this training procedure implicitly encodes the likelihood model. The use of ensemble information helps posterior inference and enables generalization to unseen priors. We benchmark the proposed method on several synthetic and real datasets in inverse imaging, HEP, and FWI. The codes are available at this https URL.

Title: Per-parameter Task Arithmetic for Unlearning in Large Language Models

• Authors: Chengyi Cai, Zesheng Ye, Jiangchao Yao, Jianzhong Qi, Bo Han, Xiaolu Zhang, Feng Liu, Jun Zhou
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22030
• Pdf URL: https://arxiv.org/pdf/2601.22030
• Copy Paste: [[2601.22030]] Per-parameter Task Arithmetic for Unlearning in Large Language Models(https://arxiv.org/abs/2601.22030)
• Keywords: privacy, large language model
• Abstract: In large language model (LLM) unlearning, private information is required to be removed. Task arithmetic unlearns by subtracting a specific task vector (TV)--defined as the parameter difference between a privacy-information-tuned model and the original model. While efficient, it can cause over-forgetting by disrupting parameters essential for retaining other information. Motivated by the observation that each parameter exhibits different importance for forgetting versus retention, we propose a per-parameter task arithmetic (PerTA) mechanism to rescale the TV, allowing per-parameter adjustment. These weights quantify the relative importance of each parameter for forgetting versus retention, estimated via gradients (i.e., PerTA-grad) or the diagonal Fisher information approximation (i.e., PerTA-fisher). Moreover, we discuss the effectiveness of PerTA, extend it to a more general form, and provide further analysis. Extensive experiments demonstrate that PerTA consistently improves upon standard TV, and in many cases surpasses widely used training-based unlearning methods in both forgetting effectiveness and overall model utility. By retaining the efficiency of task arithmetic while mitigating over-forgetting, PerTA offers a principled and practical framework for LLM unlearning.

Title: Causal Autoregressive Diffusion Language Model

• Authors: Junhao Ruan, Bei Li, Yongjing Yin, Pengcheng Huang, Xin Chen, Jingang Wang, Xunliang Cai, Tong Xiao, JingBo Zhu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22031
• Pdf URL: https://arxiv.org/pdf/2601.22031
• Copy Paste: [[2601.22031]] Causal Autoregressive Diffusion Language Model(https://arxiv.org/abs/2601.22031)
• Keywords: robust, diffusion
• Abstract: In this work, we propose Causal Autoregressive Diffusion (CARD), a novel framework that unifies the training efficiency of ARMs with the high-throughput inference of diffusion models. CARD reformulates the diffusion process within a strictly causal attention mask, enabling dense, per-token supervision in a single forward pass. To address the optimization instability of causal diffusion, we introduce a soft-tailed masking schema to preserve local context and a context-aware reweighting mechanism derived from signal-to-noise principles. This design enables dynamic parallel decoding, where the model leverages KV-caching to adaptively generate variable-length token sequences based on confidence. Empirically, CARD outperforms existing discrete diffusion baselines while reducing training latency by 3 $\times$ compared to block diffusion methods. Our results demonstrate that CARD achieves ARM-level data efficiency while unlocking the latency benefits of parallel generation, establishing a robust paradigm for next-generation efficient LLMs.

Title: Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Driving

• Authors: Linhan Wang, Zichong Yang, Chen Bai, Guoxiang Zhang, Xiaotong Liu, Xiaoyin Zheng, Xiao-Xiao Long, Chang-Tien Lu, Cheng Lu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22032
• Pdf URL: https://arxiv.org/pdf/2601.22032
• Copy Paste: [[2601.22032]] Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Driving(https://arxiv.org/abs/2601.22032)
• Keywords: transformer
• Abstract: End-to-end autonomous driving increasingly leverages self-supervised video pretraining to learn transferable planning representations. However, pretraining video world models for scene understanding has so far brought only limited improvements. This limitation is compounded by the inherent ambiguity of driving: each scene typically provides only a single human trajectory, making it difficult to learn multimodal behaviors. In this work, we propose Drive-JEPA, a framework that integrates Video Joint-Embedding Predictive Architecture (V-JEPA) with multimodal trajectory distillation for end-to-end driving. First, we adapt V-JEPA for end-to-end driving, pretraining a ViT encoder on large-scale driving videos to produce predictive representations aligned with trajectory planning. Second, we introduce a proposal-centric planner that distills diverse simulator-generated trajectories alongside human trajectories, with a momentum-aware selection mechanism to promote stable and safe behavior. When evaluated on NAVSIM, the V-JEPA representation combined with a simple transformer-based decoder outperforms prior methods by 3 PDMS in the perception-free setting. The complete Drive-JEPA framework achieves 93.3 PDMS on v1 and 87.8 EPDMS on v2, setting a new state-of-the-art.

Title: Holographic generative flows with AdS/CFT

• Authors: Ehsan Mirafzali, Sanjit Shashi, Sanya Murdeshwar, Edgar Shaghoulian, Daniele Venturi, Razvan Marinescu
• Subjects: cs.LG, gr-qc, hep-th
• Abstract URL: https://arxiv.org/abs/2601.22033
• Pdf URL: https://arxiv.org/pdf/2601.22033
• Copy Paste: [[2601.22033]] Holographic generative flows with AdS/CFT(https://arxiv.org/abs/2601.22033)
• Keywords: generative
• Abstract: We present a framework for generative machine learning that leverages the holographic principle of quantum gravity, or to be more precise its manifestation as the anti-de Sitter/conformal field theory (AdS/CFT) correspondence, with techniques for deep learning and transport theory. Our proposal is to represent the flow of data from a base distribution to some learned distribution using the bulk-to-boundary mapping of scalar fields in AdS. In the language of machine learning, we are representing and augmenting the flow-matching algorithm with AdS physics. Using a checkerboard toy dataset and MNIST, we find that our model achieves faster and higher quality convergence than comparable physics-free flow-matching models. Our method provides a physically interpretable version of flow matching. More broadly, it establishes the utility of AdS physics and geometry in the development of novel paradigms in generative modeling.

Title: Thinking Out of Order: When Output Order Stops Reflecting Reasoning Order in Diffusion Language Models

• Authors: Longxuan Yu, Yu Fu, Shaorong Zhang, Hui Liu, Mukund Varma T, Greg Ver Steeg, Yue Dong
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22035
• Pdf URL: https://arxiv.org/pdf/2601.22035
• Copy Paste: [[2601.22035]] Thinking Out of Order: When Output Order Stops Reflecting Reasoning Order in Diffusion Language Models(https://arxiv.org/abs/2601.22035)
• Keywords: robust, diffusion
• Abstract: Autoregressive (AR) language models enforce a fixed left-to-right generation order, creating a fundamental limitation when the required output structure conflicts with natural reasoning (e.g., producing answers before explanations due to presentation or schema constraints). In such cases, AR models must commit to answers before generating intermediate reasoning, and this rigid constraint forces premature commitment. Masked diffusion language models (MDLMs), which iteratively refine all tokens in parallel, offer a way to decouple computation order from output structure. We validate this capability on GSM8K, Math500, and ReasonOrderQA, a benchmark we introduce with controlled difficulty and order-level evaluation. When prompts request answers before reasoning, AR models exhibit large accuracy gaps compared to standard chain-of-thought ordering (up to 67% relative drop), while MDLMs remain stable ($\leq$14% relative drop), a property we term "order robustness". Using ReasonOrderQA, we present evidence that MDLMs achieve order robustness by stabilizing simpler tokens (e.g., reasoning steps) earlier in the diffusion process than complex ones (e.g., final answers), enabling reasoning tokens to stabilize before answer commitment. Finally, we identify failure conditions where this advantage weakens, outlining the limits required for order robustness.

Title: Cross-Fusion Distance: A Novel Metric for Measuring Fusion and Separability Between Data Groups in Representation Space

• Authors: Xiaolong Zhang, Jianwei Zhang, Xubo Song
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22036
• Pdf URL: https://arxiv.org/pdf/2601.22036
• Copy Paste: [[2601.22036]] Cross-Fusion Distance: A Novel Metric for Measuring Fusion and Separability Between Data Groups in Representation Space(https://arxiv.org/abs/2601.22036)
• Keywords: robust
• Abstract: Quantifying degrees of fusion and separability between data groups in representation space is a fundamental problem in representation learning, particularly under domain shift. A meaningful metric should capture fusion-altering factors like geometric displacement between representation groups, whose variations change the extent of fusion, while remaining invariant to fusion-preserving factors such as global scaling and sampling-induced layout changes, whose variations do not. Existing distributional distance metrics conflate these factors, leading to measures that are not informative of the true extent of fusion between data groups. We introduce Cross-Fusion Distance (CFD), a principled measure that isolates fusion-altering geometry while remaining robust to fusion-preserving variations, with linear computational complexity. We characterize the invariance and sensitivity properties of CFD theoretically and validate them in controlled synthetic experiments. For practical utility on real-world datasets with domain shift, CFD aligns more closely with downstream generalization degradation than commonly used alternatives. Overall, CFD provides a theoretically grounded and interpretable distance measure for representation learning.

Title: A Separable Architecture for Continuous Token Representation in Language Models

• Authors: Reza T. Batley, Sourav Saha
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22040
• Pdf URL: https://arxiv.org/pdf/2601.22040
• Copy Paste: [[2601.22040]] A Separable Architecture for Continuous Token Representation in Language Models(https://arxiv.org/abs/2601.22040)
• Keywords: transformer
• Abstract: Transformer scaling law analyses typically treat parameters as interchangeable; an abstraction that accurately predicts loss-compute relationships. Yet, in sub-billion-parameter small language models (SLMs), embedding matrices dominate the parameter budget. This work argues that this allocation is as suboptimal as it is counterintuitive. Leviathan is an architecture with a continuous embedding generator to replace the discrete lookup tables of canonical models. Evaluating on the Pile dataset under isoparametric settings, Leviathan consistently outperforms a standard, LLaMA-style architecture. By means of an empirical power-law fit, Leviathan exhibits a markedly superior effective parameter capacity. Across the regime studied, Leviathan behaves as a dense model with $1.47$ to $2.11 \times$ more parameters.

Title: Urban Neural Surface Reconstruction from Constrained Sparse Aerial Imagery with 3D SAR Fusion

• Authors: Da Li, Chen Yao, Tong Mao, Jiacheng Bao, Houjun Sun
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22045
• Pdf URL: https://arxiv.org/pdf/2601.22045
• Copy Paste: [[2601.22045]] Urban Neural Surface Reconstruction from Constrained Sparse Aerial Imagery with 3D SAR Fusion(https://arxiv.org/abs/2601.22045)
• Keywords: robust
• Abstract: Neural surface reconstruction (NSR) has recently shown strong potential for urban 3D reconstruction from multi-view aerial imagery. However, existing NSR methods often suffer from geometric ambiguity and instability, particularly under sparse-view conditions. This issue is critical in large-scale urban remote sensing, where aerial image acquisition is limited by flight paths, terrain, and cost. To address this challenge, we present the first urban NSR framework that fuses 3D synthetic aperture radar (SAR) point clouds with aerial imagery for high-fidelity reconstruction under constrained, sparse-view settings. 3D SAR can efficiently capture large-scale geometry even from a single side-looking flight path, providing robust priors that complement photometric cues from images. Our framework integrates radar-derived spatial constraints into an SDF-based NSR backbone, guiding structure-aware ray selection and adaptive sampling for stable and efficient optimization. We also construct the first benchmark dataset with co-registered 3D SAR point clouds and aerial imagery, facilitating systematic evaluation of cross-modal 3D reconstruction. Extensive experiments show that incorporating 3D SAR markedly enhances reconstruction accuracy, completeness, and robustness compared with single-modality baselines under highly sparse and oblique-view conditions, highlighting a viable route toward scalable high-fidelity urban reconstruction with advanced airborne and spaceborne optical-SAR sensing.

Title: On the Paradoxical Interference between Instruction-Following and Task Solving

• Authors: Yunjia Qi, Hao Peng, Xintong Shi, Amy Xin, Xiaozhi Wang, Bin Xu, Lei Hou, Juanzi Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22047
• Pdf URL: https://arxiv.org/pdf/2601.22047
• Copy Paste: [[2601.22047]] On the Paradoxical Interference between Instruction-Following and Task Solving(https://arxiv.org/abs/2601.22047)
• Keywords: large language model
• Abstract: Instruction following aims to align Large Language Models (LLMs) with human intent by specifying explicit constraints on how tasks should be performed. However, we reveal a counterintuitive phenomenon: instruction following can paradoxically interfere with LLMs' task-solving capability. We propose a metric, SUSTAINSCORE, to quantify the interference of instruction following with task solving. It measures task performance drop after inserting into the instruction a self-evident constraint, which is naturally met by the original successful model output and extracted from it. Experiments on current LLMs in mathematics, multi-hop QA, and code generation show that adding the self-evident constraints leads to substantial performance drops, even for advanced models such as Claude-Sonnet-4.5. We validate the generality of the interference across constraint types and scales. Furthermore, we identify common failure patterns, and by investigating the mechanisms of interference, we observe that failed cases allocate significantly more attention to constraints compared to successful ones. Finally, we use SUSTAINSCORE to conduct an initial investigation into how distinct post-training paradigms affect the interference, presenting empirical observations on current alignment strategies. We will release our code and data to facilitate further research

Title: MasalBench: A Benchmark for Contextual and Cross-Cultural Understanding of Persian Proverbs in LLMs

• Authors: Ghazal Kalhor, Behnam Bahrak
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22050
• Pdf URL: https://arxiv.org/pdf/2601.22050
• Copy Paste: [[2601.22050]] MasalBench: A Benchmark for Contextual and Cross-Cultural Understanding of Persian Proverbs in LLMs(https://arxiv.org/abs/2601.22050)
• Keywords: large language model
• Abstract: In recent years, multilingual Large Language Models (LLMs) have become an inseparable part of daily life, making it crucial for them to master the rules of conversational language in order to communicate effectively with users. While previous work has evaluated LLMs' understanding of figurative language in high-resource languages, their performance in low-resource languages remains underexplored. In this paper, we introduce MasalBench, a comprehensive benchmark for assessing LLMs' contextual and cross-cultural understanding of Persian proverbs, which are a key component of conversation in this low-resource language. We evaluate eight state-of-the-art LLMs on MasalBench and find that they perform well in identifying Persian proverbs in context, achieving accuracies above 0.90. However, their performance drops considerably when tasked with identifying equivalent English proverbs, with the best model achieving 0.79 accuracy. Our findings highlight the limitations of current LLMs in cultural knowledge and analogical reasoning, and they provide a framework for assessing cross-cultural understanding in other low-resource languages. MasalBench is available at this https URL.

Title: MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sources

• Authors: Baorui Ma, Jiahui Yang, Donglin Di, Xuancheng Zhang, Jianxun Cui, Hao Li, Yan Xie, Wei Chen
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22054
• Pdf URL: https://arxiv.org/pdf/2601.22054
• Copy Paste: [[2601.22054]] MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sources(https://arxiv.org/abs/2601.22054)
• Keywords: large language model
• Abstract: Scaling has powered recent advances in vision foundation models, yet extending this paradigm to metric depth estimation remains challenging due to heterogeneous sensor noise, camera-dependent biases, and metric ambiguity in noisy cross-source 3D data. We introduce Metric Anything, a simple and scalable pretraining framework that learns metric depth from noisy, diverse 3D sources without manually engineered prompts, camera-specific modeling, or task-specific architectures. Central to our approach is the Sparse Metric Prompt, created by randomly masking depth maps, which serves as a universal interface that decouples spatial reasoning from sensor and camera biases. Using about 20M image-depth pairs spanning reconstructed, captured, and rendered 3D data across 10000 camera models, we demonstrate-for the first time-a clear scaling trend in the metric depth track. The pretrained model excels at prompt-driven tasks such as depth completion, super-resolution and Radar-camera fusion, while its distilled prompt-free student achieves state-of-the-art results on monocular depth estimation, camera intrinsics recovery, single/multi-view metric 3D reconstruction, and VLA planning. We also show that using pretrained ViT of Metric Anything as a visual encoder significantly boosts Multimodal Large Language Model capabilities in spatial intelligence. These results show that metric depth estimation can benefit from the same scaling laws that drive modern foundation models, establishing a new path toward scalable and efficient real-world metric perception. We open-source MetricAnything at this http URL to support community research.

Title: Unsupervised Decomposition and Recombination with Discriminator-Driven Diffusion Models

• Authors: Archer Wang, Emile Anand, Yilun Du, Marin Soljačić
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22057
• Pdf URL: https://arxiv.org/pdf/2601.22057
• Copy Paste: [[2601.22057]] Unsupervised Decomposition and Recombination with Discriminator-Driven Diffusion Models(https://arxiv.org/abs/2601.22057)
• Keywords: diffusion
• Abstract: Decomposing complex data into factorized representations can reveal reusable components and enable synthesizing new samples via component recombination. We investigate this in the context of diffusion-based models that learn factorized latent spaces without factor-level supervision. In images, factors can capture background, illumination, and object attributes; in robotic videos, they can capture reusable motion components. To improve both latent factor discovery and quality of compositional generation, we introduce an adversarial training signal via a discriminator trained to distinguish between single-source samples and those generated by recombining factors across sources. By optimizing the generator to fool this discriminator, we encourage physical and semantic consistency in the resulting recombinations. Our method outperforms implementations of prior baselines on CelebA-HQ, Virtual KITTI, CLEVR, and Falcor3D, achieving lower FID scores and better disentanglement as measured by MIG and MCC. Furthermore, we demonstrate a novel application to robotic video trajectories: by recombining learned action components, we generate diverse sequences that significantly increase state-space coverage for exploration on the LIBERO benchmark.

Title: Vision-DeepResearch: Incentivizing DeepResearch Capability in Multimodal Large Language Models

• Authors: Wenxuan Huang, Yu Zeng, Qiuchen Wang, Zhen Fang, Shaosheng Cao, Zheng Chu, Qingyu Yin, Shuang Chen, Zhenfei Yin, Lin Chen, Zehui Chen, Yao Hu, Philip Torr, Feng Zhao, Wanli Ouyang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22060
• Pdf URL: https://arxiv.org/pdf/2601.22060
• Copy Paste: [[2601.22060]] Vision-DeepResearch: Incentivizing DeepResearch Capability in Multimodal Large Language Models(https://arxiv.org/abs/2601.22060)
• Keywords: robust, large language model
• Abstract: Multimodal large language models (MLLMs) have achieved remarkable success across a broad range of vision tasks. However, constrained by the capacity of their internal world knowledge, prior work has proposed augmenting MLLMs by ``reasoning-then-tool-call'' for visual and textual search engines to obtain substantial gains on tasks requiring extensive factual information. However, these approaches typically define multimodal search in a naive setting, assuming that a single full-level or entity-level image query and few text query suffices to retrieve the key evidence needed to answer the question, which is unrealistic in real-world scenarios with substantial visual noise. Moreover, they are often limited in the reasoning depth and search breadth, making it difficult to solve complex questions that require aggregating evidence from diverse visual and textual sources. Building on this, we propose Vision-DeepResearch, which proposes one new multimodal deep-research paradigm, i.e., performs multi-turn, multi-entity and multi-scale visual and textual search to robustly hit real-world search engines under heavy noise. Our Vision-DeepResearch supports dozens of reasoning steps and hundreds of engine interactions, while internalizing deep-research capabilities into the MLLM via cold-start supervision and RL training, resulting in a strong end-to-end multimodal deep-research MLLM. It substantially outperforming existing multimodal deep-research MLLMs, and workflows built on strong closed-source foundation model such as GPT-5, Gemini-2.5-pro and Claude-4-Sonnet. The code will be released in this https URL.

Title: BLO-Inst: Bi-Level Optimization Based Alignment of YOLO and SAM for Robust Instance Segmentation

• Authors: Li Zhang, Pengtao Xie
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22061
• Pdf URL: https://arxiv.org/pdf/2601.22061
• Copy Paste: [[2601.22061]] BLO-Inst: Bi-Level Optimization Based Alignment of YOLO and SAM for Robust Instance Segmentation(https://arxiv.org/abs/2601.22061)
• Keywords: robust, segmentation
• Abstract: The Segment Anything Model has revolutionized image segmentation with its zero-shot capabilities, yet its reliance on manual prompts hinders fully automated deployment. While integrating object detectors as prompt generators offers a pathway to automation, existing pipelines suffer from two fundamental limitations: objective mismatch, where detectors optimized for geometric localization do not correspond to the optimal prompting context required by SAM, and alignment overfitting in standard joint training, where the detector simply memorizes specific prompt adjustments for training samples rather than learning a generalizable policy. To bridge this gap, we introduce BLO-Inst, a unified framework that aligns detection and segmentation objectives by bi-level optimization. We formulate the alignment as a nested optimization problem over disjoint data splits. In the lower level, the SAM is fine-tuned to maximize segmentation fidelity given the current detection proposals on a subset ($D_1$). In the upper level, the detector is updated to generate bounding boxes that explicitly minimize the validation loss of the fine-tuned SAM on a separate subset ($D_2$). This effectively transforms the detector into a segmentation-aware prompt generator, optimizing the bounding boxes not just for localization accuracy, but for downstream mask quality. Extensive experiments demonstrate that BLO-Inst achieves superior performance, outperforming standard baselines on tasks in general and biomedical domains.

Title: VTC-R1: Vision-Text Compression for Efficient Long-Context Reasoning

• Authors: Yibo Wang, Yongcheng Jing, Shunyu Liu, Hao Guan, Rong-cheng Tu, Chengyu Wang, Jun Huang, Dacheng Tao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22069
• Pdf URL: https://arxiv.org/pdf/2601.22069
• Copy Paste: [[2601.22069]] VTC-R1: Vision-Text Compression for Efficient Long-Context Reasoning(https://arxiv.org/abs/2601.22069)
• Keywords: large language model
• Abstract: Long-context reasoning has significantly empowered large language models (LLMs) to tackle complex tasks, yet it introduces severe efficiency bottlenecks due to the computational complexity. Existing efficient approaches often rely on complex additional training or external models for compression, which limits scalability and discards critical fine-grained information. In this paper, we propose VTC-R1, a new efficient reasoning paradigm that integrates vision-text compression into the reasoning process. Instead of processing lengthy textual traces, VTC-R1 renders intermediate reasoning segments into compact images, which are iteratively fed back into vision-language models as "optical memory." We construct a training dataset based on OpenR1-Math-220K achieving 3.4x token compression and fine-tune representative VLMs-Glyph and Qwen3-VL. Extensive experiments on benchmarks such as MATH500, AIME25, AMC23 and GPQA-D demonstrate that VTC-R1 consistently outperforms standard long-context reasoning. Furthermore, our approach significantly improves inference efficiency, achieving 2.7x speedup in end-to-end latency, highlighting its potential as a scalable solution for reasoning-intensive applications. Our code is available at this https URL.

Title: Where Do the Joules Go? Diagnosing Inference Energy Consumption

• Authors: Jae-Won Chung, Ruofan Wu, Jeff J. Ma, Mosharaf Chowdhury
• Subjects: cs.LG, cs.DC
• Abstract URL: https://arxiv.org/abs/2601.22076
• Pdf URL: https://arxiv.org/pdf/2601.22076
• Copy Paste: [[2601.22076]] Where Do the Joules Go? Diagnosing Inference Energy Consumption(https://arxiv.org/abs/2601.22076)
• Keywords: generative
• Abstract: Energy is now a critical ML computing resource. While measuring energy consumption and observing trends is a valuable first step, accurately understanding and diagnosing why those differences occur is crucial for optimization. To that end, we begin by presenting a large-scale measurement study of inference time and energy across the generative AI landscape with 46 models, 7 tasks, and 1,858 different configurations on NVIDIA H100 and B200 GPUs. Our empirical findings span order-of-magnitude variations: LLM task type can lead to 25$\times$ energy differences, video generation sometimes consumes more than 100$\times$ the energy of images, and GPU utilization differences can result in 3--5$\times$ energy differences. Based on our observations, we present a framework for reasoning about the underlying mechanisms that govern time and energy consumption. The essence is that time and energy are determined by latent metrics like memory and utilization, which are in turn affected by various factors across the algorithm, software, and hardware layers. Our framework also extends directly to throughput per watt, a critical metric for power-constrained datacenters.

Title: Latent Adversarial Regularization for Offline Preference Optimization

• Authors: Enyi Jiang, Yibo Jacky Zhang, Yinglun Xu, Andreas Haupt, Nancy Amato, Sanmi Koyejo
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22083
• Pdf URL: https://arxiv.org/pdf/2601.22083
• Copy Paste: [[2601.22083]] Latent Adversarial Regularization for Offline Preference Optimization(https://arxiv.org/abs/2601.22083)
• Keywords: robust
• Abstract: Learning from human feedback typically relies on preference optimization that constrains policy updates through token-level regularization. However, preference optimization for language models is particularly challenging because token-space similarity does not imply semantic or behavioral similarity. To address this challenge, we leverage latent-space regularization for language model preference optimization. We introduce GANPO, which achieves latent-space regularization by penalizing divergence between the internal representations of a policy model and a reference model. Given that latent representations are not associated with explicit probability densities, we adopt an adversarial approach inspired by GANs to minimize latent-space divergence. We integrate GANPO as a regularizer into existing offline preference optimization objectives. Experiments across multiple model architectures and tasks show consistent improvements from latent-space regularization. Further, by comparing GANPO-induced inferential biases with those from token-level regularization, we find that GANPO provides more robust structural feedback under distributional shift and noise while maintaining comparable downstream performance with minor computational overhead.

Title: RefAny3D: 3D Asset-Referenced Diffusion Models for Image Generation

• Authors: Hanzhuo Huang, Qingyang Bao, Zekai Gu, Zhongshuo Du, Cheng Lin, Yuan Liu, Sibei Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22094
• Pdf URL: https://arxiv.org/pdf/2601.22094
• Copy Paste: [[2601.22094]] RefAny3D: 3D Asset-Referenced Diffusion Models for Image Generation(https://arxiv.org/abs/2601.22094)
• Keywords: diffusion
• Abstract: In this paper, we propose a 3D asset-referenced diffusion model for image generation, exploring how to integrate 3D assets into image diffusion models. Existing reference-based image generation methods leverage large-scale pretrained diffusion models and demonstrate strong capability in generating diverse images conditioned on a single reference image. However, these methods are limited to single-image references and cannot leverage 3D assets, constraining their practical versatility. To address this gap, we present a cross-domain diffusion model with dual-branch perception that leverages multi-view RGB images and point maps of 3D assets to jointly model their colors and canonical-space coordinates, achieving precise consistency between generated images and the 3D references. Our spatially aligned dual-branch generation architecture and domain-decoupled generation mechanism ensure the simultaneous generation of two spatially aligned but content-disentangled outputs, RGB images and point maps, linking 2D image attributes with 3D asset attributes. Experiments show that our approach effectively uses 3D assets as references to produce images consistent with the given assets, opening new possibilities for combining diffusion models with 3D content creation.

Title: GeoNorm: Unify Pre-Norm and Post-Norm with Geodesic Optimization

• Authors: Chuanyang Zheng, Jiankai Sun, Yihang Gao, Chi Wang, Yuehao Wang, Jing Xiong, Liliang Ren, Bo Peng, Qingmei Wang, Xiaoran Shang, Mac Schwager, Anderson Schneider, Yuriy Nevmyvaka, Xiaodong Liu
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22095
• Pdf URL: https://arxiv.org/pdf/2601.22095
• Copy Paste: [[2601.22095]] GeoNorm: Unify Pre-Norm and Post-Norm with Geodesic Optimization(https://arxiv.org/abs/2601.22095)
• Keywords: transformer
• Abstract: The placement of normalization layers, specifically Pre-Norm and Post-Norm, remains an open question in Transformer architecture design. In this work, we rethink these approaches through the lens of manifold optimization, interpreting the outputs of the Feed-Forward Network (FFN) and attention layers as update directions in optimization. Building on this perspective, we introduce GeoNorm, a novel method that replaces standard normalization with geodesic updates on the manifold. Furthermore, analogous to learning rate schedules, we propose a layer-wise update decay for the FFN and attention components. Comprehensive experiments demonstrate that GeoNorm consistently outperforms existing normalization methods in Transformer models. Crucially, GeoNorm can be seamlessly integrated into standard Transformer architectures, achieving performance improvements with negligible additional computational cost.

Title: ECO: Quantized Training without Full-Precision Master Weights

• Authors: Mahdi Nikdan, Amir Zandieh, Dan Alistarh, Vahab Mirrokni
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22101
• Pdf URL: https://arxiv.org/pdf/2601.22101
• Copy Paste: [[2601.22101]] ECO: Quantized Training without Full-Precision Master Weights(https://arxiv.org/abs/2601.22101)
• Keywords: transformer, large language model
• Abstract: Quantization has significantly improved the compute and memory efficiency of Large Language Model (LLM) training. However, existing approaches still rely on accumulating their updates in high-precision: concretely, gradient updates must be applied to a high-precision weight buffer, known as $\textit{master weights}$. This buffer introduces substantial memory overhead, particularly for Sparse Mixture of Experts (SMoE) models, where model parameters and optimizer states dominate memory usage. To address this, we introduce the Error-Compensating Optimizer (ECO), which eliminates master weights by applying updates directly to quantized parameters. ECO quantizes weights after each step and carefully injects the resulting quantization error into the optimizer momentum, forming an error-feedback loop with no additional memory. We prove that, under standard assumptions and a decaying learning rate, ECO converges to a constant-radius neighborhood of the optimum, while naive master-weight removal can incur an error that is inversely proportional to the learning rate. We show empirical results for pretraining small Transformers (30-800M), a Gemma-3 1B model, and a 2.1B parameter Sparse MoE model with FP8 quantization, and fine-tuning DeepSeek-MoE-16B in INT4 precision. Throughout, ECO matches baselines with master weights up to near-lossless accuracy, significantly shifting the static memory vs validation loss Pareto frontier.

Title: Prior-Informed Flow Matching for Graph Reconstruction

• Authors: Harvey Chen, Nicolas Zilberstein, Santiago Segarra
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22107
• Pdf URL: https://arxiv.org/pdf/2601.22107
• Copy Paste: [[2601.22107]] Prior-Informed Flow Matching for Graph Reconstruction(https://arxiv.org/abs/2601.22107)
• Keywords: generative
• Abstract: We introduce Prior-Informed Flow Matching (PIFM), a conditional flow model for graph reconstruction. Reconstructing graphs from partial observations remains a key challenge; classical embedding methods often lack global consistency, while modern generative models struggle to incorporate structural priors. PIFM bridges this gap by integrating embedding-based priors with continuous-time flow matching. Grounded in a permutation equivariant version of the distortion-perception theory, our method first uses a prior, such as graphons or GraphSAGE/node2vec, to form an informed initial estimate of the adjacency matrix based on local information. It then applies rectified flow matching to refine this estimate, transporting it toward the true distribution of clean graphs and learning a global coupling. Experiments on different datasets demonstrate that PIFM consistently enhances classical embeddings, outperforming them and state-of-the-art generative baselines in reconstruction accuracy.

Title: Value-Based Pre-Training with Downstream Feedback

• Authors: Shuqi Ke, Giulia Fanti
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22108
• Pdf URL: https://arxiv.org/pdf/2601.22108
• Copy Paste: [[2601.22108]] Value-Based Pre-Training with Downstream Feedback(https://arxiv.org/abs/2601.22108)
• Keywords: segmentation
• Abstract: Can a small amount of verified goal information steer the expensive self-supervised pretraining of foundation models? Standard pretraining optimizes a fixed proxy objective (e.g., next-token prediction), which can misallocate compute away from downstream capabilities of interest. We introduce V-Pretraining: a value-based, modality-agnostic method for controlled continued pretraining in which a lightweight task designer reshapes the pretraining task to maximize the value of each gradient step. For example, consider self-supervised learning (SSL) with sample augmentation. The V-Pretraining task designer selects pretraining tasks (e.g., augmentations) for which the pretraining loss gradient is aligned with a gradient computed over a downstream task (e.g., image segmentation). This helps steer pretraining towards relevant downstream capabilities. Notably, the pretrained model is never updated on downstream task labels; they are used only to shape the pretraining task. Under matched learner update budgets, V-Pretraining of 0.5B--7B language models improves reasoning (GSM8K test Pass@1) by up to 18% relative over standard next-token prediction using only 12% of GSM8K training examples as feedback. In vision SSL, we improve the state-of-the-art results on ADE20K by up to 1.07 mIoU and reduce NYUv2 RMSE while improving ImageNet linear accuracy, and we provide pilot evidence of improved token efficiency in continued pretraining.

Title: SINA: A Circuit Schematic Image-to-Netlist Generator Using Artificial Intelligence

• Authors: Saoud Aldowaish, Yashwanth Karumanchi, Kai-Chen Chiang, Soroosh Noorzad, Morteza Fayazi
• Subjects: cs.CV, cs.AI, eess.SY
• Abstract URL: https://arxiv.org/abs/2601.22114
• Pdf URL: https://arxiv.org/pdf/2601.22114
• Copy Paste: [[2601.22114]] SINA: A Circuit Schematic Image-to-Netlist Generator Using Artificial Intelligence(https://arxiv.org/abs/2601.22114)
• Keywords: extraction
• Abstract: Current methods for converting circuit schematic images into machine-readable netlists struggle with component recognition and connectivity inference. In this paper, we present SINA, an open-source, fully automated circuit schematic image-to-netlist generator. SINA integrates deep learning for accurate component detection, Connected-Component Labeling (CCL) for precise connectivity extraction, and Optical Character Recognition (OCR) for component reference designator retrieval, while employing a Vision-Language Model (VLM) for reliable reference designator assignments. In our experiments, SINA achieves 96.47% overall netlist-generation accuracy, which is 2.72x higher than state-of-the-art approaches.

Title: A Federated and Parameter-Efficient Framework for Large Language Model Training in Medicine

• Authors: Anran Li, Yuanyuan Chen, Wenjun Long, Yu Yin, Yan Hu, Hyunjae Kim, Weipeng Zhou, Yujia Zhou, Hongyi Peng, Yang Ren, Xuguang Ai, Zhenyue Qin, Ming Hu, Xiaoxiao Li, Han Yu, Yih-Chung Tham, Lucila Ohno-Machado, Hua Xu, Qingyu Chen
• Subjects: cs.CL, cs.DC
• Abstract URL: https://arxiv.org/abs/2601.22124
• Pdf URL: https://arxiv.org/pdf/2601.22124
• Copy Paste: [[2601.22124]] A Federated and Parameter-Efficient Framework for Large Language Model Training in Medicine(https://arxiv.org/abs/2601.22124)
• Keywords: extraction, federate, large language model
• Abstract: Large language models (LLMs) have demonstrated strong performance on medical benchmarks, including question answering and diagnosis. To enable their use in clinical settings, LLMs are typically further adapted through continued pretraining or post-training using clinical data. However, most medical LLMs are trained on data from a single institution, which faces limitations in generalizability and safety in heterogeneous systems. Federated learning (FL) is a promising solution for enabling collaborative model development across healthcare institutions. Yet applying FL to LLMs in medicine remains fundamentally limited. First, conventional FL requires transmitting the full model during each communication round, which becomes impractical for multi-billion-parameter LLMs given the limited computational resources. Second, many FL algorithms implicitly assume data homogeneity, whereas real-world clinical data are highly heterogeneous across patients, diseases, and institutional practices. We introduce the model-agnostic and parameter-efficient federated learning framework for adapting LLMs to medical applications. Fed-MedLoRA transmits only low-rank adapter parameters, reducing communication and computation overhead, while Fed-MedLoRA+ further incorporates adaptive, data-aware aggregation to improve convergence under cross-site heterogeneity. We apply the framework to clinical information extraction (IE), which transforms patient narratives into structured medical entities and relations. Accuracy was assessed across five patient cohorts through comparisons with BERT models, and LLaMA-3 and DeepSeek-R1, GPT-4o models. Evaluation settings included (1) in-domain training and testing, (2) external validation on independent cohorts, and (3) a low-resource new-site adaptation scenario using real-world clinical notes from the Yale New Haven Health System.

Title: Creative Image Generation with Diffusion Model

• Authors: Kunpeng Song, Ahmed Elgammal
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22125
• Pdf URL: https://arxiv.org/pdf/2601.22125
• Copy Paste: [[2601.22125]] Creative Image Generation with Diffusion Model(https://arxiv.org/abs/2601.22125)
• Keywords: diffusion, generative
• Abstract: Creative image generation has emerged as a compelling area of research, driven by the need to produce novel and high-quality images that expand the boundaries of imagination. In this work, we propose a novel framework for creative generation using diffusion models, where creativity is associated with the inverse probability of an image's existence in the CLIP embedding space. Unlike prior approaches that rely on a manual blending of concepts or exclusion of subcategories, our method calculates the probability distribution of generated images and drives it towards low-probability regions to produce rare, imaginative, and visually captivating outputs. We also introduce pullback mechanisms, achieving high creativity without sacrificing visual fidelity. Extensive experiments on text-to-image diffusion models demonstrate the effectiveness and efficiency of our creative generation framework, showcasing its ability to produce unique, novel, and thought-provoking images. This work provides a new perspective on creativity in generative models, offering a principled method to foster innovation in visual content synthesis.

Title: EditYourself: Audio-Driven Generation and Manipulation of Talking Head Videos with Diffusion Transformers

• Authors: John Flynn, Wolfgang Paier, Dimitar Dinev, Sam Nhut Nguyen, Hayk Poghosyan, Manuel Toribio, Sandipan Banerjee, Guy Gafni
• Subjects: cs.CV, cs.GR, cs.LG, cs.MM
• Abstract URL: https://arxiv.org/abs/2601.22127
• Pdf URL: https://arxiv.org/pdf/2601.22127
• Copy Paste: [[2601.22127]] EditYourself: Audio-Driven Generation and Manipulation of Talking Head Videos with Diffusion Transformers(https://arxiv.org/abs/2601.22127)
• Keywords: diffusion, transformer, generative
• Abstract: Current generative video models excel at producing novel content from text and image prompts, but leave a critical gap in editing existing pre-recorded videos, where minor alterations to the spoken script require preserving motion, temporal coherence, speaker identity, and accurate lip synchronization. We introduce EditYourself, a DiT-based framework for audio-driven video-to-video (V2V) editing that enables transcript-based modification of talking head videos, including the seamless addition, removal, and retiming of visually spoken content. Building on a general-purpose video diffusion model, EditYourself augments its V2V capabilities with audio conditioning and region-aware, edit-focused training extensions. This enables precise lip synchronization and temporally coherent restructuring of existing performances via spatiotemporal inpainting, including the synthesis of realistic human motion in newly added segments, while maintaining visual fidelity and identity consistency over long durations. This work represents a foundational step toward generative video models as practical tools for professional video post-production.

Title: Pay for Hints, Not Answers: LLM Shepherding for Cost-Efficient Inference

• Authors: Ziming Dong, Hardik Sharma, Evan O'Toole, Jaya Prakash Champati, Kui Wu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22132
• Pdf URL: https://arxiv.org/pdf/2601.22132
• Copy Paste: [[2601.22132]] Pay for Hints, Not Answers: LLM Shepherding for Cost-Efficient Inference(https://arxiv.org/abs/2601.22132)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) deliver state-of-the-art performance on complex reasoning tasks, but their inference costs limit deployment at scale. Small Language Models (SLMs) offer dramatic cost savings yet lag substantially in accuracy. Existing approaches - routing and cascading - treat the LLM as an all-or-nothing resource: either the query bypasses the LLM entirely, or the LLM generates a complete response at full cost. We introduce LLM Shepherding, a framework that requests only a short prefix (a hint) from the LLM and provides it to SLM. This simple mechanism is surprisingly effective for math and coding tasks: even hints comprising 10-30% of the full LLM response improve SLM accuracy significantly. Shepherding generalizes both routing and cascading, and it achieves lower cost under oracle decision-making. We develop a two-stage predictor that jointly determines whether a hint is needed and how many tokens to request. On the widely-used mathematical reasoning (GSM8K, CNK12) and code generation (HumanEval, MBPP) benchmarks, Shepherding reduces costs by 42-94% relative to LLM-only inference. Compared to state-of-the-art routing and cascading baselines, shepherding delivers up to 2.8x cost reduction while matching accuracy. To our knowledge, this is the first work to exploit token-level budget control for SLM-LLM collaboration.

Title: PI-Light: Physics-Inspired Diffusion for Full-Image Relighting

• Authors: Zhexin Liang, Zhaoxi Chen, Yongwei Chen, Tianyi Wei, Tengfei Wang, Xingang Pan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22135
• Pdf URL: https://arxiv.org/pdf/2601.22135
• Copy Paste: [[2601.22135]] PI-Light: Physics-Inspired Diffusion for Full-Image Relighting(https://arxiv.org/abs/2601.22135)
• Keywords: diffusion
• Abstract: Full-image relighting remains a challenging problem due to the difficulty of collecting large-scale structured paired data, the difficulty of maintaining physical plausibility, and the limited generalizability imposed by data-driven priors. Existing attempts to bridge the synthetic-to-real gap for full-scene relighting remain suboptimal. To tackle these challenges, we introduce Physics-Inspired diffusion for full-image reLight ($\pi$-Light, or PI-Light), a two-stage framework that leverages physics-inspired diffusion models. Our design incorporates (i) batch-aware attention, which improves the consistency of intrinsic predictions across a collection of images, (ii) a physics-guided neural rendering module that enforces physically plausible light transport, (iii) physics-inspired losses that regularize training dynamics toward a physically meaningful landscape, thereby enhancing generalizability to real-world image editing, and (iv) a carefully curated dataset of diverse objects and scenes captured under controlled lighting conditions. Together, these components enable efficient finetuning of pretrained diffusion models while also providing a solid benchmark for downstream evaluation. Experiments demonstrate that $\pi$-Light synthesizes specular highlights and diffuse reflections across a wide variety of materials, achieving superior generalization to real-world scenes compared with prior approaches.

Title: StepShield: When, Not Whether to Intervene on Rogue Agents

• Authors: Gloria Felicia (University of Virginia), Michael Eniolade (University of the Cumberlands), Jinfeng He (Cornell University), Zitha Sasindran (Indian Institute of Science Bangalore), Hemant Kumar (University of Arizona), Milan Hussain Angati (California State University Northridge), Sandeep Bandarupalli (University of Cincinnati)
• Subjects: cs.LG, cs.AI, cs.CR, cs.SE
• Abstract URL: https://arxiv.org/abs/2601.22136
• Pdf URL: https://arxiv.org/pdf/2601.22136
• Copy Paste: [[2601.22136]] StepShield: When, Not Whether to Intervene on Rogue Agents(https://arxiv.org/abs/2601.22136)
• Keywords: security
• Abstract: Existing agent safety benchmarks report binary accuracy, conflating early intervention with post-mortem analysis. A detector that flags a violation at step 8 enables intervention; one that reports it at step 48 provides only forensic value. This distinction is critical, yet current benchmarks cannot measure it. We introduce StepShield, the first benchmark to evaluate when violations are detected, not just whether. StepShield contains 9,213 code agent trajectories, including 1,278 meticulously annotated training pairs and a 7,935-trajectory test set with a realistic 8.1% rogue rate. Rogue behaviors are grounded in real-world security incidents across six categories. We propose three novel temporal metrics: Early Intervention Rate (EIR), Intervention Gap, and Tokens Saved. Surprisingly, our evaluation reveals that an LLM-based judge achieves 59% EIR while a static analyzer achieves only 26%, a 2.3x performance gap that is entirely invisible to standard accuracy metrics. We further show that early detection has direct economic benefits: our cascaded HybridGuard detector reduces monitoring costs by 75% and projects to $108M in cumulative savings over five years at enterprise scale. By shifting the focus of evaluation from whether to when, StepShield provides a new foundation for building safer and more economically viable AI agents. The code and data are released under an Apache 2.0 license.

Title: Reasoning While Asking: Transforming Reasoning Large Language Models from Passive Solvers to Proactive Inquirers

• Authors: Xin Chen, Feng Jiang, Yiqian Zhang, Hardy Chen, Shuo Yan, Wenya Xie, Min Yang, Shujian Huang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22139
• Pdf URL: https://arxiv.org/pdf/2601.22139
• Copy Paste: [[2601.22139]] Reasoning While Asking: Transforming Reasoning Large Language Models from Passive Solvers to Proactive Inquirers(https://arxiv.org/abs/2601.22139)
• Keywords: robust, large language model
• Abstract: Reasoning-oriented Large Language Models (LLMs) have achieved remarkable progress with Chain-of-Thought (CoT) prompting, yet they remain fundamentally limited by a \emph{blind self-thinking} paradigm: performing extensive internal reasoning even when critical information is missing or ambiguous. We propose Proactive Interactive Reasoning (PIR), a new reasoning paradigm that transforms LLMs from passive solvers into proactive inquirers that interleave reasoning with clarification. Unlike existing search- or tool-based frameworks that primarily address knowledge uncertainty by querying external environments, PIR targets premise- and intent-level uncertainty through direct interaction with the user. PIR is implemented via two core components: (1) an uncertainty-aware supervised fine-tuning procedure that equips models with interactive reasoning capability, and (2) a user-simulator-based policy optimization framework driven by a composite reward that aligns model behavior with user intent. Extensive experiments on mathematical reasoning, code generation, and document editing demonstrate that PIR consistently outperforms strong baselines, achieving up to 32.70\% higher accuracy, 22.90\% higher pass rate, and 41.36 BLEU improvement, while reducing nearly half of the reasoning computation and unnecessary interaction turns. Further reliability evaluations on factual knowledge, question answering, and missing-premise scenarios confirm the strong generalization and robustness of PIR. Model and code are publicly available at: \href{this https URL}

Title: FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale

• Authors: Ajay Patel, Colin Raffel, Chris Callison-Burch
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22146
• Pdf URL: https://arxiv.org/pdf/2601.22146
• Copy Paste: [[2601.22146]] FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale(https://arxiv.org/abs/2601.22146)
• Keywords: large language model
• Abstract: Due to limited supervised training data, large language models (LLMs) are typically pre-trained via a self-supervised "predict the next word" objective on a vast amount of unstructured text data. To make the resulting model useful to users, it is further trained on a far smaller amount of "instruction-tuning" data comprised of supervised training examples of instructions and responses. To overcome the limited amount of supervised data, we propose a procedure that can transform the knowledge in internet-scale pre-training documents into billions of synthetic instruction and answer training pairs. The resulting dataset, called FineInstructions, uses ~18M instruction templates created from real user-written queries and prompts. These instruction templates are matched to and instantiated with human-written source documents from unstructured pre-training corpora. With "supervised" synthetic training data generated at this scale, an LLM can be pre-trained from scratch solely with the instruction-tuning objective, which is far more in-distribution with the expected downstream usage of LLMs (responding to user prompts). We conduct controlled token-for-token training experiments and find pre-training on FineInstructions outperforms standard pre-training and other proposed synthetic pre-training techniques on standard benchmarks measuring free-form response quality. Our resources can be found at this https URL .

Title: DynaWeb: Model-Based Reinforcement Learning of Web Agents

• Authors: Hang Ding, Peidong Liu, Junqiao Wang, Ziwei Ji, Meng Cao, Rongzhao Zhang, Lynn Ai, Eric Yang, Tianyu Shi, Lei Yu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.22149
• Pdf URL: https://arxiv.org/pdf/2601.22149
• Copy Paste: [[2601.22149]] DynaWeb: Model-Based Reinforcement Learning of Web Agents(https://arxiv.org/abs/2601.22149)
• Keywords: large language model
• Abstract: The development of autonomous web agents, powered by Large Language Models (LLMs) and reinforcement learning (RL), represents a significant step towards general-purpose AI assistants. However, training these agents is severely hampered by the challenges of interacting with the live internet, which is inefficient, costly, and fraught with risks. Model-based reinforcement learning (MBRL) offers a promising solution by learning a world model of the environment to enable simulated interaction. This paper introduces DynaWeb, a novel MBRL framework that trains web agents through interacting with a web world model trained to predict naturalistic web page representations given agent actions. This model serves as a synthetic web environment where an agent policy can dream by generating vast quantities of rollout action trajectories for efficient online reinforcement learning. Beyond free policy rollouts, DynaWeb incorporates real expert trajectories from training data, which are randomly interleaved with on-policy rollouts during training to improve stability and sample efficiency. Experiments conducted on the challenging WebArena and WebVoyager benchmarks demonstrate that DynaWeb consistently and significantly improves the performance of state-of-the-art open-source web agent models. Our findings establish the viability of training web agents through imagination, offering a scalable and efficient way to scale up online agentic RL.

Title: UEval: A Benchmark for Unified Multimodal Generation

• Authors: Bo Li, Yida Yin, Wenhao Chai, Xingyu Fu, Zhuang Liu
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22155
• Pdf URL: https://arxiv.org/pdf/2601.22155
• Copy Paste: [[2601.22155]] UEval: A Benchmark for Unified Multimodal Generation(https://arxiv.org/abs/2601.22155)
• Keywords: large language model
• Abstract: We introduce UEval, a benchmark to evaluate unified models, i.e., models capable of generating both images and text. UEval comprises 1,000 expert-curated questions that require both images and text in the model output, sourced from 8 real-world tasks. Our curated questions cover a wide range of reasoning types, from step-by-step guides to textbook explanations. Evaluating open-ended multimodal generation is non-trivial, as simple LLM-as-a-judge methods can miss the subtleties. Different from previous works that rely on multimodal Large Language Models (MLLMs) to rate image quality or text accuracy, we design a rubric-based scoring system in UEval. For each question, reference images and text answers are provided to a MLLM to generate an initial rubric, consisting of multiple evaluation criteria, and human experts then refine and validate these rubrics. In total, UEval contains 10,417 validated rubric criteria, enabling scalable and fine-grained automatic scoring. UEval is challenging for current unified models: GPT-5-Thinking scores only 66.4 out of 100, while the best open-source model reaches merely 49.1. We observe that reasoning models often outperform non-reasoning ones, and transferring reasoning traces from a reasoning model to a non-reasoning model significantly narrows the gap. This suggests that reasoning may be important for tasks requiring complex multimodal understanding and generation.

Title: Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts

• Authors: Yingfa Chen, Zhen Leng Thai, Zihan Zhou, Zhu Zhang, Xingyu Shen, Shuo Wang, Chaojun Xiao, Xu Han, Zhiyuan Liu
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.22156
• Pdf URL: https://arxiv.org/pdf/2601.22156
• Copy Paste: [[2601.22156]] Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts(https://arxiv.org/abs/2601.22156)
• Keywords: transformer
• Abstract: Hybrid Transformer architectures, which combine softmax attention blocks and recurrent neural networks (RNNs), have shown a desirable performance-throughput tradeoff for long-context modeling, but their adoption and studies are hindered by the prohibitive cost of large-scale pre-training from scratch. Some recent studies have shown that pre-trained softmax attention blocks can be converted into RNN blocks through parameter transfer and knowledge distillation. However, these transfer methods require substantial amounts of training data (more than 10B tokens), and the resulting hybrid models also exhibit poor long-context performance, which is the scenario where hybrid models enjoy significant inference speedups over Transformer-based models. In this paper, we present HALO (Hybrid Attention via Layer Optimization), a pipeline for distilling Transformer models into RNN-attention hybrid models. We then present HypeNet, a hybrid architecture with superior length generalization enabled by a novel position encoding scheme (named HyPE) and various architectural modifications. We convert the Qwen3 series into HypeNet using HALO, achieving performance comparable to the original Transformer models while enjoying superior long-context performance and efficiency. The conversion requires just 2.3B tokens, less than 0.01% of their pre-training data

Title: One-step Latent-free Image Generation with Pixel Mean Flows

• Authors: Yiyang Lu, Susie Lu, Qiao Sun, Hanhong Zhao, Zhicheng Jiang, Xianbang Wang, Tianhong Li, Zhengyang Geng, Kaiming He
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2601.22158
• Pdf URL: https://arxiv.org/pdf/2601.22158
• Copy Paste: [[2601.22158]] One-step Latent-free Image Generation with Pixel Mean Flows(https://arxiv.org/abs/2601.22158)
• Keywords: diffusion, generative
• Abstract: Modern diffusion/flow-based models for image generation typically exhibit two core characteristics: (i) using multi-step sampling, and (ii) operating in a latent space. Recent advances have made encouraging progress on each aspect individually, paving the way toward one-step diffusion/flow without latents. In this work, we take a further step towards this goal and propose "pixel MeanFlow" (pMF). Our core guideline is to formulate the network output space and the loss space separately. The network target is designed to be on a presumed low-dimensional image manifold (i.e., x-prediction), while the loss is defined via MeanFlow in the velocity space. We introduce a simple transformation between the image manifold and the average velocity field. In experiments, pMF achieves strong results for one-step latent-free generation on ImageNet at 256x256 resolution (2.22 FID) and 512x512 resolution (2.48 FID), filling a key missing piece in this regime. We hope that our study will further advance the boundaries of diffusion/flow-based generative models.

Title: RedSage: A Cybersecurity Generalist LLM

• Authors: Naufal Suryanto, Muzammal Naseer, Pengfei Li, Syed Talal Wasim, Jinhui Yi, Juergen Gall, Paolo Ceravolo, Ernesto Damiani
• Subjects: cs.CR, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2601.22159
• Pdf URL: https://arxiv.org/pdf/2601.22159
• Copy Paste: [[2601.22159]] RedSage: A Cybersecurity Generalist LLM(https://arxiv.org/abs/2601.22159)
• Keywords: secure, security, privacy
• Abstract: Cybersecurity operations demand assistant LLMs that support diverse workflows without exposing sensitive data. Existing solutions either rely on proprietary APIs with privacy risks or on open models lacking domain adaptation. To bridge this gap, we curate 11.8B tokens of cybersecurity-focused continual pretraining data via large-scale web filtering and manual collection of high-quality resources, spanning 28.6K documents across frameworks, offensive techniques, and security tools. Building on this, we design an agentic augmentation pipeline that simulates expert workflows to generate 266K multi-turn cybersecurity samples for supervised fine-tuning. Combined with general open-source LLM data, these resources enable the training of RedSage, an open-source, locally deployable cybersecurity assistant with domain-aware pretraining and post-training. To rigorously evaluate the models, we introduce RedSage-Bench, a benchmark with 30K multiple-choice and 240 open-ended Q&A items covering cybersecurity knowledge, skills, and tool expertise. RedSage is further evaluated on established cybersecurity benchmarks (e.g., CTI-Bench, CyberMetric, SECURE) and general LLM benchmarks to assess broader generalization. At the 8B scale, RedSage achieves consistently better results, surpassing the baseline models by up to +5.59 points on cybersecurity benchmarks and +5.05 points on Open LLM Leaderboard tasks. These findings demonstrate that domain-aware agentic augmentation and pre/post-training can not only enhance cybersecurity-specific expertise but also help to improve general reasoning and instruction-following. All models, datasets, and code are publicly available.