2026-02-25

Title: Talking to Yourself: Defying Forgetting in Large Language Models

• Authors: Yutao Sun, Mingshuai Chen, Tiancheng Zhao, Phillip Miao, Zilun Zhang, Haozhan Shen, Ruizhe Zhu, Jianwei Yin
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.20162
• Pdf URL: https://arxiv.org/pdf/2602.20162
• Copy Paste: [[2602.20162]] Talking to Yourself: Defying Forgetting in Large Language Models(https://arxiv.org/abs/2602.20162)
• Keywords: language model, llm
• Abstract: Catastrophic forgetting remains a major challenge when fine-tuning large language models (LLMs) on narrow, task-specific data, often degrading their general knowledge and reasoning abilities. We propose SA-SFT, a lightweight self-augmentation routine in which an LLM generates self-dialogues prior to fine-tuning, and the resulting self-authored data are mixed with task data without modifying optimization or training schedules. Despite requiring no external data or additional tuning, SA-SFT consistently mitigates catastrophic forgetting while improving in-domain performance. Across 50 evaluation scenarios, it maintains performance comparable to the original model and achieves the best results in 40 cases, outperforming common baselines such as layer freezing and external data mixing. Guided by these empirical findings, we further present a theoretical analysis suggesting that forgetting can partly stem from style-induced parameter drift, and that self-alignment through self-generated data provides an effective means to counteract this effect. Overall, our results indicate that self-augmentation offers a simple and effective mechanism for robust LLM adaptation without incurring catastrophic forgetting.
• 摘要：在针对狭窄的特定任务数据微调大型语言模型 (LLM) 时，灾难性遗忘仍然是一个重大挑战，通常会降低其常识和推理能力。我们提出了 SA-SFT，这是一种轻量级的自我增强例程，其中法学硕士在微调之前生成自我对话，并将生成的自创作数据与任务数据混合，而无需修改优化或训练计划。尽管不需要外部数据或额外的调整，SA-SFT 始终能够减轻灾难性遗忘，同时提高域内性能。在 50 个评估场景中，它保持了与原始模型相当的性能，并在 40 个案例中取得了最佳结果，优于层冻结和外部数据混合等常见基线。在这些实证研究结果的指导下，我们进一步提出了理论分析，表明遗忘可能部分源于风格引起的参数漂移，而通过自我生成的数据进行自我调整提供了抵消这种影响的有效方法。总的来说，我们的结果表明，自我增强为法学硕士的稳健适应提供了一种简单而有效的机制，而不会导致灾难性遗忘。

Title: Benchmarking Distilled Language Models: Performance and Efficiency in Resource-Constrained Settings

• Authors: Sachin Gopal Wani, Eric Page, Ajay Dholakia, David Ellison
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.20164
• Pdf URL: https://arxiv.org/pdf/2602.20164
• Copy Paste: [[2602.20164]] Benchmarking Distilled Language Models: Performance and Efficiency in Resource-Constrained Settings(https://arxiv.org/abs/2602.20164)
• Keywords: language model
• Abstract: Knowledge distillation offers a transformative pathway to developing powerful, yet efficient, small language models (SLMs) suitable for resource-constrained environments. In this paper, we benchmark the performance and computational cost of distilled models against their vanilla and proprietary counterparts, providing a quantitative analysis of their efficiency. Our results demonstrate that distillation creates a superior performance-tocompute curve. We find that creating a distilled 8B model is over 2,000 times more compute-efficient than training its vanilla counterpart, while achieving reasoning capabilities on par with, or even exceeding, standard models ten times its size. These findings validate distillation not just as a compression technique, but as a primary strategy for building state-of-the-art, accessible AI
• 摘要：知识蒸馏为开发适用于资源受限环境的强大且高效的小型语言模型 (SLM) 提供了一条变革性途径。在本文中，我们将蒸馏模型的性能和计算成本与普通模型和专有模型进行基准测试，对其效率进行定量分析。我们的结果表明，蒸馏创造了卓越的性能计算曲线。我们发现，创建一个精炼的 8B 模型的计算效率比训练普通模型高 2,000 倍以上，同时推理能力与标准模型相当，甚至超过其规模十倍的标准模型。这些发现验证了蒸馏不仅是一种压缩技术，而且是构建最先进、易于访问的人工智能的主要策略

Title: ConceptRM: The Quest to Mitigate Alert Fatigue through Consensus-Based Purity-Driven Data Cleaning for Reflection Modelling

• Authors: Yongda Yu, Lei Zhang, Xinxin Guo, Minghui Yu, Zhengqi Zhuang, Guoping Rong, Haifeng Shen, Zhengfeng Li, Boge Wang, Guoan Zhang, Bangyu Xiang, Xiaobin Xu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.20166
• Pdf URL: https://arxiv.org/pdf/2602.20166
• Copy Paste: [[2602.20166]] ConceptRM: The Quest to Mitigate Alert Fatigue through Consensus-Based Purity-Driven Data Cleaning for Reflection Modelling(https://arxiv.org/abs/2602.20166)
• Keywords: llm, agent
• Abstract: In many applications involving intelligent agents, the overwhelming volume of alerts (mostly false) generated by the agents may desensitize users and cause them to overlook critical issues, leading to the so-called ''alert fatigue''. A common strategy is to train a reflection model as a filter to intercept false alerts with labelled data collected from user verification feedback. However, a key challenge is the noisy nature of such data as it is often collected in production environments. As cleaning noise via manual annotation incurs high costs, this paper proposes a novel method ConceptRM for constructing a high-quality corpus to train a reflection model capable of effectively intercepting false alerts. With only a small amount of expert annotations as anchors, ConceptRM creates perturbed datasets with varying noise ratios and utilizes co-teaching to train multiple distinct models for collaborative learning. By analyzing the consensus decisions of these models, it effectively identifies reliable negative samples from a noisy dataset. Experimental results demonstrate that ConceptRM significantly enhances the interception of false alerts with minimal annotation cost, outperforming several state-of-the-art LLM baselines by up to 53.31% on in-domain datasets and 41.67% on out-of-domain datasets.
• 摘要：在许多涉及智能代理的应用中，代理生成的大量警报（大部分是错误的）可能会使用户变得不敏感，导致他们忽视关键问题，从而导致所谓的“警报疲劳”。一种常见的策略是训练反射模型作为过滤器，以使用从用户验证反馈中收集的标记数据来拦截错误警报。然而，一个关键的挑战是此类数据的噪声性质，因为它通常是在生产环境中收集的。由于通过手动标注清除噪声的成本较高，本文提出了一种新方法ConceptRM，用于构建高质量语料库来训练能够有效拦截误报的反射模型。仅用少量专家注释作为锚点，ConceptRM 创建具有不同噪声比的扰动数据集，并利用协同教学来训练多个不同的模型以进行协作学习。通过分析这些模型的共识决策，它可以有效地从噪声数据集中识别出可靠的负样本。实验结果表明，ConceptRM 以最小的注释成本显着增强了对错误警报的拦截，在域内数据集上比几个最先进的 LLM 基线高出 53.31%，在域外数据集上高出 41.67%。

Title: InterviewSim: A Scalable Framework for Interview-Grounded Personality Simulation

• Authors: Yu Li, Pranav Narayanan Venkit, Yada Pruksachatkun, Chien-Sheng Wu
• Subjects: cs.CL, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2602.20294
• Pdf URL: https://arxiv.org/pdf/2602.20294
• Copy Paste: [[2602.20294]] InterviewSim: A Scalable Framework for Interview-Grounded Personality Simulation(https://arxiv.org/abs/2602.20294)
• Keywords: language model
• Abstract: Simulating real personalities with large language models requires grounding generation in authentic personal data. Existing evaluation approaches rely on demographic surveys, personality questionnaires, or short AI-led interviews as proxies, but lack direct assessment against what individuals actually said. We address this gap with an interview-grounded evaluation framework for personality simulation at a large scale. We extract over 671,000 question-answer pairs from 23,000 verified interview transcripts across 1,000 public personalities, each with an average of 11.5 hours of interview content. We propose a multi-dimensional evaluation framework with four complementary metrics measuring content similarity, factual consistency, personality alignment, and factual knowledge retention. Through systematic comparison, we demonstrate that methods grounded in real interview data substantially outperform those relying solely on biographical profiles or the model's parametric knowledge. We further reveal a trade-off in how interview data is best utilized: retrieval-augmented methods excel at capturing personality style and response quality, while chronological-based methods better preserve factual consistency and knowledge retention. Our evaluation framework enables principled method selection based on application requirements, and our empirical findings provide actionable insights for advancing personality simulation research.
• 摘要：使用大型语言模型模拟真实个性需要以真实的个人数据为基础进行生成。现有的评估方法依赖于人口统计调查、性格问卷或人工智能主导的简短访谈作为代理，但缺乏对个人实际言论的直接评估。我们通过基于访谈的大规模人格模拟评估框架来解决这一差距。我们从 1,000 名公众人物的 23,000 份经过验证的采访笔录中提取了超过 671,000 个问答对，每个问答对的采访内容平均为 11.5 小时。我们提出了一个多维度的评估框架，其中有四个互补的指标来衡量内容相似性、事实一致性、个性一致性和事实知识保留。通过系统比较，我们证明基于真实访谈数据的方法大大优于那些仅依赖传记资料或模型参数知识的方法。我们进一步揭示了如何最好地利用访谈数据的权衡：检索增强方法擅长捕捉个性风格和回答质量，而基于时间顺序的方法更好地保持事实一致性和知识保留。我们的评估框架可以根据应用需求选择有原则的方法，我们的实证研究结果为推进人格模拟研究提供了可行的见解。

Title: What Makes a Good Query? Measuring the Impact of Human-Confusing Linguistic Features on LLM Performance

• Authors: William Watson, Nicole Cho, Sumitra Ganesh, Manuela Veloso
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.20300
• Pdf URL: https://arxiv.org/pdf/2602.20300
• Copy Paste: [[2602.20300]] What Makes a Good Query? Measuring the Impact of Human-Confusing Linguistic Features on LLM Performance(https://arxiv.org/abs/2602.20300)
• Keywords: language model, llm, hallucination
• Abstract: Large Language Model (LLM) hallucinations are usually treated as defects of the model or its decoding strategy. Drawing on classical linguistics, we argue that a query's form can also shape a listener's (and model's) response. We operationalize this insight by constructing a 22-dimension query feature vector covering clause complexity, lexical rarity, and anaphora, negation, answerability, and intention grounding, all known to affect human comprehension. Using 369,837 real-world queries, we ask: Are there certain types of queries that make hallucination more likely? A large-scale analysis reveals a consistent "risk landscape": certain features such as deep clause nesting and underspecification align with higher hallucination propensity. In contrast, clear intention grounding and answerability align with lower hallucination rates. Others, including domain specificity, show mixed, dataset- and model-dependent effects. Thus, these findings establish an empirically observable query-feature representation correlated with hallucination risk, paving the way for guided query rewriting and future intervention studies.
• 摘要：大语言模型（LLM）幻觉通常被视为模型或其解码策略的缺陷。借鉴经典语言学，我们认为查询的形式也可以塑造听众（和模型）的响应。我们通过构建一个 22 维查询特征向量来操作这一见解，该向量涵盖子句复杂性、词汇稀有性、照应、否定、可回答性和意图基础，所有这些都会影响人类的理解。使用 369,837 个现实世界的查询，我们询问：是否存在某些类型的查询更容易产生幻觉？大规模分析揭示了一致的“风险景观”：某些特征（例如深层子句嵌套和不规范）与较高的幻觉倾向相一致。相比之下，明确的意图基础和责任感与较低的幻觉率相一致。其他因素（包括领域特异性）则表现出混合的、数据集和模型相关的影响。因此，这些发现建立了与幻觉风险相关的可凭经验观察的查询特征表示，为引导查询重写和未来的干预研究铺平了道路。

Title: No One Size Fits All: QueryBandits for Hallucination Mitigation

• Authors: Nicole Cho, William Watson, Alec Koppel, Sumitra Ganesh, Manuela Veloso
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.20332
• Pdf URL: https://arxiv.org/pdf/2602.20332
• Copy Paste: [[2602.20332]] No One Size Fits All: QueryBandits for Hallucination Mitigation(https://arxiv.org/abs/2602.20332)
• Keywords: language model, llm, hallucination
• Abstract: Advanced reasoning capabilities in Large Language Models (LLMs) have led to more frequent hallucinations; yet most mitigation work focuses on open-source models for post-hoc detection and parameter editing. The dearth of studies focusing on hallucinations in closed-source models is especially concerning, as they constitute the vast majority of models in institutional deployments. We introduce QueryBandits, a model-agnostic contextual bandit framework that adaptively learns online to select the optimal query-rewrite strategy by leveraging an empirically validated and calibrated reward function. Across 16 QA scenarios, our top QueryBandit (Thompson Sampling) achieves an 87.5% win rate over a No-Rewrite baseline and outperforms zero-shot static policies (e.g., Paraphrase or Expand) by 42.6% and 60.3%, respectively. Moreover, all contextual bandits outperform vanilla bandits across all datasets, with higher feature variance coinciding with greater variance in arm selection. This substantiates our finding that there is no single rewrite policy optimal for all queries. We also discover that certain static policies incur higher cumulative regret than No-Rewrite, indicating that an inflexible query-rewriting policy can worsen hallucinations. Thus, learning an online policy over semantic features with QueryBandits can shift model behavior purely through forward-pass mechanisms, enabling its use with closed-source models and bypassing the need for retraining or gradient-based adaptation.
• 摘要：大型语言模型（LLM）的高级推理能力导致更频繁的幻觉；然而，大多数缓解工作都集中在用于事后检测和参数编辑的开源模型上。缺乏针对闭源模型中幻觉的研究尤其令人担忧，因为它们构成了机构部署中的绝大多数模型。我们引入了 QueryBandits，这是一个与模型无关的上下文老虎机框架，它通过利用经过经验验证和校准的奖励函数自适应地在线学习以选择最佳的查询重写策略。在 16 个 QA 场景中，我们的顶级 QueryBandit（Thompson Sampling）在 No-Rewrite 基准上实现了 87.5% 的胜率，并且比零样本静态策略（例如，释义或扩展）分别高出 42.6% 和 60.3%。此外，所有上下文强盗在所有数据集中都优于普通强盗，较高的特征方差与手臂选择的较大方差相一致。这证实了我们的发现，即不存在适合所有查询的最佳重写策略。我们还发现，某些静态策略会比无重写策略产生更高的累积遗憾，这表明不灵活的查询重写策略可能会加剧幻觉。因此，使用 QueryBandits 学习语义特征的在线策略可以纯粹通过前向传递机制来改变模型行为，从而使其能够与闭源模型一起使用，并绕过重新训练或基于梯度的适应的需要。

Title: Case-Aware LLM-as-a-Judge Evaluation for Enterprise-Scale RAG Systems

• Authors: Mukul Chhabra, Luigi Medrano, Arush Verma
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.20379
• Pdf URL: https://arxiv.org/pdf/2602.20379
• Copy Paste: [[2602.20379]] Case-Aware LLM-as-a-Judge Evaluation for Enterprise-Scale RAG Systems(https://arxiv.org/abs/2602.20379)
• Keywords: llm, prompt, retrieval-augmented generation
• Abstract: Enterprise Retrieval-Augmented Generation (RAG) assistants operate in multi-turn, case-based workflows such as technical support and IT operations, where evaluation must reflect operational constraints, structured identifiers (e.g., error codes, versions), and resolution workflows. Existing RAG evaluation frameworks are primarily designed for benchmark-style or single-turn settings and often fail to capture enterprise-specific failure modes such as case misidentification, workflow misalignment, and partial resolution across turns. We present a case-aware LLM-as-a-Judge evaluation framework for enterprise multi-turn RAG systems. The framework evaluates each turn using eight operationally grounded metrics that separate retrieval quality, grounding fidelity, answer utility, precision integrity, and case/workflow alignment. A severity-aware scoring protocol reduces score inflation and improves diagnostic clarity across heterogeneous enterprise cases. The system uses deterministic prompting with strict JSON outputs, enabling scalable batch evaluation, regression testing, and production monitoring. Through a comparative study of two instruction-tuned models across short and long workflows, we show that generic proxy metrics provide ambiguous signals, while the proposed framework exposes enterprise-critical tradeoffs that are actionable for system improvement.
• 摘要：企业检索增强生成 (RAG) 助理在多回合、基于案例的工作流程中运行，例如技术支持和 IT 操作，其中评估必须反映操作约束、结构化标识符（例如错误代码、版本）和解决工作流程。现有的 RAG 评估框架主要是针对基准式或单轮设置而设计的，通常无法捕获企业特定的故障模式，例如案例错误识别、工作流程错位和跨轮部分解决。我们提出了一个适用于企业多轮 RAG 系统的案例感知法学硕士作为法官评估框架。该框架使用八个可操作的基础指标来评估每个回合，这些指标将检索质量、基础保真度、答案实用性、精度完整性和案例/工作流程对齐分开。严重性感知评分协议可减少分数膨胀并提高跨异构企业案例的诊断清晰度。该系统使用具有严格 JSON 输出的确定性提示，从而实现可扩展的批量评估、回归测试和生产监控。通过对短工作流程和长工作流程中两种指令调整模型的比较研究，我们表明通用代理指标提供了模糊的信号，而所提出的框架则暴露了可用于系统改进的企业关键权衡。

Title: Disentangling Geometry, Performance, and Training in Language Models

• Authors: Atharva Kulkarni, Jacob Mitchell Springer, Arjun Subramonian, Swabha Swayamdipta
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.20433
• Pdf URL: https://arxiv.org/pdf/2602.20433
• Copy Paste: [[2602.20433]] Disentangling Geometry, Performance, and Training in Language Models(https://arxiv.org/abs/2602.20433)
• Keywords: language model
• Abstract: Geometric properties of Transformer weights, particularly the unembedding matrix, have been widely useful in language model interpretability research. Yet, their utility for estimating downstream performance remains unclear. In this work, we systematically investigate the relationship between model performance and the unembedding matrix geometry, particularly its effective rank. Our experiments, involving a suite of 108 OLMo-style language models trained under controlled variation, reveal several key findings. While the best-performing models often exhibit a high effective rank, this trend is not universal across tasks and training setups. Contrary to prior work, we find that low effective rank does not cause late-stage performance degradation in small models, but instead co-occurs with it; we find adversarial cases where low-rank models do not exhibit saturation. Moreover, we show that effective rank is strongly influenced by pre-training hyperparameters, such as batch size and weight decay, which in-turn affect the model's performance. Lastly, extending our analysis to other geometric metrics and final-layer representation, we find that these metrics are largely aligned, but none can reliably predict downstream performance. Overall, our findings suggest that the model's geometry, as captured by existing metrics, primarily reflects training choices rather than performance.
• 摘要：Transformer 权重的几何特性，特别是非嵌入矩阵，在语言模型可解释性研究中广泛有用。然而，它们在估计下游性能方面的效用仍不清楚。在这项工作中，我们系统地研究了模型性能与非嵌入矩阵几何形状之间的关系，特别是其有效秩。我们的实验涉及在受控变量下训练的一套 108 个 OLMo 风格语言模型，揭示了几个关键发现。虽然表现最好的模型通常表现出较高的有效排名，但这种趋势在任务和训练设置中并不普遍。与之前的工作相反，我们发现低有效等级不会导致小模型的后期性能下降，而是与之同时发生；我们发现低秩模型没有表现出饱和的对抗性情况。此外，我们表明有效排名受到预训练超参数的强烈影响，例如批量大小和权重衰减，这反过来又影响模型的性能。最后，将我们的分析扩展到其他几何指标和最终层表示，我们发现这些指标在很大程度上是一致的，但没有一个可以可靠地预测下游性能。总的来说，我们的研究结果表明，现有指标所捕获的模型几何形状主要反映了训练选择而不是性能。

Title: From Performance to Purpose: A Sociotechnical Taxonomy for Evaluating Large Language Model Utility

• Authors: Gavin Levinson, Keith Feldman
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.20513
• Pdf URL: https://arxiv.org/pdf/2602.20513
• Copy Paste: [[2602.20513]] From Performance to Purpose: A Sociotechnical Taxonomy for Evaluating Large Language Model Utility(https://arxiv.org/abs/2602.20513)
• Keywords: language model, llm
• Abstract: As large language models (LLMs) continue to improve at completing discrete tasks, they are being integrated into increasingly complex and diverse real-world systems. However, task-level success alone does not establish a model's fit for use in practice. In applied, high-stakes settings, LLM effectiveness is driven by a wider array of sociotechnical determinants that extend beyond conventional performance measures. Although a growing set of metrics capture many of these considerations, they are rarely organized in a way that supports consistent evaluation, leaving no unified taxonomy for assessing and comparing LLM utility across use cases. To address this gap, we introduce the Language Model Utility Taxonomy (LUX), a comprehensive framework that structures utility evaluation across four domains: performance, interaction, operations, and governance. Within each domain, LUX is organized hierarchically into thematically aligned dimensions and components, each grounded in metrics that enable quantitative comparison and alignment of model selection with intended use. In addition, an external dynamic web tool is provided to support exploration of the framework by connecting each component to a repository of relevant metrics (factors) for applied evaluation.
• 摘要：随着大型语言模型 (LLM) 在完成离散任务方面不断改进，它们正在被集成到日益复杂和多样化的现实世界系统中。然而，仅任务级别的成功并不能建立模型是否适合在实践中使用。在高风险的应用环境中，法学硕士的有效性是由更广泛的社会技术决定因素驱动的，这些因素超出了传统的绩效衡量标准。尽管越来越多的指标涵盖了其中许多考虑因素，但它们很少以支持一致评估的方式进行组织，从而没有留下统一的分类法来评估和比较跨用例的 LLM 效用。为了解决这一差距，我们引入了语言模型效用分类法 (LUX)，这是一个跨四个领域构建效用评估的综合框架：性能、交互、操作和治理。在每个领域内，LUX 都按层次结构组织成主题一致的维度和组件，每个维度和组件都基于能够定量比较以及模型选择与预期用途的一致性的指标。此外，还提供了一个外部动态网络工具，通过将每个组件连接到相关指标（因素）的存储库来支持对框架的探索，以进行应用评估。

Title: Stop-Think-AutoRegress: Language Modeling with Latent Diffusion Planning

• Authors: Justin Lovelace, Christian Belardi, Sofian Zalouk, Adhitya Polavaram, Srivatsa Kundurthy, Kilian Q. Weinberger
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.20528
• Pdf URL: https://arxiv.org/pdf/2602.20528
• Copy Paste: [[2602.20528]] Stop-Think-AutoRegress: Language Modeling with Latent Diffusion Planning(https://arxiv.org/abs/2602.20528)
• Keywords: language model, llm
• Abstract: The Stop-Think-AutoRegress Language Diffusion Model (STAR-LDM) integrates latent diffusion planning with autoregressive generation. Unlike conventional autoregressive language models limited to token-by-token decisions, STAR-LDM incorporates a "thinking" phase that pauses generation to refine a semantic plan through diffusion before continuing. This enables global planning in continuous space prior to committing to discrete tokens. Evaluations show STAR-LDM significantly outperforms similar-sized models on language understanding benchmarks and achieves $>70\%$ win rates in LLM-as-judge comparisons for narrative coherence and commonsense reasoning. The architecture also allows straightforward control through lightweight classifiers, enabling fine-grained steering of attributes without model retraining while maintaining better fluency-control trade-offs than specialized approaches.
• 摘要：Stop-Think-AutoRegress 语言扩散模型 (STAR-LDM) 将潜在扩散规划与自回归生成相结合。与仅限于逐个令牌决策的传统自回归语言模型不同，STAR-LDM 包含一个“思考”阶段，该阶段会暂停生成，以便在继续之前通过扩散来完善语义计划。这使得在提交离散代币之前能够在连续空间中进行全局规划。评估显示，STAR-LDM 在语言理解基准上显着优于类似大小的模型，并在法学硕士作为法官的叙述连贯性和常识推理比较中实现了 $>70%$ 的获胜率。该架构还允许通过轻量级分类器进行直接控制，无需模型重新训练即可实现属性的细粒度控制，同时保持比专门方法更好的流畅性控制权衡。

Title: Personal Information Parroting in Language Models

• Authors: Nishant Subramani, Kshitish Ghate, Mona Diab
• Subjects: cs.CL, cs.AI, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.20580
• Pdf URL: https://arxiv.org/pdf/2602.20580
• Copy Paste: [[2602.20580]] Personal Information Parroting in Language Models(https://arxiv.org/abs/2602.20580)
• Keywords: language model, prompt
• Abstract: Modern language models (LM) are trained on large scrapes of the Web, containing millions of personal information (PI) instances, many of which LMs memorize, increasing privacy risks. In this work, we develop the regexes and rules (R&R) detector suite to detect email addresses, phone numbers, and IP addresses, which outperforms the best regex-based PI detectors. On a manually curated set of 483 instances of PI, we measure memorization: finding that 13.6% are parroted verbatim by the Pythia-6.9b model, i.e., when the model is prompted with the tokens that precede the PI in the original document, greedy decoding generates the entire PI span exactly. We expand this analysis to study models of varying sizes (160M-6.9B) and pretraining time steps (70k-143k iterations) in the Pythia model suite and find that both model size and amount of pretraining are positively correlated with memorization. Even the smallest model, Pythia-160m, parrots 2.7% of the instances exactly. Consequently, we strongly recommend that pretraining datasets be aggressively filtered and anonymized to minimize PI parroting.
• 摘要：现代语言模型 (LM) 在大量网络数据上进行训练，其中包含数百万个个人信息 (PI) 实例，其中许多 LM 会记住，从而增加了隐私风险。在这项工作中，我们开发了正则表达式和规则 (R&R) 检测器套件来检测电子邮件地址、电话号码和 IP 地址，其性能优于基于正则表达式的最佳 PI 检测器。在手动策划的 483 个 PI 实例上，我们测量了记忆情况：发现 Pythia-6.9b 模型逐字重复了 13.6%，即，当使用原始文档中 PI 之前的标记提示模型时，贪婪解码会准确生成整个 PI 范围。我们将此分析扩展到研究 Pythia 模型套件中不同大小（160M-6.9B）和预训练时间步长（70k-143k 迭代）的模型，并发现模型大小和预训练量与记忆量呈正相关。即使是最小的模型 Pythia-160m，也能准确地模仿 2.7% 的实例。因此，我们强烈建议对预训练数据集进行积极过滤和匿名化，以最大限度地减少 PI 重复行为。

Title: CARE: An Explainable Computational Framework for Assessing Client-Perceived Therapeutic Alliance Using Large Language Models

• Authors: Anqi Li, Chenxiao Wang, Yu Lu, Renjun Xu, Lizhi Ma, Zhenzhong Lan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.20648
• Pdf URL: https://arxiv.org/pdf/2602.20648
• Copy Paste: [[2602.20648]] CARE: An Explainable Computational Framework for Assessing Client-Perceived Therapeutic Alliance Using Large Language Models(https://arxiv.org/abs/2602.20648)
• Keywords: language model, llm
• Abstract: Client perceptions of the therapeutic alliance are critical for counseling effectiveness. Accurately capturing these perceptions remains challenging, as traditional post-session questionnaires are burdensome and often delayed, while existing computational approaches produce coarse scores, lack interpretable rationales, and fail to model holistic session context. We present CARE, an LLM-based framework to automatically predict multi-dimensional alliance scores and generate interpretable rationales from counseling transcripts. Built on the CounselingWAI dataset and enriched with 9,516 expert-curated rationales, CARE is fine-tuned using rationale-augmented supervision with the LLaMA-3.1-8B-Instruct backbone. Experiments show that CARE outperforms leading LLMs and substantially reduces the gap between counselor evaluations and client-perceived alliance, achieving over 70% higher Pearson correlation with client ratings. Rationale-augmented supervision further improves predictive accuracy. CARE also produces high-quality, contextually grounded rationales, validated by both automatic and human evaluations. Applied to real-world Chinese online counseling sessions, CARE uncovers common alliance-building challenges, illustrates how interaction patterns shape alliance development, and provides actionable insights, demonstrating its potential as an AI-assisted tool for supporting mental health care.
• 摘要：来访者对治疗联盟的看法对于咨询效果至关重要。准确地捕捉这些看法仍然具有挑战性，因为传统的会后调查问卷很繁重并且经常延迟，而现有的计算方法会产生粗略的分数，缺乏可解释的理由，并且无法对整体会议背景进行建模。我们提出了 CARE，一个基于法学硕士的框架，可以自动预测多维联盟分数并从咨询记录中生成可解释的理由。 CARE 基于 CounselingWAI 数据集并丰富了 9,516 个专家策划的基本原理，并使用 LLaMA-3.1-8B-Instruct 主干网的基本原理增强监督进行了微调。实验表明，CARE 的表现优于领先的法学硕士，并大大缩小了顾问评估与客户感知的联盟之间的差距，实现了与客户评级的皮尔逊相关性提高 70% 以上。基本原理增强监督进一步提高了预测准确性。 CARE 还产生高质量、基于上下文的基本原理，并通过自动和人工评估进行验证。 CARE 应用于现实世界的中国在线咨询课程，揭示了常见的联盟建设挑战，说明了互动模式如何塑造联盟发展，并提供了可行的见解，展示了其作为支持心理健康护理的人工智能辅助工具的潜力。

Title: CAMEL: Confidence-Gated Reflection for Reward Modeling

• Authors: Zirui Zhu, Hailun Xu, Yang Luo, Yong Liu, Kanchan Sarkar, Kun Xu, Yang You
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.20670
• Pdf URL: https://arxiv.org/pdf/2602.20670
• Copy Paste: [[2602.20670]] CAMEL: Confidence-Gated Reflection for Reward Modeling(https://arxiv.org/abs/2602.20670)
• Keywords: language model
• Abstract: Reward models play a fundamental role in aligning large language models with human preferences. Existing methods predominantly follow two paradigms: scalar discriminative preference models, which are efficient but lack interpretability, and generative judging models, which offer richer reasoning at the cost of higher computational overhead. We observe that the log-probability margin between verdict tokens strongly correlates with prediction correctness, providing a reliable proxy for instance difficulty without additional inference cost. Building on this insight, we propose CAMEL, a confidence-gated reflection framework that performs a lightweight single-token preference decision first and selectively invokes reflection only for low-confidence instances. To induce effective self-correction, we train the model via reinforcement learning with counterfactual prefix augmentation, which exposes the model to diverse initial verdicts and encourages genuine revision. Empirically, CAMEL achieves state-of-the-art performance on three widely used reward-model benchmarks with 82.9% average accuracy, surpassing the best prior model by 3.2% and outperforming 70B-parameter models using only 14B parameters, while establishing a strictly better accuracy-efficiency Pareto frontier.
• 摘要：奖励模型在使大型语言模型与人类偏好保持一致方面发挥着基础作用。现有的方法主要遵循两种范式：标量判别偏好模型，其高效但缺乏可解释性；以及生成判断模型，其以更高的计算开销为代价提供更丰富的推理。我们观察到，判决标记之间的对数概率裕度与预测正确性密切相关，为实例难度提供了可靠的代理，而无需额外的推理成本。基于这一见解，我们提出了 CAMEL，这是一种置信度门控反射框架，它首先执行轻量级单令牌偏好决策，并仅针对低置信度实例有选择地调用反射。为了诱导有效的自我纠正，我们通过强化学习和反事实前缀增强来训练模型，这使模型暴露于不同的初始判决并鼓励真正的修正。根据经验，CAMEL 在三个广泛使用的奖励模型基准上实现了最先进的性能，平均准确率为 82.9%，比最佳先验模型高出 3.2%，并且仅使用 14B 参数就超越了 70B 参数模型，同时建立了严格更好的准确率-效率 Pareto 前沿。

Title: ID-LoRA: Efficient Low-Rank Adaptation Inspired by Matrix Interpolative Decomposition

• Authors: Xindian Ma, Rundong Kong, Peng Zhang, Ruoxiang Huang, Yongyu Jiang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.20727
• Pdf URL: https://arxiv.org/pdf/2602.20727
• Copy Paste: [[2602.20727]] ID-LoRA: Efficient Low-Rank Adaptation Inspired by Matrix Interpolative Decomposition(https://arxiv.org/abs/2602.20727)
• Keywords: language model, llm
• Abstract: LoRA has become a universal Parameter-Efficient Fine-Tuning (PEFT) technique that equips Large Language Models (LLMs) to adapt quickly to new tasks. However, when these models are scaled up, even the latest LoRA variants still introduce considerable overhead in trainable parameters. Conversely, aggressively lowering the rank to curb this overhead markedly degrades performance in complex multi-task settings. We propose ID-LoRA, a novel PEFT framework that breaks the trade-off. Its core innovation lies in extracting and reusing clustered parameter groups from the pretrained weight matrix. These groups are then used to form multiple low-rank components, all of which share only a single initialized trainable low-rank matrix. This approach cuts the number of trainable parameters while keeping the model's capacity intact. We evaluate ID-LoRA on five diverse benchmarks: Mathematical Reasoning, Code Generation, MMLU, CommonsenseQA, and Safety Alignment. ID-LoRA outperforms both full fine-tuning and existing PEFT baselines (e.g., LoRA, DoRA, HydraLoRA) while using up to 46% fewer trainable parameters than the standard LoRA. In multi-task scenarios, it surpasses LoRA and its recent variants (e.g., DoRA and HydraLoRA) on both Code and MMLU tasks, yet requires only 54% of the trainable parameters demanded by the conventional LoRA.
• 摘要：LoRA 已成为一种通用的参数高效微调 (PEFT) 技术，使大型语言模型 (LLM) 能够快速适应新任务。 However, when these models are scaled up, even the latest LoRA variants still introduce considerable overhead in trainable parameters. Conversely, aggressively lowering the rank to curb this overhead markedly degrades performance in complex multi-task settings. We propose ID-LoRA, a novel PEFT framework that breaks the trade-off. Its core innovation lies in extracting and reusing clustered parameter groups from the pretrained weight matrix. These groups are then used to form multiple low-rank components, all of which share only a single initialized trainable low-rank matrix. This approach cuts the number of trainable parameters while keeping the model's capacity intact. We evaluate ID-LoRA on five diverse benchmarks: Mathematical Reasoning, Code Generation, MMLU, CommonsenseQA, and Safety Alignment. ID-LoRA 的性能优于完全微调和现有的 PEFT 基线（例如 LoRA、DoRA、HydraLoRA），同时使用的可训练参数比标准 LoRA 少 46%。在多任务场景中，它在代码和 MMLU 任务上都超越了 LoRA 及其最新变体（例如 DoRA 和 HydraLoRA），但只需要传统 LoRA 所需的可训练参数的 54％。

Title: Adaptive Text Anonymization: Learning Privacy-Utility Trade-offs via Prompt Optimization

• Authors: Gabriel Loiseau, Damien Sileo, Damien Riquet, Maxime Meyer, Marc Tommasi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.20743
• Pdf URL: https://arxiv.org/pdf/2602.20743
• Copy Paste: [[2602.20743]] Adaptive Text Anonymization: Learning Privacy-Utility Trade-offs via Prompt Optimization(https://arxiv.org/abs/2602.20743)
• Keywords: language model, prompt
• Abstract: Anonymizing textual documents is a highly context-sensitive problem: the appropriate balance between privacy protection and utility preservation varies with the data domain, privacy objectives, and downstream application. However, existing anonymization methods rely on static, manually designed strategies that lack the flexibility to adjust to diverse requirements and often fail to generalize across domains. We introduce adaptive text anonymization, a new task formulation in which anonymization strategies are automatically adapted to specific privacy-utility requirements. We propose a framework for task-specific prompt optimization that automatically constructs anonymization instructions for language models, enabling adaptation to different privacy goals, domains, and downstream usage patterns. To evaluate our approach, we present a benchmark spanning five datasets with diverse domains, privacy constraints, and utility objectives. Across all evaluated settings, our framework consistently achieves a better privacy-utility trade-off than existing baselines, while remaining computationally efficient and effective on open-source language models, with performance comparable to larger closed-source models. Additionally, we show that our method can discover novel anonymization strategies that explore different points along the privacy-utility trade-off frontier.
• 摘要：文本文档匿名化是一个高度上下文相关的问题：隐私保护和实用性保存之间的适当平衡随着数据域、隐私目标和下游应用程序的不同而变化。然而，现有的匿名方法依赖于静态的、手动设计的策略，缺乏适应不同需求的灵活性，并且往往无法跨领域推广。我们引入了自适应文本匿名化，这是一种新的任务制定，其中匿名策略会自动适应特定的隐私实用程序要求。我们提出了一个用于特定任务提示优化的框架，该框架自动为语言模型构建匿名指令，从而能够适应不同的隐私目标、领域和下游使用模式。为了评估我们的方法，我们提出了一个跨越五个数据集的基准，这些数据集具有不同的领域、隐私约束和实用目标。在所有评估的设置中，我们的框架始终比现有基线实现更好的隐私与实用性权衡，同时在开源语言模型上保持计算效率和有效性，其性能可与较大的闭源模型相媲美。此外，我们还表明，我们的方法可以发现新颖的匿名化策略，探索隐私-实用性权衡前沿的不同点。

Title: SibylSense: Adaptive Rubric Learning via Memory Tuning and Adversarial Probing

• Authors: Yifei Xu, Guilherme Potje, Shivam Shandilya, Tiancheng Yuan, Leonardo de Oliveira Nunes, Rakshanda Agarwal, Saeid Asgari, Adam Atkinson, Emre Kıcıman, Songwu Lu, Ranveer Chandra, Tusher Chakraborty
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.20751
• Pdf URL: https://arxiv.org/pdf/2602.20751
• Copy Paste: [[2602.20751]] SibylSense: Adaptive Rubric Learning via Memory Tuning and Adversarial Probing(https://arxiv.org/abs/2602.20751)
• Keywords: prompt
• Abstract: Designing aligned and robust rewards for open-ended generation remains a key barrier to RL post-training. Rubrics provide structured, interpretable supervision, but scaling rubric construction is difficult: expert rubrics are costly, prompted rubrics are often superficial or inconsistent, and fixed-pool discriminative rubrics can saturate and drift, enabling reward hacking. We present SibylSense, an inference-time learning approach that adapts a frozen rubric generator through a tunable memory bank of validated rubric items. Memory is updated via verifier-based item rewards measured by reference-candidate answer discriminative gaps from a handful of examples. SibylSense alternates memory tuning with a rubric-adversarial policy update that produces rubric-satisfying candidate answers, shrinking discriminative gaps and driving the rubric generator to capture new quality dimensions. Experiments on two open-ended tasks show that SibylSense yields more discriminative rubrics and improves downstream RL performance over static and non-adaptive baselines.
• 摘要：为开放式生成设计一致且强大的奖励仍然是 RL 训练后的主要障碍。评分标准提供了结构化的、可解释的监督，但扩展评分标准的构建很困难：专家评分标准成本高昂，提示评分标准往往很肤浅或不一致，固定池的歧视性评分标准可能会饱和和漂移，从而导致奖励黑客行为。我们提出了 SibylSense，这是一种推理时间学习方法，它通过经过验证的评分项项目的可调内存库来适应冻结的评分生成器。记忆是通过基于验证者的项目奖励来更新的，该奖励是通过来自少数示例的参考候选答案的判别性差距来衡量的。 SibylSense 将内存调整与标题对抗策略更新交替使用，生成令人满意的候选答案，缩小判别差距并驱动标题生成器捕获新的质量维度。对两个开放式任务的实验表明，SibylSense 产生了更具区分性的规则，并在静态和非自适应基线上提高了下游 RL 性能。

Title: Overton Pluralistic Reinforcement Learning for Large Language Models

• Authors: Yu Fu, Seongho Son, Ilija Bogunovic
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.20759
• Pdf URL: https://arxiv.org/pdf/2602.20759
• Copy Paste: [[2602.20759]] Overton Pluralistic Reinforcement Learning for Large Language Models(https://arxiv.org/abs/2602.20759)
• Keywords: language model, gpt, prompt
• Abstract: Existing alignment paradigms remain limited in capturing the pluralistic nature of human values. Overton Pluralism addresses this gap by generating responses with diverse perspectives from a single query. This paper introduces OP-GRPO (Overton Pluralistic Group Relative Policy Optimization), a reinforcement learning framework for implicit Overton Pluralism that enables a single large language model to produce pluralistic responses without explicit prompting or modular orchestration. Our workflow consists of two main steps. First, similarity estimator training fine-tunes a Sentence Transformer for Overton Pluralism tasks to provide more accurate coverage evaluation of generated responses. Second, OP-GRPO training incorporates this similarity estimator into a dual-reward system designed to ensure both broad coverage of genuine human perspectives and the uniqueness of each perspective, thereby promoting diversity. Empirical results demonstrate a "small models, big perspective coverage" effect. The trained Qwen2.5-3B-Instruct model surpasses a 20B GPT-OSS baseline with a 37.4 percent relative accuracy gain on a Natural Language Inference benchmark, and also outperforms a modular architecture baseline with a 19.1 percent relative improvement. Additional evaluations using GPT-4.1 as a large language model judge further confirm the robustness of the approach.
• 摘要：现有的一致性范式在捕捉人类价值观的多元化本质方面仍然有限。奥弗顿多元论通过从单个查询生成具有不同观点的响应来解决这一差距。本文介绍了 OP-GRPO（Overton 多元组相对策略优化），这是一种隐式 Overton 多元主义的强化学习框架，使单个大型语言模型能够在没有显式提示或模块化编排的情况下产生多元响应。我们的工作流程包括两个主要步骤。首先，相似性估计器训练对 Overton Pluralism 任务的 Sentence Transformer 进行微调，以对生成的响应提供更准确的覆盖评估。其次，OP-GRPO 培训将这种相似性估计器纳入双重奖励系统中，旨在确保广泛覆盖真实的人类观点和每个观点的独特性，从而促进多样性。实证结果证明了“小模型，大视角覆盖”的效果。经过训练的 Qwen2.5-3B-Instruct 模型超越了 20B GPT-OSS 基线，在自然语言推理基准上相对准确度提高了 37.4%，并且还优于模块化架构基线，相对提高了 19.1%。使用 GPT-4.1 作为大型语言模型判断的附加评估进一步证实了该方法的稳健性。

Title: Don't Ignore the Tail: Decoupling top-K Probabilities for Efficient Language Model Distillation

• Authors: Sayantan Dasgupta, Trevor Cohn, Timothy Baldwin
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.20816
• Pdf URL: https://arxiv.org/pdf/2602.20816
• Copy Paste: [[2602.20816]] Don't Ignore the Tail: Decoupling top-K Probabilities for Efficient Language Model Distillation(https://arxiv.org/abs/2602.20816)
• Keywords: language model
• Abstract: The core learning signal used in language model distillation is the standard Kullback-Leibler (KL) divergence between the student and teacher distributions. Traditional KL divergence tends to be dominated by the next tokens with the highest probabilities, i.e., the teacher's modes, thereby diminishing the influence of less probable yet potentially informative components of the output distribution. We propose a new tail-aware divergence that decouples the contribution of the teacher model's top-K predicted probabilities from that of lower-probability predictions, while maintaining the same computational profile as the KL Divergence. Our decoupled approach reduces the impact of the teacher modes and, consequently, increases the contribution of the tail of the distribution. Experimental results demonstrate that our modified distillation method yields competitive performance in both pre-training and supervised distillation of decoder models across various datasets. Furthermore, the distillation process is efficient and can be performed with a modest academic budget for large datasets, eliminating the need for industry-scale computing.
• 摘要：语言模型蒸馏中使用的核心学习信号是学生和教师分布之间的标准 Kullback-Leibler (KL) 散度。传统的 KL 散度往往由具有最高概率的下一个标记（即教师模式）主导，从而减少了输出分布中概率较小但具有潜在信息性的组成部分的影响。我们提出了一种新的尾部感知散度，它将教师模型的前 K 个预测概率的贡献与较低概率预测的贡献分开，同时保持与 KL 散度相同的计算配置文件。我们的解耦方法减少了教师模式的影响，从而增加了分布尾部的贡献。实验结果表明，我们改进的蒸馏方法在跨各种数据集的解码器模型的预训练和监督蒸馏方面都产生了具有竞争力的性能。此外，蒸馏过程非常高效，并且可以用适度的学术预算来执行大型数据集，从而消除了对工业规模计算的需求。

Title: FinAnchor: Aligned Multi-Model Representations for Financial Prediction

• Authors: Zirui He, Huopu Zhang, Yanguang Liu, Sirui Wu, Mengnan Du
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.20859
• Pdf URL: https://arxiv.org/pdf/2602.20859
• Copy Paste: [[2602.20859]] FinAnchor: Aligned Multi-Model Representations for Financial Prediction(https://arxiv.org/abs/2602.20859)
• Keywords: llm
• Abstract: Financial prediction from long documents involves significant challenges, as actionable signals are often sparse and obscured by noise, and the optimal LLM for generating embeddings varies across tasks and time periods. In this paper, we propose FinAnchor(Financial Anchored Representations), a lightweight framework that integrates embeddings from multiple LLMs without fine-tuning the underlying models. FinAnchor addresses the incompatibility of feature spaces by selecting an anchor embedding space and learning linear mappings to align representations from other models into this anchor. These aligned features are then aggregated to form a unified representation for downstream prediction. Across multiple financial NLP tasks, FinAnchor consistently outperforms strong single-model baselines and standard ensemble methods, demonstrating the effectiveness of anchoring heterogeneous representations for robust financial prediction.
• 摘要：长文档的财务预测面临重大挑战，因为可操作的信号通常稀疏且被噪声掩盖，并且生成嵌入的最佳 LLM 因任务和时间段的不同而异。在本文中，我们提出了 FinAnchor（金融锚定表示），这是一个轻量级框架，集成了多个法学硕士的嵌入，无需微调底层模型。 FinAnchor 通过选择锚点嵌入空间并学习线性映射以将其他模型的表示与该锚点对齐来解决特征空间的不兼容性问题。然后聚合这些对齐的特征以形成下游预测的统一表示。在多个金融 NLP 任务中，FinAnchor 始终优于强大的单模型基线和标准集成方法，证明了锚定异构表示以实现稳健的金融预测的有效性。

Title: Exa-PSD: a new Persian sentiment analysis dataset on Twitter

• Authors: Seyed Himan Ghaderi, Saeed Sarbazi Azad, Mohammad Mehdi Jaziriyan, Ahmad Akbari
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.20892
• Pdf URL: https://arxiv.org/pdf/2602.20892
• Copy Paste: [[2602.20892]] Exa-PSD: a new Persian sentiment analysis dataset on Twitter(https://arxiv.org/abs/2602.20892)
• Keywords: language model
• Abstract: Today, Social networks such as Twitter are the most widely used platforms for communication of people. Analyzing this data has useful information to recognize the opinion of people in tweets. Sentiment analysis plays a vital role in NLP, which identifies the opinion of the individuals about a specific topic. Natural language processing in Persian has many challenges despite the adventure of strong language models. The datasets available in Persian are generally in special topics such as products, foods, hotels, etc while users may use ironies, colloquial phrases in social media To overcome these challenges, there is a necessity for having a dataset of Persian sentiment analysis on Twitter. In this paper, we introduce the Exa sentiment analysis Persian dataset, which is collected from Persian tweets. This dataset contains 12,000 tweets, annotated by 5 native Persian taggers. The aforementioned data is labeled in 3 classes: positive, neutral and negative. We present the characteristics and statistics of this dataset and use the pre-trained Pars Bert and Roberta as the base model to evaluate this dataset. Our evaluation reached a 79.87 Macro F-score, which shows the model and data can be adequately valuable for a sentiment analysis system.
• 摘要：如今，Twitter 等社交网络是人们使用最广泛的交流平台。分析这些数据可以提供有用的信息来识别推文中人们的观点。情感分析在 NLP 中起着至关重要的作用，它可以识别个人对特定主题的看法。尽管存在强大的语言模型，但波斯语的自然语言处理仍面临许多挑战。波斯语可用的数据集通常涉及产品、食品、酒店等特殊主题，而用户可能会在社交媒体中使用讽刺、口语短语。为了克服这些挑战，有必要在 Twitter 上拥有波斯语情绪分析数据集。在本文中，我们介绍了 Exa 情感分析波斯语数据集，该数据集是从波斯语推文中收集的。该数据集包含 12,000 条推文，由 5 个本地波斯语标注器注释。上述数据被标记为 3 类：正面、中性和负面。我们展示了该数据集的特征和统计数据，并使用预训练的 Pars Bert 和 Roberta 作为基础模型来评估该数据集。我们的评估达到了 79.87 的宏观 F 分数，这表明该模型和数据对于情感分析系统具有足够的价值。

Title: The Art of Efficient Reasoning: Data, Reward, and Optimization

• Authors: Taiqiang Wu, Zenan Zu, Bo Zhou, Ngai Wong
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.20945
• Pdf URL: https://arxiv.org/pdf/2602.20945
• Copy Paste: [[2602.20945]] The Art of Efficient Reasoning: Data, Reward, and Optimization(https://arxiv.org/abs/2602.20945)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: Large Language Models (LLMs) consistently benefit from scaled Chain-of-Thought (CoT) reasoning, but also suffer from heavy computational overhead. To address this issue, efficient reasoning aims to incentivize short yet accurate thinking trajectories, typically through reward shaping with Reinforcement Learning (RL). In this paper, we systematically investigate the mechanics of efficient reasoning for LLMs. For comprehensive evaluation, we advocate for more fine-grained metrics, including length distribution conditioned on correctness and performance across a wide spectrum of token budgets ranging from 2k to 32k. First, we reveal that the training process follows a two-stage paradigm: length adaptation and reasoning refinement. After that, we conduct extensive experiments (about 0.2 million GPU hours) in a unified protocol, deconstructing training prompts and rollouts, reward shaping, and optimization strategies. In particular, a key finding is to train on relatively easier prompts, ensuring the density of positive reward signals and thus avoiding the length collapse. Meanwhile, the learned length bias can be generalized across domains. We distill all findings into valuable insights and practical guidelines, and further validate them across the Qwen3 series, ranging from 0.6B to 30B, demonstrating the robustness and generalization.
• 摘要：大型语言模型 (LLM) 始终受益于规模化的思想链 (CoT) 推理，但也遭受沉重的计算开销。为了解决这个问题，有效推理的目的是激励简短而准确的思维轨迹，通常是通过强化学习（RL）的奖励塑造。在本文中，我们系统地研究了法学硕士有效推理的机制。为了进行全面评估，我们主张采用更细粒度的指标，包括以 2k 到 32k 范围内的各种代币预算的正确性和性能为条件的长度分布。首先，我们揭示训练过程遵循两阶段范式：长度适应和推理细化。之后，我们在统一的协议中进行了大量的实验（约 20 万个 GPU 小时），解构训练提示和部署、奖励塑造和优化策略。特别是，一个关键的发现是在相对容易的提示上进行训练，确保积极奖励信号的密度，从而避免长度崩溃。同时，学习到的长度偏差可以跨领域推广。我们将所有发现提炼成有价值的见解和实用指南，并在 Qwen3 系列中进一步验证它们，范围从 0.6B 到 30B，展示了稳健性和泛化性。

Title: Blackbird Language Matrices: A Framework to Investigate the Linguistic Competence of Language Models

• Authors: Paola Merlo, Chunyang Jiang, Giuseppe Samo, Vivi Nastase
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.20966
• Pdf URL: https://arxiv.org/pdf/2602.20966
• Copy Paste: [[2602.20966]] Blackbird Language Matrices: A Framework to Investigate the Linguistic Competence of Language Models(https://arxiv.org/abs/2602.20966)
• Keywords: language model, llm
• Abstract: This article describes a novel language task, the Blackbird Language Matrices (BLM) task, inspired by intelligence tests, and illustrates the BLM datasets, their construction and benchmarking, and targeted experiments on chunking and systematicity. BLMs are multiple-choice problems, structured at multiple levels: within each sentence, across the input sequence, within each candidate answer. Because of their rich structure, these curated, but naturalistic datasets are key to answer some core questions about current large language models abilities: do LLMs detect linguistic objects and their properties? Do they detect and use systematic patterns across sentences? Are they more prone to linguistic or reasoning errors, and how do these interact? We show that BLMs, while challenging, can be solved at good levels of performance, in more than one language, with simple baseline models or, at better performance levels, with more tailored models. We show that their representations contain the grammatical objects and attributes relevant to solve a linguistic task. We also show that these solutions are reached by detecting systematic patterns across sentences. The paper supports the point of view that curated, structured datasets support multi-faceted investigations of properties of language and large language models. Because they present a curated, articulated structure, because they comprise both learning contexts and expected answers, and because they are partly built by hand, BLMs fall in the category of datasets that can support explainability investigations, and be useful to ask why large language models behave the way they do.
• 摘要：本文描述了一种新颖的语言任务，即黑鸟语言矩阵（BLM）任务，其灵感来自于智力测试，并说明了 BLM 数据集、其构建和基准测试，以及关于分块和系统性的有针对性的实验。 BLM 是多项选择问题，具有多个级别的结构：在每个句子内、跨输入序列、在每个候选答案内。由于其丰富的结构，这些精心策划的自然数据集是回答有关当前大型语言模型能力的一些核心问题的关键：法学硕士是否检测语言对象及其属性？他们是否检测并使用跨句子的系统模式？他们是否更容易出现语言或推理错误？这些错误如何相互作用？我们表明，BLM 虽然具有挑战性，但可以通过简单的基线模型以多种语言以良好的性能水平来解决，或者通过更定制的模型以更好的性能水平来解决。我们证明它们的表示包含与解决语言任务相关的语法对象和属性。我们还表明，这些解决方案是通过检测句子之间的系统模式来实现的。该论文支持这样的观点：精心策划的结构化数据集支持对语言和大型语言模型属性的多方面研究。因为它们呈现出精心策划的、清晰的结构，因为它们包含学习上下文和预期答案，并且因为它们部分是手工构建的，所以 BLM 属于可以支持可解释性调查的数据集类别，并且对于询问为什么大型语言模型的行为方式很有用。

Title: Linear Reasoning vs. Proof by Cases: Obstacles for Large Language Models in FOL Problem Solving

• Authors: Yuliang Ji, Fuchen Shen, Jian Wu, Qiujie Xie, Yue Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.20973
• Pdf URL: https://arxiv.org/pdf/2602.20973
• Copy Paste: [[2602.20973]] Linear Reasoning vs. Proof by Cases: Obstacles for Large Language Models in FOL Problem Solving(https://arxiv.org/abs/2602.20973)
• Keywords: language model, llm
• Abstract: To comprehensively evaluate the mathematical reasoning capabilities of Large Language Models (LLMs), researchers have introduced abundant mathematical reasoning datasets. However, most existing datasets primarily focus on linear reasoning, neglecting other parts such as proof by contradiction and proof by cases, which are crucial for investigating LLMs' reasoning abilities. To address this limitation, we first introduce a novel first-order logic (FOL) dataset named PC-FOL, annotated by professional mathematicians, focusing on case-based reasoning problems. All instances in this dataset are equipped with a manually written natural language proof, clearly distinguishing it from conventional linear reasoning datasets. Our experimental results over leading LLMs demonstrate a substantial performance gap between linear reasoning and case-based reasoning problems. To further investigate this phenomenon, we provide a theoretical analysis grounded in graphical model, which provides an explanation for the observed disparity between the two types of reasoning problems. We hope this work can reveal the core challenges in the field of automated natural language mathematical proof generation, paving the way for future research.
• 摘要：为了全面评估大型语言模型（LLM）的数学推理能力，研究人员引入了丰富的数学推理数据集。然而，大多数现有数据集主要关注线性推理，忽略了其他部分，例如矛盾证明和案例证明，这些对于考察LLM的推理能力至关重要。为了解决这个限制，我们首先引入一个名为 PC-FOL 的新型一阶逻辑（FOL）数据集，由专业数学家注释，重点关注基于案例的推理问题。该数据集中的所有实例都配备了手动编写的自然语言证明，明显区别于传统的线性推理数据集。我们对领先法学硕士的实验结果表明，线性推理和基于案例的推理问题之间存在巨大的性能差距。为了进一步研究这种现象，我们提供了基于图模型的理论分析，它为观察到的两类推理问题之间的差异提供了解释。我们希望这项工作能够揭示自动化自然语言数学证明生成领域的核心挑战，为未来的研究铺平道路。

Title: Evaluating Proactive Risk Awareness of Large Language Models

• Authors: Xuan Luo, Yubin Chen, Zhiyu Hou, Linpu Yu, Geng Tu, Jing Li, Ruifeng Xu
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2602.20976
• Pdf URL: https://arxiv.org/pdf/2602.20976
• Copy Paste: [[2602.20976]] Evaluating Proactive Risk Awareness of Large Language Models(https://arxiv.org/abs/2602.20976)
• Keywords: language model, llm
• Abstract: As large language models (LLMs) are increasingly embedded in everyday decision-making, their safety responsibilities extend beyond reacting to explicit harmful intent toward anticipating unintended but consequential risks. In this work, we introduce a proactive risk awareness evaluation framework that measures whether LLMs can anticipate potential harms and provide warnings before damage occurs. We construct the Butterfly dataset to instantiate this framework in the environmental and ecological domain. It contains 1,094 queries that simulate ordinary solution-seeking activities whose responses may induce latent ecological impact. Through experiments across five widely used LLMs, we analyze the effects of response length, languages, and modality. Experimental results reveal consistent, significant declines in proactive awareness under length-restricted responses, cross-lingual similarities, and persistent blind spots in (multimodal) species protection. These findings highlight a critical gap between current safety alignment and the requirements of real-world ecological responsibility, underscoring the need for proactive safeguards in LLM deployment.
• 摘要：随着大型语言模型 (LLM) 越来越多地融入日常决策中，它们的安全责任不仅限于对明确的有害意图做出反应，还包括预测意外但后果严重的风险。在这项工作中，我们引入了一个主动的风险意识评估框架，该框架衡量法学硕士是否能够预测潜在的危害并在损害发生之前提供警告。我们构建了 Butterfly 数据集来在环境和生态领域实例化该框架。它包含 1,094 个查询，模拟普通的解决方案寻求活动，其响应可能会引起潜在的生态影响。通过五个广泛使用的法学硕士的实验，我们分析了响应长度、语言和模式的影响。实验结果表明，在长度受限的反应、跨语言相似性以及（多模式）物种保护中持续存在的盲点下，主动意识持续显着下降。这些发现凸显了当前安全一致性与现实世界生态责任要求之间的重大差距，强调了法学硕士部署中采取主动保障措施的必要性。

Title: Beyond the Star Rating: A Scalable Framework for Aspect-Based Sentiment Analysis Using LLMs and Text Classification

• Authors: Vishal Patil, Shree Vaishnavi Bacha, Revanth Yamani, Yidan Sun, Mayank Kejriwal
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21082
• Pdf URL: https://arxiv.org/pdf/2602.21082
• Copy Paste: [[2602.21082]] Beyond the Star Rating: A Scalable Framework for Aspect-Based Sentiment Analysis Using LLMs and Text Classification(https://arxiv.org/abs/2602.21082)
• Keywords: language model, gpt, llm, chat
• Abstract: Customer-provided reviews have become an important source of information for business owners and other customers alike. However, effectively analyzing millions of unstructured reviews remains challenging. While large language models (LLMs) show promise for natural language understanding, their application to large-scale review analysis has been limited by computational costs and scalability concerns. This study proposes a hybrid approach that uses LLMs for aspect identification while employing classic machine-learning methods for sentiment classification at scale. Using ChatGPT to analyze sampled restaurant reviews, we identified key aspects of dining experiences and developed sentiment classifiers using human-labeled reviews, which we subsequently applied to 4.7 million reviews collected over 17 years from a major online platform. Regression analysis reveals that our machine-labeled aspects significantly explain variance in overall restaurant ratings across different aspects of dining experiences, cuisines, and geographical regions. Our findings demonstrate that combining LLMs with traditional machine learning approaches can effectively automate aspect-based sentiment analysis of large-scale customer feedback, suggesting a practical framework for both researchers and practitioners in the hospitality industry and potentially, other service sectors.
• 摘要：客户提供的评论已成为企业主和其他客户的重要信息来源。然而，有效分析数百万条非结构化评论仍然具有挑战性。虽然大型语言模型 (LLM) 在自然语言理解方面展现出希望，但它们在大规模评论分析中的应用受到计算成本和可扩展性问题的限制。本研究提出了一种混合方法，使用法学硕士进行方面识别，同时采用经典的机器学习方法进行大规模情感分类。我们使用 ChatGPT 分析餐厅评论样本，确定了就餐体验的关键方面，并使用人工标记的评论开发了情感分类器，随后将其应用于 17 年来从主要在线平台收集的 470 万条评论。回归分析表明，我们的机器标记方面可以显着解释就餐体验、美食和地理区域不同方面的整体餐厅评级的差异。我们的研究结果表明，将法学硕士与传统机器学习方法相结合，可以有效地对大规模客户反馈进行基于方面的情感分析自动化，为酒店业以及其他服务行业的研究人员和从业者提供了一个实用的框架。

Title: Prompt-Level Distillation: A Non-Parametric Alternative to Model Fine-Tuning for Efficient Reasoning

• Authors: Sanket Badhe, Deep Shah
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2602.21103
• Pdf URL: https://arxiv.org/pdf/2602.21103
• Copy Paste: [[2602.21103]] Prompt-Level Distillation: A Non-Parametric Alternative to Model Fine-Tuning for Efficient Reasoning(https://arxiv.org/abs/2602.21103)
• Keywords: prompt, chain-of-thought
• Abstract: Advanced reasoning typically requires Chain-of-Thought prompting, which is accurate but incurs prohibitive latency and substantial test-time inference costs. The standard alternative, fine-tuning smaller models, often sacrifices interpretability while introducing significant resource and operational overhead. To address these limitations, we introduce Prompt-Level Distillation (PLD). We extract explicit reasoning patterns from a Teacher model and organize them into a structured list of expressive instructions for the Student model's System Prompt. Evaluated on the StereoSet and Contract-NLI datasets using Gemma-3 4B, PLD improved Macro F1 scores from 57\% to 90.0\% and 67\% to 83\% respectively, enabling this compact model to match frontier performance with negligible latency overhead. These expressive instructions render the decision-making process transparent, allowing for full human verification of logic, making this approach ideal for regulated industries such as law, finance, and content moderation, as well as high-volume use cases and edge devices.
• 摘要：高级推理通常需要思想链提示，这是准确的，但会产生令人望而却步的延迟和大量的测试时间推理成本。标准的替代方案是对较小的模型进行微调，通常会牺牲可解释性，同时引入大量的资源和运营开销。为了解决这些限制，我们引入了即时级蒸馏 (PLD)。我们从教师模型中提取明确的推理模式，并将它们组织成学生模型系统提示的表达指令的结构化列表。使用 Gemma-3 4B 对 StereoSet 和 Contract-NLI 数据集进行评估，PLD 将 Macro F1 分数分别从 57% 提高到 90.0% 和 67% 提高到 83%，使这个紧凑的模型能够以可忽略的延迟开销匹配前沿性能。这些富有表现力的指令使决策过程变得透明，允许对逻辑进行全面的人工验证，使这种方法成为法律、金融和内容审核等受监管行业以及大容量用例和边缘设备的理想选择。

Title: On Data Engineering for Scaling LLM Terminal Capabilities

• Authors: Renjie Pi, Grace Lam, Mohammad Shoeybi, Pooya Jannaty, Bryan Catanzaro, Wei Ping
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21193
• Pdf URL: https://arxiv.org/pdf/2602.21193
• Copy Paste: [[2602.21193]] On Data Engineering for Scaling LLM Terminal Capabilities(https://arxiv.org/abs/2602.21193)
• Keywords: language model, llm, long context, agent
• Abstract: Despite rapid recent progress in the terminal capabilities of large language models, the training data strategies behind state-of-the-art terminal agents remain largely undisclosed. We address this gap through a systematic study of data engineering practices for terminal agents, making two key contributions: (1) Terminal-Task-Gen, a lightweight synthetic task generation pipeline that supports seed-based and skill-based task construction, and (2) a comprehensive analysis of data and training strategies, including filtering, curriculum learning, long context training, and scaling behavior. Our pipeline yields Terminal-Corpus, a large-scale open-source dataset for terminal tasks. Using this dataset, we train Nemotron-Terminal, a family of models initialized from Qwen3(8B, 14B, 32B) that achieve substantial gains on Terminal-Bench 2.0: Nemotron-Terminal-8B improves from 2.5% to 13.0% Nemotron-Terminal-14B improves from 4.0% to 20.2%, and Nemotron-Terminal-32B improves from 3.4% to 27.4%, matching the performance of significantly larger models. To accelerate research in this domain, we open-source our model checkpoints and most of our synthetic datasets at this https URL.
• 摘要：尽管大型语言模型的终端能力最近取得了快速进展，但最先进的终端代理背后的训练数据策略在很大程度上仍未公开。我们通过对终端代理的数据工程实践进行系统研究来解决这一差距，做出了两个关键贡献：（1）Terminal-Task-Gen，一种轻量级合成任务生成管道，支持基于种子和基于技能的任务构建；（2）对数据和训练策略的全面分析，包括过滤、课程学习、长上下文训练和扩展行为。我们的管道产生了 Terminal-Corpus，这是一个用于终端任务的大规模开源数据集。使用此数据集，我们训练 Nemotron-Terminal，这是从 Qwen3（8B、14B、32B）初始化的一系列模型，在 Terminal-Bench 2.0 上取得了显着的进步：Nemotron-Terminal-8B 从 2.5% 提高到 13.0%，Nemotron-Terminal-14B 从 4.0% 提高到 20.2%，Nemotron-Terminal-32B 从 3.4% 提高达到 27.4%，与更大型号的性能相当。为了加速该领域的研究，我们在此 https URL 开源了我们的模型检查点和大部分合成数据集。