2025-12-10

Title: Short-Context Dominance: How Much Local Context Natural Language Actually Needs?

• Authors: Vala Vakilian, Zimeng Wang, Ankit Singh Rawat, Christos Thrampoulidis
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.08082
• Pdf URL: https://arxiv.org/pdf/2512.08082
• Copy Paste: [[2512.08082]] Short-Context Dominance: How Much Local Context Natural Language Actually Needs?(https://arxiv.org/abs/2512.08082)
• Keywords: language model, llm
• Abstract: We investigate the short-context dominance hypothesis: that for most sequences, a small local prefix suffices to predict their next tokens. Using large language models as statistical oracles, we measure the minimum context length (MCL) needed to reproduce accurate full-context predictions across datasets with sequences of varying lengths. For sequences with 1-7k tokens from long-context documents, we consistently find that 75-80% require only the last 96 tokens at most. Given the dominance of short-context tokens, we then ask whether it is possible to detect challenging long-context sequences for which a short local prefix does not suffice for prediction. We introduce a practical proxy to MCL, called Distributionally Aware MCL (DaMCL), that does not require knowledge of the actual next-token and is compatible with sampling strategies beyond greedy decoding. Our experiments validate that simple thresholding of the metric defining DaMCL achieves high performance in detecting long vs. short context sequences. Finally, to counter the bias that short-context dominance induces in LLM output distributions, we develop an intuitive decoding algorithm that leverages our detector to identify and boost tokens that are long-range-relevant. Across Q&A tasks and model architectures, we confirm that mitigating the bias improves performance.
• 摘要：We investigate the short-context dominance hypothesis: that for most sequences, a small local prefix suffices to predict their next tokens.使用大型语言模型作为统计预言，我们测量了在具有不同长度序列的数据集上重现准确的全上下文预测所需的最小上下文长度（MCL）。 For sequences with 1-7k tokens from long-context documents, we consistently find that 75-80% require only the last 96 tokens at most.考虑到短上下文标记的主导地位，我们然后询问是否有可能检测到具有挑战性的长上下文序列，而短的本地前缀不足以进行预测。我们引入了一种实用的 MCL 代理，称为分布式感知 MCL (DaMCL)，它不需要了解实际的下一个令牌，并且与贪婪解码之外的采样策略兼容。我们的实验验证了定义 DaMCL 的度量的简单阈值在检测长上下文序列与短上下文序列方面实现了高性能。最后，为了对抗短上下文主导在 LLM 输出分布中引起的偏差，我们开发了一种直观的解码算法，利用我们的检测器来识别和增强远程相关的标记。 Across Q&A tasks and model architectures, we confirm that mitigating the bias improves performance.

Title: Adaptation of Embedding Models to Financial Filings via LLM Distillation

• Authors: Eliot Brenner, Dominic Seyler, Manjunath Hegde, Andrei Simion, Koustuv Dasgupta, Bing Xiang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.08088
• Pdf URL: https://arxiv.org/pdf/2512.08088
• Copy Paste: [[2512.08088]] Adaptation of Embedding Models to Financial Filings via LLM Distillation(https://arxiv.org/abs/2512.08088)
• Keywords: language model, llm, agent
• Abstract: Despite advances in generative large language models (LLMs), practical application of specialized conversational AI agents remains constrained by computation costs, latency requirements, and the need for precise domain-specific relevance measures. While existing embedding models address the first two constraints, they underperform on information retrieval in specialized domains like finance. This paper introduces a scalable pipeline that trains specialized models from an unlabeled corpus using a general purpose retrieval embedding model as foundation. Our method yields an average of 27.7% improvement in MRR$\texttt{@}$5, 44.6% improvement in mean DCG$\texttt{@}$5 across 14 financial filing types measured over 21,800 query-document pairs, and improved NDCG on 3 of 4 document classes in FinanceBench. We adapt retrieval embeddings (bi-encoder) for RAG, not LLM generators, using LLM-judged relevance to distill domain knowledge into a compact retriever. There are prior works which pair synthetically generated queries with real passages to directly fine-tune the retrieval model. Our pipeline differs from these by introducing interaction between student and teacher models that interleaves retrieval-based mining of hard positive/negative examples from the unlabeled corpus with iterative retraining of the student model's weights using these examples. Each retrieval iteration uses the refined student model to mine the corpus for progressively harder training examples for the subsequent training iteration. The methodology provides a cost-effective solution to bridging the gap between general-purpose models and specialized domains without requiring labor-intensive human annotation.
• 摘要：尽管生成式大语言模型 (LLM) 取得了进步，但专用会话 AI 代理的实际应用仍然受到计算成本、延迟要求以及精确的特定领域相关性测量的需求的限制。虽然现有的嵌入模型解决了前两个限制，但它们在金融等专业领域的信息检索方面表现不佳。本文介绍了一种可扩展的管道，该管道使用通用检索嵌入模型作为基础，从未标记的语料库中训练专用模型。我们的方法在测量超过 21,800 个查询文档对的 14 种财务归档类型中，MRR$\te​​xttt{@}$5 平均提高了 27.7%，平均 DCG$\texttt{@}$5 提高了 44.6%，并改进了 FinanceBench 中 4 个文档类中的 3 个的 NDCG。我们采用 RAG 的检索嵌入（双编码器），而不是 LLM 生成器，使用 LLM 判断的相关性将领域知识提取到紧凑的检索器中。之前有一些工作将综合生成的查询与真实的段落配对，以直接微调检索模型。我们的流程与这些不同，它引入了学生模型和教师模型之间的交互，该模型将来自未标记语料库的基于检索的硬正/负示例的挖掘与使用这些示例对学生模型权重的迭代再训练交织在一起。每次检索迭代都使用精炼的学生模型来挖掘语料库，为后续训练迭代提供逐渐困难的训练示例。该方法提供了一种经济有效的解决方案，可以弥合通用模型和专业领域之间的差距，而无需劳动密集型的人工注释。

Title: Universal Adversarial Suffixes Using Calibrated Gumbel-Softmax Relaxation

• Authors: Sampriti Soor, Suklav Ghosh, Arijit Sur
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.08123
• Pdf URL: https://arxiv.org/pdf/2512.08123
• Copy Paste: [[2512.08123]] Universal Adversarial Suffixes Using Calibrated Gumbel-Softmax Relaxation(https://arxiv.org/abs/2512.08123)
• Keywords: language model, prompt
• Abstract: Language models (LMs) are often used as zero-shot or few-shot classifiers by scoring label words, but they remain fragile to adversarial prompts. Prior work typically optimizes task- or model-specific triggers, making results difficult to compare and limiting transferability. We study universal adversarial suffixes: short token sequences (4-10 tokens) that, when appended to any input, broadly reduce accuracy across tasks and models. Our approach learns the suffix in a differentiable "soft" form using Gumbel-Softmax relaxation and then discretizes it for inference. Training maximizes calibrated cross-entropy on the label region while masking gold tokens to prevent trivial leakage, with entropy regularization to avoid collapse. A single suffix trained on one model transfers effectively to others, consistently lowering both accuracy and calibrated confidence. Experiments on sentiment analysis, natural language inference, paraphrase detection, commonsense QA, and physical reasoning with Qwen2-1.5B, Phi-1.5, and TinyLlama-1.1B demonstrate consistent attack effectiveness and transfer across tasks and model families.
• 摘要：语言模型（LM）通常通过对标签词进行评分来用作零样本或少样本分类器，但它们仍然容易受到对抗性提示的影响。先前的工作通常会优化特定于任务或模型的触发器，从而使结果难以比较并限制可转移性。我们研究通用对抗性后缀：短标记序列（4-10 个标记），当附加到任何输入时，会广泛降低任务和模型的准确性。我们的方法使用 Gumbel-Softmax 松弛以可微的“软”形式学习后缀，然后将其离散化以进行推理。训练最大化标签区域上的校准交叉熵，同时屏蔽黄金标记以防止微不足道的泄漏，并通过熵正则化来避免崩溃。在一个模型上训练的单个后缀会有效地转移到其他模型上，从而持续降低准确性和校准置信度。使用 Qwen2-1.5B、Phi-1.5 和 TinyLlama-1.1B 进行的情感分析、自然语言推理、释义检测、常识 QA 和物理推理实验证明了一致的攻击有效性以及跨任务和模型系列的迁移。

Title: Universal Adversarial Suffixes for Language Models Using Reinforcement Learning with Calibrated Reward

• Authors: Sampriti Soor, Suklav Ghosh, Arijit Sur
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.08131
• Pdf URL: https://arxiv.org/pdf/2512.08131
• Copy Paste: [[2512.08131]] Universal Adversarial Suffixes for Language Models Using Reinforcement Learning with Calibrated Reward(https://arxiv.org/abs/2512.08131)
• Keywords: language model, chat
• Abstract: Language models are vulnerable to short adversarial suffixes that can reliably alter predictions. Previous works usually find such suffixes with gradient search or rule-based methods, but these are brittle and often tied to a single task or model. In this paper, a reinforcement learning framework is used where the suffix is treated as a policy and trained with Proximal Policy Optimization against a frozen model as a reward oracle. Rewards are shaped using calibrated cross-entropy, removing label bias and aggregating across surface forms to improve transferability. The proposed method is evaluated on five diverse NLP benchmark datasets, covering sentiment, natural language inference, paraphrase, and commonsense reasoning, using three distinct language models: Qwen2-1.5B Instruct, TinyLlama-1.1B Chat, and Phi-1.5. Results show that RL-trained suffixes consistently degrade accuracy and transfer more effectively across tasks and models than previous adversarial triggers of similar genres.
• 摘要：语言模型很容易受到短对抗性后缀的影响，这些后缀可以可靠地改变预测。以前的工作通常通过梯度搜索或基于规则的方法找到此类后缀，但这些后缀很脆弱，并且通常与单个任务或模型相关。在本文中，使用强化学习框架，其中后缀被视为策略，并针对作为奖励预言的冻结模型进行近端策略优化训练。奖励是使用校准的交叉熵形成的，消除了标签偏差并在表面形式上聚合以提高可转移性。该方法在五个不同的 NLP 基准数据集上进行评估，涵盖情感、自然语言推理、释义和常识推理，使用三种不同的语言模型：Qwen2-1.5B Instruct、TinyLlama-1.1B Chat 和 Phi-1.5。结果表明，强化学习训练的后缀持续降低准确性，并且比之前类似类型的对抗性触发器更有效地跨任务和模型转移。

Title: ClinicalTrialsHub: Bridging Registries and Literature for Comprehensive Clinical Trial Access

• Authors: Jiwoo Park, Ruoqi Liu, Avani Jagdale, Andrew Srisuwananukorn, Jing Zhao, Lang Li, Ping Zhang, Sachin Kumar
• Subjects: cs.CL, cs.AI, cs.HC, cs.IR
• Abstract URL: https://arxiv.org/abs/2512.08193
• Pdf URL: https://arxiv.org/pdf/2512.08193
• Copy Paste: [[2512.08193]] ClinicalTrialsHub: Bridging Registries and Literature for Comprehensive Clinical Trial Access(https://arxiv.org/abs/2512.08193)
• Keywords: language model, gpt
• Abstract: We present ClinicalTrialsHub, an interactive search-focused platform that consolidates all data from this http URL and augments it by automatically extracting and structuring trial-relevant information from PubMed research articles. Our system effectively increases access to structured clinical trial data by 83.8% compared to relying on this http URL alone, with potential to make access easier for patients, clinicians, researchers, and policymakers, advancing evidence-based medicine. ClinicalTrialsHub uses large language models such as GPT-5.1 and Gemini-3-Pro to enhance accessibility. The platform automatically parses full-text research articles to extract structured trial information, translates user queries into structured database searches, and provides an attributed question-answering system that generates evidence-grounded answers linked to specific source sentences. We demonstrate its utility through a user study involving clinicians, clinical researchers, and PhD students of pharmaceutical sciences and nursing, and a systematic automatic evaluation of its information extraction and question answering capabilities.
• 摘要：我们推出了 ClinicalTrialsHub，这是一个以搜索为中心的交互式平台，它整合了来自该 http URL 的所有数据，并通过自动从 PubMed 研究文章中提取和构建试验相关信息来增强数据。与单独依赖此 http URL 相比，我们的系统有效地将结构化临床试验数据的访问量增加了 83.8%，有可能使患者、临床医生、研究人员和政策制定者更容易访问，从而推进循证医学。 ClinicalTrialsHub 使用 GPT-5.1 和 Gemini-3-Pro 等大型语言模型来增强可访问性。该平台自动解析全文研究文章以提取结构化试验信息，将用户查询转换为结构化数据库搜索，并提供属性问答系统，生成与特定源句子相关的基于证据的答案。我们通过一项涉及临床医生、临床研究人员以及药学和护理学博士生的用户研究以及对其信息提取和问题回答能力的系统自动评估来展示其实用性。

Title: Are generative AI text annotations systematically biased?

• Authors: Sjoerd B. Stolwijk, Mark Boukes, Damian Trilling
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.08404
• Pdf URL: https://arxiv.org/pdf/2512.08404
• Copy Paste: [[2512.08404]] Are generative AI text annotations systematically biased?(https://arxiv.org/abs/2512.08404)
• Keywords: gpt, llm, prompt
• Abstract: This paper investigates bias in GLLM annotations by conceptually replicating manual annotations of Boukes (2024). Using various GLLMs (Llama3.1:8b, Llama3.3:70b, GPT4o, Qwen2.5:72b) in combination with five different prompts for five concepts (political content, interactivity, rationality, incivility, and ideology). We find GLLMs perform adequate in terms of F1 scores, but differ from manual annotations in terms of prevalence, yield substantively different downstream results, and display systematic bias in that they overlap more with each other than with manual annotations. Differences in F1 scores fail to account for the degree of bias.
• 摘要：本文通过概念性地复制 Boukes (2024) 的手动注释来研究 GLLM 注释中的偏差。使用各种 GLLM（Llama3.1:8b、Llama3.3:70b、GPT4o、Qwen2.5:72b）结合针对五个概念（政治内容、交互性、理性、不文明和意识形态）的五个不同提示。我们发现 GLLM 在 F1 分数方面表现良好，但在流行程度方面与手动注释不同，产生实质性不同的下游结果，并且表现出系统偏差，因为它们彼此重叠比手动注释更多。 F1 分数的差异无法解释偏差程度。

Title: What Triggers my Model? Contrastive Explanations Inform Gender Choices by Translation Models

• Authors: Janiça Hackenbuchner, Arda Tezcan, Joke Daems
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.08440
• Pdf URL: https://arxiv.org/pdf/2512.08440
• Copy Paste: [[2512.08440]] What Triggers my Model? Contrastive Explanations Inform Gender Choices by Translation Models(https://arxiv.org/abs/2512.08440)
• Keywords: language model, llm
• Abstract: Interpretability can be implemented as a means to understand decisions taken by (black box) models, such as machine translation (MT) or large language models (LLMs). Yet, research in this area has been limited in relation to a manifested problem in these models: gender bias. With this research, we aim to move away from simply measuring bias to exploring its origins. Working with gender-ambiguous natural source data, this study examines which context, in the form of input tokens in the source sentence, influences (or triggers) the translation model choice of a certain gender inflection in the target language. To analyse this, we use contrastive explanations and compute saliency attribution. We first address the challenge of a lacking scoring threshold and specifically examine different attribution levels of source words on the model gender decisions in the translation. We compare salient source words with human perceptions of gender and demonstrate a noticeable overlap between human perceptions and model attribution. Additionally, we provide a linguistic analysis of salient words. Our work showcases the relevance of understanding model translation decisions in terms of gender, how this compares to human decisions and that this information should be leveraged to mitigate gender bias.
• 摘要：可解释性可以作为理解（黑盒）模型（例如机器翻译（MT）或大型语言模型（LLM））所做出的决策的一种手段来实现。然而，由于这些模型中明显的问题：性别偏见，该领域的研究受到限制。通过这项研究，我们的目标是从简单地衡量偏见转向探索其根源。本研究使用性别模糊的自然源数据，检查哪些上下文以源句子中的输入标记的形式影响（或触发）目标语言中某种性别变化的翻译模型选择。为了分析这一点，我们使用对比解释并计算显着性归因。我们首先解决缺乏评分阈值的挑战，并专门检查翻译中模型性别决策中源词的不同归因水平。我们将显着的源词与人类对性别的看法进行比较，并证明人类看法和模型归因之间存在明显的重叠。此外，我们还提供了显着单词的语言分析。我们的工作展示了理解模型翻译决策在性别方面的相关性、这与人类决策的比较，以及应该利用这些信息来减轻性别偏见。

Title: Soft Inductive Bias Approach via Explicit Reasoning Perspectives in Inappropriate Utterance Detection Using Large Language Models

• Authors: Ju-Young Kim, Ji-Hong Park, Se-Yeon Lee, Sujin Park, Gun-Woo Kim
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.08480
• Pdf URL: https://arxiv.org/pdf/2512.08480
• Copy Paste: [[2512.08480]] Soft Inductive Bias Approach via Explicit Reasoning Perspectives in Inappropriate Utterance Detection Using Large Language Models(https://arxiv.org/abs/2512.08480)
• Keywords: language model, chain-of-thought
• Abstract: Recent incidents in certain online games and communities, where anonymity is guaranteed, show that unchecked inappropriate remarks frequently escalate into verbal abuse and even criminal behavior, raising significant social concerns. Consequently, there is a growing need for research on techniques that can detect inappropriate utterances within conversational texts to help build a safer communication environment. Although large-scale language models trained on Korean corpora and chain-of-thought reasoning have recently gained attention, research applying these approaches to inappropriate utterance detection remains limited. In this study, we propose a soft inductive bias approach that explicitly defines reasoning perspectives to guide the inference process, thereby promoting rational decision-making and preventing errors that may arise during reasoning. We fine-tune a Korean large language model using the proposed method and conduct both quantitative performance comparisons and qualitative evaluations across different training strategies. Experimental results show that the Kanana-1.5 model achieves an average accuracy of 87.0046, improving by approximately 3.89 percent over standard supervised learning. These findings indicate that the proposed method goes beyond simple knowledge imitation by large language models and enables more precise and consistent judgments through constrained reasoning perspectives, demonstrating its effectiveness for inappropriate utterance detection.
• 摘要：近期，一些网络游戏和匿名社区发生的事件表明，不当言论若不加以遏制，往往会升级为谩骂甚至犯罪行为，引发社会严重关切。因此，越来越需要研究能够检测会话文本中不当话语的技术，以帮助建立更安全的通信环境。尽管基于韩语语料库和思维链推理训练的大规模语言模型最近引起了人们的关注，但将这些方法应用于不当话语检测的研究仍然有限。在本研究中，我们提出了一种软归纳偏差方法，明确定义推理视角来指导推理过程，从而促进理性决策并防止推理过程中可能出现的错误。我们使用所提出的方法对韩语大语言模型进行微调，并对不同的训练策略进行定量性能比较和定性评估。实验结果表明，Kanana-1.5模型的平均准确率达到87.0046，比标准监督学习提高了约3.89%。这些发现表明，所提出的方法超越了大型语言模型的简单知识模仿，并通过约束推理视角实现了更精确和一致的判断，证明了其对不当话语检测的有效性。

Title: Curriculum Guided Massive Multi Agent System Solving For Robust Long Horizon Tasks

• Authors: Indrajit Kar, Kalathur Chenchu Kishore Kumar
• Subjects: cs.CL, cs.AI, cs.CV, cs.MA
• Abstract URL: https://arxiv.org/abs/2512.08545
• Pdf URL: https://arxiv.org/pdf/2512.08545
• Copy Paste: [[2512.08545]] Curriculum Guided Massive Multi Agent System Solving For Robust Long Horizon Tasks(https://arxiv.org/abs/2512.08545)
• Keywords: language model, agent
• Abstract: Large Language Models and multi-agent systems have shown promise in decomposing complex tasks, yet they struggle with long-horizon reasoning tasks and escalating computation cost. This work introduces a hierarchical multi-agent architecture that distributes reasoning across a 64*64 grid of lightweight agents, supported by a selective oracle. A spatial curriculum progressively expands the operational region of the grid, ensuring that agents master easier central tasks before tackling harder peripheral ones. To improve reliability, the system integrates Negative Log-Likelihood as a measure of confidence, allowing the curriculum to prioritize regions where agents are both accurate and well calibrated. A Thompson Sampling curriculum manager adaptively chooses training zones based on competence and NLL-driven reward signals. We evaluate the approach on a spatially grounded Tower of Hanoi benchmark, which mirrors the long-horizon structure of many robotic manipulation and planning tasks. Results demonstrate improved stability, reduced oracle usage, and stronger long-range reasoning from distributed agent cooperation.
• 摘要：大型语言模型和多智能体系统在分解复杂任务方面表现出了良好的前景，但它们却难以应对长期推理任务和不断增加的计算成本。这项工作引入了一种分层多代理架构，该架构在 64*64 的轻量级代理网格中分布推理，并由选择性预言机支持。空间课程逐步扩大网格的操作区域，确保代理在处理更困难的外围任务之前掌握更容易的中心任务。为了提高可靠性，该系统集成了负对数似然作为置信度衡量标准，允许课程优先考虑智能体准确且校准良好的区域。 Thompson Sampling 课程经理根据能力和 NLL 驱动的奖励信号自适应地选择培训区域。我们在空间接地的河内塔基准上评估了该方法，该基准反映了许多机器人操作和规划任务的长期结构。结果表明，分布式代理合作提高了稳定性，减少了预言机的使用，并增强了远程推理能力。

Title: HealthcareNLP: where are we and what is next?

• Authors: Lifeng Han, Paul Rayson, Suzan Verberne, Andrew Moore, Goran Nenadic
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.08617
• Pdf URL: https://arxiv.org/pdf/2512.08617
• Copy Paste: [[2512.08617]] HealthcareNLP: where are we and what is next?(https://arxiv.org/abs/2512.08617)
• Keywords: llm, retrieval augmented generation
• Abstract: This proposed tutorial focuses on Healthcare Domain Applications of NLP, what we have achieved around HealthcareNLP, and the challenges that lie ahead for the future. Existing reviews in this domain either overlook some important tasks, such as synthetic data generation for addressing privacy concerns, or explainable clinical NLP for improved integration and implementation, or fail to mention important methodologies, including retrieval augmented generation and the neural symbolic integration of LLMs and KGs. In light of this, the goal of this tutorial is to provide an introductory overview of the most important sub-areas of a patient- and resource-oriented HealthcareNLP, with three layers of hierarchy: data/resource layer: annotation guidelines, ethical approvals, governance, synthetic data; NLP-Eval layer: NLP tasks such as NER, RE, sentiment analysis, and linking/coding with categorised methods, leading to explainable HealthAI; patients layer: Patient Public Involvement and Engagement (PPIE), health literacy, translation, simplification, and summarisation (also NLP tasks), and shared decision-making support. A hands-on session will be included in the tutorial for the audience to use HealthcareNLP applications. The target audience includes NLP practitioners in the healthcare application domain, NLP researchers who are interested in domain applications, healthcare researchers, and students from NLP fields. The type of tutorial is "Introductory to CL/NLP topics (HealthcareNLP)" and the audience does not need prior knowledge to attend this. Tutorial materials: this https URL
• 摘要：本教程重点介绍 NLP 的医疗保健领域应用、我们在 HealthcareNLP 方面取得的成就以及未来面临的挑战。该领域的现有评论要么忽视了一些重要的任务，例如用于解决隐私问题的合成数据生成，或用于改进集成和实施的可解释的临床 NLP，要么没有提及重要的方法，包括检索增强生成以及法学硕士和知识图谱的神经符号集成。有鉴于此，本教程的目标是提供面向患者和资源的 HealthcareNLP 最重要子领域的介绍性概述，具有三层层次结构：数据/资源层：注释指南、伦理批准、治理、合成数据； NLP-Eval 层：NLP 任务，例如 NER、RE、情感分析以及使用分类方法进行链接/编码，从而产生可解释的 HealthAI；患者层：患者公众参与（PPIE）、健康素养、翻译、简化和总结（也是 NLP 任务）以及共享决策支持。教程中将包含一个实践课程，供观众使用 HealthcareNLP 应用程序。目标受众包括医疗保健应用领域的NLP从业者、对领域应用感兴趣的NLP研究人员、医疗保健研究人员以及NLP领域的学生。教程类型为“CL/NLP 主题入门（HealthcareNLP）”，观众无需具备任何先验知识即可参加。教程材料：这个https URL

Title: QSTN: A Modular Framework for Robust Questionnaire Inference with Large Language Models

• Authors: Maximilian Kreutner, Jens Rupprecht, Georg Ahnert, Ahmed Salem, Markus Strohmaier
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2512.08646
• Pdf URL: https://arxiv.org/pdf/2512.08646
• Copy Paste: [[2512.08646]] QSTN: A Modular Framework for Robust Questionnaire Inference with Large Language Models(https://arxiv.org/abs/2512.08646)
• Keywords: language model, llm, prompt
• Abstract: We introduce QSTN, an open-source Python framework for systematically generating responses from questionnaire-style prompts to support in-silico surveys and annotation tasks with large language models (LLMs). QSTN enables robust evaluation of questionnaire presentation, prompt perturbations, and response generation methods. Our extensive evaluation ($>40 $ million survey responses) shows that question structure and response generation methods have a significant impact on the alignment of generated survey responses with human answers, and can be obtained for a fraction of the compute cost. In addition, we offer a no-code user interface that allows researchers to set up robust experiments with LLMs without coding knowledge. We hope that QSTN will support the reproducibility and reliability of LLM-based research in the future.
• 摘要：我们引入了 QSTN，这是一个开源 Python 框架，用于系统地根据问卷式提示生成响应，以支持使用大型语言模型 (LLM) 的计算机调查和注释任务。 QSTN 能够对问卷呈现、提示扰动和响应生成方法进行稳健评估。我们的广泛评估（超过 40 亿美元的调查回复）表明，问题结构和回复生成方法对生成的调查回复与人类答案的一致性具有重大影响，并且只需一小部分计算成本即可获得。此外，我们还提供无代码用户界面，允许研究人员在没有编码知识的情况下使用法学硕士建立稳健的实验。我们希望 QSTN 能够支持未来基于 LLM 的研究的可重复性和可靠性。

Title: An Agentic AI System for Multi-Framework Communication Coding

• Authors: Bohao Yang, Rui Yang, Joshua M. Biro, Haoyuan Wang, Jessica L. Handley, Brianna Richardson, Sophia Bessias, Nicoleta Economou-Zavlanos, Armando D. Bedoya, Monica Agrawal, Michael M. Zavlanos, Anand Chowdhury, Raj M. Ratwani, Kai Sun, Kathryn I. Pollak, Michael J. Pencina, Chuan Hong
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.08659
• Pdf URL: https://arxiv.org/pdf/2512.08659
• Copy Paste: [[2512.08659]] An Agentic AI System for Multi-Framework Communication Coding(https://arxiv.org/abs/2512.08659)
• Keywords: language model, prompt, retrieval-augmented generation, agent
• Abstract: Clinical communication is central to patient outcomes, yet large-scale human annotation of patient-provider conversation remains labor-intensive, inconsistent, and difficult to scale. Existing approaches based on large language models typically rely on single-task models that lack adaptability, interpretability, and reliability, especially when applied across various communication frameworks and clinical domains. In this study, we developed a Multi-framework Structured Agentic AI system for Clinical Communication (MOSAIC), built on a LangGraph-based architecture that orchestrates four core agents, including a Plan Agent for codebook selection and workflow planning, an Update Agent for maintaining up-to-date retrieval databases, a set of Annotation Agents that applies codebook-guided retrieval-augmented generation (RAG) with dynamic few-shot prompting, and a Verification Agent that provides consistency checks and feedback. To evaluate performance, we compared MOSAIC outputs against gold-standard annotations created by trained human coders. We developed and evaluated MOSAIC using 26 gold standard annotated transcripts for training and 50 transcripts for testing, spanning rheumatology and OB/GYN domains. On the test set, MOSAIC achieved an overall F1 score of 0.928. Performance was highest in the Rheumatology subset (F1 = 0.962) and strongest for Patient Behavior (e.g., patients asking questions, expressing preferences, or showing assertiveness). Ablations revealed that MOSAIC outperforms baseline benchmarking.
• 摘要：临床沟通对于患者治疗结果至关重要，但对患者与提供者对话的大规模人工注释仍然是劳动密集型、不一致且难以扩展。基于大型语言模型的现有方法通常依赖于缺乏适应性、可解释性和可靠性的单任务模型，特别是在跨各种通信框架和临床领域应用时。在本研究中，我们开发了一种用于临床沟通的多框架结构化代理人工智能系统（MOSAIC），该系统建立在基于 LangGraph 的架构之上，协调四个核心代理，包括用于码本选择和工作流程规划的计划代理、用于维护最新检索数据库的更新代理、一组应用具有动态少样本提示的码本引导检索增强生成（RAG）的注释代理，以及提供一致性检查和反馈的验证代理。为了评估性能，我们将 MOSAIC 输出与经过培训的人类编码员创建的黄金标准注释进行了比较。我们使用 26 个用于训练的黄金标准注释转录本和 50 个用于测试的转录本来开发和评估 MOSAIC，涵盖风湿病学和妇产科领域。在测试集上，MOSAIC 的总体 F1 得分为 0.928。风湿病学子集的表现最高 (F1 = 0.962)，患者行为表现最强（例如，患者提出问题、表达偏好或表现出自信）。 Ablations 显示 MOSAIC 的性能优于基线基准测试。

Title: Automatic Essay Scoring and Feedback Generation in Basque Language Learning

• Authors: Ekhi Azurmendi, Xabier Arregi, Oier Lopez de Lacalle
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.08713
• Pdf URL: https://arxiv.org/pdf/2512.08713
• Copy Paste: [[2512.08713]] Automatic Essay Scoring and Feedback Generation in Basque Language Learning(https://arxiv.org/abs/2512.08713)
• Keywords: gpt
• Abstract: This paper introduces the first publicly available dataset for Automatic Essay Scoring (AES) and feedback generation in Basque, targeting the CEFR C1 proficiency level. The dataset comprises 3,200 essays from HABE, each annotated by expert evaluators with criterion specific scores covering correctness, richness, coherence, cohesion, and task alignment enriched with detailed feedback and error examples. We fine-tune open-source models, including RoBERTa-EusCrawl and Latxa 8B/70B, for both scoring and explanation generation. Our experiments show that encoder models remain highly reliable for AES, while supervised fine-tuning (SFT) of Latxa significantly enhances performance, surpassing state-of-the-art (SoTA) closed-source systems such as GPT-5 and Claude Sonnet 4.5 in scoring consistency and feedback quality. We also propose a novel evaluation methodology for assessing feedback generation, combining automatic consistency metrics with expert-based validation of extracted learner errors. Results demonstrate that the fine-tuned Latxa model produces criterion-aligned, pedagogically meaningful feedback and identifies a wider range of error types than proprietary models. This resource and benchmark establish a foundation for transparent, reproducible, and educationally grounded NLP research in low-resource languages such as Basque.
• 摘要：本文介绍了第一个公开可用的巴斯克语论文自动评分 (AES) 和反馈生成数据集，目标是 CEFR C1 熟练程度。该数据集包含来自 HABE 的 3,200 篇文章，每篇文章都由专家评估员注释，并根据具体标准评分，涵盖正确性、丰富性、连贯性、凝聚力和任务一致性，并附有详细的反馈和错误示例。我们微调开源模型，包括 RoBERTa-EusCrawl 和 Latxa 8B/70B，用于评分和解释生成。我们的实验表明，编码器模型对于 AES 仍然高度可靠，而 Latxa 的监督微调 (SFT) 显着提高了性能，在评分一致性和反馈质量方面超越了 GPT-5 和 Claude Sonnet 4.5 等最先进的 (SoTA) 闭源系统。我们还提出了一种新的评估方法来评估反馈的生成，将自动一致性指标与基于专家的对提取的学习者错误的验证相结合。结果表明，经过微调的 Latxa 模型可产生符合标准、具有教学意义的反馈，并比专有模型识别更广泛的错误类型。该资源和基准为巴斯克语等资源匮乏语言的透明、可重复且以教育为基础的 NLP 研究奠定了基础。

Title: Fluent Alignment with Disfluent Judges: Post-training for Lower-resource Languages

• Authors: David Samuel, Lilja Øvrelid, Erik Velldal, Andrey Kutuzov
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.08777
• Pdf URL: https://arxiv.org/pdf/2512.08777
• Copy Paste: [[2512.08777]] Fluent Alignment with Disfluent Judges: Post-training for Lower-resource Languages(https://arxiv.org/abs/2512.08777)
• Keywords: language model
• Abstract: We propose a post-training method for lower-resource languages that preserves fluency of language models even when aligned by disfluent reward models. Preference-optimization is now a well-researched topic, but previous work has mostly addressed models for English and Chinese. Lower-resource languages lack both datasets written by native speakers and language models capable of generating fluent synthetic data. Thus, in this work, we focus on developing a fluent preference-aligned language model without any instruction-tuning data in the target language. Our approach uses an on-policy training method, which we compare with two common approaches: supervised finetuning on machine-translated data and multilingual finetuning. We conduct a case study on Norwegian Bokmål and evaluate fluency through native-speaker assessments. The results show that the on-policy aspect is crucial and outperforms the alternatives without relying on any hard-to-obtain data.
• 摘要：我们提出了一种针对资源较低的语言的后训练方法，即使在通过不流利的奖励模型进行对齐时，也能保持语言模型的流畅性。偏好优化现在是一个深入研究的话题，但之前的工作主要涉及英语和汉语模型。资源较低的语言缺乏由母语人士编写的数据集和能够生成流畅的合成数据的语言模型。因此，在这项工作中，我们专注于开发一个流畅的偏好对齐语言模型，而无需目标语言中的任何指令调整数据。我们的方法使用策略训练方法，我们将其与两种常见方法进行比较：机器翻译数据的监督微调和多语言微调。我们对挪威语博克马尔语进行了案例研究，并通过母语人士评估来评估流利程度。结果表明，在政策方面至关重要，并且在不依赖任何难以获得的数据的情况下优于其他选择。

Title: A Systematic Evaluation of Preference Aggregation in Federated RLHF for Pluralistic Alignment of LLMs

• Authors: Mahmoud Srewa, Tianyu Zhao, Salma Elmalaki
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.08786
• Pdf URL: https://arxiv.org/pdf/2512.08786
• Copy Paste: [[2512.08786]] A Systematic Evaluation of Preference Aggregation in Federated RLHF for Pluralistic Alignment of LLMs(https://arxiv.org/abs/2512.08786)
• Keywords: language model, llm
• Abstract: This paper addresses the challenge of aligning large language models (LLMs) with diverse human preferences within federated learning (FL) environments, where standard methods often fail to adequately represent diverse viewpoints. We introduce a comprehensive evaluation framework that systematically assesses the trade-off between alignment quality and fairness when using different aggregation strategies for human preferences. In our federated setting, each group locally evaluates rollouts and produces reward signals, and the server aggregates these group-level rewards without accessing any raw data. Specifically, we evaluate standard reward aggregation techniques (min, max, and average) and introduce a novel adaptive scheme that dynamically adjusts preference weights based on a group's historical alignment performance. Our experiments on question-answering (Q/A) tasks using a PPO-based RLHF pipeline demonstrate that our adaptive approach consistently achieves superior fairness while maintaining competitive alignment scores. This work offers a robust methodology for evaluating LLM behavior across diverse populations and provides a practical solution for developing truly pluralistic and fairly aligned models.
• 摘要：本文解决了在联邦学习 (FL) 环境中使大型语言模型 (LLM) 与不同人类偏好保持一致的挑战，其中标准方法通常无法充分代表不同的观点。我们引入了一个全面的评估框架，该框架可以系统地评估在针对人类偏好使用不同聚合策略时对齐质量和公平性之间的权衡。在我们的联合设置中，每个组在本地评估部署并产生奖励信号，并且服务器在不访问任何原始数据的情况下聚合这些组级奖励。具体来说，我们评估标准奖励聚合技术（最小值、最大值和平均值），并引入一种新颖的自适应方案，该方案根据群体的历史一致性表现动态调整偏好权重。我们使用基于 PPO 的 RLHF 管道对问答 (Q/A) 任务进行的实验表明，我们的自适应方法始终能够实现卓越的公平性，同时保持有竞争力的对齐分数。这项工作提供了一种强大的方法来评估不同人群的法学硕士行为，并为开发真正多元化和公平一致的模型提供了实用的解决方案。

Title: Ask, Answer, and Detect: Role-Playing LLMs for Personality Detection with Question-Conditioned Mixture-of-Experts

• Authors: Yifan Lyu, Liang Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.08814
• Pdf URL: https://arxiv.org/pdf/2512.08814
• Copy Paste: [[2512.08814]] Ask, Answer, and Detect: Role-Playing LLMs for Personality Detection with Question-Conditioned Mixture-of-Experts(https://arxiv.org/abs/2512.08814)
• Keywords: language model, llm
• Abstract: Understanding human personality is crucial for web applications such as personalized recommendation and mental health assessment. Existing studies on personality detection predominantly adopt a "posts -> user vector -> labels" modeling paradigm, which encodes social media posts into user representations for predicting personality labels (e.g., MBTI labels). While recent advances in large language models (LLMs) have improved text encoding capacities, these approaches remain constrained by limited supervision signals due to label scarcity, and under-specified semantic mappings between user language and abstract psychological constructs. We address these challenges by proposing ROME, a novel framework that explicitly injects psychological knowledge into personality detection. Inspired by standardized self-assessment tests, ROME leverages LLMs' role-play capability to simulate user responses to validated psychometric questionnaires. These generated question-level answers transform free-form user posts into interpretable, questionnaire-grounded evidence linking linguistic cues to personality labels, thereby providing rich intermediate supervision to mitigate label scarcity while offering a semantic reasoning chain that guides and simplifies the text-to-personality mapping learning. A question-conditioned Mixture-of-Experts module then jointly routes over post and question representations, learning to answer questionnaire items under explicit supervision. The predicted answers are summarized into an interpretable answer vector and fused with the user representation for final prediction within a multi-task learning framework, where question answering serves as a powerful auxiliary task for personality detection. Extensive experiments on two real-world datasets demonstrate that ROME consistently outperforms state-of-the-art baselines, achieving improvements (15.41% on Kaggle dataset).
• 摘要：了解人的性格对于个性化推荐和心理健康评估等网络应用至关重要。现有的性格检测研究主要采用“帖子 -> 用户向量 -> 标签”建模范式，将社交媒体帖子编码为用户表示以预测个性标签（例如 MBTI 标签）。虽然大型语言模型（LLM）的最新进展提高了文本编码能力，但由于标签稀缺以及用户语言和抽象心理结构之间的语义映射不明确，这些方法仍然受到监督信号有限的限制。我们通过提出 ROME 来应对这些挑战，这是一个新颖的框架，明确地将心理学知识注入到人格检测中。受标准化自我评估测试的启发，ROME 利用法学硕士的角色扮演能力来模拟用户对经过验证的心理测量问卷的反应。这些生成的问题级答案将自由形式的用户帖子转换为可解释的、基于问卷的证据，将语言线索与个性标签联系起来，从而提供丰富的中间监督以减轻标签稀缺性，同时提供语义推理链来指导和简化文本到个性映射学习。然后，以问题为条件的专家混合模块联合路由帖子和问题表示，学习在明确的监督下回答问卷项目。预测的答案被总结为可解释的答案向量，并与用户表示融合，以在多任务学习框架内进行最终预测，其中问答作为人格检测的强大辅助任务。对两个真实世界数据集的大量实验表明，ROME 始终优于最先进的基线，实现了改进（在 Kaggle 数据集上为 15.41%）。

Title: Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoders

• Authors: Guangzhi Xiong, Zhenghao He, Bohan Liu, Sanchit Sinha, Aidong Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.08892
• Pdf URL: https://arxiv.org/pdf/2512.08892
• Copy Paste: [[2512.08892]] Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoders(https://arxiv.org/abs/2512.08892)
• Keywords: language model, llm, hallucination, retrieval-augmented generation
• Abstract: Retrieval-Augmented Generation (RAG) improves the factuality of large language models (LLMs) by grounding outputs in retrieved evidence, but faithfulness failures, where generations contradict or extend beyond the provided sources, remain a critical challenge. Existing hallucination detection methods for RAG often rely either on large-scale detector training, which requires substantial annotated data, or on querying external LLM judges, which leads to high inference costs. Although some approaches attempt to leverage internal representations of LLMs for hallucination detection, their accuracy remains limited. Motivated by recent advances in mechanistic interpretability, we employ sparse autoencoders (SAEs) to disentangle internal activations, successfully identifying features that are specifically triggered during RAG hallucinations. Building on a systematic pipeline of information-based feature selection and additive feature modeling, we introduce RAGLens, a lightweight hallucination detector that accurately flags unfaithful RAG outputs using LLM internal representations. RAGLens not only achieves superior detection performance compared to existing methods, but also provides interpretable rationales for its decisions, enabling effective post-hoc mitigation of unfaithful RAG. Finally, we justify our design choices and reveal new insights into the distribution of hallucination-related signals within LLMs. The code is available at this https URL.
• 摘要：检索增强生成（RAG）通过将输出基于检索到的证据来提高大型语言模型（LLM）的真实性，但忠实性失败（即世代相矛盾或超出所提供的来源）仍然是一个严峻的挑战。现有的 RAG 幻觉检测方法通常依赖于大规模检测器训练（需要大量注释数据）或依赖于查询外部 LLM 法官（这会导致较高的推理成本）。尽管一些方法试图利用法学硕士的内部表征来进行幻觉检测，但它们的准确性仍然有限。受机械可解释性最新进展的推动，我们采用稀疏自动编码器 (SAE) 来解开内部激活，成功识别在 RAG 幻觉期间专门触发的特征。基于基于信息的特征选择和附加特征建模的系统管道，我们引入了 RAGLens，这是一种轻量级幻觉检测器，可使用 LLM 内部表示准确标记不忠实的 RAG 输出。与现有方法相比，RAGLens 不仅实现了卓越的检测性能，而且还为其决策提供了可解释的基本原理，从而能够有效地事后缓解不忠实的 RAG。最后，我们证明了我们的设计选择的合理性，并揭示了对法学硕士内幻觉相关信号分布的新见解。该代码可从此 https URL 获取。