2025-12-08

Title: Fine-Tuning BERT for Domain-Specific Question Answering: Toward Educational NLP Resources at University Scale

• Authors: Aurélie Montfrond
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.05179
• Pdf URL: https://arxiv.org/pdf/2512.05179
• Copy Paste: [[2512.05179]] Fine-Tuning BERT for Domain-Specific Question Answering: Toward Educational NLP Resources at University Scale(https://arxiv.org/abs/2512.05179)
• Keywords: chat
• Abstract: Prior work on scientific question answering has largely emphasized chatbot-style systems, with limited exploration of fine-tuning foundation models for domain-specific reasoning. In this study, we developed a chatbot for the University of Limerick's Department of Electronic and Computer Engineering to provide course information to students. A custom dataset of 1,203 question-answer pairs in SQuAD format was constructed using the university book of modules, supplemented with manually and synthetically generated entries. We fine-tuned BERT (Devlin et al., 2019) using PyTorch and evaluated performance with Exact Match and F1 scores. Results show that even modest fine-tuning improves hypothesis framing and knowledge extraction, demonstrating the feasibility of adapting foundation models to educational domains. While domain-specific BERT variants such as BioBERT and SciBERT exist for biomedical and scientific literature, no foundation model has yet been tailored to university course materials. Our work addresses this gap by showing that fine-tuning BERT with academic QA pairs yields effective results, highlighting the potential to scale towards the first domain-specific QA model for universities and enabling autonomous educational knowledge systems.
• 摘要：先前有关科学问答的工作主要强调聊天机器人式系统，对特定领域推理的微调基础模型的探索有限。在这项研究中，我们为利默里克大学电子与计算机工程系开发了一个聊天机器人，为学生提供课程信息。使用大学模块手册构建了 SQuAD 格式的 1,203 个问答对的自定义数据集，并辅以手动和综合生成的条目。我们使用 PyTorch 微调 BERT（Devlin 等人，2019），并使用精确匹配和 F1 分数评估性能。结果表明，即使是适度的微调也能改善假设框架和知识提取，证明将基础模型应用于教育领域的可行性。虽然生物医学和科学文献中存在特定领域的 BERT 变体（例如 BioBERT 和 SciBERT），但尚未针对大学课程材料定制基础模型。我们的工作通过证明用学术 QA 对微调 BERT 可以产生有效的结果来解决这一差距，强调了扩展到大学第一个特定领域 QA 模型并实现自主教育知识系统的潜力。

Title: Decoding the Black Box: Discerning AI Rhetorics About and Through Poetic Prompting

• Authors: P.D. Edgar, Alia Hall
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2512.05243
• Pdf URL: https://arxiv.org/pdf/2512.05243
• Copy Paste: [[2512.05243]] Decoding the Black Box: Discerning AI Rhetorics About and Through Poetic Prompting(https://arxiv.org/abs/2512.05243)
• Keywords: language model, llm, prompt
• Abstract: Prompt engineering has emerged as a useful way studying the algorithmic tendencies and biases of large language models. Meanwhile creatives and academics have leveraged LLMs to develop creative works and explore the boundaries of their writing capabilities through text generation and code. This study suggests that creative text prompting, specifically Poetry Prompt Patterns, may be a useful addition to the toolbox of the prompt engineer, and outlines the process by which this approach may be taken. Then, the paper uses poetic prompts to assess descriptions and evaluations of three models of a renowned poet and test the consequences of the willingness of models to adapt or rewrite original creative works for presumed audiences.
• 摘要：即时工程已成为研究大型语言模型的算法趋势和偏差的有效方法。与此同时，创意人士和学者利用法学硕士开发创意作品，并通过文本生成和代码探索其写作能力的界限。这项研究表明，创造性的文本提示，特别是诗歌提示模式，可能是提示工程师工具箱的有用补充，并概述了可以采用这种方法的过程。然后，本文使用诗歌提示来评估对一位著名诗人的三个模型的描述和评价，并测试模型为假定的受众改编或重写原创作品的意愿的后果。

Title: Enhancing Clinical Note Generation with ICD-10, Clinical Ontology Knowledge Graphs, and Chain-of-Thought Prompting Using GPT-4

• Authors: Ivan Makohon, Mohamad Najafi, Jian Wu, Mathias Brochhausen, Yaohang Li
• Subjects: cs.CL, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2512.05256
• Pdf URL: https://arxiv.org/pdf/2512.05256
• Copy Paste: [[2512.05256]] Enhancing Clinical Note Generation with ICD-10, Clinical Ontology Knowledge Graphs, and Chain-of-Thought Prompting Using GPT-4(https://arxiv.org/abs/2512.05256)
• Keywords: language model, gpt, llm, prompt, chain-of-thought
• Abstract: In the past decade a surge in the amount of electronic health record (EHR) data in the United States, attributed to a favorable policy environment created by the Health Information Technology for Economic and Clinical Health (HITECH) Act of 2009 and the 21st Century Cures Act of 2016. Clinical notes for patients' assessments, diagnoses, and treatments are captured in these EHRs in free-form text by physicians, who spend a considerable amount of time entering and editing them. Manually writing clinical notes takes a considerable amount of a doctor's valuable time, increasing the patient's waiting time and possibly delaying diagnoses. Large language models (LLMs) possess the ability to generate news articles that closely resemble human-written ones. We investigate the usage of Chain-of-Thought (CoT) prompt engineering to improve the LLM's response in clinical note generation. In our prompts, we use as input International Classification of Diseases (ICD) codes and basic patient information. We investigate a strategy that combines the traditional CoT with semantic search results to improve the quality of generated clinical notes. Additionally, we infuse a knowledge graph (KG) built from clinical ontology to further enrich the domain-specific knowledge of generated clinical notes. We test our prompting technique on six clinical cases from the CodiEsp test dataset using GPT-4 and our results show that it outperformed the clinical notes generated by standard one-shot prompts.
• 摘要：过去十年，美国电子健康记录 (EHR) 数据量激增，这得益于 2009 年《经济和临床健康健康信息技术 (HITECH) 法案》和 2016 年《21 世纪治愈法案》创造的有利政策环境。这些 EHR 中的患者评估、诊断和治疗的临床记录由医生以自由格式文本形式记录，医生花费大量时间输入和编辑这些数据。手动撰写临床记录会占用医生大量的宝贵时间，增加患者的等待时间，并可能延误诊断。大型语言模型（LLM）能够生成与人类撰写的新闻文章非常相似的新闻文章。我们研究了思想链 (CoT) 提示工程的使用，以提高法学硕士在临床记录生成方面的响应。在我们的提示中，我们使用国际疾病分类 (ICD) 代码和基本患者信息作为输入。我们研究了一种将传统 CoT 与语义搜索结果相结合的策略，以提高生成的临床记录的质量。此外，我们注入了根据临床本体构建的知识图（KG），以进一步丰富生成的临床记录的特定领域知识。我们使用 GPT-4 在 CodiEsp 测试数据集中的 6 个临床病例上测试了我们的提示技术，结果表明它优于标准一次性提示生成的临床记录。

Title: To Think or Not to Think: The Hidden Cost of Meta-Training with Excessive CoT Examples

• Authors: Vignesh Kothapalli, Ata Fatahibaarzi, Hamed Firooz, Maziar Sanjabi
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.05318
• Pdf URL: https://arxiv.org/pdf/2512.05318
• Copy Paste: [[2512.05318]] To Think or Not to Think: The Hidden Cost of Meta-Training with Excessive CoT Examples(https://arxiv.org/abs/2512.05318)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: Chain-of-thought (CoT) prompting combined with few-shot in-context learning (ICL) has unlocked significant reasoning capabilities in large language models (LLMs). However, ICL with CoT examples is ineffective on novel tasks when the pre-training knowledge is insufficient. We study this problem in a controlled setting using the CoT-ICL Lab framework, and propose meta-training techniques to learn novel abstract reasoning tasks in-context. Although CoT examples facilitate reasoning, we noticed that their excessive inclusion during meta-training degrades performance when CoT supervision is limited. To mitigate such behavior, we propose CoT-Recipe, a formal approach to modulate the mix of CoT and non-CoT examples in meta-training sequences. We demonstrate that careful modulation via CoT-Recipe can increase the accuracy of transformers on novel tasks by up to 300% even when there are no CoT examples available in-context. We confirm the broader effectiveness of these techniques by applying them to pretrained LLMs (Qwen2.5 series) for symbolic reasoning tasks and observing gains of up to 130% in accuracy.
• 摘要：思想链 (CoT) 提示与少样本上下文学习 (ICL) 相结合，释放了大型语言模型 (LLM) 中的重要推理能力。然而，当预训练知识不足时，带有 CoT 示例的 ICL 对于新任务是无效的。我们使用 CoT-ICL Lab 框架在受控环境中研究这个问题，并提出元训练技术来学习上下文中新颖的抽象推理任务。尽管 CoT 示例有助于推理，但我们注意到，当 CoT 监督有限时，元训练期间过度包含它们会降低性能。为了减轻这种行为，我们提出了 CoT-Recipe，一种在元训练序列中调节 CoT 和非 CoT 示例混合的正式方法。我们证明，即使上下文中没有可用的 CoT 示例，通过 CoT-Recipe 进行仔细调制也​​可以将 Transformer 在新任务上的准确性提高高达 300%。我们通过将这些技术应用于预训练的 LLM（Qwen2.5 系列）来执行符号推理任务，并观察到准确度提高了高达 130%，从而证实了这些技术的更广泛有效性。

Title: Exposing Pink Slime Journalism: Linguistic Signatures and Robust Detection Against LLM-Generated Threats

• Authors: Sadat Shahriar, Navid Ayoobi, Arjun Mukherjee, Mostafa Musharrat, Sai Vishnu Vamsi
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.05331
• Pdf URL: https://arxiv.org/pdf/2512.05331
• Copy Paste: [[2512.05331]] Exposing Pink Slime Journalism: Linguistic Signatures and Robust Detection Against LLM-Generated Threats(https://arxiv.org/abs/2512.05331)
• Keywords: language model, llm
• Abstract: The local news landscape, a vital source of reliable information for 28 million Americans, faces a growing threat from Pink Slime Journalism, a low-quality, auto-generated articles that mimic legitimate local reporting. Detecting these deceptive articles requires a fine-grained analysis of their linguistic, stylistic, and lexical characteristics. In this work, we conduct a comprehensive study to uncover the distinguishing patterns of Pink Slime content and propose detection strategies based on these insights. Beyond traditional generation methods, we highlight a new adversarial vector: modifications through large language models (LLMs). Our findings reveal that even consumer-accessible LLMs can significantly undermine existing detection systems, reducing their performance by up to 40% in F1-score. To counter this threat, we introduce a robust learning framework specifically designed to resist LLM-based adversarial attacks and adapt to the evolving landscape of automated pink slime journalism, and showed and improvement by up to 27%.
• 摘要：当地新闻景观是 2800 万美国人可靠信息的重要来源，但它面临着来自“粉红史莱姆新闻”日益增长的威胁，这是一种模仿合法当地报道的低质量、自动生成的文章。检测这些欺骗性文章需要对其语言、文体和词汇特征进行精细分析。在这项工作中，我们进行了一项全面的研究，以揭示粉红史莱姆内容的区别模式，并根据这些见解提出检测策略。除了传统的生成方法之外，我们还强调了一种新的对抗向量：通过大型语言模型（LLM）进行修改。我们的研究结果表明，即使是消费者可访问的法学硕士也可能会严重破坏现有的检测系统，使其 F1 分数性能降低高达 40%。为了应对这一威胁，我们引入了一个强大的学习框架，专门设计用于抵御基于 LLM 的对抗性攻击，并适应自动化粉红史莱姆新闻业不断发展的格局，并表现出高达 27% 的改进。

Title: Mitigating Self-Preference by Authorship Obfuscation

• Authors: Taslim Mahbub, Shi Feng
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.05379
• Pdf URL: https://arxiv.org/pdf/2512.05379
• Copy Paste: [[2512.05379]] Mitigating Self-Preference by Authorship Obfuscation(https://arxiv.org/abs/2512.05379)
• Keywords: language model
• Abstract: Language models (LMs) judges are widely used to evaluate the quality of LM outputs. Despite many advantages, LM judges display concerning biases that can impair their integrity in evaluations. One such bias is self-preference: LM judges preferring their own answers over those produced by other LMs or humans. The bias is hard to eliminate as frontier LM judges can distinguish their own outputs from those of others, even when the evaluation candidates are not labeled with their sources. In this paper, we investigate strategies to mitigate self-preference by reducing the LM judges' ability to recognize their own outputs. We apply black-box perturbations to evaluation candidates in pairwise comparison to obfuscate the authorship and reduce self-recognition. We find that perturbations as simple as synonym replacement for a few words predictably reduce self-preference. However, we also uncover fundamental challenges to eliminating the bias: when we extrapolate our perturbations to a more complete neutralization of stylistic differences between the evaluation candidates, self-preference recovers. Our findings suggest that self-recognition and self-preference can happen on many semantic levels, and complete mitigation remains challenging despite promising initial results.
• 摘要：语言模型（LM）评判器被广泛用于评估 LM 输出的质量。尽管有许多优势，LM 法官仍表现出可能损害其评估诚信的偏见。其中一种偏见是自我偏好：LM 法官更喜欢自己的答案，而不是其他 LM 或人类给出的答案。这种偏见很难消除，因为前沿 LM 法官可以将自己的输出与其他人的输出区分开来，即使评估候选者没有标明其来源。在本文中，我们研究了通过降低 LM 法官识别自己输出的能力来减轻自我偏好的策略。我们在成对比较中对评估候选者应用黑盒扰动，以混淆作者身份并减少自我认知。我们发现，像替换几个单词的同义词这样简单的扰动可以预见地降低自我偏好。然而，我们也发现了消除偏见的根本挑战：当我们将扰动推断为更完全地消除评估候选人之间的风格差异时，自我偏好就会恢复。我们的研究结果表明，自我认知和自我偏好可以在许多语义层面上发生，尽管初步结果很有希望，但完全缓解仍然具有挑战性。

Title: Learning from Self Critique and Refinement for Faithful LLM Summarization

• Authors: Ting-Yao Hu, Hema Swetha Koppula, Hadi Pouransari, Cem Koc, Oncel Tuzel, Raviteja Vemulapalli
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.05387
• Pdf URL: https://arxiv.org/pdf/2512.05387
• Copy Paste: [[2512.05387]] Learning from Self Critique and Refinement for Faithful LLM Summarization(https://arxiv.org/abs/2512.05387)
• Keywords: language model, llm, hallucination
• Abstract: Large Language Models (LLMs) often suffer from hallucinations: output content that is not grounded in the input context, when performing long-form text generation tasks such as summarization. Prior works have shown that hallucinations can be reduced by iteratively critiquing and refining previously generated outputs using either the same model or a more powerful teacher model as the critique. However, these approaches either require additional test-time compute or assume access to more powerful teacher models, making them costly and less practical. In this work, we propose Self Critique and Refinement-based Preference Optimization (SCRPO), which is a self-supervised training framework that first constructs a preference dataset by leveraging the LLM's own critique and refinement capabilities, and then applies preference learning to improve the same LLM for faithful summarization. Experiments on three summarization benchmarks (XSUM CNNDM and SAMSum), demonstrate that our approach outperforms state-of-the-art self-supervised learning methods in terms of faithfulness metrics while either maintaining or improving other metrics that measure the overall quality of the summary. Moreover, compared to test-time refinement, our approach not only improves efficiency but also results in more faithful summaries.
• 摘要：大型语言模型 (LLM) 经常会出现幻觉：在执行摘要等长文本生成任务时，输出的内容不以输入上下文为基础。先前的工作表明，可以通过使用相同的模型或更强大的教师模型作为批评来迭代地批评和细化先前生成的输出来减少幻觉。然而，这些方法要么需要额外的测试时计算，要么需要访问更强大的教师模型，这使得它们成本高昂且实用性较差。在这项工作中，我们提出了基于自我批判和细化的偏好优化（SCRPO），这是一种自我监督的训练框架，首先利用LLM自身的批判和细化能力构建偏好数据集，然后应用偏好学习来改进相同的LLM以实现忠实的总结。对三个摘要基准（XSUM CNNDM 和 SAMSum）的实验表明，我们的方法在忠实度指标方面优于最先进的自监督学习方法，同时维持或改进了衡量摘要整体质量的其他指标。此外，与测试时细化相比，我们的方法不仅提高了效率，而且还产生了更忠实的摘要。

Title: SQ-format: A Unified Sparse-Quantized Hardware-friendly Data Format for LLMs

• Authors: Ruixuan Huang, Hao Zeng, Hantao Huang, Jinyuan Shi, Minghui Yu, Ian En-Hsu Yen, Shuai Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.05409
• Pdf URL: https://arxiv.org/pdf/2512.05409
• Copy Paste: [[2512.05409]] SQ-format: A Unified Sparse-Quantized Hardware-friendly Data Format for LLMs(https://arxiv.org/abs/2512.05409)
• Keywords: language model, llm
• Abstract: Post-training quantization (PTQ) plays a crucial role in the democratization of large language models (LLMs). However, existing low-bit quantization and sparsification techniques are difficult to balance accuracy and efficiency due to the limited hardware support. For example, W4A8 can only achieve the same peak TOPS as W8A8 whereas the GPU-supported sparse data format (2:4 semi-structure sparse) is seldomly adopted due to the loss of accuracy. To bridge this gap, in this paper, we propose the Sparse-Quantized Format (SQ-format), which is a unified data format for quantization and sparsification potentially easily supported by new hardware and existing GPUs. SQ-format makes use of the fact that sparse matrix can be accelerated in high-precision, and low-precision matrix multiplication can also be accelerated accordingly. As such, SQ-format is proposed to achieve Pareto improvement between performance and throughput. This format is particularly suitable for activations with outlier inequality status and makes their static compression possible. We show the state-of-the-art PTQ performance with SQ-format, propose the hardware required to support it, and further offer the design exploration and insights for the next-generation AI accelerators.
• 摘要：训练后量化（PTQ）在大型语言模型（LLM）的民主化中发挥着至关重要的作用。然而，现有的低比特量化和稀疏化技术由于硬件支持有限，很难平衡精度和效率。例如，W4A8只能达到与W8A8相同的峰值TOPS，而GPU支持的稀疏数据格式（2:4半结构稀疏）由于精度损失而很少采用。为了弥补这一差距，在本文中，我们提出了稀疏量化格式（SQ-format），它是一种用于量化和稀疏化的统一数据格式，可能很容易受到新硬件和现有 GPU 的支持。 SQ格式利用了稀疏矩阵可以在高精度下加速的特点，低精度矩阵乘法也可以相应加速。因此，提出SQ格式来实现性能和吞吐量之间的帕累托改进。这种格式特别适合具有异常不平等状态的激活，并使其静态压缩成为可能。我们以 SQ 格式展示最先进的 PTQ 性能，提出支持它所需的硬件，并进一步提供下一代 AI 加速器的设计探索和见解。

Title: LMSpell: Neural Spell Checking for Low-Resource Languages

• Authors: Akesh Gunathilakea, Nadil Karunarathnea, Tharusha Bandaranayakea, Nisansa de Silvaa, Surangika Ranathunga
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.05414
• Pdf URL: https://arxiv.org/pdf/2512.05414
• Copy Paste: [[2512.05414]] LMSpell: Neural Spell Checking for Low-Resource Languages(https://arxiv.org/abs/2512.05414)
• Keywords: language model, llm, hallucination
• Abstract: Spell correction is still a challenging problem for low-resource languages (LRLs). While pretrained language models (PLMs) have been employed for spell correction, their use is still limited to a handful of languages, and there has been no proper comparison across PLMs. We present the first empirical study on the effectiveness of PLMs for spell correction, which includes LRLs. We find that Large Language Models (LLMs) outperform their counterparts (encoder-based and encoder-decoder) when the fine-tuning dataset is large. This observation holds even in languages for which the LLM is not pre-trained. We release LMSpell, an easy- to use spell correction toolkit across PLMs. It includes an evaluation function that compensates for the hallucination of LLMs. Further, we present a case study with Sinhala to shed light on the plight of spell correction for LRLs.
• 摘要：对于低资源语言（LRL）来说，拼写纠正仍然是一个具有挑战性的问题。虽然预训练语言模型 (PLM) 已用于拼写校正，但它们的使用仍然仅限于少数语言，并且各个 PLM 之间还没有进行适当的比较。我们首次对 PLM 在拼写纠正方面的有效性进行了实证研究，其中包括 LRL。我们发现，当微调数据集很大时，大型语言模型（LLM）的性能优于其对应模型（基于编码器和编码器-解码器）。即使对于法学硕士未经过预训练的语言，这一观察结果也成立。我们发布了 LMSpell，这是一个跨 PLM 的易于使用的拼写纠正工具包。它包括一个评估功能，可以弥补法学硕士的幻觉。此外，我们还介绍了僧伽罗语的案例研究，以阐明 LRL 拼写纠正的困境。

Title: ArtistMus: A Globally Diverse, Artist-Centric Benchmark for Retrieval-Augmented Music Question Answering

• Authors: Daeyong Kwon, SeungHeon Doh, Juhan Nam
• Subjects: cs.CL, cs.AI, cs.IR, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.05430
• Pdf URL: https://arxiv.org/pdf/2512.05430
• Copy Paste: [[2512.05430]] ArtistMus: A Globally Diverse, Artist-Centric Benchmark for Retrieval-Augmented Music Question Answering(https://arxiv.org/abs/2512.05430)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Recent advances in large language models (LLMs) have transformed open-domain question answering, yet their effectiveness in music-related reasoning remains limited due to sparse music knowledge in pretraining data. While music information retrieval and computational musicology have explored structured and multimodal understanding, few resources support factual and contextual music question answering (MQA) grounded in artist metadata or historical context. We introduce MusWikiDB, a vector database of 3.2M passages from 144K music-related Wikipedia pages, and ArtistMus, a benchmark of 1,000 questions on 500 diverse artists with metadata such as genre, debut year, and topic. These resources enable systematic evaluation of retrieval-augmented generation (RAG) for MQA. Experiments show that RAG markedly improves factual accuracy; open-source models gain up to +56.8 percentage points (for example, Qwen3 8B improves from 35.0 to 91.8), approaching proprietary model performance. RAG-style fine-tuning further boosts both factual recall and contextual reasoning, improving results on both in-domain and out-of-domain benchmarks. MusWikiDB also yields approximately 6 percentage points higher accuracy and 40% faster retrieval than a general-purpose Wikipedia corpus. We release MusWikiDB and ArtistMus to advance research in music information retrieval and domain-specific question answering, establishing a foundation for retrieval-augmented reasoning in culturally rich domains such as music.
• 摘要：大型语言模型（LLM）的最新进展已经改变了开放域问答，但由于预训练数据中的音乐知识稀疏，它们在音乐相关推理中的有效性仍然有限。虽然音乐信息检索和计算音乐学已经探索了结构化和多模态理解，但很少有资源支持基于艺术家元数据或历史背景的事实和上下文音乐问答（MQA）。我们引入了 MusWikiDB（一个包含来自 144K 个音乐相关维基百科页面的 320 万个段落的矢量数据库）和 ArtistMus（一个针对 500 名不同艺术家的 1,000 个问题的基准，包含流派、首次亮相年份和主题等元数据）。这些资源支持对 MQA 的检索增强生成 (RAG) 进行系统评估。实验表明RAG显着提高了事实准确性；开源模型的性能提升高达 +56.8 个百分点（例如，Qwen3 8B 从 35.0 提高到 91.8），接近专有模型的性能。 RAG 式的微调进一步增强了事实回忆和上下文推理，从而改善了域内和域外基准测试的结果。与通用维基百科语料库相比，MusWikiDB 的准确度高出大约 6 个百分点，检索速度提高了 40%。我们发布 MusWikiDB 和 ArtistMus 来推进音乐信息检索和特定领域问答的研究，为音乐等文化丰富领域的检索增强推理奠定基础。

Title: Dynamic Alignment for Collective Agency: Toward a Scalable Self-Improving Framework for Open-Ended LLM Alignment

• Authors: Panatchakorn Anantaprayoon, Nataliia Babina, Jad Tarifi, Nima Asgharbeygi
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.05464
• Pdf URL: https://arxiv.org/pdf/2512.05464
• Copy Paste: [[2512.05464]] Dynamic Alignment for Collective Agency: Toward a Scalable Self-Improving Framework for Open-Ended LLM Alignment(https://arxiv.org/abs/2512.05464)
• Keywords: language model, llm, agent
• Abstract: Large Language Models (LLMs) are typically aligned with human values using preference data or predefined principles such as helpfulness, honesty, and harmlessness. However, as AI systems progress toward Artificial General Intelligence (AGI) and Artificial Superintelligence (ASI), such value systems may become insufficient. In addition, human feedback-based alignment remains resource-intensive and difficult to scale. While AI-feedback-based self-improving alignment methods have been explored as a scalable alternative, they have largely remained constrained to conventional alignment values. In this work, we explore both a more holistic alignment objective and a scalable, self-improving alignment approach. Aiming to transcend conventional alignment norms, we introduce Collective Agency (CA)-a unified and open-ended alignment value that encourages integrated agentic capabilities. We also propose Dynamic Alignment-an alignment framework that enables an LLM to iteratively align itself. Dynamic Alignment comprises two key components: (1) automated training dataset generation with LLMs, and (2) a self-rewarding mechanism, where the policy model evaluates its own output candidates and assigns rewards for GRPO-based learning. Experimental results demonstrate that our approach successfully aligns the model to CA while preserving general NLP capabilities.
• 摘要：大型语言模型 (LLM) 通常使用偏好数据或预定义的原则（例如乐于助人、诚实和无害）与人类价值观保持一致。然而，随着人工智能系统向通用人工智能（AGI）和超级人工智能（ASI）发展，这样的价值体系可能会变得不够。此外，基于人类反馈的调整仍然需要大量资源且难以扩展。虽然基于人工智能反馈的自我改进对齐方法已被探索为可扩展的替代方案，但它们在很大程度上仍然受到传统对齐值的限制。在这项工作中，我们探索了更全面的对齐目标和可扩展的、自我改进的对齐方法。为了超越传统的联盟规范，我们引入了集体代理（CA）——一种统一且开放的联盟价值，鼓励集成代理能力。我们还提出了动态对齐——一种对齐框架，使法学硕士能够迭代地自我对齐。动态调整包括两个关键组成部分：(1) 使用法学硕士自动生成训练数据集，以及 (2) 自我奖励机制，其中策略模型评估其自己的输出候选并为基于 GRPO 的学习分配奖励。实验结果表明，我们的方法成功地将模型与 CA 对齐，同时保留了一般 NLP 功能。

Title: SEA-SafeguardBench: Evaluating AI Safety in SEA Languages and Cultures

• Authors: Panuthep Tasawong, Jian Gang Ngui, Alham Fikri Aji, Trevor Cohn, Peerat Limkonchotiwat
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.05501
• Pdf URL: https://arxiv.org/pdf/2512.05501
• Copy Paste: [[2512.05501]] SEA-SafeguardBench: Evaluating AI Safety in SEA Languages and Cultures(https://arxiv.org/abs/2512.05501)
• Keywords: language model, llm
• Abstract: Safeguard models help large language models (LLMs) detect and block harmful content, but most evaluations remain English-centric and overlook linguistic and cultural diversity. Existing multilingual safety benchmarks often rely on machine-translated English data, which fails to capture nuances in low-resource languages. Southeast Asian (SEA) languages are underrepresented despite the region's linguistic diversity and unique safety concerns, from culturally sensitive political speech to region-specific misinformation. Addressing these gaps requires benchmarks that are natively authored to reflect local norms and harm scenarios. We introduce SEA-SafeguardBench, the first human-verified safety benchmark for SEA, covering eight languages, 21,640 samples, across three subsets: general, in-the-wild, and content generation. The experimental results from our benchmark demonstrate that even state-of-the-art LLMs and guardrails are challenged by SEA cultural and harm scenarios and underperform when compared to English texts.
• 摘要：保障模型有助于大型语言模型 (LLM) 检测和阻止有害内容，但大多数评估仍然以英语为中心，忽视了语言和文化多样性。现有的多语言安全基准通常依赖于机器翻译的英语数据，而这无法捕捉资源匮乏的语言中的细微差别。尽管东南亚（SEA）语言具有多样性，并且存在独特的安全问题（从文化敏感的政治言论到特定地区的错误信息），但该地区的语言代表性不足。解决这些差距需要本地编写的基准来反映当地规范和危害场景。我们推出 SEA-SafeguardBench，这是第一个经过人工验证的 SEA 安全基准，涵盖八种语言、21,640 个样本，涵盖三个子集：通用、野外和内容生成。我们基准测试的实验结果表明，即使是最先进的法学硕士和护栏也会受到东南亚文化和危害情景的挑战，并且与英文文本相比表现不佳。

Title: Automated Identification of Incidentalomas Requiring Follow-Up: A Multi-Anatomy Evaluation of LLM-Based and Supervised Approaches

• Authors: Namu Park, Farzad Ahmed, Zhaoyi Sun, Kevin Lybarger, Ethan Breinhorst, Julie Hu, Ozlem Uzuner, Martin Gunn, Meliha Yetisgen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.05537
• Pdf URL: https://arxiv.org/pdf/2512.05537
• Copy Paste: [[2512.05537]] Automated Identification of Incidentalomas Requiring Follow-Up: A Multi-Anatomy Evaluation of LLM-Based and Supervised Approaches(https://arxiv.org/abs/2512.05537)
• Keywords: language model, gpt, llm, prompt
• Abstract: Objective: To evaluate large language models (LLMs) against supervised baselines for fine-grained, lesion-level detection of incidentalomas requiring follow-up, addressing the limitations of current document-level classification systems. Methods: We utilized a dataset of 400 annotated radiology reports containing 1,623 verified lesion findings. We compared three supervised transformer-based encoders (BioClinicalModernBERT, ModernBERT, Clinical Longformer) against four generative LLM configurations (Llama 3.1-8B, GPT-4o, GPT-OSS-20b). We introduced a novel inference strategy using lesion-tagged inputs and anatomy-aware prompting to ground model reasoning. Performance was evaluated using class-specific F1-scores. Results: The anatomy-informed GPT-OSS-20b model achieved the highest performance, yielding an incidentaloma-positive macro-F1 of 0.79. This surpassed all supervised baselines (maximum macro-F1: 0.70) and closely matched the inter-annotator agreement of 0.76. Explicit anatomical grounding yielded statistically significant performance gains across GPT-based models (p < 0.05), while a majority-vote ensemble of the top systems further improved the macro-F1 to 0.90. Error analysis revealed that anatomy-aware LLMs demonstrated superior contextual reasoning in distinguishing actionable findings from benign lesions. Conclusion: Generative LLMs, when enhanced with structured lesion tagging and anatomical context, significantly outperform traditional supervised encoders and achieve performance comparable to human experts. This approach offers a reliable, interpretable pathway for automated incidental finding surveillance in radiology workflows.
• 摘要：目标：根据监督基线评估大型语言模型（LLM），以对需要随访的偶发瘤进行细粒度、病变级别的检测，解决当前文档级别分类系统的局限性。方法：我们使用了 400 份带注释的放射学报告的数据集，其中包含 1,623 个经过验证的病变发现。我们将三种基于 Transformer 的监督编码器（BioClinicalModernBERT、ModernBERT、Clinical Longformer）与四种生成式 LLM 配置（Llama 3.1-8B、GPT-4o、GPT-OSS-20b）进行了比较。我们引入了一种新颖的推理策略，使用病变标记输入和解剖感知提示来进行基础模型推理。使用特定类别的 F1 分数来评估表现。结果：基于解剖学的 GPT-OSS-20b 模型获得了最高的性能，偶发瘤阳性宏观 F1 为 0.79。这超过了所有监督基线（最大宏 F1：0.70），并且与注释者间一致性 0.76 紧密匹配。显式解剖基础在基于 GPT 的模型中产生了统计上显着的性能提升 (p < 0.05)，而顶级系统的多数票集成进一步将宏 F1 提高到 0.90。错误分析显示，具有解剖学意识的法学硕士在区分可操作的发现与良性病变方面表现出卓越的情境推理能力。结论：生成式 LLM 在通过结构化病变标记和解剖背景进行增强时，显着优于传统的监督编码器，并达到与人类专家相当的性能。这种方法为放射学工作流程中的自动偶然发现监测提供了可靠、可解释的途径。

Title: Structured Reasoning with Tree-of-Thoughts for Bengali Math Word Problems

• Authors: Aurprita Mahmood, Sabrin alam, Neloy kumer Sagor, Md. Abdul Hadi, Md. Sehab Al Islam, Minhajul Islam
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.05580
• Pdf URL: https://arxiv.org/pdf/2512.05580
• Copy Paste: [[2512.05580]] Structured Reasoning with Tree-of-Thoughts for Bengali Math Word Problems(https://arxiv.org/abs/2512.05580)
• Keywords: language model, gpt, llm, prompt, chain-of-thought, tree-of-thought
• Abstract: Mathematical Word Problems (MWPs) are among the most challenging tasks in natural language processing because they require both linguistic understanding and multi-step numerical reasoning. While Chain-of-Thought (CoT) prompting has shown promise, its linear structure often propagates errors, limiting overall effectiveness. To address this limitation, we present the a systematic study of Tree-of-Thought (ToT) reasoning for Bengali MWPs using the SOMADHAN dataset. Owing to computational and token-cost constraints, we evaluate a curated set of 100 representative problems across multiple large language models (LLMs), including GPT-OSS and LLaMA variants, under standard prompting, CoT, and ToT strategies. Our results show that CoT improves baseline accuracy from 78% (standard prompting) to 83% on average, while ToT further increases performance by up to 5 percentage points, achieving 88% accuracy with GPT-OSS-120B. These improvements highlight that ToT is particularly effective in medium-to-large-scale models but may offer less advantage for smaller ones. Overall, our findings establish ToT as a robust framework for solving mathematical problems in low-resource languages such as Bengali. More broadly, this study shows that structured reasoning methods like ToT can provide more reliable and globally consistent outcomes than CoT, paving the way for better reasoning strategies in multilingual NLP.
• 摘要：数学应用题（MWP）是自然语言处理中最具挑战性的任务之一，因为它们需要语言理解和多步骤数值推理。虽然思想链 (CoT) 提示已显示出希望，但其线性结构经常会传播错误，从而限制了整体有效性。为了解决这一局限性，我们使用 SOMADHAN 数据集对孟加拉 MWP 的思想树 (ToT) 推理进行了系统研究。由于计算和令牌成本的限制，我们在标准提示、CoT 和 ToT 策略下评估了跨多个大型语言模型 (LLM)（包括 GPT-OSS 和 LLaMA 变体）的一组精选的 100 个代表性问题。我们的结果表明，CoT 平均将基线准确率从 78%（标准提示）提高到 83%，而 ToT 进一步将性能提高多达 5 个百分点，GPT-OSS-120B 的准确率达到 88%。这些改进凸显出 ToT 在中型到大型模型中特别有效，但对于小型模型来说优势可能较小。总的来说，我们的研究结果将 ToT 确立为用孟加拉语等低资源语言解决数学问题的强大框架。更广泛地说，这项研究表明，与 CoT 相比，ToT 等结构化推理方法可以提供更可靠、全球一致的结果，为多语言 NLP 中更好的推理策略铺平道路。

Title: A Greek Government Decisions Dataset for Public-Sector Analysis and Insight

• Authors: Giorgos Antoniou, Giorgos Filandrianos, Aggelos Vlachos, Giorgos Stamou, Lampros Kollimenos, Konstantinos Skianis, Michalis Vazirgiannis
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.05647
• Pdf URL: https://arxiv.org/pdf/2512.05647
• Copy Paste: [[2512.05647]] A Greek Government Decisions Dataset for Public-Sector Analysis and Insight(https://arxiv.org/abs/2512.05647)
• Keywords: language model, llm, chat, retrieval-augmented generation
• Abstract: We introduce an open, machine-readable corpus of Greek government decisions sourced from the national transparency platform Diavgeia. The resource comprises 1 million decisions, featuring and high-quality raw text extracted from PDFs. It is released with raw extracted text in Markdown format, alongside a fully reproducible extraction pipeline. Beyond the core dataset, we conduct qualitative analyses to explore boilerplate patterns and design a retrieval-augmented generation (RAG) task by formulating a set of representative questions, creating high-quality answers, and evaluating a baseline RAG system on its ability to retrieve and reason over public decisions. This evaluation demonstrates the potential of large-scale public-sector corpora to support advanced information access and transparency through structured retrieval and reasoning over governmental documents, and highlights how such a RAG pipeline could simulate a chat-based assistant capable of interactively answering questions about public decisions. Due to its scale, quality, and domain coverage, the corpus can also serve as high-value pre-training or fine-tuning material for new Language Models (LMs) and Large Language Models (LLMs) respectively, including specialized models for legal and governmental domains, and as a foundation for novel approaches in domain adaptation, knowledge-grounded generation, and explainable AI. Finally, we discuss limitations, outline future directions, and make both the data and the code accessible.
• 摘要：我们引入了一个开放的、机器可读的希腊政府决策语料库，该语料库源自国家透明度平台 Diavgeia。该资源包含 100 万个决策，包含从 PDF 中提取的高质量原始文本。它以 Markdown 格式的原始提取文本以及完全可重现的提取管道一起发布。除了核心数据集之外，我们还进行定性分析来探索样板模式，并通过制定一组代表性问题、创建高质量答案以及评估基线 RAG 系统检索和推理公共决策的能力来设计检索增强生成 (RAG) 任务。该评估展示了大规模公共部门语料库通过对政府文件进行结构化检索和推理来支持高级信息访问和透明度的潜力，并强调了这样的 RAG 管道如何模拟基于聊天的助手，能够交互式地回答有关公共决策的问题。由于其规模、质量和领域覆盖范围，该语料库还可以分别作为新语言模型（LM）和大型语言模型（LLM）的高价值预训练或微调材料，包括法律和政府领域的专门模型，并作为领域适应、基于知识的生成和可解释人工智能等新方法的基础。最后，我们讨论局限性，概述未来的方向，并使数据和代码都易于访问。

Title: Grounded Multilingual Medical Reasoning for Question Answering with Large Language Models

• Authors: Pietro Ferrazzi, Aitor Soroa, Rodrigo Agerri
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.05658
• Pdf URL: https://arxiv.org/pdf/2512.05658
• Copy Paste: [[2512.05658]] Grounded Multilingual Medical Reasoning for Question Answering with Large Language Models(https://arxiv.org/abs/2512.05658)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) with reasoning capabilities have recently demonstrated strong potential in medical Question Answering (QA). Existing approaches are largely English-focused and primarily rely on distillation from general-purpose LLMs, raising concerns about the reliability of their medical knowledge. In this work, we present a method to generate multilingual reasoning traces grounded in factual medical knowledge. We produce 500k traces in English, Italian, and Spanish, using a retrievalaugmented generation approach over medical information from Wikipedia. The traces are generated to solve medical questions drawn from MedQA and MedMCQA, which we extend to Italian and Spanish. We test our pipeline in both in-domain and outof-domain settings across Medical QA benchmarks, and demonstrate that our reasoning traces improve performance both when utilized via in-context learning (few-shot) and supervised fine-tuning, yielding state-of-the-art results among 8B-parameter LLMs. We believe that these resources can support the development of safer, more transparent clinical decision-support tools in multilingual settings. We release the full suite of resources: reasoning traces, translated QA datasets, Medical-Wikipedia, and fine-tuned models.
• 摘要：具有推理能力的大型语言模型 (LLM) 最近在医学问答 (QA) 领域展现出强大的潜力。现有的方法主要以英语为中心，主要依赖于通用法学硕士的提炼，这引起了人们对其医学知识可靠性的担忧。在这项工作中，我们提出了一种基于事实医学知识生成多语言推理轨迹的方法。我们对维基百科的医疗信息使用检索增强生成方法，生成 500k 条英语、意大利语和西班牙语轨迹。生成这些痕迹是为了解决来自 MedQA 和 MedMCQA 的医学问题，我们将其扩展到意大利语和西班牙语。我们在医学 QA 基准测试中在域内和域外设置中测试了我们的流程，并证明我们的推理轨迹在通过上下文学习（少样本）和监督微调使用时都能提高性能，从而在 8B 参数 LLM 中产生最先进的结果。我们相信这些资源可以支持在多语言环境中开发更安全、更透明的临床决策支持工具。我们发布了全套资源：推理轨迹、翻译的 QA 数据集、医学维基百科和微调模型。

Title: Interleaved Latent Visual Reasoning with Selective Perceptual Modeling

• Authors: Shuai Dong, Siyuan Wang, Xingyu Liu, Zhongyu Wei
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2512.05665
• Pdf URL: https://arxiv.org/pdf/2512.05665
• Copy Paste: [[2512.05665]] Interleaved Latent Visual Reasoning with Selective Perceptual Modeling(https://arxiv.org/abs/2512.05665)
• Keywords: language model, llm
• Abstract: Interleaved reasoning paradigms enhance Multimodal Large Language Models (MLLMs) with visual feedback but are hindered by the prohibitive computational cost of repeatedly re-encoding pixel-dense images. A promising alternative, latent visual reasoning, circumvents this bottleneck yet currently forces a critical trade-off: methods either sacrifice precise perceptual modeling by over-compressing features or fail to model dynamic problems due to static, non-interleaved structures. We introduce Interleaved Latent Visual Reasoning (ILVR), a framework that unifies dynamic state evolution with precise perceptual modeling. ILVR interleaves textual generation with latent visual representations that act as specific, evolving cues for subsequent reasoning. To enable this, we employ a self-supervision strategy where a Momentum Teacher Model selectively distills relevant features from helper images into sparse supervision targets. This adaptive selection mechanism guides the model to autonomously generate context-aware visual signals. Extensive experiments on multimodal reasoning benchmarks demonstrate that ILVR significantly outperforms existing approaches, effectively bridging the gap between fine-grained perception and sequential multimodal reasoning.
• 摘要：交错推理范例通过视觉反馈增强了多模态大型语言模型（MLLM），但受到重复重新编码像素密集图像的高昂计算成本的阻碍。一种有前景的替代方案——潜在视觉推理——绕过了这一瓶颈，但目前却面临着一个关键的权衡：方法要么通过过度压缩特征而牺牲精确的感知建模，要么由于静态、非交错结构而无法对动态问题进行建模。我们引入了交错潜在视觉推理（ILVR），这是一个将动态演化与精确感知建模相结合的框架。 ILVR 将文本生成与潜在的视觉表示交织在一起，作为后续推理的特定的、不断发展的线索。为了实现这一点，我们采用了一种自我监督策略，其中动量教师模型有选择地将辅助图像中的相关特征提取到稀疏的监督目标中。这种自适应选择机制引导模型自主生成上下文感知的视觉信号。对多模态推理基准的大量实验表明，ILVR 显着优于现有方法，有效地弥合了细粒度感知和顺序多模态推理之间的差距。

Title: MedTutor-R1: Socratic Personalized Medical Teaching with Multi-Agent Simulation

• Authors: Zhitao He, Haolin Yang, Zeyu Qin, Yi R Fung
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.05671
• Pdf URL: https://arxiv.org/pdf/2512.05671
• Copy Paste: [[2512.05671]] MedTutor-R1: Socratic Personalized Medical Teaching with Multi-Agent Simulation(https://arxiv.org/abs/2512.05671)
• Keywords: language model, llm, agent
• Abstract: The significant gap between rising demands for clinical training and the scarcity of expert instruction poses a major challenge to medical education. With powerful capabilities in personalized guidance, Large Language Models (LLMs) offer a promising solution to bridge this gap. However, current research focuses mainly on one-on-one knowledge instruction, overlooking collaborative reasoning, a key skill for students developed in teamwork like ward rounds. To this end, we develop ClinEdu, a multi-agent pedagogical simulator with personality-driven patients and diverse student cohorts, enabling controlled testing of complex pedagogical processes and scalable generation of teaching data. Based on ClinEdu, we construct ClinTeach, a large Socratic teaching dialogue dataset that captures the complexities of group instruction. We then train MedTutor-R1, the first multimodal Socratic tutor designed for one-to-many instruction in clinical medical education. MedTutor-R1 is first instruction-tuned on our ClinTeach dataset and then optimized with reinforcement learning, using rewards derived from a three-axis rubric, covering structural fidelity, analytical quality, and clinical safety, to refine its adaptive Socratic strategies. For authentic in-situ assessment, we use simulation-based interactive evaluation that redeploys the tutor back into ClinEdu. Experimental results demonstrate that our MedTutor-R1 outperforms the base model by over 20% in average pedagogical score and is comparable to o3, while also exhibiting high adaptability in handling a varying number of students. This promising performance underscores the effectiveness of our pedagogical simulator, ClinEdu.
• 摘要：不断增长的临床培训需求与专家指导的稀缺之间的巨大差距给医学教育带来了重大挑战。凭借强大的个性化指导功能，大型语言模型 (LLM) 提供了一种有前景的解决方案来弥补这一差距。然而，目前的研究主要集中在一对一的知识教学，忽视了协作推理，这是学生在查房等团队合作中培养的一项关键技能。为此，我们开发了 ClinEdu，这是一种多智能体教学模拟器，具有个性驱动的患者和多样化的学生群体，能够对复杂的教学过程进行受控测试并生成可扩展的教学数据。基于 ClinEdu，我们构建了 ClinTeach，这是一个大型苏格拉底式教学对话数据集，可以捕捉小组教学的复杂性。然后，我们训练 MedTutor-R1，这是第一个多模式苏格拉底式导师，专为临床医学教育中的一对多教学而设计。 MedTutor-R1 首先在我们的 ClinTeach 数据集上进行指令调整，然后通过强化学习进行优化，使用从三轴标题衍生的奖励，涵盖结构保真度、分析质量和临床安全性，以完善其自适应苏格拉底策略。对于真实的现场评估，我们使用基于模拟的交互式评估，将导师重新部署回 ClinEdu。实验结果表明，我们的 MedTutor-R1 的平均教学分数比基本模型高出 20% 以上，与 o3 相当，同时在处理不同数量的学生方面也表现出高度的适应性。这一令人鼓舞的表现凸显了我们的教学模拟器 ClinEdu 的有效性。

Title: Faithfulness metric fusion: Improving the evaluation of LLM trustworthiness across domains

• Authors: Ben Malin, Tatiana Kalganova, Nikolaos Boulgouris
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.05700
• Pdf URL: https://arxiv.org/pdf/2512.05700
• Copy Paste: [[2512.05700]] Faithfulness metric fusion: Improving the evaluation of LLM trustworthiness across domains(https://arxiv.org/abs/2512.05700)
• Keywords: language model, llm
• Abstract: We present a methodology for improving the accuracy of faithfulness evaluation in Large Language Models (LLMs). The proposed methodology is based on the combination of elementary faithfulness metrics into a combined (fused) metric, for the purpose of improving the faithfulness of LLM outputs. The proposed strategy for metric fusion deploys a tree-based model to identify the importance of each metric, which is driven by the integration of human judgements evaluating the faithfulness of LLM responses. This fused metric is demonstrated to correlate more strongly with human judgements across all tested domains for faithfulness. Improving the ability to evaluate the faithfulness of LLMs, allows for greater confidence to be placed within models, allowing for their implementation in a greater diversity of scenarios. Additionally, we homogenise a collection of datasets across question answering and dialogue-based domains and implement human judgements and LLM responses within this dataset, allowing for the reproduction and trialling of faithfulness evaluation across domains.
• 摘要：我们提出了一种提高大型语言模型（LLM）忠实度评估准确性的方法。所提出的方法基于将基本忠实度指标组合成组合（融合）指标，以提高法学硕士输出的忠实度。所提出的度量融合策略部署了一个基于树的模型来识别每个度量的重要性，这是由评估 LLM 响应可信度的人类判断的整合驱动的。事实证明，这种融合指标与所有测试领域的人类忠诚度判断具有更强的相关性。提高评估法学硕士真实性的能力，可以让人们对模型更有信心，从而可以在更多样化的场景中实施它们。此外，我们将跨问答和基于对话的领域的数据集集合同质化，并在该数据集中实施人类判断和法学硕士响应，从而允许跨领域的忠诚度评估的复制和试验。

Title: Efficient Text Classification with Conformal In-Context Learning

• Authors: Ippokratis Pantelidis, Korbinian Randl, Aron Henriksson
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.05732
• Pdf URL: https://arxiv.org/pdf/2512.05732
• Copy Paste: [[2512.05732]] Efficient Text Classification with Conformal In-Context Learning(https://arxiv.org/abs/2512.05732)
• Keywords: language model, llm, prompt
• Abstract: Large Language Models (LLMs) demonstrate strong in-context learning abilities, yet their effectiveness in text classification depends heavily on prompt design and incurs substantial computational cost. Conformal In-Context Learning (CICLe) has been proposed as a resource-efficient framework that integrates a lightweight base classifier with Conformal Prediction to guide LLM prompting by adaptively reducing the set of candidate classes. However, its broader applicability and efficiency benefits beyond a single domain have not yet been systematically explored. In this paper, we present a comprehensive evaluation of CICLe across diverse NLP classification benchmarks. The results show that CICLe consistently improves over its base classifier and outperforms few-shot prompting baselines when the sample size is sufficient for training the base classifier, and performs comparably in low-data regimes. In terms of efficiency, CICLe reduces the number of shots and prompt length by up to 34.45% and 25.16%, respectively, and enables the use of smaller models with competitive performance. CICLe is furthermore particularly advantageous for text classification tasks with high class imbalance. These findings highlight CICLe as a practical and scalable approach for efficient text classification, combining the robustness of traditional classifiers with the adaptability of LLMs, and achieving substantial gains in data and computational efficiency.
• 摘要：大型语言模型（LLM）表现出强大的上下文学习能力，但它们在文本分类方面的有效性在很大程度上取决于提示设计并会产生大量的计算成本。保形上下文学习 (CICLe) 被提出作为一种资源高效的框架，它将轻量级基础分类器与保形预测集成在一起，通过自适应地减少候选类集来指导 LLM 提示。然而，其超越单一领域的更广泛的适用性和效率优势尚未得到系统探索。在本文中，我们提出了跨不同 NLP 分类基准的 CICLe 综合评估。结果表明，当样本量足以训练基分类器时，CICLe 持续改进其基分类器，并且优于少样本提示基线，并且在低数据情况下表现相当。在效率方面，CICLe将镜头数量和提示长度分别减少了高达34.45%和25.16%，并且可以使用具有竞争力性能的更小模型。此外，CICLe 对于具有高度类别不平衡的文本分类任务特别有利。这些发现凸显了 CICLe 作为一种实用且可扩展的高效文本分类方法，将传统分类器的稳健性与法学硕士的适应性相结合，并在数据和计算效率方面取得了显着的进步。

Title: Capturing Classic Authorial Style in Long-Form Story Generation with GRPO Fine-Tuning

• Authors: Jinlong Liu, Mohammed Bahja, Venelin Kovatchev, Mark Lee
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.05747
• Pdf URL: https://arxiv.org/pdf/2512.05747
• Copy Paste: [[2512.05747]] Capturing Classic Authorial Style in Long-Form Story Generation with GRPO Fine-Tuning(https://arxiv.org/abs/2512.05747)
• Keywords: language model, gpt, llm, agent
• Abstract: Recent advances in large language models (LLMs) show impressive performance in open-ended story generation, but fine-grained stylistic control remains limited. Existing methods often rely on shallow cues (e.g., names or topics) to simulate authorial style, without robust evaluation. In this work, we present a training framework for style-conditioned story generation using Group Relative Policy Optimization (GRPO) and a custom multi-reward setup. The style reward is derived from a fine-tuned sentence transformer using authorship verification (AV) signals, combined with content and completeness scores to stabilize long-form narrative generation. We conduct experiments using fiction by Mark Twain, a prominent 19th-century American author, with The Adventures of Huckleberry Finn serving as the reference style exemplar. Our 8B model outperforms larger baselines such as GPT-4o and Claude Sonnet 4 in AV-style metrics, achieving a style score of 0.628 and competitive content quality. Results demonstrate the feasibility of agentic stylistic generation with moderate model size and task-specific training. While the output is clearly style-aligned, narrative completeness remains a challenge, indicating future work is needed to better model global coherence and story resolution.
• 摘要：大型语言模型 (LLM) 的最新进展在开放式故事生成方面显示出令人印象深刻的性能，但细粒度的风格控制仍然有限。现有的方法通常依赖于浅层线索（例如名称或主题）来模拟作者风格，而没有稳健的评估。在这项工作中，我们提出了一个使用组相对策略优化（GRPO）和自定义多重奖励设置来生成风格条件故事的训练框架。风格奖励源自使用作者身份验证（AV）信号的微调句子转换器，并结合内容和完整性分数来稳定长篇叙事的生成。我们以 19 世纪美国著名作家马克·吐温的小说为实验对象，以《哈克贝利·费恩历险记》作为参考风格范例。我们的 8B 模型在 AV 风格指标方面优于更大的基线，例如 GPT-4o 和 Claude Sonnet 4，实现了 0.628 的风格得分和具有竞争力的内容质量。结果证明了具有中等模型大小和特定任务训练的代理风格生成的可行性。虽然输出明显符合风格，但叙事完整性仍然是一个挑战，这表明未来需要更好地模拟全球连贯性和故事解决方案。

Title: Prompting Science Report 4: Playing Pretend: Expert Personas Don't Improve Factual Accuracy

• Authors: Savir Basil, Ina Shapiro, Dan Shapiro, Ethan Mollick, Lilach Mollick, Lennart Meincke
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.05858
• Pdf URL: https://arxiv.org/pdf/2512.05858
• Copy Paste: [[2512.05858]] Prompting Science Report 4: Playing Pretend: Expert Personas Don't Improve Factual Accuracy(https://arxiv.org/abs/2512.05858)
• Keywords: prompt
• Abstract: This is the fourth in a series of short reports that help business, education, and policy leaders understand the technical details of working with AI through rigorous testing. Here, we ask whether assigning personas to models improves performance on difficult objective multiple-choice questions. We study both domain-specific expert personas and low-knowledge personas, evaluating six models on GPQA Diamond (Rein et al. 2024) and MMLU-Pro (Wang et al. 2024), graduate-level questions spanning science, engineering, and law. We tested three approaches: -In-Domain Experts: Assigning the model an expert persona ("you are a physics expert") matched to the problem type (physics problems) had no significant impact on performance (with the exception of the Gemini 2.0 Flash model). -Off-Domain Experts (Domain-Mismatched): Assigning the model an expert persona ("you are a physics expert") not matched to the problem type (law problems) resulted in marginal differences. -Low-Knowledge Personas: We assigned the model negative capability personas (layperson, young child, toddler), which were generally harmful to benchmark accuracy. Across both benchmarks, persona prompts generally did not improve accuracy relative to a no-persona baseline. Expert personas showed no consistent benefit across models, with few exceptions. Domain-mismatched expert personas sometimes degraded performance. Low-knowledge personas often reduced accuracy. These results are about the accuracy of answers only; personas may serve other purposes (such as altering the tone of outputs), beyond improving factual performance.
• 摘要：这是一系列简短报告中的第四份，旨在帮助企业、教育和政策领导者通过严格的测试了解使用人工智能的技术细节。在这里，我们询问将角色分配给模型是否可以提高困难的客观多项选择问题的性能。我们研究特定领域的专家角色和低知识角色，评估 GPQA Diamond（Rein 等人，2024）和 MMLU-Pro（Wang 等人，2024）的六个模型，以及涵盖科学、工程和法律的研究生级别问题。我们测试了三种方法： - 域内专家：为模型分配与问题类型（物理问题）匹配的专家角色（“您是物理专家”），这对性能没有显着影响（Gemini 2.0 Flash 模型除外）。 - 域外专家（域不匹配）：为模型分配与问题类型（法律问题）不匹配的专家角色（“您是物理专家”）会导致边际差异。 -低知识角色：我们分配了模型负面能力角色（外行、幼儿、幼儿），这通常对基准准确性有害。在这两个基准测试中，相对于无角色基线，角色提示通常不会提高准确性。除了少数例外，专家角色在不同模型中并没有表现出一致的优势。领域不匹配的专家角色有时会降低性能。知识匮乏的角色通常会降低准确性。这些结果仅与答案的准确性有关；除了提高事实表现之外，人物角色还可以用于其他目的（例如改变输出的基调）。

Title: Optimizing Medical Question-Answering Systems: A Comparative Study of Fine-Tuned and Zero-Shot Large Language Models with RAG Framework

• Authors: Tasnimul Hassan, Md Faisal Karim, Haziq Jeelani, Elham Behnam, Robert Green, Fayeq Jeelani Syed
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.05863
• Pdf URL: https://arxiv.org/pdf/2512.05863
• Copy Paste: [[2512.05863]] Optimizing Medical Question-Answering Systems: A Comparative Study of Fine-Tuned and Zero-Shot Large Language Models with RAG Framework(https://arxiv.org/abs/2512.05863)
• Keywords: language model, llm, hallucination, retrieval-augmented generation
• Abstract: Medical question-answering (QA) systems can benefit from advances in large language models (LLMs), but directly applying LLMs to the clinical domain poses challenges such as maintaining factual accuracy and avoiding hallucinations. In this paper, we present a retrieval-augmented generation (RAG) based medical QA system that combines domain-specific knowledge retrieval with open-source LLMs to answer medical questions. We fine-tune two state-of-the-art open LLMs (LLaMA~2 and Falcon) using Low-Rank Adaptation (LoRA) for efficient domain specialization. The system retrieves relevant medical literature to ground the LLM's answers, thereby improving factual correctness and reducing hallucinations. We evaluate the approach on benchmark datasets (PubMedQA and MedMCQA) and show that retrieval augmentation yields measurable improvements in answer accuracy compared to using LLMs alone. Our fine-tuned LLaMA~2 model achieves 71.8% accuracy on PubMedQA, substantially improving over the 55.4% zero-shot baseline, while maintaining transparency by providing source references. We also detail the system design and fine-tuning methodology, demonstrating that grounding answers in retrieved evidence reduces unsupported content by approximately 60%. These results highlight the potential of RAG-augmented open-source LLMs for reliable biomedical QA, pointing toward practical clinical informatics applications.
• 摘要：医学问答 (QA) 系统可以受益于大语言模型 (LLM) 的进步，但直接将 LLM 应用于临床领域会带来挑战，例如保持事实准确性和避免幻觉。在本文中，我们提出了一种基于检索增强生成（RAG）的医学问答系统，该系统将特定领域的知识检索与开源法学硕士相结合来回答医学问题。我们使用低秩适应 (LoRA) 微调两个最先进的开放式 LLM（LLaMA~2 和 Falcon），以实现高效的领域专业化。该系统检索相关医学文献来为法学硕士的答案提供依据，从而提高事实的正确性并减少幻觉。我们在基准数据集（PubMedQA 和 MedMCQA）上评估了该方法，并表明与单独使用法学硕士相比，检索增强可以显着提高答案准确性。我们经过微调的 LLaMA~2 模型在 PubMedQA 上实现了 71.8% 的准确率，比 55.4% 的零样本基线有了显着提高，同时通过提供源参考来保持透明度。我们还详细介绍了系统设计和微调方法，证明在检索到的证据中建立基础答案可将不受支持的内容减少约 60%。这些结果凸显了 RAG 增强型开源法学硕士在可靠的生物医学 QA 方面的潜力，指向实际的临床信息学应用。

Title: M4-RAG: A Massive-Scale Multilingual Multi-Cultural Multimodal RAG

• Authors: David Anugraha, Patrick Amadeus Irawan, Anshul Singh, En-Shiun Annie Lee, Genta Indra Winata
• Subjects: cs.CL, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2512.05959
• Pdf URL: https://arxiv.org/pdf/2512.05959
• Copy Paste: [[2512.05959]] M4-RAG: A Massive-Scale Multilingual Multi-Cultural Multimodal RAG(https://arxiv.org/abs/2512.05959)
• Keywords: language model, retrieval-augmented generation
• Abstract: Vision-language models (VLMs) have achieved strong performance in visual question answering (VQA), yet they remain constrained by static training data. Retrieval-Augmented Generation (RAG) mitigates this limitation by enabling access to up-to-date, culturally grounded, and multilingual information; however, multilingual multimodal RAG remains largely underexplored. We introduce M4-RAG, a massive-scale benchmark covering 42 languages and 56 regional dialects and registers, comprising over 80,000 culturally diverse image-question pairs for evaluating retrieval-augmented VQA across languages and modalities. To balance realism with reproducibility, we build a controlled retrieval environment containing millions of carefully curated multilingual documents relevant to the query domains, approximating real-world retrieval conditions while ensuring consistent experimentation. Our systematic evaluation reveals that although RAG consistently benefits smaller VLMs, it fails to scale to larger models and often even degrades their performance, exposing a critical mismatch between model size and current retrieval effectiveness. M4-RAG provides a foundation for advancing next-generation RAG systems capable of reasoning seamlessly across languages, modalities, and cultural contexts.
• 摘要：视觉语言模型（VLM）在视觉问答（VQA）方面取得了强大的性能，但它们仍然受到静态训练数据的限制。检索增强生成 (RAG) 通过允许访问最新的、基于文化的多语言信息来缓解这一限制；然而，多语言多模式 RAG 在很大程度上仍未得到充分探索。我们推出了 M4-RAG，这是一个涵盖 42 种语言和 56 种地区方言和语域的大规模基准，包含超过 80,000 个文化多样化的图像问题对，用于评估跨语言和模式的检索增强 VQA。为了平衡真实性和可重复性，我们构建了一个受控检索环境，其中包含数百万个精心策划的与查询域相关的多语言文档，近似真实世界的检索条件，同时确保实验的一致性。我们的系统评估表明，尽管 RAG 始终有利于较小的 VLM，但它无法扩展到较大的模型，甚至常常降低其性能，从而暴露出模型大小和当前检索有效性之间的严重不匹配。 M4-RAG 为推进下一代 RAG 系统奠定了基础，该系统能够跨语言、模式和文化背景进行无缝推理。