2025-08-27

Title: Semantic Attractors and the Emergence of Meaning: Towards a Teleological Model of AGI

• Authors: Hans-Joachim Rudolph
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.18290
• Pdf URL: https://arxiv.org/pdf/2508.18290
• Copy Paste: [[2508.18290]] Semantic Attractors and the Emergence of Meaning: Towards a Teleological Model of AGI(https://arxiv.org/abs/2508.18290)
• Keywords: language model, agent
• Abstract: This essay develops a theoretical framework for a semantic Artificial General Intelligence (AGI) based on the notion of semantic attractors in complex-valued meaning spaces. Departing from current transformer-based language models, which operate on statistical next-token prediction, we explore a model in which meaning is not inferred probabilistically but formed through recursive tensorial transformation. Using cyclic operations involving the imaginary unit \emph{i}, we describe a rotational semantic structure capable of modeling irony, homonymy, and ambiguity. At the center of this model, however, is a semantic attractor -- a teleological operator that, unlike statistical computation, acts as an intentional agent (Microvitum), guiding meaning toward stability, clarity, and expressive depth. Conceived in terms of gradient flows, tensor deformations, and iterative matrix dynamics, the attractor offers a model of semantic transformation that is not only mathematically suggestive, but also philosophically significant. We argue that true meaning emerges not from simulation, but from recursive convergence toward semantic coherence, and that this requires a fundamentally new kind of cognitive architecture -- one designed to shape language, not just predict it.
• 摘要：本文基于复杂价值的含义空间中语义吸引子的概念，为语义人工通用智能（AGI）开发了一个理论框架。偏离了基于统计的下一步预测的当前基于变压器的语言模型，我们探索了一个模型，其中含义不是概率上推断出来的，而是通过递归张力转换形成的。使用涉及假想单元\ emph {i}的循环操作，我们描述了能够建模讽刺，同义词和歧义的旋转语义结构。然而，该模型的中心是一种语义吸引子 - 一个目的论操作员，与统计计算不同，它是有意的代理（微伏），指导含义稳定，清晰度和表达深度。吸引子以梯度流，张量变形和迭代矩阵动力学来构想，它提供了一种语义转换模型，不仅在数学上具有暗示性，而且在哲学上具有重要意义。我们认为，真正的含义不是源于模拟，而是从递归融合到语义连贯性中出现的，这需要一种从根本上开始的认知体系结构 - 一种旨在塑造语言的一种，而不仅仅是预测它。

Title: LLMs Can't Handle Peer Pressure: Crumbling under Multi-Agent Social Interactions

• Authors: Maojia Song, Tej Deep Pala, Weisheng Jin, Amir Zadeh, Chuan Li, Dorien Herremans, Soujanya Poria
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.18321
• Pdf URL: https://arxiv.org/pdf/2508.18321
• Copy Paste: [[2508.18321]] LLMs Can't Handle Peer Pressure: Crumbling under Multi-Agent Social Interactions(https://arxiv.org/abs/2508.18321)
• Keywords: language model, llm, prompt, agent
• Abstract: Large language models (LLMs) are increasingly deployed in multi-agent systems (MAS) as components of collaborative intelligence, where peer interactions dynamically shape individual decision-making. Although prior work has focused on conformity bias, we extend the analysis to examine how LLMs form trust from previous impressions, resist misinformation, and integrate peer input during interaction, key factors for achieving collective intelligence under complex social dynamics. We present KAIROS, a benchmark simulating quiz contests with peer agents of varying reliability, offering fine-grained control over conditions such as expert-novice roles, noisy crowds, and adversarial peers. LLMs receive both historical interactions and current peer responses, allowing systematic investigation into how trust, peer action, and self-confidence influence decisions. As for mitigation strategies, we evaluate prompting, supervised fine-tuning, and reinforcement learning, Group Relative Policy Optimisation (GRPO), across multiple models. Our results reveal that GRPO with multi-agent context combined with outcome-based rewards and unconstrained reasoning achieves the best overall performance, but also decreases the robustness to social influence compared to Base models. The code and datasets are available at: this https URL.
• 摘要：大型语言模型（LLMS）越来越多地部署在多代理系统（MAS）中，作为协作智能的组成部分，其中同伴互动动态地塑造了个人决策。尽管先前的工作集中在整合偏见上，但我们扩展了分析，以研究LLMS如何从先前的印象中形成信任，抵抗错误信息，并在互动过程中整合同伴输入，这是在复杂的社会动态下实现集体智能的关键因素。我们介绍了Kairos，这是一个基准测试测验竞赛，具有不同的可靠性，对诸如专家 - novice角色，嘈杂的人群和对抗性同伴等条件等条件的精细控制。 LLM既获得历史互动又获得当前的同伴反应，从而可以对信任，同伴行动和自信心如何影响决策进行系统的调查。至于缓解策略，我们评估了跨多种模型的提示，监督微调和强化学习，小组相对政策优化（GRPO）。我们的结果表明，具有多代理上下文的GRPO结合基于结果的奖励和不受限制的推理可以实现最佳的总体表现，但与基本模型相比，社会影响力的稳健性也降低了。代码和数据集可在以下网址提供：此HTTPS URL。

Title: Language-Specific Layer Matters: Efficient Multilingual Enhancement for Large Vision-Language Models

• Authors: Yuchun Fan, Yilin Wang, Yongyu Mu, Lei Huang, Bei Li, Xiaocheng Feng, Tong Xiao, Jingbo Zhu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.18381
• Pdf URL: https://arxiv.org/pdf/2508.18381
• Copy Paste: [[2508.18381]] Language-Specific Layer Matters: Efficient Multilingual Enhancement for Large Vision-Language Models(https://arxiv.org/abs/2508.18381)
• Keywords: language model
• Abstract: Large vision-language models (LVLMs) have demonstrated exceptional capabilities in understanding visual information with human languages but also exhibit an imbalance in multilingual capabilities. In this work, we delve into the multilingual working pattern of LVLMs and identify a salient correlation between the multilingual understanding ability of LVLMs and language-specific neuron activations in shallow layers. Building on this insight, we introduce PLAST, a training recipe that achieves efficient multilingual enhancement for LVLMs by Precise LAnguage-Specific layers fine-Tuning. PLAST first identifies layers involved in multilingual understanding by monitoring language-specific neuron activations. These layers are then precisely fine-tuned with question-translation pairs to achieve multilingual alignment. Our empirical results on MM-Bench and MMMB demonstrate that PLAST effectively improves the multilingual capabilities of LVLMs and achieves significant efficiency with only 14% of the parameters tuned. Further analysis reveals that PLAST can be generalized to low-resource and complex visual reasoning tasks, facilitating the language-specific visual information engagement in shallow layers.
• 摘要：大型视觉模型（LVLM）表现出了出色的功能，可以用人类语言理解视觉信息，但在多语言能力方面也表现出不平衡。在这项工作中，我们深入研究了LVLMS的多语言工作模式，并确定LVLM的多语言理解能力与浅层中语言特定的神经元激活之间的显着相关性。在这种见解的基础上，我们介绍了Plast，这是一种培训配方，通过精确的语言特定层进行微调，可实现LVLM的有效多语言增强。塑料首先通过监测特定语言的神经元激活来识别参与多语言理解的层。然后，这些层通过问题翻译对精确地进行了微调，以实现多语言对齐。我们对MM Bench和MMMB的经验结果表明，塑料有效地提高了LVLMS的多语言能力，并且仅通过调整了14％的参数，才能达到明显的效率。进一步的分析表明，可以将塑料推广到低资源和复杂的视觉推理任务，从而促进浅层层中特定于语言的视觉信息参与。

Title: Backprompting: Leveraging Synthetic Production Data for Health Advice Guardrails

• Authors: Kellen Tan Cheng, Anna Lisa Gentile, Chad DeLuca, Guang-Jie Ren
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.18384
• Pdf URL: https://arxiv.org/pdf/2508.18384
• Copy Paste: [[2508.18384]] Backprompting: Leveraging Synthetic Production Data for Health Advice Guardrails(https://arxiv.org/abs/2508.18384)
• Keywords: language model, gpt, llm, prompt
• Abstract: The pervasiveness of large language models (LLMs) in enterprise settings has also brought forth a significant amount of risks associated with their usage. Guardrails technologies aim to mitigate this risk by filtering LLMs' input/output text through various detectors. However, developing and maintaining robust detectors faces many challenges, one of which is the difficulty in acquiring production-quality labeled data on real LLM outputs prior to deployment. In this work, we propose backprompting, a simple yet intuitive solution to generate production-like labeled data for health advice guardrails development. Furthermore, we pair our backprompting method with a sparse human-in-the-loop clustering technique to label the generated data. Our aim is to construct a parallel corpus roughly representative of the original dataset yet resembling real LLM output. We then infuse existing datasets with our synthetic examples to produce robust training data for our detector. We test our technique in one of the most difficult and nuanced guardrails: the identification of health advice in LLM output, and demonstrate improvement versus other solutions. Our detector is able to outperform GPT-4o by up to 3.73%, despite having 400x less parameters.
• 摘要：企业环境中大型语言模型（LLM）的普遍性也带来了与其使用相关的大量风险。 GuardRails Technologies旨在通过通过各种检测器过滤LLMS的输入/输出文本来降低这种风险。但是，开发和维护强大的探测器面临许多挑战，其中之一是在部署前在实际LLM输出上获取生产质量的数据的困难。在这项工作中，我们提出了反击，这是一种简单而直观的解决方案，用于生成类似生产的标签数据，以进行健康建议护栏开发。此外，我们将反弹方法与稀疏的人类聚类聚类技术配对，以标记生成的数据。我们的目的是构建一个平行语料库，该语料库大致代表了原始数据集但类似于实际LLM输出。然后，我们将现有数据集注入我们的合成示例，以为我们的检测器产生强大的培训数据。我们在最困难，最细微的护栏之一中测试了我们的技术：LLM输出中健康建议的识别，并证明了改进与其他解决方案。尽管参数减少了400倍，但我们的检测器能够胜过高达3.73％的GPT-4O。

Title: Latent Self-Consistency for Reliable Majority-Set Selection in Short- and Long-Answer Reasoning

• Authors: Jeong-seok Oh, Jay-yoon Lee
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.18395
• Pdf URL: https://arxiv.org/pdf/2508.18395
• Copy Paste: [[2508.18395]] Latent Self-Consistency for Reliable Majority-Set Selection in Short- and Long-Answer Reasoning(https://arxiv.org/abs/2508.18395)
• Keywords: language model, llm
• Abstract: Probabilistic decoding in Large Language Models (LLMs) often yields inconsistent outputs, particularly on complex or long-form questions. Self-Consistency (SC) mitigates this for short-form QA by majority voting over exact strings, whereas Universal Self-Consistency (USC) and Weighted Unigram Consistency Score (WUCS) extend to long-form responses but lose accuracy on short-form benchmarks. We introduce Latent Self-Consistency (LSC), which selects the most semantically consistent response using learnable token embeddings. A lightweight forward generation of summary tokens increases inference time by less than 1% and requires no changes to the model architecture. Across 6 short-form and 5 long-form reasoning benchmarks (e.g., MATH, MMLU, TruthfulQA), LSC surpasses SC, USC and WUCS on all short-form and long-form ones on average, while maintaining negligible computational overhead. These results position LSC as a practical consistency-selection method that works reliably across answer formats. Additionally, LSC provides well-calibrated confidence estimates, maintaining low Expected Calibration Error across both answer formats.
• 摘要：在大语言模型（LLM）中进行的概率解码通常会产生不一致的产出，尤其是在复杂或长期的问题上。自洽（SC）通过对精确字符串进行多数投票来减轻QA的质量，而普遍的自洽（USC）和加权的Unigram一致性得分（WUCS）扩展到长期响应，但在短形式基准上失去了准确性。我们介绍了潜在的自通抗性（LSC），它使用可学习的令牌嵌入选择了最一致的响应。轻巧的摘要代币的远期代码增加了推理时间不到1％，并且不需要更改模型体系结构。在6个短形式和5个长形式的推理基准（例如，Math，MMLU，ThrotfulQA），LSC平均而言，LSC超过SC，USC和WUCS，同时保持可忽略不计的计算高架。这些结果将LSC定位为一种实用的一致性选择方法，可在答案格式上可靠地工作。此外，LSC提供了良好的置信度估计值，可在两种答案格式中保持低预期的校准误差。

Title: How Reliable are LLMs for Reasoning on the Re-ranking task?

• Authors: Nafis Tanveer Islam, Zhiming Zhao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.18444
• Pdf URL: https://arxiv.org/pdf/2508.18444
• Copy Paste: [[2508.18444]] How Reliable are LLMs for Reasoning on the Re-ranking task?(https://arxiv.org/abs/2508.18444)
• Keywords: language model, llm
• Abstract: With the improving semantic understanding capability of Large Language Models (LLMs), they exhibit a greater awareness and alignment with human values, but this comes at the cost of transparency. Although promising results are achieved via experimental analysis, an in-depth understanding of the LLM's internal workings is unavoidable to comprehend the reasoning behind the re-ranking, which provides end users with an explanation that enables them to make an informed decision. Moreover, in newly developed systems with limited user engagement and insufficient ranking data, accurately re-ranking content remains a significant challenge. While various training methods affect the training of LLMs and generate inference, our analysis has found that some training methods exhibit better explainability than others, implying that an accurate semantic understanding has not been learned through all training methods; instead, abstract knowledge has been gained to optimize evaluation, which raises questions about the true reliability of LLMs. Therefore, in this work, we analyze how different training methods affect the semantic understanding of the re-ranking task in LLMs and investigate whether these models can generate more informed textual reasoning to overcome the challenges of transparency or LLMs and limited training data. To analyze the LLMs for re-ranking tasks, we utilize a relatively small ranking dataset from the environment and the Earth science domain to re-rank retrieved content. Furthermore, we also analyze the explainable information to see if the re-ranking can be reasoned using explainability.
• 摘要：随着大型语言模型（LLM）的语义理解能力的提高，它们表现出更大的认识和对人类价值观的一致性，但这是以透明度为代价的。尽管通过实验分析实现了有希望的结果，但不可避免地要深入了解LLM的内部运作方式，无法理解重新排名背后的推理，这为最终用户提供了一个解释，使他们能够做出明智的决定。此外，在新开发的系统中，用户参与度有限且排名数据不足，准确的重新排列内容仍然是一个重大挑战。尽管各种培训方法影响了LLM的培训并产生推理，但我们的分析发现，某些培训方法比其他培训方法更好，这意味着并未通过所有培训方法学习准确的语义理解。取而代之的是，已经获得了抽象知识来优化评估，从而提出了有关LLMS真实可靠性的问题。因此，在这项工作中，我们分析了不同的培训方法如何影响对LLM中重新排列任务的语义理解，并研究这些模型是否可以产生更明智的文本推理，以克服透明度或LLMS的挑战以及有限的培训数据。为了分析LLM的重新排列任务，我们利用来自环境和地球科学领域的相对较小的排名数据集来重新检索内容。此外，我们还分析了可解释的信息，以查看是否可以使用解释性来进行重新排列。

Title: Integrating gender inclusivity into large language models via instruction tuning

• Authors: Alina Wróblewska, Bartosz Żuk
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.18466
• Pdf URL: https://arxiv.org/pdf/2508.18466
• Copy Paste: [[2508.18466]] Integrating gender inclusivity into large language models via instruction tuning(https://arxiv.org/abs/2508.18466)
• Keywords: language model, llm, prompt
• Abstract: Imagine a language with masculine, feminine, and neuter grammatical genders, yet, due to historical and political conventions, masculine forms are predominantly used to refer to men, women and mixed-gender groups. This is the reality of contemporary Polish. A social consequence of this unfair linguistic system is that large language models (LLMs) trained on Polish texts inherit and reinforce this masculine bias, generating gender-imbalanced outputs. This study addresses this issue by tuning LLMs using the IPIS dataset, a collection of human-crafted gender-inclusive proofreading in Polish and Polish-to-English translation instructions. Grounded in a theoretical linguistic framework, we design a system prompt with explicit gender-inclusive guidelines for Polish. In our experiments, we IPIS-tune multilingual LLMs (Llama-8B, Mistral-7B and Mistral-Nemo) and Polish-specific LLMs (Bielik and PLLuM). Our approach aims to integrate gender inclusivity as an inherent feature of these models, offering a systematic solution to mitigate gender bias in Polish language generation.
• 摘要：想象一种具有男性，女性和中性语法性别的语言，但是由于历史和政治惯例，男性形式主要用于指代男人，女人和混合性别群体。这是当代抛光的现实。这种不公平的语言制度的社会后果是，对波兰文本训练的大型语言模型（LLM）继承并加强了这种男性偏见，从而产生了性别不平衡的输出。这项研究通过使用IPIS数据集调整LLM来解决此问题，该数据集是在波兰语和波兰语到英语翻译指令中的人力制作的包括性别的性别校对。我们以理论语言框架为基础，设计了一个系统提示，并具有针对波兰语的明确指南。在我们的实验中，我们iPis-Tune多语言LLM（Llama-8B，Mistral-7b和Mistral-Nemo）和波兰特异性LLM（Bielik和Pllum）。我们的方法旨在将性别包容性整合为这些模型的固有特征，从而提供系统的解决方案，以减轻波兰语言产生的性别偏见。

Title: Principled Detection of Hallucinations in Large Language Models via Multiple Testing

• Authors: Jiawei Li, Akshayaa Magesh, Venugopal V. Veeravalli
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.18473
• Pdf URL: https://arxiv.org/pdf/2508.18473
• Copy Paste: [[2508.18473]] Principled Detection of Hallucinations in Large Language Models via Multiple Testing(https://arxiv.org/abs/2508.18473)
• Keywords: language model, llm, hallucination
• Abstract: While Large Language Models (LLMs) have emerged as powerful foundational models to solve a variety of tasks, they have also been shown to be prone to hallucinations, i.e., generating responses that sound confident but are actually incorrect or even nonsensical. In this work, we formulate the problem of detecting hallucinations as a hypothesis testing problem and draw parallels to the problem of out-of-distribution detection in machine learning models. We propose a multiple-testing-inspired method to solve the hallucination detection problem, and provide extensive experimental results to validate the robustness of our approach against state-of-the-art methods.
• 摘要：尽管大型语言模型（LLM）已成为解决各种任务的强大基础模型，但它们也被证明是幻觉容易出现的，即产生听起来自信但实际上是不正确甚至是荒谬的响应。在这项工作中，我们提出了将幻觉作为假设测试问题的问题，并与机器学习模型中分布外检测的问题相似。我们提出了一种多次测试启发的方法来解决幻觉检测问题，并提供了广泛的实验结果，以验证我们针对最新方法的方法的鲁棒性。

Title: The Mind's Eye: A Multi-Faceted Reward Framework for Guiding Visual Metaphor Generation

• Authors: Girish A. Koushik, Fatemeh Nazarieh, Katherine Birch, Shenbin Qian, Diptesh Kanojia
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2508.18569
• Pdf URL: https://arxiv.org/pdf/2508.18569
• Copy Paste: [[2508.18569]] The Mind's Eye: A Multi-Faceted Reward Framework for Guiding Visual Metaphor Generation(https://arxiv.org/abs/2508.18569)
• Keywords: gpt, prompt
• Abstract: Visual metaphor generation is a challenging task that aims to generate an image given an input text metaphor. Inherently, it needs language understanding to bind a source concept with a target concept, in a way that preserves meaning while ensuring visual coherence. We propose a self-evaluating visual metaphor generation framework that focuses on metaphor alignment. Our self-evaluation approach combines existing metrics with our newly proposed metaphor decomposition score and a meaning alignment (MA) metric. Within this setup, we explore two novel approaches: a training-free pipeline that explicitly decomposes prompts into source-target-meaning (S-T-M) mapping for image synthesis, and a complementary training-based pipeline that improves alignment using our proposed self-evaluation reward schema, without any large-scale retraining. On the held-out test set, the training-free approach surpasses strong closed baselines (GPT-4o, Imagen) on decomposition, CLIP, and MA scores, with the training-based approach close behind. We evaluate our framework output using a user-facing study, and observed that participants preferred GPT-4o overall, while our training-free pipeline led open-source methods and edged Imagen on abstract metaphors. Our analyses show S-T-M prompting helps longer or more abstract metaphors, with closed models excelling on short, concrete cases; we also observe sensitivity to sampler settings. Overall, structured prompting and lightweight RL perform metaphor alignment well under modest compute, and remaining gaps to human preference appear driven by aesthetics and sampling.
• 摘要：视觉隐喻生成是一项具有挑战性的任务，旨在给定输入文本隐喻生成图像。固有地，它需要语言理解才能用目标概念绑定源概念，以确保视觉连贯性的同时保留意义。我们提出了一个自我评估的视觉隐喻生成框架，该框架着重于隐喻对齐。我们的自我评估方法将现有指标与我们新提出的隐喻分解评分和含义一致性（MA）度量相结合。在此设置中，我们探讨了两种新颖的方法：一条无训练的管道，将提示分解为图像合成的源目标含量（S-T-M）映射，以及基于互补的培训管道，可以使用我们建议的自我评估奖励架构改善对齐方式，而无需进行任何大规模验证。在持有的测试集中，无训练方法在分解，剪辑和MA分数上超过了强闭合基线（GPT-4O，Imagen），基于训练的方法近在咫尺。我们使用面向用户的研究评估了框架输出，并观察到参与者总体上更喜欢GPT-4O，而我们的无培训管道LED开源方法并在抽象隐喻上进行了质量图像。我们的分析表明，S-T-M提示有助于更长或更高的抽象隐喻，而封闭的模型在简短的具体案例中表现出色。我们还观察到对采样器设置的敏感性。总体而言，结构化提示和轻巧的RL在适度的计算下很好地执行了隐喻对准，并且剩余的差距似乎是由美学和抽样驱动的。

Title: What do language models model? Transformers, automata, and the format of thought

• Authors: Colin Klein
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.18598
• Pdf URL: https://arxiv.org/pdf/2508.18598
• Copy Paste: [[2508.18598]] What do language models model? Transformers, automata, and the format of thought(https://arxiv.org/abs/2508.18598)
• Keywords: language model, llm
• Abstract: What do large language models actually model? Do they tell us something about human capacities, or are they models of the corpus we've trained them on? I give a non-deflationary defence of the latter position. Cognitive science tells us that linguistic capabilities in humans rely supralinear formats for computation. The transformer architecture, by contrast, supports at best a linear formats for processing. This argument will rely primarily on certain invariants of the computational architecture of transformers. I then suggest a positive story about what transformers are doing, focusing on Liu et al. (2022)'s intriguing speculations about shortcut automata. I conclude with why I don't think this is a terribly deflationary story. Language is not (just) a means for expressing inner state but also a kind of 'discourse machine' that lets us make new language given appropriate context. We have learned to use this technology in one way; LLMs have also learned to use it too, but via very different means.
• 摘要：大型语言模型实际上是什么模型？他们是告诉我们有关人类能力的信息，还是我们已经训练过的语料库的模型？我对后一种立场进行了非流畅的防御。认知科学告诉我们，人类的语言能力依靠上线性格式进行计算。相比之下，变压器体系结构充其量最多支持用于处理的线性格式。该论点将主要依靠变压器计算架构的某些不变性。然后，我提出一个关于变压器在做什么的积极故事，重点是Liu等。 （2022）关于快捷自动机的有趣猜测。我总而言之，为什么我不认为这是一个非常轻巧的故事。语言不是（仅）表达内在状态的一种手段，而是一种“话语机器”，可以让我们在适当的上下文中制作新的语言。我们已经学会了以一种方式使用这项技术。 LLM也学会了使用它，但是通过非常不同的方式。

Title: Scaling Laws for Task-Stratified Knowledge in Post-Training Quantized Large Language Models

• Authors: Chenxi Zhou, Pengfei Cao, Jiang Li, Jun Zhao, Kang Liu
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.18609
• Pdf URL: https://arxiv.org/pdf/2508.18609
• Copy Paste: [[2508.18609]] Scaling Laws for Task-Stratified Knowledge in Post-Training Quantized Large Language Models(https://arxiv.org/abs/2508.18609)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) present significant deployment challenges due to their scale, with post-training quantization (PTQ) emerging as a practical compression solution. However, a comprehensive understanding of how PTQ precisely impacts diverse LLM knowledge capabilities remains elusive, and existing scaling laws for quantized models often overlook crucial PTQ-specific parameters and task-specific sensitivities. This paper addresses these gaps by conducting an extensive empirical investigation to establish task-stratified scaling laws. We disentangle LLM knowledge into memorization and utilization capabilities and develop a unified quantitative framework that incorporates model size, effective bit-width, calibration set size, and group size. Our central finding reveals that knowledge memorization exhibits markedly greater sensitivity to variations in effective bit-width, calibration set size, and model size compared to the more robust knowledge utilization. These findings offer a fine-grained understanding of PTQ's impact and provide guidance for developing knowledge-aware quantization strategies that can better preserve targeted cognitive functions.
• 摘要：大型语言模型（LLMS）由于其规模而引起了重大部署挑战，培训后量化（PTQ）作为一种实际的压缩解决方案出现。但是，对PTQ如何准确影响不同的LLM知识能力的全面理解仍然难以捉摸，并且对量化模型的现有比例定律通常忽略了至关重要的PTQ特异性参数和特定于任务的敏感性。本文通过进行广泛的实证研究来建立任务分层的缩放定律来解决这些差距。我们将LLM知识分解为记忆和利用能力，并开发一个统一的定量框架，该框架结合了模型大小，有效的位宽度，校准集大小和组大小。我们的中心发现表明，与更强大的知识利用相比，知识记忆对有效的位宽度，校准集大小和模型大小的变化显着敏感。这些发现对PTQ的影响有很好的了解，并为开发知识感知的量化策略提供了指导，以更好地保留有针对性的认知功能。

Title: Thinking Before You Speak: A Proactive Test-time Scaling Approach

• Authors: Cong Li, Wenchang Chai, Hejun Wu, Yan Pan, Pengxu Wei, Liang Lin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.18648
• Pdf URL: https://arxiv.org/pdf/2508.18648
• Copy Paste: [[2508.18648]] Thinking Before You Speak: A Proactive Test-time Scaling Approach(https://arxiv.org/abs/2508.18648)
• Keywords: language model, llm, prompt
• Abstract: Large Language Models (LLMs) often exhibit deficiencies with complex reasoning tasks, such as maths, which we attribute to the discrepancy between human reasoning patterns and those presented in the LLMs' training data. When dealing with complex problems, humans tend to think carefully before expressing solutions. However, they often do not articulate their inner thoughts, including their intentions and chosen methodologies. Consequently, critical insights essential for bridging reasoning steps may be absent in training data collected from human sources. To bridge this gap, we proposes inserting \emph{insight}s between consecutive reasoning steps, which review the status and initiate the next reasoning steps. Unlike prior prompting strategies that rely on a single or a workflow of static prompts to facilitate reasoning, \emph{insight}s are \emph{proactively} generated to guide reasoning processes. We implement our idea as a reasoning framework, named \emph{Thinking Before You Speak} (TBYS), and design a pipeline for automatically collecting and filtering in-context examples for the generation of \emph{insight}s, which alleviates human labeling efforts and fine-tuning overheads. Experiments on challenging mathematical datasets verify the effectiveness of TBYS. Project website: this https URL
• 摘要：大型语言模型（LLM）经常表现出具有复杂推理任务的缺陷，例如数学，我们将其归因于人类推理模式与LLMS培训数据中呈现的差异。在处理复杂问题时，人类倾向于在表达解决方案之前仔细考虑。但是，他们通常不会表达自己的内在思想，包括他们的意图和选择的方法。因此，从人类来源收集的培训数据中可能不存在桥接推理步骤必不可少的关键见解。为了弥合这一差距，我们建议在连续的推理步骤之间插入\ emph {Insight}，这些步骤会审查状态并启动下一个推理步骤。与先前提示依靠静态提示的单个或工作流以促进推理的策略不同，生成的\ emph {insight} s是\ emph {主动}来指导推理过程。我们将思想作为推理框架实施，称为\ emph {在说话之前}（TBYS），并设计了一条管道，用于自动收集和过滤在\ emph {Insight} s中的文本示例，以减轻人类的标签努力和微调的高架架设。有关挑战性数学数据集的实验验证了TBY的有效性。项目网站：此HTTPS URL

Title: Breaking the Trade-Off Between Faithfulness and Expressiveness for Large Language Models

• Authors: Chenxu Yang, Qingyi Si, Zheng Lin
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.18651
• Pdf URL: https://arxiv.org/pdf/2508.18651
• Copy Paste: [[2508.18651]] Breaking the Trade-Off Between Faithfulness and Expressiveness for Large Language Models(https://arxiv.org/abs/2508.18651)
• Keywords: language model, llm, hallucination
• Abstract: Grounding responses in external knowledge represents an effective strategy for mitigating hallucinations in Large Language Models (LLMs). However, current LLMs struggle to seamlessly integrate knowledge while simultaneously maintaining faithfulness (or fidelity) and expressiveness, capabilities that humans naturally possess. This limitation results in outputs that either lack support from external knowledge, thereby compromising faithfulness, or appear overly verbose and unnatural, thus sacrificing expressiveness. In this work, to break the trade-off between faithfulness and expressiveness, we propose Collaborative Decoding (CoDe), a novel approach that dynamically integrates output probabilities generated with and without external knowledge. This integration is guided by distribution divergence and model confidence, enabling the selective activation of relevant and reliable expressions from the model's internal parameters. Furthermore, we introduce a knowledge-aware reranking mechanism that prevents over-reliance on prior parametric knowledge while ensuring proper utilization of provided external information. Through comprehensive experiments, our plug-and-play CoDe framework demonstrates superior performance in enhancing faithfulness without compromising expressiveness across diverse LLMs and evaluation metrics, validating both its effectiveness and generalizability.
• 摘要：外部知识的基础响应代表了缓解大语言模型（LLM）幻觉的有效策略。但是，当前的LLM努力无缝整合知识，同时保持忠诚（或忠诚）和表现力，人类自然具有的能力。这种局限性导致输出要么缺乏外部知识的支持，从而损害了忠诚，或者出现过于冗长和不自然，从而牺牲了表现力。在这项工作中，为了打破忠诚和表现力之间的权衡，我们提出了协作解码（代码），这是一种动态整合以有或没有外部知识生成的输出概率的新颖方法。该集成以分布差异和模型置信度为指导，从而可以从模型的内部参数中选择性激活相关和可靠的表达式。此外，我们介绍了一种知识感知的重新依据机制，该机制可以防止过度依赖先前的参数知识，同时确保适当利用提供的外部信息。通过全面的实验，我们的插件代码框架在增强忠诚度方面表现出了卓越的表现，而不会在不同的LLM和评估指标上损害表现力，从而验证了其有效性和普遍性。

Title: Emotion Omni: Enabling Empathetic Speech Response Generation through Large Language Models

• Authors: Haoyu Wang, Guangyan Zhang, Jiale Chen, Jingyu Li, Yuehai Wang, Yiwen Guo
• Subjects: cs.CL, cs.SD
• Abstract URL: https://arxiv.org/abs/2508.18655
• Pdf URL: https://arxiv.org/pdf/2508.18655
• Copy Paste: [[2508.18655]] Emotion Omni: Enabling Empathetic Speech Response Generation through Large Language Models(https://arxiv.org/abs/2508.18655)
• Keywords: language model, llm
• Abstract: With the development of speech large language models (speech LLMs), users can now interact directly with assistants via speech. However, most existing models simply convert the response content into speech without fully understanding the rich emotional and paralinguistic cues embedded in the user's query. In many cases, the same sentence can have different meanings depending on the emotional expression. Furthermore, emotional understanding is essential for improving user experience in human-machine interaction. Currently, most speech LLMs with empathetic capabilities are trained on massive datasets. This approach requires vast amounts of data and significant computational resources. Therefore, a key challenge lies in how to develop a speech LLM capable of generating empathetic responses with limited data and without the need for large-scale training. To address this challenge, we propose Emotion Omni, a novel model architecture designed to understand the emotional content of user speech input and generate empathetic speech responses. Additionally, we developed a data generation pipeline based on an open-source TTS framework to construct a 200k emotional dialogue dataset, which supports the construction of an empathetic speech assistant. The demos are available at this https URL
• 摘要：随着语音大语模型（语音LLM）的发展，用户现在可以通过语音直接与助手互动。但是，大多数现有模型只需将响应内容转换为语音，而无需完全了解用户查询中嵌入的丰富情感和副语言提示。在许多情况下，根据情感表达，同一句子可能具有不同的含义。此外，情感理解对于改善人机互动中的用户体验至关重要。目前，大多数具有善解人意能力的语音LLM都在大规模数据集上进行了培训。这种方法需要大量数据和大量的计算资源。因此，一个关键的挑战在于如何开发能够使用有限的数据产生同情反应的语音LLM，而无需大规模培训。为了应对这一挑战，我们提出了情感Omni，这是一种新颖的模型体系结构，旨在了解用户语音输入的情感内容并产生善解人意的语音回答。此外，我们基于开源TTS框架开发了数据生成管道，以构建200K情感对话数据集，该数据集支持促进性语音助手的构建。该演示可在此HTTPS URL上找到

Title: Tailored Teaching with Balanced Difficulty: Elevating Reasoning in Multimodal Chain-of-Thought via Prompt Curriculum

• Authors: Xinglong Yang, Quan Feng, Zhongying Pan, Xiang Chen, Yu Tian, Wentong Li, Shuofei Qiao, Yuxia Geng, Xingyu Zhao, Sheng-Jun Huang
• Subjects: cs.CL, cs.AI, cs.MM
• Abstract URL: https://arxiv.org/abs/2508.18673
• Pdf URL: https://arxiv.org/pdf/2508.18673
• Copy Paste: [[2508.18673]] Tailored Teaching with Balanced Difficulty: Elevating Reasoning in Multimodal Chain-of-Thought via Prompt Curriculum(https://arxiv.org/abs/2508.18673)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: The effectiveness of Multimodal Chain-of-Thought (MCoT) prompting is often limited by the use of randomly or manually selected examples. These examples fail to account for both model-specific knowledge distributions and the intrinsic complexity of the tasks, resulting in suboptimal and unstable model performance. To address this, we propose a novel framework inspired by the pedagogical principle of "tailored teaching with balanced difficulty". We reframe prompt selection as a prompt curriculum design problem: constructing a well ordered set of training examples that align with the model's current capabilities. Our approach integrates two complementary signals: (1) model-perceived difficulty, quantified through prediction disagreement in an active learning setup, capturing what the model itself finds challenging; and (2) intrinsic sample complexity, which measures the inherent difficulty of each question-image pair independently of any model. By jointly analyzing these signals, we develop a difficulty-balanced sampling strategy that ensures the selected prompt examples are diverse across both dimensions. Extensive experiments conducted on five challenging benchmarks and multiple popular Multimodal Large Language Models (MLLMs) demonstrate that our method yields substantial and consistent improvements and greatly reduces performance discrepancies caused by random sampling, providing a principled and robust approach for enhancing multimodal reasoning.
• 摘要：多模式链（MCOT）提示的有效性通常受到随机或手动选择的示例的使用限制。这些示例无法说明特定于模型的知识分布和任务的内在复杂性，从而导致了次优和不稳定的模型性能。为了解决这个问题，我们提出了一个新颖的框架，该框架灵感来自“平衡难度量身定制的教学”的教学原则。我们将及时选择作为及时的课程设计问题进行重新定位：构建一组有序的培训示例，这些培训示例与模型的当前功能保持一致。我们的方法集成了两个互补信号：（1）模型感知的难度，通过在主动学习设置中的预测分歧来量化，捕获模型本身发现具有挑战性的内容； （2）内在样品复杂性，它独立于任何模型来测量每个问题图对的固有难度。通过共同分析这些信号，我们制定了难以平衡的抽样策略，以确保所选及时的示例在两个维度上都是多样的。对五个具有挑战性的基准和多种流行的多模式模型（MLLM）进行的广泛实验表明，我们的方法可实现实质性和一致的改进，并大大降低了由随机抽样引起的性能差异，从而提供了一种原则性和强大的方法来增强多模式推理。

Title: Knowing or Guessing? Robust Medical Visual Question Answering via Joint Consistency and Contrastive Learning

• Authors: Songtao Jiang, Yuxi Chen, Sibo Song, Yan Zhang, Yeying Jin, Yang Feng, Jian Wu, Zuozhu Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.18687
• Pdf URL: https://arxiv.org/pdf/2508.18687
• Copy Paste: [[2508.18687]] Knowing or Guessing? Robust Medical Visual Question Answering via Joint Consistency and Contrastive Learning(https://arxiv.org/abs/2508.18687)
• Keywords: language model
• Abstract: In high-stakes medical applications, consistent answering across diverse question phrasings is essential for reliable diagnosis. However, we reveal that current Medical Vision-Language Models (Med-VLMs) exhibit concerning fragility in Medical Visual Question Answering, as their answers fluctuate significantly when faced with semantically equivalent rephrasings of medical questions. We attribute this to two limitations: (1) insufficient alignment of medical concepts, leading to divergent reasoning patterns, and (2) hidden biases in training data that prioritize syntactic shortcuts over semantic understanding. To address these challenges, we construct RoMed, a dataset built upon original VQA datasets containing 144k questions with variations spanning word-level, sentence-level, and semantic-level perturbations. When evaluating state-of-the-art (SOTA) models like LLaVA-Med on RoMed, we observe alarming performance drops (e.g., a 40\% decline in Recall) compared to original VQA benchmarks, exposing critical robustness gaps. To bridge this gap, we propose Consistency and Contrastive Learning (CCL), which integrates two key components: (1) knowledge-anchored consistency learning, aligning Med-VLMs with medical knowledge rather than shallow feature patterns, and (2) bias-aware contrastive learning, mitigating data-specific priors through discriminative representation refinement. CCL achieves SOTA performance on three popular VQA benchmarks and notably improves answer consistency by 50\% on the challenging RoMed test set, demonstrating significantly enhanced robustness. Code will be released.
• 摘要：在高风险的医疗应用中，跨不同问题短语的一致答案对于可靠的诊断至关重要。但是，我们揭示了当前的医学视觉模型（MED-VLM）在医学视觉问题回答中表现出脆弱性，因为当面对语义上等效的医学问题时，它们的答案显着波动。我们将其归因于两个局限性：（1）医学概念的一致性不足，导致推理模式不同，以及（2）培训数据中隐藏的偏见，将句法快捷​​方式优先于语义理解。为了应对这些挑战，我们构建了一个romed，这是一个基于原始VQA数据集构建的，该数据集包含144K问题，其中包含跨越单词级别，句子级别和语义级扰动的变化。当评估诸如Romed的Llava-Med之类的最新模型（SOTA）模型时，我们观察到令人震惊的性能下降（例如，召回下降40 \％）与原始的VQA基准相比，揭示了关键的稳健性差距。为了弥合这一差距，我们提出了一致性和对比性学习（CCL），该学习集成了两个关键组成部分：（1）知识锚定的一致性学习，将Med-vlms与医学知识而不是浅的特征模式保持一致，以及（2）偏见 - 意识到的对比性学习，通过歧视性代表来减轻数据特定的数据。 CCL在三个流行的VQA基准上实现了SOTA性能，并且在挑战性的Romed测试集中显着提高了答案一致性，并提高了50 \％，这表明鲁棒性显着增强。代码将发布。

Title: Attention2Probability: Attention-Driven Terminology Probability Estimation for Robust Speech-to-Text System

• Authors: Yanfan Du, Jun Zhang, Bin Wang, Jin Qiu, Lu Huang, Yuan Ge, Xiaoqian Liu, Tong Xiao, Jingbo Zhu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.18701
• Pdf URL: https://arxiv.org/pdf/2508.18701
• Copy Paste: [[2508.18701]] Attention2Probability: Attention-Driven Terminology Probability Estimation for Robust Speech-to-Text System(https://arxiv.org/abs/2508.18701)
• Keywords: language model
• Abstract: Recent advances in speech large language models (SLMs) have improved speech recognition and translation in general domains, but accurately generating domain-specific terms or neologisms remains challenging. To address this, we propose Attention2Probability: attention-driven terminology probability estimation for robust speech-to-text system, which is lightweight, flexible, and accurate. Attention2Probability converts cross-attention weights between speech and terminology into presence probabilities, and it further employs curriculum learning to enhance retrieval accuracy. Furthermore, to tackle the lack of data for speech-to-text tasks with terminology intervention, we create and release a new speech dataset with terminology to support future research in this area. Experimental results show that Attention2Probability significantly outperforms the VectorDB method on our test set. Specifically, its maximum recall rates reach 92.57% for Chinese and 86.83% for English. This high recall is achieved with a latency of only 8.71ms per query. Intervening in SLMs' recognition and translation tasks using Attention2Probability-retrieved terms improves terminology accuracy by 6-17%, while revealing that the current utilization of terminology by SLMs has limitations.
• 摘要：语音大语言模型（SLM）的最新进展改善了一般领域中的语音识别和翻译，但是准确地产生了特定于领域的术语或新词，仍然具有挑战性。为了解决这个问题，我们提出了注意力2探针：鲁棒语音到文本系统的注意力驱动术语概率估计，这是轻巧，灵活和准确的。注意力2将语音和术语之间的交叉注意权重转化为存在概率，并且进一步采用了课程学习来提高检索准确性。此外，为了通过术语干预解决缺乏语音到文本任务的数据，我们创建并发布了一个具有术语的新的语音数据集，以支持该领域的未来研究。实验结果表明，注意力2明显优于我们测试集中的VectordB方法。具体而言，其最大召回率达到中文的92.57％，英语的召回率达到86.83％。每个查询的延迟仅为8.71ms，可以实现这种高召回率。介入SLM的识别和使用activation2probability-Retresed术语的识别和翻译任务提高了术语准确性6-17％，同时揭示了SLMS当前对术语的利用率有局限性。

Title: Filtering for Creativity: Adaptive Prompting for Multilingual Riddle Generation in LLMs

• Authors: Duy Le, Kent Ziti, Evan Girard-Sun, Sean O'Brien, Vasu Sharma, Kevin Zhu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.18709
• Pdf URL: https://arxiv.org/pdf/2508.18709
• Copy Paste: [[2508.18709]] Filtering for Creativity: Adaptive Prompting for Multilingual Riddle Generation in LLMs(https://arxiv.org/abs/2508.18709)
• Keywords: language model, gpt, llm, prompt, chain-of-thought
• Abstract: Multilingual riddle generation challenges large language models (LLMs) to balance cultural fluency with creative abstraction. Standard prompting strategies -- zero-shot, few-shot, chain-of-thought -- tend to reuse memorized riddles or perform shallow paraphrasing. We introduce Adaptive Originality Filtering (AOF), a prompting framework that filters redundant generations using cosine-based similarity rejection, while enforcing lexical novelty and cross-lingual fidelity. Evaluated across three LLMs and four language pairs, AOF-enhanced GPT-4o achieves \texttt{0.177} Self-BLEU and \texttt{0.915} Distinct-2 in Japanese, signaling improved lexical diversity and reduced redundancy compared to other prompting methods and language pairs. Our findings show that semantic rejection can guide culturally grounded, creative generation without task-specific fine-tuning.
• 摘要：多语言谜语生成挑战大型语言模型（LLMS），以平衡文化流利性和创造性抽象。标准提示策略 - 零射，很少，经营链 - 往往会重复使用记忆的谜语或进行浅释义。我们介绍了自适应原创性过滤（AOF），这是一种提示框架，使用基于余弦的相似性拒绝过滤冗余，同时实施词汇新颖性和跨语言保真度。在三个LLM和四个语言对中进行了评估，AOF增强的GPT-4O实现了\ Texttt {0.177} selfbleu和\ texttt {0.915} {0.915} distintion-2在日语中，与其他提示方法和其他提示方法和语言对相比，信号提高了词汇多样性，并减少了冗余。我们的发现表明，语义拒绝可以指导文化基础，创造性的生成，而无需特定于任务的微调。

Title: EMMM, Explain Me My Model! Explainable Machine Generated Text Detection in Dialogues

• Authors: Angela Yifei Yuan, Haoyi Li, Soyeon Caren Han, Christopher Leckie
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.18715
• Pdf URL: https://arxiv.org/pdf/2508.18715
• Copy Paste: [[2508.18715]] EMMM, Explain Me My Model! Explainable Machine Generated Text Detection in Dialogues(https://arxiv.org/abs/2508.18715)
• Keywords: language model, llm
• Abstract: The rapid adoption of large language models (LLMs) in customer service introduces new risks, as malicious actors can exploit them to conduct large-scale user impersonation through machine-generated text (MGT). Current MGT detection methods often struggle in online conversational settings, reducing the reliability and interpretability essential for trustworthy AI deployment. In customer service scenarios where operators are typically non-expert users, explanation become crucial for trustworthy MGT detection. In this paper, we propose EMMM, an explanation-then-detection framework that balances latency, accuracy, and non-expert-oriented interpretability. Experimental results demonstrate that EMMM provides explanations accessible to non-expert users, with 70\% of human evaluators preferring its outputs, while achieving competitive accuracy compared to state-of-the-art models and maintaining low latency, generating outputs within 1 second. Our code and dataset are open-sourced at this https URL.
• 摘要：在客户服务中，大型语言模型（LLM）的迅速采用引入了新的风险，因为恶意演员可以通过机器生成的文本（MGT）利用他们来进行大规模用户模仿。当前的MGT检测方法通常在在线对话环境中挣扎，从而降低了值得信赖的AI部署必不可少的可靠性和解释性。在运营商通常是非专家用户的客户服务方案中，解释对于值得信赖的MGT检测至关重要。在本文中，我们提出了EMMM，这是一个解释 - 然后进行检测框架，该框架可以平衡潜伏期，准确性和非专家的解释性。实验结果表明，EMMM提供了非专家用户可以访问的解释，其中70％的人类评估者更喜欢其输出，同时与最先进的模型相比实现竞争精度并保持低延迟，并在1秒内产生输出。我们的代码和数据集在此HTTPS URL上开源。

Title: Beyond Quality: Unlocking Diversity in Ad Headline Generation with Large Language Models

• Authors: Chang Wang, Siyu Yan, Depeng Yuan, Yuqi Chen, Yanhua Huang, Yuanhang Zheng, Shuhao Li, Yinqi Zhang, Kedi Chen, Mingrui Zhu, Ruiwen Xu
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.18739
• Pdf URL: https://arxiv.org/pdf/2508.18739
• Copy Paste: [[2508.18739]] Beyond Quality: Unlocking Diversity in Ad Headline Generation with Large Language Models(https://arxiv.org/abs/2508.18739)
• Keywords: language model, llm
• Abstract: The generation of ad headlines plays a vital role in modern advertising, where both quality and diversity are essential to engage a broad range of audience segments. Current approaches primarily optimize language models for headline quality or click-through rates (CTR), often overlooking the need for diversity and resulting in homogeneous outputs. To address this limitation, we propose DIVER, a novel framework based on large language models (LLMs) that are jointly optimized for both diversity and quality. We first design a semantic- and stylistic-aware data generation pipeline that automatically produces high-quality training pairs with ad content and multiple diverse headlines. To achieve the goal of generating high-quality and diversified ad headlines within a single forward pass, we propose a multi-stage multi-objective optimization framework with supervised fine-tuning (SFT) and reinforcement learning (RL). Experiments on real-world industrial datasets demonstrate that DIVER effectively balances quality and diversity. Deployed on a large-scale content-sharing platform serving hundreds of millions of users, our framework improves advertiser value (ADVV) and CTR by 4.0% and 1.4%.
• 摘要：广告头条的一代在现代广告中起着至关重要的作用，在现代广告中，质量和多样性对于参与广泛的受众群体至关重要。当前的方法主要针对标题质量或点击率（CTR）优化语言模型，通常忽略了对多样性的需求，并导致产出均匀的产出。为了解决这一限制，我们提出了Diver，这是一个基于大型语言模型（LLMS）的新型框架，这些框架均已针对多样性和质量共同优化。我们首先设计了一种语义和风格感知的数据生成管道，该管道自动生产具有广告内容和多种不同标题的高质量培训对。为了实现单个前向通行证中产生高质量和多元化的广告头条的目标，我们提出了一个多阶段的多阶段多目标优化框架，并使用有监督的微调（SFT）和加固学习（RL）。对现实世界工业数据集的实验表明，潜水员有效地平衡了质量和多样性。我们的框架部署在一个为数亿用户提供服务的大规模内容共享平台上，将广告商价值（ADVV）和CTR提高了4.0％和1.4％。

Title: Chronological Passage Assembling in RAG framework for Temporal Question Answering

• Authors: Byeongjeong Kim, Jeonghyun Park, Joonho Yang, Hwanhee Lee
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.18748
• Pdf URL: https://arxiv.org/pdf/2508.18748
• Copy Paste: [[2508.18748]] Chronological Passage Assembling in RAG framework for Temporal Question Answering(https://arxiv.org/abs/2508.18748)
• Keywords: retrieval-augmented generation
• Abstract: Long-context question answering over narrative tasks is challenging because correct answers often hinge on reconstructing a coherent timeline of events while preserving contextual flow in a limited context window. Retrieval-augmented generation (RAG) indexing methods aim to address this challenge by selectively retrieving only necessary document segments. However, narrative texts possess unique characteristics that limit the effectiveness of these existing approaches. Specifically, understanding narrative texts requires more than isolated segments, as the broader context and sequential relationships between segments are crucial for comprehension. To address these limitations, we propose ChronoRAG, a novel RAG framework specialized for narrative texts. This approach focuses on two essential aspects: refining dispersed document information into coherent and structured passages, and preserving narrative flow by explicitly capturing and maintaining the temporal order among retrieved passages. We empirically demonstrate the effectiveness of ChronoRAG through experiments on the NarrativeQA dataset, showing substantial improvements in tasks requiring both factual identification and comprehension of complex sequential relationships, underscoring that reasoning over temporal order is crucial in resolving narrative QA.
• 摘要：对叙事任务的回答的长篇文章问题是具有挑战性的，因为正确的答案通常取决于重建事件的连贯时间表，同时在有限的上下文窗口中保留上下文流程。检索增强的生成（RAG）索引方法旨在通过选择性检索必要的文档段来解决这一挑战。但是，叙事文本具有限制这些现有方法的有效性的独特特征。具体而言，理解叙事文本不仅需要孤立的细分，因为段之间的更广泛的背景和顺序关系对于理解至关重要。为了解决这些局限性，我们提出了Chronorag，这是一个专门用于叙事文本的新颖的RAG框架。这种方法侧重于两个基本方面：将分散的文档信息提高到连贯和结构化的段落中，并通过明确捕获和维护所检索的段落之间的时间顺序来保存叙事流程。我们通过实验证明了Chronorag通过对叙事QA数据集的实验的有效性，显示了需要对复杂顺序关系的事实识别和理解的任务的实质性改进，这强调了在时间上的推理对于解决叙述质量质量质量质量质量质量质量质量质量质量质量至关重要。

Title: ThinkDial: An Open Recipe for Controlling Reasoning Effort in Large Language Models

• Authors: Qianyu He, Siyu Yuan, Xuefeng Li, Mingxuan Wang, Jiangjie Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.18773
• Pdf URL: https://arxiv.org/pdf/2508.18773
• Copy Paste: [[2508.18773]] ThinkDial: An Open Recipe for Controlling Reasoning Effort in Large Language Models(https://arxiv.org/abs/2508.18773)
• Keywords: language model, gpt, llm, chain-of-thought
• Abstract: Large language models (LLMs) with chain-of-thought reasoning have demonstrated remarkable problem-solving capabilities, but controlling their computational effort remains a significant challenge for practical deployment. Recent proprietary systems like OpenAI's gpt-oss series have introduced discrete operational modes for intuitive reasoning control, but the open-source community has largely failed to achieve such capabilities. In this paper, we introduce ThinkDial, the first open-recipe end-to-end framework that successfully implements gpt-oss-style controllable reasoning through discrete operational modes. Our system enables seamless switching between three distinct reasoning regimes: High mode (full reasoning capability), Medium mode (50 percent token reduction with <10 percent performance degradation), and Low mode (75 percent token reduction with <15 percent performance degradation). We achieve this through an end-to-end training paradigm that integrates budget-mode control throughout the entire pipeline: budget-mode supervised fine-tuning that embeds controllable reasoning capabilities directly into the learning process, and two-phase budget-aware reinforcement learning with adaptive reward shaping. Extensive experiments demonstrate that ThinkDial achieves target compression-performance trade-offs with clear response length reductions while maintaining performance thresholds. The framework also exhibits strong generalization capabilities on out-of-distribution tasks.
• 摘要：具有经过经过经验推理的大型语言模型（LLM）表现出了显着的解决问题的能力，但是控制其计算工作仍然是实践部署的重大挑战。诸如OpenAI的GPT-Oss系列之类的最新专有系统引入了直观推理控制的离散操作模式，但开源社区在很大程度上未能实现此类能力。在本文中，我们介绍了ThinkDial，这是第一个开放式端口端到端框架，通过离散操作模式成功地实现了GPT-oss式可控推理。我们的系统可以在三个不同的推理方案之间进行无缝切换：高模式（完全推理能力），中型模式（50％的令牌减少，<10％的性能降低）和低模式（75％的令牌减少，<15％的性能降低）。我们通过端到端的培训范式在整个管道中整合预算模式控制的范式：预算模式监督的微型调整，将可控制的推理能力直接嵌入学习过程中，以及两相预算吸引的增强学习和自适应奖励的奖励。广泛的实验表明，思维式实现目标压缩性绩效的权衡，并保持响应长度的明确降低，同时保持绩效阈值。该框架在分布任务上还具有强大的概括能力。

Title: Harnessing Rule-Based Reinforcement Learning for Enhanced Grammatical Error Correction

• Authors: Yilin Li, Xunjian Yin, Yilin Chen, Xiaojun Wan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.18780
• Pdf URL: https://arxiv.org/pdf/2508.18780
• Copy Paste: [[2508.18780]] Harnessing Rule-Based Reinforcement Learning for Enhanced Grammatical Error Correction(https://arxiv.org/abs/2508.18780)
• Keywords: llm
• Abstract: Grammatical error correction is a significant task in NLP. Traditional methods based on encoder-decoder models have achieved certain success, but the application of LLMs in this field is still underexplored. Current research predominantly relies on supervised fine-tuning to train LLMs to directly generate the corrected sentence, which limits the model's powerful reasoning ability. To address this limitation, we propose a novel framework based on Rule-Based RL. Through experiments on the Chinese datasets, our Rule-Based RL framework achieves \textbf{state-of-the-art }performance, with a notable increase in \textbf{recall}. This result clearly highlights the advantages of using RL to steer LLMs, offering a more controllable and reliable paradigm for future development in GEC.
• 摘要：语法误差校正是NLP的重要任务。基于编码器模型的传统方法已取得了一定的成功，但是LLM在该领域的应用仍未得到充分展望。当前的研究主要依赖于监督的微调来训练LLM直接生成校正后的句子，这限制了模型的强大推理能力。为了解决此限制，我们提出了一个基于基于规则的RL的新框架。通过中文数据集的实验，我们基于规则的RL框架实现了\ textbf {最新的}性能，并显着提高了\ textbf {reckle}。该结果显然强调了使用RL转向LLM的优势，为GEC的未来开发提供了更可控制和可靠的范式。

Title: Controllable Conversational Theme Detection Track at DSTC 12

• Authors: Igor Shalyminov, Hang Su, Jake Vincent, Siffi Singh, Jason Cai, James Gung, Raphael Shu, Saab Mansour
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.18783
• Pdf URL: https://arxiv.org/pdf/2508.18783
• Copy Paste: [[2508.18783]] Controllable Conversational Theme Detection Track at DSTC 12(https://arxiv.org/abs/2508.18783)
• Keywords: language model, llm
• Abstract: Conversational analytics has been on the forefront of transformation driven by the advances in Speech and Natural Language Processing techniques. Rapid adoption of Large Language Models (LLMs) in the analytics field has taken the problems that can be automated to a new level of complexity and scale. In this paper, we introduce Theme Detection as a critical task in conversational analytics, aimed at automatically identifying and categorizing topics within conversations. This process can significantly reduce the manual effort involved in analyzing expansive dialogs, particularly in domains like customer support or sales. Unlike traditional dialog intent detection, which often relies on a fixed set of intents for downstream system logic, themes are intended as a direct, user-facing summary of the conversation's core inquiry. This distinction allows for greater flexibility in theme surface forms and user-specific customizations. We pose Controllable Conversational Theme Detection problem as a public competition track at Dialog System Technology Challenge (DSTC) 12 -- it is framed as joint clustering and theme labeling of dialog utterances, with the distinctive aspect being controllability of the resulting theme clusters' granularity achieved via the provided user preference data. We give an overview of the problem, the associated dataset and the evaluation metrics, both automatic and human. Finally, we discuss the participant teams' submissions and provide insights from those. The track materials (data and code) are openly available in the GitHub repository.
• 摘要：对话分析一直处于转型和自然语言处理技术的进步所驱动的最前沿。在分析领域快速采用大型语言模型（LLM）已将可以自动化的问题提高到新的复杂性和规模水平。在本文中，我们将主题检测作为对话分析中的关键任务介绍，旨在自动识别和分类对话中的主题。此过程可以大大减少分析广泛对话的手动工作，尤其是在客户支持或销售等领域中。与传统对话框的意图检测通常依赖于对下游系统逻辑的固定意图，主题旨在作为对话核心查询的直接，面向用户的摘要。这种区别允许在主题表面形式和特定于用户的自定义方面具有更大的灵活性。我们在对话系统技术挑战赛（DSTC）中提出了可控的对话主题检测问题，作为公共竞争轨道12-它被构架为对话言话语的联合聚类和主题标签，其独特的方面是由此产生的主题群集通过提供的用户偏好数据实现的粒度的可控性。我们概述了自动和人类的问题，相关的数据集和评估指标。最后，我们讨论了参与者团队的意见，并提供了这些参与者的见解。轨道材料（数据和代码）在GitHub存储库中公开可用。

Title: LaTeXTrans: Structured LaTeX Translation with Multi-Agent Coordination

• Authors: Ziming Zhu, Chenglong Wang, Shunjie Xing, Yifu Huo, Fengning Tian, Quan Du, Di Yang, Chunliang Zhang, Tong Xiao, Jingbo Zhu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.18791
• Pdf URL: https://arxiv.org/pdf/2508.18791
• Copy Paste: [[2508.18791]] LaTeXTrans: Structured LaTeX Translation with Multi-Agent Coordination(https://arxiv.org/abs/2508.18791)
• Keywords: agent
• Abstract: Despite the remarkable progress of modern machine translation (MT) systems on general-domain texts, translating structured LaTeX-formatted documents remains a significant challenge. These documents typically interleave natural language with domain-specific syntax, such as mathematical equations, tables, figures, and cross-references, all of which must be accurately preserved to maintain semantic integrity and compilability. In this paper, we introduce LaTeXTrans, a collaborative multi-agent system designed to address this challenge. LaTeXTrans ensures format preservation, structural fidelity, and terminology consistency through six specialized agents: 1) a Parser that decomposes LaTeX into translation-friendly units via placeholder substitution and syntax filtering; 2) a Translator, Validator, Summarizer, and Terminology Extractor that work collaboratively to ensure context-aware, self-correcting, and terminology-consistent translations; 3) a Generator that reconstructs the translated content into well-structured LaTeX documents. Experimental results demonstrate that LaTeXTrans can outperform mainstream MT systems in both translation accuracy and structural fidelity, offering an effective and practical solution for translating LaTeX-formatted documents.
• 摘要：尽管现代机器翻译（MT）系统在通用域文本上取得了显着进展，但翻译结构化的乳胶形式的文档仍然是一个重大挑战。这些文档通常将自然语言与域特异性语法交织在一起，例如数学方程式，表，数字和交叉引用，所有这些都必须准确保留以维持语义完整性和汇编。在本文中，我们介绍了乳胶，这是一种旨在应对这一挑战的协作多代理系统。通过六种专用代理商确保格式保存，结构保真度和术语一致性：1）通过占位持有人的替代和语法过滤将乳胶分解为对翻译友好单位的解析器； 2）翻译器，验证器，摘要和术语提取器，可协同工作，以确保上下文感知，自我校正和术语一致的翻译； 3）将翻译内容重建为结构良好的乳胶文档的发电机。实验结果表明，乳胶可以以翻译精度和结构保真度优于主流MT系统，提供有效且实用的解决方案，用于翻译乳胶形式的文档。

Title: LLM-based Contrastive Self-Supervised AMR Learning with Masked Graph Autoencoders for Fake News Detection

• Authors: Shubham Gupta, Shraban Kumar Chatterjee, Suman Kundu
• Subjects: cs.CL, cs.SI
• Abstract URL: https://arxiv.org/abs/2508.18819
• Pdf URL: https://arxiv.org/pdf/2508.18819
• Copy Paste: [[2508.18819]] LLM-based Contrastive Self-Supervised AMR Learning with Masked Graph Autoencoders for Fake News Detection(https://arxiv.org/abs/2508.18819)
• Keywords: language model, llm
• Abstract: The proliferation of misinformation in the digital age has led to significant societal challenges. Existing approaches often struggle with capturing long-range dependencies, complex semantic relations, and the social dynamics influencing news dissemination. Furthermore, these methods require extensive labelled datasets, making their deployment resource-intensive. In this study, we propose a novel self-supervised misinformation detection framework that integrates both complex semantic relations using Abstract Meaning Representation (AMR) and news propagation dynamics. We introduce an LLM-based graph contrastive loss (LGCL) that utilizes negative anchor points generated by a Large Language Model (LLM) to enhance feature separability in a zero-shot manner. To incorporate social context, we employ a multi view graph masked autoencoder, which learns news propagation features from social context graph. By combining these semantic and propagation-based features, our approach effectively differentiates between fake and real news in a self-supervised manner. Extensive experiments demonstrate that our self-supervised framework achieves superior performance compared to other state-of-the-art methodologies, even with limited labelled datasets while improving generalizability.
• 摘要：数字时代的错误信息的扩散导致了重大的社会挑战。现有的方法通常在捕获长期依赖性，复杂的语义关系以及影响新闻传播的社会动态方面遇到困难。此外，这些方法需要广泛的标记数据集，从而使其部署资源密集型。在这项研究中，我们提出了一个新型的自我监督错误信息检测框架，该框架使用抽象含义表示（AMR）和新闻传播动态整合复杂的语义关系。我们引入了基于LLM的图形对比损失（LGCL），该损失（LGCL）利用了由大语言模型（LLM）生成的负锚点以零拍的方式增强特征可分离性。为了结合社会环境，我们采用了多视图掩盖自动编码器，该图形从社交上下文图中学习新闻传播功能。通过将这些基于语义和传播的特征结合在一起，我们的方法以自我监督的方式有效地区分了假新闻和真实新闻。广泛的实验表明，与其他最先进的方法相比，我们的自我监管框架的性能也比其他标记的数据集有限，同时提高了概括性。

Title: Arrows of Math Reasoning Data Synthesis for Large Language Models: Diversity, Complexity and Correctness

• Authors: Sirui Chen, Changxin Tian, Binbin Hu, Kunlong Chen, Ziqi Liu, Zhiqiang Zhang, Jun Zhou
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.18824
• Pdf URL: https://arxiv.org/pdf/2508.18824
• Copy Paste: [[2508.18824]] Arrows of Math Reasoning Data Synthesis for Large Language Models: Diversity, Complexity and Correctness(https://arxiv.org/abs/2508.18824)
• Keywords: language model, llm
• Abstract: Enhancing the mathematical reasoning of large language models (LLMs) demands high-quality training data, yet conventional methods face critical challenges in scalability, cost, and data reliability. To address these limitations, we propose a novel program-assisted synthesis framework that systematically generates a high-quality mathematical corpus with guaranteed diversity, complexity, and correctness. This framework integrates mathematical knowledge systems and domain-specific tools to create executable programs. These programs are then translated into natural language problem-solution pairs and vetted by a bilateral validation mechanism that verifies solution correctness against program outputs and ensures program-problem consistency. We have generated 12.3 million such problem-solving triples. Experiments demonstrate that models fine-tuned on our data significantly improve their inference capabilities, achieving state-of-the-art performance on several benchmark datasets and showcasing the effectiveness of our synthesis approach.
• 摘要：增强大语模型（LLM）的数学推理需要高质量的培训数据，但是常规方法在可伸缩性，成本和数据可靠性方面面临着关键的挑战。为了解决这些局限性，我们提出了一个新颖的程序辅助合成框架，该框架系统地生成具有保证多样性，复杂性和正确性的高质量数学语料库。该框架集成了数学知识系统和特定领域的工具来创建可执行程序。然后将这些程序转化为自然语言问题解决方案对，并通过双边验证机制进行审查，该机制验证了针对程序输出的解决方案的正确性，并确保了程序问题的一致性。我们已经产生了1,230万此类解决问题的三元组。实验表明，对我们数据进行微调的模型可显着提高其推论能力，从而在几个基准数据集上实现最新性能，并展示我们合成方法的有效性。

Title: ConfTuner: Training Large Language Models to Express Their Confidence Verbally

• Authors: Yibo Li, Miao Xiong, Jiaying Wu, Bryan Hooi
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.18847
• Pdf URL: https://arxiv.org/pdf/2508.18847
• Copy Paste: [[2508.18847]] ConfTuner: Training Large Language Models to Express Their Confidence Verbally(https://arxiv.org/abs/2508.18847)
• Keywords: language model, gpt, llm, prompt
• Abstract: Large Language Models (LLMs) are increasingly deployed in high-stakes domains such as science, law, and healthcare, where accurate expressions of uncertainty are essential for reliability and trust. However, current LLMs are often observed to generate incorrect answers with high confidence, a phenomenon known as "overconfidence". Recent efforts have focused on calibrating LLMs' verbalized confidence: i.e., their expressions of confidence in text form, such as "I am 80% confident that...". Existing approaches either rely on prompt engineering or fine-tuning with heuristically generated uncertainty estimates, both of which have limited effectiveness and generalizability. Motivated by the notion of proper scoring rules for calibration in classical machine learning models, we introduce ConfTuner, a simple and efficient fine-tuning method that introduces minimal overhead and does not require ground-truth confidence scores or proxy confidence estimates. ConfTuner relies on a new loss function, tokenized Brier score, which we theoretically prove to be a proper scoring rule, intuitively meaning that it "correctly incentivizes the model to report its true probability of being correct". ConfTuner improves calibration across diverse reasoning tasks and generalizes to black-box models such as GPT-4o. Our results further show that better-calibrated confidence enables downstream gains in self-correction and model cascade, advancing the development of trustworthy LLM systems. The code is available at this https URL.
• 摘要：大型语言模型（LLM）越来越多地部署在科学，法律和医疗保健等高风险领域中，在这种领域中，准确的不确定性表达对于可靠性和信任至关重要。但是，通常观察到当前的LLM会以高信心产生错误的答案，这种现象被称为“过度自信”。最近的努力集中在校准LLMS的口头信心：即，他们对文本形式的信心表达，例如“我80％的信心……”。现有的方法要么依赖于迅速的工程或通过启发式产生的不确定性估计值进行微调，这两种估计都有有限的有效性和可推广性。在经典的机器学习模型中，由适当评分规则的概念进行的概念，我们引入了Conftuner，这是一种简单有效的微调方法，它引入了最小的开销，并且不需要地面真实的置信度分数或代理置信度估计。 Conftuner依靠新的损失函数，令牌化的Brier分数，从理论上讲，我们被证明是一个适当的评分规则，直觉上意味着它“正确激励模型以报告其正确的正确概率”。 Conftuner改善了各种推理任务的校准，并将其推广到Black-Box模型，例如GPT-4O。我们的结果进一步表明，更好地校准的置信度可以使自校正和模型级联的下游增长，从而推进了值得信赖的LLM系统的发展。该代码可在此HTTPS URL上找到。

Title: ReflectivePrompt: Reflective evolution in autoprompting algorithms

• Authors: Viktor N. Zhuravlev, Artur R. Khairullin, Ernest A. Dyagin, Alena N. Sitkina, Nikita I. Kulin
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.18870
• Pdf URL: https://arxiv.org/pdf/2508.18870
• Copy Paste: [[2508.18870]] ReflectivePrompt: Reflective evolution in autoprompting algorithms(https://arxiv.org/abs/2508.18870)
• Keywords: language model, llm, prompt
• Abstract: Autoprompting is the process of automatically selecting optimized prompts for language models, which has been gaining popularity with the rapid advancement of prompt engineering, driven by extensive research in the field of large language models (LLMs). This paper presents ReflectivePrompt - a novel autoprompting method based on evolutionary algorithms that employs a reflective evolution approach for more precise and comprehensive search of optimal prompts. ReflectivePrompt utilizes short-term and long-term reflection operations before crossover and elitist mutation to enhance the quality of the modifications they introduce. This method allows for the accumulation of knowledge obtained throughout the evolution process and updates it at each epoch based on the current population. ReflectivePrompt was tested on 33 datasets for classification and text generation tasks using open-access large language models: t-lite-instruct-0.1 and gemma3-27b-it. The method demonstrates, on average, a significant improvement (e.g., 28% on BBH compared to EvoPrompt) in metrics relative to current state-of-the-art approaches, thereby establishing itself as one of the most effective solutions in evolutionary algorithm-based autoprompting.
• 摘要：自动弹药是自动选择对语言模型的优化提示的过程，该提示在大型语言模型（LLMS）领域的广泛研究驱动下，随着迅速工程的快速发展，这一过程一直受欢迎。本文介绍了反思性提示 - 一种基于进化算法的新型自动爆发方法，该方法采用反射性进化方法来更精确，更全面地搜索最佳提示。反射性提示在交叉和精英突变之前利用短期和长期反思操作来提高其引入的修改质量。该方法允许在整个演化过程中获得的知识积累，并根据当前人群在每个时期内对其进行更新。在33个数据集上测试了反思性提示，用于使用开放式大语模型进行分类和文本生成任务：T-Lite-Instruct-0.1和Gemma3-27b-it。该方法平均表明，相对于当前的最新方法，指标的平均改善（例如，与EVOPROMPT相比，BBH相比为28％），从而确立了基于进化算法的自动爆发剂中最有效的解决方案之一。

Title: Empowering Computing Education Researchers Through LLM-Assisted Content Analysis

• Authors: Laurie Gale, Sebastian Mateos Nicolajsen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.18872
• Pdf URL: https://arxiv.org/pdf/2508.18872
• Copy Paste: [[2508.18872]] Empowering Computing Education Researchers Through LLM-Assisted Content Analysis(https://arxiv.org/abs/2508.18872)
• Keywords: language model, llm
• Abstract: Computing education research (CER) is often instigated by practitioners wanting to improve both their own and the wider discipline's teaching practice. However, the latter is often difficult as many researchers lack the colleagues, resources, or capacity to conduct research that is generalisable or rigorous enough to advance the discipline. As a result, research methods that enable sense-making with larger volumes of qualitative data, while not increasing the burden on the researcher, have significant potential within CER. In this discussion paper, we propose such a method for conducting rigorous analysis on large volumes of textual data, namely a variation of LLM-assisted content analysis (LACA). This method combines content analysis with the use of large language models, empowering researchers to conduct larger-scale research which they would otherwise not be able to perform. Using a computing education dataset, we illustrate how LACA could be applied in a reproducible and rigorous manner. We believe this method has potential in CER, enabling more generalisable findings from a wider range of research. This, together with the development of similar methods, can help to advance both the practice and research quality of the CER discipline.
• 摘要：计算教育研究（CER）通常是由想要改善自己和更广泛学科的教学实践的从业者促进的。但是，后者通常很困难，因为许多研究人员都缺乏可以进行普遍或严格的研究以推进学科的研究的同事，资源或能力。结果，可以在CER中具有较大量的定性数据来实现感官创造的研究方法，同时又不增加研究人员的负担，具有巨大的潜力。在本讨论文件中，我们提出了一种对大量文本数据进行严格分析的方法，即LLM辅助内容分析（LACA）的变化。这种方法将内容分析与使用大语言模型的使用结合在一起，使研究人员能够进行大规模研究，否则他们将无法执行这些研究。使用计算教育数据集，我们说明了如何以可重复和严格的方式应用LACA。我们认为，这种方法具有CER的潜力，从广泛的研究中实现了更普遍的发现。这与开发类似方法可以帮助提高CER学科的实践和研究质量。

Title: Diverse And Private Synthetic Datasets Generation for RAG evaluation: A multi-agent framework

• Authors: Ilias Driouich, Hongliu Cao, Eoin Thomas
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.18929
• Pdf URL: https://arxiv.org/pdf/2508.18929
• Copy Paste: [[2508.18929]] Diverse And Private Synthetic Datasets Generation for RAG evaluation: A multi-agent framework(https://arxiv.org/abs/2508.18929)
• Keywords: language model, retrieval-augmented generation, agent
• Abstract: Retrieval-augmented generation (RAG) systems improve large language model outputs by incorporating external knowledge, enabling more informed and context-aware responses. However, the effectiveness and trustworthiness of these systems critically depends on how they are evaluated, particularly on whether the evaluation process captures real-world constraints like protecting sensitive information. While current evaluation efforts for RAG systems have primarily focused on the development of performance metrics, far less attention has been given to the design and quality of the underlying evaluation datasets, despite their pivotal role in enabling meaningful, reliable assessments. In this work, we introduce a novel multi-agent framework for generating synthetic QA datasets for RAG evaluation that prioritize semantic diversity and privacy preservation. Our approach involves: (1) a Diversity agent leveraging clustering techniques to maximize topical coverage and semantic variability, (2) a Privacy Agent that detects and mask sensitive information across multiple domains and (3) a QA curation agent that synthesizes private and diverse QA pairs suitable as ground truth for RAG evaluation. Extensive experiments demonstrate that our evaluation sets outperform baseline methods in diversity and achieve robust privacy masking on domain-specific datasets. This work offers a practical and ethically aligned pathway toward safer, more comprehensive RAG system evaluation, laying the foundation for future enhancements aligned with evolving AI regulations and compliance standards.
• 摘要：检索增强的生成（RAG）系统通过合并外部知识来改善大型语言模型输出，从而实现更多知识和背景感知的响应。但是，这些系统的有效性和可信赖性在很大程度上取决于它们的评估方式，特别是评估过程是否捕获了诸如保护敏感信息之类的现实世界约束。尽管当前对抹布系统的评估工作主要集中在绩效指标的发展上，但尽管在实现有意义的，可靠的评估方面的角色关注，但对基础评估数据集的设计和质量的关注得多。在这项工作中，我们介绍了一个新型的多代理框架，用于生成合成质量检查数据集，以进行抹布评估，以优先使用语义多样性和隐私保护。我们的方法涉及：（1）利用聚类技术来最大化局部覆盖范围和语义可变性的多样性剂，（2）一种隐私剂，可检测并跨多个域中检测和掩盖敏感信息，以及（3）QA策划代理，综合了私人和多样的QA成对作为RAG评估的地面真理。广泛的实验表明，我们的评估设置在多样性方面的表现优于基线方法，并在特定领域的数据集上实现了强大的隐私掩蔽。这项工作为实用和道德上的途径提供了通往更安全，更全面的抹布系统评估的途径，为未来增强的基础奠定了与不断发展的AI法规和合规性标准相符的基础。

Title: Automatic Prompt Optimization with Prompt Distillation

• Authors: Viktor N. Zhuravlev, Artur R. Khairullin, Ernest A. Dyagin, Alena N. Sitkina, Nikita I. Kulin
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.18992
• Pdf URL: https://arxiv.org/pdf/2508.18992
• Copy Paste: [[2508.18992]] Automatic Prompt Optimization with Prompt Distillation(https://arxiv.org/abs/2508.18992)
• Keywords: language model, llm, prompt
• Abstract: Autoprompting is the process of automatically selecting optimized prompts for language models, which is gaining popularity due to the rapid development of prompt engineering driven by extensive research in the field of large language models (LLMs). This paper presents DistillPrompt -- a novel autoprompting method based on large language models that employs a multi-stage integration of task-specific information into prompts using training data. DistillPrompt utilizes distillation, compression, and aggregation operations to explore the prompt space more thoroughly. The method was tested on different datasets for text classification and generation tasks using the t-lite-instruct-0.1 language model. The results demonstrate a significant average improvement (e.g., 20.12% across the entire dataset compared to Grips) in key metrics over existing methods in the field, establishing DistillPrompt as one of the most effective non-gradient approaches in autoprompting.
• 摘要：自动弹药是自动选择对语言模型的优化提示的过程，由于大型语言模型（LLMS）领域的广泛研究驱动的迅速工程的快速发展，这一过程正在受欢迎。本文介绍了蒸馏器 - 一种基于大语言模型的新型自动爆发方法，该方法将特定于任务的信息多级集成到使用培训数据的提示中。 Distill Prompt利用蒸馏，压缩和聚合操作更彻底地探索及时空间。该方法在不同的数据集上测试了使用T-Lite-Instruct-0.1语言模型的文本分类和生成任务。结果表明，关键指标的平均平均改善（例如，整个数据集的20.12％）比现场中的现有方法的平均改善，将蒸馏液作为自动爆发中最有效的非梯度方法之一。

Title: MovieCORE: COgnitive REasoning in Movies

• Authors: Gueter Josmy Faure, Min-Hung Chen, Jia-Fong Yeh, Ying Cheng, Hung-Ting Su, Yung-Hao Tang, Shang-Hong Lai, Winston H. Hsu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.19026
• Pdf URL: https://arxiv.org/pdf/2508.19026
• Copy Paste: [[2508.19026]] MovieCORE: COgnitive REasoning in Movies(https://arxiv.org/abs/2508.19026)
• Keywords: language model, llm, agent
• Abstract: This paper introduces MovieCORE, a novel video question answering (VQA) dataset designed to probe deeper cognitive understanding of movie content. Unlike existing datasets that focus on surface-level comprehension, MovieCORE emphasizes questions that engage System-2 thinking while remaining specific to the video material. We present an innovative agentic brainstorming approach, utilizing multiple large language models (LLMs) as thought agents to generate and refine high-quality question-answer pairs. To evaluate dataset quality, we develop a set of cognitive tests assessing depth, thought-provocation potential, and syntactic complexity. We also propose a comprehensive evaluation scheme for assessing VQA model performance on deeper cognitive tasks. To address the limitations of existing video-language models (VLMs), we introduce an agentic enhancement module, Agentic Choice Enhancement (ACE), which improves model reasoning capabilities post-training by up to 25%. Our work contributes to advancing movie understanding in AI systems and provides valuable insights into the capabilities and limitations of current VQA models when faced with more challenging, nuanced questions about cinematic content. Our project page, dataset and code can be found at this https URL.
• 摘要：本文介绍了Moviecore，这是一个新颖的视频答案（VQA）数据集，旨在探究对电影内容的更深入的认知理解。与专注于表面级别理解的现有数据集不同，Moviecore强调了涉及System-2思维的问题，同时又是视频材料的特定问题。我们介绍了一种创新的代理头脑风暴方法，利用多种大语言模型（LLM）作为思想剂来产生和完善高质量的问题 - 答案对。为了评估数据集质量，我们开发了一系列认知测试，以评估深度，思想积累潜力和句法复杂性。我们还提出了一项全面的评估方案，用于评估对更深层次的认知任务的VQA模型性能。为了解决现有视频模型（VLMS）的局限性，我们引入了一个代理增强模块，代理选择增强（ACE），该模型将模型推理功能提高到培训后最多25％。我们的工作有助于在AI系统中提高电影理解，并在面临有关电影内容的更具挑战性，细致的问题时，对当前VQA模型的功能和局限性提供了宝贵的见解。我们的项目页面，数据集和代码可以在此HTTPS URL上找到。

Title: HiPlan: Hierarchical Planning for LLM-Based Agents with Adaptive Global-Local Guidance

• Authors: Ziyue Li, Yuan Chang, Gaihong Yu, Xiaoqiu Le
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.19076
• Pdf URL: https://arxiv.org/pdf/2508.19076
• Copy Paste: [[2508.19076]] HiPlan: Hierarchical Planning for LLM-Based Agents with Adaptive Global-Local Guidance(https://arxiv.org/abs/2508.19076)
• Keywords: language model, llm, agent
• Abstract: Large language model (LLM)-based agents have demonstrated remarkable capabilities in decision-making tasks, but struggle significantly with complex, long-horizon planning scenarios. This arises from their lack of macroscopic guidance, causing disorientation and failures in complex tasks, as well as insufficient continuous oversight during execution, rendering them unresponsive to environmental changes and prone to deviations. To tackle these challenges, we introduce HiPlan, a hierarchical planning framework that provides adaptive global-local guidance to boost LLM-based agents'decision-making. HiPlan decomposes complex tasks into milestone action guides for general direction and step-wise hints for detailed actions. During the offline phase, we construct a milestone library from expert demonstrations, enabling structured experience reuse by retrieving semantically similar tasks and milestones. In the execution phase, trajectory segments from past milestones are dynamically adapted to generate step-wise hints that align current observations with the milestone objectives, bridging gaps and correcting deviations. Extensive experiments across two challenging benchmarks demonstrate that HiPlan substantially outperforms strong baselines, and ablation studies validate the complementary benefits of its hierarchical components.
• 摘要：大型语言模型（LLM）的代理商在决策任务中表现出了显着的能力，但在复杂的长马计划方案中遇到了重大斗争。这是由于他们缺乏宏观的指导，导致复杂任务中的迷失方向和失败以及在执行过程中的持续监督不足，因此使它们对环境变化和容易偏离偏差无反应。为了应对这些挑战，我们介绍了Hiplan，这是一个层次规划框架，提供了自适应的全球本地指导，以增强基于LLM的代理商的制作制作。 Hiplan将复杂的任务分解为里程碑动作指南，以提供一般方向和逐步提示，以详细行动。在离线阶段，我们从专家演示中构建了一个里程碑式库，从而通过检索语义上相似的任务和里程碑来使结构化的经验重复使用。在执行阶段，从过去的里程碑中进行了动态调整的轨迹段，以生成逐步的提示，这些提示将当前的观察结果与里程碑目标，弥合空白和纠正偏差保持一致。对两个具有挑战性的基准进行的广泛实验表明，Hiplan的表现大大优于强质基础，而消融研究证明了其层次组件的互补益处。

Title: It's All About In-Context Learning! Teaching Extremely Low-Resource Languages to LLMs

• Authors: Yue Li, Zhixue Zhao, Carolina Scarton
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.19089
• Pdf URL: https://arxiv.org/pdf/2508.19089
• Copy Paste: [[2508.19089]] It's All About In-Context Learning! Teaching Extremely Low-Resource Languages to LLMs(https://arxiv.org/abs/2508.19089)
• Keywords: language model, llm
• Abstract: Extremely low-resource languages, especially those written in rare scripts, as shown in Figure 1, remain largely unsupported by large language models (LLMs). This is due in part to compounding factors such as the lack of training data. This paper delivers the first comprehensive analysis of whether LLMs can acquire such languages purely via in-context learning (ICL), with or without auxiliary alignment signals, and how these methods compare to parameter-efficient fine-tuning (PEFT). We systematically evaluate 20 under-represented languages across three state-of-the-art multilingual LLMs. Our findings highlight the limitation of PEFT when both language and its script are extremely under-represented by the LLM. In contrast, zero-shot ICL with language alignment is impressively effective on extremely low-resource languages, while few-shot ICL or PEFT is more beneficial for languages relatively better represented by LLMs. For LLM practitioners working on extremely low-resource languages, we summarise guidelines grounded by our results on adapting LLMs to low-resource languages, e.g., avoiding fine-tuning a multilingual model on languages of unseen scripts.
• 摘要：如图1所示，极低的资源语言，尤其是用稀有脚本编写的语言，在很大程度上没有大型语言模型（LLMS）所支持的语言。这部分是由于缺乏培训数据等复杂因素。本文对LLM的首次综合分析提供了是否可以纯粹可以通过内在学习（ICL），有或没有辅助对齐信号获得此类语言，以及这些方法与参数有效的微调（PEFT）的比较。我们系统地评估了三种最先进的多语言LLM的20种代表性不足的语言。我们的发现突出了PEFT的局限性，而LLM的语言和脚本极为不足。相比之下，语言一致性的零射击ICL对极低的资源语言具有令人印象深刻的有效性，而很少的ICL或PEFT对LLMS代表的语言更有益。对于从事极低资源语言的LLM从业人员，我们总结了基于将LLMS调整为低资源语言的结果所基于的指南，例如，避免对看不见脚本的语言进行微调模型。

Title: Retrieval-Augmented Generation for Natural Language Art Provenance Searches in the Getty Provenance Index

• Authors: Mathew Henrickson
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.19093
• Pdf URL: https://arxiv.org/pdf/2508.19093
• Copy Paste: [[2508.19093]] Retrieval-Augmented Generation for Natural Language Art Provenance Searches in the Getty Provenance Index(https://arxiv.org/abs/2508.19093)
• Keywords: retrieval-augmented generation
• Abstract: This research presents a Retrieval-Augmented Generation (RAG) framework for art provenance studies, focusing on the Getty Provenance Index. Provenance research establishes the ownership history of artworks, which is essential for verifying authenticity, supporting restitution and legal claims, and understanding the cultural and historical context of art objects. The process is complicated by fragmented, multilingual archival data that hinders efficient retrieval. Current search portals require precise metadata, limiting exploratory searches. Our method enables natural-language and multilingual searches through semantic retrieval and contextual summarization, reducing dependence on metadata structures. We assess RAG's capability to retrieve and summarize auction records using a 10,000-record sample from the Getty Provenance Index - German Sales. The results show this approach provides a scalable solution for navigating art market archives, offering a practical tool for historians and cultural heritage professionals conducting historically sensitive research.
• 摘要：这项研究介绍了艺术出处研究的检索型生成（RAG）框架，重点是盖蒂出处指数。出处研究建立了艺术品的所有权历史，这对于验证真实性，支持恢复原状和法律主张以及了解艺术对象的文化和历史背景至关重要。该过程因碎片的多语言档案数据而复杂化，这些数据阻碍了有效的检索。当前的搜索门户需要精确的元数据，限制探索性搜索。我们的方法通过语义检索和上下文摘要实现了自然语言和多语言搜索，从而降低了对元数据结构的依赖。我们评估了RAG使用Getty出处指数 - 德国销售中的10,000个记录样本检索和总结拍卖记录的能力。结果表明，这种方法为导航艺术市场档案提供了可扩展的解决方案，为进行历史敏感研究的历史学家和文化遗产专业人士提供了实用的工具。

Title: Beyond the Black Box: Integrating Lexical and Semantic Methods in Quantitative Discourse Analysis with BERTopic

• Authors: Thomas Compton
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.19099
• Pdf URL: https://arxiv.org/pdf/2508.19099
• Copy Paste: [[2508.19099]] Beyond the Black Box: Integrating Lexical and Semantic Methods in Quantitative Discourse Analysis with BERTopic(https://arxiv.org/abs/2508.19099)
• Keywords: language model
• Abstract: Quantitative Discourse Analysis has seen growing adoption with the rise of Large Language Models and computational tools. However, reliance on black box software such as MAXQDA and NVivo risks undermining methodological transparency and alignment with research goals. This paper presents a hybrid, transparent framework for QDA that combines lexical and semantic methods to enable triangulation, reproducibility, and interpretability. Drawing from a case study in historical political discourse, we demonstrate how custom Python pipelines using NLTK, spaCy, and Sentence Transformers allow fine-grained control over preprocessing, lemmatisation, and embedding generation. We further detail our iterative BERTopic modelling process, incorporating UMAP dimensionality reduction, HDBSCAN clustering, and c-TF-IDF keyword extraction, optimised through parameter tuning and multiple runs to enhance topic coherence and coverage. By juxtaposing precise lexical searches with context-aware semantic clustering, we argue for a multi-layered approach that mitigates the limitations of either method in isolation. Our workflow underscores the importance of code-level transparency, researcher agency, and methodological triangulation in computational discourse studies. Code and supplementary materials are available via GitHub.
• 摘要：定量话语分析随着大语言模型和计算工具的兴起的兴起，采用的采用日益增长。但是，依赖黑匣子软件（例如MAXQDA和NVIVO）风险破坏方法透明度和与研究目标的一致性。本文提出了QDA的混合透明框架，结合了词汇和语义方法，以实现三角测量，再现性和解释性。从历史政治话语中的案例研究中得出，我们证明了使用NLTK，Spacy和句子变形金刚的定制Python管道如何允许对预处理，诱饵和嵌入产生的细粒度控制。我们进一步详细介绍了迭代伯气建模过程，这些过程结合了UMAP维度降低，HDBSCAN聚类和C-TF-IDF关键字提取，通过参数调整和多个运行进行了优化，以增强主题相干性和覆盖范围。通过将精确的词汇搜索与上下文感知的语义聚类并列，我们主张一种多层方法，该方法可以减轻任何一种方法的局限性。我们的工作流程强调了计算话语研究中代码级透明度，研究人员机构和方法论三角剖分的重要性。可以通过GitHub获得代码和补充材料。

Title: Do LVLMs Know What They Know? A Systematic Study of Knowledge Boundary Perception in LVLMs

• Authors: Zhikai Ding, Shiyu Ni, Keping Bi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.19111
• Pdf URL: https://arxiv.org/pdf/2508.19111
• Copy Paste: [[2508.19111]] Do LVLMs Know What They Know? A Systematic Study of Knowledge Boundary Perception in LVLMs(https://arxiv.org/abs/2508.19111)
• Keywords: language model, llm
• Abstract: Large vision-language models (LVLMs) demonstrate strong visual question answering (VQA) capabilities but are shown to hallucinate. A reliable model should perceive its knowledge boundaries-knowing what it knows and what it does not. This paper investigates LVLMs' perception of their knowledge boundaries by evaluating three types of confidence signals: probabilistic confidence, answer consistency-based confidence, and verbalized confidence. Experiments on three LVLMs across three VQA datasets show that, although LVLMs possess a reasonable perception level, there is substantial room for improvement. Among the three confidences, probabilistic and consistency-based signals are more reliable indicators, while verbalized confidence often leads to overconfidence. To enhance LVLMs' perception, we adapt several established confidence calibration methods from Large Language Models (LLMs) and propose three effective methods. Additionally, we compare LVLMs with their LLM counterparts, finding that jointly processing visual and textual inputs decreases question-answering performance but reduces confidence, resulting in an improved perception level compared to LLMs.
• 摘要：大型视觉模型（LVLM）表现出强大的视觉响应（VQA）功能，但显示出幻觉。可靠的模型应该感知其知识边界，从而知道其所知道的和不知道的知识。本文通过评估三种类型的置信信号来研究LVLMS对知识边界的感知：概率信心，基于答案一致性的信心和口头上的信心。在三个VQA数据集上进行三个LVLM的实验表明，尽管LVLM具有合理的感知水平，但仍有很大的改进空间。在这三种信心中，概率和基于一致性的信号是更可靠的指标，而口头上的信心通常会导致过度自信。为了增强LVLM的感知，我们适应了大型语言模型（LLM）的几种既定置信度校准方法，并提出了三种有效的方法。此外，我们将LVLM与LLM对应物进行了比较，发现共同处理视觉和文本输入会降低问题的效果，但会降低信心，从而降低了与LLMS相比的感知水平提高。

Title: Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning

• Authors: Alan Li, Yixin Liu, Arpan Sarkar, Doug Downey, Arman Cohan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.19202
• Pdf URL: https://arxiv.org/pdf/2508.19202
• Copy Paste: [[2508.19202]] Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning(https://arxiv.org/abs/2508.19202)
• Keywords: llm
• Abstract: Scientific problem solving poses unique challenges for LLMs, requiring both deep domain knowledge and the ability to apply such knowledge through complex reasoning. While automated scientific reasoners hold great promise for assisting human scientists, there is currently no widely adopted holistic benchmark for evaluating scientific reasoning, and few approaches systematically disentangle the distinct roles of knowledge and reasoning in these tasks. To address these gaps, we introduce SciReas, a diverse suite of existing benchmarks for scientific reasoning tasks, and SciReas-Pro, a selective subset that requires more complex reasoning. Our holistic evaluation surfaces insights about scientific reasoning performance that remain hidden when relying on individual benchmarks alone. We then propose KRUX, a probing framework for studying the distinct roles of reasoning and knowledge in scientific tasks. Combining the two, we conduct an in-depth analysis that yields several key findings: (1) Retrieving task-relevant knowledge from model parameters is a critical bottleneck for LLMs in scientific reasoning; (2) Reasoning models consistently benefit from external knowledge added in-context on top of the reasoning enhancement; (3) Enhancing verbalized reasoning improves LLMs' ability to surface task-relevant knowledge. Finally, we conduct a lightweight analysis, comparing our science-focused data composition with concurrent efforts on long CoT SFT, and release SciLit01, a strong 8B baseline for scientific reasoning.
• 摘要：解决科学问题解决了LLM的独特挑战，需要深层领域知识和通过复杂推理应用此类知识的能力。虽然自动化的科学推理者对协助人类科学家有着巨大的希望，但目前尚无广泛采用的整体基准来评估科学推理，而且很少有方法可以系统地剥夺知识和推理在这些任务中的独特作用。为了解决这些差距，我们介绍了Scireas，这是科学推理任务的各种现有基准，以及Scireas-Pro，这是一个需要更复杂推理的选择性子集。我们的整体评估表现出有关科学推理绩效的见解，这些绩效仅依靠单个基准时就隐藏了。然后，我们提出了KRUX，这是研究推理和知识在科学任务中的独特作用的探测框架。结合两者，我们进行了深入的分析，该分析得出了几个关键发现：（1）从模型参数中检索与任务相关的知识是LLM在科学推理中的关键瓶颈； （2）推理模型始终受益于在推理增强之上添加的外部知识中的外部知识； （3）增强言语推理可以提高LLMS表达与任务相关的知识的能力。最后，我们进行了轻量级分析，将我们以科学为中心的数据组成与长COT SFT的同时努力进行了比较，并释放SCILIT01，这是科学推理的强大的8B基线。

Title: Evaluating the Evaluators: Are readability metrics good measures of readability?

• Authors: Isabel Cachola, Daniel Khashabi, Mark Dredze
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.19221
• Pdf URL: https://arxiv.org/pdf/2508.19221
• Copy Paste: [[2508.19221]] Evaluating the Evaluators: Are readability metrics good measures of readability?(https://arxiv.org/abs/2508.19221)
• Keywords: language model
• Abstract: Plain Language Summarization (PLS) aims to distill complex documents into accessible summaries for non-expert audiences. In this paper, we conduct a thorough survey of PLS literature, and identify that the current standard practice for readability evaluation is to use traditional readability metrics, such as Flesch-Kincaid Grade Level (FKGL). However, despite proven utility in other fields, these metrics have not been compared to human readability judgments in PLS. We evaluate 8 readability metrics and show that most correlate poorly with human judgments, including the most popular metric, FKGL. We then show that Language Models (LMs) are better judges of readability, with the best-performing model achieving a Pearson correlation of 0.56 with human judgments. Extending our analysis to PLS datasets, which contain summaries aimed at non-expert audiences, we find that LMs better capture deeper measures of readability, such as required background knowledge, and lead to different conclusions than the traditional metrics. Based on these findings, we offer recommendations for best practices in the evaluation of plain language summaries. We release our analysis code and survey data.
• 摘要：普通语言摘要（PLS）旨在将复杂的文档提炼成非专业受众的可访问摘要。在本文中，我们对PLS文献进行了彻底的调查，并确定当前的可读性评估标准实践是使用传统的可读性指标，例如Flesch-Kincaid等级（FKGL）。但是，尽管在其他领域有证明的效用，但这些指标尚未与PL中的人类可读性判断进行比较。我们评估了8个可读性指标，并表明与人类判断（包括最受欢迎的指标FKGL）的相关性最差。然后，我们证明语言模型（LMS）是更好的可读性法官，表现最佳的模型达到了与人类判断的0.56相关性。将我们的分析扩展到PLS数据集（包含针对非专业观众的摘要）的PLS数据集，我们发现LMS可以更好地捕获更深入的可读性衡量标准，例如所需的背景知识，并且与传统指标相比得出不同的结论。根据这些发现，我们为评估普通语言摘要的最佳实践提供了建议。我们发布我们的分析代码和调查数据。

Title: Generative Interfaces for Language Models

• Authors: Jiaqi Chen, Yanzhe Zhang, Yutong Zhang, Yijia Shao, Diyi Yang
• Subjects: cs.CL, cs.AI, cs.HC
• Abstract URL: https://arxiv.org/abs/2508.19227
• Pdf URL: https://arxiv.org/pdf/2508.19227
• Copy Paste: [[2508.19227]] Generative Interfaces for Language Models(https://arxiv.org/abs/2508.19227)
• Keywords: language model, llm, chat
• Abstract: Large language models (LLMs) are increasingly seen as assistants, copilots, and consultants, capable of supporting a wide range of tasks through natural conversation. However, most systems remain constrained by a linear request-response format that often makes interactions inefficient in multi-turn, information-dense, and exploratory tasks. To address these limitations, we propose Generative Interfaces for Language Models, a paradigm in which LLMs respond to user queries by proactively generating user interfaces (UIs) that enable more adaptive and interactive engagement. Our framework leverages structured interface-specific representations and iterative refinements to translate user queries into task-specific UIs. For systematic evaluation, we introduce a multidimensional assessment framework that compares generative interfaces with traditional chat-based ones across diverse tasks, interaction patterns, and query types, capturing functional, interactive, and emotional aspects of user experience. Results show that generative interfaces consistently outperform conversational ones, with humans preferring them in over 70% of cases. These findings clarify when and why users favor generative interfaces, paving the way for future advancements in human-AI interaction.
• 摘要：大型语言模型（LLM）越来越被视为助手，副驾驶和顾问，能够通过自然对话来支持各种任务。但是，大多数系统仍受到线性请求响应格式的约束，该格式通常会使多转化，信息密集和探索性任务效率低下的交互作用。为了解决这些局限性，我们提出了语言模型的生成界面，这是一个范式，其中LLM通过主动生成用户界面（UIS）来响应用户查询，从而启用更自适应和互动的参与度。我们的框架利用结构化的界面特异性表示和迭代修复，将用户查询转化为特定于任务的UI。为了进行系统的评估，我们介绍了一个多维评估框架，该框架将生成界面与跨不同任务，交互模式和查询类型的传统聊天界面进行了比较，从而捕获用户体验的功能，交互和情感方面。结果表明，生成界面的表现始终优于对话，人类更喜欢在超过70％的情况下。这些发现阐明了用户何时以及为什么偏爱生成界面，为人类互动中的未来进步铺平了道路。