2025-05-15

Title: Human-AI Collaboration or Academic Misconduct? Measuring AI Use in Student Writing Through Stylometric Evidence

• Authors: Eduardo Araujo Oliveira, Madhavi Mohoni, Sonsoles López-Pernas, Mohammed Saqr
• Subjects: cs.CL, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2505.08828
• Pdf URL: https://arxiv.org/pdf/2505.08828
• Copy Paste: [[2505.08828]] Human-AI Collaboration or Academic Misconduct? Measuring AI Use in Student Writing Through Stylometric Evidence(https://arxiv.org/abs/2505.08828)
• Keywords: llm
• Abstract: As human-AI collaboration becomes increasingly prevalent in educational contexts, understanding and measuring the extent and nature of such interactions pose significant challenges. This research investigates the use of authorship verification (AV) techniques not as a punitive measure, but as a means to quantify AI assistance in academic writing, with a focus on promoting transparency, interpretability, and student development. Building on prior work, we structured our investigation into three stages: dataset selection and expansion, AV method development, and systematic evaluation. Using three datasets - including a public dataset (PAN-14) and two from University of Melbourne students from various courses - we expanded the data to include LLM-generated texts, totalling 1,889 documents and 540 authorship problems from 506 students. We developed an adapted Feature Vector Difference AV methodology to construct robust academic writing profiles for students, designed to capture meaningful, individual characteristics of their writing. The method's effectiveness was evaluated across multiple scenarios, including distinguishing between student-authored and LLM-generated texts and testing resilience against LLMs' attempts to mimic student writing styles. Results demonstrate the enhanced AV classifier's ability to identify stylometric discrepancies and measure human-AI collaboration at word and sentence levels while providing educators with a transparent tool to support academic integrity investigations. This work advances AV technology, offering actionable insights into the dynamics of academic writing in an AI-driven era.
• 摘要：随着人类合作在教育环境中变得越来越普遍，理解和衡量这种相互作用的程度和性质构成了重大挑战。这项研究调查了作者身份验证（AV）技术的使用不是惩罚性措施，而是用于量化学术写作中AI援助的手段，重点是促进透明度，可解释性和学生发展。在先前工作的基础上，我们将调查构建​​为三个阶段：数据集选择和扩展，AV方法开发和系统评估。我们使用三个数据集 - 包括一个来自各种课程的墨尔本大学学生的公共数据集（PAN -14） - 我们将数据扩展到包括LLM生成的文本，总共1,889个文档和540名来自506名学生的作者问题。我们开发了一种适当的功能矢量差异方法，以为学生构建强大的学术写作资料，旨在捕捉其写作的有意义的个人特征。在多种情况下评估了该方法的有效性，包括区分学生作者和LLM生成的文本以及针对LLMS模仿学生写作风格的尝试的弹性。结果表明，增强的AV分类器能够识别造型差异并在单词和句子水平上衡量人类协作的能力，同时为教育者提供透明的工具来支持学术诚信调查。这项工作推动了AV技术，为在AI驱动时代的学术写作动态提供了可行的见解。

Title: A suite of LMs comprehend puzzle statements as well as humans

• Authors: Adele E Goldberg, Supantho Rakshit, Jennifer Hu, Kyle Mahowald
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.08996
• Pdf URL: https://arxiv.org/pdf/2505.08996
• Copy Paste: [[2505.08996]] A suite of LMs comprehend puzzle statements as well as humans(https://arxiv.org/abs/2505.08996)
• Keywords: language model, gpt, llm, prompt, chat
• Abstract: Recent claims suggest that large language models (LMs) underperform humans in comprehending minimally complex English statements (Dentella et al., 2024). Here, we revisit those findings and argue that human performance was overestimated, while LLM abilities were underestimated. Using the same stimuli, we report a preregistered study comparing human responses in two conditions: one allowed rereading (replicating the original study), and one that restricted rereading (a more naturalistic comprehension test). Human accuracy dropped significantly when rereading was restricted (73%), falling below that of Falcon-180B-Chat (76%) and GPT-4 (81%). The newer GPT-o1 model achieves perfect accuracy. Results further show that both humans and models are disproportionately challenged by queries involving potentially reciprocal actions (e.g., kissing), suggesting shared pragmatic sensitivities rather than model-specific deficits. Additional analyses using Llama-2-70B log probabilities, a recoding of open-ended model responses, and grammaticality ratings of other sentences reveal systematic underestimation of model performance. We find that GPT-4o can align with either naive or expert grammaticality judgments, depending on prompt framing. These findings underscore the need for more careful experimental design and coding practices in LLM evaluation, and they challenge the assumption that current models are inherently weaker than humans at language comprehension.
• 摘要：最近的主张表明，大型语言模型（LMS）在理解最低限制的英语陈述时表现不佳（Dentella等，2024）。在这里，我们重新审视了这些发现，并认为人类绩效被高估了，而LLM能力被低估了。使用相同的刺激，我们报告了一项预先策略的研究，比较了在两个条件下人类反应的比较：一种允许重新读取（复制原始研究），而一项限制了重新读取（更自然的理解测试）。当限制重读（73％）时，人类准确性显着下降，低于Falcon-180b-chat（76％）和GPT-4（81％）。较新的GPT-O1型号实现了完美的准确性。结果进一步表明，人类和模型都受到涉及潜在相互作用（例如接吻）的查询的挑战，表明了共同的务实敏感性，而不是特定于模型的缺陷。使用Llama-2-70B对数概率的其他分析，开放式模型响应的重新编码以及其他句子的语法等级揭示了模型性能的系统低估。我们发现，根据迅速的框架，GPT-4O可以与天真或专家语法判断保持一致。这些发现强调了在LLM评估中需要更仔细的实验​​设计和编码实践的必要性，并且它们挑战了当前模型本质上比人类在语言理解上弱弱的假设。

Title: For GPT-4 as with Humans: Information Structure Predicts Acceptability of Long-Distance Dependencies

• Authors: Nicole Cuneo, Eleanor Graves, Supantho Rakshit, Adele E. Goldberg
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.09005
• Pdf URL: https://arxiv.org/pdf/2505.09005
• Copy Paste: [[2505.09005]] For GPT-4 as with Humans: Information Structure Predicts Acceptability of Long-Distance Dependencies(https://arxiv.org/abs/2505.09005)
• Keywords: gpt
• Abstract: It remains debated how well any LM understands natural language or generates reliable metalinguistic judgments. Moreover, relatively little work has demonstrated that LMs can represent and respect subtle relationships between form and function proposed by linguists. We here focus on a particular such relationship established in recent work: English speakers' judgments about the information structure of canonical sentences predicts independently collected acceptability ratings on corresponding 'long distance dependency' [LDD] constructions, across a wide array of base constructions and multiple types of LDDs. To determine whether any LM captures this relationship, we probe GPT-4 on the same tasks used with humans and new this http URL reveal reliable metalinguistic skill on the information structure and acceptability tasks, replicating a striking interaction between the two, despite the zero-shot, explicit nature of the tasks, and little to no chance of contamination [Studies 1a, 1b]. Study 2 manipulates the information structure of base sentences and confirms a causal relationship: increasing the prominence of a constituent in a context sentence increases the subsequent acceptability ratings on an LDD construction. The findings suggest a tight relationship between natural and GPT-4 generated English, and between information structure and syntax, which begs for further exploration.
• 摘要：它仍然在争论任何LM对自然语言的理解程度或产生可靠的金属语言判断。此外，相对较少的工作表明，LM可以代表和尊重语言学家提出的形式和功能之间的微妙关系。我们在这里着重于最近工作中建立的特定关系：说英语的人对规范句子的信息结构的判断预测，在相应的“长距离依赖” [LDD]结构上，独立收集的可接受性等级，跨越了各种基础结构和多种LDD。要确定任何LM是否捕获这种关系，我们对与人类使用的相同任务进行了探测，而新的HTTP URL揭示了在信息结构和可接受性任务上可靠的属性技巧，尽管零摄像，但零摄像，明确的任务性质，几乎没有命令的机会，并且没有命令的机会，并且没有命令的机会，并且没有命令的机会，并且没有命令的机会[研究1B]，研究1B]。研究2操纵基本句子的信息结构并确认因果关系：在上下文句子中增加成分的突出性会增加随后的LDD构造上的可接受性评分。研究结果表明，自然和GPT-4产生的英语之间以及信息结构与语法之间存在紧密的关系，这是为了进一步探索。

Title: Atomic Consistency Preference Optimization for Long-Form Question Answering

• Authors: Jingfeng Chen, Raghuveer Thirukovalluru, Junlin Wang, Kaiwei Luo, Bhuwan Dhingra
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.09039
• Pdf URL: https://arxiv.org/pdf/2505.09039
• Copy Paste: [[2505.09039]] Atomic Consistency Preference Optimization for Long-Form Question Answering(https://arxiv.org/abs/2505.09039)
• Keywords: language model, gpt, llm, hallucination
• Abstract: Large Language Models (LLMs) frequently produce factoid hallucinations - plausible yet incorrect answers. A common mitigation strategy is model alignment, which improves factual accuracy by training on curated factual and non-factual pairs. However, this approach often relies on a stronger model (e.g., GPT-4) or an external knowledge base to assess factual correctness, which may not always be accessible. To address this, we propose Atomic Consistency Preference Optimization (ACPO), a self-supervised preference-tuning method that enhances factual accuracy without external supervision. ACPO leverages atomic consistency signals, i.e., the agreement of individual facts across multiple stochastic responses, to identify high- and low-quality data pairs for model alignment. By eliminating the need for costly GPT calls, ACPO provides a scalable and efficient approach to improving factoid question-answering. Despite being self-supervised, empirical results demonstrate that ACPO outperforms FactAlign, a strong supervised alignment baseline, by 1.95 points on the LongFact and BioGen datasets, highlighting its effectiveness in enhancing factual reliability without relying on external models or knowledge bases.
• 摘要：大型语言模型（LLM）经常产生Factoid幻觉 - 合理但不正确的答案。一种常见的缓解策略是模型对齐，它通过对策划的事实和非事实对培训来提高事实准确性。但是，这种方法通常依赖于更强大的模型（例如GPT-4）或外部知识库来评估事实正确性，这可能并不总是可以访问。为了解决这个问题，我们提出了原子一致性偏好优化（ACPO），这是一种自我监管的偏好调节方法，在没有外部监督的情况下增强了事实准确性。 ACPO利用原子一致性信号，即，在多个随机响应中的个别事​​实一致，以确定模型对齐的高质量和低质量数据对。通过消除对昂贵的GPT呼叫的需求，ACPO提供了一种可扩展有效的方法来改善Factoid的提问。尽管经过自我监督，但经验结果表明，ACPO在长期和Biogen数据集上优于强大的监督对准基线的事实，强大的监督对准基线，强调了其在不依赖外部模型或知识基础的情况下增强事实可靠性的有效性。

Title: A Comprehensive Analysis of Large Language Model Outputs: Similarity, Diversity, and Bias

• Authors: Brandon Smith, Mohamed Reda Bouadjenek, Tahsin Alamgir Kheya, Phillip Dawson, Sunil Aryal
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.09056
• Pdf URL: https://arxiv.org/pdf/2505.09056
• Copy Paste: [[2505.09056]] A Comprehensive Analysis of Large Language Model Outputs: Similarity, Diversity, and Bias(https://arxiv.org/abs/2505.09056)
• Keywords: language model, gpt, llm, prompt
• Abstract: Large Language Models (LLMs) represent a major step toward artificial general intelligence, significantly advancing our ability to interact with technology. While LLMs perform well on Natural Language Processing tasks -- such as translation, generation, code writing, and summarization -- questions remain about their output similarity, variability, and ethical implications. For instance, how similar are texts generated by the same model? How does this compare across different models? And which models best uphold ethical standards? To investigate, we used 5{,}000 prompts spanning diverse tasks like generation, explanation, and rewriting. This resulted in approximately 3 million texts from 12 LLMs, including proprietary and open-source systems from OpenAI, Google, Microsoft, Meta, and Mistral. Key findings include: (1) outputs from the same LLM are more similar to each other than to human-written texts; (2) models like WizardLM-2-8x22b generate highly similar outputs, while GPT-4 produces more varied responses; (3) LLM writing styles differ significantly, with Llama 3 and Mistral showing higher similarity, and GPT-4 standing out for distinctiveness; (4) differences in vocabulary and tone underscore the linguistic uniqueness of LLM-generated content; (5) some LLMs demonstrate greater gender balance and reduced bias. These results offer new insights into the behavior and diversity of LLM outputs, helping guide future development and ethical evaluation.
• 摘要：大型语言模型（LLMS）代表了迈向人工通用智能的重大步骤，从而大大提高了我们与技术互动的能力。尽管LLM在自然语言处理任务（例如翻译，发电，代码写作和摘要）上表现良好，但有关其输出相似性，可变性和道德含义的问题仍然存在。例如，同一模型生成的文本如何相似？这如何比较不同的模型？哪些模式最好地维护道德标准？为了进行调查，我们使用了5个{，} 000个提示，涵盖了各种任务，例如发电，解释和重写。这导致了来自12个LLM的大约300万条文本，包括来自OpenAI，Google，Microsoft，Meta和Mistral的专有和开源系统。关键发现包括：（1）来自同一LLM的输出比彼此更相似，而不是与人写的文本； （2）像Wizardlm-2-8x22b这样的模型产生高度相似的输出，而GPT-4产生了更多的响应； （3）LLM写作风格差异很大，Llama 3和Mistral显示出更高的相似性，而GPT-4则脱颖而出。 （4）词汇和音调的差异强调了LLM生成内容的语言独特性； （5）一些LLM表现出更大的性别平衡和偏见减少。这些结果为LLM产出的行为和多样性提供了新的见解，有助于指导未来的发展和道德评估。

Title: S-DAT: A Multilingual, GenAI-Driven Framework for Automated Divergent Thinking Assessment

• Authors: Jennifer Haase, Paul H. P. Hanel, Sebastian Pokutta
• Subjects: cs.CL, cs.HC
• Abstract URL: https://arxiv.org/abs/2505.09068
• Pdf URL: https://arxiv.org/pdf/2505.09068
• Copy Paste: [[2505.09068]] S-DAT: A Multilingual, GenAI-Driven Framework for Automated Divergent Thinking Assessment(https://arxiv.org/abs/2505.09068)
• Keywords: language model
• Abstract: This paper introduces S-DAT (Synthetic-Divergent Association Task), a scalable, multilingual framework for automated assessment of divergent thinking (DT) -a core component of human creativity. Traditional creativity assessments are often labor-intensive, language-specific, and reliant on subjective human ratings, limiting their scalability and cross-cultural applicability. In contrast, S-DAT leverages large language models and advanced multilingual embeddings to compute semantic distance -- a language-agnostic proxy for DT. We evaluate S-DAT across eleven diverse languages, including English, Spanish, German, Russian, Hindi, and Japanese (Kanji, Hiragana, Katakana), demonstrating robust and consistent scoring across linguistic contexts. Unlike prior DAT approaches, the S-DAT shows convergent validity with other DT measures and correct discriminant validity with convergent thinking. This cross-linguistic flexibility allows for more inclusive, global-scale creativity research, addressing key limitations of earlier approaches. S-DAT provides a powerful tool for fairer, more comprehensive evaluation of cognitive flexibility in diverse populations and can be freely assessed online: this https URL.
• 摘要：本文介绍了S-DAT（合成分散关联任务），这是一个可扩展的，多语言的框架，用于自动评估发散思维（DT） - 人类创造力的核心组成部分。传统的创造力评估通常是劳动密集型的，特定于语言的，并且依赖于主观人类评级，从而限制了其可扩展性和跨文化的适用性。相比之下，S-DAT利用大型语言模型和高级多语言嵌入来计算语义距离 - DT语言不合时宜的代理。我们评估了11种不同语言的S-DAT，包括英语，西班牙语，德语，俄语，印地语和日语（汉字，海拉加纳，katakana），在语言环境中表现出强劲而稳定的评分。与先前的DAT方法不同，S-DAT在其他DT测量中显示出收敛的有效性，并通过收敛思维正确判别有效性。这种跨语言的灵活性允许更具包容性的全球规模的创造力研究，从而解决了早期方法的关键局限性。 S-DAT提供了一种强大的工具，可用于对不同人群的认知灵活性进行更公平，更全面的评估，并且可以在线自由评估：此HTTPS URL。

Title: CEC-Zero: Chinese Error Correction Solution Based on LLM

• Authors: Sophie Zhang, Zhiming Lin
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.09082
• Pdf URL: https://arxiv.org/pdf/2505.09082
• Copy Paste: [[2505.09082]] CEC-Zero: Chinese Error Correction Solution Based on LLM(https://arxiv.org/abs/2505.09082)
• Keywords: language model, llm
• Abstract: Recent advancements in large language models (LLMs) demonstrate exceptional Chinese text processing capabilities, particularly in Chinese Spelling Correction (CSC). While LLMs outperform traditional BERT-based models in accuracy and robustness, challenges persist in reliability and generalization. This paper proposes CEC-Zero, a novel reinforcement learning (RL) framework enabling LLMs to self-correct through autonomous error strategy learning without external supervision. By integrating RL with LLMs' generative power, the method eliminates dependency on annotated data or auxiliary models. Experiments reveal RL-enhanced LLMs achieve industry-viable accuracy and superior cross-domain generalization, offering a scalable solution for reliability optimization in Chinese NLP applications. This breakthrough facilitates LLM deployment in practical Chinese text correction scenarios while establishing a new paradigm for self-improving language models.
• 摘要：大型语言模型（LLMS）的最新进展表明了中国文本处理功能异常，尤其是中国拼写校正（CSC）。尽管LLM在准确性和鲁棒性方面都优于传统的基于BERT的模型，但挑战仍在可靠性和概括方面存在。本文提出了CEC-Zero，这是一种新颖的增强学习（RL）框架，使LLM可以通过自主错误策略学习而无需外部监督。通过将RL与LLMS的生成能力集成，该方法消除了对注释数据或辅助模型的依赖性。实验表明，RL增强的LLM达到了行业可行的准确性和出色的跨域泛化，为中国NLP应用中的可靠性优化提供了可扩展的解决方案。这一突破促进了LLM在实用的中文文本校正方案中的部署，同时建立了用于自我改善语言模型的新范式。

Title: A Scalable Unsupervised Framework for multi-aspect labeling of Multilingual and Multi-Domain Review Data

• Authors: Jiin Park, Misuk Kim
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.09286
• Pdf URL: https://arxiv.org/pdf/2505.09286
• Copy Paste: [[2505.09286]] A Scalable Unsupervised Framework for multi-aspect labeling of Multilingual and Multi-Domain Review Data(https://arxiv.org/abs/2505.09286)
• Keywords: language model, agent
• Abstract: Effectively analyzing online review data is essential across industries. However, many existing studies are limited to specific domains and languages or depend on supervised learning approaches that require large-scale labeled datasets. To address these limitations, we propose a multilingual, scalable, and unsupervised framework for cross-domain aspect detection. This framework is designed for multi-aspect labeling of multilingual and multi-domain review data. In this study, we apply automatic labeling to Korean and English review datasets spanning various domains and assess the quality of the generated labels through extensive experiments. Aspect category candidates are first extracted through clustering, and each review is then represented as an aspect-aware embedding vector using negative sampling. To evaluate the framework, we conduct multi-aspect labeling and fine-tune several pretrained language models to measure the effectiveness of the automatically generated labels. Results show that these models achieve high performance, demonstrating that the labels are suitable for training. Furthermore, comparisons with publicly available large language models highlight the framework's superior consistency and scalability when processing large-scale data. A human evaluation also confirms that the quality of the automatic labels is comparable to those created manually. This study demonstrates the potential of a robust multi-aspect labeling approach that overcomes limitations of supervised methods and is adaptable to multilingual, multi-domain environments. Future research will explore automatic review summarization and the integration of artificial intelligence agents to further improve the efficiency and depth of review analysis.
• 摘要：有效分析在线审核数据在整个行业中都是必不可少的。但是，许多现有的研究仅限于特定的领域和语言，或依赖需要大规模标记数据集的监督学习方法。为了解决这些局限性，我们提出了一个多语言，可扩展和无监督的框架，用于跨域方面检测。该框架设计用于多语言和多域审核数据的多光值标记。在这项研究中，我们将自动标记应用于跨越各个领域的韩语和英语评论数据集，并通过广泛的实验评估生成的标签的质量。首先通过聚类提取方面类别的候选者，然后每次审查用负面采样表示为嵌入矢量。为了评估该框架，我们进行了多光值标签并微调几种验证的语言模型，以衡量自动生成的标签的有效性。结果表明，这些模型达到了高性能，表明标签适合训练。此外，在处理大规模数据时，与公开可用的大语言模型进行比较突出了该框架的卓越一致性和可扩展性。人类评估还证实，自动标签的质量与手动创建的标签相当。这项研究证明了一种强大的多光值标签方法的潜力，该方法克服了监督方法的局限性，并且适用于多语言多域环境。未来的研究将探讨自动审查摘要以及人工智能剂的整合，以进一步提高审查分析的效率和深度。

Title: Scent of Knowledge: Optimizing Search-Enhanced Reasoning with Information Foraging

• Authors: Hongjin Qian, Zheng Liu
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2505.09316
• Pdf URL: https://arxiv.org/pdf/2505.09316
• Copy Paste: [[2505.09316]] Scent of Knowledge: Optimizing Search-Enhanced Reasoning with Information Foraging(https://arxiv.org/abs/2505.09316)
• Keywords: language model, llm, retrieval-augmented generation, agent
• Abstract: Augmenting large language models (LLMs) with external retrieval has become a standard method to address their inherent knowledge cutoff limitations. However, traditional retrieval-augmented generation methods employ static, pre-inference retrieval strategies, making them inadequate for complex tasks involving ambiguous, multi-step, or evolving information needs. Recent advances in test-time scaling techniques have demonstrated significant potential in enabling LLMs to dynamically interact with external tools, motivating the shift toward adaptive inference-time retrieval. Inspired by Information Foraging Theory (IFT), we propose InForage, a reinforcement learning framework that formalizes retrieval-augmented reasoning as a dynamic information-seeking process. Unlike existing approaches, InForage explicitly rewards intermediate retrieval quality, encouraging LLMs to iteratively gather and integrate information through adaptive search behaviors. To facilitate training, we construct a human-guided dataset capturing iterative search and reasoning trajectories for complex, real-world web tasks. Extensive evaluations across general question answering, multi-hop reasoning tasks, and a newly developed real-time web QA dataset demonstrate InForage's superior performance over baseline methods. These results highlight InForage's effectiveness in building robust, adaptive, and efficient reasoning agents.
• 摘要：增强具有外部检索的大型语言模型（LLM）已成为解决其固有知识截止限制的标准方法。但是，传统的检索演示生成方法采用了静态的，推理的检索策略，从而使它们无法完成涉及模棱两可，多步骤或不断发展的信息需求的复杂任务。测试时间缩放技术的最新进展表明，在使LLMS能够与外部工具动态相互作用方面具有巨大潜力，从而激发了向自适应推理时间检索的转变。受到信息觅食理论（IFT）的启发，我们提出了Ifforage，这是一个强化学习框架，将检索提示的推理正式为动态信息寻求信息。与现有方法不同，Inforage明确奖励了中间检索质量，鼓励LLM通过自适应搜索行为迭代地收集和整合信息。为了促进培训，我们构建了一个人类引导的数据集，该数据集捕获了复杂的现实Web任务的迭代搜索和推理轨迹。一般问答，多跳的推理任务以及新开发的实时Web QA数据集的广泛评估表明，Iftorage优于基线方法。这些结果突出了Inforage在建立强大，适应性和有效推理剂方面的有效性。

Title: Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMs

• Authors: Jingcheng Niu, Xingdi Yuan, Tong Wang, Hamidreza Saghir, Amir H. Abdi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.09338
• Pdf URL: https://arxiv.org/pdf/2505.09338
• Copy Paste: [[2505.09338]] Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMs(https://arxiv.org/abs/2505.09338)
• Keywords: language model, llm, prompt
• Abstract: We observe a novel phenomenon, contextual entrainment, across a wide range of language models (LMs) and prompt settings, providing a new mechanistic perspective on how LMs become distracted by ``irrelevant'' contextual information in the input prompt. Specifically, LMs assign significantly higher logits (or probabilities) to any tokens that have previously appeared in the context prompt, even for random tokens. This suggests that contextual entrainment is a mechanistic phenomenon, occurring independently of the relevance or semantic relation of the tokens to the question or the rest of the sentence. We find statistically significant evidence that the magnitude of contextual entrainment is influenced by semantic factors. Counterfactual prompts have a greater effect compared to factual ones, suggesting that while contextual entrainment is a mechanistic phenomenon, it is modulated by semantic factors. We hypothesise that there is a circuit of attention heads -- the entrainment heads -- that corresponds to the contextual entrainment phenomenon. Using a novel entrainment head discovery method based on differentiable masking, we identify these heads across various settings. When we ``turn off'' these heads, i.e., set their outputs to zero, the effect of contextual entrainment is significantly attenuated, causing the model to generate output that capitulates to what it would produce if no distracting context were provided. Our discovery of contextual entrainment, along with our investigation into LM distraction via the entrainment heads, marks a key step towards the mechanistic analysis and mitigation of the distraction problem.
• 摘要：我们在广泛的语言模型（LMS）和及时设置中观察到了一种新颖的现象，即情境夹带，并提供了有关LMS在输入提示中如何分心LMS的新机械观点。具体来说，LMS将明显更高的逻辑（或概率）分配给以前在上下文提示中出现的任何令牌，即使是随机令牌。这表明上下文夹带是一种机械现象，它独立于代币与问题或句子的其余部分的相关性或语义关系。我们发现统计上有重要的证据表明，上下文夹带的幅度受语义因素的影响。与事实相比，反事实提示具有更大的作用，这表明尽管上下文夹带是一种机械现象，但它受语义因素的调节。我们假设有一个注意力头（夹带头）与上下文夹带现象相对应。使用基于可区分掩蔽的新颖夹带头发现方法，我们在各种设置中识别这些头。当我们``关闭''这些头部（即将其输出设置为零）时，上下文夹带的效果会大大减弱，从而导致该模型产生输出，如果没有提供分散注意力的上下文，则会产生其产生的内容。我们发现上下文夹带的发现，以及我们通过夹带头对LM分心的调查，标志着朝着机械分析和缓解分散问题问题的关键一步。

Title: Qwen3 Technical Report

• Authors: An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, Zihan Qiu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.09388
• Pdf URL: https://arxiv.org/pdf/2505.09388
• Copy Paste: [[2505.09388]] Qwen3 Technical Report(https://arxiv.org/abs/2505.09388)
• Keywords: language model, gpt, llm, chat, agent
• Abstract: In this work, we present Qwen3, the latest version of the Qwen model family. Qwen3 comprises a series of large language models (LLMs) designed to advance performance, efficiency, and multilingual capabilities. The Qwen3 series includes models of both dense and Mixture-of-Expert (MoE) architectures, with parameter scales ranging from 0.6 to 235 billion. A key innovation in Qwen3 is the integration of thinking mode (for complex, multi-step reasoning) and non-thinking mode (for rapid, context-driven responses) into a unified framework. This eliminates the need to switch between different models--such as chat-optimized models (e.g., GPT-4o) and dedicated reasoning models (e.g., QwQ-32B)--and enables dynamic mode switching based on user queries or chat templates. Meanwhile, Qwen3 introduces a thinking budget mechanism, allowing users to allocate computational resources adaptively during inference, thereby balancing latency and performance based on task complexity. Moreover, by leveraging the knowledge from the flagship models, we significantly reduce the computational resources required to build smaller-scale models, while ensuring their highly competitive performance. Empirical evaluations demonstrate that Qwen3 achieves state-of-the-art results across diverse benchmarks, including tasks in code generation, mathematical reasoning, agent tasks, etc., competitive against larger MoE models and proprietary models. Compared to its predecessor Qwen2.5, Qwen3 expands multilingual support from 29 to 119 languages and dialects, enhancing global accessibility through improved cross-lingual understanding and generation capabilities. To facilitate reproducibility and community-driven research and development, all Qwen3 models are publicly accessible under Apache 2.0.
• 摘要：在这项工作中，我们介绍了Qwen3，这是QWEN模型系列的最新版本。 QWEN3包括一系列旨在提高性能，效率和多语言功能的大型语言模型（LLM）。 QWEN3系列包括稠密和混合物（MOE）架构的模型，参数尺度范围为0.6至2350亿。 QWEN3中的一个关键创新是整合思维模式（用于复杂，多步推理）和非思考模式（用于快速，上下文驱动的响应）中的统一框架。这消除了在不同模型之间切换的需求，例如聊天优化的模型（例如GPT-4O）和专用的推理模型（例如QWQ-32B） - 并根据用户查询或聊天模板启用动态模式切换。同时，QWEN3引入了一种思维预算机制，使用户可以在推理过程中自适应地分配计算资源，从而根据任务复杂性平衡延迟和性能。此外，通过利用旗舰模型中的知识，我们大大减少了建立较小规模的模型所需的计算资源，同时确保其竞争激烈的性能。经验评估表明，QWEN3在不同的基准中取得了最新的结果，包括代码生成，数学推理，代理任务等方面的任务，与更大的MOE模型和专有模型竞争。与其前身QWEN2.5相比，QWEN3将多语言支持从29种到119种语言和方言扩展，从而通过提高跨语性理解和发电能力来增强全球可访问性。为了促进可重复性和社区驱动的研发，所有QWEN3模型均可在Apache 2.0下公开访问。

Title: Multilingual Machine Translation with Quantum Encoder Decoder Attention-based Convolutional Variational Circuits

• Authors: Subrit Dikshit, Ritu Tiwari, Priyank Jain
• Subjects: cs.CL, cs.AI, cs.ET
• Abstract URL: https://arxiv.org/abs/2505.09407
• Pdf URL: https://arxiv.org/pdf/2505.09407
• Copy Paste: [[2505.09407]] Multilingual Machine Translation with Quantum Encoder Decoder Attention-based Convolutional Variational Circuits(https://arxiv.org/abs/2505.09407)
• Keywords: language model, gpt, chat
• Abstract: Cloud-based multilingual translation services like Google Translate and Microsoft Translator achieve state-of-the-art translation capabilities. These services inherently use large multilingual language models such as GRU, LSTM, BERT, GPT, T5, or similar encoder-decoder architectures with attention mechanisms as the backbone. Also, new age natural language systems, for instance ChatGPT and DeepSeek, have established huge potential in multiple tasks in natural language processing. At the same time, they also possess outstanding multilingual translation capabilities. However, these models use the classical computing realm as a backend. QEDACVC (Quantum Encoder Decoder Attention-based Convolutional Variational Circuits) is an alternate solution that explores the quantum computing realm instead of the classical computing realm to study and demonstrate multilingual machine translation. QEDACVC introduces the quantum encoder-decoder architecture that simulates and runs on quantum computing hardware via quantum convolution, quantum pooling, quantum variational circuit, and quantum attention as software alterations. QEDACVC achieves an Accuracy of 82% when trained on the OPUS dataset for English, French, German, and Hindi corpora for multilingual translations.
• 摘要：基于云的多语言翻译服务，例如Google Translate和Microsoft Translator获得最新的翻译功能。这些服务固有地使用大型多语言模型，例如GRU，LSTM，BERT，GPT，T5或类似的Encoder-Decoder Architectures，其注意机制作为骨干。此外，新时代的自然语言系统（例如Chatgpt and Deepseek）在自然语言处理的多个任务中已经确立了巨大的潜力。同时，它们还具有出色的多语言翻译功能。但是，这些模型将经典计算领域用作后端。 QEDACVC（基于量子编码器解码器基于注意的卷积变化电路）是替代量子计算领域的替代解决方案，而不是经典的计算领域，以研究和演示多语言的机器翻译。 QEDACVC引入了量子编码器架构，该体系结构通过量子卷积，量子池，量子变异电路在量子计算硬件上模拟并运行，并作为软件更改。当在使用英语，法语，德语和印地语语料库的Opus数据集中培训用于多语言翻译时，QedACVC的精度为82％。

Title: PT-MoE: An Efficient Finetuning Framework for Integrating Mixture-of-Experts into Prompt Tuning

• Authors: Zongqian Li, Yixuan Su, Nigel Collier
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.09519
• Pdf URL: https://arxiv.org/pdf/2505.09519
• Copy Paste: [[2505.09519]] PT-MoE: An Efficient Finetuning Framework for Integrating Mixture-of-Experts into Prompt Tuning(https://arxiv.org/abs/2505.09519)
• Keywords: language model, prompt
• Abstract: Parameter-efficient fine-tuning (PEFT) methods have shown promise in adapting large language models, yet existing approaches exhibit counter-intuitive phenomena: integrating router into prompt tuning (PT) increases training efficiency yet does not improve performance universally; parameter reduction through matrix decomposition can improve performance in specific domains. Motivated by these observations and the modular nature of PT, we propose PT-MoE, a novel framework that integrates matrix decomposition with mixture-of-experts (MoE) routing for efficient PT. Results across 17 datasets demonstrate that PT-MoE achieves state-of-the-art performance in both question answering (QA) and mathematical problem solving tasks, improving F1 score by 1.49 points over PT and 2.13 points over LoRA in QA tasks, while enhancing mathematical accuracy by 10.75 points over PT and 0.44 points over LoRA, all while using 25% fewer parameters than LoRA. Our analysis reveals that while PT methods generally excel in QA tasks and LoRA-based methods in math datasets, the integration of matrix decomposition and MoE in PT-MoE yields complementary benefits: decomposition enables efficient parameter sharing across experts while MoE provides dynamic adaptation, collectively enabling PT-MoE to demonstrate cross-task consistency and generalization abilities. These findings, along with ablation studies on routing mechanisms and architectural components, provide insights for future PEFT methods.
• 摘要：参数有效的微调（PEFT）方法在调整大语言模型方面已经显示出希望，但是现有的方法表现出反直觉现象：将路由器整合到及时调整（PT）提高训练效率，但并不能普遍提高性能；通过基质分解的参数降低可以提高特定域中的性能。由这些观察结果和PT的模块化性质的动机，我们提出了PT-MOE，这是一个新型框架，将矩阵分解与Experts（MOE）混合物（MOE）路由集成以获得有效的PT。 17个数据集中的结果表明，PT-MOE在问答（QA）和数学问题解决任务中都能达到最先进的表现，在PT上提高了F1分数1.49点，而Lora在QA任务中的得分为2.13点，同时在PT上增加了10.75点的数学精度和0.44点的数学精确度，同时使用LORA，而不是使用25％的LORA。我们的分析表明，虽然PT方法通常在质量检查任务和基于LORA的方法中表现出色，但在PT-MOE中，矩阵分解和MOE的整合产生了互补的好处：分解可以使专家之间有效的参数共享，而MOE则可以进行动态适应，从而实现PT-MOE，从而表现出跨压缩的一致性和一般的一般化度和一般化的一般化。这些发现，以及有关路由机制和建筑组件的消融研究，为未来的PEFT方法提供了见解。

Title: WorldView-Bench: A Benchmark for Evaluating Global Cultural Perspectives in Large Language Models

• Authors: Abdullah Mushtaq, Imran Taj, Rafay Naeem, Ibrahim Ghaznavi, Junaid Qadir
• Subjects: cs.CL, cs.AI, cs.CY, cs.MA
• Abstract URL: https://arxiv.org/abs/2505.09595
• Pdf URL: https://arxiv.org/pdf/2505.09595
• Copy Paste: [[2505.09595]] WorldView-Bench: A Benchmark for Evaluating Global Cultural Perspectives in Large Language Models(https://arxiv.org/abs/2505.09595)
• Keywords: language model, llm, prompt, agent
• Abstract: Large Language Models (LLMs) are predominantly trained and aligned in ways that reinforce Western-centric epistemologies and socio-cultural norms, leading to cultural homogenization and limiting their ability to reflect global civilizational plurality. Existing benchmarking frameworks fail to adequately capture this bias, as they rely on rigid, closed-form assessments that overlook the complexity of cultural inclusivity. To address this, we introduce WorldView-Bench, a benchmark designed to evaluate Global Cultural Inclusivity (GCI) in LLMs by analyzing their ability to accommodate diverse worldviews. Our approach is grounded in the Multiplex Worldview proposed by Senturk et al., which distinguishes between Uniplex models, reinforcing cultural homogenization, and Multiplex models, which integrate diverse perspectives. WorldView-Bench measures Cultural Polarization, the exclusion of alternative perspectives, through free-form generative evaluation rather than conventional categorical benchmarks. We implement applied multiplexity through two intervention strategies: (1) Contextually-Implemented Multiplex LLMs, where system prompts embed multiplexity principles, and (2) Multi-Agent System (MAS)-Implemented Multiplex LLMs, where multiple LLM agents representing distinct cultural perspectives collaboratively generate responses. Our results demonstrate a significant increase in Perspectives Distribution Score (PDS) entropy from 13% at baseline to 94% with MAS-Implemented Multiplex LLMs, alongside a shift toward positive sentiment (67.7%) and enhanced cultural balance. These findings highlight the potential of multiplex-aware AI evaluation in mitigating cultural bias in LLMs, paving the way for more inclusive and ethically aligned AI systems.
• 摘要：大型语言模型（LLMS）主要受过培训和对齐的方式，以加强以西方为中心的认识论和社会文化规范，从而导致文化同质化并限制其反映全球文明多元化的能力。现有的基准测试框架无法充分捕捉这种偏见，因为它们依赖于忽略文化包容性复杂性的僵化，封闭形式的评估。为了解决这个问题，我们介绍了Worldview-Bench，这是一种基准，旨在通过分析其适应多种世界观的能力来评估LLMS中的全球文化包容性（GCI）。我们的方法基于Senturk等人提出的多重世界观，该世界观区分了单流模型，加强文化同质化和多元模型，这些模型整合了各种观点。 Worldview Bench通过自由形式的生成评估而不是常规的分类基准来衡量文化两极分化，排除替代观点。我们通过两种干预策略实现了应用的多重性：（1）系统提示嵌入的多重性原理，以及（2）多工具系统（MAS）插入的多重LLMS，其中多个LLM代理代表不同的文化观点会协同产生响应。我们的结果表明，随着MAS的多重LLMS，观点分布评分（PDS）的熵从基线的13％显着提高到94％，以及向积极情绪的转变（67.7％）和增强的文化平衡。这些发现凸显了多路复用AI评估在减轻LLM中文化偏见中的潜力，为更具包容性和道德上一致的AI系统铺平了道路。