2025-07-16

Title: Truth Sleuth and Trend Bender: AI Agents to fact-check YouTube videos and influence opinions

• Authors: Logé Cécile, Ghori Rehan
• Subjects: cs.CL, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2507.10577
• Pdf URL: https://arxiv.org/pdf/2507.10577
• Copy Paste: [[2507.10577]] Truth Sleuth and Trend Bender: AI Agents to fact-check YouTube videos and influence opinions(https://arxiv.org/abs/2507.10577)
• Keywords: prompt, retrieval-augmented generation, agent
• Abstract: Misinformation poses a significant threat in today's digital world, often spreading rapidly through platforms like YouTube. This paper introduces a novel approach to combating misinformation by developing an AI-powered system that not only fact-checks claims made in YouTube videos but also actively engages users in the comment section and challenge misleading narratives. Our system comprises two main agents: Truth Sleuth and Trend Bender. Truth Sleuth extracts claims from a YouTube video, uses a Retrieval-Augmented Generation (RAG) approach - drawing on sources like Wikipedia, Google Search, Google FactCheck - to accurately assess their veracity and generates a nuanced and comprehensive report. Through rigorous prompt engineering, Trend Bender leverages this report along with a curated corpus of relevant articles to generate insightful and persuasive comments designed to stimulate a productive debate. With a carefully set up self-evaluation loop, this agent is able to iteratively improve its style and refine its output. We demonstrate the system's capabilities through experiments on established benchmark datasets and a real-world deployment on YouTube, showcasing its potential to engage users and potentially influence perspectives. Our findings highlight the high accuracy of our fact-checking agent, and confirm the potential of AI-driven interventions in combating misinformation and fostering a more informed online space.
• 摘要：错误信息在当今的数字世界中构成了重大威胁，通常通过YouTube等平台迅速传播。本文介绍了一种新颖的方法，可以通过开发一个由AI驱动的系统来打击错误信息，该系统不仅是事实检查YouTube视频中提出的事实，而且还积极吸引用户参与评论部分并挑战误导性叙事。我们的系统包括两个主要代理：真相侦探和趋势弯曲者。 Truth Sleuth摘录从YouTube视频中提取的主张，使用检索功能的一代（RAG）方法 - 借鉴Wikipedia，Google Search，Google Factcheck等资源，以准确评估其真实性并产生细微差别和全面的报告。通过严格的及时工程，趋势弯曲者利用本报告以及一系列相关文章的策划，以引起旨在刺激富有成效辩论的有见地和有说服力的评论。经过精心设置的自我评估循环，该代理可以迭代地改善其样式并完善其输出。我们通过在YouTube上建立的基准数据集和现实世界部署的实验来演示系统的功能，展示其吸引用户的潜力并潜在地影响观点。我们的发现突出了我们事实检查代理的高度准确性，并确认了AI驱动的干预措施在打击错误信息和促进更明智的在线空间方面的潜力。

Title: An Offline Mobile Conversational Agent for Mental Health Support: Learning from Emotional Dialogues and Psychological Texts with Student-Centered Evaluation

• Authors: Vimaleswar A, Prabhu Nandan Sahu, Nilesh Kumar Sahu, Haroon R Lone
• Subjects: cs.CL, cs.AI, cs.CY, cs.HC
• Abstract URL: https://arxiv.org/abs/2507.10580
• Pdf URL: https://arxiv.org/pdf/2507.10580
• Copy Paste: [[2507.10580]] An Offline Mobile Conversational Agent for Mental Health Support: Learning from Emotional Dialogues and Psychological Texts with Student-Centered Evaluation(https://arxiv.org/abs/2507.10580)
• Keywords: language model, llm, agent
• Abstract: Mental health plays a crucial role in the overall well-being of an individual. In recent years, digital platforms have been increasingly used to expand mental health and emotional support. However, there are persistent challenges related to limited user accessibility, internet connectivity, and data privacy, which highlight the need for an offline, smartphone-based solution. To address these challenges, we propose EmoSApp (Emotional Support App): an entirely offline, smartphone-based conversational app designed for mental health and emotional support. The system leverages Large Language Models (LLMs), specifically fine-tuned, quantized and deployed using Torchtune and Executorch for resource-constrained devices, allowing all inferences to occur on the smartphone. To equip EmoSApp with robust domain expertise, we fine-tuned the LLaMA-3.2-1B-Instruct model on our custom curated ``Knowledge dataset'' of 14,582 mental-health QA pairs, along with the multi-turn conversational data. Through qualitative human evaluation with the student population, we demonstrate that EmoSApp has the ability to respond coherently, empathetically, maintain interactive dialogue, and provide relevant suggestions to user's mental health problems. Additionally, quantitative evaluations on nine standard commonsense and reasoning benchmarks demonstrate the efficacy of our fine-tuned, quantized model in low-resource settings. By prioritizing on-device deployment and specialized domain adaptation, EmoSApp serves as a blueprint for future innovations in portable, secure, and highly tailored AI-driven mental health solutions.
• 摘要：心理健康在个人的整体福祉中起着至关重要的作用。近年来，数字平台越来越多地用于扩大心理健康和情感支持。但是，与有限的用户可访问性，Internet连接性和数据隐私相关的持续挑战，这突出了对离线，基于智能手机的解决方案的需求。为了应对这些挑战，我们提出了EMOSAPP（情感支持应用程序）：一个完全离线的，基于智能手机的对话应用，旨在精神健康和情感支持。该系统利用大型语言模型（LLMS），专门调整，使用Torchtune和Executorch进行资源受限的设备进行了微调，量化和部署，从而使所有推论都可以在智能手机上进行。为了配备EMOSAPP具有强大的领域专业知识，我们在我们的自定义策划的``知识数据集''中微调了Llama-3.2-1b-Instruct模型，其中14,582个心理健康QA对，以及多转向对话数据。通过与学生群体的定性人类评估，我们证明了EMOSAPP具有连贯，善解人意，维持互动对话的能力，并为用户的心理健康问题提供相关建议。此外，对九个标准常识和推理基准的定量评估证明了我们在低资源环境中进行微调，量化模型的功效。通过优先考虑设备部署和专业领域的适应，EMOSAPP可以作为便携式，安全且高度量身定制的AI-drien驱动心理健康解决方案的未来创新的蓝图。

Title: Transforming Sensitive Documents into Quantitative Data: An AI-Based Preprocessing Toolchain for Structured and Privacy-Conscious Analysis

• Authors: Anders Ledberg, Anna Thalén
• Subjects: cs.CL, stat.ME
• Abstract URL: https://arxiv.org/abs/2507.10582
• Pdf URL: https://arxiv.org/pdf/2507.10582
• Copy Paste: [[2507.10582]] Transforming Sensitive Documents into Quantitative Data: An AI-Based Preprocessing Toolchain for Structured and Privacy-Conscious Analysis(https://arxiv.org/abs/2507.10582)
• Keywords: language model, llm, prompt
• Abstract: Unstructured text from legal, medical, and administrative sources offers a rich but underutilized resource for research in public health and the social sciences. However, large-scale analysis is hampered by two key challenges: the presence of sensitive, personally identifiable information, and significant heterogeneity in structure and language. We present a modular toolchain that prepares such text data for embedding-based analysis, relying entirely on open-weight models that run on local hardware, requiring only a workstation-level GPU and supporting privacy-sensitive research. The toolchain employs large language model (LLM) prompting to standardize, summarize, and, when needed, translate texts to English for greater comparability. Anonymization is achieved via LLM-based redaction, supplemented with named entity recognition and rule-based methods to minimize the risk of disclosure. We demonstrate the toolchain on a corpus of 10,842 Swedish court decisions under the Care of Abusers Act (LVM), comprising over 56,000 pages. Each document is processed into an anonymized, standardized summary and transformed into a document-level embedding. Validation, including manual review, automated scanning, and predictive evaluation shows the toolchain effectively removes identifying information while retaining semantic content. As an illustrative application, we train a predictive model using embedding vectors derived from a small set of manually labeled summaries, demonstrating the toolchain's capacity for semi-automated content analysis at scale. By enabling structured, privacy-conscious analysis of sensitive documents, our toolchain opens new possibilities for large-scale research in domains where textual data was previously inaccessible due to privacy and heterogeneity constraints.
• 摘要：法律，医疗和行政资料中的非结构化文本为公共卫生和社会科学研究提供了丰富但未充分利用的资源。但是，大规模分析受到两个关键挑战的阻碍：敏感，个人身份信息的存在以及结构和语言的重要异质性。我们提出了一个模块化工具链，该工具链为基于嵌入的分析准备了此类文本数据，完全依赖于在本地硬件上运行的开放权重模型，仅需要工作站级的GPU并支持对隐私敏感的研究。该工具链采用大型语言模型（LLM）提示标准化，总结，并在需要时将文本转换为英语以获得更大的可比性。通过基于LLM的修订来实现匿名化，并补充了指定的实体识别和基于规则的方法，以最大程度地减少披露风险。我们根据《虐待者法》（LVM）的10,842个瑞典法院判决的语料库证明了该工具链，其中包括超过56,000页。每个文档都被处理成一个匿名，标准化的摘要，并将其转换为文档级嵌入。验证，包括手动审查，自动扫描和预测性评估，显示工具链有效地消除了识别信息，同时保留语义内容。作为说明性应用程序，我们使用嵌入载体衍生自手动标记的摘要的嵌入向量训练预测模型，这表明了工具链按大规模进行半自动内容分析的能力。通过启用对敏感文档的结构化的，具有隐私意识的分析，我们的工具链为在域中的大规模研究开辟了新的可能性，在这些域中，由于隐私和异质性约束，文本数据以前无法访问。

Title: A Taxonomy for Design and Evaluation of Prompt-Based Natural Language Explanations

• Authors: Isar Nejadgholi, Mona Omidyeganeh, Marc-Antoine Drouin, Jonathan Boisvert
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.10585
• Pdf URL: https://arxiv.org/pdf/2507.10585
• Copy Paste: [[2507.10585]] A Taxonomy for Design and Evaluation of Prompt-Based Natural Language Explanations(https://arxiv.org/abs/2507.10585)
• Keywords: language model, prompt
• Abstract: Effective AI governance requires structured approaches for stakeholders to access and verify AI system behavior. With the rise of large language models, Natural Language Explanations (NLEs) are now key to articulating model behavior, which necessitates a focused examination of their characteristics and governance implications. We draw on Explainable AI (XAI) literature to create an updated XAI taxonomy, adapted to prompt-based NLEs, across three dimensions: (1) Context, including task, data, audience, and goals; (2) Generation and Presentation, covering generation methods, inputs, interactivity, outputs, and forms; and (3) Evaluation, focusing on content, presentation, and user-centered properties, as well as the setting of the evaluation. This taxonomy provides a framework for researchers, auditors, and policymakers to characterize, design, and enhance NLEs for transparent AI systems.
• 摘要：有效的AI治理要求利益相关者访问和验证AI系统行为的结构化方法。随着大语言模型的兴起，自然语言解释（NLE）现在是表达模型行为的关键，这需要集中研究其特征和治理含义。我们借鉴了可解释的AI（XAI）文献来创建更新的XAI分类法，该分类法适用于迅速的NLE，跨三个维度：（1）上下文，包括任务，数据，受众和目标； （2）生成和演示，涵盖生成方法，输入，交互性，输出和形式； （3）评估，专注于内容，演示和以用户为中心的属性以及评估的设置。该分类法为研究人员，审计师和决策者提供了一个框架，以表征，设计和增强透明AI系统的NLE。

Title: AutoRAG-LoRA: Hallucination-Triggered Knowledge Retuning via Lightweight Adapters

• Authors: Kaushik Dwivedi, Padmanabh Patanjali Mishra
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.10586
• Pdf URL: https://arxiv.org/pdf/2507.10586
• Copy Paste: [[2507.10586]] AutoRAG-LoRA: Hallucination-Triggered Knowledge Retuning via Lightweight Adapters(https://arxiv.org/abs/2507.10586)
• Keywords: language model, llm, hallucination, prompt, retrieval-augmented generation
• Abstract: Large Language Models (LLMs) have demonstrated remarkable fluency across a range of natural language tasks, yet remain vulnerable to hallucinations - factual inaccuracies that undermine trust in real world deployment. We present AutoRAG-LoRA, a modular framework for Retrieval-Augmented Generation (RAG) that tackles hallucination in large language models through lightweight LoRA-based adapters and KL-regularized training. Our pipeline integrates automated prompt rewriting, hybrid retrieval, and low-rank adapter tuning to ground responses in retrieved evidence. A hallucination detection module, using both classifier-based and self-evaluation techniques, assigns confidence scores to generated outputs, triggering an optional feedback correction loop. This loop enforces factual alignment via contrastive KL loss and adapter fine tuning. We demonstrate that AutoRAG-LoRA significantly reduces the factual drift while preserving the efficiency and modularity of the model.
• 摘要：大型语言模型（LLMS）在一系列自然语言任务中表现出了显着的流利性，但仍然容易受到幻觉的影响 - 破坏对现实世界部署的信任的事实不准确。我们提出了Autorag-lora，这是一个用于检索的模块化框架（RAG），该框架通过基于Lora的轻质适配器和KL登记的培训来解决大语言模型中的幻觉。我们的管道集成了自动化的及时重写，混合检索和低级别适配器调整到地面响应中，以在检索到的证据中。使用基于分类器和自我评估技术的幻觉检测模块将置信分数分配给生成的输出，从而触发可选的反馈校正循环。该循环通过对比的KL损失和适配器微调来实现事实对准。我们证明，Autorag-Lora在保留模型的效率和模块化的同时大大降低了事实漂移。

Title: Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing

• Authors: Dennis Ulmer, Alexandra Lorson, Ivan Titov, Christian Hardmeier
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.10587
• Pdf URL: https://arxiv.org/pdf/2507.10587
• Copy Paste: [[2507.10587]] Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing(https://arxiv.org/abs/2507.10587)
• Keywords: language model, llm
• Abstract: Human users increasingly rely on natural language interactions with large language models (LLMs) in order to receive help on a large variety of tasks and problems. However, the trustworthiness and perceived legitimacy of LLMs is undermined by the fact that their output is frequently stated in very confident terms, even when its accuracy is questionable. Therefore, there is a need to signal the confidence of the language model to a user in order to reap the benefits of human-machine collaboration and mitigate potential harms. Verbalized uncertainty is the expression of confidence with linguistic means, an approach that integrates perfectly into language-based interfaces. Nevertheless, most recent research in natural language processing (NLP) overlooks the nuances surrounding human uncertainty communication and the data biases that influence machine uncertainty communication. We argue for anthropomimetic uncertainty, meaning that intuitive and trustworthy uncertainty communication requires a degree of linguistic authenticity and personalization to the user, which could be achieved by emulating human communication. We present a thorough overview over the research in human uncertainty communication, survey ongoing research, and perform additional analyses to demonstrate so-far overlooked biases in verbalized uncertainty. We conclude by pointing out unique factors in human-machine communication of uncertainty and deconstruct anthropomimetic uncertainty into future research directions for NLP.
• 摘要：人类用户越来越依赖于与大语言模型（LLM）的自然语言互动，以便在各种任务和问题上获得帮助。但是，LLM的可信赖性和认为的合法性受到以下事实的损害，即即使其准确性值得怀疑，它们的输出经常被非常自信地说明。因此，有必要向用户发出语言模型的信心，以便获得人机协作的好处并减轻潜在的危害。口头上的不确定性是用语言手段表达信心的表达，这种方法可以完美地集成到基于语言的界面中。然而，最新的自然语言处理研究（NLP）忽略了人类不确定性通信的细微差别以及影响机器不确定性通信的数据偏见。我们主张拟人化的不确定性，这意味着直觉和值得信赖的不确定性交流需要对用户的语言真实性和个性化程度，这可以通过模仿人类的交流来实现。我们对人类不确定性沟通，调查正在进行的研究的研究进行了详尽的概述，并进行了其他分析，以证明言语不确定性的偏见。最后，我们指出了不确定性的人机通信中的独特因素，并解构了NLP未来研究方向的拟人化不确定性。

Title: PLEX: Perturbation-free Local Explanations for LLM-Based Text Classification

• Authors: Yogachandran Rahulamathavan, Misbah Farooq, Varuna De Silva
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.10596
• Pdf URL: https://arxiv.org/pdf/2507.10596
• Copy Paste: [[2507.10596]] PLEX: Perturbation-free Local Explanations for LLM-Based Text Classification(https://arxiv.org/abs/2507.10596)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) excel in text classification, but their complexity hinders interpretability, making it difficult to understand the reasoning behind their predictions. Explainable AI (XAI) methods like LIME and SHAP offer local explanations by identifying influential words, but they rely on computationally expensive perturbations. These methods typically generate thousands of perturbed sentences and perform inferences on each, incurring a substantial computational burden, especially with LLMs. To address this, we propose \underline{P}erturbation-free \underline{L}ocal \underline{Ex}planation (PLEX), a novel method that leverages the contextual embeddings extracted from the LLM and a ``Siamese network" style neural network trained to align with feature importance scores. This one-off training eliminates the need for subsequent perturbations, enabling efficient explanations for any new sentence. We demonstrate PLEX's effectiveness on four different classification tasks (sentiment, fake news, fake COVID-19 news and depression), showing more than 92\% agreement with LIME and SHAP. Our evaluation using a ``stress test" reveals that PLEX accurately identifies influential words, leading to a similar decline in classification accuracy as observed with LIME and SHAP when these words are removed. Notably, in some cases, PLEX demonstrates superior performance in capturing the impact of key features. PLEX dramatically accelerates explanation, reducing time and computational overhead by two and four orders of magnitude, respectively. This work offers a promising solution for explainable LLM-based text classification.
• 摘要：大型语言模型（LLM）在文本分类中表现出色，但是它们的复杂性阻碍了可解释性，因此很难理解其预测背后的推理。可解释的AI（XAI）方法（例如石灰和Shap）通过识别有影响力的词来提供局部解释，但它们依赖于计算昂贵的扰动。这些方法通常会产生数千个受干扰的句子，并对每种句子执行推断，从而产生重大的计算负担，尤其是在LLMS上。为了解决这个问题，我们提出\下划线{p}无逆转\下划线{l} ocal \下划线{ex}平面（plex），一种新方法，它利用了从LLM中提取的上下文嵌入的新方法，从LLM中提取的上下文嵌入，并``siamese''s``siamese网络训练了一个需要培训的样式网络，供您进行培训，以培训这一范围，以验证这一范围，以验证这一范围，以替代了一部分。对任何新句子的有效解释。值得注意的是，在某些情况下，PLEX在捕获关键特征的影响方面表现出卓越的性能。 PLEX显着加速了解释，将时间和计算开销降低了两个和四个数量级。这项工作为基于LLM的文本分类提供了有希望的解决方案。

Title: Emergence of Hierarchical Emotion Organization in Large Language Models

• Authors: Bo Zhao, Maya Okawa, Eric J. Bigelow, Rose Yu, Tomer Ullman, Ekdeep Singh Lubana, Hidenori Tanaka
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.10599
• Pdf URL: https://arxiv.org/pdf/2507.10599
• Copy Paste: [[2507.10599]] Emergence of Hierarchical Emotion Organization in Large Language Models(https://arxiv.org/abs/2507.10599)
• Keywords: language model, llm, agent
• Abstract: As large language models (LLMs) increasingly power conversational agents, understanding how they model users' emotional states is critical for ethical deployment. Inspired by emotion wheels -- a psychological framework that argues emotions organize hierarchically -- we analyze probabilistic dependencies between emotional states in model outputs. We find that LLMs naturally form hierarchical emotion trees that align with human psychological models, and larger models develop more complex hierarchies. We also uncover systematic biases in emotion recognition across socioeconomic personas, with compounding misclassifications for intersectional, underrepresented groups. Human studies reveal striking parallels, suggesting that LLMs internalize aspects of social perception. Beyond highlighting emergent emotional reasoning in LLMs, our results hint at the potential of using cognitively-grounded theories for developing better model evaluations.
• 摘要：随着大型语言模型（LLM）越来越多地对话代理人，了解它们如何建模用户的情绪状态对于道德部署至关重要。受情感轮毂的启发 - 一种心理框架，认为情绪在层次上组织 - 我们分析了模型输出中情绪状态之间的概率依赖性。我们发现，LLM自然形成与人类心理模型保持一致的分层情感树，并且更大的模型会发展出更复杂的层次结构。我们还发现了社会经济角色跨情绪识别的系统偏见，对交叉，代表性不足的群体的错误分类变得更加复杂。人类研究表明，引人注目的相似之处，表明LLM会内化社会感知的各个方面。除了强调LLM中出现的情绪推理外，我们的结果还暗示了使用认知基理论来开发更好的模型评估的潜力。

Title: Language Models for Adult Service Website Text Analysis

• Authors: Nickolas Freeman, Thanh Nguyen, Gregory Bott, Jason Parton, Collin Francel
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.10743
• Pdf URL: https://arxiv.org/pdf/2507.10743
• Copy Paste: [[2507.10743]] Language Models for Adult Service Website Text Analysis(https://arxiv.org/abs/2507.10743)
• Keywords: language model
• Abstract: Sex trafficking refers to the use of force, fraud, or coercion to compel an individual to perform in commercial sex acts against their will. Adult service websites (ASWs) have and continue to be linked to sex trafficking, offering a platform for traffickers to advertise their victims. Thus, organizations involved in the fight against sex trafficking often use ASW data when attempting to identify potential sex trafficking victims. A critical challenge in transforming ASW data into actionable insight is text analysis. Previous research using ASW data has shown that ASW ad text is important for linking ads. However, working with this text is challenging due to its extensive use of emojis, poor grammar, and deliberate obfuscation to evade law enforcement scrutiny. We conduct a comprehensive study of language modeling approaches for this application area, including simple information retrieval methods, pre-trained transformers, and custom transformer models. We demonstrate that characteristics of ASW text data allow efficient custom transformer models to be trained with relatively small GPU resources and used efficiently for inference on consumer hardware. Our custom models outperform fine-tuned variants of well-known encoder-only transformer models, including BERT-base, RoBERTa, and ModernBERT, on accuracy, recall, F1 score, and ROC AUC. We demonstrate the use of our best-performing custom configuration on three tasks related to ASW data analysis: (i) decomposing the giant component in a graph representation of ASW data, (ii) clustering ASW ad text, and (iii) using the learned token embeddings to understand the use of emojis in the illicit context we study. The models we develop represent a significant advancement in ASW text analysis, which can be leveraged in a variety of downstream applications and research.
• 摘要：性贩运是指使用武力，欺诈或胁迫迫使个人违背自己的意愿进行商业性行为。成人服务网站（ASWS）已并且继续与性贩运联系起来，为贩运者提供宣传其受害者的平台。因此，参与反对性贩运的组织在试图识别潜在的性贩运受害者时通常会使用ASW数据。将ASW数据转换为可行的见解的关键挑战是文本分析。使用ASW数据的先前研究表明，ASW广告文本对于链接广告很重要。但是，由于广泛使用表情符号，语法差和故意混淆以逃避执法审查，因此使用此文本是具有挑战性的。我们对该应用领域的语言建模方法进行了全面研究，包括简单的信息检索方法，预训练的变压器和自定义变压器模型。我们证明，ASW文本数据的特征使有效的自定义变压器模型可以接受相对较小的GPU资源培训，并有效地用于推断消费者硬件。我们的自定义模型优于众所周知的仅编码变压器模型的微调变体，包括Bert-Base，Roberta和Modernbert，关于准确性，召回，F1分数和ROC AUC。我们证明了在与ASW数据分析相关的三个任务上使用最佳性能的自定义配置：（i）将巨型组件分解为ASW数据的图表，（ii）群集ASW ASW AD AD文本，（iii）使用学识渊博的令牌嵌入者在我们研究我们研究中使用emojis的使用。我们开发的模型代表了ASW文本分析的重大进步，可以在各种下游应用程序和研究中利用。

Title: Applying Text Embedding Models for Efficient Analysis in Labeled Property Graphs

• Authors: Michal Podstawski
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2507.10772
• Pdf URL: https://arxiv.org/pdf/2507.10772
• Copy Paste: [[2507.10772]] Applying Text Embedding Models for Efficient Analysis in Labeled Property Graphs(https://arxiv.org/abs/2507.10772)
• Keywords: language model
• Abstract: Labeled property graphs often contain rich textual attributes that can enhance analytical tasks when properly leveraged. This work explores the use of pretrained text embedding models to enable efficient semantic analysis in such graphs. By embedding textual node and edge properties, we support downstream tasks including node classification and relation prediction with improved contextual understanding. Our approach integrates language model embeddings into the graph pipeline without altering its structure, demonstrating that textual semantics can significantly enhance the accuracy and interpretability of property graph analysis.
• 摘要：标记的属性图通常包含丰富的文本属性，这些属性可以在正确利用时可以增强分析任务。这项工作探讨了验证的文本嵌入模型以在此类图中实现有效的语义分析。通过嵌入文本节点和边缘属性，我们支持下游任务，包括节点分类和关系预测，并改善上下文理解。我们的方法将嵌入语言模型嵌入到图形管道中而不改变其结构的情况下，表明文本语义可以显着提高属性图分析的准确性和解释性。

Title: LLMs on Trial: Evaluating Judicial Fairness for Large Language Models

• Authors: Yiran Hu, Zongyue Xue, Haitao Li, Siyuan Zheng, Qingjing Chen, Shaochun Wang, Xihan Zhang, Ning Zheng, Yun Liu, Qingyao Ai, Yiqun Liu, Charles L.A. Clarke, Weixing Shen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.10852
• Pdf URL: https://arxiv.org/pdf/2507.10852
• Copy Paste: [[2507.10852]] LLMs on Trial: Evaluating Judicial Fairness for Large Language Models(https://arxiv.org/abs/2507.10852)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) are increasingly used in high-stakes fields where their decisions impact rights and equity. However, LLMs' judicial fairness and implications for social justice remain underexplored. When LLMs act as judges, the ability to fairly resolve judicial issues is a prerequisite to ensure their trustworthiness. Based on theories of judicial fairness, we construct a comprehensive framework to measure LLM fairness, leading to a selection of 65 labels and 161 corresponding values. Applying this framework to the judicial system, we compile an extensive dataset, JudiFair, comprising 177,100 unique case facts. To achieve robust statistical inference, we develop three evaluation metrics, inconsistency, bias, and imbalanced inaccuracy, and introduce a method to assess the overall fairness of multiple LLMs across various labels. Through experiments with 16 LLMs, we uncover pervasive inconsistency, bias, and imbalanced inaccuracy across models, underscoring severe LLM judicial unfairness. Particularly, LLMs display notably more pronounced biases on demographic labels, with slightly less bias on substance labels compared to procedure ones. Interestingly, increased inconsistency correlates with reduced biases, but more accurate predictions exacerbate biases. While we find that adjusting the temperature parameter can influence LLM fairness, model size, release date, and country of origin do not exhibit significant effects on judicial fairness. Accordingly, we introduce a publicly available toolkit containing all datasets and code, designed to support future research in evaluating and improving LLM fairness.
• 摘要：大型语言模型（LLM）越来越多地用于其决策影响权利和公平性的高风险领域。但是，LLMS的司法公平和对社会正义的影响仍然没有得到充实。当LLM充当法官时，公平解决司法问题的能力是确保其信任度的先决条件。基于司法公平理论，我们构建了一个综合框架来衡量LLM公平性，从而选择了65个标签和161个相应的值。将此框架应用于司法系统，我们编制了一个广泛的数据集，朱迪法尔（Judifair）包括177,100个独特的案例事实。为了实现强大的统计推断，我们开发了三个评估指标，不一致，偏见和不平衡的不准确性，并引入了一种评估各种标签中多个LLM的整体公平性的方法。通过使用16个LLM的实验，我们发现了整个模型的普遍不一致，偏见和不平衡的不平衡，这突显了严重的LLM司法不公平。特别是，LLM在人群标签上显示出更明显的偏见，与手术标签相比，物质标签的偏差略少一些。有趣的是，增加的不一致与偏差减少相关，但更准确的预测加剧了偏见。尽管我们发现调整温度参数会影响LLM的公平性，模型大小，释放日期和原产国对司法公平没有显着影响。因此，我们引入了一个公开可用的工具包，其中包含所有数据集和代码，旨在支持评估和改善LLM公平性的未来研究。

Title: HanjaBridge: Resolving Semantic Ambiguity in Korean LLMs via Hanja-Augmented Pre-Training

• Authors: Seungho Choi
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.10920
• Pdf URL: https://arxiv.org/pdf/2507.10920
• Copy Paste: [[2507.10920]] HanjaBridge: Resolving Semantic Ambiguity in Korean LLMs via Hanja-Augmented Pre-Training(https://arxiv.org/abs/2507.10920)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) often show poor performance in low-resource languages like Korean, partly due to unique linguistic challenges such as homophonous Sino-Korean words that are indistinguishable in Hangul script. To address this semantic ambiguity, we propose HanjaBridge, a novel meaning-injection technique integrated into a continual pre-training (CPT) framework. Instead of deterministically mapping a word to a single Hanja (Chinese character), HanjaBridge presents the model with all possible Hanja candidates for a given homograph, encouraging the model to learn contextual disambiguation. This process is paired with token-level knowledge distillation to prevent catastrophic forgetting. Experimental results show that HanjaBridge significantly improves Korean language understanding, achieving a 21\% relative improvement on the KoBALT benchmark. Notably, by reinforcing semantic alignment between Korean and Chinese through shared Hanja, we observe a strong positive cross-lingual transfer. Furthermore, these gains persist even when Hanja augmentation is omitted at inference time, ensuring practical efficiency with no additional run-time cost.
• 摘要：大型语言模型（LLM）通常在韩语（例如韩语）中表现出较差的表现，部分原因是诸如hangul脚本中无法区分的同句中korean单词等独特的语言挑战。为了解决这种语义歧义，我们提出了Hanjabridge，这是一种集成到持续的预训练（CPT）框架中的新型意义注射技术。 Hanjabridge并没有确定性地将单词映射到单个hanja（汉字），而是为给定同型的所有可能的Hanja候选人提供了模型，鼓励模型学习上下文的歧义。此过程与令牌级知识蒸馏配对，以防止灾难性的遗忘。实验结果表明，汉贾布里奇（Hanjabridge）显着提高了韩国语言的理解，从而实现了21 \％的Kobalt基准测试。值得注意的是，通过通过共享的汉雅（Hanja）加强韩语和中国人之间的语义一致性，我们观察到了强烈的积极跨语性转移。此外，即使在推理时省略了汉加的增强，这些收益仍然存在，从而确保没有额外的运行时间成本，从而确保实践效率。

Title: Modeling Understanding of Story-Based Analogies Using Large Language Models

• Authors: Kalit Inani, Keshav Kabra, Vijay Marupudi, Sashank Varma
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.10957
• Pdf URL: https://arxiv.org/pdf/2507.10957
• Copy Paste: [[2507.10957]] Modeling Understanding of Story-Based Analogies Using Large Language Models(https://arxiv.org/abs/2507.10957)
• Keywords: language model, gpt, llm, prompt
• Abstract: Recent advancements in Large Language Models (LLMs) have brought them closer to matching human cognition across a variety of tasks. How well do these models align with human performance in detecting and mapping analogies? Prior research has shown that LLMs can extract similarities from analogy problems but lack robust human-like reasoning. Building on Webb, Holyoak, and Lu (2023), the current study focused on a story-based analogical mapping task and conducted a fine-grained evaluation of LLM reasoning abilities compared to human performance. First, it explored the semantic representation of analogies in LLMs, using sentence embeddings to assess whether they capture the similarity between the source and target texts of an analogy, and the dissimilarity between the source and distractor texts. Second, it investigated the effectiveness of explicitly prompting LLMs to explain analogies. Throughout, we examine whether LLMs exhibit similar performance profiles to those observed in humans by evaluating their reasoning at the level of individual analogies, and not just at the level of overall accuracy (as prior studies have done). Our experiments include evaluating the impact of model size (8B vs. 70B parameters) and performance variation across state-of-the-art model architectures such as GPT-4 and LLaMA3. This work advances our understanding of the analogical reasoning abilities of LLMs and their potential as models of human reasoning.
• 摘要：大型语言模型（LLM）的最新进展使他们更接近各种任务中的人类认知。这些模型在检测和映射类比中与人类绩效保持一致？先前的研究表明，LLM可以从类比问题中提取相似性，但缺乏类似人类的推理。当前的研究以Webb，Holyoak和Lu（2023）为基础，重点是基于故事的类似映射任务，与人类绩效相比，对LLM推理能力进行了精细的评估。首先，它探索了LLMS中类比的语义表示，使用句子嵌入来评估它们是否捕获了类比的源和目标文本之间的相似性，以及源文本和分散术文本之间的差异。其次，它研究了明确促使LLMS解释类比的有效性。在整个过程中，我们检查了LLM是否表现出与人类在单个类比水平上的推理，而不仅仅在整体准确性水平（如先前的研究已经完成的）中，是否表现出与人类在人类中观察到的相似的表现。我们的实验包括评估模型大小（8b vs. 70b参数）的影响以及诸如GPT-4和Llama3之类的最新模型体系结构之间的性能变化。这项工作促进了我们对LLM的类似推理能力及其作为人类推理模型的潜力的理解。

Title: DS@GT at eRisk 2025: From prompts to predictions, benchmarking early depression detection with conversational agent based assessments and temporal attention models

• Authors: Anthony Miyaguchi, David Guecha, Yuwen Chiu, Sidharth Gaur
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.10958
• Pdf URL: https://arxiv.org/pdf/2507.10958
• Copy Paste: [[2507.10958]] DS@GT at eRisk 2025: From prompts to predictions, benchmarking early depression detection with conversational agent based assessments and temporal attention models(https://arxiv.org/abs/2507.10958)
• Keywords: llm, prompt, agent
• Abstract: This Working Note summarizes the participation of the DS@GT team in two eRisk 2025 challenges. For the Pilot Task on conversational depression detection with large language-models (LLMs), we adopted a prompt-engineering strategy in which diverse LLMs conducted BDI-II-based assessments and produced structured JSON outputs. Because ground-truth labels were unavailable, we evaluated cross-model agreement and internal consistency. Our prompt design methodology aligned model outputs with BDI-II criteria and enabled the analysis of conversational cues that influenced the prediction of symptoms. Our best submission, second on the official leaderboard, achieved DCHR = 0.50, ADODL = 0.89, and ASHR = 0.27.
• 摘要：该工作说明总结了DS@GT团队在两个Erisk 2025挑战中的参与。对于大型语言模型（LLMS）的对话抑郁症检测的试验任务，我们采用了迅速的工程策略，其中不同的LLMS进行了基于BDI-II的评估并产生了结构化的JSON输出。由于无法获得地面真相标签，因此我们评估了跨模型一致性和内部一致性。我们的及时设计方法与BDI-II标准相一致，并使影响症状预测的对话提示分析。我们最好的提交在官方排行榜上第二，达到了DCHR = 0.50，ADODL = 0.89，ASHR = 0.27。

Title: Teach Me Sign: Stepwise Prompting LLM for Sign Language Production

• Authors: Zhaoyi An, Rei Kawakami
• Subjects: cs.CL, cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2507.10972
• Pdf URL: https://arxiv.org/pdf/2507.10972
• Copy Paste: [[2507.10972]] Teach Me Sign: Stepwise Prompting LLM for Sign Language Production(https://arxiv.org/abs/2507.10972)
• Keywords: language model, llm, prompt
• Abstract: Large language models, with their strong reasoning ability and rich knowledge, have brought revolution to many tasks of AI, but their impact on sign language generation remains limited due to its complexity and unique rules. In this paper, we propose TEAch Me Sign (TEAM-Sign), treating sign language as another natural language. By fine-tuning an LLM, we enable it to learn the correspondence between text and sign language, and facilitate generation. Considering the differences between sign and spoken language, we employ a stepwise prompting strategy to extract the inherent sign language knowledge within the LLM, thereby supporting the learning and generation process. Experimental results on How2Sign and Phoenix14T datasets demonstrate that our approach effectively leverages both the sign language knowledge and reasoning capabilities of LLM to align the different distribution and grammatical rules between sign and spoken language.
• 摘要：具有强大的推理能力和丰富知识的大型语言模型已将革命带入了许多AI的任务，但是由于其复杂性和独特的规则，它们对手语的影响仍然有限。在本文中，我们建议教我标志（团队签名），将手语视为另一种自然语言。通过微调LLM，我们使其能够学习文本和手语之间的对应关系，并促进生成。考虑到符号和口语之间的差异，我们采用逐步提示策略来提取LLM中固有的手语知识，从而支持学习和生成过程。关于2Sign和Phoenix14t数据集的实验结果表明，我们的方法有效地利用了LLM的手语知识和推理能力，以使符号和口语之间的不同分布和语法规则保持一致。

Title: Team HUMANE at AVeriTeC 2025: HerO 2 for Efficient Fact Verification

• Authors: Yejun Yoon, Jaeyoon Jung, Seunghyun Yoon, Kunwoo Park
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.11004
• Pdf URL: https://arxiv.org/pdf/2507.11004
• Copy Paste: [[2507.11004]] Team HUMANE at AVeriTeC 2025: HerO 2 for Efficient Fact Verification(https://arxiv.org/abs/2507.11004)
• Keywords: language model
• Abstract: This paper presents HerO 2, Team HUMANE's system for the AVeriTeC shared task at the FEVER-25 workshop. HerO 2 is an enhanced version of HerO, the best-performing open-source model from the previous year's challenge. It improves evidence quality through document summarization and answer reformulation, optimizes veracity prediction via post-training quantization under computational constraints, and enhances overall system performance by integrating updated language model (LM) backbones. HerO 2 ranked second on the leaderboard while achieving the shortest runtime among the top three systems, demonstrating both high efficiency and strong potential for real-world fact verification. The code is available at this https URL.
• 摘要：本文介绍了Hulane Team在Fever-25研讨会上为Averitec共享任务的HERO 2的Hero 2。 Hero 2是英雄的增强版本，这是上一年挑战赛表现最好的开源模型。它通过文档汇总和答案重新制作提高了证据质量，通过计算限制下的培训后量化来优化真实性预测，并通过整合更新的语言模型（LM）骨干来增强整体系统性能。英雄2在排行榜上排名第二，同时在前三名系统中达到了最短的运行时间，这既表现出高效率又表现出强大的现实事实验证潜力。该代码可在此HTTPS URL上找到。

Title: Journalism-Guided Agentic In-Context Learning for News Stance Detection

• Authors: Dahyun Lee, Jonghyeon Choi, Jiyoung Han, Kunwoo Park
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.11049
• Pdf URL: https://arxiv.org/pdf/2507.11049
• Copy Paste: [[2507.11049]] Journalism-Guided Agentic In-Context Learning for News Stance Detection(https://arxiv.org/abs/2507.11049)
• Keywords: language model, agent
• Abstract: As online news consumption grows, personalized recommendation systems have become integral to digital journalism. However, these systems risk reinforcing filter bubbles and political polarization by failing to incorporate diverse perspectives. Stance detection -- identifying a text's position on a target -- can help mitigate this by enabling viewpoint-aware recommendations and data-driven analyses of media bias. Yet, existing stance detection research remains largely limited to short texts and high-resource languages. To address these gaps, we introduce \textsc{K-News-Stance}, the first Korean dataset for article-level stance detection, comprising 2,000 news articles with article-level and 19,650 segment-level stance annotations across 47 societal issues. We also propose \textsc{JoA-ICL}, a \textbf{Jo}urnalism-guided \textbf{A}gentic \textbf{I}n-\textbf{C}ontext \textbf{L}earning framework that employs a language model agent to predict the stances of key structural segments (e.g., leads, quotes), which are then aggregated to infer the overall article stance. Experiments show that \textsc{JoA-ICL} outperforms existing stance detection methods, highlighting the benefits of segment-level agency in capturing the overall position of long-form news articles. Two case studies further demonstrate its broader utility in promoting viewpoint diversity in news recommendations and uncovering patterns of media bias.
• 摘要：随着在线新闻消费的增长，个性化的推荐系统已成为数字新闻业不可或缺的一部分。但是，这些系统可能会因未能纳入各种观点而冒着加强过滤器泡沫和政治两极分化的风险。立场检测 - 确定文本在目标上的立场 - 可以通过启用观点感知的建议和数据驱动的媒体偏见分析来帮助减轻这种情况。然而，现有的立场检测研究在很大程度上仍然限于简短的文本和高资源语言。为了解决这些差距，我们介绍了\ textsc {k-news-stance}，这是第一个用于文章级别立场检测的韩国数据集，其中包含2,000篇新闻文章，其中包含文章级别和19,650个跨47个社会问题的细分级别的立场注释。 We also propose \textsc{JoA-ICL}, a \textbf{Jo}urnalism-guided \textbf{A}gentic \textbf{I}n-\textbf{C}ontext \textbf{L}earning framework that employs a language model agent to predict the stances of key structural segments (e.g., leads, quotes), which然后汇总以推断整个文章立场。实验表明，\ textsc {joa-icl}胜过现有的立场检测方法，强调了细分级代理商在捕获长形新闻文章的整体位置方面的好处。两项案例研究进一步证明了其在新闻建议和揭示媒体偏见模式中的观点多样性方面更广泛的效用。

Title: LLM-Augmented Symptom Analysis for Cardiovascular Disease Risk Prediction: A Clinical NLP

• Authors: Haowei Yang, Ziyu Shen, Junli Shao, Luyao Men, Xinyue Han, Jing Dong
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.11052
• Pdf URL: https://arxiv.org/pdf/2507.11052
• Copy Paste: [[2507.11052]] LLM-Augmented Symptom Analysis for Cardiovascular Disease Risk Prediction: A Clinical NLP(https://arxiv.org/abs/2507.11052)
• Keywords: language model, llm, hallucination, prompt
• Abstract: Timely identification and accurate risk stratification of cardiovascular disease (CVD) remain essential for reducing global mortality. While existing prediction models primarily leverage structured data, unstructured clinical notes contain valuable early indicators. This study introduces a novel LLM-augmented clinical NLP pipeline that employs domain-adapted large language models for symptom extraction, contextual reasoning, and correlation from free-text reports. Our approach integrates cardiovascular-specific fine-tuning, prompt-based inference, and entity-aware reasoning. Evaluations on MIMIC-III and CARDIO-NLP datasets demonstrate improved performance in precision, recall, F1-score, and AUROC, with high clinical relevance (kappa = 0.82) assessed by cardiologists. Challenges such as contextual hallucination, which occurs when plausible information contracts with provided source, and temporal ambiguity, which is related with models struggling with chronological ordering of events are addressed using prompt engineering and hybrid rule-based verification. This work underscores the potential of LLMs in clinical decision support systems (CDSS), advancing early warning systems and enhancing the translation of patient narratives into actionable risk assessments.
• 摘要：心血管疾病（CVD）的及时识别和准确的风险分层对于降低全球死亡率仍然至关重要。尽管现有的预测模型主要利用结构化数据，但非结构化的临床笔记包含有价值的早期指标。这项研究介绍了一条新型的LLM临床NLP管道，该管道采用了域适应的大型语言模型来提取症状提取，上下文推理和自由文本报告中的相关性。我们的方法整合了心血管特异性的微调，基于及时的推理和实体感知的推理。对MIMIC-III和Cardio-NLP数据集的评估表明，心脏病专家评估的具有很高的临床相关性（KAPPA = 0.82）的精度，回忆，F1得分和AUROC的性能提高了。诸如上下文幻觉之类的挑战是在使用及时的工程和基于混合规则的验证验证来解决事件时遇到的模型时发生的合理信息和时间歧义时发生的。这项工作强调了LLM在临床决策支持系统（CDSS）中的潜力，推进预警系统，并将患者叙事转化为可操作的风险评估。

Title: Beyond Traditional Algorithms: Leveraging LLMs for Accurate Cross-Border Entity Identification

• Authors: Andres Azqueta-Gavaldón, Joaquin Ramos Cosgrove
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.11086
• Pdf URL: https://arxiv.org/pdf/2507.11086
• Copy Paste: [[2507.11086]] Beyond Traditional Algorithms: Leveraging LLMs for Accurate Cross-Border Entity Identification(https://arxiv.org/abs/2507.11086)
• Keywords: language model, llm
• Abstract: The growing prevalence of cross-border financial activities in global markets has underscored the necessity of accurately identifying and classifying foreign entities. This practice is essential within the Spanish financial system for ensuring robust risk management, regulatory adherence, and the prevention of financial misconduct. This process involves a labor-intensive entity-matching task, where entities need to be validated against available reference sources. Challenges arise from linguistic variations, special characters, outdated names, and changes in legal forms, complicating traditional matching algorithms like Jaccard, cosine, and Levenshtein distances. These methods struggle with contextual nuances and semantic relationships, leading to mismatches. To address these limitations, we explore Large Language Models (LLMs) as a flexible alternative. LLMs leverage extensive training to interpret context, handle abbreviations, and adapt to legal transitions. We evaluate traditional methods, Hugging Face-based LLMs, and interface-based LLMs (e.g., Microsoft Copilot, Alibaba's Qwen 2.5) using a dataset of 65 Portuguese company cases. Results show traditional methods achieve accuracies over 92% but suffer high false positive rates (20-40%). Interface-based LLMs outperform, achieving accuracies above 93%, F1 scores exceeding 96%, and lower false positives (40-80%).
• 摘要：在全球市场中，跨境金融活动的越来越流行凸显了准确识别和分类外国实体的必要性。在西班牙金融体系中，这种做法对于确保强大的风险管理，监管依从性和预防金融不当行为至关重要。此过程涉及一项劳动密集型实体匹配任务，在该任务中，需要根据可用参考来源进行验证实体。挑战是由语言变化，特殊角色，过时的名称以及法律形式的变化引起的，使传统匹配算法（如Jaccard，Cosine和Levenshtein距离）变得复杂。这些方法在上下文的细微差别和语义关系上挣扎，导致不匹配。为了解决这些限制，我们将大型语言模型（LLMS）作为灵活的替代方案。 LLMS利用广泛的培训来解释背景，处理缩写并适应法律过渡。我们使用65个葡萄牙公司案例的数据集评估了传统方法，基于面部的LLM和基于接口的LLM（例如Microsoft Copilot，Alibaba的QWEN 2.5）。结果表明，传统方法的准确性超过92％，但遭受高误报率（20-40％）。基于界面的LLM跑赢大于93％，F1分数超过96％，较低的假阳性（40-80％）的精度均优于大胜（40-80％）。

Title: The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMs

• Authors: Zichen Wen, Jiashu Qu, Dongrui Liu, Zhiyuan Liu, Ruixi Wu, Yicun Yang, Xiangqi Jin, Haoyun Xu, Xuyang Liu, Weijia Li, Chaochao Lu, Jing Shao, Conghui He, Linfeng Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.11097
• Pdf URL: https://arxiv.org/pdf/2507.11097
• Copy Paste: [[2507.11097]] The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMs(https://arxiv.org/abs/2507.11097)
• Keywords: language model, llm, prompt
• Abstract: Diffusion-based large language models (dLLMs) have recently emerged as a powerful alternative to autoregressive LLMs, offering faster inference and greater interactivity via parallel decoding and bidirectional modeling. However, despite strong performance in code generation and text infilling, we identify a fundamental safety concern: existing alignment mechanisms fail to safeguard dLLMs against context-aware, masked-input adversarial prompts, exposing novel vulnerabilities. To this end, we present DIJA, the first systematic study and jailbreak attack framework that exploits unique safety weaknesses of dLLMs. Specifically, our proposed DIJA constructs adversarial interleaved mask-text prompts that exploit the text generation mechanisms of dLLMs, i.e., bidirectional modeling and parallel decoding. Bidirectional modeling drives the model to produce contextually consistent outputs for masked spans, even when harmful, while parallel decoding limits model dynamic filtering and rejection sampling of unsafe content. This causes standard alignment mechanisms to fail, enabling harmful completions in alignment-tuned dLLMs, even when harmful behaviors or unsafe instructions are directly exposed in the prompt. Through comprehensive experiments, we demonstrate that DIJA significantly outperforms existing jailbreak methods, exposing a previously overlooked threat surface in dLLM architectures. Notably, our method achieves up to 100% keyword-based ASR on Dream-Instruct, surpassing the strongest prior baseline, ReNeLLM, by up to 78.5% in evaluator-based ASR on JailbreakBench and by 37.7 points in StrongREJECT score, while requiring no rewriting or hiding of harmful content in the jailbreak prompt. Our findings underscore the urgent need for rethinking safety alignment in this emerging class of language models. Code is available at this https URL.
• 摘要：基于扩散的大语言模型（DLLM）最近成为自回归LLM的强大替代方法，通过并行解码和双向建模提供更快的推理和更大的交互性。但是，尽管在代码生成和文本填充方面的性能很强，但我们确定了一个基本的安全问题：现有的一致性机制无法保护DLLMS免受上下文感知的，蒙版输入的对抗提示，从而暴露了新的脆弱性。为此，我们提出了Dija，这是第一个系统的研究和越狱攻击框架，它利用了DLLM的独特安全弱点。具体而言，我们提出的DIJA构建了对抗性的掩模文本，提示利用DLLM的文本生成机理，即双向建模和并行解码。双向建模驱动该模型，即使在有害的同时，也可以为掩盖跨度产生上下文一致的输出，同时模型的动态滤波和不安全内容的拒绝采样。这会导致标准对齐机制失败，即使在提示中直接暴露有害行为或不安全的说明，也可以在对齐调整的DLLM中实现有害的完成。通过全面的实验，我们证明了DIJA极大地胜过现有的越狱方法，并在DLLM架构中展示了先前被忽视的威胁表面。值得注意的是，我们的方法在Dream-Instruct上最多可获得100％的基于关键字的ASR，超过了先前的基线Renellm，在基于评估的ASR的ASR上，最高可达78.5％，在越狱式越狱中获得37.7分，同时不需要重新定位或在越狱及时的有害内容及时提高有害内容。我们的发现强调了在这种新兴语言模型中重新思考安全对准的迫切需要。代码可在此HTTPS URL上找到。

Title: Multi-Trigger Poisoning Amplifies Backdoor Vulnerabilities in LLMs

• Authors: Sanhanat Sivapiromrat, Caiqi Zhang, Marco Basaldella, Nigel Collier
• Subjects: cs.CL, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.11112
• Pdf URL: https://arxiv.org/pdf/2507.11112
• Copy Paste: [[2507.11112]] Multi-Trigger Poisoning Amplifies Backdoor Vulnerabilities in LLMs(https://arxiv.org/abs/2507.11112)
• Keywords: language model, llm
• Abstract: Recent studies have shown that Large Language Models (LLMs) are vulnerable to data poisoning attacks, where malicious training examples embed hidden behaviours triggered by specific input patterns. However, most existing works assume a phrase and focus on the attack's effectiveness, offering limited understanding of trigger mechanisms and how multiple triggers interact within the model. In this paper, we present a framework for studying poisoning in LLMs. We show that multiple distinct backdoor triggers can coexist within a single model without interfering with each other, enabling adversaries to embed several triggers concurrently. Using multiple triggers with high embedding similarity, we demonstrate that poisoned triggers can achieve robust activation even when tokens are substituted or separated by long token spans. Our findings expose a broader and more persistent vulnerability surface in LLMs. To mitigate this threat, we propose a post hoc recovery method that selectively retrains specific model components based on a layer-wise weight difference analysis. Our method effectively removes the trigger behaviour with minimal parameter updates, presenting a practical and efficient defence against multi-trigger poisoning.
• 摘要：最近的研究表明，大型语言模型（LLMS）容易受到数据中毒攻击的影响，其中恶意训练示例嵌入了由特定输入模式触发的隐藏行为。但是，大多数现有作品都采用短语，并专注于攻击的有效性，对触发机制以及多个触发器在模型中的相互作用如何有限。在本文中，我们提出了研究LLM中毒的框架。我们表明，多个不同的后门触发器可以在单个模型中共存而无需彼此干扰，从而使对手能够同时嵌入多个触发器。使用多个具有高嵌入相似性的触发器，我们证明了中毒触发因素即使在代币被长令牌跨度取代或分离时，也可以实现强大的激活。我们的发现暴露了LLMS中更广泛，更持久的脆弱性表面。为了减轻这种威胁，我们提出了一种事后恢复方法，该方法根据层的重量差分析有选择地重新培训特定的模型组件。我们的方法通过最小的参数更新有效地消除了触发行为，从而对多触发中毒提出了实用有效的防御。

Title: MSA at ImageCLEF 2025 Multimodal Reasoning: Multilingual Multimodal Reasoning With Ensemble Vision Language Models

• Authors: Seif Ahmed, Mohamed T. Younes, Abdelrahman Moustafa, Abdelrahman Allam, Hamza Moustafa
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.11114
• Pdf URL: https://arxiv.org/pdf/2507.11114
• Copy Paste: [[2507.11114]] MSA at ImageCLEF 2025 Multimodal Reasoning: Multilingual Multimodal Reasoning With Ensemble Vision Language Models(https://arxiv.org/abs/2507.11114)
• Keywords: language model, prompt
• Abstract: We present a robust ensemble-based system for multilingual multimodal reasoning, designed for the ImageCLEF 2025 EXAMS V challenge. Our approach integrates Gemini 2.5 Flash for visual description, Gemini 1.5 Pro for caption refinement and consistency checks, and Gemini 2.5 Pro as a reasoner which handles final answer selection, all coordinated through carefully engineered few-shot and zero-shot prompts. We conducted an extensive ablation study, training several large language models (Gemini 2.5 Flash, Phi 4, Gemma 3, Mistral) on an English dataset and its multilingual augmented version. Additionally, we evaluated Gemini 2.5 Flash in a zero-shot setting for comparison and found it to substantially outperform the trained models. Prompt design also proved critical: enforcing concise, language-normalized formats and prohibiting explanatory text boosted model accuracy on the English validation set from 55.9% to 61.7%. On the official leaderboard, our system (Team MSA) achieved first place overall in the multilingual track with 81.4% accuracy, and led 11 out of 13 individual language tracks, with top results such as 95.07% for Croatian and 92.12% for Italian. These findings highlight that lightweight OCR-VLM ensembles, when paired with precise prompt strategies and cross-lingual augmentation, can outperform heavier end-to-end models in high-stakes, multilingual educational settings.
• 摘要：我们提出了一个基于强大的集合系统，用于多语言多模式推理，该系统为Imageclef 2025考试V挑战而设计。我们的方法集成了Gemini 2.5闪光灯进行视觉描述，Gemini 1.5 Pro用于标题改进和一致性检查，以及Gemini 2.5 Pro作为理解最终答案选择的推理器，所有这些都通过精心设计的几次射击和零发提示进行了协调。我们进行了一项广泛的消融研究，培训了几种大型语言模型（Gemini 2.5 Flash，Phi 4，Gemma 3，Mistral）及其多语言增强版本。此外，我们在零拍设置中评估了Gemini 2.5 Flash以进行比较，并发现它显着优于训练有素的模型。及时设计也被证明至关重要：在英语验证设置的第55.9％至61.7％的情况下，执行简洁，语言归一化的格式和禁止解释性文本的模型精度。在官方排行榜上，我们的系统（MSA团队）以81.4％的精度在多语言轨道中获得了第一名，并在13个单独的语言轨道中领先11个，克罗地亚人的最高结果为95.07％，意大利人为92.12％。这些发现凸显了轻巧的OCR-VLM合奏，与精确的及时策略和跨语言增强配对时，可以在高风险的多种语言教育环境中胜过更重的端到端模型。

Title: What Should LLMs Forget? Quantifying Personal Data in LLMs for Right-to-Be-Forgotten Requests

• Authors: Dimitri Staufer
• Subjects: cs.CL, cs.CY, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.11128
• Pdf URL: https://arxiv.org/pdf/2507.11128
• Copy Paste: [[2507.11128]] What Should LLMs Forget? Quantifying Personal Data in LLMs for Right-to-Be-Forgotten Requests(https://arxiv.org/abs/2507.11128)
• Keywords: language model, llm, prompt
• Abstract: Large Language Models (LLMs) can memorize and reveal personal information, raising concerns regarding compliance with the EU's GDPR, particularly the Right to Be Forgotten (RTBF). Existing machine unlearning methods assume the data to forget is already known but do not address how to identify which individual-fact associations are stored in the model. Privacy auditing techniques typically operate at the population level or target a small set of identifiers, limiting applicability to individual-level data inquiries. We introduce WikiMem, a dataset of over 5,000 natural language canaries covering 243 human-related properties from Wikidata, and a model-agnostic metric to quantify human-fact associations in LLMs. Our approach ranks ground-truth values against counterfactuals using calibrated negative log-likelihood across paraphrased prompts. We evaluate 200 individuals across 15 LLMs (410M-70B parameters), showing that memorization correlates with subject web presence and model scale. We provide a foundation for identifying memorized personal data in LLMs at the individual level, enabling the dynamic construction of forget sets for machine unlearning and RTBF requests.
• 摘要：大型语言模型（LLMS）可以记住并揭示个人信息，从而引起人们对遵守欧盟GDPR的担忧，尤其是被遗忘的权利（RTBF）。现有的计算机学习方法假设已经知道要忘记的数据，但没有解决如何识别模型中存储哪些个人事实关联。隐私审核技术通常在人群级别运行或针对一小部分标识符，从而将适用性限制在个人级别的数据查询中。我们介绍了Wikimem，该数据集是5,000多种自然语言金丝雀的数据集，其中涵盖了来自Wikidata的243个与人类相关特性的数据集，以及一个模型不合时宜的度量标准，可量化LLMS中的人类事实关联。我们的方法使用跨释义的提示中使用校准的负对数似然性对反事实的基本真相值对反事实进行了排名。我们评估了15个LLM（410M-70B参数）的200个个体，表明记忆与受试者Web的存在和模型量表相关。我们为在单个级别的LLMS中识别记忆的个人数据提供了基础，从而为机器学习和RTBF请求提供了动态构造。

Title: Temperature and Persona Shape LLM Agent Consensus With Minimal Accuracy Gains in Qualitative Coding

• Authors: Conrad Borchers, Bahar Shahrokhian, Francesco Balzan, Elham Tajik, Sreecharan Sankaranarayanan, Sebastian Simon
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.11198
• Pdf URL: https://arxiv.org/pdf/2507.11198
• Copy Paste: [[2507.11198]] Temperature and Persona Shape LLM Agent Consensus With Minimal Accuracy Gains in Qualitative Coding(https://arxiv.org/abs/2507.11198)
• Keywords: language model, llm, agent
• Abstract: Large Language Models (LLMs) enable new possibilities for qualitative research at scale, including coding and data annotation. While multi-agent systems (MAS) can emulate human coding workflows, their benefits over single-agent coding remain poorly understood. We conducted an experimental study of how agent persona and temperature shape consensus-building and coding accuracy of dialog segments based on a codebook with 8 codes. Our open-source MAS mirrors deductive human coding through structured agent discussion and consensus arbitration. Using six open-source LLMs (with 3 to 32 billion parameters) and 18 experimental configurations, we analyze over 77,000 coding decisions against a gold-standard dataset of human-annotated transcripts from online math tutoring sessions. Temperature significantly impacted whether and when consensus was reached across all six LLMs. MAS with multiple personas (including neutral, assertive, or empathetic), significantly delayed consensus in four out of six LLMs compared to uniform personas. In three of those LLMs, higher temperatures significantly diminished the effects of multiple personas on consensus. However, neither temperature nor persona pairing lead to robust improvements in coding accuracy. Single agents matched or outperformed MAS consensus in most conditions. Only one model (OpenHermesV2:7B) and code category showed above-chance gains from MAS deliberation when temperature was 0.5 or lower and especially when the agents included at least one assertive persona. Qualitative analysis of MAS collaboration for these configurations suggests that MAS may nonetheless aid in narrowing ambiguous code applications that could improve codebooks and human-AI coding. We contribute new insight into the limits of LLM-based qualitative methods, challenging the notion that diverse MAS personas lead to better outcomes. We open-source our MAS and experimentation code.
• 摘要：大型语言模型（LLMS）为定性研究提供了新的可能性，包括编码和数据注释。虽然多代理系统（MAS）可以效仿人体编码工作流，但它们对单个代理编码的好处仍然知之甚少。我们对基于包含8个代码的代码手册的对话段共识和编码精度进行了代理角色和温度形状建立和编码精度的实验研究。我们的开源MAS反映了通过结构化的代理讨论和共识仲裁演绎的人为编码。使用六个开源LLM（具有3至320亿个参数）和18种实验配置，我们分析了77,000多个编码决策，以通过在线数学辅导会话中的人类宣布的人类宣布的成绩单进行了77,000多个编码决策。在所有六个LLM中是否达成共识，温度显着影响是否达成共识。与统一的人物相比，具有多个角色（包括中性，自信或同理心）的MAS在六个LLM中的四分之一显着延迟了共识。在其中三个LLM中，较高的温度显着降低了多个角色对共识的影响。但是，温度和角色配对都不会导致编码准确性方面的强大改善。在大多数情况下，单个代理匹配或胜过MAS的共识。在温度为0.5或较低时，尤其是当代理人至少包括一个自信的角色时，只有一种模型（OpenHermesv2：7b）和代码类别显示了MAS审议时从MAS审议中获得的收益。对这些配置的MAS协作的定性分析表明，MAS仍可能有助于缩小模棱两可的代码应用，以改善代码书和人类AID编码。我们对基于LLM的定性方法的局限性有了新的见解，这挑战了多样化的MAS角色导致更好结果的观念。我们开源的MAS和实验代码。

Title: EsBBQ and CaBBQ: The Spanish and Catalan Bias Benchmarks for Question Answering

• Authors: Valle Ruiz-Fernández, Mario Mina, Júlia Falcão, Luis Vasquez-Reina, Anna Sallés, Aitor Gonzalez-Agirre, Olatz Perez-de-Viñaspre
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.11216
• Pdf URL: https://arxiv.org/pdf/2507.11216
• Copy Paste: [[2507.11216]] EsBBQ and CaBBQ: The Spanish and Catalan Bias Benchmarks for Question Answering(https://arxiv.org/abs/2507.11216)
• Keywords: language model, llm
• Abstract: Previous literature has largely shown that Large Language Models (LLMs) perpetuate social biases learnt from their pre-training data. Given the notable lack of resources for social bias evaluation in languages other than English, and for social contexts outside of the United States, this paper introduces the Spanish and the Catalan Bias Benchmarks for Question Answering (EsBBQ and CaBBQ). Based on the original BBQ, these two parallel datasets are designed to assess social bias across 10 categories using a multiple-choice QA setting, now adapted to the Spanish and Catalan languages and to the social context of Spain. We report evaluation results on different LLMs, factoring in model family, size and variant. Our results show that models tend to fail to choose the correct answer in ambiguous scenarios, and that high QA accuracy often correlates with greater reliance on social biases.
• 摘要：先前的文献很大程度上表明，大型语言模型（LLMS）使社会偏见从其前培训数据中学到。鉴于明显缺乏在英语以外的其他语言以及美国以外的社会环境中的社会偏见评估资源，因此本文介绍了西班牙语和加泰罗尼亚的偏见基准，以提出问题回答（ESBBQ和CABBQ）。基于原始烧烤，这两个并行数据集旨在使用多项选择的质量检查设置评估10个类别的社会偏见，现在适合西班牙语和加泰罗尼亚语言以及西班牙的社交背景。我们报告了对不同LLM的评估结果，包括模型家族，大小和变体中的分解。我们的结果表明，模型倾向于在模棱两可的场景中选择正确的答案，并且高质量质量质量准确性通常与更依赖社会偏见的依赖相关。

Title: An Agentic Flow for Finite State Machine Extraction using Prompt Chaining

• Authors: Fares Wael, Youssef Maklad, Ali Hamdi, Wael Elsersy
• Subjects: cs.CL, cs.AI, cs.NI
• Abstract URL: https://arxiv.org/abs/2507.11222
• Pdf URL: https://arxiv.org/pdf/2507.11222
• Copy Paste: [[2507.11222]] An Agentic Flow for Finite State Machine Extraction using Prompt Chaining(https://arxiv.org/abs/2507.11222)
• Keywords: language model, llm, prompt, chain-of-thought, agent
• Abstract: Finite-State Machines (FSMs) are critical for modeling the operational logic of network protocols, enabling verification, analysis, and vulnerability discovery. However, existing FSM extraction techniques face limitations such as scalability, incomplete coverage, and ambiguity in natural language specifications. In this paper, we propose FlowFSM, a novel agentic framework that leverages Large Language Models (LLMs) combined with prompt chaining and chain-of-thought reasoning to extract accurate FSMs from raw RFC documents. FlowFSM systematically processes protocol specifications, identifies state transitions, and constructs structured rule-books by chaining agent outputs. Experimental evaluation across FTP and RTSP protocols demonstrates that FlowFSM achieves high extraction precision while minimizing hallucinated transitions, showing promising results. Our findings highlight the potential of agent-based LLM systems in the advancement of protocol analysis and FSM inference for cybersecurity and reverse engineering applications.
• 摘要：有限状态机器（FSM）对于建模网络协议的操作逻辑，启用验证，分析和脆弱性发现至关重要。但是，现有的FSM提取技术面临限制，例如可扩展性，不完整的覆盖范围和自然语言规范的歧义。在本文中，我们提出了FlowFSM，这是一种新型的代理框架，利用大型语言模型（LLMS）与迅速的链接和经过经过经过经过经过经过经过做法的推理相结合，以从RAW RFC文档中提取准确的FSM。 FlowFSM系统地处理协议规范，标识状态转换，并通过链接代理输出来构造结构化规则手册。跨FTP和RTSP方案进行的实验评估表明，FlowFSM可实现高提取精度，同时最大程度地减少幻觉过渡，从而显示出令人鼓舞的结果。我们的发现突出了基于代理的LLM系统在协议分析和FSM推断网络安全和逆向工程应用中的潜力。

Title: Sparse Autoencoders Can Capture Language-Specific Concepts Across Diverse Languages

• Authors: Lyzander Marciano Andrylie, Inaya Rahmanisa, Mahardika Krisna Ihsani, Alfan Farizki Wicaksono, Haryo Akbarianto Wibowo, Alham Fikri Aji
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.11230
• Pdf URL: https://arxiv.org/pdf/2507.11230
• Copy Paste: [[2507.11230]] Sparse Autoencoders Can Capture Language-Specific Concepts Across Diverse Languages(https://arxiv.org/abs/2507.11230)
• Keywords: language model, llm
• Abstract: Understanding the multilingual mechanisms of large language models (LLMs) provides insight into how they process different languages, yet this remains challenging. Existing studies often focus on individual neurons, but their polysemantic nature makes it difficult to isolate language-specific units from cross-lingual representations. To address this, we explore sparse autoencoders (SAEs) for their ability to learn monosemantic features that represent concrete and abstract concepts across languages in LLMs. While some of these features are language-independent, the presence of language-specific features remains underexplored. In this work, we introduce SAE-LAPE, a method based on feature activation probability, to identify language-specific features within the feed-forward network. We find that many such features predominantly appear in the middle to final layers of the model and are interpretable. These features influence the model's multilingual performance and language output and can be used for language identification with performance comparable to fastText along with more interpretability. Our code is available at this https URL .
• 摘要：了解大语模型（LLM）的多语言机制提供了有关它们如何处理不同语言的见解，但这仍然具有挑战性。现有的研究通常集中在单个神经元上，但是它们的多性性性质使得很难将特定于语言的单元与跨语性表示。为了解决这个问题，我们探索了稀疏的自动编码器（SAE），以了解其具有跨LLMS语言的具体和抽象概念的单语义特征的能力。尽管其中一些功能是无关的，但语言特定功能的存在仍然没有被忽视。在这项工作中，我们介绍了基于特征激活概率的方法SAE-lape，以识别馈送前进网络中特定于语言的特征。我们发现许多这样的功能主要出现在模型的中间至最后一层，并且是可解释的。这些功能会影响该模型的多语言性能和语言输出，可用于语言识别，具有与FastText相当的性能以及更多的可解释性。我们的代码可在此HTTPS URL上找到。

Title: KV-Latent: Dimensional-level KV Cache Reduction with Frequency-aware Rotary Positional Embedding

• Authors: Luohe Shi, Zuchao Li, Lefei Zhang, Guoming Liu, Baoyuan Qi, Hai Zhao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.11273
• Pdf URL: https://arxiv.org/pdf/2507.11273
• Copy Paste: [[2507.11273]] KV-Latent: Dimensional-level KV Cache Reduction with Frequency-aware Rotary Positional Embedding(https://arxiv.org/abs/2507.11273)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) based on Transformer Decoders have become the preferred choice for conversational generative AI. Despite the overall superiority of the Decoder architecture, the gradually increasing Key-Value (KV) cache during inference has emerged as a primary efficiency bottleneck, both in aspects of memory consumption and data transfer bandwidth limitations. To address these challenges, we propose a paradigm called KV-Latent. By down-sampling the Key-Value vector dimensions into a latent space, we can significantly reduce the KV Cache footprint and improve inference speed, only with a small amount of extra training, less than 1\% of pre-training takes. Besides, we enhanced the stability of Rotary Positional Embedding applied on lower-dimensional vectors by modifying its frequency sampling mechanism, avoiding noise introduced by higher frequencies while retaining position attenuation. Our experiments, including both models with Grouped Query Attention and those without, have yielded satisfactory results. Finally, we conducted comparative experiments to study the impact of separately reducing Key and Value components on model's performance. Our approach allows for the construction of more efficient language model systems, and opens the new possibility on KV Cache saving and efficient LLMs. Our code is available at this https URL.
• 摘要：基于变压器解码器的大型语言模型（LLM）已成为对话生成AI的首选选择。尽管解码器架构的总体优势，但在推理过程中逐渐增加的键值（KV）缓存还是在内存消耗和数据传输带宽限制方面都作为主要效率瓶颈。为了应对这些挑战，我们提出了一个称为KV-Latent的范式。通过将键值矢量尺寸减小为潜在空间，我们可以显着降低KV高速缓存足迹并提高推理速度，仅在少量的额外训练中，少于1 \％的预训练所需。此外，我们通过修改其频率采样机制，增强了施加在较低维矢量上的旋转位置嵌入的稳定性，从而避免了较高频率引入的噪声，同时避免了位置衰减。我们的实验，包括两个具有分组查询注意力的模型，而无需进行查询的模型，都产生了令人满意的结果。最后，我们进行了比较实验，以研究分别降低密钥和价值成分对模型性能的影响。我们的方法允许构建更有效的语言模型系统，并在KV缓存节省和有效的LLM上打开了新的可能性。我们的代码可在此HTTPS URL上找到。

Title: FMC: Formalization of Natural Language Mathematical Competition Problems

• Authors: Jiaxuan Xie, Chengwu Liu, Ye Yuan, Siqi Li, Zhiping Xiao, Ming Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.11275
• Pdf URL: https://arxiv.org/pdf/2507.11275
• Copy Paste: [[2507.11275]] FMC: Formalization of Natural Language Mathematical Competition Problems(https://arxiv.org/abs/2507.11275)
• Keywords: language model, llm
• Abstract: Efficient and accurate autoformalization methods, which leverage large-scale datasets of extensive natural language mathematical problems to construct formal language datasets, are key to advancing formal mathematical reasoning. In this paper, we propose an autoformalization pipeline based on large language models with error feedback, achieving a fully automatic and training-free formalization approach. Using this pipeline, we curate an Olympiad-level dataset aligning natural language problems with Lean formalizations. The dataset comprises $3,922$ mathematical problems in natural language and $9,787$ in Lean, of which $64.46\%$ were assessed as at least above-average quality, making it suitable as a benchmark for automated theorem provers. Additionally, we investigate the formalization and reasoning capabilities of various LLMs and empirically demonstrate that few-shot learning, error feedback, and increasing sampling numbers enhance the autoformalization process. Experiments of three automated theorem provers on the \dataset\ dataset also highlight its challenging nature and its value as a benchmark for formal reasoning tasks.
• 摘要：有效，准确的自动化方法利用广泛的自然语言数学问题的大规模数据集来构建正式的语言数据集，这是推进正式数学推理的关键。在本文中，我们提出了一种基于具有错误反馈的大语言模型的自动化管道，实现了一种全自动且无训练的形式化方法。使用此管道，我们策划了一个奥林匹克级的数据集，将自然语言问题与精益形式化结合。该数据集包含$ 3,922 $的自然语言数学问题和$ 9,787 $ in Lean，其中$ 64.46 \％$被评估为至少高于平均水平的质量，使其适合自动定理的基准。此外，我们研究了各种LLM的形式化和推理能力，并从经验上证明，很少有学习，错误反馈和增加采样数量可以增强自动化过程。 \ dataset \ dataset上三个自动定理掠夺的实验还强调了其挑战性的性质及其价值作为正式推理任务的基准。

Title: Fine-Grained Chinese Hate Speech Understanding: Span-Level Resources, Coded Term Lexicon, and Enhanced Detection Frameworks

• Authors: Zewen Bai, Liang Yang, Shengdi Yin, Yuanyuan Sun, Hongfei Lin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.11292
• Pdf URL: https://arxiv.org/pdf/2507.11292
• Copy Paste: [[2507.11292]] Fine-Grained Chinese Hate Speech Understanding: Span-Level Resources, Coded Term Lexicon, and Enhanced Detection Frameworks(https://arxiv.org/abs/2507.11292)
• Keywords: llm
• Abstract: The proliferation of hate speech has inflicted significant societal harm, with its intensity and directionality closely tied to specific targets and arguments. In recent years, numerous machine learning-based methods have been developed to detect hateful comments on online platforms automatically. However, research on Chinese hate speech detection lags behind, and interpretability studies face two major challenges: first, the scarcity of span-level fine-grained annotated datasets limits models' deep semantic understanding of hate speech; second, insufficient research on identifying and interpreting coded hate speech restricts model explainability in complex real-world scenarios. To address these, we make the following contributions: (1) We introduce the Span-level Target-Aware Toxicity Extraction dataset (STATE ToxiCN), the first span-level Chinese hate speech dataset, and evaluate the hate semantic understanding of existing models using it. (2) We conduct the first comprehensive study on Chinese coded hate terms, LLMs' ability to interpret hate semantics. (3) We propose a method to integrate an annotated lexicon into models, significantly enhancing hate speech detection performance. Our work provides valuable resources and insights to advance the interpretability of Chinese hate speech detection research.
• 摘要：仇恨言论的扩散造成了严重的社会伤害，其强度和方向性与特定目标和论点紧密相关。近年来，已经开发了许多基于机器学习的方法来自动检测在线平台上的仇恨评论。然而，对中国仇恨言论检测的研究滞后，解释性研究面临两个主要挑战：首先，跨度级的细颗粒注释的数据集的稀缺限制了模型模型对仇恨言论的深刻语义理解；其次，在复杂的现实世界情景中识别和解释编码的仇恨言论限制的研究不足。为了解决这些问题，我们做出以下贡献：（1）我们介绍了跨越目标感知的毒性提取数据集（州有毒），这是第一个跨度级中国仇恨语音数据集，并评估使用它对现有模型的仇恨语义理解。 （2）我们对中国编码的仇恨术语进行了首次全面研究，即LLMS解释仇恨语义的能力。 （3）我们提出了一种将带注释的词典集成到模型中的方法，从而显着提高了仇恨言语检测性能。我们的工作提供了宝贵的资源和见解，以提高中国仇恨言论检测研究的解释性。

Title: Dr.Copilot: A Multi-Agent Prompt Optimized Assistant for Improving Patient-Doctor Communication in Romanian

• Authors: Andrei Niculae, Adrian Cosma, Cosmin Dumitrache, Emilian Rǎdoi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.11299
• Pdf URL: https://arxiv.org/pdf/2507.11299
• Copy Paste: [[2507.11299]] Dr.Copilot: A Multi-Agent Prompt Optimized Assistant for Improving Patient-Doctor Communication in Romanian(https://arxiv.org/abs/2507.11299)
• Keywords: language model, llm, prompt, agent
• Abstract: Text-based telemedicine has become increasingly common, yet the quality of medical advice in doctor-patient interactions is often judged more on how advice is communicated rather than its clinical accuracy. To address this, we introduce this http URL , a multi-agent large language model (LLM) system that supports Romanian-speaking doctors by evaluating and enhancing the presentation quality of their written responses. Rather than assessing medical correctness, this http URL provides feedback along 17 interpretable axes. The system comprises of three LLM agents with prompts automatically optimized via DSPy. Designed with low-resource Romanian data and deployed using open-weight models, it delivers real-time specific feedback to doctors within a telemedicine platform. Empirical evaluations and live deployment with 41 doctors show measurable improvements in user reviews and response quality, marking one of the first real-world deployments of LLMs in Romanian medical settings.
• 摘要：基于文本的远程医疗已经变得越来越普遍，但是医生互动中医疗建议的质量通常可以更多地判断为如何传达建议，而不是其临床准确性。为了解决这个问题，我们介绍了这款HTTP URL，这是一种多门语言模型（LLM）系统，该系统通过评估和提高其书面响应的表现质量来支持讲罗马尼亚的医生。该HTTP URL并没有评估医疗正确性，而是沿17个可解释的轴提供反馈。该系统由三种LLM代理组成，并通过DSPY自动优化提示。它采用低资源的罗马尼亚数据并使用开放式模型部署，它为远程医疗平台内的医生提供了实时的特定反馈。 41位医生的经验评估和实时部署显示了用户评论和响应质量的可衡量改进，这标志着LLMS在罗马尼亚医疗环境中的最早现实世界部署之一。

Title: Internal Value Alignment in Large Language Models through Controlled Value Vector Activation

• Authors: Haoran Jin, Meng Li, Xiting Wang, Zhihao Xu, Minlie Huang, Yantao Jia, Defu Lian
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.11316
• Pdf URL: https://arxiv.org/pdf/2507.11316
• Copy Paste: [[2507.11316]] Internal Value Alignment in Large Language Models through Controlled Value Vector Activation(https://arxiv.org/abs/2507.11316)
• Keywords: language model, llm, prompt
• Abstract: Aligning Large Language Models (LLMs) with human values has attracted increasing attention since it provides clarity, transparency, and the ability to adapt to evolving scenarios. In this paper, we introduce a Controlled Value Vector Activation (ConVA) method that directly aligns the internal values of LLMs by interpreting how a value is encoded in their latent representations and modifies relevant activations to ensure consistent values in LLMs. To ensure an accurate and unbiased interpretation, we propose a context-controlled value vector identification method. To consistently control values without sacrificing model performance, we introduce a gated value vector activation method for effective and minimum degree of value control. Experiments show that our method achieves the highest control success rate across 10 basic values without hurting LLM performance and fluency, and ensures target values even with opposite and potentially malicious input prompts. Source code and data are available at~ this https URL.
• 摘要：将大型语言模型（LLM）与人类价值观保持一致，因为它提供了清晰，透明度和适应不断发展的情景的能力，因此引起了人们的关注。在本文中，我们引入了一个受控的值向量激活（CORVA）方法，该方法通过解释如何在其潜在表示中编码值并修改相关激活以确保LLMS中的值一致的值来直接与LLM的内部值保持一致。为了确保准确，公正的解释，我们提出了一种上下文控制的值矢量识别方法。为了始终如一地控制值而不牺牲模型性能，我们引入了一种封闭的值矢量激活方法，以有效和最低价值控制程度。实验表明，我们的方法在10个基本值中达到了最高的控制成功率，而不会损害LLM的性能和流利度，即使有相反的恶意输入提示，也可以确保目标值。源代码和数据可在〜this HTTPS URL上获得。

Title: Automated Novelty Evaluation of Academic Paper: A Collaborative Approach Integrating Human and Large Language Model Knowledge

• Authors: Wenqing Wu, Chengzhi Zhang, Yi Zhao
• Subjects: cs.CL, cs.AI, cs.DL, cs.HC
• Abstract URL: https://arxiv.org/abs/2507.11330
• Pdf URL: https://arxiv.org/pdf/2507.11330
• Copy Paste: [[2507.11330]] Automated Novelty Evaluation of Academic Paper: A Collaborative Approach Integrating Human and Large Language Model Knowledge(https://arxiv.org/abs/2507.11330)
• Keywords: language model, llm
• Abstract: Novelty is a crucial criterion in the peer review process for evaluating academic papers. Traditionally, it's judged by experts or measure by unique reference combinations. Both methods have limitations: experts have limited knowledge, and the effectiveness of the combination method is uncertain. Moreover, it's unclear if unique citations truly measure novelty. The large language model (LLM) possesses a wealth of knowledge, while human experts possess judgment abilities that the LLM does not possess. Therefore, our research integrates the knowledge and abilities of LLM and human experts to address the limitations of novelty assessment. The most common novelty in academic papers is the introduction of new methods. In this paper, we propose leveraging human knowledge and LLM to assist pretrained language models (PLMs, e.g. BERT etc.) in predicting the method novelty of papers. Specifically, we extract sentences related to the novelty of the academic paper from peer review reports and use LLM to summarize the methodology section of the academic paper, which are then used to fine-tune PLMs. In addition, we have designed a text-guided fusion module with novel Sparse-Attention to better integrate human and LLM knowledge. We compared the method we proposed with a large number of baselines. Extensive experiments demonstrate that our method achieves superior performance.
• 摘要：新颖性是评估学术论文的同行评审过程中的关键标准。传统上，它是由专家判断的，或者是通过独特的参考组合来衡量的。两种方法都有局限性：专家的知识有限，并且组合方法的有效性尚不确定。而且，目前尚不清楚独特的引用是否真正衡量新颖性。大型语言模型（LLM）具有丰富的知识，而人类专家具有LLM不具备的判断力。因此，我们的研究将LLM和人类专家的知识和能力整合在一起，以解决新颖性评估的局限性。学术论文中最常见的新颖性是引入新方法。在本文中，我们提出了利用人类知识和LLM来协助验证的语言模型（PLM，例如BERT等）预测论文的新颖性。具体而言，我们从同行评审报告中提取了与学术论文的新颖性有关的句子，并使用LLM来总结学术论文的方法论部分，然后将其用于微调PLM。此外，我们设计了一个带有新型稀疏注意的文本引导的融合模块，以更好地整合人类和LLM知识。我们将我们提出的方法与大量基线进行了比较。广泛的实验表明，我们的方法实现了卓越的性能。

Title: What is the Best Process Model Representation? A Comparative Analysis for Process Modeling with Large Language Models

• Authors: Alexis Brissard, Frédéric Cuppens, Amal Zouaq
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.11356
• Pdf URL: https://arxiv.org/pdf/2507.11356
• Copy Paste: [[2507.11356]] What is the Best Process Model Representation? A Comparative Analysis for Process Modeling with Large Language Models(https://arxiv.org/abs/2507.11356)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) are increasingly applied for Process Modeling (PMo) tasks such as Process Model Generation (PMG). To support these tasks, researchers have introduced a variety of Process Model Representations (PMRs) that serve as model abstractions or generation targets. However, these PMRs differ widely in structure, complexity, and usability, and have never been systematically compared. Moreover, recent PMG approaches rely on distinct evaluation strategies and generation techniques, making comparison difficult. This paper presents the first empirical study that evaluates multiple PMRs in the context of PMo with LLMs. We introduce the PMo Dataset, a new dataset containing 55 process descriptions paired with models in nine different PMRs. We evaluate PMRs along two dimensions: suitability for LLM-based PMo and performance on PMG. \textit{Mermaid} achieves the highest overall score across six PMo criteria, whereas \textit{BPMN text} delivers the best PMG results in terms of process element similarity.
• 摘要：大型语言模型（LLM）越来越多地用于过程建模（PMO）任务，例如过程模型生成（PMG）。为了支持这些任务，研究人员介绍了作为模型抽象或发电目标的各种过程模型表示（PMR）。但是，这些PMR在结构，复杂性和可用性方面差异很大，从未系统地比较。此外，最近的PMG方法依赖于不同的评估策略和发电技术，因此比较变得困难。本文介绍了第一项经验研究，该研究在使用LLMS的PMO的背景下评估了多个PMR。我们介绍了PMO数据集，PMO数据集是一个新数据集，其中包含55个流程描述与9种不同PMR中的型号配对。我们沿两个维度评估PMR：基于LLM的PMO的适用性和PMG的性能。 \ textit {mermaid}在六个PMO标准中取得了最高的总分，而\ textit {bpmn text}在过程元素相似性方面提供了最佳的PMG结果。

Title: DCR: Quantifying Data Contamination in LLMs Evaluation

• Authors: Cheng Xu, Nan Yan, Shuhao Guan, Changhong Jin, Yuke Mei, Yibing Guo, M-Tahar Kechadi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.11405
• Pdf URL: https://arxiv.org/pdf/2507.11405
• Copy Paste: [[2507.11405]] DCR: Quantifying Data Contamination in LLMs Evaluation(https://arxiv.org/abs/2507.11405)
• Keywords: language model, llm
• Abstract: The rapid advancement of large language models (LLMs) has heightened concerns about benchmark data contamination (BDC), where models inadvertently memorize evaluation data, inflating performance metrics and undermining genuine generalization assessment. This paper introduces the Data Contamination Risk (DCR) framework, a lightweight, interpretable pipeline designed to detect and quantify BDC across four granular levels: semantic, informational, data, and label. By synthesizing contamination scores via a fuzzy inference system, DCR produces a unified DCR Factor that adjusts raw accuracy to reflect contamination-aware performance. Validated on 9 LLMs (0.5B-72B) across sentiment analysis, fake news detection, and arithmetic reasoning tasks, the DCR framework reliably diagnoses contamination severity and with accuracy adjusted using the DCR Factor to within 4% average error across the three benchmarks compared to the uncontaminated baseline. Emphasizing computational efficiency and transparency, DCR provides a practical tool for integrating contamination assessment into routine evaluations, fostering fairer comparisons and enhancing the credibility of LLM benchmarking practices.
• 摘要：大型语言模型（LLMS）的快速发展对基准数据污染（BDC）的关注加剧了，其中模型无意中记住评估数据，膨胀性能指标并破坏了真正的概括评估。本文介绍了数据污染风险（DCR）框架，这是一种轻巧，可解释的管道，旨在检测和量化四个颗粒级的BDC：语义，信息，数据和标签。通过通过模糊的推理系统合成污染分数，DCR产生统一的DCR因子，可调节原始精度以反映污染感知性能。 DCR框架在情感分析，虚假新闻检测和算术推理任务的9个LLM（0.5B-72B）上进行了验证，DCR框架可靠地诊断出污染的严重性，并且使用DCR因子的准确性在三个基准的4％平均误差范围内调整至4％的平均误差以内。 DCR强调计算效率和透明度，提供了一种实用工具，可以将污染评估整合到常规评估中，促进更公平的比较并增强LLM基准测试实践的可信度。

Title: EXAONE 4.0: Unified Large Language Models Integrating Non-reasoning and Reasoning Modes

• Authors: LG AI Research: Kyunghoon Bae, Eunbi Choi, Kibong Choi, Stanley Jungkyu Choi, Yemuk Choi, Kyubeen Han, Seokhee Hong, Junwon Hwang, Taewan Hwang, Joonwon Jang, Hyojin Jeon, Kijeong Jeon, Gerrard Jeongwon Jo, Hyunjik Jo, Jiyeon Jung, Euisoon Kim, Hyosang Kim, Jihoon Kim, Joonkee Kim, Seonghwan Kim, Soyeon Kim, Sunkyoung Kim, Yireun Kim, Yongil Kim, Youchul Kim, Edward Hwayoung Lee, Gwangho Lee, Haeju Lee, Honglak Lee, Jinsik Lee, Kyungmin Lee, Sangha Park, Young Min Paik, Yongmin Park, Youngyong Park, Sanghyun Seo, Sihoon Yang, Heuiyeen Yeen, Sihyuk Yi, Hyeongu Yun
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.11407
• Pdf URL: https://arxiv.org/pdf/2507.11407
• Copy Paste: [[2507.11407]] EXAONE 4.0: Unified Large Language Models Integrating Non-reasoning and Reasoning Modes(https://arxiv.org/abs/2507.11407)
• Keywords: language model, agent
• Abstract: This technical report introduces EXAONE 4.0, which integrates a Non-reasoning mode and a Reasoning mode to achieve both the excellent usability of EXAONE 3.5 and the advanced reasoning abilities of EXAONE Deep. To pave the way for the agentic AI era, EXAONE 4.0 incorporates essential features such as agentic tool use, and its multilingual capabilities are extended to support Spanish in addition to English and Korean. The EXAONE 4.0 model series consists of two sizes: a mid-size 32B model optimized for high performance, and a small-size 1.2B model designed for on-device applications. The EXAONE 4.0 demonstrates superior performance compared to open-weight models in its class and remains competitive even against frontier-class models. The models are publicly available for research purposes and can be easily downloaded via this https URL.
• 摘要：该技术报告介绍了Exaone 4.0，该报告集成了非争议模式和推理模式，以实现Exaone 3.5的出色可用性和Exaone Deep的高级推理能力。为了为代理AI时代铺平道路，Exaone 4.0结合了基本功能，例如代理工具的使用，其多语言功能还扩展到除英语和韩文外，还可以支持西班牙语。 Exaone 4.0模型系列包括两种尺寸：用于高性能优化的中型32B型号，以及专为设备应用程序设计的小型1.2B型号。 Exaone 4.0与同类的开放权重模型相比表现出卓越的性能，即使在边境级型号上也保持竞争力。这些模型可公开用于研究目的，可以通过此HTTPS URL轻松下载。

Title: KisMATH: Do LLMs Have Knowledge of Implicit Structures in Mathematical Reasoning?

• Authors: Soumadeep Saha, Akshay Chaturvedi, Saptarshi Saha, Utpal Garain, Nicholas Asher
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.11408
• Pdf URL: https://arxiv.org/pdf/2507.11408
• Copy Paste: [[2507.11408]] KisMATH: Do LLMs Have Knowledge of Implicit Structures in Mathematical Reasoning?(https://arxiv.org/abs/2507.11408)
• Keywords: language model, llm, chain-of-thought
• Abstract: Chain-of-thought traces have been shown to improve performance of large language models in a plethora of reasoning tasks, yet there is no consensus on the mechanism through which this performance boost is achieved. To shed more light on this, we introduce Causal CoT Graphs (CCGs), which are directed acyclic graphs automatically extracted from reasoning traces that model fine-grained causal dependencies in the language model output. A collection of $1671$ mathematical reasoning problems from MATH500, GSM8K and AIME, and their associated CCGs are compiled into our dataset -- \textbf{KisMATH}. Our detailed empirical analysis with 15 open-weight LLMs shows that (i) reasoning nodes in the CCG are mediators for the final answer, a condition necessary for reasoning; and (ii) LLMs emphasise reasoning paths given by the CCG, indicating that models internally realise structures akin to our graphs. KisMATH enables controlled, graph-aligned interventions and opens up avenues for further investigation into the role of chain-of-thought in LLM reasoning.
• 摘要：已经证明，经过深思熟虑的痕迹可以在许多推理任务中提高大语言模型的性能，但是就实现这种绩效提升的机制尚无共识。为了更加阐明这一点，我们介绍了因果cot图（CCG），这些因素是定向的无环形图，它会自动从模拟语言模型输出中细粒因果关系依赖性的推理痕迹中提取。来自Math500，GSM8K和AIME及其相关CCG的$ 1671 $数学推理问题的集合被编译到我们的数据集中 - \ textbf {kismath}。我们对15个开放式LLM的详细经验分析表明，CCG中的推理节点是最终答案的中介者，这是推理所必需的条件； （ii）LLM强调CCG给出的推理路径，表明模型内部实现类似于我们图形的结构。 Kismath可以进行受控的，图形对准的干预措施，并开放途径，以进一步研究三重研究链在LLM推理中的作用。

Title: Seq vs Seq: An Open Suite of Paired Encoders and Decoders

• Authors: Orion Weller, Kathryn Ricci, Marc Marone, Antoine Chaffin, Dawn Lawrie, Benjamin Van Durme
• Subjects: cs.CL, cs.IR, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.11412
• Pdf URL: https://arxiv.org/pdf/2507.11412
• Copy Paste: [[2507.11412]] Seq vs Seq: An Open Suite of Paired Encoders and Decoders(https://arxiv.org/abs/2507.11412)
• Keywords: language model, llm
• Abstract: The large language model (LLM) community focuses almost exclusively on decoder-only language models, since they are easier to use for text generation. However, a large subset of the community still uses encoder-only models for tasks such as classification or retrieval. Previous work has attempted to compare these architectures, but is forced to make comparisons with models that have different numbers of parameters, training techniques, and datasets. We introduce the SOTA open-data Ettin suite of models: paired encoder-only and decoder-only models ranging from 17 million parameters to 1 billion, trained on up to 2 trillion tokens. Using the same recipe for both encoder-only and decoder-only models produces SOTA recipes in both categories for their respective sizes, beating ModernBERT as an encoder and Llama 3.2 and SmolLM2 as decoders. Like previous work, we find that encoder-only models excel at classification and retrieval tasks while decoders excel at generative tasks. However, we show that adapting a decoder model to encoder tasks (and vice versa) through continued training is subpar compared to using only the reverse objective (i.e. a 400M encoder outperforms a 1B decoder on MNLI, and vice versa for generative tasks). We open-source all artifacts of this study including training data, training order segmented by checkpoint, and 200+ checkpoints to allow future work to analyze or extend all aspects of training.
• 摘要：大型语言模型（LLM）社区几乎完全关注仅解码器的语言模型，因为它们易于用于文本生成。但是，社区的很大一部分仍使用仅编码模型来进行分类或检索等任务。以前的工作试图比较这些体系结构，但被迫与具有不同参数，培训技术和数据集的模型进行比较。我们介绍了模型的SOTA Open-DATA Ettin套件：仅配对的仅编码和解码器的模型，范围从1700万个参数到10亿，经过多达2万亿个代币的培训。使用相同的配方适用于仅编码和仅解码器模型，可为其各自尺寸的两个类别生产SOTA食谱，以编码器和Llama 3.2和Smollm2作为解码器击败Modernbert。像以前的工作一样，我们发现仅编码模型在分类和检索任务上表现出色，而解码在生成任务中出色。但是，我们表明，与仅使用反向目标相比，通过持续训练来调整解码器模型来编码任务（反之亦然）（即400m编码器在MNLI上优于1B解码器，而对生成任务的vice Vice反之亦然）。我们开源本研究的所有工件，包括培训数据，检查点分割的培训顺序以及200多个检查站，以允许将来的工作分析或扩展培训的所有方面。

Title: Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?

• Authors: Yanjian Zhang, Guillaume Wisniewski, Nadi Tomeh, Thierry Charnois
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.11423
• Pdf URL: https://arxiv.org/pdf/2507.11423
• Copy Paste: [[2507.11423]] Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?(https://arxiv.org/abs/2507.11423)
• Keywords: language model, llm, prompt
• Abstract: Human reasoning involves different strategies, each suited to specific problems. Prior work shows that large language model (LLMs) tend to favor a single reasoning strategy, potentially limiting their effectiveness in diverse reasoning challenges. In this work, we investigate whether prompting can control LLMs reasoning strategies and assess its impact on logical problem-solving. While our experiments show that no single strategy consistently improves accuracy, performance could be enhanced if models could adaptively choose the optimal strategy. We propose methods to guide LLMs in strategy selection, highlighting new ways to refine their reasoning abilities.
• 摘要：人类推理涉及不同的策略，每种策略都适合特定问题。先前的工作表明，大型语言模型（LLMS）倾向于赞成一种单一的推理策略，可能会限制其在各种推理挑战中的有效性。在这项工作中，我们调查提示是否可以控制LLMS推理策略并评估其对解决逻辑问题解决的影响。虽然我们的实验表明，没有一个策略始终如一地提高准确性，但如果模型可以自适应地选择最佳策略，则可以提高性能。我们提出了指导LLM在策略选择中的方法，并强调了完善其推理能力的新方法。

Title: HKGAI-V1: Towards Regional Sovereign Large Language Model for Hong Kong

• Authors: Sirui Han, Junqi Zhu, Ruiyuan Zhang, Yike Guo
• Subjects: cs.CL, cs.CE, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.11502
• Pdf URL: https://arxiv.org/pdf/2507.11502
• Copy Paste: [[2507.11502]] HKGAI-V1: Towards Regional Sovereign Large Language Model for Hong Kong(https://arxiv.org/abs/2507.11502)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: This paper presents the development of HKGAI-V1, a foundational sovereign large language model (LLM), developed as part of an initiative to establish value-aligned AI infrastructure specifically tailored for Hong Kong. Addressing the region's unique multilingual environment (Cantonese, Mandarin, and English), its distinct socio-legal context under the "one country, two systems" framework, and specific local cultural and value considerations, the model is built upon the DeepSeek architecture and systematically aligned with regional norms through a multifaceted full parameter fine-tuning process. It is further integrated with a retrieval-augmented generation (RAG) system to ensure timely and factually grounded information access. The core contribution lies in the design and implementation of a comprehensive, region-specific AI alignment and safety framework, demonstrated through two key achievements: 1) The successful development of HKGAI-V1 itself - which outper-forms general-purpose models in handling Hong Kong-specific culturally sensitive queries, and embodies a "governance-embedded" approach to digital sovereignty - empowers Hong Kong to exercise control over AI applications in critical sectors including public services, legal systems, and edu-cation. 2) The development of the proprietary Adversarial HK Value Benchmark, a rigorous tool for evaluating model alignment with local ethical and legal stand-ards under challenging conditions. By documenting these achievements, the paper provides not only a technological artifact but also a replicable blueprint for developing advanced, regionally focused AI systems deeply rooted in their local identities.
• 摘要：本文介绍了HKGAI-V1的发展，HKGAI-V1是一种基础主权大语模型（LLM），是作为一项计划的一部分，旨在建立专门针对香港量身定制的价值一致的AI基础架构。在该地区独特的多语言环境（广东话，普通话和英语）上，其在“一个国家，两个系统”框架以及特定的本地文化和价值方面的独特社会法律背景下，该模型建立在DeepSeek的建筑上，并通过多模式的全面参数微调流程与区域性规范进行系统地对齐。它进一步与检索型生成（RAG）系统进行了整合，以确保及时且实际的信息访问。 The core contribution lies in the design and implementation of a comprehensive, region-specific AI alignment and safety framework, demonstrated through two key achievements: 1) The successful development of HKGAI-V1 itself - which outper-forms general-purpose models in handling Hong Kong-specific culturally sensitive queries, and embodies a "governance-embedded" approach to digital sovereignty - empowers Hong Kong to对关键部门的AI应用程序进行控制，包括公共服务，法律系统和Edu-Cation。 2）专有的对抗性香气值基准的开发，这是一种严格的工具，用于评估模型在挑战性条件下与当地道德和法律替补的一致性。通过记录这些成就，本文不仅提供了技术文物，而且提供了可复制的蓝图，用于开发高级，重点的AI系统，深深地植根于其当地身份。

Title: Real-World Summarization: When Evaluation Reaches Its Limits

• Authors: Patrícia Schmidtová, Ondřej Dušek, Saad Mahamood
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.11508
• Pdf URL: https://arxiv.org/pdf/2507.11508
• Copy Paste: [[2507.11508]] Real-World Summarization: When Evaluation Reaches Its Limits(https://arxiv.org/abs/2507.11508)
• Keywords: llm
• Abstract: We examine evaluation of faithfulness to input data in the context of hotel highlights: brief LLM-generated summaries that capture unique features of accommodations. Through human evaluation campaigns involving categorical error assessment and span-level annotation, we compare traditional metrics, trainable methods, and LLM-as-a-judge approaches. Our findings reveal that simpler metrics like word overlap correlate surprisingly well with human judgments (Spearman correlation rank of 0.63), often outperforming more complex methods when applied to out-of-domain data. We further demonstrate that while LLMs can generate high-quality highlights, they prove unreliable for evaluation as they tend to severely under- or over-annotate. Our analysis of real-world business impacts shows incorrect and non-checkable information pose the greatest risks. We also highlight challenges in crowdsourced evaluations.
• 摘要：我们检查了酒店亮点中对输入数据的忠诚度的评估：简短的LLM生成的摘要，这些摘要捕获了住宿的独特功能。通过涉及分类错误评估和跨度注释的人类评估运动，我们比较了传统指标，可训练的方法和LLM-AS-A-A-A-Gudge方法。我们的发现表明，诸如单词重叠之类的简单指标与人类判断（Spearman相关等级为0.63）非常相关，当将应用于室外数据时通常比更复杂的方法相关。我们进一步证明，尽管LLM可以产生高质量的亮点，但由于它们倾向于严重不足或过度注释，因此它们对评估不可靠。我们对实际业务影响的分析表明，不正确和不可检查的信息构成了最大的风险。我们还强调了众包评估中的挑战。