2024-11-13

Title: Target-driven Attack for Large Language Models

• Authors: Chong Zhang, Mingyu Jin, Dong Shu, Taowen Wang, Dongfang Liu, Xiaobo Jin
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.07268
• Pdf URL: https://arxiv.org/pdf/2411.07268
• Copy Paste: [[2411.07268]] Target-driven Attack for Large Language Models(https://arxiv.org/abs/2411.07268)
• Keywords: language model, llm
• Abstract: Current large language models (LLM) provide a strong foundation for large-scale user-oriented natural language tasks. Many users can easily inject adversarial text or instructions through the user interface, thus causing LLM model security challenges like the language model not giving the correct answer. Although there is currently a large amount of research on black-box attacks, most of these black-box attacks use random and heuristic strategies. It is unclear how these strategies relate to the success rate of attacks and thus effectively improve model robustness. To solve this problem, we propose our target-driven black-box attack method to maximize the KL divergence between the conditional probabilities of the clean text and the attack text to redefine the attack's goal. We transform the distance maximization problem into two convex optimization problems based on the attack goal to solve the attack text and estimate the covariance. Furthermore, the projected gradient descent algorithm solves the vector corresponding to the attack text. Our target-driven black-box attack approach includes two attack strategies: token manipulation and misinformation attack. Experimental results on multiple Large Language Models and datasets demonstrate the effectiveness of our attack method.
• 摘要：当前的大型语言模型（LLM）为大规模面向用户的自然语言任务提供了坚实的基础。许多用户可以通过用户界面轻易地注入对抗性文本或指令，从而导致语言模型无法给出正确答案等LLM模型安全挑战。虽然目前有大量关于黑盒攻击的研究，但这些黑盒攻击大多采用随机和启发式策略，尚不清楚这些策略如何与攻击成功率相关，从而有效地提高模型鲁棒性。为了解决这个问题，我们提出了目标驱动的黑盒攻击方法，最大化干净文本和攻击文本的条件概率之间的KL散度，以重新定义攻击的目标。我们根据攻击目标将距离最大化问题转化为两个凸优化问题，以求解攻击文本并估计协方差。此外，投影梯度下降算法求解与攻击文本相对应的向量。我们的目标驱动的黑盒攻击方法包括两种攻击策略：令牌操纵和错误信息攻击。在多个大型语言模型和数据集上的实验结果证明了我们的攻击方法的有效性。

Title: Richer Output for Richer Countries: Uncovering Geographical Disparities in Generated Stories and Travel Recommendations

• Authors: Kirti Bhagat, Kinshuk Vasisht, Danish Pruthi
• Subjects: cs.CL, cs.AI, cs.CY, cs.LG
• Abstract URL: https://arxiv.org/abs/2411.07320
• Pdf URL: https://arxiv.org/pdf/2411.07320
• Copy Paste: [[2411.07320]] Richer Output for Richer Countries: Uncovering Geographical Disparities in Generated Stories and Travel Recommendations(https://arxiv.org/abs/2411.07320)
• Keywords: language model
• Abstract: While a large body of work inspects language models for biases concerning gender, race, occupation and religion, biases of geographical nature are relatively less explored. Some recent studies benchmark the degree to which large language models encode geospatial knowledge. However, the impact of the encoded geographical knowledge (or lack thereof) on real-world applications has not been documented. In this work, we examine large language models for two common scenarios that require geographical knowledge: (a) travel recommendations and (b) geo-anchored story generation. Specifically, we study four popular language models, and across about $100$K travel requests, and $200$K story generations, we observe that travel recommendations corresponding to poorer countries are less unique with fewer location references, and stories from these regions more often convey emotions of hardship and sadness compared to those from wealthier nations.
• 摘要：虽然大量的研究检查了语言模型中是否存在与性别、种族、职业和宗教有关的偏见，但对地理偏见的研究相对较少。最近的一些研究对大型语言模型编码地理空间知识的程度进行了基准测试。然而，编码的地理知识（或缺乏地理知识）对现实世界应用的影响尚未得到记录。在这项工作中，我们研究了两种需要地理知识的常见场景的大型语言模型：（a）旅行推荐和（b）地理锚定故事生成。具体来说，我们研究了四种流行的语言模型，在大约 10 万美元的旅行请求和 20 万美元的故事生成中，我们观察到与较贫穷国家相对应的旅行推荐不太独特，位置参考较少，与富裕国家相比，这些地区的故事更经常传达艰辛和悲伤的情绪。

Title: SetLexSem Challenge: Using Set Operations to Evaluate the Lexical and Semantic Robustness of Language Models

• Authors: Bardiya Akhbari, Manish Gawali, Nicholas A. Dronen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.07336
• Pdf URL: https://arxiv.org/pdf/2411.07336
• Copy Paste: [[2411.07336]] SetLexSem Challenge: Using Set Operations to Evaluate the Lexical and Semantic Robustness of Language Models(https://arxiv.org/abs/2411.07336)
• Keywords: language model, llm
• Abstract: Set theory is foundational to mathematics and, when sets are finite, to reasoning about the world. An intelligent system should perform set operations consistently, regardless of superficial variations in the operands. Initially designed for semantically-oriented NLP tasks, large language models (LLMs) are now being evaluated on algorithmic tasks. Because sets are comprised of arbitrary symbols (e.g. numbers, words), they provide an opportunity to test, systematically, the invariance of LLMs' algorithmic abilities under simple lexical or semantic variations. To this end, we present the SetLexSem Challenge, a synthetic benchmark that evaluates the performance of LLMs on set operations. SetLexSem assesses the robustness of LLMs' instruction-following abilities under various conditions, focusing on the set operations and the nature and construction of the set members. Evaluating seven LLMs with SetLexSem, we find that they exhibit poor robustness to variation in both operation and operands. We show -- via the framework's systematic sampling of set members along lexical and semantic dimensions -- that LLMs are not only not robust to variation along these dimensions but demonstrate unique failure modes in particular, easy-to-create semantic groupings of "deceptive" sets. We find that rigorously measuring language model robustness to variation in frequency and length is challenging and present an analysis that measures them independently. The code for reproducing the results of this paper, and for generating the SetLexSem Challenge dataset, is available at \href{this https URL}{this https URL}.
• 摘要：集合论是数学的基础，当集合是有限的，集合论是推理世界的基础。智能系统应该一致地执行集合运算，而不管操作数的表面变化如何。大型语言模型 (LLM) 最初是为面向语义的 NLP 任务设计的，现在正在算法任务上进行评估。由于集合由任意符号（例如数字、单词）组成，因此它们提供了一个机会，可以系统地测试 LLM 在简单的词汇或语义变化下的算法能力的不变性。为此，我们提出了 SetLexSem 挑战，这是一个综合基准，用于评估 LLM 在集合运算上的表现。SetLexSem 评估 LLM 在各种条件下的指令跟踪能力的稳健性，重点关注集合运算以及集合成员的性质和构造。使用 SetLexSem 评估七个 LLM，我们发现它们对操作和操作数的变化都表现出较差的稳健性。我们通过框架沿词汇和语义维度对集合成员进行系统抽样，表明 LLM 不仅对这些维度的变化不具有鲁棒性，而且表现出独特的故障模式，特别是易于创建的“欺骗性”集合的语义分组。我们发现严格测量语言模型对频率和长度变化的鲁棒性具有挑战性，并提出了一种独立测量它们的分析。用于重现本文结果和生成 SetLexSem Challenge 数据集的代码可在 \href{this https URL}{this https URL} 中找到。

Title: BeeManc at the PLABA Track of TAC-2024: RoBERTa for task 1 and LLaMA3.1 and GPT-4o for task 2

• Authors: Zhidong Ling, Zihao Li, Pablo Romeo, Lifeng Han, Goran Nenadic
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.07381
• Pdf URL: https://arxiv.org/pdf/2411.07381
• Copy Paste: [[2411.07381]] BeeManc at the PLABA Track of TAC-2024: RoBERTa for task 1 and LLaMA3.1 and GPT-4o for task 2(https://arxiv.org/abs/2411.07381)
• Keywords: gpt, prompt
• Abstract: This report is the system description of the BeeManc team for shared task Plain Language Adaptation of Biomedical Abstracts (PLABA) 2024. This report contains two sections corresponding to the two sub-tasks in PLABA 2024. In task one, we applied fine-tuned ReBERTa-Base models to identify and classify the difficult terms, jargon and acronyms in the biomedical abstracts and reported the F1 score. Due to time constraints, we didn't finish the replacement task. In task two, we leveraged Llamma3.1-70B-Instruct and GPT-4o with the one-shot prompts to complete the abstract adaptation and reported the scores in BLEU, SARI, BERTScore, LENS, and SALSA. From the official Evaluation from PLABA-2024 on Task 1A and 1B, our \textbf{much smaller fine-tuned RoBERTa-Base} model ranked 3rd and 2nd respectively on the two sub-task, and the \textbf{1st on averaged F1 scores across the two tasks} from 9 evaluated systems. Our share our fine-tuned models and related resources at \url{this https URL}
• 摘要：本报告是 BeeManc 团队对共享任务“生物医学摘要的通俗语言改编 (PLABA) 2024”的系统描述。本报告包含两个部分，分别对应 PLABA 2024 中的两个子任务。在任务一中，我们应用经过微调的 ReBERTa-Base 模型来识别和分类生物医学摘要中的难词、术语和首字母缩略词，并报告 F1 分数。由于时间限制，我们没有完成替换任务。在任务二中，我们利用 Llamma3.1-70B-Instruct 和 GPT-4o 以及一次性提示完成了摘要改编，并报告了 BLEU、SARI、BERTScore、LENS 和 SALSA 中的分数。根据 PLABA-2024 对任务 1A 和 1B 的官方评估，我们的 \textbf{小得多的微调 RoBERTa-Base} 模型在两个子任务中分别排名第三和第二，并且在 9 个评估系统中的 \textbf{两个任务的平均 F1 分数上排名第一。我们在 \url{此 https URL} 上分享了我们的微调模型和相关资源

Title: Toward Optimal Search and Retrieval for RAG

• Authors: Alexandria Leto, Cecilia Aguerrebere, Ishwar Bhati, Ted Willke, Mariano Tepper, Vy Ai Vo
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.07396
• Pdf URL: https://arxiv.org/pdf/2411.07396
• Copy Paste: [[2411.07396]] Toward Optimal Search and Retrieval for RAG(https://arxiv.org/abs/2411.07396)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Retrieval-augmented generation (RAG) is a promising method for addressing some of the memory-related challenges associated with Large Language Models (LLMs). Two separate systems form the RAG pipeline, the retriever and the reader, and the impact of each on downstream task performance is not well-understood. Here, we work towards the goal of understanding how retrievers can be optimized for RAG pipelines for common tasks such as Question Answering (QA). We conduct experiments focused on the relationship between retrieval and RAG performance on QA and attributed QA and unveil a number of insights useful to practitioners developing high-performance RAG pipelines. For example, lowering search accuracy has minor implications for RAG performance while potentially increasing retrieval speed and memory efficiency.
• 摘要：检索增强生成 (RAG) 是一种很有前途的方法，可以解决与大型语言模型 (LLM) 相关的一些内存相关挑战。两个独立的系统构成了 RAG 管道，即检索器和读取器，每个系统对下游任务性能的影响尚不清楚。在这里，我们致力于了解如何针对常见任务（例如问答 (QA)）优化检索器以使其适用于 RAG 管道。我们开展了实验，重点研究了 QA 和归因 QA 中检索和 RAG 性能之间的关系，并揭示了许多对开发高性能 RAG 管道的从业者有用的见解。例如，降低搜索准确度对 RAG 性能的影响很小，但可能会提高检索速度和内存效率。

Title: Beyond Keywords: A Context-based Hybrid Approach to Mining Ethical Concern-related App Reviews

• Authors: Aakash Sorathiya, Gouri Ginde
• Subjects: cs.CL, cs.AI, cs.SE
• Abstract URL: https://arxiv.org/abs/2411.07398
• Pdf URL: https://arxiv.org/pdf/2411.07398
• Copy Paste: [[2411.07398]] Beyond Keywords: A Context-based Hybrid Approach to Mining Ethical Concern-related App Reviews(https://arxiv.org/abs/2411.07398)
• Keywords: language model, llm
• Abstract: With the increasing proliferation of mobile applications in our everyday experiences, the concerns surrounding ethics have surged significantly. Users generally communicate their feedback, report issues, and suggest new functionalities in application (app) reviews, frequently emphasizing safety, privacy, and accountability concerns. Incorporating these reviews is essential to developing successful products. However, app reviews related to ethical concerns generally use domain-specific language and are expressed using a more varied vocabulary. Thus making automated ethical concern-related app review extraction a challenging and time-consuming effort. This study proposes a novel Natural Language Processing (NLP) based approach that combines Natural Language Inference (NLI), which provides a deep comprehension of language nuances, and a decoder-only (LLaMA-like) Large Language Model (LLM) to extract ethical concern-related app reviews at scale. Utilizing 43,647 app reviews from the mental health domain, the proposed methodology 1) Evaluates four NLI models to extract potential privacy reviews and compares the results of domain-specific privacy hypotheses with generic privacy hypotheses; 2) Evaluates four LLMs for classifying app reviews to privacy concerns; and 3) Uses the best NLI and LLM models further to extract new privacy reviews from the dataset. Results show that the DeBERTa-v3-base-mnli-fever-anli NLI model with domain-specific hypotheses yields the best performance, and Llama3.1-8B-Instruct LLM performs best in the classification of app reviews. Then, using NLI+LLM, an additional 1,008 new privacy-related reviews were extracted that were not identified through the keyword-based approach in previous research, thus demonstrating the effectiveness of the proposed approach.
• 摘要：随着移动应用程序在我们日常生活中日益普及，人们对道德问题的担忧也显著增加。用户通常会在应用程序评论中传达他们的反馈、报告问题并建议新功能，经常强调安全、隐私和问责问题。融入这些评论对于开发成功的产品至关重要。然而，与道德问题相关的应用评论通常使用特定领域的语言，并使用更多样化的词汇来表达。因此，自动提取与道德问题相关的应用评论是一项具有挑战性且耗时的工作。本研究提出了一种基于自然语言处理 (NLP) 的新型方法，该方法结合了自然语言推理 (NLI)（可以深入理解语言细微差别）和仅解码器（类似 LLaMA）的大型语言模型 (LLM)，以大规模提取与道德问题相关的应用评论。利用来自心理健康领域的 43,647 条应用评论，提出的方法 1) 评估四种 NLI 模型以提取潜在的隐私评论，并将特定领域的隐私假设的结果与一般隐私假设进行比较； 2) 评估四个 LLM 将应用评论归类为隐私问题的方法；3) 进一步使用最佳 NLI 和 LLM 模型从数据集中提取新的隐私评论。结果表明，具有领域特定假设的 DeBERTa-v3-base-mnli-fever-anli NLI 模型性能最佳，而 Llama3.1-8B-Instruct LLM 在应用评论分类中表现最佳。然后，使用 NLI+LLM，又提取了 1,008 条新的隐私相关评论，这些评论在之前的研究中没有通过基于关键字的方法识别出来，从而证明了所提方法的有效性。

Title: Controllable Context Sensitivity and the Knob Behind It

• Authors: Julian Minder, Kevin Du, Niklas Stoehr, Giovanni Monea, Chris Wendler, Robert West, Ryan Cotterell
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.07404
• Pdf URL: https://arxiv.org/pdf/2411.07404
• Copy Paste: [[2411.07404]] Controllable Context Sensitivity and the Knob Behind It(https://arxiv.org/abs/2411.07404)
• Keywords: language model, retrieval-augmented generation
• Abstract: When making predictions, a language model must trade off how much it relies on its context vs. its prior knowledge. Choosing how sensitive the model is to its context is a fundamental functionality, as it enables the model to excel at tasks like retrieval-augmented generation and question-answering. In this paper, we search for a knob which controls this sensitivity, determining whether language models answer from the context or their prior knowledge. To guide this search, we design a task for controllable context sensitivity. In this task, we first feed the model a context (Paris is in England) and a question (Where is Paris?); we then instruct the model to either use its prior or contextual knowledge and evaluate whether it generates the correct answer for both intents (either France or England). When fine-tuned on this task, instruction-tuned versions of Llama-3.1, Mistral-v0.3, and Gemma-2 can solve it with high accuracy (85-95%). Analyzing these high-performing models, we narrow down which layers may be important to context sensitivity using a novel linear time algorithm. Then, in each model, we identify a 1-D subspace in a single layer that encodes whether the model follows context or prior knowledge. Interestingly, while we identify this subspace in a fine-tuned model, we find that the exact same subspace serves as an effective knob in not only that model but also non-fine-tuned instruct and base models of that model family. Finally, we show a strong correlation between a model's performance and how distinctly it separates context-agreeing from context-ignoring answers in this subspace. These results suggest a single subspace facilitates how the model chooses between context and prior knowledge, hinting at a simple fundamental mechanism that controls this behavior.
• 摘要：在进行预测时，语言模型必须在其对上下文的依赖程度与对先验知识的依赖程度之间进行权衡。选择模型对上下文的敏感程度是一项基本功能，因为它使模型能够出色地完成检索增强生成和问答等任务。在本文中，我们寻找一个控制这种敏感度的旋钮，确定语言模型是根据上下文还是先验知识来回答。为了指导这一搜索，我们设计了一个可控上下文敏感度的任务。在这个任务中，我们首先为模型提供一个上下文（巴黎在英国）和一个问题（巴黎在哪里？）；然后我们指示模型使用其先验知识或上下文知识，并评估它是否为两个意图（法国或英国）生成了正确的答案。当在这个任务上进行微调时，指令调整版本的 Llama-3.1、Mistral-v0.3 和 Gemma-2 可以高精度（85-95%）地解决它。通过分析这些高性能模型，我们使用一种新颖的线性时间算法缩小了对上下文敏感性可能很重要的层的范围。然后，在每个模型中，我们在单个层中识别一个 1-D 子空间，该子空间对模型是否遵循上下文或先验知识进行编码。有趣的是，虽然我们在微调模型中识别了这个子空间，但我们发现完全相同的子空间不仅在该模型中充当有效旋钮，而且在该模型系列的非微调指令和基础模型中也充当有效旋钮。最后，我们展示了模型的性能与它在这个子空间中将上下文一致答案与上下文忽略答案区分开来之间的强相关性。这些结果表明单个子空间有助于模型在上下文和先验知识之间进行选择，暗示了控制这种行为的简单基本机制。

Title: Using Generative AI and Multi-Agents to Provide Automatic Feedback

• Authors: Shuchen Guo, Ehsan Latif, Yifan Zhou, Xuan Huang, Xiaoming Zhai
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.07407
• Pdf URL: https://arxiv.org/pdf/2411.07407
• Copy Paste: [[2411.07407]] Using Generative AI and Multi-Agents to Provide Automatic Feedback(https://arxiv.org/abs/2411.07407)
• Keywords: language model, llm, agent
• Abstract: This study investigates the use of generative AI and multi-agent systems to provide automatic feedback in educational contexts, particularly for student constructed responses in science assessments. The research addresses a key gap in the field by exploring how multi-agent systems, called AutoFeedback, can improve the quality of GenAI-generated feedback, overcoming known issues such as over-praise and over-inference that are common in single-agent large language models (LLMs). The study developed a multi-agent system consisting of two AI agents: one for generating feedback and another for validating and refining it. The system was tested on a dataset of 240 student responses, and its performance was compared to that of a single-agent LLM. Results showed that AutoFeedback significantly reduced the occurrence of over-praise and over-inference errors, providing more accurate and pedagogically sound feedback. The findings suggest that multi-agent systems can offer a more reliable solution for generating automated feedback in educational settings, highlighting their potential for scalable and personalized learning support. These results have important implications for educators and researchers seeking to leverage AI in formative assessments, offering a pathway to more effective feedback mechanisms that enhance student learning outcomes.
• 摘要：本研究调查了生成式人工智能和多智能体系统在教育环境中提供自动反馈的使用情况，特别是针对学生在科学评估中构建的答案。该研究通过探索多智能体系统（称为 AutoFeedback）如何提高 GenAI 生成的反馈的质量，克服单智能体大型语言模型 (LLM) 中常见的过度赞扬和过度推理等已知问题，解决了该领域的一个关键差距。该研究开发了一个由两个人工智能代理组成的多智能体系统：一个用于生成反馈，另一个用于验证和改进反馈。该系统在 240 个学生答案的数据集上进行了测试，并将其性能与单智能体 LLM 的性能进行了比较。结果表明，AutoFeedback 显著减少了过度赞扬和过度推理错误的发生，提供了更准确、更符合教学法的反馈。研究结果表明，多智能体系统可以为在教育环境中生成自动反馈提供更可靠的解决方案，凸显了它们在可扩展和个性化学习支持方面的潜力。这些结果对于寻求在形成性评估中利用人工智能的教育工作者和研究人员具有重要意义，为提高学生学习成果的更有效的反馈机制提供了途径。

Title: Untangling Hate Speech Definitions: A Semantic Componential Analysis Across Cultures and Domains

• Authors: Katerina Korre, Arianna Muti, Federico Ruggeri, Alberto Barrón-Cedeño
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.07417
• Pdf URL: https://arxiv.org/pdf/2411.07417
• Copy Paste: [[2411.07417]] Untangling Hate Speech Definitions: A Semantic Componential Analysis Across Cultures and Domains(https://arxiv.org/abs/2411.07417)
• Keywords: llm, prompt
• Abstract: Hate speech relies heavily on cultural influences, leading to varying individual interpretations. For that reason, we propose a Semantic Componential Analysis (SCA) framework for a cross-cultural and cross-domain analysis of hate speech definitions. We create the first dataset of definitions derived from five domains: online dictionaries, research papers, Wikipedia articles, legislation, and online platforms, which are later analyzed into semantic components. Our analysis reveals that the components differ from definition to definition, yet many domains borrow definitions from one another without taking into account the target culture. We conduct zero-shot model experiments using our proposed dataset, employing three popular open-sourced LLMs to understand the impact of different definitions on hate speech detection. Our findings indicate that LLMs are sensitive to definitions: responses for hate speech detection change according to the complexity of definitions used in the prompt.
• 摘要：仇恨言论在很大程度上依赖于文化影响，导致不同的个人解读。为此，我们提出了一个语义成分分析 (SCA) 框架，用于跨文化和跨领域分析仇恨言论定义。我们创建了第一个定义数据集，该数据集来自五个领域：在线词典、研究论文、维基百科文章、立法和在线平台，然后将其分析为语义成分。我们的分析表明，各个定义之间的成分各不相同，但许多领域相互借用定义而不考虑目标文化。我们使用我们提出的数据集进行零样本模型实验，采用三种流行的开源 LLM 来了解不同定义对仇恨言论检测的影响。我们的研究结果表明，LLM 对定义很敏感：仇恨言论检测的响应会根据提示中使用的定义的复杂性而变化。

Title: Efficient and Accurate Prompt Optimization: the Benefit of Memory in Exemplar-Guided Reflection

• Authors: Cilin Yan, Jingyun Wang, Lin Zhang, Ruihui Zhao, Xiaopu Wu, Kai Xiong, Qingsong Liu, Guoliang Kang, Yangyang Kang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.07446
• Pdf URL: https://arxiv.org/pdf/2411.07446
• Copy Paste: [[2411.07446]] Efficient and Accurate Prompt Optimization: the Benefit of Memory in Exemplar-Guided Reflection(https://arxiv.org/abs/2411.07446)
• Keywords: language model, llm, prompt
• Abstract: Automatic prompt engineering aims to enhance the generation quality of large language models (LLMs). Recent works utilize feedbacks generated from erroneous cases to guide the prompt optimization. During inference, they may further retrieve several semantically-related exemplars and concatenate them to the optimized prompts to improve the performance. However, those works only utilize the feedback at the current step, ignoring historical and unseleccted feedbacks which are potentially beneficial. Moreover, the selection of exemplars only considers the general semantic relationship and may not be optimal in terms of task performance and matching with the optimized prompt. In this work, we propose an Exemplar-Guided Reflection with Memory mechanism (ERM) to realize more efficient and accurate prompt optimization. Specifically, we design an exemplar-guided reflection mechanism where the feedback generation is additionally guided by the generated exemplars. We further build two kinds of memory to fully utilize the historical feedback information and support more effective exemplar retrieval. Empirical evaluations show our method surpasses previous state-of-the-arts with less optimization steps, i.e., improving F1 score by 10.1 on LIAR dataset, and reducing half of the optimization steps on ProTeGi.
• 摘要：自动提示工程旨在提高大型语言模型 (LLM) 的生成质量。最近的研究利用从错误案例生成的反馈来指导提示优化。在推理过程中，它们可能会进一步检索几个语义相关的范例并将它们连接到优化的提示以提高性能。然而，这些工作只利用当前步骤的反馈，忽略了可能有益的历史和未选择的反馈。此外，范例的选择只考虑一般的语义关系，在任务性能和与优化提示的匹配方面可能不是最佳的。在本文中，我们提出了一种带记忆机制的范例引导反射 (ERM)，以实现更高效、更准确的提示优化。具体而言，我们设计了一种范例引导的反射机制，其中反馈生成还由生成的范例引导。我们进一步构建了两种记忆，以充分利用历史反馈信息并支持更有效的范例检索。实证评估表明，我们的方法以更少的优化步骤超越了以前的最先进技术，例如，在 LIAR 数据集上将 F1 分数提高了 10.1，并在 ProTeGi 上减少了一半的优化步骤。

Title: DecoPrompt : Decoding Prompts Reduces Hallucinations when Large Language Models Meet False Premises

• Authors: Nan Xu, Xuezhe Ma
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.07457
• Pdf URL: https://arxiv.org/pdf/2411.07457
• Copy Paste: [[2411.07457]] DecoPrompt : Decoding Prompts Reduces Hallucinations when Large Language Models Meet False Premises(https://arxiv.org/abs/2411.07457)
• Keywords: language model, llm, hallucination, prompt
• Abstract: While large language models (LLMs) have demonstrated increasing power, they have also called upon studies on their hallucinated outputs that deviate from factually correct statements. In this paper, we focus on one important scenario of false premises, where LLMs are distracted by misaligned claims although the model possesses the required factual knowledge to answer original questions accurately. Inspired by the observation that entropy of the false-premise prompt is closely related to its likelihood to elicit hallucination generation, we propose a new prompting algorithm, named DecoPrompt, to mitigate hallucination. DecoPrompt leverages LLMs to "decode" the false-premise prompts without really eliciting hallucination output from LLMs. We perform experiments on two datasets, demonstrating that DecoPrompt can reduce hallucinations effectively on outputs from different LLMs. Moreover, DecoPrompt exhibits cross-model transferability, which facilitates its applications to scenarios such as LLMs of large sizes or unavailable model logits.
• 摘要：虽然大型语言模型 (LLM) 已显示出越来越强大的能力，但它们也要求研究其与事实正确陈述相悖的幻觉输出。在本文中，我们重点关注一个重要的错误前提场景，其中 LLM 被不一致的主张所分散注意力，尽管该模型拥有准确回答原始问题所需的事实知识。受错误前提提示的熵与其引发幻觉产生的可能性密切相关的观察启发，我们提出了一种名为 DecoPrompt 的新提示算法来减轻幻觉。DecoPrompt 利用 LLM 来“解码”错误前提提示，而无需真正从 LLM 中引发幻觉输出。我们在两个数据集上进行了实验，证明 DecoPrompt 可以有效减少不同 LLM 输出的幻觉。此外，DecoPrompt 表现出跨模型可迁移性，这有助于将其应用于大型 LLM 或不可用的模型逻辑等场景。

Title: IdentifyMe: A Challenging Long-Context Mention Resolution Benchmark

• Authors: Kawshik Manikantan, Makarand Tapaswi, Vineet Gandhi, Shubham Toshniwal
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2411.07466
• Pdf URL: https://arxiv.org/pdf/2411.07466
• Copy Paste: [[2411.07466]] IdentifyMe: A Challenging Long-Context Mention Resolution Benchmark(https://arxiv.org/abs/2411.07466)
• Keywords: gpt, llm
• Abstract: Recent evaluations of LLMs on coreference resolution have revealed that traditional output formats and evaluation metrics do not fully capture the models' referential understanding. To address this, we introduce IdentifyMe, a new benchmark for mention resolution presented in a multiple-choice question (MCQ) format, commonly used for evaluating LLMs. IdentifyMe features long narratives and employs heuristics to exclude easily identifiable mentions, creating a more challenging task. The benchmark also consists of a curated mixture of different mention types and corresponding entities, allowing for a fine-grained analysis of model performance. We evaluate both closed- and open source LLMs on IdentifyMe and observe a significant performance gap (20-30%) between the state-of-the-art sub-10B open models vs. closed ones. We observe that pronominal mentions, which have limited surface information, are typically much harder for models to resolve than nominal mentions. Additionally, we find that LLMs often confuse entities when their mentions overlap in nested structures. The highest-scoring model, GPT-4o, achieves 81.9% accuracy, highlighting the strong referential capabilities of state-of-the-art LLMs while also indicating room for further improvement.
• 摘要：最近对 LLM 的共指解析评估表明，传统的输出格式和评估指标不能完全捕捉模型的指称理解。为了解决这个问题，我们引入了 IdentityMe，这是一个以多项选择题 (MCQ) 格式呈现的提及解析的新基准，通常用于评估 LLM。IdentifyMe 具有长篇叙述，并采用启发式方法来排除容易识别的提及，从而创建了更具挑战性的任务。该基准还包含不同提及类型和相应实体的精选混合，允许对模型性能进行细粒度分析。我们在 IdentityMe 上评估了闭源和开源 LLM，并观察到最先进的 10B 以下开放模型与闭源模型之间存在显著的性能差距 (20-30%)。我们观察到，代词提及具有有限的表面信息，通常比名词提及更难被模型解析。此外，我们发现当 LLM 的提及在嵌套结构中重叠时，它们通常会混淆实体。得分最高的模型 GPT-4o 的准确率达到了 81.9%，突显了最先进的 LLM 强大的参考能力，同时也表明还有进一步改进的空间。

Title: Controlled Evaluation of Syntactic Knowledge in Multilingual Language Models

• Authors: Daria Kryvosheieva, Roger Levy
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.07474
• Pdf URL: https://arxiv.org/pdf/2411.07474
• Copy Paste: [[2411.07474]] Controlled Evaluation of Syntactic Knowledge in Multilingual Language Models(https://arxiv.org/abs/2411.07474)
• Keywords: language model
• Abstract: Language models (LMs) are capable of acquiring elements of human-like syntactic knowledge. Targeted syntactic evaluation tests have been employed to measure how well they form generalizations about syntactic phenomena in high-resource languages such as English. However, we still lack a thorough understanding of LMs' capacity for syntactic generalizations in low-resource languages, which are responsible for much of the diversity of syntactic patterns worldwide. In this study, we develop targeted syntactic evaluation tests for three low-resource languages (Basque, Hindi, and Swahili) and use them to evaluate five families of open-access multilingual Transformer LMs. We find that some syntactic tasks prove relatively easy for LMs while others (agreement in sentences containing indirect objects in Basque, agreement across a prepositional phrase in Swahili) are challenging. We additionally uncover issues with publicly available Transformers, including a bias toward the habitual aspect in Hindi in multilingual BERT and underperformance compared to similar-sized models in XGLM-4.5B.
• 摘要：语言模型 (LM) 能够获取类似人类的句法知识元素。有针对性的句法评估测试已用于衡量它们对英语等高资源语言中的句法现象形成概括的能力。然而，我们仍然缺乏对 LM 在低资源语言中进行句法概括的能力的透彻了解，而低资源语言是造成全球句法模式多样性的主要原因。在本研究中，我们为三种低资源语言（巴斯克语、印地语和斯瓦希里语）开发了有针对性的句法评估测试，并使用它们来评估五种开放获取的多语言 Transformer LM 系列。我们发现一些句法任务对于 LM 来说相对容易，而其他一些任务（巴斯克语中包含间接宾语的句子的一致性、斯瓦希里语中介词短语的一致性）则具有挑战性。我们还发现了公开可用的 Transformer 存在的问题，包括多语言 BERT 对印地语中习惯方面的偏见以及与 XGLM-4.5B 中类似大小的模型相比表现不佳。

Title: Rapid Response: Mitigating LLM Jailbreaks with a Few Examples

• Authors: Alwin Peng, Julian Michael, Henry Sleight, Ethan Perez, Mrinank Sharma
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.07494
• Pdf URL: https://arxiv.org/pdf/2411.07494
• Copy Paste: [[2411.07494]] Rapid Response: Mitigating LLM Jailbreaks with a Few Examples(https://arxiv.org/abs/2411.07494)
• Keywords: language model, llm
• Abstract: As large language models (LLMs) grow more powerful, ensuring their safety against misuse becomes crucial. While researchers have focused on developing robust defenses, no method has yet achieved complete invulnerability to attacks. We propose an alternative approach: instead of seeking perfect adversarial robustness, we develop rapid response techniques to look to block whole classes of jailbreaks after observing only a handful of attacks. To study this setting, we develop RapidResponseBench, a benchmark that measures a defense's robustness against various jailbreak strategies after adapting to a few observed examples. We evaluate five rapid response methods, all of which use jailbreak proliferation, where we automatically generate additional jailbreaks similar to the examples observed. Our strongest method, which fine-tunes an input classifier to block proliferated jailbreaks, reduces attack success rate by a factor greater than 240 on an in-distribution set of jailbreaks and a factor greater than 15 on an out-of-distribution set, having observed just one example of each jailbreaking strategy. Moreover, further studies suggest that the quality of proliferation model and number of proliferated examples play an key role in the effectiveness of this defense. Overall, our results highlight the potential of responding rapidly to novel jailbreaks to limit LLM misuse.
• 摘要：随着大型语言模型 (LLM) 变得越来越强大，确保其免受滥用变得至关重要。虽然研究人员一直致力于开发强大的防御措施，但目前还没有一种方法能够完全抵御攻击。我们提出了一种替代方法：我们不是寻求完美的对抗鲁棒性，而是开发快速响应技术，在仅观察到少数攻击后，就试图阻止整个越狱类别。为了研究这种情况，我们开发了 RapidResponseBench，这是一个基准，在适应一些观察到的示例后，可以衡量防御对各种越狱策略的鲁棒性。我们评估了五种快速响应方法，所有这些方法都使用越狱扩散，我们会自动生成与观察到的示例类似的其他越狱。我们最强大的方法是微调输入分类器以阻止扩散的越狱，在仅观察到每种越狱策略的一个示例后，将分布内越狱集的攻击成功率降低了 240 倍以上，将分布外越狱集的攻击成功率降低了 15 倍以上。此外，进一步的研究表明，扩散模型的质量和扩散示例的数量对这种防御的有效性起着关键作用。总体而言，我们的结果凸显了快速应对新型越狱以限制 LLM 滥用的潜力。

Title: Fair Summarization: Bridging Quality and Diversity in Extractive Summaries

• Authors: Sina Bagheri Nezhad, Sayan Bandyapadhyay, Ameeta Agrawal
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.07521
• Pdf URL: https://arxiv.org/pdf/2411.07521
• Copy Paste: [[2411.07521]] Fair Summarization: Bridging Quality and Diversity in Extractive Summaries(https://arxiv.org/abs/2411.07521)
• Keywords: gpt
• Abstract: Fairness in multi-document summarization of user-generated content remains a critical challenge in natural language processing (NLP). Existing summarization methods often fail to ensure equitable representation across different social groups, leading to biased outputs. In this paper, we introduce two novel methods for fair extractive summarization: FairExtract, a clustering-based approach, and FairGPT, which leverages GPT-3.5-turbo with fairness constraints. We evaluate these methods using Divsumm summarization dataset of White-aligned, Hispanic, and African-American dialect tweets and compare them against relevant baselines. The results obtained using a comprehensive set of summarization quality metrics such as SUPERT, BLANC, SummaQA, BARTScore, and UniEval, as well as a fairness metric F, demonstrate that FairExtract and FairGPT achieve superior fairness while maintaining competitive summarization quality. Additionally, we introduce composite metrics (e.g., SUPERT+F, BLANC+F) that integrate quality and fairness into a single evaluation framework, offering a more nuanced understanding of the trade-offs between these objectives. This work highlights the importance of fairness in summarization and sets a benchmark for future research in fairness-aware NLP models.
• 摘要：用户生成内容多文档摘要的公平性仍然是自然语言处理 (NLP) 中的一个关键挑战。现有的摘要方法通常无法确保不同社会群体之间的公平代表性，从而导致输出结果存在偏差。在本文中，我们介绍了两种新的公平提取摘要方法：基于聚类的方法 FairExtract 和利用具有公平性约束的 GPT-3.5-turbo 的 FairGPT。我们使用白人、西班牙裔和非裔美国人方言推文的 Divsumm 摘要数据集评估这些方法，并将它们与相关基线进行比较。使用一组全面的摘要质量指标（例如 SUPERT、BLANC、SummaQA、BARTScore 和 UniEval）以及公平性指标 F 获得的结果表明，FairExtract 和 FairGPT 在保持有竞争力的摘要质量的同时实现了卓越的公平性。此外，我们引入了综合指标（例如 SUPERT+F、BLANC+F），将质量和公平性整合到一个评估框架中，从而更细致地理解这些目标之间的权衡。这项工作强调了公平性在摘要中的重要性，并为公平性感知 NLP 模型的未来研究树立了标杆。

Title: Prompt-enhanced Network for Hateful Meme Classification

• Authors: Junxi Liu, Yanyan Feng, Jiehai Chen, Yun Xue, Fenghuan Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.07527
• Pdf URL: https://arxiv.org/pdf/2411.07527
• Copy Paste: [[2411.07527]] Prompt-enhanced Network for Hateful Meme Classification(https://arxiv.org/abs/2411.07527)
• Keywords: language model, prompt
• Abstract: The dynamic expansion of social media has led to an inundation of hateful memes on media platforms, accentuating the growing need for efficient identification and removal. Acknowledging the constraints of conventional multimodal hateful meme classification, which heavily depends on external knowledge and poses the risk of including irrelevant or redundant content, we developed Pen -- a prompt-enhanced network framework based on the prompt learning approach. Specifically, after constructing the sequence through the prompt method and encoding it with a language model, we performed region information global extraction on the encoded sequence for multi-view perception. By capturing global information about inference instances and demonstrations, Pen facilitates category selection by fully leveraging sequence information. This approach significantly improves model classification accuracy. Additionally, to bolster the model's reasoning capabilities in the feature space, we introduced prompt-aware contrastive learning into the framework to improve the quality of sample feature distributions. Through extensive ablation experiments on two public datasets, we evaluate the effectiveness of the Pen framework, concurrently comparing it with state-of-the-art model baselines. Our research findings highlight that Pen surpasses manual prompt methods, showcasing superior generalization and classification accuracy in hateful meme classification tasks. Our code is available at this https URL.
• 摘要：社交媒体的动态扩张导致媒体平台上充斥着大量仇恨表情包，这加剧了对有效识别和删除的需求。考虑到传统的多模态仇恨表情包分类严重依赖外部知识，存在包含不相关或冗余内容的风险，我们开发了 Pen——一种基于提示学习方法的提示增强网络框架。具体来说，在通过提示方法构建序列并使用语言模型对其进行编码后，我们对编码序列进行区域信息全局提取，以实现多视角感知。通过捕获有关推理实例和演示的全局信息，Pen 充分利用序列信息来促进类别选择。这种方法显著提高了模型分类准确率。此外，为了增强模型在特征空间中的推理能力，我们在框架中引入了提示感知对比学习，以提高样本特征分布的质量。通过在两个公共数据集上进行大量消融实验，我们评估了 Pen 框架的有效性，同时将其与最先进的模型基线进行了比较。我们的研究结果表明，Pen 超越了手动提示方法，在仇恨模因分类任务中展现出卓越的泛化能力和分类准确率。我们的代码可在此 https URL 上找到。

Title: Large Language Models as Neurolinguistic Subjects: Identifying Internal Representations for Form and Meaning

• Authors: Linyang He, Ercong Nie, Helmut Schmid, Hinrich Schütze, Nima Mesgarani, Jonathan Brennan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.07533
• Pdf URL: https://arxiv.org/pdf/2411.07533
• Copy Paste: [[2411.07533]] Large Language Models as Neurolinguistic Subjects: Identifying Internal Representations for Form and Meaning(https://arxiv.org/abs/2411.07533)
• Keywords: language model, llm
• Abstract: This study investigates the linguistic understanding of Large Language Models (LLMs) regarding signifier (form) and signified (meaning) by distinguishing two LLM evaluation paradigms: psycholinguistic and neurolinguistic. Traditional psycholinguistic evaluations often reflect statistical biases that may misrepresent LLMs' true linguistic capabilities. We introduce a neurolinguistic approach, utilizing a novel method that combines minimal pair and diagnostic probing to analyze activation patterns across model layers. This method allows for a detailed examination of how LLMs represent form and meaning, and whether these representations are consistent across languages. Our contributions are three-fold: (1) We compare neurolinguistic and psycholinguistic methods, revealing distinct patterns in LLM assessment; (2) We demonstrate that LLMs exhibit higher competence in form compared to meaning, with the latter largely correlated to the former; (3) We present new conceptual minimal pair datasets for Chinese (COMPS-ZH) and German (COMPS-DE), complementing existing English datasets.
• 摘要：本研究通过区分两种 LLM 评估范式：心理语言学和神经语言学，研究了大型语言模型 (LLM) 对能指（形式）和所指（意义）的语言理解。传统的心理语言学评估往往反映统计偏差，可能会歪曲 LLM 的真实语言能力。我们引入了一种神经语言学方法，利用一种结合最小对和诊断探测的新方法来分析跨模型层的激活模式。这种方法可以详细检查 LLM 如何表示形式和意义，以及这些表示在不同语言之间是否一致。我们的贡献有三方面：(1) 我们比较了神经语言学和心理语言学方法，揭示了 LLM 评估中的不同模式；(2) 我们证明 LLM 在形式上的能力高于意义，后者与前者有很大相关性；(3) 我们为中文 (COMPS-ZH) 和德语 (COMPS-DE) 提供了新的概念最小对数据集，以补充现有的英语数据集。

Title: Problem-Oriented Segmentation and Retrieval: Case Study on Tutoring Conversations

• Authors: Rose E. Wang, Pawan Wirawarn, Kenny Lam, Omar Khattab, Dorottya Demszky
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.07598
• Pdf URL: https://arxiv.org/pdf/2411.07598
• Copy Paste: [[2411.07598]] Problem-Oriented Segmentation and Retrieval: Case Study on Tutoring Conversations(https://arxiv.org/abs/2411.07598)
• Keywords: language model, llm
• Abstract: Many open-ended conversations (e.g., tutoring lessons or business meetings) revolve around pre-defined reference materials, like worksheets or meeting bullets. To provide a framework for studying such conversation structure, we introduce Problem-Oriented Segmentation & Retrieval (POSR), the task of jointly breaking down conversations into segments and linking each segment to the relevant reference item. As a case study, we apply POSR to education where effectively structuring lessons around problems is critical yet difficult. We present LessonLink, the first dataset of real-world tutoring lessons, featuring 3,500 segments, spanning 24,300 minutes of instruction and linked to 116 SAT math problems. We define and evaluate several joint and independent approaches for POSR, including segmentation (e.g., TextTiling), retrieval (e.g., ColBERT), and large language models (LLMs) methods. Our results highlight that modeling POSR as one joint task is essential: POSR methods outperform independent segmentation and retrieval pipelines by up to +76% on joint metrics and surpass traditional segmentation methods by up to +78% on segmentation metrics. We demonstrate POSR's practical impact on downstream education applications, deriving new insights on the language and time use in real-world lesson structures.
• 摘要：许多开放式对话（例如辅导课或商务会议）都围绕预定义的参考资料（如工作表或会议要点）展开。为了提供研究此类对话结构的框架，我们引入了面向问题的细分和检索 (POSR)，即将对话联合分解为多个片段，并将每个片段链接到相关的参考项目。作为案例研究，我们将 POSR 应用于教育领域，在教育领域，有效地围绕问题构建课程至关重要但又很困难。我们介绍了 LessonLink，这是第一个真实世界辅导课程的数据集，包含 3,500 个片段，涵盖 24,300 分钟的教学时间，并与 116 道 SAT 数学题相关联。我们定义并评估了 POSR 的几种联合和独立方法，包括细分（例如 TextTiling）、检索（例如 ColBERT）和大型语言模型 (LLM) 方法。我们的结果表明，将 POSR 建模为一个联合任务至关重要：POSR 方法在联合指标上的表现比独立分割和检索流程高出多达 76%，在分割指标上的表现比传统分割方法高出多达 78%。我们展示了 POSR 对下游教育应用的实际影响，从而对现实世界课程结构中的语言和时间使用产生了新的见解。

Title: Multimodal Clinical Reasoning through Knowledge-augmented Rationale Generation

• Authors: Shuai Niu, Jing Ma, Liang Bai, Zhihua Wang, Yida Xu, Yunya Song, Xian Yang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.07611
• Pdf URL: https://arxiv.org/pdf/2411.07611
• Copy Paste: [[2411.07611]] Multimodal Clinical Reasoning through Knowledge-augmented Rationale Generation(https://arxiv.org/abs/2411.07611)
• Keywords: language model, llm
• Abstract: Clinical rationales play a pivotal role in accurate disease diagnosis; however, many models predominantly use discriminative methods and overlook the importance of generating supportive rationales. Rationale distillation is a process that transfers knowledge from large language models (LLMs) to smaller language models (SLMs), thereby enhancing the latter's ability to break down complex tasks. Despite its benefits, rationale distillation alone is inadequate for addressing domain knowledge limitations in tasks requiring specialized expertise, such as disease diagnosis. Effectively embedding domain knowledge in SLMs poses a significant challenge. While current LLMs are primarily geared toward processing textual data, multimodal LLMs that incorporate time series data, especially electronic health records (EHRs), are still evolving. To tackle these limitations, we introduce ClinRaGen, an SLM optimized for multimodal rationale generation in disease diagnosis. ClinRaGen incorporates a unique knowledge-augmented attention mechanism to merge domain knowledge with time series EHR data, utilizing a stepwise rationale distillation strategy to produce both textual and time series-based clinical rationales. Our evaluations show that ClinRaGen markedly improves the SLM's capability to interpret multimodal EHR data and generate accurate clinical rationales, supporting more reliable disease diagnosis, advancing LLM applications in healthcare, and narrowing the performance divide between LLMs and SLMs.
• 摘要：临床原理在准确的疾病诊断中起着关键作用；然而，许多模型主要使用判别方法，而忽视了生成支持性原理的重要性。原理提炼是一个将知识从大型语言模型 (LLM) 转移到小型语言模型 (SLM) 的过程，从而增强后者分解复杂任务的能力。尽管原理提炼有其好处，但单靠原理提炼不足以解决需要专业知识的任务（例如疾病诊断）中的领域知识限制。有效地将领域知识嵌入 SLM 是一项重大挑战。虽然当前的 LLM 主要用于处理文本数据，但结合时间序列数据（尤其是电子健康记录 (EHR)）的多模态 LLM 仍在不断发展。为了解决这些限制，我们引入了 ClinRaGen，这是一种针对疾病诊断中的多模态原理生成进行了优化的 SLM。ClinRaGen 结合了独特的知识增强注意力机制，将领域知识与时间序列 EHR 数据相结合，利用逐步原理提炼策略来生成基于文本和时间序列的临床原理。我们的评估表明，ClinRaGen 显著提高了 SLM 解释多模式 EHR 数据和生成准确临床原理的能力，支持更可靠的疾病诊断，推进 LLM 在医疗保健领域的应用，并缩小 LLM 和 SLM 之间的性能差距。

Title: Mitigating Bias in Queer Representation within Large Language Models: A Collaborative Agent Approach

• Authors: Tianyi Huang (1), Arya Somasundaram (1) ((1) App-In Club)
• Subjects: cs.CL, cs.MA
• Abstract URL: https://arxiv.org/abs/2411.07656
• Pdf URL: https://arxiv.org/pdf/2411.07656
• Copy Paste: [[2411.07656]] Mitigating Bias in Queer Representation within Large Language Models: A Collaborative Agent Approach(https://arxiv.org/abs/2411.07656)
• Keywords: language model, gpt, llm, agent
• Abstract: Large Language Models (LLMs) often perpetuate biases in pronoun usage, leading to misrepresentation or exclusion of queer individuals. This paper addresses the specific problem of biased pronoun usage in LLM outputs, particularly the inappropriate use of traditionally gendered pronouns ("he," "she") when inclusive language is needed to accurately represent all identities. We introduce a collaborative agent pipeline designed to mitigate these biases by analyzing and optimizing pronoun usage for inclusivity. Our multi-agent framework includes specialized agents for both bias detection and correction. Experimental evaluations using the Tango dataset-a benchmark focused on gender pronoun usage-demonstrate that our approach significantly improves inclusive pronoun classification, achieving a 32.6 percentage point increase over GPT-4o in correctly disagreeing with inappropriate traditionally gendered pronouns $(\chi^2 = 38.57, p < 0.0001)$. These results accentuate the potential of agent-driven frameworks in enhancing fairness and inclusivity in AI-generated content, demonstrating their efficacy in reducing biases and promoting socially responsible AI.
• 摘要：大型语言模型 (LLM) 通常会延续代词使用中的偏见，导致对酷儿个体的错误表述或排斥。本文讨论了 LLM 输出中代词使用偏见的具体问题，特别是在需要包容性语言来准确表示所有身份时不恰当地使用传统性别代词（“他”、“她”）。我们引入了一个协作代理管道，旨在通过分析和优化代词使用以实现包容性来减轻这些偏见。我们的多代理框架包括用于偏见检测和纠正的专用代理。使用 Tango 数据集（一个专注于性别代词使用的基准）进行的实验评估表明，我们的方法显着提高了包容性代词分类，在正确反对不适当的传统性别代词方面比 GPT-4o 提高了 32.6 个百分点 $(\chi^2 = 38.57, p < 0.0001)$。这些结果强调了代理驱动框架在增强人工智能生成内容的公平性和包容性方面的潜力，证明了其在减少偏见和促进人工智能社会责任方面的有效性。

Title: Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows

• Authors: Fangyu Lei, Jixuan Chen, Yuxiao Ye, Ruisheng Cao, Dongchan Shin, Hongjin Su, Zhaoqing Suo, Hongcheng Gao, Wenjing Hu, Pengcheng Yin, Victor Zhong, Caiming Xiong, Ruoxi Sun, Qian Liu, Sida Wang, Tao Yu
• Subjects: cs.CL, cs.AI, cs.DB
• Abstract URL: https://arxiv.org/abs/2411.07763
• Pdf URL: https://arxiv.org/pdf/2411.07763
• Copy Paste: [[2411.07763]] Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows(https://arxiv.org/abs/2411.07763)
• Keywords: language model, long context, agent
• Abstract: Real-world enterprise text-to-SQL workflows often involve complex cloud or local data across various database systems, multiple SQL queries in various dialects, and diverse operations from data transformation to analytics. We introduce Spider 2.0, an evaluation framework comprising 632 real-world text-to-SQL workflow problems derived from enterprise-level database use cases. The databases in Spider 2.0 are sourced from real data applications, often containing over 1,000 columns and stored in local or cloud database systems such as BigQuery and Snowflake. We show that solving problems in Spider 2.0 frequently requires understanding and searching through database metadata, dialect documentation, and even project-level codebases. This challenge calls for models to interact with complex SQL workflow environments, process extremely long contexts, perform intricate reasoning, and generate multiple SQL queries with diverse operations, often exceeding 100 lines, which goes far beyond traditional text-to-SQL challenges. Our evaluations indicate that based on o1-preview, our code agent framework successfully solves only 17.0% of the tasks, compared with 91.2% on Spider 1.0 and 73.0% on BIRD. Our results on Spider 2.0 show that while language models have demonstrated remarkable performance in code generation -- especially in prior text-to-SQL benchmarks -- they require significant improvement in order to achieve adequate performance for real-world enterprise usage. Progress on Spider 2.0 represents crucial steps towards developing intelligent, autonomous, code agents for real-world enterprise settings. Our code, baseline models, and data are available at this https URL.
• 摘要：现实世界中的企业文本到 SQL 工作流通常涉及跨各种数据库系统的复杂云或本地数据、各种方言的多个 SQL 查询以及从数据转换到分析的各种操作。我们推出了 Spider 2.0，这是一个评估框架，包含 632 个来自企业级数据库用例的真实文本到 SQL 工作流问题。Spider 2.0 中的数据库来自真实数据应用程序，通常包含 1,000 多个列，并存储在本地或云数据库系统（例如 BigQuery 和 Snowflake）中。我们表明，解决 Spider 2.0 中的问题通常需要理解和搜索数据库元数据、方言文档甚至项目级代码库。这一挑战要求模型与复杂的 SQL 工作流环境交互，处理极长的上下文，执行复杂的推理，并生成具有多种操作的多个 SQL 查询，通常超过 100 行，这远远超出了传统的文本到 SQL 挑战。我们的评估表明，基于 o1-preview，我们的代码代理框架仅成功解决了 17.0% 的任务，而 Spider 1.0 上成功解决了 91.2%，BIRD 上成功解决了 73.0%。我们在 Spider 2.0 上的结果表明，虽然语言模型在代码生成方面表现出色——尤其是在之前的文本到 SQL 基准测试中——但它们需要显著改进才能达到适合实际企业使用的性能。Spider 2.0 的进展代表着朝着为实际企业环境开发智能、自主的代码代理迈出了关键一步。我们的代码、基线模型和数据可在此 https URL 上找到。

Title: Likelihood as a Performance Gauge for Retrieval-Augmented Generation

• Authors: Tianyu Liu, Jirui Qi, Paul He, Arianna Bisazza, Mrinmaya Sachan, Ryan Cotterell
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2411.07773
• Pdf URL: https://arxiv.org/pdf/2411.07773
• Copy Paste: [[2411.07773]] Likelihood as a Performance Gauge for Retrieval-Augmented Generation(https://arxiv.org/abs/2411.07773)
• Keywords: language model, prompt, retrieval-augmented generation
• Abstract: Recent work finds that retrieval-augmented generation with large language models is prone to be influenced by the order of retrieved documents in the context. However, the lack of in-depth analysis limits the use of this phenomenon for prompt engineering in practice. In this study, we posit that likelihoods serve as an effective gauge for language model performance. Through experiments on two question-answering datasets with a variety of state-of-the-art language models, we reveal correlations between answer accuracy and the likelihood of the question at both the corpus level and the instance level. In addition, we find that question likelihood can also indicate the position of the task-relevant information in the context. Based on these findings, we propose two methods that use question likelihood as a gauge for selecting and constructing prompts that lead to better performance. We demonstrate their effectiveness with experiments. In addition, our likelihood-based methods are efficient, as they only need to compute the likelihood of the input, requiring much fewer language model passes than heuristic prompt engineering methods that require generating responses. Our analysis deepens our understanding of how input prompts affect model performance and provides a promising direction for efficient prompt optimization.
• 摘要：最近的研究发现，使用大型语言模型的检索增强生成很容易受到上下文中检索文档的顺序的影响。然而，缺乏深入的分析限制了这种现象在实践中用于提示工程。在本研究中，我们假设似然度是语言模型性能的有效衡量标准。通过对两个问答数据集进行实验，使用各种最先进的语言模型，我们在语料库级别和实例级别揭示了答案准确性与问题似然度之间的相关性。此外，我们发现问题似然度还可以指示任务相关信息在上下文中的位置。基于这些发现，我们提出了两种使用问题似然度作为选择和构建提示的衡量标准的方法，从而获得更好的性能。我们通过实验证明了它们的有效性。此外，我们的基于似然度的方法是高效的，因为它们只需要计算输入的似然度，与需要生成响应的启发式提示工程方法相比，所需的语言模型传递次数要少得多。我们的分析加深了我们对输入提示如何影响模型性能的理解，并为有效的提示优化提供了一个有希望的方向。

Title: Query Optimization for Parametric Knowledge Refinement in Retrieval-Augmented Large Language Models

• Authors: Youan Cong, Cheng Wang, Pritom Saha Akash, Kevin Chen-Chuan Chang
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2411.07820
• Pdf URL: https://arxiv.org/pdf/2411.07820
• Copy Paste: [[2411.07820]] Query Optimization for Parametric Knowledge Refinement in Retrieval-Augmented Large Language Models(https://arxiv.org/abs/2411.07820)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: We introduce the \textit{Extract-Refine-Retrieve-Read} (ERRR) framework, a novel approach designed to bridge the pre-retrieval information gap in Retrieval-Augmented Generation (RAG) systems through query optimization tailored to meet the specific knowledge requirements of Large Language Models (LLMs). Unlike conventional query optimization techniques used in RAG, the ERRR framework begins by extracting parametric knowledge from LLMs, followed by using a specialized query optimizer for refining these queries. This process ensures the retrieval of only the most pertinent information essential for generating accurate responses. Moreover, to enhance flexibility and reduce computational costs, we propose a trainable scheme for our pipeline that utilizes a smaller, tunable model as the query optimizer, which is refined through knowledge distillation from a larger teacher model. Our evaluations on various question-answering (QA) datasets and with different retrieval systems show that ERRR consistently outperforms existing baselines, proving to be a versatile and cost-effective module for improving the utility and accuracy of RAG systems.
• 摘要：我们引入了 \textit{提取-细化-检索-读取} (ERRR) 框架，这是一种新颖的方法，旨在通过定制查询优化来弥补检索增强生成 (RAG) 系统中的预检索信息差距，以满足大型语言模型 (LLM) 的特定知识要求。与 RAG 中使用的传统查询优化技术不同，ERRR 框架首先从 LLM 中提取参数知识，然后使用专门的查询优化器来细化这些查询。此过程可确保仅检索生成准确响应所必需的最相关信息。此外，为了增强灵活性并降低计算成本，我们为我们的管道提出了一种可训练方案，该方案利用较小的可调模型作为查询优化器，并通过从较大的教师模型中进行知识提炼来细化。我们对各种问答 (QA) 数据集和不同检索系统的评估表明，ERRR 的表现始终优于现有基线，证明它是一个多功能且经济高效的模块，可用于提高 RAG 系统的实用性和准确性。

Title: Chain Association-based Attacking and Shielding Natural Language Processing Systems

• Authors: Jiacheng Huang, Long Chen
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.07843
• Pdf URL: https://arxiv.org/pdf/2411.07843
• Copy Paste: [[2411.07843]] Chain Association-based Attacking and Shielding Natural Language Processing Systems(https://arxiv.org/abs/2411.07843)
• Keywords: language model
• Abstract: Association as a gift enables people do not have to mention something in completely straightforward words and allows others to understand what they intend to refer to. In this paper, we propose a chain association-based adversarial attack against natural language processing systems, utilizing the comprehension gap between humans and machines. We first generate a chain association graph for Chinese characters based on the association paradigm for building search space of potential adversarial examples. Then, we introduce an discrete particle swarm optimization algorithm to search for the optimal adversarial examples. We conduct comprehensive experiments and show that advanced natural language processing models and applications, including large language models, are vulnerable to our attack, while humans appear good at understanding the perturbed text. We also explore two methods, including adversarial training and associative graph-based recovery, to shield systems from chain association-based attack. Since a few examples that use some derogatory terms, this paper contains materials that may be offensive or upsetting to some people.
• 摘要：联想作为一种天赋，使人们不必用完全直白的语言提及某件事，而让别人明白他们想要指代什么。在本文中，我们利用人与机器之间的理解差距，提出了一种针对自然语言处理系统的基于链式联想的对抗性攻击。我们首先基于联想范式生成汉字链式联想图，以构建潜在对抗性示例的搜索空间。然后，我们引入离散粒子群优化算法来搜索最佳对抗性示例。我们进行了全面的实验，表明先进的自然语言处理模型和应用程序（包括大型语言模型）容易受到我们的攻击，而人类似乎擅长理解受干扰的文本。我们还探索了两种方法，包括对抗性训练和基于联想图的恢复，以保护系统免受基于链式联想的攻击。由于一些示例使用了一些贬义词，本文包含的内容可能会冒犯或让某些人感到不安。

Title: Ethical Concern Identification in NLP: A Corpus of ACL Anthology Ethics Statements

• Authors: Antonia Karamolegkou, Sandrine Schiller Hansen, Ariadni Christopoulou, Filippos Stamatiou, Anne Lauscher, Anders Søgaard
• Subjects: cs.CL, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2411.07845
• Pdf URL: https://arxiv.org/pdf/2411.07845
• Copy Paste: [[2411.07845]] Ethical Concern Identification in NLP: A Corpus of ACL Anthology Ethics Statements(https://arxiv.org/abs/2411.07845)
• Keywords: llm
• Abstract: What ethical concerns, if any, do LLM researchers have? We introduce EthiCon, a corpus of 1,580 ethical concern statements extracted from scientific papers published in the ACL Anthology. We extract ethical concern keywords from the statements and show promising results in automating the concern identification process. Through a survey, we compare the ethical concerns of the corpus to the concerns listed by the general public and professionals in the field. Finally, we compare our retrieved ethical concerns with existing taxonomies pointing to gaps and future research directions.
• 摘要：LLM 研究人员有哪些道德顾虑（如果有的话）？我们介绍了 EthiCon，这是一个从 ACL Anthology 上发表的科学论文中提取的 1,580 条道德顾虑声明的语料库。我们从这些声明中提取了道德顾虑关键词，并在自动化顾虑识别过程中显示出令人鼓舞的结果。通过一项调查，我们将语料库中的道德顾虑与公众和该领域专业人士列出的顾虑进行了比较。最后，我们将检索到的道德顾虑与现有分类法进行比较，指出了差距和未来的研究方向。

Title: Tucano: Advancing Neural Text Generation for Portuguese

• Authors: Nicholas Kluge Corrêa, Aniket Sen, Sophia Falk, Shiza Fatimah
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2411.07854
• Pdf URL: https://arxiv.org/pdf/2411.07854
• Copy Paste: [[2411.07854]] Tucano: Advancing Neural Text Generation for Portuguese(https://arxiv.org/abs/2411.07854)
• Keywords: language model
• Abstract: Significant advances have been made in natural language processing in recent years. However, our current deep learning approach to language modeling requires substantial resources in terms of data and computation. One of the side effects of this data-hungry paradigm is the current schism between languages, separating those considered high-resource, where most of the development happens and resources are available, and the low-resource ones, which struggle to attain the same level of performance and autonomy. This study aims to introduce a new set of resources to stimulate the future development of neural text generation in Portuguese. In this work, we document the development of GigaVerbo, a concatenation of deduplicated Portuguese text corpora amounting to 200 billion tokens. Via this corpus, we trained a series of decoder-transformers named Tucano. Our models perform equal or superior to other Portuguese and multilingual language models of similar size in several Portuguese benchmarks. The evaluation of our models also reveals that model performance on many currently available benchmarks used by the Portuguese NLP community has little to no correlation with the scaling of token ingestion during training, highlighting the limitations of such evaluations when it comes to the assessment of Portuguese generative language models. All derivatives of our study are openly released on GitHub and Hugging Face. See this https URL
• 摘要：近年来，自然语言处理取得了重大进展。然而，我们目前的语言建模深度学习方法需要大量的数据和计算资源。这种数据密集型范式的副作用之一是语言之间的分裂，将资源丰富的语言与资源匮乏的语言区分开来，前者大部分开发工作都在这里进行，资源充足，后者则难以达到同样的性能和自主性水平。这项研究旨在引入一套新的资源，以促进葡萄牙语神经文本生成的未来发展。在这项工作中，我们记录了 GigaVerbo 的开发过程，GigaVerbo 是一个去重葡萄牙语文本语料库的集合，共计 2000 亿个标记。通过这个语料库，我们训练了一系列名为 Tucano 的解码器转换器。在几个葡萄牙语基准测试中，我们的模型表现与其他类似规模的葡萄牙语和多语言语言模型相当或更胜一筹。对我们模型的评估还表明，葡萄牙语 NLP 社区目前使用的许多基准测试中的模型性能与训练期间 token 摄入的扩展几乎没有关联，这凸显了此类评估在评估葡萄牙语生成语言模型时的局限性。我们研究的所有衍生品均在 GitHub 和 Hugging Face 上公开发布。请参阅此 https URL

Title: Verbosity $\neq$ Veracity: Demystify Verbosity Compensation Behavior of Large Language Models

• Authors: Yusen Zhang, Sarkar Snigdha Sarathi Das, Rui Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.07858
• Pdf URL: https://arxiv.org/pdf/2411.07858
• Copy Paste: [[2411.07858]] Verbosity $\neq$ Veracity: Demystify Verbosity Compensation Behavior of Large Language Models(https://arxiv.org/abs/2411.07858)
• Keywords: language model, gpt, llm, prompt
• Abstract: When unsure about an answer, humans often respond with more words than necessary, hoping that part of the response will be correct. We observe a similar behavior in large language models (LLMs), which we term "Verbosity Compensation" (VC). VC is harmful because it confuses the user understanding, leading to low efficiency, and influences the LLM services by increasing the latency and cost of generating useless tokens. In this paper, we present the first work that defines and analyzes Verbosity Compensation, explores its causes, and proposes a simple mitigating approach. We define Verbosity Compensation as the behavior of generating responses that can be compressed without information loss when prompted to write concisely. Our experiments, conducted on five datasets of knowledge and reasoning-based QA tasks with 14 newly developed LLMs, reveal three conclusions. 1) We reveal a pervasive presence of verbosity compensation across all models and all datasets. Notably, GPT-4 exhibits a VC frequency of 50.40%. 2) We reveal the large performance gap between verbose and concise responses, with a notable difference of 27.61% on the Qasper dataset. We also demonstrate that this difference does not naturally diminish as LLM capability increases. Both 1) and 2) highlight the urgent need to mitigate the frequency of VC behavior and disentangle verbosity with veracity. We propose a simple yet effective cascade algorithm that replaces the verbose responses with the other model-generated responses. The results show that our approach effectively alleviates the VC of the Mistral model from 63.81% to 16.16% on the Qasper dataset. 3) We also find that verbose responses exhibit higher uncertainty across all five datasets, suggesting a strong connection between verbosity and model uncertainty. Our dataset and code are available at this https URL.
• 摘要：当不确定答案时，人类通常会用比必要更多的词语来回应，希望部分回应是正确的。我们在大型语言模型 (LLM) 中观察到了类似的行为，我们将其称为“冗长补偿”(VC)。VC 是有害的，因为它会混淆用户的理解，导致效率低下，并通过增加生成无用标记的延迟和成本来影响 LLM 服务。在本文中，我们介绍了第一篇定义和分析冗长补偿、探讨其原因并提出一种简单缓解方法的工作。我们将冗长补偿定义为在提示简洁书写时生成可以压缩而不会丢失信息的响应的行为。我们使用 14 个新开发的 LLM 在五个基于知识和推理的 QA 任务数据集上进行的实验得出了三个结论。1) 我们发现冗长补偿在所有模型和所有数据集中普遍存在。值得注意的是，GPT-4 的 VC 频率为 50.40%。 2) 我们发现详细和简洁响应之间存在巨大的性能差距，在 Qasper 数据集上差异显著，为 27.61%。我们还表明，随着 LLM 能力的提高，这种差异不会自然减小。1) 和 2) 都强调了降低 VC 行为频率并将冗长与准确性区分开来的迫切需要。我们提出了一种简单但有效的级联算法，用其他模型生成的响应替换详细响应。结果表明，我们的方法有效地将 Mistral 模型在 Qasper 数据集上的 VC 从 63.81% 降低到 16.16%。3) 我们还发现，详细响应在所有五个数据集中都表现出更高的不确定性，这表明冗长和模型不确定性之间存在很强的联系。我们的数据集和代码可在此 https URL 上找到。

Title: Trustful LLMs: Customizing and Grounding Text Generation with Knowledge Bases and Dual Decoders

• Authors: Xiaofeng Zhu, Jaya Krishna Mandivarapu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.07870
• Pdf URL: https://arxiv.org/pdf/2411.07870
• Copy Paste: [[2411.07870]] Trustful LLMs: Customizing and Grounding Text Generation with Knowledge Bases and Dual Decoders(https://arxiv.org/abs/2411.07870)
• Keywords: language model, gpt, llm, hallucination, chat, retrieval-augmented generation
• Abstract: Although people are impressed by the content generation skills of large language models, the use of LLMs, such as ChatGPT, is limited by the domain grounding of the content. The correctness and groundedness of the generated content need to be based on a verified context, such as results from Retrieval-Augmented Generation (RAG). One important issue when adapting LLMs to a customized domain is that the generated responses are often incomplete, or the additions are not verified and may even be hallucinated. Prior studies on hallucination detection have focused on evaluation metrics, which are not easily adaptable to dynamic domains and can be vulnerable to attacks like jail-breaking. In this work, we propose 1) a post-processing algorithm that leverages knowledge triplets in RAG context to correct hallucinations and 2) a dual-decoder model that fuses RAG context to guide the generation process.
• 摘要：尽管人们对大型语言模型的内容生成技能印象深刻，但 ChatGPT 等 LLM 的使用受到内容领域基础的限制。生成内容的正确性和基础性需要基于经过验证的上下文，例如检索增强生成 (RAG) 的结果。将 LLM 适配到自定义域时的一个重要问题是生成的响应通常不完整，或者添加的内容未经验证甚至可能是幻觉。先前对幻觉检测的研究侧重于评估指标，这些指标不易适应动态领域，并且容易受到越狱等攻击。在这项工作中，我们提出了 1) 一种利用 RAG 上下文中的知识三元组来纠正幻觉的后处理算法和 2) 一种融合 RAG 上下文来指导生成过程的双解码器模型。

Title: CryptoLLM: Unleashing the Power of Prompted LLMs for SmartQnA and Classification of Crypto Posts

• Authors: Aniket Deroy, Subhankar Maity
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.07917
• Pdf URL: https://arxiv.org/pdf/2411.07917
• Copy Paste: [[2411.07917]] CryptoLLM: Unleashing the Power of Prompted LLMs for SmartQnA and Classification of Crypto Posts(https://arxiv.org/abs/2411.07917)
• Keywords: gpt, llm, prompt
• Abstract: The rapid growth of social media has resulted in an large volume of user-generated content, particularly in niche domains such as cryptocurrency. This task focuses on developing robust classification models to accurately categorize cryptocurrency-related social media posts into predefined classes, including but not limited to objective, positive, negative, etc. Additionally, the task requires participants to identify the most relevant answers from a set of posts in response to specific questions. By leveraging advanced LLMs, this research aims to enhance the understanding and filtering of cryptocurrency discourse, thereby facilitating more informed decision-making in this volatile sector. We have used a prompt-based technique to solve the classification task for reddit posts and twitter posts. Also, we have used 64-shot technique along with prompts on GPT-4-Turbo model to determine whether a answer is relevant to a question or not.
• 摘要：社交媒体的快速发展导致了大量用户生成内容，特别是在加密货币等小众领域。此任务侧重于开发强大的分类模型，以准确地将与加密货币相关的社交媒体帖子分类为预定义的类别，包括但不限于客观、积极、消极等。此外，该任务还要求参与者从一组帖子中找出与特定问题最相关的答案。通过利用高级 LLM，本研究旨在增强对加密货币话语的理解和过滤，从而促进在这个动荡的领域做出更明智的决策。我们使用基于提示的技术来解决 reddit 帖子和 twitter 帖子的分类任务。此外，我们还使用了 64 次拍摄技术以及 GPT-4-Turbo 模型上的提示来确定答案是否与问题相关。

Title: From General to Specific: Utilizing General Hallucation to Automatically Measure the Role Relationship Fidelity for Specific Role-Play Agents

• Authors: Chuyi Kong, Ziyang Luo, Hongzhan Lin, Zhiyuan Fan, Yaxin Fan, Yuxi Sun, Jing Ma
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.07965
• Pdf URL: https://arxiv.org/pdf/2411.07965
• Copy Paste: [[2411.07965]] From General to Specific: Utilizing General Hallucation to Automatically Measure the Role Relationship Fidelity for Specific Role-Play Agents(https://arxiv.org/abs/2411.07965)
• Keywords: language model, gpt, llm, hallucination, prompt, chat, agent
• Abstract: The advanced role-playing capabilities of Large Language Models (LLMs) have paved the way for developing Role-Playing Agents (RPAs). However, existing benchmarks, such as HPD, which incorporates manually scored character relationships into the context for LLMs to sort coherence, and SocialBench, which uses specific profiles generated by LLMs in the context of multiple-choice tasks to assess character preferences, face limitations like poor generalizability, implicit and inaccurate judgments, and excessive context length. To address the above issues, we propose an automatic, scalable, and generalizable paradigm. Specifically, we construct a benchmark by extracting relations from a general knowledge graph and leverage RPA's inherent hallucination properties to prompt it to interact across roles, employing ChatGPT for stance detection and defining relationship hallucination along with three related metrics. Extensive experiments validate the effectiveness and stability of our metrics. Our findings further explore factors influencing these metrics and discuss the trade-off between relationship hallucination and factuality.
• 摘要：大型语言模型 (LLM) 的高级角色扮演能力为开发角色扮演代理 (RPA) 铺平了道路。然而，现有的基准测试，例如 HPD，将手动评分的角色关系纳入 LLM 的上下文中以对连贯性进行排序，以及 SocialBench，它使用 LLM 在多项选择任务的上下文中生成的特定配置文件来评估角色偏好，面临着诸如通用性差、判断隐含和不准确以及上下文过长等限制。为了解决上述问题，我们提出了一种自动、可扩展和可推广的范式。具体来说，我们通过从一般知识图中提取关系来构建基准，并利用 RPA 固有的幻觉属性来促使它跨角色交互，使用 ChatGPT 进行立场检测并定义关系幻觉以及三个相关指标。大量实验验证了我们指标的有效性和稳定性。我们的研究结果进一步探讨了影响这些指标的因素，并讨论了关系幻觉和事实性之间的权衡。

Title: Derivational Morphology Reveals Analogical Generalization in Large Language Models

• Authors: Valentin Hofmann, Leonie Weissweiler, David Mortensen, Hinrich Schütze, Janet Pierrehumbert
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2411.07990
• Pdf URL: https://arxiv.org/pdf/2411.07990
• Copy Paste: [[2411.07990]] Derivational Morphology Reveals Analogical Generalization in Large Language Models(https://arxiv.org/abs/2411.07990)
• Keywords: language model, gpt, llm
• Abstract: What mechanisms underlie linguistic generalization in large language models (LLMs)? This question has attracted considerable attention, with most studies analyzing the extent to which the language skills of LLMs resemble rules. As of yet, it is not known whether linguistic generalization in LLMs could equally well be explained as the result of analogical processes, which can be formalized as similarity operations on stored exemplars. A key shortcoming of prior research is its focus on linguistic phenomena with a high degree of regularity, for which rule-based and analogical approaches make the same predictions. Here, we instead examine derivational morphology, specifically English adjective nominalization, which displays notable variability. We introduce a new method for investigating linguistic generalization in LLMs: focusing on GPT-J, we fit cognitive models that instantiate rule-based and analogical learning to the LLM training data and compare their predictions on a set of nonce adjectives with those of the LLM, allowing us to draw direct conclusions regarding underlying mechanisms. As expected, rule-based and analogical models explain the predictions of GPT-J equally well for adjectives with regular nominalization patterns. However, for adjectives with variable nominalization patterns, the analogical model provides a much better match. Furthermore, GPT-J's behavior is sensitive to the individual word frequencies, even for regular forms, a behavior that is consistent with an analogical account of regular forms but not a rule-based one. These findings refute the hypothesis that GPT-J's linguistic generalization on adjective nominalization involves rules, suggesting similarity operations on stored exemplars as the underlying mechanism. Overall, our study suggests that analogical processes play a bigger role in the linguistic generalization of LLMs than previously thought.
• 摘要：大型语言模型 (LLM) 中的语言泛化背后的机制是什么？这个问题引起了广泛关注，大多数研究分析了 LLM 的语言技能与规则的相似程度。到目前为止，尚不清楚 LLM 中的语言泛化是否可以同样很好地解释为类比过程的结果，类比过程可以形式化为对存储的样本的相似性操作。先前研究的一个关键缺点是它关注的是具有高度规律性的语言现象，基于规则的方法和类比方法对这些现象做出相同的预测。在这里，我们改为研究派生形态，特别是英语形容词名词化，它表现出明显的多变性。我们引入了一种研究 LLM 中语言泛化的新方法：专注于 GPT-J，我们将基于规则和类比学习的认知模型与 LLM 训练数据相匹配，并将它们对一组随机形容词的预测与 LLM 的预测进行比较，从而让我们能够直接得出关于潜在机制的结论。正如预期的那样，基于规则的模型和类比模型同样很好地解释了 GPT-J 对具有常规名词化模式的形容词的预测。然而，对于具有可变名词化模式的形容词，类比模型提供了更好的匹配。此外，GPT-J 的行为对单个词频很敏感，即使对于常规形式也是如此，这种行为与常规形式的类比解释一致，但与基于规则的解释不一致。这些发现驳斥了 GPT-J 对形容词名词化的语言泛化涉及规则的假设，表明对存储的范例的相似性操作是潜在机制。总的来说，我们的研究表明，类比过程在 LLM 的语言泛化中发挥的作用比以前认为的更大。

Title: ExpressivityArena: Can LLMs Express Information Implicitly?

• Authors: Joshua Tint, Som Sagar, Aditya Taparia, Kelly Raines, Bimsara Pathiraja, Caleb Liu, Ransalu Senanayake
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.08010
• Pdf URL: https://arxiv.org/pdf/2411.08010
• Copy Paste: [[2411.08010]] ExpressivityArena: Can LLMs Express Information Implicitly?(https://arxiv.org/abs/2411.08010)
• Keywords: language model, llm
• Abstract: While Large Language Models (LLMs) have demonstrated remarkable performance in certain dimensions, their ability to express implicit language cues that human use for effective communication remains unclear. This paper presents ExpressivityArena, a Python library for measuring the implicit communication abilities of LLMs. We provide a comprehensive framework to evaluate expressivity of arbitrary LLMs and explore its practical implications. To this end, we refine the definition and measurements of ``expressivity,'' and use our framework in a set of small experiments. These experiments test LLMs in creative and logical tasks such as poetry, coding, and emotion-based responses. They are then evaluated by an automated grader, through ExpressivityArena, which we verify to be the most pragmatic for testing expressivity. Building on these experiments, we deepen our understanding of the expressivity of LLMs by assessing their ability to remain expressive in conversations. Our findings indicate that LLMs are capable of generating and understanding expressive content, however, with some limitations. These insights will inform the future development and deployment of expressive LLMs. We provide the code for ExpressivityArena alongside our paper.
• 摘要：虽然大型语言模型 (LLM) 在某些方面表现出色，但它们表达人类用于有效交流的隐性语言线索的能力仍不清楚。本文介绍了 ExpressivityArena，这是一个用于测量 LLM 隐性交流能力的 Python 库。我们提供了一个全面的框架来评估任意 LLM 的表达能力并探索其实际意义。为此，我们改进了“表达能力”的定义和测量方法，并在一系列小型实验中使用了我们的框架。这些实验在诗歌、编码和基于情感的反应等创造性和逻辑性任务中测试 LLM。然后，它们由自动评分器通过 ExpressivityArena 进行评估，我们证实这是测试表达能力最实用的方法。在这些实验的基础上，我们通过评估 LLM 在对话中保持表达能力的能力来加深对 LLM 表达能力的理解。我们的研究结果表明，LLM 能够生成和理解富有表现力的内容，但也存在一些局限性。这些见解将为未来富有表现力的 LLM 的开发和部署提供参考。我们在论文中提供了 ExpressivityArena 的代码。

Title: Language Models as Causal Effect Generators

• Authors: Lucius E.J. Bynum, Kyunghyun Cho
• Subjects: cs.CL, cs.AI, cs.LG, stat.AP, stat.ME, stat.ML
• Abstract URL: https://arxiv.org/abs/2411.08019
• Pdf URL: https://arxiv.org/pdf/2411.08019
• Copy Paste: [[2411.08019]] Language Models as Causal Effect Generators(https://arxiv.org/abs/2411.08019)
• Keywords: language model, llm
• Abstract: We present a framework for large language model (LLM) based data generation with controllable causal structure. In particular, we define a procedure for turning any language model and any directed acyclic graph (DAG) into a sequence-driven structural causal model (SD-SCM). Broadly speaking, an SD-SCM is a causal model with user-defined structure and LLM-defined structural equations. We characterize how an SD-SCM allows sampling from observational, interventional, and counterfactual distributions according to the desired causal structure. We then leverage this procedure to propose a new type of benchmark for causal inference methods, generating individual-level counterfactual data without needing to manually specify functional relationships between variables. We create an example benchmark consisting of thousands of datasets, and test a suite of popular estimation methods on these datasets for average, conditional average, and individual treatment effect estimation, both with and without hidden confounding. Apart from generating data, the same procedure also allows us to test for the presence of a causal effect that might be encoded in an LLM. This procedure can underpin auditing LLMs for misinformation, discrimination, or otherwise undesirable behavior. We believe SD-SCMs can serve as a useful tool in any application that would benefit from sequential data with controllable causal structure.
• 摘要：我们提出了一个基于大型语言模型 (LLM) 的可控因果结构数据生成框架。具体来说，我们定义了一个将任何语言模型和任何有向无环图 (DAG) 转换为序列驱动的结构因果模型 (SD-SCM) 的过程。广义上讲，SD-SCM 是一种具有用户定义结构和 LLM 定义结构方程的因果模型。我们描述了 SD-SCM 如何根据所需的因果结构从观察、干预和反事实分布中进行抽样。然后，我们利用此过程为因果推理方法提出了一种新型基准，无需手动指定变量之间的函数关系即可生成个体级反事实数据。我们创建了一个由数千个数据集组成的示例基准，并在这些数据集上测试了一套流行的估计方法，以进行平均值、条件平均值和个体治疗效果估计，包括有无隐藏混杂因素。除了生成数据之外，同样的程序还允许我们测试可能编码在 LLM 中的因果效应的存在。此程序可以支持审计 LLM 是否存在错误信息、歧视或其他不良行为。我们相信，SD-SCM 可以成为任何受益于具有可控因果结构的序列数据的应用程序中的有用工具。