2023-12-27

language model

Title: Dynamic Syntax Mapping: A New Approach to Unsupervised Syntax Parsing. (arXiv:2312.14966v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2312.14966
• Code URL: null
• Copy Paste: [[2312.14966]] Dynamic Syntax Mapping: A New Approach to Unsupervised Syntax Parsing(http://arxiv.org/abs/2312.14966)
• Summary:

  The intricate hierarchical structure of syntax is fundamental to the intricate and systematic nature of human language. This study investigates the premise that language models, specifically their attention distributions, can encapsulate syntactic dependencies. We introduce Dynamic Syntax Mapping (DSM), an innovative approach for the agnostic induction of these structures. Our method diverges from traditional syntax models which rely on predefined annotation schemata. Instead, we focus on a core characteristic inherent in dependency relations: syntactic substitutability. This concept refers to the interchangeability of words within the same syntactic category at either end of a dependency. By leveraging this property, we generate a collection of syntactically invariant sentences, which serve as the foundation for our parsing framework. Our findings reveal that the use of an increasing array of substitutions notably enhances parsing precision on natural language data. Specifically, in the context of long-distance subject-verb agreement, DSM exhibits a remarkable advancement over prior methodologies. Furthermore, DSM's adaptability is demonstrated through its successful application in varied parsing scenarios, underscoring its broad applicability.

Title: Towards a Unified Multimodal Reasoning Framework. (arXiv:2312.15021v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2312.15021
• Code URL: https://github.com/tomohiro-sawada/cs7643-final-project
• Copy Paste: [[2312.15021]] Towards a Unified Multimodal Reasoning Framework(http://arxiv.org/abs/2312.15021)
• Summary:

  Recent advancements in deep learning have led to the development of powerful language models (LMs) that excel in various tasks. Despite these achievements, there is still room for improvement, particularly in enhancing reasoning abilities and incorporating multimodal data. This report investigates the potential impact of combining Chain-of-Thought (CoT) reasoning and Visual Question Answering (VQA) techniques to improve LM's accuracy in solving multiple-choice questions. By employing TextVQA and ScienceQA datasets, we assessed the effectiveness of three text embedding methods and three visual embedding approaches. Our experiments aimed to fill the gap in current research by investigating the combined impact of CoT and VQA, contributing to the understanding of how these techniques can improve the reasoning capabilities of state-of-the-art models like GPT-4. Results from our experiments demonstrated the potential of these approaches in enhancing LM's reasoning and question-answering capabilities, providing insights for further research and development in the field, and paving the way for more accurate and reliable AI systems that can handle complex reasoning tasks across multiple modalities.

Title: Moderating New Waves of Online Hate with Chain-of-Thought Reasoning in Large Language Models. (arXiv:2312.15099v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2312.15099
• Code URL: https://github.com/cactilab/hateguard
• Copy Paste: [[2312.15099]] Moderating New Waves of Online Hate with Chain-of-Thought Reasoning in Large Language Models(http://arxiv.org/abs/2312.15099)
• Summary:

  Online hate is an escalating problem that negatively impacts the lives of Internet users, and is also subject to rapid changes due to evolving events, resulting in new waves of online hate that pose a critical threat. Detecting and mitigating these new waves present two key challenges: it demands reasoning-based complex decision-making to determine the presence of hateful content, and the limited availability of training samples hinders updating the detection model. To address this critical issue, we present a novel framework called HATEGUARD for effectively moderating new waves of online hate. HATEGUARD employs a reasoning-based approach that leverages the recently introduced chain-of-thought (CoT) prompting technique, harnessing the capabilities of large language models (LLMs). HATEGUARD further achieves prompt-based zero-shot detection by automatically generating and updating detection prompts with new derogatory terms and targets in new wave samples to effectively address new waves of online hate. To demonstrate the effectiveness of our approach, we compile a new dataset consisting of tweets related to three recently witnessed new waves: the 2022 Russian invasion of Ukraine, the 2021 insurrection of the US Capitol, and the COVID-19 pandemic. Our studies reveal crucial longitudinal patterns in these new waves concerning the evolution of events and the pressing need for techniques to rapidly update existing moderation tools to counteract them. Comparative evaluations against state-of-the-art tools illustrate the superiority of our framework, showcasing a substantial 22.22% to 83.33% improvement in detecting the three new waves of online hate. Our work highlights the severe threat posed by the emergence of new waves of online hate and represents a paradigm shift in addressing this threat practically.

gpt

Title: Assessing the Impact of Prompting, Persona, and Chain of Thought Methods on ChatGPT's Arithmetic Capabilities. (arXiv:2312.15006v1 [cs.AI])

• Paper URL: http://arxiv.org/abs/2312.15006
• Code URL: null
• Copy Paste: [[2312.15006]] Assessing the Impact of Prompting, Persona, and Chain of Thought Methods on ChatGPT's Arithmetic Capabilities(http://arxiv.org/abs/2312.15006)
• Summary:

  This study critically evaluates the mathematical proficiency of OpenAI's language model, ChatGPT, by juxtaposing its default computational capabilities against the efficiency of three prescriptive methods: strategic prompting, persona implementation, and the Chain of Thought approach. The evaluation harnessed the diverse and extensive problem sets from the MATH, GSM8K, and MMLU data-sets, which encompassing a broad spectrum of mathematical conundrums and levels of complexity. A sophisticated grading script was designed to determine the efficacy of these interventions in enhancing the model's mathematical precision. Contrary to expectations, our empirical analysis revealed that none of the trialed methods substantially improved ChatGPT's baseline performance. In some cases, these interventions inadvertently disrupted the model's response generation. This investigation concluded that while the pursuit of innovative strategies for augmenting language model performance remains crucial, the specific methods examined within this study did not induce significant improvements in ChatGPT's computational aptitude. These findings underscore the importance of further comprehensive research and exploration of novel techniques to enhance the precision and dependability of such models across diverse domains.

llm

Title: Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Intervention. (arXiv:2312.15033v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2312.15033
• Code URL: null
• Copy Paste: [[2312.15033]] Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Intervention(http://arxiv.org/abs/2312.15033)
• Summary:

  Large Language Models (LLMs) have achieved unprecedented breakthroughs in various natural language processing domains. However, the enigmatic ``black-box'' nature of LLMs remains a significant challenge for interpretability, hampering transparent and accountable applications. While past approaches, such as attention visualization, pivotal subnetwork extraction, and concept-based analyses, offer some insight, they often focus on either local or global explanations within a single dimension, occasionally falling short in providing comprehensive clarity. In response, we propose a novel methodology anchored in sparsity-guided techniques, aiming to provide a holistic interpretation of LLMs. Our framework, termed SparseCBM, innovatively integrates sparsity to elucidate three intertwined layers of interpretation: input, subnetwork, and concept levels. In addition, the newly introduced dimension of interpretable inference-time intervention facilitates dynamic adjustments to the model during deployment. Through rigorous empirical evaluations on real-world datasets, we demonstrate that SparseCBM delivers a profound understanding of LLM behaviors, setting it apart in both interpreting and ameliorating model inaccuracies. Codes are provided in supplements.

long context

lora

Title: Gradient Shaping for Multi-Constraint Safe Reinforcement Learning. (arXiv:2312.15127v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2312.15127
• Code URL: null
• Copy Paste: [[2312.15127]] Gradient Shaping for Multi-Constraint Safe Reinforcement Learning(http://arxiv.org/abs/2312.15127)
• Summary:

  Online safe reinforcement learning (RL) involves training a policy that maximizes task efficiency while satisfying constraints via interacting with the environments. In this paper, our focus lies in addressing the complex challenges associated with solving multi-constraint (MC) safe RL problems. We approach the safe RL problem from the perspective of Multi-Objective Optimization (MOO) and propose a unified framework designed for MC safe RL algorithms. This framework highlights the manipulation of gradients derived from constraints. Leveraging insights from this framework and recognizing the significance of \textit{redundant} and \textit{conflicting} constraint conditions, we introduce the Gradient Shaping (GradS) method for general Lagrangian-based safe RL algorithms to improve the training efficiency in terms of both reward and constraint satisfaction. Our extensive experimentation demonstrates the effectiveness of our proposed method in encouraging exploration and learning a policy that improves both safety and reward performance across various challenging MC safe RL tasks as well as good scalability to the number of constraints.

hallucination

prompt

Title: Learning to Prompt Knowledge Transfer for Open-World Continual Learning. (arXiv:2312.14990v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2312.14990
• Code URL: null
• Copy Paste: [[2312.14990]] Learning to Prompt Knowledge Transfer for Open-World Continual Learning(http://arxiv.org/abs/2312.14990)
• Summary:

  This paper studies the problem of continual learning in an open-world scenario, referred to as Open-world Continual Learning (OwCL). OwCL is increasingly rising while it is highly challenging in two-fold: i) learning a sequence of tasks without forgetting knowns in the past, and ii) identifying unknowns (novel objects/classes) in the future. Existing OwCL methods suffer from the adaptability of task-aware boundaries between knowns and unknowns, and do not consider the mechanism of knowledge transfer. In this work, we propose Pro-KT, a novel prompt-enhanced knowledge transfer model for OwCL. Pro-KT includes two key components: (1) a prompt bank to encode and transfer both task-generic and task-specific knowledge, and (2) a task-aware open-set boundary to identify unknowns in the new tasks. Experimental results using two real-world datasets demonstrate that the proposed Pro-KT outperforms the state-of-the-art counterparts in both the detection of unknowns and the classification of knowns markedly.

Title: Generative AI and the History of Architecture. (arXiv:2312.15106v1 [cs.AI])

• Paper URL: http://arxiv.org/abs/2312.15106
• Code URL: null
• Copy Paste: [[2312.15106]] Generative AI and the History of Architecture(http://arxiv.org/abs/2312.15106)
• Summary:

  Recent generative AI platforms are able to create texts or impressive images from simple text prompts. This makes them powerful tools for summarizing knowledge about architectural history or deriving new creative work in early design tasks like ideation, sketching and modelling. But, how good is the understanding of the generative AI models of the history of architecture? Has it learned to properly distinguish styles, or is it hallucinating information? In this chapter, we investigate this question for generative AI platforms for text and image generation for different architectural styles, to understand the capabilities and boundaries of knowledge of those tools. We also analyze how they are already being used by analyzing a data set of 101 million Midjourney queries to see if and how practitioners are already querying for specific architectural concepts.

code

Title: Diffusion Models for Generative Artificial Intelligence: An Introduction for Applied Mathematicians. (arXiv:2312.14977v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2312.14977
• Code URL: null
• Copy Paste: [[2312.14977]] Diffusion Models for Generative Artificial Intelligence: An Introduction for Applied Mathematicians(http://arxiv.org/abs/2312.14977)
• Summary:

  Generative artificial intelligence (AI) refers to algorithms that create synthetic but realistic output. Diffusion models currently offer state of the art performance in generative AI for images. They also form a key component in more general tools, including text-to-image generators and large language models. Diffusion models work by adding noise to the available training data and then learning how to reverse the process. The reverse operation may then be applied to new random data in order to produce new outputs. We provide a brief introduction to diffusion models for applied mathematicians and statisticians. Our key aims are (a) to present illustrative computational examples, (b) to give a careful derivation of the underlying mathematical formulas involved, and (c) to draw a connection with partial differential equation (PDE) diffusion models. We provide code for the computational experiments. We hope that this topic will be of interest to advanced undergraduate students and postgraduate students. Portions of the material may also provide useful motivational examples for those who teach courses in stochastic processes, inference, machine learning, PDEs or scientific computing.

Title: Unsupervised Auditory and Semantic Entrainment Models with Deep Neural Networks. (arXiv:2312.15098v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2312.15098
• Code URL: null
• Copy Paste: [[2312.15098]] Unsupervised Auditory and Semantic Entrainment Models with Deep Neural Networks(http://arxiv.org/abs/2312.15098)
• Summary:

  Speakers tend to engage in adaptive behavior, known as entrainment, when they become similar to their interlocutor in various aspects of speaking. We present an unsupervised deep learning framework that derives meaningful representation from textual features for developing semantic entrainment. We investigate the model's performance by extracting features using different variations of the BERT model (DistilBERT and XLM-RoBERTa) and Google's universal sentence encoder (USE) embeddings on two human-human (HH) corpora (The Fisher Corpus English Part 1, Columbia games corpus) and one human-machine (HM) corpus (Voice Assistant Conversation Corpus (VACC)). In addition to semantic features we also trained DNN-based models utilizing two auditory embeddings (TRIpLet Loss network (TRILL) vectors, Low-level descriptors (LLD) features) and two units of analysis (Inter pausal unit and Turn). The results show that semantic entrainment can be assessed with our model, that models can distinguish between HH and HM interactions and that the two units of analysis for extracting acoustic features provide comparable findings.

chat

retrieval augmented generation

retrieval-augmented generation

rag

Title: Bridging AI and Clinical Practice: Integrating Automated Sleep Scoring Algorithm with Uncertainty-Guided Physician Review. (arXiv:2312.14996v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2312.14996
• Code URL: null
• Copy Paste: [[2312.14996]] Bridging AI and Clinical Practice: Integrating Automated Sleep Scoring Algorithm with Uncertainty-Guided Physician Review(http://arxiv.org/abs/2312.14996)
• Summary:

  Purpose: This study aims to enhance the clinical use of automated sleep-scoring algorithms by incorporating an uncertainty estimation approach to efficiently assist clinicians in the manual review of predicted hypnograms, a necessity due to the notable inter-scorer variability inherent in polysomnography (PSG) databases. Our efforts target the extent of review required to achieve predefined agreement levels, examining both in-domain and out-of-domain data, and considering subjects diagnoses. Patients and methods: Total of 19578 PSGs from 13 open-access databases were used to train U-Sleep, a state-of-the-art sleep-scoring algorithm. We leveraged a comprehensive clinical database of additional 8832 PSGs, covering a full spectrum of ages and sleep-disorders, to refine the U-Sleep, and to evaluate different uncertainty-quantification approaches, including our novel confidence network. The ID data consisted of PSGs scored by over 50 physicians, and the two OOD sets comprised recordings each scored by a unique senior physician. Results: U-Sleep demonstrated robust performance, with Cohen's kappa (K) at 76.2% on ID and 73.8-78.8% on OOD data. The confidence network excelled at identifying uncertain predictions, achieving AUROC scores of 85.7% on ID and 82.5-85.6% on OOD data. Independently of sleep-disorder status, statistical evaluations revealed significant differences in confidence scores between aligning vs discording predictions, and significant correlations of confidence scores with classification performance metrics. To achieve K of at least 90% with physician intervention, examining less than 29.0% of uncertain epochs was required, substantially reducing physicians workload, and facilitating near-perfect agreement.

Title: SODA: Protecting Proprietary Information in On-Device Machine Learning Models. (arXiv:2312.15036v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2312.15036
• Code URL: null
• Copy Paste: [[2312.15036]] SODA: Protecting Proprietary Information in On-Device Machine Learning Models(http://arxiv.org/abs/2312.15036)
• Summary:

  The growth of low-end hardware has led to a proliferation of machine learning-based services in edge applications. These applications gather contextual information about users and provide some services, such as personalized offers, through a machine learning (ML) model. A growing practice has been to deploy such ML models on the user's device to reduce latency, maintain user privacy, and minimize continuous reliance on a centralized source. However, deploying ML models on the user's edge device can leak proprietary information about the service provider. In this work, we investigate on-device ML models that are used to provide mobile services and demonstrate how simple attacks can leak proprietary information of the service provider. We show that different adversaries can easily exploit such models to maximize their profit and accomplish content theft. Motivated by the need to thwart such attacks, we present an end-to-end framework, SODA, for deploying and serving on edge devices while defending against adversarial usage. Our results demonstrate that SODA can detect adversarial usage with 89% accuracy in less than 50 queries with minimal impact on service performance, latency, and storage.

Title: Learning Rich Rankings. (arXiv:2312.15081v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2312.15081
• Code URL: https://github.com/arjunsesh/lrr-neurips
• Copy Paste: [[2312.15081]] Learning Rich Rankings(http://arxiv.org/abs/2312.15081)
• Summary:

  Although the foundations of ranking are well established, the ranking literature has primarily been focused on simple, unimodal models, e.g. the Mallows and Plackett-Luce models, that define distributions centered around a single total ordering. Explicit mixture models have provided some tools for modelling multimodal ranking data, though learning such models from data is often difficult. In this work, we contribute a contextual repeated selection (CRS) model that leverages recent advances in choice modeling to bring a natural multimodality and richness to the rankings space. We provide rigorous theoretical guarantees for maximum likelihood estimation under the model through structure-dependent tail risk and expected risk bounds. As a by-product, we also furnish the first tight bounds on the expected risk of maximum likelihood estimators for the multinomial logit (MNL) choice model and the Plackett-Luce (PL) ranking model, as well as the first tail risk bound on the PL ranking model. The CRS model significantly outperforms existing methods for modeling real world ranking data in a variety of settings, from racing to rank choice voting.

Title: A Note on Stability in Asynchronous Stochastic Approximation without Communication Delays. (arXiv:2312.15091v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2312.15091
• Code URL: null
• Copy Paste: [[2312.15091]] A Note on Stability in Asynchronous Stochastic Approximation without Communication Delays(http://arxiv.org/abs/2312.15091)
• Summary:

  In this paper, we study asynchronous stochastic approximation algorithms without communication delays. Our main contribution is a stability proof for these algorithms that extends a method of Borkar and Meyn by accommodating more general noise conditions. We also derive convergence results from this stability result and discuss their application in important average-reward reinforcement learning problems.

Title: Less or More From Teacher: Exploiting Trilateral Geometry For Knowledge Distillation. (arXiv:2312.15112v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2312.15112
• Code URL: null
• Copy Paste: [[2312.15112]] Less or More From Teacher: Exploiting Trilateral Geometry For Knowledge Distillation(http://arxiv.org/abs/2312.15112)
• Summary:

  Knowledge distillation aims to train a compact student network using soft supervision from a larger teacher network and hard supervision from ground truths. However, determining an optimal knowledge fusion ratio that balances these supervisory signals remains challenging. Prior methods generally resort to a constant or heuristic-based fusion ratio, which often falls short of a proper balance. In this study, we introduce a novel adaptive method for learning a sample-wise knowledge fusion ratio, exploiting both the correctness of teacher and student, as well as how well the student mimics the teacher on each sample. Our method naturally leads to the intra-sample trilateral geometric relations among the student prediction ($S$), teacher prediction ($T$), and ground truth ($G$). To counterbalance the impact of outliers, we further extend to the inter-sample relations, incorporating the teacher's global average prediction $\bar{T}$ for samples within the same class. A simple neural network then learns the implicit mapping from the intra- and inter-sample relations to an adaptive, sample-wise knowledge fusion ratio in a bilevel-optimization manner. Our approach provides a simple, practical, and adaptable solution for knowledge distillation that can be employed across various architectures and model sizes. Extensive experiments demonstrate consistent improvements over other loss re-weighting methods on image classification, attack detection, and click-through rate prediction.

Title: Improving the Performance of Echo State Networks Through Feedback. (arXiv:2312.15141v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2312.15141
• Code URL: null
• Copy Paste: [[2312.15141]] Improving the Performance of Echo State Networks Through Feedback(http://arxiv.org/abs/2312.15141)
• Summary:

  Reservoir computing, using nonlinear dynamical systems, offers a cost-effective alternative to neural networks for complex tasks involving processing of sequential data, time series modeling, and system identification. Echo state networks (ESNs), a type of reservoir computer, mirror neural networks but simplify training. They apply fixed, random linear transformations to the internal state, followed by nonlinear changes. This process, guided by input signals and linear regression, adapts the system to match target characteristics, reducing computational demands. A potential drawback of ESNs is that the fixed reservoir may not offer the complexity needed for specific problems. While directly altering (training) the internal ESN would reintroduce the computational burden, an indirect modification can be achieved by redirecting some output as input. This feedback can influence the internal reservoir state, yielding ESNs with enhanced complexity suitable for broader challenges. In this paper, we demonstrate that by feeding some component of the reservoir state back into the network through the input, we can drastically improve upon the performance of a given ESN. We rigorously prove that, for any given ESN, feedback will almost always improve the accuracy of the output. For a set of three tasks, each representing different problem classes, we find that with feedback the average error measures are reduced by $30\%-60\%$. Remarkably, feedback provides at least an equivalent performance boost to doubling the initial number of computational nodes, a computationally expensive and technologically challenging alternative. These results demonstrate the broad applicability and substantial usefulness of this feedback scheme.

multi-run

chain-of-thought

tree-of-thought

agent

Title: Scaling Is All You Need: Training Strong Policies for Autonomous Driving with JAX-Accelerated Reinforcement Learning. (arXiv:2312.15122v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2312.15122
• Code URL: null
• Copy Paste: [[2312.15122]] Scaling Is All You Need: Training Strong Policies for Autonomous Driving with JAX-Accelerated Reinforcement Learning(http://arxiv.org/abs/2312.15122)
• Summary:

  Reinforcement learning has been used to train policies that outperform even the best human players in various games. However, a large amount of data is needed to achieve good performance, which in turn requires building large-scale frameworks and simulators. In this paper, we study how large-scale reinforcement learning can be applied to autonomous driving, analyze how the resulting policies perform as the experiment size is scaled, and what the most important factors contributing to policy performance are. To do this, we first introduce a hardware-accelerated autonomous driving simulator, which allows us to efficiently collect experience from billions of agent steps. This simulator is paired with a large-scale, multi-GPU reinforcement learning framework. We demonstrate that simultaneous scaling of dataset size, model size, and agent steps trained provides increasingly strong driving policies in regard to collision, traffic rule violations, and progress. In particular, our best policy reduces the failure rate by 57% while improving progress by 23% compared to the current state-of-the-art machine learning policies for autonomous driving.

Title: Federated Q-Learning: Linear Regret Speedup with Low Communication Cost. (arXiv:2312.15023v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2312.15023
• Code URL: null
• Copy Paste: [[2312.15023]] Federated Q-Learning: Linear Regret Speedup with Low Communication Cost(http://arxiv.org/abs/2312.15023)
• Summary:

  In this paper, we consider federated reinforcement learning for tabular episodic Markov Decision Processes (MDP) where, under the coordination of a central server, multiple agents collaboratively explore the environment and learn an optimal policy without sharing their raw data. While linear speedup in the number of agents has been achieved for some metrics, such as convergence rate and sample complexity, in similar settings, it is unclear whether it is possible to design a model-free algorithm to achieve linear regret speedup with low communication cost. We propose two federated Q-Learning algorithms termed as FedQ-Hoeffding and FedQ-Bernstein, respectively, and show that the corresponding total regrets achieve a linear speedup compared with their single-agent counterparts when the time horizon is sufficiently large, while the communication cost scales logarithmically in the total number of time steps $T$. Those results rely on an event-triggered synchronization mechanism between the agents and the server, a novel step size selection when the server aggregates the local estimates of the state-action values to form the global estimates, and a set of new concentration inequalities to bound the sum of non-martingale differences. This is the first work showing that linear regret speedup and logarithmic communication cost can be achieved by model-free algorithms in federated reinforcement learning.