2024-11-13

Title: Learning From Graph-Structured Data: Addressing Design Issues and Exploring Practical Applications in Graph Representation Learning

• Authors: Chenqing Hua
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.07269
• Pdf URL: https://arxiv.org/pdf/2411.07269
• Copy Paste: [[2411.07269]] Learning From Graph-Structured Data: Addressing Design Issues and Exploring Practical Applications in Graph Representation Learning(https://arxiv.org/abs/2411.07269)
• Keywords: generative
• Abstract: Graphs serve as fundamental descriptors for systems composed of interacting elements, capturing a wide array of data types, from molecular interactions to social networks and knowledge graphs. In this paper, we present an exhaustive review of the latest advancements in graph representation learning and Graph Neural Networks (GNNs). GNNs, tailored to handle graph-structured data, excel in deriving insights and predictions from intricate relational information, making them invaluable for tasks involving such data. Graph representation learning, a pivotal approach in analyzing graph-structured data, facilitates numerous downstream tasks and applications across machine learning, data mining, biomedicine, and healthcare. Our work delves into the capabilities of GNNs, examining their foundational designs and their application in addressing real-world challenges. We introduce a GNN equipped with an advanced high-order pooling function, adept at capturing complex node interactions within graph-structured data. This pooling function significantly enhances the GNN's efficacy in both node- and graph-level tasks. Additionally, we propose a molecular graph generative model with a GNN as its core framework. This GNN backbone is proficient in learning invariant and equivariant molecular characteristics. Employing these features, the molecular graph generative model is capable of simultaneously learning and generating molecular graphs with atom-bond structures and precise atom positions. Our models undergo thorough experimental evaluations and comparisons with established methods, showcasing their superior performance in addressing diverse real-world challenges with various datasets.

Title: Anomaly Detection in OKTA Logs using Autoencoders

• Authors: Jericho Cain, Hayden Beadles, Karthik Venkatesan
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2411.07314
• Pdf URL: https://arxiv.org/pdf/2411.07314
• Copy Paste: [[2411.07314]] Anomaly Detection in OKTA Logs using Autoencoders(https://arxiv.org/abs/2411.07314)
• Keywords: anomaly
• Abstract: Okta logs are used today to detect cybersecurity events using various rule-based models with restricted look back periods. These functions have limitations, such as a limited retrospective analysis, a predefined rule set, and susceptibility to generating false positives. To address this, we adopt unsupervised techniques, specifically employing autoencoders. To properly use an autoencoder, we need to transform and simplify the complexity of the log data we receive from our users. This transformed and filtered data is then fed into the autoencoder, and the output is evaluated.

Title: Exploring Variational Autoencoders for Medical Image Generation: A Comprehensive Study

• Authors: Khadija Rais, Mohamed Amroune, Abdelmadjid Benmachiche, Mohamed Yassine Haouam
• Subjects: cs.LG, cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2411.07348
• Pdf URL: https://arxiv.org/pdf/2411.07348
• Copy Paste: [[2411.07348]] Exploring Variational Autoencoders for Medical Image Generation: A Comprehensive Study(https://arxiv.org/abs/2411.07348)
• Keywords: generative
• Abstract: Variational autoencoder (VAE) is one of the most common techniques in the field of medical image generation, where this architecture has shown advanced researchers in recent years and has developed into various architectures. VAE has advantages including improving datasets by adding samples in smaller datasets and in datasets with imbalanced classes, and this is how data augmentation works. This paper provides a comprehensive review of studies on VAE in medical imaging, with a special focus on their ability to create synthetic images close to real data so that they can be used for data augmentation. This study reviews important architectures and methods used to develop VAEs for medical images and provides a comparison with other generative models such as GANs on issues such as image quality, and low diversity of generated samples. We discuss recent developments and applications in several medical fields highlighting the ability of VAEs to improve segmentation and classification accuracy.

Title: Feature-Space Semantic Invariance: Enhanced OOD Detection for Open-Set Domain Generalization

• Authors: Haoliang Wang, Chen Zhao, Feng Chen
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.07392
• Pdf URL: https://arxiv.org/pdf/2411.07392
• Copy Paste: [[2411.07392]] Feature-Space Semantic Invariance: Enhanced OOD Detection for Open-Set Domain Generalization(https://arxiv.org/abs/2411.07392)
• Keywords: generative
• Abstract: Open-set domain generalization addresses a real-world challenge: training a model to generalize across unseen domains (domain generalization) while also detecting samples from unknown classes not encountered during training (open-set recognition). However, most existing approaches tackle these issues separately, limiting their practical applicability. To overcome this limitation, we propose a unified framework for open-set domain generalization by introducing Feature-space Semantic Invariance (FSI). FSI maintains semantic consistency across different domains within the feature space, enabling more accurate detection of OOD instances in unseen domains. Additionally, we adopt a generative model to produce synthetic data with novel domain styles or class labels, enhancing model robustness. Initial experiments show that our method improves AUROC by 9.1% to 18.9% on ColoredMNIST, while also significantly increasing in-distribution classification accuracy.

Title: Using Generative AI and Multi-Agents to Provide Automatic Feedback

• Authors: Shuchen Guo, Ehsan Latif, Yifan Zhou, Xuan Huang, Xiaoming Zhai
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.07407
• Pdf URL: https://arxiv.org/pdf/2411.07407
• Copy Paste: [[2411.07407]] Using Generative AI and Multi-Agents to Provide Automatic Feedback(https://arxiv.org/abs/2411.07407)
• Keywords: generative
• Abstract: This study investigates the use of generative AI and multi-agent systems to provide automatic feedback in educational contexts, particularly for student constructed responses in science assessments. The research addresses a key gap in the field by exploring how multi-agent systems, called AutoFeedback, can improve the quality of GenAI-generated feedback, overcoming known issues such as over-praise and over-inference that are common in single-agent large language models (LLMs). The study developed a multi-agent system consisting of two AI agents: one for generating feedback and another for validating and refining it. The system was tested on a dataset of 240 student responses, and its performance was compared to that of a single-agent LLM. Results showed that AutoFeedback significantly reduced the occurrence of over-praise and over-inference errors, providing more accurate and pedagogically sound feedback. The findings suggest that multi-agent systems can offer a more reliable solution for generating automated feedback in educational settings, highlighting their potential for scalable and personalized learning support. These results have important implications for educators and researchers seeking to leverage AI in formative assessments, offering a pathway to more effective feedback mechanisms that enhance student learning outcomes.

Title: XPoint: A Self-Supervised Visual-State-Space based Architecture for Multispectral Image Registration

• Authors: Ismail Can Yagmur, Hasan F. Ates, Bahadir K. Gunturk
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2411.07430
• Pdf URL: https://arxiv.org/pdf/2411.07430
• Copy Paste: [[2411.07430]] XPoint: A Self-Supervised Visual-State-Space based Architecture for Multispectral Image Registration(https://arxiv.org/abs/2411.07430)
• Keywords: self-supervised
• Abstract: Accurate multispectral image matching presents significant challenges due to non-linear intensity variations across spectral modalities, extreme viewpoint changes, and the scarcity of labeled datasets. Current state-of-the-art methods are typically specialized for a single spectral difference, such as visibleinfrared, and struggle to adapt to other modalities due to their reliance on expensive supervision, such as depth maps or camera poses. To address the need for rapid adaptation across modalities, we introduce XPoint, a self-supervised, modular image-matching framework designed for adaptive training and fine-tuning on aligned multispectral datasets, allowing users to customize key components based on their specific tasks. XPoint employs modularity and self-supervision to allow for the adjustment of elements such as the base detector, which generates pseudoground truth keypoints invariant to viewpoint and spectrum variations. The framework integrates a VMamba encoder, pretrained on segmentation tasks, for robust feature extraction, and includes three joint decoder heads: two are dedicated to interest point and descriptor extraction; and a task-specific homography regression head imposes geometric constraints for superior performance in tasks like image registration. This flexible architecture enables quick adaptation to a wide range of modalities, demonstrated by training on Optical-Thermal data and fine-tuning on settings such as visual-near infrared, visual-infrared, visual-longwave infrared, and visual-synthetic aperture radar. Experimental results show that XPoint consistently outperforms or matches state-ofthe-art methods in feature matching and image registration tasks across five distinct multispectral datasets. Our source code is available at this https URL.

Title: All-in-one Weather-degraded Image Restoration via Adaptive Degradation-aware Self-prompting Model

• Authors: Yuanbo Wen, Tao Gao, Ziqi Li, Jing Zhang, Kaihao Zhang, Ting Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2411.07445
• Pdf URL: https://arxiv.org/pdf/2411.07445
• Copy Paste: [[2411.07445]] All-in-one Weather-degraded Image Restoration via Adaptive Degradation-aware Self-prompting Model(https://arxiv.org/abs/2411.07445)
• Keywords: diffusion
• Abstract: Existing approaches for all-in-one weather-degraded image restoration suffer from inefficiencies in leveraging degradation-aware priors, resulting in sub-optimal performance in adapting to different weather conditions. To this end, we develop an adaptive degradation-aware self-prompting model (ADSM) for all-in-one weather-degraded image restoration. Specifically, our model employs the contrastive language-image pre-training model (CLIP) to facilitate the training of our proposed latent prompt generators (LPGs), which represent three types of latent prompts to characterize the degradation type, degradation property and image caption. Moreover, we integrate the acquired degradation-aware prompts into the time embedding of diffusion model to improve degradation perception. Meanwhile, we employ the latent caption prompt to guide the reverse sampling process using the cross-attention mechanism, thereby guiding the accurate image reconstruction. Furthermore, to accelerate the reverse sampling procedure of diffusion model and address the limitations of frequency perception, we introduce a wavelet-oriented noise estimating network (WNE-Net). Extensive experiments conducted on eight publicly available datasets demonstrate the effectiveness of our proposed approach in both task-specific and all-in-one applications.

Title: Tracing the Roots: Leveraging Temporal Dynamics in Diffusion Trajectories for Origin Attribution

• Authors: Andreas Floros, Seyed-Mohsen Moosavi-Dezfooli, Pier Luigi Dragotti
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2411.07449
• Pdf URL: https://arxiv.org/pdf/2411.07449
• Copy Paste: [[2411.07449]] Tracing the Roots: Leveraging Temporal Dynamics in Diffusion Trajectories for Origin Attribution(https://arxiv.org/abs/2411.07449)
• Keywords: diffusion
• Abstract: Diffusion models have revolutionized image synthesis, garnering significant research interest in recent years. Diffusion is an iterative algorithm in which samples are generated step-by-step, starting from pure noise. This process introduces the notion of diffusion trajectories, i.e., paths from the standard Gaussian distribution to the target image distribution. In this context, we study discriminative algorithms operating on these trajectories. Specifically, given a pre-trained diffusion model, we consider the problem of classifying images as part of the training dataset, generated by the model or originating from an external source. Our approach demonstrates the presence of patterns across steps that can be leveraged for classification. We also conduct ablation studies, which reveal that using higher-order gradient features to characterize the trajectories leads to significant performance gains and more robust algorithms.

Title: MureObjectStitch: Multi-reference Image Composition

• Authors: Jiaxuan Chen, Bo Zhang, Li Niu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2411.07462
• Pdf URL: https://arxiv.org/pdf/2411.07462
• Copy Paste: [[2411.07462]] MureObjectStitch: Multi-reference Image Composition(https://arxiv.org/abs/2411.07462)
• Keywords: generative
• Abstract: Generative image composition aims to regenerate the given foreground object in the background image to produce a realistic composite image. In this work, we propose an effective finetuning strategy for generative image composition model, in which we finetune a pretrained model using one or more images containing the same foreground object. Moreover, we propose a multi-reference strategy, which allows the model to take in multiple reference images of the foreground object. The experiments on MureCOM dataset verify the effectiveness of our method.

Title: MSEG-VCUQ: Multimodal SEGmentation with Enhanced Vision Foundation Models, Convolutional Neural Networks, and Uncertainty Quantification for High-Speed Video Phase Detection Data

• Authors: Chika Maduabuchi, Ericmoore Jossou, Matteo Bucci
• Subjects: cs.CV, cs.LG, eess.IV
• Abstract URL: https://arxiv.org/abs/2411.07463
• Pdf URL: https://arxiv.org/pdf/2411.07463
• Copy Paste: [[2411.07463]] MSEG-VCUQ: Multimodal SEGmentation with Enhanced Vision Foundation Models, Convolutional Neural Networks, and Uncertainty Quantification for High-Speed Video Phase Detection Data(https://arxiv.org/abs/2411.07463)
• Keywords: foundation model
• Abstract: Purpose: High-speed video (HSV) phase detection (PD) segmentation is vital in nuclear reactors, chemical processing, and electronics cooling for detecting vapor, liquid, and microlayer phases. Traditional segmentation models face pixel-level accuracy and generalization issues in multimodal data. MSEG-VCUQ introduces VideoSAM, a hybrid framework leveraging convolutional neural networks (CNNs) and transformer-based vision models to enhance segmentation accuracy and generalizability across complex multimodal PD tasks. Methods: VideoSAM combines U-Net CNN and the Segment Anything Model (SAM) for advanced feature extraction and segmentation across diverse HSV PD modalities, spanning fluids like water, FC-72, nitrogen, and argon under varied heat flux conditions. The framework also incorporates uncertainty quantification (UQ) to assess pixel-based discretization errors, delivering reliable metrics such as contact line density and dry area fraction under experimental conditions. Results: VideoSAM outperforms SAM and modality-specific CNN models in segmentation accuracy, excelling in environments with complex phase boundaries, overlapping bubbles, and dynamic liquid-vapor interactions. Its hybrid architecture supports cross-dataset generalization, adapting effectively to varying modalities. The UQ module provides accurate error estimates, enhancing the reliability of segmentation outputs for advanced HSV PD research. Conclusion: MSEG-VCUQ, via VideoSAM, offers a robust solution for HSV PD segmentation, addressing previous limitations with advanced deep learning and UQ techniques. The open-source datasets and tools introduced enable scalable, precise, and adaptable segmentation for multimodal PD datasets, supporting advancements in HSV analysis and autonomous experimentation.

Title: Semi-Truths: A Large-Scale Dataset of AI-Augmented Images for Evaluating Robustness of AI-Generated Image detectors

• Authors: Anisha Pal, Julia Kruk, Mansi Phute, Manognya Bhattaram, Diyi Yang, Duen Horng Chau, Judy Hoffman
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2411.07472
• Pdf URL: https://arxiv.org/pdf/2411.07472
• Copy Paste: [[2411.07472]] Semi-Truths: A Large-Scale Dataset of AI-Augmented Images for Evaluating Robustness of AI-Generated Image detectors(https://arxiv.org/abs/2411.07472)
• Keywords: diffusion
• Abstract: Text-to-image diffusion models have impactful applications in art, design, and entertainment, yet these technologies also pose significant risks by enabling the creation and dissemination of misinformation. Although recent advancements have produced AI-generated image detectors that claim robustness against various augmentations, their true effectiveness remains uncertain. Do these detectors reliably identify images with different levels of augmentation? Are they biased toward specific scenes or data distributions? To investigate, we introduce SEMI-TRUTHS, featuring 27,600 real images, 223,400 masks, and 1,472,700 AI-augmented images that feature targeted and localized perturbations produced using diverse augmentation techniques, diffusion models, and data distributions. Each augmented image is accompanied by metadata for standardized and targeted evaluation of detector robustness. Our findings suggest that state-of-the-art detectors exhibit varying sensitivities to the types and degrees of perturbations, data distributions, and augmentation methods used, offering new insights into their performance and limitations. The code for the augmentation and evaluation pipeline is available at this https URL.

Title: FM-TS: Flow Matching for Time Series Generation

• Authors: Yang Hu, Xiao Wang, Lirong Wu, Huatian Zhang, Stan Z. Li, Sheng Wang, Tianlong Chen
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.07506
• Pdf URL: https://arxiv.org/pdf/2411.07506
• Copy Paste: [[2411.07506]] FM-TS: Flow Matching for Time Series Generation(https://arxiv.org/abs/2411.07506)
• Keywords: diffusion
• Abstract: Time series generation has emerged as an essential tool for analyzing temporal data across numerous fields. While diffusion models have recently gained significant attention in generating high-quality time series, they tend to be computationally demanding and reliant on complex stochastic processes. To address these limitations, we introduce FM-TS, a rectified Flow Matching-based framework for Time Series generation, which simplifies the time series generation process by directly optimizing continuous trajectories. This approach avoids the need for iterative sampling or complex noise schedules typically required in diffusion-based models. FM-TS is more efficient in terms of training and inference. Moreover, FM-TS is highly adaptive, supporting both conditional and unconditional time series generation. Notably, through our novel inference design, the model trained in an unconditional setting can seamlessly generalize to conditional tasks without the need for retraining. Extensive benchmarking across both settings demonstrates that FM-TS consistently delivers superior performance compared to existing approaches while being more efficient in terms of training and inference. For instance, in terms of discriminative score, FM-TS achieves 0.005, 0.019, 0.011, 0.005, 0.053, and 0.106 on the Sines, Stocks, ETTh, MuJoCo, Energy, and fMRI unconditional time series datasets, respectively, significantly outperforming the second-best method which achieves 0.006, 0.067, 0.061, 0.008, 0.122, and 0.167 on the same datasets. We have achieved superior performance in solar forecasting and MuJoCo imputation tasks, significantly enhanced by our innovative $t$ power sampling method. The code is available at this https URL.

Title: Contrastive Language Prompting to Ease False Positives in Medical Anomaly Detection

• Authors: YeongHyeon Park, Myung Jin Kim, Hyeong Seok Kim
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2411.07546
• Pdf URL: https://arxiv.org/pdf/2411.07546
• Copy Paste: [[2411.07546]] Contrastive Language Prompting to Ease False Positives in Medical Anomaly Detection(https://arxiv.org/abs/2411.07546)
• Keywords: anomaly
• Abstract: A pre-trained visual-language model, contrastive language-image pre-training (CLIP), successfully accomplishes various downstream tasks with text prompts, such as finding images or localizing regions within the image. Despite CLIP's strong multi-modal data capabilities, it remains limited in specialized environments, such as medical applications. For this purpose, many CLIP variants-i.e., BioMedCLIP, and MedCLIP-SAMv2-have emerged, but false positives related to normal regions persist. Thus, we aim to present a simple yet important goal of reducing false positives in medical anomaly detection. We introduce a Contrastive LAnguage Prompting (CLAP) method that leverages both positive and negative text prompts. This straightforward approach identifies potential lesion regions by visual attention to the positive prompts in the given image. To reduce false positives, we attenuate attention on normal regions using negative prompts. Extensive experiments with the BMAD dataset, including six biomedical benchmarks, demonstrate that CLAP method enhances anomaly detection performance. Our future plans include developing an automated fine prompting method for more practical usage.

Title: Disentangling Tabular Data towards Better One-Class Anomaly Detection

• Authors: Jianan Ye, Zhaorui Tan, Yijie Hu, Xi Yang, Guangliang Cheng, Kaizhu Huang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.07574
• Pdf URL: https://arxiv.org/pdf/2411.07574
• Copy Paste: [[2411.07574]] Disentangling Tabular Data towards Better One-Class Anomaly Detection(https://arxiv.org/abs/2411.07574)
• Keywords: anomaly
• Abstract: Tabular anomaly detection under the one-class classification setting poses a significant challenge, as it involves accurately conceptualizing "normal" derived exclusively from a single category to discern anomalies from normal data variations. Capturing the intrinsic correlation among attributes within normal samples presents one promising method for learning the concept. To do so, the most recent effort relies on a learnable mask strategy with a reconstruction task. However, this wisdom may suffer from the risk of producing uniform masks, i.e., essentially nothing is masked, leading to less effective correlation learning. To address this issue, we presume that attributes related to others in normal samples can be divided into two non-overlapping and correlated subsets, defined as CorrSets, to capture the intrinsic correlation effectively. Accordingly, we introduce an innovative method that disentangles CorrSets from normal tabular data. To our knowledge, this is a pioneering effort to apply the concept of disentanglement for one-class anomaly detection on tabular data. Extensive experiments on 20 tabular datasets show that our method substantially outperforms the state-of-the-art methods and leads to an average performance improvement of 6.1% on AUC-PR and 2.1% on AUC-ROC.

Title: Artificial Intelligence for Biomedical Video Generation

• Authors: Linyuan Li, Jianing Qiu, Anujit Saha, Lin Li, Poyuan Li, Mengxian He, Ziyu Guo, Wu Yuan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2411.07619
• Pdf URL: https://arxiv.org/pdf/2411.07619
• Copy Paste: [[2411.07619]] Artificial Intelligence for Biomedical Video Generation(https://arxiv.org/abs/2411.07619)
• Keywords: generative
• Abstract: As a prominent subfield of Artificial Intelligence Generated Content (AIGC), video generation has achieved notable advancements in recent years. The introduction of Sora-alike models represents a pivotal breakthrough in video generation technologies, significantly enhancing the quality of synthesized videos. Particularly in the realm of biomedicine, video generation technology has shown immense potential such as medical concept explanation, disease simulation, and biomedical data augmentation. In this article, we thoroughly examine the latest developments in video generation models and explore their applications, challenges, and future opportunities in the biomedical sector. We have conducted an extensive review and compiled a comprehensive list of datasets from various sources to facilitate the development and evaluation of video generative models in biomedicine. Given the rapid progress in this field, we have also created a github repository to regularly update the advances of biomedical video generation at: this https URL

Title: Unraveling the Connections between Flow Matching and Diffusion Probabilistic Models in Training-free Conditional Generation

• Authors: Kaiyu Song, Hanjiang Lai
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2411.07625
• Pdf URL: https://arxiv.org/pdf/2411.07625
• Copy Paste: [[2411.07625]] Unraveling the Connections between Flow Matching and Diffusion Probabilistic Models in Training-free Conditional Generation(https://arxiv.org/abs/2411.07625)
• Keywords: diffusion
• Abstract: Training-free conditional generation aims to leverage the unconditional diffusion models to implement the conditional generation, where flow-matching (FM) and diffusion probabilistic models (DPMs) are two mature unconditional diffusion models that achieve high-quality generation. Two questions were asked in this paper: What are the underlying connections between FM and DPMs in training-free conditional generation? Can we leverage DPMs to improve the training-free conditional generation for FM? We first show that a probabilistic diffusion path can be associated with the FM and DPMs. Then, we reformulate the ordinary differential equation (ODE) of FM based on the score function of DPMs, and thus, the conditions in FM can be incorporated as those in DPMs. Finally, we propose two posterior sampling methods to estimate the conditional term and achieve a training-free conditional generation of FM. Experimental results show that our proposed method could be implemented for various conditional generation tasks. Our method can generate higher-quality results than the state-of-the-art methods.

Title: Leveraging Previous Steps: A Training-free Fast Solver for Flow Diffusion

• Authors: Kaiyu Song, Hanjiang Lai
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2411.07627
• Pdf URL: https://arxiv.org/pdf/2411.07627
• Copy Paste: [[2411.07627]] Leveraging Previous Steps: A Training-free Fast Solver for Flow Diffusion(https://arxiv.org/abs/2411.07627)
• Keywords: diffusion
• Abstract: Flow diffusion models (FDMs) have recently shown potential in generation tasks due to the high generation quality. However, the current ordinary differential equation (ODE) solver for FDMs, e.g., the Euler solver, still suffers from slow generation since ODE solvers need many number function evaluations (NFE) to keep high-quality generation. In this paper, we propose a novel training-free flow-solver to reduce NFE while maintaining high-quality generation. The key insight for the flow-solver is to leverage the previous steps to reduce the NFE, where a cache is created to reuse these results from the previous steps. Specifically, the Taylor expansion is first used to approximate the ODE. To calculate the high-order derivatives of Taylor expansion, the flow-solver proposes to use the previous steps and a polynomial interpolation to approximate it, where the number of orders we could approximate equals the number of previous steps we cached. We also prove that the flow-solver has a more minor approximation error and faster generation speed. Experimental results on the CIFAR-10, CelebA-HQ, LSUN-Bedroom, LSUN-Church, ImageNet, and real text-to-image generation prove the efficiency of the flow-solver. Specifically, the flow-solver improves the FID-30K from 13.79 to 6.75, from 46.64 to 19.49 with $\text{NFE}=10$ on CIFAR-10 and LSUN-Church, respectively.

Title: Evaluating the Generation of Spatial Relations in Text and Image Generative Models

• Authors: Shang Hong Sim, Clarence Lee, Alvin Tan, Cheston Tan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2411.07664
• Pdf URL: https://arxiv.org/pdf/2411.07664
• Copy Paste: [[2411.07664]] Evaluating the Generation of Spatial Relations in Text and Image Generative Models(https://arxiv.org/abs/2411.07664)
• Keywords: generative
• Abstract: Understanding spatial relations is a crucial cognitive ability for both humans and AI. While current research has predominantly focused on the benchmarking of text-to-image (T2I) models, we propose a more comprehensive evaluation that includes \textit{both} T2I and Large Language Models (LLMs). As spatial relations are naturally understood in a visuo-spatial manner, we develop an approach to convert LLM outputs into an image, thereby allowing us to evaluate both T2I models and LLMs \textit{visually}. We examined the spatial relation understanding of 8 prominent generative models (3 T2I models and 5 LLMs) on a set of 10 common prepositions, as well as assess the feasibility of automatic evaluation methods. Surprisingly, we found that T2I models only achieve subpar performance despite their impressive general image-generation abilities. Even more surprisingly, our results show that LLMs are significantly more accurate than T2I models in generating spatial relations, despite being primarily trained on textual data. We examined reasons for model failures and highlight gaps that can be filled to enable more spatially faithful generations.

Title: Emotion Classification of Children Expressions

• Authors: Sanchayan Vivekananthan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2411.07708
• Pdf URL: https://arxiv.org/pdf/2411.07708
• Copy Paste: [[2411.07708]] Emotion Classification of Children Expressions(https://arxiv.org/abs/2411.07708)
• Keywords: diffusion
• Abstract: This paper proposes a process for a classification model for the facial expressions. The proposed process would aid in specific categorisation of children's emotions from 2 emotions namely 'Happy' and 'Sad'. Since the existing emotion recognition systems algorithms primarily train on adult faces, the model developed is achieved by using advanced concepts of models with Squeeze-andExcitation blocks, Convolutional Block Attention modules, and robust data augmentation. Stable Diffusion image synthesis was used for expanding and diversifying the data set generating realistic and various training samples. The model designed using Batch Normalisation, Dropout, and SE Attention mechanisms for the classification of children's emotions achieved an accuracy rate of 89\% due to these methods improving the precision of emotion recognition in children. The relative importance of this issue is raised in this study with an emphasis on the call for a more specific model in emotion detection systems for the young generation with specific direction on how the young people can be assisted to manage emotions while online.

Title: Spatially Regularized Graph Attention Autoencoder Framework for Detecting Rainfall Extremes

• Authors: Mihir Agarwal, Progyan Das, Udit Bhatia
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2411.07753
• Pdf URL: https://arxiv.org/pdf/2411.07753
• Copy Paste: [[2411.07753]] Spatially Regularized Graph Attention Autoencoder Framework for Detecting Rainfall Extremes(https://arxiv.org/abs/2411.07753)
• Keywords: anomaly
• Abstract: We introduce a novel Graph Attention Autoencoder (GAE) with spatial regularization to address the challenge of scalable anomaly detection in spatiotemporal rainfall data across India from 1990 to 2015. Our model leverages a Graph Attention Network (GAT) to capture spatial dependencies and temporal dynamics in the data, further enhanced by a spatial regularization term ensuring geographic coherence. We construct two graph datasets employing rainfall, pressure, and temperature attributes from the Indian Meteorological Department and ERA5 Reanalysis on Single Levels, respectively. Our network operates on graph representations of the data, where nodes represent geographic locations, and edges, inferred through event synchronization, denote significant co-occurrences of rainfall events. Through extensive experiments, we demonstrate that our GAE effectively identifies anomalous rainfall patterns across the Indian landscape. Our work paves the way for sophisticated spatiotemporal anomaly detection methodologies in climate science, contributing to better climate change preparedness and response strategies.

Title: Novel View Synthesis with Pixel-Space Diffusion Models

• Authors: Noam Elata, Bahjat Kawar, Yaron Ostrovsky-Berman, Miriam Farber, Ron Sokolovsky
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2411.07765
• Pdf URL: https://arxiv.org/pdf/2411.07765
• Copy Paste: [[2411.07765]] Novel View Synthesis with Pixel-Space Diffusion Models(https://arxiv.org/abs/2411.07765)
• Keywords: diffusion, generative
• Abstract: Synthesizing a novel view from a single input image is a challenging task. Traditionally, this task was approached by estimating scene depth, warping, and inpainting, with machine learning models enabling parts of the pipeline. More recently, generative models are being increasingly employed in novel view synthesis (NVS), often encompassing the entire end-to-end system. In this work, we adapt a modern diffusion model architecture for end-to-end NVS in the pixel space, substantially outperforming previous state-of-the-art (SOTA) techniques. We explore different ways to encode geometric information into the network. Our experiments show that while these methods may enhance performance, their impact is minor compared to utilizing improved generative models. Moreover, we introduce a novel NVS training scheme that utilizes single-view datasets, capitalizing on their relative abundance compared to their multi-view counterparts. This leads to improved generalization capabilities to scenes with out-of-domain content.

Title: Federated Low-Rank Adaptation with Differential Privacy over Wireless Networks

• Authors: Tianqu Kang, Zixin Wang, Hengtao He, Jun Zhang, Shenghui Song, Khaled B. Letaief
• Subjects: cs.LG, cs.CR, eess.SP
• Abstract URL: https://arxiv.org/abs/2411.07806
• Pdf URL: https://arxiv.org/pdf/2411.07806
• Copy Paste: [[2411.07806]] Federated Low-Rank Adaptation with Differential Privacy over Wireless Networks(https://arxiv.org/abs/2411.07806)
• Keywords: foundation model
• Abstract: Fine-tuning large pre-trained foundation models (FMs) on distributed edge devices presents considerable computational and privacy challenges. Federated fine-tuning (FedFT) mitigates some privacy issues by facilitating collaborative model training without the need to share raw data. To lessen the computational burden on resource-limited devices, combining low-rank adaptation (LoRA) with federated learning enables parameter-efficient fine-tuning. Additionally, the split FedFT architecture partitions an FM between edge devices and a central server, reducing the necessity for complete model deployment on individual devices. However, the risk of privacy eavesdropping attacks in FedFT remains a concern, particularly in sensitive areas such as healthcare and finance. In this paper, we propose a split FedFT framework with differential privacy (DP) over wireless networks, where the inherent wireless channel noise in the uplink transmission is utilized to achieve DP guarantees without adding an extra artificial noise. We shall investigate the impact of the wireless noise on convergence performance of the proposed framework. We will also show that by updating only one of the low-rank matrices in the split FedFT with DP, the proposed method can mitigate the noise amplification effect. Simulation results will demonstrate that the proposed framework achieves higher accuracy under strict privacy budgets compared to baseline methods.

Title: Tucano: Advancing Neural Text Generation for Portuguese

• Authors: Nicholas Kluge Corrêa, Aniket Sen, Sophia Falk, Shiza Fatimah
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2411.07854
• Pdf URL: https://arxiv.org/pdf/2411.07854
• Copy Paste: [[2411.07854]] Tucano: Advancing Neural Text Generation for Portuguese(https://arxiv.org/abs/2411.07854)
• Keywords: generative
• Abstract: Significant advances have been made in natural language processing in recent years. However, our current deep learning approach to language modeling requires substantial resources in terms of data and computation. One of the side effects of this data-hungry paradigm is the current schism between languages, separating those considered high-resource, where most of the development happens and resources are available, and the low-resource ones, which struggle to attain the same level of performance and autonomy. This study aims to introduce a new set of resources to stimulate the future development of neural text generation in Portuguese. In this work, we document the development of GigaVerbo, a concatenation of deduplicated Portuguese text corpora amounting to 200 billion tokens. Via this corpus, we trained a series of decoder-transformers named Tucano. Our models perform equal or superior to other Portuguese and multilingual language models of similar size in several Portuguese benchmarks. The evaluation of our models also reveals that model performance on many currently available benchmarks used by the Portuguese NLP community has little to no correlation with the scaling of token ingestion during training, highlighting the limitations of such evaluations when it comes to the assessment of Portuguese generative language models. All derivatives of our study are openly released on GitHub and Hugging Face. See this https URL

Title: Diverse capability and scaling of diffusion and auto-regressive models when learning abstract rules

• Authors: Binxu Wang, Jiaqi Shang, Haim Sompolinsky
• Subjects: cs.LG, cs.AI, cs.CV, cs.NE
• Abstract URL: https://arxiv.org/abs/2411.07873
• Pdf URL: https://arxiv.org/pdf/2411.07873
• Copy Paste: [[2411.07873]] Diverse capability and scaling of diffusion and auto-regressive models when learning abstract rules(https://arxiv.org/abs/2411.07873)
• Keywords: diffusion, generative
• Abstract: Humans excel at discovering regular structures from limited samples and applying inferred rules to novel settings. We investigate whether modern generative models can similarly learn underlying rules from finite samples and perform reasoning through conditional sampling. Inspired by Raven's Progressive Matrices task, we designed GenRAVEN dataset, where each sample consists of three rows, and one of 40 relational rules governing the object position, number, or attributes applies to all rows. We trained generative models to learn the data distribution, where samples are encoded as integer arrays to focus on rule learning. We compared two generative model families: diffusion (EDM, DiT, SiT) and autoregressive models (GPT2, Mamba). We evaluated their ability to generate structurally consistent samples and perform panel completion via unconditional and conditional sampling. We found diffusion models excel at unconditional generation, producing more novel and consistent samples from scratch and memorizing less, but performing less well in panel completion, even with advanced conditional sampling methods. Conversely, autoregressive models excel at completing missing panels in a rule-consistent manner but generate less consistent samples unconditionally. We observe diverse data scaling behaviors: for both model families, rule learning emerges at a certain dataset size - around 1000s examples per rule. With more training data, diffusion models improve both their unconditional and conditional generation capabilities. However, for autoregressive models, while panel completion improves with more training data, unconditional generation consistency declines. Our findings highlight complementary capabilities and limitations of diffusion and autoregressive models in rule learning and reasoning tasks, suggesting avenues for further research into their mechanisms and potential for human-like reasoning.

Title: JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation

• Authors: Yiyang Ma, Xingchao Liu, Xiaokang Chen, Wen Liu, Chengyue Wu, Zhiyu Wu, Zizheng Pan, Zhenda Xie, Haowei Zhang, Xingkai yu, Liang Zhao, Yisong Wang, Jiaying Liu, Chong Ruan
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2411.07975
• Pdf URL: https://arxiv.org/pdf/2411.07975
• Copy Paste: [[2411.07975]] JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation(https://arxiv.org/abs/2411.07975)
• Keywords: generative
• Abstract: We present JanusFlow, a powerful framework that unifies image understanding and generation in a single model. JanusFlow introduces a minimalist architecture that integrates autoregressive language models with rectified flow, a state-of-the-art method in generative modeling. Our key finding demonstrates that rectified flow can be straightforwardly trained within the large language model framework, eliminating the need for complex architectural modifications. To further improve the performance of our unified model, we adopt two key strategies: (i) decoupling the understanding and generation encoders, and (ii) aligning their representations during unified training. Extensive experiments show that JanusFlow achieves comparable or superior performance to specialized models in their respective domains, while significantly outperforming existing unified approaches across standard benchmarks. This work represents a step toward more efficient and versatile vision-language models.

Title: Wavelet Latent Diffusion (Wala): Billion-Parameter 3D Generative Model with Compact Wavelet Encodings

• Authors: Aditya Sanghi, Aliasghar Khani, Pradyumna Reddy, Arianna Rampini, Derek Cheung, Kamal Rahimi Malekshan, Kanika Madan, Hooman Shayani
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2411.08017
• Pdf URL: https://arxiv.org/pdf/2411.08017
• Copy Paste: [[2411.08017]] Wavelet Latent Diffusion (Wala): Billion-Parameter 3D Generative Model with Compact Wavelet Encodings(https://arxiv.org/abs/2411.08017)
• Keywords: diffusion, generative
• Abstract: Large-scale 3D generative models require substantial computational resources yet often fall short in capturing fine details and complex geometries at high resolutions. We attribute this limitation to the inefficiency of current representations, which lack the compactness required to model the generative models effectively. To address this, we introduce a novel approach called Wavelet Latent Diffusion, or WaLa, that encodes 3D shapes into wavelet-based, compact latent encodings. Specifically, we compress a $256^3$ signed distance field into a $12^3 \times 4$ latent grid, achieving an impressive 2427x compression ratio with minimal loss of detail. This high level of compression allows our method to efficiently train large-scale generative networks without increasing the inference time. Our models, both conditional and unconditional, contain approximately one billion parameters and successfully generate high-quality 3D shapes at $256^3$ resolution. Moreover, WaLa offers rapid inference, producing shapes within two to four seconds depending on the condition, despite the model's scale. We demonstrate state-of-the-art performance across multiple datasets, with significant improvements in generation quality, diversity, and computational efficiency. We open-source our code and, to the best of our knowledge, release the largest pretrained 3D generative models across different modalities.

Title: GaussianAnything: Interactive Point Cloud Latent Diffusion for 3D Generation

• Authors: Yushi Lan, Shangchen Zhou, Zhaoyang Lyu, Fangzhou Hong, Shuai Yang, Bo Dai, Xingang Pan, Chen Change Loy
• Subjects: cs.CV, cs.AI, cs.GR
• Abstract URL: https://arxiv.org/abs/2411.08033
• Pdf URL: https://arxiv.org/pdf/2411.08033
• Copy Paste: [[2411.08033]] GaussianAnything: Interactive Point Cloud Latent Diffusion for 3D Generation(https://arxiv.org/abs/2411.08033)
• Keywords: diffusion
• Abstract: While 3D content generation has advanced significantly, existing methods still face challenges with input formats, latent space design, and output representations. This paper introduces a novel 3D generation framework that addresses these challenges, offering scalable, high-quality 3D generation with an interactive Point Cloud-structured Latent space. Our framework employs a Variational Autoencoder (VAE) with multi-view posed RGB-D(epth)-N(ormal) renderings as input, using a unique latent space design that preserves 3D shape information, and incorporates a cascaded latent diffusion model for improved shape-texture disentanglement. The proposed method, GaussianAnything, supports multi-modal conditional 3D generation, allowing for point cloud, caption, and single/multi-view image inputs. Notably, the newly proposed latent space naturally enables geometry-texture disentanglement, thus allowing 3D-aware editing. Experimental results demonstrate the effectiveness of our approach on multiple datasets, outperforming existing methods in both text- and image-conditioned 3D generation.

Title: Scaling Properties of Diffusion Models for Perceptual Tasks

• Authors: Rahul Ravishankar, Zeeshan Patel, Jathushan Rajasegaran, Jitendra Malik
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.08034
• Pdf URL: https://arxiv.org/pdf/2411.08034
• Copy Paste: [[2411.08034]] Scaling Properties of Diffusion Models for Perceptual Tasks(https://arxiv.org/abs/2411.08034)
• Keywords: diffusion
• Abstract: In this paper, we argue that iterative computation with diffusion models offers a powerful paradigm for not only generation but also visual perception tasks. We unify tasks such as depth estimation, optical flow, and segmentation under image-to-image translation, and show how diffusion models benefit from scaling training and test-time compute for these perception tasks. Through a careful analysis of these scaling behaviors, we present various techniques to efficiently train diffusion models for visual perception tasks. Our models achieve improved or comparable performance to state-of-the-art methods using significantly less data and compute. To use our code and models, see this https URL .