2024-07-25

Title: PateGail: A Privacy-Preserving Mobility Trajectory Generator with Imitation Learning

• Authors: Huandong Wang, Changzheng Gao, Yuchen Wu, Depeng Jin, Lina Yao, Yong Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.16729
• Pdf URL: https://arxiv.org/pdf/2407.16729
• Copy Paste: [[2407.16729]] PateGail: A Privacy-Preserving Mobility Trajectory Generator with Imitation Learning(https://arxiv.org/abs/2407.16729)
• Keywords: generative
• Abstract: Generating human mobility trajectories is of great importance to solve the lack of large-scale trajectory data in numerous applications, which is caused by privacy concerns. However, existing mobility trajectory generation methods still require real-world human trajectories centrally collected as the training data, where there exists an inescapable risk of privacy leakage. To overcome this limitation, in this paper, we propose PateGail, a privacy-preserving imitation learning model to generate mobility trajectories, which utilizes the powerful generative adversary imitation learning model to simulate the decision-making process of humans. Further, in order to protect user privacy, we train this model collectively based on decentralized mobility data stored in user devices, where personal discriminators are trained locally to distinguish and reward the real and generated human trajectories. In the training process, only the generated trajectories and their rewards obtained based on personal discriminators are shared between the server and devices, whose privacy is further preserved by our proposed perturbation mechanisms with theoretical proof to satisfy differential privacy. Further, to better model the human decision-making process, we propose a novel aggregation mechanism of the rewards obtained from personal discriminators. We theoretically prove that under the reward obtained based on the aggregation mechanism, our proposed model maximizes the lower bound of the discounted total rewards of users. Extensive experiments show that the trajectories generated by our model are able to resemble real-world trajectories in terms of five key statistical metrics, outperforming state-of-the-art algorithms by over 48.03%. Furthermore, we demonstrate that the synthetic trajectories are able to efficiently support practical applications, including mobility prediction and location recommendation.

Title: VisMin: Visual Minimal-Change Understanding

• Authors: Rabiul Awal, Saba Ahmadi, Le Zhang, Aishwarya Agrawal
• Subjects: cs.CV, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2407.16772
• Pdf URL: https://arxiv.org/pdf/2407.16772
• Copy Paste: [[2407.16772]] VisMin: Visual Minimal-Change Understanding(https://arxiv.org/abs/2407.16772)
• Keywords: diffusion
• Abstract: Fine-grained understanding of objects, attributes, and relationships between objects is crucial for visual-language models (VLMs). Existing benchmarks primarily focus on evaluating VLMs' capability to distinguish between two very similar \textit{captions} given an image. In this paper, we introduce a new, challenging benchmark termed \textbf{Vis}ual \textbf{Min}imal-Change Understanding (VisMin), which requires models to predict the correct image-caption match given two images and two captions. The image pair and caption pair contain minimal changes, i.e., only one aspect changes at a time from among the following: \textit{object}, \textit{attribute}, \textit{count}, and \textit{spatial relation}. These changes test the models' understanding of objects, attributes (such as color, material, shape), counts, and spatial relationships between objects. We built an automatic framework using large language models and diffusion models, followed by a rigorous 4-step verification process by human annotators. Empirical experiments reveal that current VLMs exhibit notable deficiencies in understanding spatial relationships and counting abilities. We also generate a large-scale training dataset to finetune CLIP and Idefics2, showing significant improvements in fine-grained understanding across benchmarks and in CLIP's general image-text alignment. We release all resources, including the benchmark, training data, and finetuned model checkpoints, at \url{this https URL}.

Title: Distribution-Aware Robust Learning from Long-Tailed Data with Noisy Labels

• Authors: Jae Soon Baik, In Young Yoon, Kun Hoon Kim, Jun Won Choi
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2407.16802
• Pdf URL: https://arxiv.org/pdf/2407.16802
• Copy Paste: [[2407.16802]] Distribution-Aware Robust Learning from Long-Tailed Data with Noisy Labels(https://arxiv.org/abs/2407.16802)
• Keywords: self-supervised
• Abstract: Deep neural networks have demonstrated remarkable advancements in various fields using large, well-annotated datasets. However, real-world data often exhibit long-tailed distributions and label noise, significantly degrading generalization performance. Recent studies addressing these issues have focused on noisy sample selection methods that estimate the centroid of each class based on high-confidence samples within each target class. The performance of these methods is limited because they use only the training samples within each class for class centroid estimation, making the quality of centroids susceptible to long-tailed distributions and noisy labels. In this study, we present a robust training framework called Distribution-aware Sample Selection and Contrastive Learning (DaSC). Specifically, DaSC introduces a Distribution-aware Class Centroid Estimation (DaCC) to generate enhanced class centroids. DaCC performs weighted averaging of the features from all samples, with weights determined based on model predictions. Additionally, we propose a confidence-aware contrastive learning strategy to obtain balanced and robust representations. The training samples are categorized into high-confidence and low-confidence samples. Our method then applies Semi-supervised Balanced Contrastive Loss (SBCL) using high-confidence samples, leveraging reliable label information to mitigate class bias. For the low-confidence samples, our method computes Mixup-enhanced Instance Discrimination Loss (MIDL) to improve their representations in a self-supervised manner. Our experimental results on CIFAR and real-world noisy-label datasets demonstrate the superior performance of the proposed DaSC compared to previous approaches.

Title: SAR to Optical Image Translation with Color Supervised Diffusion Model

• Authors: Xinyu Bai, Feng Xu
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2407.16921
• Pdf URL: https://arxiv.org/pdf/2407.16921
• Copy Paste: [[2407.16921]] SAR to Optical Image Translation with Color Supervised Diffusion Model(https://arxiv.org/abs/2407.16921)
• Keywords: diffusion, generative
• Abstract: Synthetic Aperture Radar (SAR) offers all-weather, high-resolution imaging capabilities, but its complex imaging mechanism often poses challenges for interpretation. In response to these limitations, this paper introduces an innovative generative model designed to transform SAR images into more intelligible optical images, thereby enhancing the interpretability of SAR images. Specifically, our model backbone is based on the recent diffusion models, which have powerful generative capabilities. We employ SAR images as conditional guides in the sampling process and integrate color supervision to counteract color shift issues effectively. We conducted experiments on the SEN12 dataset and employed quantitative evaluations using peak signal-to-noise ratio, structural similarity, and fréchet inception distance. The results demonstrate that our model not only surpasses previous methods in quantitative assessments but also significantly enhances the visual quality of the generated images.

Title: Synthetic Trajectory Generation Through Convolutional Neural Networks

• Authors: Jesse Merhi, Erik Buchholz, Salil S. Kanhere
• Subjects: cs.CR, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2407.16938
• Pdf URL: https://arxiv.org/pdf/2407.16938
• Copy Paste: [[2407.16938]] Synthetic Trajectory Generation Through Convolutional Neural Networks(https://arxiv.org/abs/2407.16938)
• Keywords: generative
• Abstract: Location trajectories provide valuable insights for applications from urban planning to pandemic control. However, mobility data can also reveal sensitive information about individuals, such as political opinions, religious beliefs, or sexual orientations. Existing privacy-preserving approaches for publishing this data face a significant utility-privacy trade-off. Releasing synthetic trajectory data generated through deep learning offers a promising solution. Due to the trajectories' sequential nature, most existing models are based on recurrent neural networks (RNNs). However, research in generative adversarial networks (GANs) largely employs convolutional neural networks (CNNs) for image generation. This discrepancy raises the question of whether advances in computer vision can be applied to trajectory generation. In this work, we introduce a Reversible Trajectory-to-CNN Transformation (RTCT) that adapts trajectories into a format suitable for CNN-based models. We integrated this transformation with the well-known DCGAN in a proof-of-concept (PoC) and evaluated its performance against an RNN-based trajectory GAN using four metrics across two datasets. The PoC was superior in capturing spatial distributions compared to the RNN model but had difficulty replicating sequential and temporal properties. Although the PoC's utility is not sufficient for practical applications, the results demonstrate the transformation's potential to facilitate the use of CNNs for trajectory generation, opening up avenues for future research. To support continued research, all source code has been made available under an open-source license.

Title: GV-Rep: A Large-Scale Dataset for Genetic Variant Representation Learning

• Authors: Zehui Li, Vallijah Subasri, Guy-Bart Stan, Yiren Zhao, Bo Wang
• Subjects: cs.LG, q-bio.GN
• Abstract URL: https://arxiv.org/abs/2407.16940
• Pdf URL: https://arxiv.org/pdf/2407.16940
• Copy Paste: [[2407.16940]] GV-Rep: A Large-Scale Dataset for Genetic Variant Representation Learning(https://arxiv.org/abs/2407.16940)
• Keywords: foundation model
• Abstract: Genetic variants (GVs) are defined as differences in the DNA sequences among individuals and play a crucial role in diagnosing and treating genetic diseases. The rapid decrease in next generation sequencing cost has led to an exponential increase in patient-level GV data. This growth poses a challenge for clinicians who must efficiently prioritize patient-specific GVs and integrate them with existing genomic databases to inform patient management. To addressing the interpretation of GVs, genomic foundation models (GFMs) have emerged. However, these models lack standardized performance assessments, leading to considerable variability in model evaluations. This poses the question: How effectively do deep learning methods classify unknown GVs and align them with clinically-verified GVs? We argue that representation learning, which transforms raw data into meaningful feature spaces, is an effective approach for addressing both indexing and classification challenges. We introduce a large-scale Genetic Variant dataset, named GV-Rep, featuring variable-length contexts and detailed annotations, designed for deep learning models to learn GV representations across various traits, diseases, tissue types, and experimental contexts. Our contributions are three-fold: (i) Construction of a comprehensive dataset with 7 million records, each labeled with characteristics of the corresponding variants, alongside additional data from 17,548 gene knockout tests across 1,107 cell types, 1,808 variant combinations, and 156 unique clinically verified GVs from real-world patients. (ii) Analysis of the structure and properties of the dataset. (iii) Experimentation of the dataset with pre-trained GFMs. The results show a significant gap between GFMs current capabilities and accurate GV representation. We hope this dataset will help advance genomic deep learning to bridge this gap.

Title: McGAN: Generating Manufacturable Designs by Embedding Manufacturing Rules into Conditional Generative Adversarial Network

• Authors: Zhichao Wang, Xiaoliang Yan, Shreyes Melkote, David Rosen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2407.16943
• Pdf URL: https://arxiv.org/pdf/2407.16943
• Copy Paste: [[2407.16943]] McGAN: Generating Manufacturable Designs by Embedding Manufacturing Rules into Conditional Generative Adversarial Network(https://arxiv.org/abs/2407.16943)
• Keywords: generative
• Abstract: Generative design (GD) methods aim to automatically generate a wide variety of designs that satisfy functional or aesthetic design requirements. However, research to date generally lacks considerations of manufacturability of the generated designs. To this end, we propose a novel GD approach by using deep neural networks to encode design for manufacturing (DFM) rules, thereby modifying part designs to make them manufacturable by a given manufacturing process. Specifically, a three-step approach is proposed: first, an instance segmentation method, Mask R-CNN, is used to decompose a part design into subregions. Second, a conditional generative adversarial neural network (cGAN), Pix2Pix, transforms unmanufacturable decomposed subregions into manufacturable subregions. The transformed subregions of designs are subsequently reintegrated into a unified manufacturable design. These three steps, Mask-RCNN, Pix2Pix, and reintegration, form the basis of the proposed Manufacturable conditional GAN (McGAN) framework. Experimental results show that McGAN can transform existing unmanufacturable designs to generate their corresponding manufacturable counterparts automatically that realize the specified manufacturing rules in an efficient and robust manner. The effectiveness of McGAN is demonstrated through two-dimensional design case studies of an injection molding process.

Title: Affective Behaviour Analysis via Progressive Learning

• Authors: Chen Liu, Wei Zhang, Feng Qiu, Lincheng Li, Xin Yu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2407.16945
• Pdf URL: https://arxiv.org/pdf/2407.16945
• Copy Paste: [[2407.16945]] Affective Behaviour Analysis via Progressive Learning(https://arxiv.org/abs/2407.16945)
• Keywords: self-supervised
• Abstract: Affective Behavior Analysis aims to develop emotionally intelligent technology that can recognize and respond to human emotions. To advance this, the 7th Affective Behavior Analysis in-the-wild (ABAW) competition establishes two tracks: i.e., the Multi-task Learning (MTL) Challenge and the Compound Expression (CE) challenge based on Aff-Wild2 and C-EXPR-DB datasets. In this paper, we present our methods and experimental results for the two competition tracks. Specifically, it can be summarized in the following four aspects: 1) To attain high-quality facial features, we train a Masked-Auto Encoder in a self-supervised manner. 2) We devise a temporal convergence module to capture the temporal information between video frames and explore the impact of window size and sequence length on each sub-task. 3) To facilitate the joint optimization of various sub-tasks, we explore the impact of sub-task joint training and feature fusion from individual tasks on each task performance improvement. 4) We utilize curriculum learning to transition the model from recognizing single expressions to recognizing compound expressions, thereby improving the accuracy of compound expression recognition. Extensive experiments demonstrate the superiority of our designs.

Title: Diffree: Text-Guided Shape Free Object Inpainting with Diffusion Model

• Authors: Lirui Zhao, Tianshuo Yang, Wenqi Shao, Yuxin Zhang, Yu Qiao, Ping Luo, Kaipeng Zhang, Rongrong Ji
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.16982
• Pdf URL: https://arxiv.org/pdf/2407.16982
• Copy Paste: [[2407.16982]] Diffree: Text-Guided Shape Free Object Inpainting with Diffusion Model(https://arxiv.org/abs/2407.16982)
• Keywords: diffusion
• Abstract: This paper addresses an important problem of object addition for images with only text guidance. It is challenging because the new object must be integrated seamlessly into the image with consistent visual context, such as lighting, texture, and spatial location. While existing text-guided image inpainting methods can add objects, they either fail to preserve the background consistency or involve cumbersome human intervention in specifying bounding boxes or user-scribbled masks. To tackle this challenge, we introduce Diffree, a Text-to-Image (T2I) model that facilitates text-guided object addition with only text control. To this end, we curate OABench, an exquisite synthetic dataset by removing objects with advanced image inpainting techniques. OABench comprises 74K real-world tuples of an original image, an inpainted image with the object removed, an object mask, and object descriptions. Trained on OABench using the Stable Diffusion model with an additional mask prediction module, Diffree uniquely predicts the position of the new object and achieves object addition with guidance from only text. Extensive experiments demonstrate that Diffree excels in adding new objects with a high success rate while maintaining background consistency, spatial appropriateness, and object relevance and quality.

Title: DreamCar: Leveraging Car-specific Prior for in-the-wild 3D Car Reconstruction

• Authors: Xiaobiao Du, Haiyang Sun, Ming Lu, Tianqing Zhu, Xin Yu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2407.16988
• Pdf URL: https://arxiv.org/pdf/2407.16988
• Copy Paste: [[2407.16988]] DreamCar: Leveraging Car-specific Prior for in-the-wild 3D Car Reconstruction(https://arxiv.org/abs/2407.16988)
• Keywords: generative
• Abstract: Self-driving industries usually employ professional artists to build exquisite 3D cars. However, it is expensive to craft large-scale digital assets. Since there are already numerous datasets available that contain a vast number of images of cars, we focus on reconstructing high-quality 3D car models from these datasets. However, these datasets only contain one side of cars in the forward-moving scene. We try to use the existing generative models to provide more supervision information, but they struggle to generalize well in cars since they are trained on synthetic datasets not car-specific. In addition, The reconstructed 3D car texture misaligns due to a large error in camera pose estimation when dealing with in-the-wild images. These restrictions make it challenging for previous methods to reconstruct complete 3D cars. To address these problems, we propose a novel method, named DreamCar, which can reconstruct high-quality 3D cars given a few images even a single image. To generalize the generative model, we collect a car dataset, named Car360, with over 5,600 vehicles. With this dataset, we make the generative model more robust to cars. We use this generative prior specific to the car to guide its reconstruction via Score Distillation Sampling. To further complement the supervision information, we utilize the geometric and appearance symmetry of cars. Finally, we propose a pose optimization method that rectifies poses to tackle texture misalignment. Extensive experiments demonstrate that our method significantly outperforms existing methods in reconstructing high-quality 3D cars. \href{this https URL}{Our code is available.}

Title: Unveiling In-Context Learning: A Coordinate System to Understand Its Working Mechanism

• Authors: Anhao Zhao, Fanghua Ye, Jinlan Fu, Xiaoyu Shen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17011
• Pdf URL: https://arxiv.org/pdf/2407.17011
• Copy Paste: [[2407.17011]] Unveiling In-Context Learning: A Coordinate System to Understand Its Working Mechanism(https://arxiv.org/abs/2407.17011)
• Keywords: in-context
• Abstract: Large language models (LLMs) exhibit remarkable in-context learning (ICL) capabilities. However, the underlying working mechanism of ICL remains poorly understood. Recent research presents two conflicting views on ICL: One attributes it to LLMs' inherent ability of task recognition, deeming label correctness and shot numbers of demonstrations as not crucial; the other emphasizes the impact of similar examples in the demonstrations, stressing the need for label correctness and more shots. In this work, we provide a Two-Dimensional Coordinate System that unifies both views into a systematic framework. The framework explains the behavior of ICL through two orthogonal variables: whether LLMs can recognize the task and whether similar examples are presented in the demonstrations. We propose the peak inverse rank metric to detect the task recognition ability of LLMs and study LLMs' reactions to different definitions of similarity. Based on these, we conduct extensive experiments to elucidate how ICL functions across each quadrant on multiple representative classification tasks. Finally, we extend our analyses to generation tasks, showing that our coordinate system can also be used to interpret ICL for generation tasks effectively.

Title: EAFormer: Scene Text Segmentation with Edge-Aware Transformers

• Authors: Haiyang Yu, Teng Fu, Bin Li, Xiangyang Xue
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2407.17020
• Pdf URL: https://arxiv.org/pdf/2407.17020
• Copy Paste: [[2407.17020]] EAFormer: Scene Text Segmentation with Edge-Aware Transformers(https://arxiv.org/abs/2407.17020)
• Keywords: generative
• Abstract: Scene text segmentation aims at cropping texts from scene images, which is usually used to help generative models edit or remove texts. The existing text segmentation methods tend to involve various text-related supervisions for better performance. However, most of them ignore the importance of text edges, which are significant for downstream applications. In this paper, we propose Edge-Aware Transformers, termed EAFormer, to segment texts more accurately, especially at the edge of texts. Specifically, we first design a text edge extractor to detect edges and filter out edges of non-text areas. Then, we propose an edge-guided encoder to make the model focus more on text edges. Finally, an MLP-based decoder is employed to predict text masks. We have conducted extensive experiments on commonly-used benchmarks to verify the effectiveness of EAFormer. The experimental results demonstrate that the proposed method can perform better than previous methods, especially on the segmentation of text edges. Considering that the annotations of several benchmarks (e.g., COCO_TS and MLT_S) are not accurate enough to fairly evaluate our methods, we have relabeled these datasets. Through experiments, we observe that our method can achieve a higher performance improvement when more accurate annotations are used for training.

Title: Sparse Inducing Points in Deep Gaussian Processes: Enhancing Modeling with Denoising Diffusion Variational Inference

• Authors: Jian Xu, Delu Zeng, John Paisley
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2407.17033
• Pdf URL: https://arxiv.org/pdf/2407.17033
• Copy Paste: [[2407.17033]] Sparse Inducing Points in Deep Gaussian Processes: Enhancing Modeling with Denoising Diffusion Variational Inference(https://arxiv.org/abs/2407.17033)
• Keywords: diffusion
• Abstract: Deep Gaussian processes (DGPs) provide a robust paradigm for Bayesian deep learning. In DGPs, a set of sparse integration locations called inducing points are selected to approximate the posterior distribution of the model. This is done to reduce computational complexity and improve model efficiency. However, inferring the posterior distribution of inducing points is not straightforward. Traditional variational inference approaches to posterior approximation often lead to significant bias. To address this issue, we propose an alternative method called Denoising Diffusion Variational Inference (DDVI) that uses a denoising diffusion stochastic differential equation (SDE) to generate posterior samples of inducing variables. We rely on score matching methods for denoising diffusion model to approximate score functions with a neural network. Furthermore, by combining classical mathematical theory of SDEs with the minimization of KL divergence between the approximate and true processes, we propose a novel explicit variational lower bound for the marginal likelihood function of DGP. Through experiments on various datasets and comparisons with baseline methods, we empirically demonstrate the effectiveness of DDVI for posterior inference of inducing points for DGP models.

Title: Contrastive Learning Is Not Optimal for Quasiperiodic Time Series

• Authors: Adrian Atienza, Jakob Bardram, Sadasivan Puthusserypady
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2407.17073
• Pdf URL: https://arxiv.org/pdf/2407.17073
• Copy Paste: [[2407.17073]] Contrastive Learning Is Not Optimal for Quasiperiodic Time Series(https://arxiv.org/abs/2407.17073)
• Keywords: self-supervised
• Abstract: Despite recent advancements in Self-Supervised Learning (SSL) for time series analysis, a noticeable gap persists between the anticipated achievements and actual performance. While these methods have demonstrated formidable generalization capabilities with minimal labels in various domains, their effectiveness in distinguishing between different classes based on a limited number of annotated records is notably lacking. Our hypothesis attributes this bottleneck to the prevalent use of Contrastive Learning, a shared training objective in previous state-of-the-art (SOTA) methods. By mandating distinctiveness between representations for negative pairs drawn from separate records, this approach compels the model to encode unique record-based patterns but simultaneously neglects changes occurring across the entire record. To overcome this challenge, we introduce Distilled Embedding for Almost-Periodic Time Series (DEAPS) in this paper, offering a non-contrastive method tailored for quasiperiodic time series, such as electrocardiogram (ECG) data. By avoiding the use of negative pairs, we not only mitigate the model's blindness to temporal changes but also enable the integration of a "Gradual Loss (Lgra)" function. This function guides the model to effectively capture dynamic patterns evolving throughout the record. The outcomes are promising, as DEAPS demonstrates a notable improvement of +10% over existing SOTA methods when just a few annotated records are presented to fit a Machine Learning (ML) model based on the learned representation.

Title: SAFETY-J: Evaluating Safety with Critique

• Authors: Yixiu Liu, Yuxiang Zheng, Shijie Xia, Yuan Guo, Jiajun Li, Yi Tu, Chaoling Song, Pengfei Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17075
• Pdf URL: https://arxiv.org/pdf/2407.17075
• Copy Paste: [[2407.17075]] SAFETY-J: Evaluating Safety with Critique(https://arxiv.org/abs/2407.17075)
• Keywords: generative
• Abstract: The deployment of Large Language Models (LLMs) in content generation raises significant safety concerns, particularly regarding the transparency and interpretability of content evaluations. Current methods, primarily focused on binary safety classifications, lack mechanisms for detailed critique, limiting their utility for model improvement and user trust. To address these limitations, we introduce SAFETY-J, a bilingual generative safety evaluator for English and Chinese with critique-based judgment. SAFETY-J utilizes a robust training dataset that includes diverse dialogues and augmented query-response pairs to assess safety across various scenarios comprehensively. We establish an automated meta-evaluation benchmark that objectively assesses the quality of critiques with minimal human intervention, facilitating scalable and continuous improvement. Additionally, SAFETY-J employs an iterative preference learning technique to dynamically refine safety assessments based on meta-evaluations and critiques. Our evaluations demonstrate that SAFETY-J provides more nuanced and accurate safety evaluations, thereby enhancing both critique quality and predictive reliability in complex content scenarios. To facilitate further research and application, we will open-source SAFETY-J's training protocols, datasets, and code.

Title: A Survey Forest Diagram : Gain a Divergent Insight View on a Specific Research Topic

• Authors: Jinghong Li, Wen Gu, Koichi Ota, Shinobu Hasegawa
• Subjects: cs.CL, cs.AI, cs.DL
• Abstract URL: https://arxiv.org/abs/2407.17081
• Pdf URL: https://arxiv.org/pdf/2407.17081
• Copy Paste: [[2407.17081]] A Survey Forest Diagram : Gain a Divergent Insight View on a Specific Research Topic(https://arxiv.org/abs/2407.17081)
• Keywords: generative
• Abstract: With the exponential growth in the number of papers and the trend of AI research, the use of Generative AI for information retrieval and question-answering has become popular for conducting research surveys. However, novice researchers unfamiliar with a particular field may not significantly improve their efficiency in interacting with Generative AI because they have not developed divergent thinking in that field. This study aims to develop an in-depth Survey Forest Diagram that guides novice researchers in divergent thinking about the research topic by indicating the citation clues among multiple papers, to help expand the survey perspective for novice researchers.

Title: When Text and Images Don't Mix: Bias-Correcting Language-Image Similarity Scores for Anomaly Detection

• Authors: Adam Goodge, Bryan Hooi, Wee Siong Ng
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.17083
• Pdf URL: https://arxiv.org/pdf/2407.17083
• Copy Paste: [[2407.17083]] When Text and Images Don't Mix: Bias-Correcting Language-Image Similarity Scores for Anomaly Detection(https://arxiv.org/abs/2407.17083)
• Keywords: anomaly
• Abstract: Contrastive Language-Image Pre-training (CLIP) achieves remarkable performance in various downstream tasks through the alignment of image and text input embeddings and holds great promise for anomaly detection. However, our empirical experiments show that the embeddings of text inputs unexpectedly tightly cluster together, far away from image embeddings, contrary to the model's contrastive training objective to align image-text input pairs. We show that this phenomenon induces a `similarity bias' - in which false negative and false positive errors occur due to bias in the similarities between images and the normal label text embeddings. To address this bias, we propose a novel methodology called BLISS which directly accounts for this similarity bias through the use of an auxiliary, external set of text inputs. BLISS is simple, it does not require strong inductive biases about anomalous behaviour nor an expensive training process, and it significantly outperforms baseline methods on benchmark image datasets, even when access to normal data is extremely limited.

Title: MemBench: Memorized Image Trigger Prompt Dataset for Diffusion Models

• Authors: Chunsan Hong, Tae-Hyun Oh, Minhyuk Sung
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2407.17095
• Pdf URL: https://arxiv.org/pdf/2407.17095
• Copy Paste: [[2407.17095]] MemBench: Memorized Image Trigger Prompt Dataset for Diffusion Models(https://arxiv.org/abs/2407.17095)
• Keywords: diffusion
• Abstract: Diffusion models have achieved remarkable success in Text-to-Image generation tasks, leading to the development of many commercial models. However, recent studies have reported that diffusion models often generate replicated images in train data when triggered by specific prompts, potentially raising social issues ranging from copyright to privacy concerns. To sidestep the memorization, there have been recent studies for developing memorization mitigation methods for diffusion models. Nevertheless, the lack of benchmarks impedes the assessment of the true effectiveness of these methods. In this work, we present MemBench, the first benchmark for evaluating image memorization mitigation methods. Our benchmark includes a large number of memorized image trigger prompts in Stable Diffusion, the most popularly used model nowadays. Furthermore, in contrast to the prior work evaluating mitigation performance only on trigger prompts, we present metrics evaluating on both trigger prompts and general prompts, so that we can see whether mitigation methods address the memorization issue while maintaining performance for general prompts. This is an important development considering the practical applications which previous works have overlooked. Through evaluation on MemBench, we verify that the performance of existing image memorization mitigation methods is still insufficient for application to diffusion models.

Title: PiPa++: Towards Unification of Domain Adaptive Semantic Segmentation via Self-supervised Learning

• Authors: Mu Chen, Zhedong Zheng, Yi Yang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.17101
• Pdf URL: https://arxiv.org/pdf/2407.17101
• Copy Paste: [[2407.17101]] PiPa++: Towards Unification of Domain Adaptive Semantic Segmentation via Self-supervised Learning(https://arxiv.org/abs/2407.17101)
• Keywords: self-supervised
• Abstract: Unsupervised domain adaptive segmentation aims to improve the segmentation accuracy of models on target domains without relying on labeled data from those domains. This approach is crucial when labeled target domain data is scarce or unavailable. It seeks to align the feature representations of the source domain (where labeled data is available) and the target domain (where only unlabeled data is present), thus enabling the model to generalize well to the target domain. Current image- and video-level domain adaptation have been addressed using different and specialized frameworks, training strategies and optimizations despite their underlying connections. In this paper, we propose a unified framework PiPa++, which leverages the core idea of ``comparing'' to (1) explicitly encourage learning of discriminative pixel-wise features with intraclass compactness and inter-class separability, (2) promote the robust feature learning of the identical patch against different contexts or fluctuations, and (3) enable the learning of temporal continuity under dynamic environments. With the designed task-smart contrastive sampling strategy, PiPa++ enables the mining of more informative training samples according to the task demand. Extensive experiments demonstrate the effectiveness of our method on both image-level and video-level domain adaption benchmarks. Moreover, the proposed method is compatible with other UDA approaches to further improve the performance without introducing extra parameters.

Title: A Self-Supervised Image Registration Approach for Measuring Local Response Patterns in Metastatic Ovarian Cancer

• Authors: Inês P. Machado, Anna Reithmeir, Fryderyk Kogl, Leonardo Rundo, Gabriel Funingana, Marika Reinius, Gift Mungmeeprued, Zeyu Gao, Cathal McCague, Eric Kerfoot, Ramona Woitek, Evis Sala, Yangming Ou, James Brenton, Julia Schnabel, Mireia Crispin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2407.17114
• Pdf URL: https://arxiv.org/pdf/2407.17114
• Copy Paste: [[2407.17114]] A Self-Supervised Image Registration Approach for Measuring Local Response Patterns in Metastatic Ovarian Cancer(https://arxiv.org/abs/2407.17114)
• Keywords: self-supervised
• Abstract: High-grade serous ovarian carcinoma (HGSOC) is characterised by significant spatial and temporal heterogeneity, typically manifesting at an advanced metastatic stage. A major challenge in treating advanced HGSOC is effectively monitoring localised change in tumour burden across multiple sites during neoadjuvant chemotherapy (NACT) and predicting long-term pathological response and overall patient survival. In this work, we propose a self-supervised deformable image registration algorithm that utilises a general-purpose image encoder for image feature extraction to co-register contrast-enhanced computerised tomography scan images acquired before and after neoadjuvant chemotherapy. This approach addresses challenges posed by highly complex tumour deformations and longitudinal lesion matching during treatment. Localised tumour changes are calculated using the Jacobian determinant maps of the registration deformation at multiple disease sites and their macroscopic areas, including hypo-dense (i.e., cystic/necrotic), hyper-dense (i.e., calcified), and intermediate density (i.e., soft tissue) portions. A series of experiments is conducted to understand the role of a general-purpose image encoder and its application in quantifying change in tumour burden during neoadjuvant chemotherapy in HGSOC. This work is the first to demonstrate the feasibility of a self-supervised image registration approach in quantifying NACT-induced localised tumour changes across the whole disease burden of patients with complex multi-site HGSOC, which could be used as a potential marker for ovarian cancer patient's long-term pathological response and survival.

Title: Unpaired Photo-realistic Image Deraining with Energy-informed Diffusion Model

• Authors: Yuanbo Wen, Tao Gao, Ting Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2407.17193
• Pdf URL: https://arxiv.org/pdf/2407.17193
• Copy Paste: [[2407.17193]] Unpaired Photo-realistic Image Deraining with Energy-informed Diffusion Model(https://arxiv.org/abs/2407.17193)
• Keywords: diffusion
• Abstract: Existing unpaired image deraining approaches face challenges in accurately capture the distinguishing characteristics between the rainy and clean domains, resulting in residual degradation and color distortion within the reconstructed images. To this end, we propose an energy-informed diffusion model for unpaired photo-realistic image deraining (UPID-EDM). Initially, we delve into the intricate visual-language priors embedded within the contrastive language-image pre-training model (CLIP), and demonstrate that the CLIP priors aid in the discrimination of rainy and clean images. Furthermore, we introduce a dual-consistent energy function (DEF) that retains the rain-irrelevant characteristics while eliminating the rain-relevant features. This energy function is trained by the non-corresponding rainy and clean images. In addition, we employ the rain-relevance discarding energy function (RDEF) and the rain-irrelevance preserving energy function (RPEF) to direct the reverse sampling procedure of a pre-trained diffusion model, effectively removing the rain streaks while preserving the image contents. Extensive experiments demonstrate that our energy-informed model surpasses the existing unpaired learning approaches in terms of both supervised and no-reference metrics.

Title: Graph Neural Networks: A suitable Alternative to MLPs in Latent 3D Medical Image Classification?

• Authors: Johannes Kiechle, Daniel M. Lang, Stefan M. Fischer, Lina Felsner, Jan C. Peeken, Julia A. Schnabel
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2407.17219
• Pdf URL: https://arxiv.org/pdf/2407.17219
• Copy Paste: [[2407.17219]] Graph Neural Networks: A suitable Alternative to MLPs in Latent 3D Medical Image Classification?(https://arxiv.org/abs/2407.17219)
• Keywords: foundation model
• Abstract: Recent studies have underscored the capabilities of natural imaging foundation models to serve as powerful feature extractors, even in a zero-shot setting for medical imaging data. Most commonly, a shallow multi-layer perceptron (MLP) is appended to the feature extractor to facilitate end-to-end learning and downstream prediction tasks such as classification, thus representing the de facto standard. However, as graph neural networks (GNNs) have become a practicable choice for various tasks in medical research in the recent past, we direct attention to the question of how effective GNNs are compared to MLP prediction heads for the task of 3D medical image classification, proposing them as a potential alternative. In our experiments, we devise a subject-level graph for each volumetric dataset instance. Therein latent representations of all slices in the volume, encoded through a DINOv2 pretrained vision transformer (ViT), constitute the nodes and their respective node features. We use public datasets to compare the classification heads numerically and evaluate various graph construction and graph convolution methods in our experiments. Our findings show enhancements of the GNN in classification performance and substantial improvements in runtime compared to an MLP prediction head. Additional robustness evaluations further validate the promising performance of the GNN, promoting them as a suitable alternative to traditional MLP classification heads. Our code is publicly available at: this https URL

Title: Sublinear Regret for An Actor-Critic Algorithm in Continuous-Time Linear-Quadratic Reinforcement Learning

• Authors: Yilie Huang, Yanwei Jia, Xun Yu Zhou
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.17226
• Pdf URL: https://arxiv.org/pdf/2407.17226
• Copy Paste: [[2407.17226]] Sublinear Regret for An Actor-Critic Algorithm in Continuous-Time Linear-Quadratic Reinforcement Learning(https://arxiv.org/abs/2407.17226)
• Keywords: diffusion
• Abstract: We study reinforcement learning (RL) for a class of continuous-time linear-quadratic (LQ) control problems for diffusions where volatility of the state processes depends on both state and control variables. We apply a model-free approach that relies neither on knowledge of model parameters nor on their estimations, and devise an actor-critic algorithm to learn the optimal policy parameter directly. Our main contributions include the introduction of a novel exploration schedule and a regret analysis of the proposed algorithm. We provide the convergence rate of the policy parameter to the optimal one, and prove that the algorithm achieves a regret bound of $O(N^{\frac{3}{4}})$ up to a logarithmic factor. We conduct a simulation study to validate the theoretical results and demonstrate the effectiveness and reliability of the proposed algorithm. We also perform numerical comparisons between our method and those of the recent model-based stochastic LQ RL studies adapted to the state- and control-dependent volatility setting, demonstrating a better performance of the former in terms of regret bounds.

Title: LPGen: Enhancing High-Fidelity Landscape Painting Generation through Diffusion Model

• Authors: Wanggong Yang, Xiaona Wang, Yingrui Qiu, Yifei Zhao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2407.17229
• Pdf URL: https://arxiv.org/pdf/2407.17229
• Copy Paste: [[2407.17229]] LPGen: Enhancing High-Fidelity Landscape Painting Generation through Diffusion Model(https://arxiv.org/abs/2407.17229)
• Keywords: diffusion
• Abstract: Generating landscape paintings expands the possibilities of artistic creativity and imagination. Traditional landscape painting methods involve using ink or colored ink on rice paper, which requires substantial time and effort. These methods are susceptible to errors and inconsistencies and lack precise control over lines and colors. This paper presents LPGen, a high-fidelity, controllable model for landscape painting generation, introducing a novel multi-modal framework that integrates image prompts into the diffusion model. We extract its edges and contours by computing canny edges from the target landscape image. These, along with natural language text prompts and drawing style references, are fed into the latent diffusion model as conditions. We implement a decoupled cross-attention strategy to ensure compatibility between image and text prompts, facilitating multi-modal image generation. A decoder generates the final image. Quantitative and qualitative analyses demonstrate that our method outperforms existing approaches in landscape painting generation and exceeds the current state-of-the-art. The LPGen network effectively controls the composition and color of landscape paintings, generates more accurate images, and supports further research in deep learning-based landscape painting generation.

Title: SCIsegV2: A Universal Tool for Segmentation of Intramedullary Lesions in Spinal Cord Injury

• Authors: Enamundram Naga Karthik, Jan Valošek, Lynn Farner, Dario Pfyffer, Simon Schading-Sassenhausen, Anna Lebret, Gergely David, Andrew C. Smith, Kenneth A. Weber II, Maryam Seif, RHSCIR Network Imaging Group, Patrick Freund, Julien Cohen-Adad
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.17265
• Pdf URL: https://arxiv.org/pdf/2407.17265
• Copy Paste: [[2407.17265]] SCIsegV2: A Universal Tool for Segmentation of Intramedullary Lesions in Spinal Cord Injury(https://arxiv.org/abs/2407.17265)
• Keywords: generative
• Abstract: Spinal cord injury (SCI) is a devastating incidence leading to permanent paralysis and loss of sensory-motor functions potentially resulting in the formation of lesions within the spinal cord. Imaging biomarkers obtained from magnetic resonance imaging (MRI) scans can predict the functional recovery of individuals with SCI and help choose the optimal treatment strategy. Currently, most studies employ manual quantification of these MRI-derived biomarkers, which is a subjective and tedious task. In this work, we propose (i) a universal tool for the automatic segmentation of intramedullary SCI lesions, dubbed \texttt{SCIsegV2}, and (ii) a method to automatically compute the width of the tissue bridges from the segmented lesion. Tissue bridges represent the spared spinal tissue adjacent to the lesion, which is associated with functional recovery in SCI patients. The tool was trained and validated on a heterogeneous dataset from 7 sites comprising patients from different SCI phases (acute, sub-acute, and chronic) and etiologies (traumatic SCI, ischemic SCI, and degenerative cervical myelopathy). Tissue bridges quantified automatically did not significantly differ from those computed manually, suggesting that the proposed automatic tool can be used to derive relevant MRI biomarkers. \texttt{SCIsegV2} and the automatic tissue bridges computation are open-source and available in Spinal Cord Toolbox (v6.4 and above) via the \texttt{sct\_deepseg -task seg\_sc\_lesion\_t2w\_sci} and \texttt{sct\_analyze\_lesion} functions, respectively.

Title: LangOcc: Self-Supervised Open Vocabulary Occupancy Estimation via Volume Rendering

• Authors: Simon Boeder, Fabian Gigengack, Benjamin Risse
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2407.17310
• Pdf URL: https://arxiv.org/pdf/2407.17310
• Copy Paste: [[2407.17310]] LangOcc: Self-Supervised Open Vocabulary Occupancy Estimation via Volume Rendering(https://arxiv.org/abs/2407.17310)
• Keywords: self-supervised
• Abstract: Semantic occupancy has recently gained significant traction as a prominent method for 3D scene representation. However, most existing camera-based methods rely on costly datasets with fine-grained 3D voxel labels or LiDAR scans for training, which limits their practicality and scalability, raising the need for self-supervised approaches in this domain. Moreover, most methods are tied to a predefined set of classes which they can detect. In this work we present a novel approach for open vocabulary occupancy estimation called \textit{LangOcc}, that is trained only via camera images, and can detect arbitrary semantics via vision-language alignment. In particular, we distill the knowledge of the strong vision-language aligned encoder CLIP into a 3D occupancy model via differentiable volume rendering. Our model estimates vision-language aligned features in a 3D voxel grid using only images. It is trained in a self-supervised manner by rendering our estimations back to 2D space, where ground-truth features can be computed. This training mechanism automatically supervises the scene geometry, allowing for a straight-forward and powerful training method without any explicit geometry supervision. LangOcc outperforms LiDAR-supervised competitors in open vocabulary occupancy by a large margin, solely relying on vision-based training. We also achieve state-of-the-art results in self-supervised semantic occupancy estimation on the Occ3D-nuScenes dataset, despite not being limited to a specific set of categories, thus demonstrating the effectiveness of our proposed vision-language training.

Title: Global and Local Confidence Based Fraud Detection Graph Neural Network

• Authors: Jiaxun Liu, Yue Tian, Guanjun Liu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2407.17333
• Pdf URL: https://arxiv.org/pdf/2407.17333
• Copy Paste: [[2407.17333]] Global and Local Confidence Based Fraud Detection Graph Neural Network(https://arxiv.org/abs/2407.17333)
• Keywords: anomaly
• Abstract: This paper presents the Global and Local Confidence Graph Neural Network (GLC-GNN), an innovative approach to graph-based anomaly detection that addresses the challenges of heterophily and camouflage in fraudulent activities. By introducing a prototype to encapsulate the global features of a graph and calculating a Global Confidence (GC) value for each node, GLC-GNN effectively distinguishes between benign and fraudulent nodes. The model leverages GC to generate attention values for message aggregation, enhancing its ability to capture both homophily and heterophily. Through extensive experiments on four open datasets, GLC-GNN demonstrates superior performance over state-of-the-art models in accuracy and convergence speed, while maintaining a compact model size and expedited training process. The integration of global and local confidence measures in GLC-GNN offers a robust solution for detecting anomalies in graphs, with significant implications for fraud detection across diverse domains.

Title: Boosting Large Language Models with Socratic Method for Conversational Mathematics Teaching

• Authors: Yuyang Ding, Hanglei Hu, Jie Zhou, Qin Chen, Bo Jiang, Liang He
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17349
• Pdf URL: https://arxiv.org/pdf/2407.17349
• Copy Paste: [[2407.17349]] Boosting Large Language Models with Socratic Method for Conversational Mathematics Teaching(https://arxiv.org/abs/2407.17349)
• Keywords: generative
• Abstract: With the introduction of large language models (LLMs), automatic math reasoning has seen tremendous success. However, current methods primarily focus on providing solutions or using techniques like Chain-of-Thought to enhance problem-solving accuracy. In this paper, we focus on improving the capability of mathematics teaching via a Socratic teaching-based LLM (\texttt{SocraticLLM}), which guides learners toward profound thinking with clarity and self-discovery via conversation. We collect and release a high-quality mathematical teaching dataset, named \texttt{SocraticMATH}, which provides Socratic-style conversations of problems with extra knowledge. Also, we propose a knowledge-enhanced LLM as a strong baseline to generate reliable responses with review, guidance/heuristic, rectification, and summarization. Experimental results show the great advantages of \texttt{SocraticLLM} by comparing it with several strong generative models. The codes and datasets are available on \url{this https URL}.

Title: ViPer: Visual Personalization of Generative Models via Individual Preference Learning

• Authors: Sogand Salehi, Mahdi Shafiei, Teresa Yeo, Roman Bachmann, Amir Zamir
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2407.17365
• Pdf URL: https://arxiv.org/pdf/2407.17365
• Copy Paste: [[2407.17365]] ViPer: Visual Personalization of Generative Models via Individual Preference Learning(https://arxiv.org/abs/2407.17365)
• Keywords: generative
• Abstract: Different users find different images generated for the same prompt desirable. This gives rise to personalized image generation which involves creating images aligned with an individual's visual preference. Current generative models are, however, unpersonalized, as they are tuned to produce outputs that appeal to a broad audience. Using them to generate images aligned with individual users relies on iterative manual prompt engineering by the user which is inefficient and undesirable. We propose to personalize the image generation process by first capturing the generic preferences of the user in a one-time process by inviting them to comment on a small selection of images, explaining why they like or dislike each. Based on these comments, we infer a user's structured liked and disliked visual attributes, i.e., their visual preference, using a large language model. These attributes are used to guide a text-to-image model toward producing images that are tuned towards the individual user's visual preference. Through a series of user studies and large language model guided evaluations, we demonstrate that the proposed method results in generations that are well aligned with individual users' visual preferences.

Title: Five reasons against assuming a data-generating distribution in Machine Learning

• Authors: Benedikt Höltgen, Robert C. Williamson
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2407.17395
• Pdf URL: https://arxiv.org/pdf/2407.17395
• Copy Paste: [[2407.17395]] Five reasons against assuming a data-generating distribution in Machine Learning(https://arxiv.org/abs/2407.17395)
• Keywords: generative
• Abstract: Machine Learning research, as most of Statistics, heavily relies on the concept of a data-generating probability distribution. As data points are thought to be sampled from such a distribution, we can learn from observed data about this distribution and, thus, predict future data points drawn from it (with some probability of success). Drawing on scholarship across disciplines, we here argue that this framework is not always a good model. Not only do such true probability distributions not exist; the framework can also be misleading and obscure both the choices made and the goals pursued in machine learning practice. We suggest an alternative framework that focuses on finite populations rather than abstract distributions; while classical learning theory can be left almost unchanged, it opens new opportunities, especially to model sampling. We compile these considerations into five reasons for modelling machine learning -- in some settings -- with finite distributions rather than generative distributions, both to be more faithful to practice and to provide novel theoretical insights.

Title: Vision Language Model-Empowered Contract Theory for AIGC Task Allocation in Teleoperation

• Authors: Zijun Zhan, Yaxian Dong, Yuqing Hu, Shuai Li, Shaohua Cao, Zhu Han
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.17428
• Pdf URL: https://arxiv.org/pdf/2407.17428
• Copy Paste: [[2407.17428]] Vision Language Model-Empowered Contract Theory for AIGC Task Allocation in Teleoperation(https://arxiv.org/abs/2407.17428)
• Keywords: diffusion
• Abstract: Integrating low-light image enhancement techniques, in which diffusion-based AI-generated content (AIGC) models are promising, is necessary to enhance nighttime teleoperation. Remarkably, the AIGC model is computation-intensive, thus necessitating the allocation of AIGC tasks to edge servers with ample computational resources. Given the distinct cost of the AIGC model trained with varying-sized datasets and AIGC tasks possessing disparate demand, it is imperative to formulate a differential pricing strategy to optimize the utility of teleoperators and edge servers concurrently. Nonetheless, the pricing strategy formulation is under information asymmetry, i.e., the demand (e.g., the difficulty level of AIGC tasks and their distribution) of AIGC tasks is hidden information to edge servers. Additionally, manually assessing the difficulty level of AIGC tasks is tedious and unnecessary for teleoperators. To this end, we devise a framework of AIGC task allocation assisted by the Vision Language Model (VLM)-empowered contract theory, which includes two components: VLM-empowered difficulty assessment and contract theory-assisted AIGC task allocation. The first component enables automatic and accurate AIGC task difficulty assessment. The second component is capable of formulating the pricing strategy for edge servers under information asymmetry, thereby optimizing the utility of both edge servers and teleoperators. The simulation results demonstrated that our proposed framework can improve the average utility of teleoperators and edge servers by 10.88~12.43% and 1.4~2.17%, respectively. Code and data are available at this https URL.

Title: Looking at Model Debiasing through the Lens of Anomaly Detection

• Authors: Vito Paolo Pastore, Massimiliano Ciranni, Davide Marinelli, Francesca Odone, Vittorio Murino
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2407.17449
• Pdf URL: https://arxiv.org/pdf/2407.17449
• Copy Paste: [[2407.17449]] Looking at Model Debiasing through the Lens of Anomaly Detection(https://arxiv.org/abs/2407.17449)
• Keywords: anomaly
• Abstract: It is widely recognized that deep neural networks are sensitive to bias in the data. This means that during training these models are likely to learn spurious correlations between data and labels, resulting in limited generalization abilities and low performance. In this context, model debiasing approaches can be devised aiming at reducing the model's dependency on such unwanted correlations, either leveraging the knowledge of bias information or not. In this work, we focus on the latter and more realistic scenario, showing the importance of accurately predicting the bias-conflicting and bias-aligned samples to obtain compelling performance in bias mitigation. On this ground, we propose to conceive the problem of model bias from an out-of-distribution perspective, introducing a new bias identification method based on anomaly detection. We claim that when data is mostly biased, bias-conflicting samples can be regarded as outliers with respect to the bias-aligned distribution in the feature space of a biased model, thus allowing for precisely detecting them with an anomaly detection method. Coupling the proposed bias identification approach with bias-conflicting data upsampling and augmentation in a two-step strategy, we reach state-of-the-art performance on synthetic and real benchmark datasets. Ultimately, our proposed approach shows that the data bias issue does not necessarily require complex debiasing methods, given that an accurate bias identification procedure is defined.

Title: SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency

• Authors: Yiming Xie, Chun-Han Yao, Vikram Voleti, Huaizu Jiang, Varun Jampani
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2407.17470
• Pdf URL: https://arxiv.org/pdf/2407.17470
• Copy Paste: [[2407.17470]] SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency(https://arxiv.org/abs/2407.17470)
• Keywords: diffusion, generative
• Abstract: We present Stable Video 4D (SV4D), a latent video diffusion model for multi-frame and multi-view consistent dynamic 3D content generation. Unlike previous methods that rely on separately trained generative models for video generation and novel view synthesis, we design a unified diffusion model to generate novel view videos of dynamic 3D objects. Specifically, given a monocular reference video, SV4D generates novel views for each video frame that are temporally consistent. We then use the generated novel view videos to optimize an implicit 4D representation (dynamic NeRF) efficiently, without the need for cumbersome SDS-based optimization used in most prior works. To train our unified novel view video generation model, we curated a dynamic 3D object dataset from the existing Objaverse dataset. Extensive experimental results on multiple datasets and user studies demonstrate SV4D's state-of-the-art performance on novel-view video synthesis as well as 4D generation compared to prior works.