# documents

scrape_date: 2025-03-17T13:26:50.821265
content_type: markdown
hash: 8ac916c896f15ce9954208ffe3f551911759b256356bfbda4b8225f681374116
url:
https://github.com/akitenkrad/akitenkrad-blog/blob/e493935c054a5ba82143a5b8b5a0daeebcafb376/src/content/posts/arxiv/202310/20231021000000/index.md

title:
akitenkrad-blog/src/content/posts/arxiv/202310/20231021000000/index.md at e493935c054a5ba82143a5b8b5a0daeebcafb376 · akitenkrad/akitenkrad-blog · GitHub

content:
Skip to content

## Navigation Menu

Toggle navigation

[ ](/)

[ Sign in ](/login?return_to=https%3A%2F%2Fgithub.com%2Fakitenkrad%2Fakitenkrad-blog%2Fblob%2Fe493935c054a5ba82143a5b8b5a0daeebcafb376%2Fsrc%2Fcontent%2Fposts%2Farxiv%2F202310%2F20231021000000%2Findex.md)

  * Product 

    * [ GitHub Copilot Write better code with AI  ](https://github.com/features/copilot)
    * [ Security Find and fix vulnerabilities  ](https://github.com/features/security)
    * [ Actions Automate any workflow  ](https://github.com/features/actions)
    * [ Codespaces Instant dev environments  ](https://github.com/features/codespaces)
    * [ Issues Plan and track work  ](https://github.com/features/issues)
    * [ Code Review Manage code changes  ](https://github.com/features/code-review)
    * [ Discussions Collaborate outside of code  ](https://github.com/features/discussions)
    * [ Code Search Find more, search less  ](https://github.com/features/code-search)

Explore
    * [ All features ](https://github.com/features)
    * [ Documentation  ](https://docs.github.com)
    * [ GitHub Skills  ](https://skills.github.com)
    * [ Blog  ](https://github.blog)

  * Solutions 

By company size
    * [ Enterprises ](https://github.com/enterprise)
    * [ Small and medium teams ](https://github.com/team)
    * [ Startups ](https://github.com/enterprise/startups)
    * [ Nonprofits ](/solutions/industry/nonprofits)

By use case
    * [ DevSecOps ](/solutions/use-case/devsecops)
    * [ DevOps ](/solutions/use-case/devops)
    * [ CI/CD ](/solutions/use-case/ci-cd)
    * [ View all use cases ](/solutions/use-case)

By industry
    * [ Healthcare ](/solutions/industry/healthcare)
    * [ Financial services ](/solutions/industry/financial-services)
    * [ Manufacturing ](/solutions/industry/manufacturing)
    * [ Government ](/solutions/industry/government)
    * [ View all industries ](/solutions/industry)

[ View all solutions  ](/solutions)

  * Resources 

Topics
    * [ AI ](/resources/articles/ai)
    * [ DevOps ](/resources/articles/devops)
    * [ Security ](/resources/articles/security)
    * [ Software Development ](/resources/articles/software-development)
    * [ View all ](/resources/articles)

Explore
    * [ Learning Pathways  ](https://resources.github.com/learn/pathways)
    * [ Events & Webinars  ](https://resources.github.com)
    * [ Ebooks & Whitepapers ](https://github.com/resources/whitepapers)
    * [ Customer Stories ](https://github.com/customer-stories)
    * [ Partners  ](https://partner.github.com)
    * [ Executive Insights ](https://github.com/solutions/executive-insights)

  * Open Source 

    * [ GitHub Sponsors Fund open source developers  ](/sponsors)

    * [ The ReadME Project GitHub community articles  ](https://github.com/readme)

Repositories
    * [ Topics ](https://github.com/topics)
    * [ Trending ](https://github.com/trending)
    * [ Collections ](https://github.com/collections)

  * Enterprise 

    * [ Enterprise platform AI-powered developer platform  ](/enterprise)

Available add-ons
    * [ Advanced Security Enterprise-grade security features  ](https://github.com/enterprise/advanced-security)
    * [ Copilot for business Enterprise-grade AI features  ](/features/copilot/copilot-business)
    * [ Premium Support Enterprise-grade 24/7 support  ](/premium-support)

  * [Pricing](https://github.com/pricing)



Search or jump to...

# Search code, repositories, users, issues, pull requests...

Search 

Clear




[Search syntax tips](https://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax)

#  Provide feedback 

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Cancel  Submit feedback 

#  Saved searches 

## Use saved searches to filter your results more quickly

Name

Query

To see all available qualifiers, see our [documentation](https://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax). 

Cancel  Create saved search 

[ Sign in ](/login?return_to=https%3A%2F%2Fgithub.com%2Fakitenkrad%2Fakitenkrad-blog%2Fblob%2Fe493935c054a5ba82143a5b8b5a0daeebcafb376%2Fsrc%2Fcontent%2Fposts%2Farxiv%2F202310%2F20231021000000%2Findex.md)

[ Sign up ](/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fblob%2Fshow&source=header-repo&source_repo=akitenkrad%2Fakitenkrad-blog) Reseting focus

You signed in with another tab or window. [Reload]() to refresh your session. You signed out in another tab or window. [Reload]() to refresh your session. You switched accounts on another tab or window. [Reload]() to refresh your session. Dismiss alert

{{ message }}

[ akitenkrad ](/akitenkrad) / **[akitenkrad-blog](/akitenkrad/akitenkrad-blog) ** Public

  * [ Notifications ](/login?return_to=%2Fakitenkrad%2Fakitenkrad-blog) You must be signed in to change notification settings
  * [ Fork 0 ](/login?return_to=%2Fakitenkrad%2Fakitenkrad-blog)
  * [ Star  2 ](/login?return_to=%2Fakitenkrad%2Fakitenkrad-blog)




  * [ Code ](/akitenkrad/akitenkrad-blog)
  * [ Issues 0 ](/akitenkrad/akitenkrad-blog/issues)
  * [ Pull requests 0 ](/akitenkrad/akitenkrad-blog/pulls)
  * [ Actions ](/akitenkrad/akitenkrad-blog/actions)
  * [ Projects 0 ](/akitenkrad/akitenkrad-blog/projects)
  * [ Security ](/akitenkrad/akitenkrad-blog/security)
  * [ Insights ](/akitenkrad/akitenkrad-blog/pulse)



Additional navigation options

  * [ Code  ](/akitenkrad/akitenkrad-blog)
  * [ Issues  ](/akitenkrad/akitenkrad-blog/issues)
  * [ Pull requests  ](/akitenkrad/akitenkrad-blog/pulls)
  * [ Actions  ](/akitenkrad/akitenkrad-blog/actions)
  * [ Projects  ](/akitenkrad/akitenkrad-blog/projects)
  * [ Security  ](/akitenkrad/akitenkrad-blog/security)
  * [ Insights  ](/akitenkrad/akitenkrad-blog/pulse)



## Files

e493935

## Breadcrumbs

  1. [akitenkrad-blog](/akitenkrad/akitenkrad-blog/tree/e493935c054a5ba82143a5b8b5a0daeebcafb376)
  2. /[src](/akitenkrad/akitenkrad-blog/tree/e493935c054a5ba82143a5b8b5a0daeebcafb376/src)
  3. /[content](/akitenkrad/akitenkrad-blog/tree/e493935c054a5ba82143a5b8b5a0daeebcafb376/src/content)
  4. /[posts](/akitenkrad/akitenkrad-blog/tree/e493935c054a5ba82143a5b8b5a0daeebcafb376/src/content/posts)
  5. /[arxiv](/akitenkrad/akitenkrad-blog/tree/e493935c054a5ba82143a5b8b5a0daeebcafb376/src/content/posts/arxiv)
  6. /[202310](/akitenkrad/akitenkrad-blog/tree/e493935c054a5ba82143a5b8b5a0daeebcafb376/src/content/posts/arxiv/202310)
  7. /[20231021000000](/akitenkrad/akitenkrad-blog/tree/e493935c054a5ba82143a5b8b5a0daeebcafb376/src/content/posts/arxiv/202310/20231021000000)



/

# index.md

Copy path

Blame

Blame

## Latest commit

## History

[History](/akitenkrad/akitenkrad-blog/commits/e493935c054a5ba82143a5b8b5a0daeebcafb376/src/content/posts/arxiv/202310/20231021000000/index.md)

[](/akitenkrad/akitenkrad-blog/commits/e493935c054a5ba82143a5b8b5a0daeebcafb376/src/content/posts/arxiv/202310/20231021000000/index.md)

3657 lines (2145 loc) · 279 KB

e493935

## Breadcrumbs

  1. [akitenkrad-blog](/akitenkrad/akitenkrad-blog/tree/e493935c054a5ba82143a5b8b5a0daeebcafb376)
  2. /[src](/akitenkrad/akitenkrad-blog/tree/e493935c054a5ba82143a5b8b5a0daeebcafb376/src)
  3. /[content](/akitenkrad/akitenkrad-blog/tree/e493935c054a5ba82143a5b8b5a0daeebcafb376/src/content)
  4. /[posts](/akitenkrad/akitenkrad-blog/tree/e493935c054a5ba82143a5b8b5a0daeebcafb376/src/content/posts)
  5. /[arxiv](/akitenkrad/akitenkrad-blog/tree/e493935c054a5ba82143a5b8b5a0daeebcafb376/src/content/posts/arxiv)
  6. /[202310](/akitenkrad/akitenkrad-blog/tree/e493935c054a5ba82143a5b8b5a0daeebcafb376/src/content/posts/arxiv/202310)
  7. /[20231021000000](/akitenkrad/akitenkrad-blog/tree/e493935c054a5ba82143a5b8b5a0daeebcafb376/src/content/posts/arxiv/202310/20231021000000)



/

# index.md

Top

## File metadata and controls

  * Preview

  * Code

  * Blame




3657 lines (2145 loc) · 279 KB

[Raw](https://github.com/akitenkrad/akitenkrad-blog/raw/e493935c054a5ba82143a5b8b5a0daeebcafb376/src/content/posts/arxiv/202310/20231021000000/index.md)

draft | title | date | author | description | tags | menu | math  
---|---|---|---|---|---|---|---  
false | arXiv @ 2023.10.21 | 2023-10-21 | akitenkrad |  |  | arXiv | Published:2023  
---|---  
  
| sidebar  
---  
| name | identifier | parent | weight  
---|---|---|---  
arXiv @ 2023.10.21 | arxiv_20231021 | 202310_arxiv | 10  
  
true  
<iframe src="pie.html" width=900 height=620 style="border:none"></iframe>

## Primary Categories

  * cs.CV (27)
  * cs.LG (30)
  * cs.AI (5)
  * cs.CL (59)
  * cs.RO (4)
  * eess.SP (1)
  * cs.HC (6)
  * cond-mat.mtrl-sci (1)
  * eess.SY (1)
  * stat.ML (3)
  * cs.DC (2)
  * eess.IV (4)
  * cs.CR (4)
  * cs.SE (2)
  * cs.DL (2)
  * cs.NE (2)
  * cs.MM (1)
  * cs.LO (1)
  * q-fin.PR (1)
  * eess.AS (1)
  * cs.IR (1)
  * q-bio.NC (1)
  * cs.SD (1)



## cs.CV (27)

### (1/160) A Car Model Identification System for Streamlining the Automobile Sales Process (Said Togru et al., 2023)

{{}}

Said Togru, Marco Moldovan. (2023)  
**A Car Model Identification System for Streamlining the Automobile Sales Process**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs.CV  
Keywords: Transformer, Transformers  
[Paper Link](http://arxiv.org/abs/2310.13198v2)

* * *

**ABSTRACT**  
This project presents an automated solution for the efficient identification of car models and makes from images, aimed at streamlining the vehicle listing process on online car-selling platforms. Through a thorough exploration encompassing various efficient network architectures including Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), and hybrid models, we achieved a notable accuracy of 81.97% employing the EfficientNet (V2 b2) architecture. To refine performance, a combination of strategies, including data augmentation, fine-tuning pretrained models, and extensive hyperparameter tuning, were applied. The trained model offers the potential for automating information extraction, promising enhanced user experiences across car-selling websites.

{{}}

### (2/160) Breaking through Deterministic Barriers: Randomized Pruning Mask Generation and Selection (Jianwei Li et al., 2023)

{{}}

Jianwei Li, Weizhi Gao, Qi Lei, Dongkuan Xu. (2023)  
**Breaking through Deterministic Barriers: Randomized Pruning Mask Generation and Selection**

* * *

Primary Category: cs.CV  
Categories: cs-CL, cs-CV, cs.CV  
Keywords: GLUE, Pruning  
[Paper Link](http://arxiv.org/abs/2310.13183v1)

* * *

**ABSTRACT**  
It is widely acknowledged that large and sparse models have higher accuracy than small and dense models under the same model size constraints. This motivates us to train a large model and then remove its redundant neurons or weights by pruning. Most existing works pruned the networks in a deterministic way, the performance of which solely depends on a single pruning criterion and thus lacks variety. Instead, in this paper, we propose a model pruning strategy that first generates several pruning masks in a designed random way. Subsequently, along with an effective mask-selection rule, the optimal mask is chosen from the pool of mask candidates. To further enhance efficiency, we introduce an early mask evaluation strategy, mitigating the overhead associated with training multiple masks. Our extensive experiments demonstrate that this approach achieves state-of-the-art performance across eight datasets from GLUE, particularly excelling at high levels of sparsity.

{{}}

### (3/160) LeTFuser: Light-weight End-to-end Transformer-Based Sensor Fusion for Autonomous Driving with Multi-Task Learning (Pedram Agand et al., 2023)

{{}}

Pedram Agand, Mohammad Mahdavian, Manolis Savva, Mo Chen. (2023)  
**LeTFuser: Light-weight End-to-end Transformer-Based Sensor Fusion for Autonomous Driving with Multi-Task Learning**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs.CV  
Keywords: Transformer  
[Paper Link](http://arxiv.org/abs/2310.13135v1)

* * *

**ABSTRACT**  
In end-to-end autonomous driving, the utilization of existing sensor fusion techniques for imitation learning proves inadequate in challenging situations that involve numerous dynamic agents. To address this issue, we introduce LeTFuser, a transformer-based algorithm for fusing multiple RGB-D camera representations. To perform perception and control tasks simultaneously, we utilize multi-task learning. Our model comprises of two modules, the first being the perception module that is responsible for encoding the observation data obtained from the RGB-D cameras. It carries out tasks such as semantic segmentation, semantic depth cloud mapping (SDC), and traffic light state recognition. Our approach employs the Convolutional vision Transformer (CvT) \cite{wu2021cvt} to better extract and fuse features from multiple RGB cameras due to local and global feature extraction capability of convolution and transformer modules, respectively. Following this, the control module undertakes the decoding of the encoded characteristics together with supplementary data, comprising a rough simulator for static and dynamic environments, as well as various measurements, in order to anticipate the waypoints associated with a latent feature space. We use two methods to process these outputs and generate the vehicular controls (e.g. steering, throttle, and brake) levels. The first method uses a PID algorithm to follow the waypoints on the fly, whereas the second one directly predicts the control policy using the measurement features and environmental state. We evaluate the model and conduct a comparative analysis with recent models on the CARLA simulator using various scenarios, ranging from normal to adversarial conditions, to simulate real-world scenarios. Our code is available at \url{[https://github.com/pagand/e2etransfuser/tree/cvpr-w}](https://github.com/pagand/e2etransfuser/tree/cvpr-w%7D) to facilitate future studies.

{{}}

### (4/160) RSAdapter: Adapting Multimodal Models for Remote Sensing Visual Question Answering (Yuduo Wang et al., 2023)

{{}}

Yuduo Wang, Pedram Ghamisi. (2023)  
**RSAdapter: Adapting Multimodal Models for Remote Sensing Visual Question Answering**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs-LG, cs.CV  
Keywords: Image Captioning, QA, Question Answering, Text Generation  
[Paper Link](http://arxiv.org/abs/2310.13120v1)

* * *

**ABSTRACT**  
In recent years, with the rapid advancement of transformer models, transformer-based multimodal architectures have found wide application in various downstream tasks, including but not limited to Image Captioning, Visual Question Answering (VQA), and Image-Text Generation. However, contemporary approaches to Remote Sensing (RS) VQA often involve resource-intensive techniques, such as full fine-tuning of large models or the extraction of image-text features from pre-trained multimodal models, followed by modality fusion using decoders. These approaches demand significant computational resources and time, and a considerable number of trainable parameters are introduced. To address these challenges, we introduce a novel method known as RSAdapter, which prioritizes runtime and parameter efficiency. RSAdapter comprises two key components: the Parallel Adapter and an additional linear transformation layer inserted after each fully connected (FC) layer within the Adapter. This approach not only improves adaptation to pre-trained multimodal models but also allows the parameters of the linear transformation layer to be integrated into the preceding FC layers during inference, reducing inference costs. To demonstrate the effectiveness of RSAdapter, we conduct an extensive series of experiments using three distinct RS-VQA datasets and achieve state-of-the-art results on all three datasets. The code for RSAdapter will be available online at <https://github.com/Y-D-Wang/RSAdapter>.

{{}}

### (5/160) AVTENet: Audio-Visual Transformer-based Ensemble Network Exploiting Multiple Experts for Video Deepfake Detection (Ammarah Hashmi et al., 2023)

{{}}

Ammarah Hashmi, Sahibzada Adil Shahzad, Chia-Wen Lin, Yu Tsao, Hsin-Min Wang. (2023)  
**AVTENet: Audio-Visual Transformer-based Ensemble Network Exploiting Multiple Experts for Video Deepfake Detection**

* * *

Primary Category: cs.CV  
Categories: cs-AI, cs-CV, cs-LG, cs-MM, cs-SD, cs.CV, eess-AS  
Keywords: AI, Transformer  
[Paper Link](http://arxiv.org/abs/2310.13103v1)

* * *

**ABSTRACT**  
Forged content shared widely on social media platforms is a major social problem that requires increased regulation and poses new challenges to the research community. The recent proliferation of hyper-realistic deepfake videos has drawn attention to the threat of audio and visual forgeries. Most previous work on detecting AI-generated fake videos only utilizes visual modality or audio modality. While there are some methods in the literature that exploit audio and visual modalities to detect forged videos, they have not been comprehensively evaluated on multi-modal datasets of deepfake videos involving acoustic and visual manipulations. Moreover, these existing methods are mostly based on CNN and suffer from low detection accuracy. Inspired by the recent success of Transformer in various fields, to address the challenges posed by deepfake technology, in this paper, we propose an Audio-Visual Transformer-based Ensemble Network (AVTENet) framework that considers both acoustic manipulation and visual manipulation to achieve effective video forgery detection. Specifically, the proposed model integrates several purely transformer-based variants that capture video, audio, and audio-visual salient cues to reach a consensus in prediction. For evaluation, we use the recently released benchmark multi-modal audio-video FakeAVCeleb dataset. For a detailed analysis, we evaluate AVTENet, its variants, and several existing methods on multiple test sets of the FakeAVCeleb dataset. Experimental results show that our best model outperforms all existing methods and achieves state-of-the-art performance on Testset-I and Testset-II of the FakeAVCeleb dataset.

{{}}

### (6/160) HumanTOMATO: Text-aligned Whole-body Motion Generation (Shunlin Lu et al., 2023)

{{}}

Shunlin Lu, Ling-Hao Chen, Ailing Zeng, Jing Lin, Ruimao Zhang, Lei Zhang, Heung-Yeung Shum. (2023)  
**HumanTOMATO: Text-aligned Whole-body Motion Generation**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs.CV  
Keywords: GPT  
[Paper Link](http://arxiv.org/abs/2310.12978v1)

* * *

**ABSTRACT**  
This work targets a novel text-driven whole-body motion generation task, which takes a given textual description as input and aims at generating high-quality, diverse, and coherent facial expressions, hand gestures, and body motions simultaneously. Previous works on text-driven motion generation tasks mainly have two limitations: they ignore the key role of fine-grained hand and face controlling in vivid whole-body motion generation, and lack a good alignment between text and motion. To address such limitations, we propose a Text-aligned whOle-body Motion generATiOn framework, named HumanTOMATO, which is the first attempt to our knowledge towards applicable holistic motion generation in this research area. To tackle this challenging task, our solution includes two key designs: (1) a Holistic Hierarchical VQ-VAE (aka H$^2$VQ) and a Hierarchical-GPT for fine-grained body and hand motion reconstruction and generation with two structured codebooks; and (2) a pre-trained text-motion-alignment model to help generated motion align with the input textual description explicitly. Comprehensive experiments verify that our model has significant advantages in both the quality of generated motions and their alignment with text.

{{}}

### (7/160) FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objects (Mayank Lunayach et al., 2023)

{{}}

Mayank Lunayach, Sergey Zakharov, Dian Chen, Rares Ambrus, Zsolt Kira, Muhammad Zubair Irshad. (2023)  
**FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objects**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs-RO, cs.CV  
Keywords: Self-Supervised  
[Paper Link](http://arxiv.org/abs/2310.12974v1)

* * *

**ABSTRACT**  
In this work, we address the challenging task of 3D object recognition without the reliance on real-world 3D labeled data. Our goal is to predict the 3D shape, size, and 6D pose of objects within a single RGB-D image, operating at the category level and eliminating the need for CAD models during inference. While existing self-supervised methods have made strides in this field, they often suffer from inefficiencies arising from non-end-to-end processing, reliance on separate models for different object categories, and slow surface extraction during the training of implicit reconstruction models; thus hindering both the speed and real-world applicability of the 3D recognition process. Our proposed method leverages a multi-stage training pipeline, designed to efficiently transfer synthetic performance to the real-world domain. This approach is achieved through a combination of 2D and 3D supervised losses during the synthetic domain training, followed by the incorporation of 2D supervised and 3D self-supervised losses on real-world data in two additional learning stages. By adopting this comprehensive strategy, our method successfully overcomes the aforementioned limitations and outperforms existing self-supervised 6D pose and size estimation baselines on the NOCS test-set with a 16.4% absolute improvement in mAP for 6D pose estimation while running in near real-time at 5 Hz.

{{}}

### (8/160) Frozen Transformers in Language Models Are Effective Visual Encoder Layers (Ziqi Pang et al., 2023)

{{}}

Ziqi Pang, Ziyang Xie, Yunze Man, Yu-Xiong Wang. (2023)  
**Frozen Transformers in Language Models Are Effective Visual Encoder Layers**

* * *

Primary Category: cs.CV  
Categories: cs-AI, cs-CL, cs-CV, cs-LG, cs.CV  
Keywords: LLaMA, Language Model, Transformer, Transformers  
[Paper Link](http://arxiv.org/abs/2310.12973v1)

* * *

**ABSTRACT**  
This paper reveals that large language models (LLMs), despite being trained solely on textual data, are surprisingly strong encoders for purely visual tasks in the absence of language. Even more intriguingly, this can be achieved by a simple yet previously overlooked strategy -- employing a frozen transformer block from pre-trained LLMs as a constituent encoder layer to directly process visual tokens. Our work pushes the boundaries of leveraging LLMs for computer vision tasks, significantly departing from conventional practices that typically necessitate a multi-modal vision-language setup with associated language prompts, inputs, or outputs. We demonstrate that our approach consistently enhances performance across a diverse range of tasks, encompassing pure 2D and 3D visual recognition tasks (e.g., image and point cloud classification), temporal modeling tasks (e.g., action recognition), non-semantic tasks (e.g., motion forecasting), and multi-modal tasks (e.g., 2D/3D visual question answering and image-text retrieval). Such improvements are a general phenomenon, applicable to various types of LLMs (e.g., LLaMA and OPT) and different LLM transformer blocks. We additionally propose the information filtering hypothesis to explain the effectiveness of pre-trained LLMs in visual encoding -- the pre-trained LLM transformer blocks discern informative visual tokens and further amplify their effect. This hypothesis is empirically supported by the observation that the feature activation, after training with LLM transformer blocks, exhibits a stronger focus on relevant regions. We hope that our work inspires new perspectives on utilizing LLMs and deepening our understanding of their underlying mechanisms. Code is available at <https://github.com/ziqipang/LM4VisualEncoding>.

{{}}

### (9/160) Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encoding (Zhejun Zhang et al., 2023)

{{}}

Zhejun Zhang, Alexander Liniger, Christos Sakaridis, Fisher Yu, Luc Van Gool. (2023)  
**Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encoding**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs-RO, cs.CV  
Keywords: Transformer, Transformers  
[Paper Link](http://arxiv.org/abs/2310.12970v1)

* * *

**ABSTRACT**  
The real-world deployment of an autonomous driving system requires its components to run on-board and in real-time, including the motion prediction module that predicts the future trajectories of surrounding traffic participants. Existing agent-centric methods have demonstrated outstanding performance on public benchmarks. However, they suffer from high computational overhead and poor scalability as the number of agents to be predicted increases. To address this problem, we introduce the K-nearest neighbor attention with relative pose encoding (KNARPE), a novel attention mechanism allowing the pairwise-relative representation to be used by Transformers. Then, based on KNARPE we present the Heterogeneous Polyline Transformer with Relative pose encoding (HPTR), a hierarchical framework enabling asynchronous token update during the online inference. By sharing contexts among agents and reusing the unchanged contexts, our approach is as efficient as scene-centric methods, while performing on par with state-of-the-art agent-centric methods. Experiments on Waymo and Argoverse-2 datasets show that HPTR achieves superior performance among end-to-end methods that do not apply expensive post-processing or model ensembling. The code is available at <https://github.com/zhejz/HPTR>.

{{}}

### (10/160) CLAIR: Evaluating Image Captions with Large Language Models (David Chan et al., 2023)

{{}}

David Chan, Suzanne Petryk, Joseph E. Gonzalez, Trevor Darrell, John Canny. (2023)  
**CLAIR: Evaluating Image Captions with Large Language Models**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs.CV  
Keywords: AI, Language Model  
[Paper Link](http://arxiv.org/abs/2310.12971v1)

* * *

**ABSTRACT**  
The evaluation of machine-generated image captions poses an interesting yet persistent challenge. Effective evaluation measures must consider numerous dimensions of similarity, including semantic relevance, visual structure, object interactions, caption diversity, and specificity. Existing highly-engineered measures attempt to capture specific aspects, but fall short in providing a holistic score that aligns closely with human judgments. Here, we propose CLAIR, a novel method that leverages the zero-shot language modeling capabilities of large language models (LLMs) to evaluate candidate captions. In our evaluations, CLAIR demonstrates a stronger correlation with human judgments of caption quality compared to existing measures. Notably, on Flickr8K-Expert, CLAIR achieves relative correlation improvements over SPICE of 39.6% and over image-augmented methods such as RefCLIP-S of 18.3%. Moreover, CLAIR provides noisily interpretable results by allowing the language model to identify the underlying reasoning behind its assigned score. Code is available at <https://davidmchan.github.io/clair/>

{{}}

### (11/160) 3D-GPT: Procedural 3D Modeling with Large Language Models (Chunyi Sun et al., 2023)

{{}}

Chunyi Sun, Junlin Han, Weijian Deng, Xinlong Wang, Zishan Qin, Stephen Gould. (2023)  
**3D-GPT: Procedural 3D Modeling with Large Language Models**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs-GR, cs-LG, cs.CV  
Keywords: GPT, Language Model  
[Paper Link](http://arxiv.org/abs/2310.12945v1)

* * *

**ABSTRACT**  
In the pursuit of efficient automated content creation, procedural generation, leveraging modifiable parameters and rule-based systems, emerges as a promising approach. Nonetheless, it could be a demanding endeavor, given its intricate nature necessitating a deep understanding of rules, algorithms, and parameters. To reduce workload, we introduce 3D-GPT, a framework utilizing large language models~(LLMs) for instruction-driven 3D modeling. 3D-GPT positions LLMs as proficient problem solvers, dissecting the procedural 3D modeling tasks into accessible segments and appointing the apt agent for each task. 3D-GPT integrates three core agents: the task dispatch agent, the conceptualization agent, and the modeling agent. They collaboratively achieve two objectives. First, it enhances concise initial scene descriptions, evolving them into detailed forms while dynamically adapting the text based on subsequent instructions. Second, it integrates procedural generation, extracting parameter values from enriched text to effortlessly interface with 3D software for asset creation. Our empirical investigations confirm that 3D-GPT not only interprets and executes instructions, delivering reliable results but also collaborates effectively with human designers. Furthermore, it seamlessly integrates with Blender, unlocking expanded manipulation possibilities. Our work highlights the potential of LLMs in 3D modeling, offering a basic framework for future advancements in scene generation and animation.

{{}}

### (12/160) Unsupervised Object Localization in the Era of Self-Supervised ViTs: A Survey (Oriane Siméoni et al., 2023)

{{}}

Oriane Siméoni, Éloi Zablocki, Spyros Gidaris, Gilles Puy, Patrick Pérez. (2023)  
**Unsupervised Object Localization in the Era of Self-Supervised ViTs: A Survey**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs.CV  
Keywords: Self-Supervised  
[Paper Link](http://arxiv.org/abs/2310.12904v1)

* * *

**ABSTRACT**  
The recent enthusiasm for open-world vision systems show the high interest of the community to perform perception tasks outside of the closed-vocabulary benchmark setups which have been so popular until now. Being able to discover objects in images/videos without knowing in advance what objects populate the dataset is an exciting prospect. But how to find objects without knowing anything about them? Recent works show that it is possible to perform class-agnostic unsupervised object localization by exploiting self-supervised pre-trained features. We propose here a survey of unsupervised object localization methods that discover objects in images without requiring any manual annotation in the era of self-supervised ViTs. We gather links of discussed methods in the repository <https://github.com/valeoai/Awesome-Unsupervised-Object-Localization>.

{{}}

### (13/160) Neural Degradation Representation Learning for All-In-One Image Restoration (Mingde Yao et al., 2023)

{{}}

Mingde Yao, Ruikang Xu, Yuanshen Guan, Jie Huang, Zhiwei Xiong. (2023)  
**Neural Degradation Representation Learning for All-In-One Image Restoration**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs.CV  
Keywords: Representation Learning  
[Paper Link](http://arxiv.org/abs/2310.12848v1)

* * *

**ABSTRACT**  
Existing methods have demonstrated effective performance on a single degradation type. In practical applications, however, the degradation is often unknown, and the mismatch between the model and the degradation will result in a severe performance drop. In this paper, we propose an all-in-one image restoration network that tackles multiple degradations. Due to the heterogeneous nature of different types of degradations, it is difficult to process multiple degradations in a single network. To this end, we propose to learn a neural degradation representation (NDR) that captures the underlying characteristics of various degradations. The learned NDR decomposes different types of degradations adaptively, similar to a neural dictionary that represents basic degradation components. Subsequently, we develop a degradation query module and a degradation injection module to effectively recognize and utilize the specific degradation based on NDR, enabling the all-in-one restoration ability for multiple degradations. Moreover, we propose a bidirectional optimization strategy to effectively drive NDR to learn the degradation representation by optimizing the degradation and restoration processes alternately. Comprehensive experiments on representative types of degradations (including noise, haze, rain, and downsampling) demonstrate the effectiveness and generalization capability of our method.

{{}}

### (14/160) 2D-3D Interlaced Transformer for Point Cloud Segmentation with Scene-Level Supervision (Cheng-Kun Yang et al., 2023)

{{}}

Cheng-Kun Yang, Min-Hung Chen, Yung-Yu Chuang, Yen-Yu Lin. (2023)  
**2D-3D Interlaced Transformer for Point Cloud Segmentation with Scene-Level Supervision**

* * *

Primary Category: cs.CV  
Categories: cs-AI, cs-CV, cs-LG, cs.CV  
Keywords: Transformer  
[Paper Link](http://arxiv.org/abs/2310.12817v1)

* * *

**ABSTRACT**  
We present a Multimodal Interlaced Transformer (MIT) that jointly considers 2D and 3D data for weakly supervised point cloud segmentation. Research studies have shown that 2D and 3D features are complementary for point cloud segmentation. However, existing methods require extra 2D annotations to achieve 2D-3D information fusion. Considering the high annotation cost of point clouds, effective 2D and 3D feature fusion based on weakly supervised learning is in great demand. To this end, we propose a transformer model with two encoders and one decoder for weakly supervised point cloud segmentation using only scene-level class tags. Specifically, the two encoders compute the self-attended features for 3D point clouds and 2D multi-view images, respectively. The decoder implements interlaced 2D-3D cross-attention and carries out implicit 2D and 3D feature fusion. We alternately switch the roles of queries and key-value pairs in the decoder layers. It turns out that the 2D and 3D features are iteratively enriched by each other. Experiments show that it performs favorably against existing weakly supervised point cloud segmentation methods by a large margin on the S3DIS and ScanNet benchmarks. The project page will be available at <https://jimmy15923.github.io/mit_web/>.

{{}}

### (15/160) Anomaly Heterogeneity Learning for Open-set Supervised Anomaly Detection (Jiawen Zhu et al., 2023)

{{}}

Jiawen Zhu, Choubo Ding, Yu Tian, Guansong Pang. (2023)  
**Anomaly Heterogeneity Learning for Open-set Supervised Anomaly Detection**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs.CV  
Keywords: Anomaly Detection  
[Paper Link](http://arxiv.org/abs/2310.12790v2)

* * *

**ABSTRACT**  
Open-set supervised anomaly detection (OSAD) - a recently emerging anomaly detection area - aims at utilizing a few samples of anomaly classes seen during training to detect unseen anomalies (i.e., samples from open-set anomaly classes), while effectively identifying the seen anomalies. Benefiting from the prior knowledge illustrated by the seen anomalies, current OSAD methods can often largely reduce false positive errors. However, these methods treat the anomaly examples as from a homogeneous distribution, rendering them less effective in generalizing to unseen anomalies that can be drawn from any distribution. In this paper, we propose to learn heterogeneous anomaly distributions using the limited anomaly examples to address this issue. To this end, we introduce a novel approach, namely Anomaly Heterogeneity Learning (AHL), that simulates a diverse set of heterogeneous (seen and unseen) anomaly distributions and then utilizes them to learn a unified heterogeneous abnormality model. Further, AHL is a generic framework that existing OSAD models can plug and play for enhancing their abnormality modeling. Extensive experiments on nine real-world anomaly detection datasets show that AHL can 1) substantially enhance different state-of-the-art (SOTA) OSAD models in detecting both seen and unseen anomalies, achieving new SOTA performance on a large set of datasets, and 2) effectively generalize to unseen anomalies in new target domains.

{{}}

### (16/160) DT/MARS-CycleGAN: Improved Object Detection for MARS Phenotyping Robot (David Liu et al., 2023)

{{}}

David Liu, Zhengkun Li, Zihao Wu, Changying Li. (2023)  
**DT/MARS-CycleGAN: Improved Object Detection for MARS Phenotyping Robot**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs.CV  
Keywords: Object Detection  
[Paper Link](http://arxiv.org/abs/2310.12787v2)

* * *

**ABSTRACT**  
Robotic crop phenotyping has emerged as a key technology to assess crops' morphological and physiological traits at scale. These phenotypical measurements are essential for developing new crop varieties with the aim of increasing productivity and dealing with environmental challenges such as climate change. However, developing and deploying crop phenotyping robots face many challenges such as complex and variable crop shapes that complicate robotic object detection, dynamic and unstructured environments that baffle robotic control, and real-time computing and managing big data that challenge robotic hardware/software. This work specifically tackles the first challenge by proposing a novel Digital-Twin(DT)MARS-CycleGAN model for image augmentation to improve our Modular Agricultural Robotic System (MARS)'s crop object detection from complex and variable backgrounds. Our core idea is that in addition to the cycle consistency losses in the CycleGAN model, we designed and enforced a new DT-MARS loss in the deep learning model to penalize the inconsistency between real crop images captured by MARS and synthesized images sensed by DT MARS. Therefore, the generated synthesized crop images closely mimic real images in terms of realism, and they are employed to fine-tune object detectors such as YOLOv8. Extensive experiments demonstrated that our new DT/MARS-CycleGAN framework significantly boosts our MARS' crop object/row detector's performance, contributing to the field of robotic crop phenotyping.

{{}}

### (17/160) Minimalist and High-Performance Semantic Segmentation with Plain Vision Transformers (Yuanduo Hong et al., 2023)

{{}}

Yuanduo Hong, Jue Wang, Weichao Sun, Huihui Pan. (2023)  
**Minimalist and High-Performance Semantic Segmentation with Plain Vision Transformers**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs.CV  
Keywords: Semantic Segmentation, Transformer, Transformers  
[Paper Link](http://arxiv.org/abs/2310.12755v1)

* * *

**ABSTRACT**  
In the wake of Masked Image Modeling (MIM), a diverse range of plain, non-hierarchical Vision Transformer (ViT) models have been pre-trained with extensive datasets, offering new paradigms and significant potential for semantic segmentation. Current state-of-the-art systems incorporate numerous inductive biases and employ cumbersome decoders. Building upon the original motivations of plain ViTs, which are simplicity and generality, we explore high-performance `minimalist' systems to this end. Our primary purpose is to provide simple and efficient baselines for practical semantic segmentation with plain ViTs. Specifically, we first explore the feasibility and methodology for achieving high-performance semantic segmentation using the last feature map. As a result, we introduce the PlainSeg, a model comprising only three 3$\times$3 convolutions in addition to the transformer layers (either encoder or decoder). In this process, we offer insights into two underlying principles: (i) high-resolution features are crucial to high performance in spite of employing simple up-sampling techniques and (ii) the slim transformer decoder requires a much larger learning rate than the wide transformer decoder. On this basis, we further present the PlainSeg-Hier, which allows for the utilization of hierarchical features. Extensive experiments on four popular benchmarks demonstrate the high performance and efficiency of our methods. They can also serve as powerful tools for assessing the transfer ability of base models in semantic segmentation. Code is available at \url{[https://github.com/ydhongHIT/PlainSeg}](https://github.com/ydhongHIT/PlainSeg%7D).

{{}}

### (18/160) Recoverable Privacy-Preserving Image Classification through Noise-like Adversarial Examples (Jun Liu et al., 2023)

{{}}

Jun Liu, Jiantao Zhou, Jinyu Tian, Weiwei Sun. (2023)  
**Recoverable Privacy-Preserving Image Classification through Noise-like Adversarial Examples**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs.CV  
Keywords: Image Classification  
[Paper Link](http://arxiv.org/abs/2310.12707v1)

* * *

**ABSTRACT**  
With the increasing prevalence of cloud computing platforms, ensuring data privacy during the cloud-based image related services such as classification has become crucial. In this study, we propose a novel privacypreserving image classification scheme that enables the direct application of classifiers trained in the plaintext domain to classify encrypted images, without the need of retraining a dedicated classifier. Moreover, encrypted images can be decrypted back into their original form with high fidelity (recoverable) using a secret key. Specifically, our proposed scheme involves utilizing a feature extractor and an encoder to mask the plaintext image through a newly designed Noise-like Adversarial Example (NAE). Such an NAE not only introduces a noise-like visual appearance to the encrypted image but also compels the target classifier to predict the ciphertext as the same label as the original plaintext image. At the decoding phase, we adopt a Symmetric Residual Learning (SRL) framework for restoring the plaintext image with minimal degradation. Extensive experiments demonstrate that 1) the classification accuracy of the classifier trained in the plaintext domain remains the same in both the ciphertext and plaintext domains; 2) the encrypted images can be recovered into their original form with an average PSNR of up to 51+ dB for the SVHN dataset and 48+ dB for the VGGFace2 dataset; 3) our system exhibits satisfactory generalization capability on the encryption, decryption and classification tasks across datasets that are different from the training one; and 4) a high-level of security is achieved against three potential threat models. The code is available at <https://github.com/csjunjun/RIC.git>.

{{}}

### (19/160) Exploiting Low-confidence Pseudo-labels for Source-free Object Detection (Zhihong Chen et al., 2023)

{{}}

Zhihong Chen, Zilei Wang, Yixin Zhang. (2023)  
**Exploiting Low-confidence Pseudo-labels for Source-free Object Detection**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs.CV  
Keywords: Contrastive Learning, Object Detection  
[Paper Link](http://arxiv.org/abs/2310.12705v1)

* * *

**ABSTRACT**  
Source-free object detection (SFOD) aims to adapt a source-trained detector to an unlabeled target domain without access to the labeled source data. Current SFOD methods utilize a threshold-based pseudo-label approach in the adaptation phase, which is typically limited to high-confidence pseudo-labels and results in a loss of information. To address this issue, we propose a new approach to take full advantage of pseudo-labels by introducing high and low confidence thresholds. Specifically, the pseudo-labels with confidence scores above the high threshold are used conventionally, while those between the low and high thresholds are exploited using the Low-confidence Pseudo-labels Utilization (LPU) module. The LPU module consists of Proposal Soft Training (PST) and Local Spatial Contrastive Learning (LSCL). PST generates soft labels of proposals for soft training, which can mitigate the label mismatch problem. LSCL exploits the local spatial relationship of proposals to improve the model's ability to differentiate between spatially adjacent proposals, thereby optimizing representational features further. Combining the two components overcomes the challenges faced by traditional methods in utilizing low-confidence pseudo-labels. Extensive experiments on five cross-domain object detection benchmarks demonstrate that our proposed method outperforms the previous SFOD methods, achieving state-of-the-art performance.

{{}}

### (20/160) Representation Learning via Consistent Assignment of Views over Random Partitions (Thalles Silva et al., 2023)

{{}}

Thalles Silva, Adín Ramírez Rivera. (2023)  
**Representation Learning via Consistent Assignment of Views over Random Partitions**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs-LG, cs.CV  
Keywords: Representation Learning  
[Paper Link](http://arxiv.org/abs/2310.12692v1)

* * *

**ABSTRACT**  
We present Consistent Assignment of Views over Random Partitions (CARP), a self-supervised clustering method for representation learning of visual features. CARP learns prototypes in an end-to-end online fashion using gradient descent without additional non-differentiable modules to solve the cluster assignment problem. CARP optimizes a new pretext task based on random partitions of prototypes that regularizes the model and enforces consistency between views' assignments. Additionally, our method improves training stability and prevents collapsed solutions in joint-embedding training. Through an extensive evaluation, we demonstrate that CARP's representations are suitable for learning downstream tasks. We evaluate CARP's representations capabilities in 17 datasets across many standard protocols, including linear evaluation, few-shot classification, k-NN, k-means, image retrieval, and copy detection. We compare CARP performance to 11 existing self-supervised methods. We extensively ablate our method and demonstrate that our proposed random partition pretext task improves the quality of the learned representations by devising multiple random classification tasks. In transfer learning tasks, CARP achieves the best performance on average against many SSL methods trained for a longer time.

{{}}

### (21/160) Heart Disease Detection using Vision-Based Transformer Models from ECG Images (Zeynep Hilal Kilimci et al., 2023)

{{}}

Zeynep Hilal Kilimci, Mustafa Yalcin, Ayhan Kucukmanisa, Amit Kumar Mishra. (2023)  
**Heart Disease Detection using Vision-Based Transformer Models from ECG Images**

* * *

Primary Category: cs.CV  
Categories: cs-AI, cs-CV, cs.CV  
Keywords: Google, Microsoft, Transformer  
[Paper Link](http://arxiv.org/abs/2310.12630v1)

* * *

**ABSTRACT**  
Heart disease, also known as cardiovascular disease, is a prevalent and critical medical condition characterized by the impairment of the heart and blood vessels, leading to various complications such as coronary artery disease, heart failure, and myocardial infarction. The timely and accurate detection of heart disease is of paramount importance in clinical practice. Early identification of individuals at risk enables proactive interventions, preventive measures, and personalized treatment strategies to mitigate the progression of the disease and reduce adverse outcomes. In recent years, the field of heart disease detection has witnessed notable advancements due to the integration of sophisticated technologies and computational approaches. These include machine learning algorithms, data mining techniques, and predictive modeling frameworks that leverage vast amounts of clinical and physiological data to improve diagnostic accuracy and risk stratification. In this work, we propose to detect heart disease from ECG images using cutting-edge technologies, namely vision transformer models. These models are Google-Vit, Microsoft-Beit, and Swin-Tiny. To the best of our knowledge, this is the initial endeavor concentrating on the detection of heart diseases through image-based ECG data by employing cuttingedge technologies namely, transformer models. To demonstrate the contribution of the proposed framework, the performance of vision transformer models are compared with state-of-the-art studies. Experiment results show that the proposed framework exhibits remarkable classification results.

{{}}

### (22/160) Cross-attention Spatio-temporal Context Transformer for Semantic Segmentation of Historical Maps (Sidi Wu et al., 2023)

{{}}

Sidi Wu, Yizi Chen, Konrad Schindler, Lorenz Hurni. (2023)  
**Cross-attention Spatio-temporal Context Transformer for Semantic Segmentation of Historical Maps**

* * *

Primary Category: cs.CV  
Categories: cs-AI, cs-CV, cs.CV  
Keywords: Semantic Segmentation, Transformer  
[Paper Link](http://arxiv.org/abs/2310.12616v1)

* * *

**ABSTRACT**  
Historical maps provide useful spatio-temporal information on the Earth's surface before modern earth observation techniques came into being. To extract information from maps, neural networks, which gain wide popularity in recent years, have replaced hand-crafted map processing methods and tedious manual labor. However, aleatoric uncertainty, known as data-dependent uncertainty, inherent in the drawing/scanning/fading defects of the original map sheets and inadequate contexts when cropping maps into small tiles considering the memory limits of the training process, challenges the model to make correct predictions. As aleatoric uncertainty cannot be reduced even with more training data collected, we argue that complementary spatio-temporal contexts can be helpful. To achieve this, we propose a U-Net-based network that fuses spatio-temporal features with cross-attention transformers (U-SpaTem), aggregating information at a larger spatial range as well as through a temporal sequence of images. Our model achieves a better performance than other state-or-art models that use either temporal or spatial contexts. Compared with pure vision transformers, our model is more lightweight and effective. To the best of our knowledge, leveraging both spatial and temporal contexts have been rarely explored before in the segmentation task. Even though our application is on segmenting historical maps, we believe that the method can be transferred into other fields with similar problems like temporal sequences of satellite images. Our code is freely accessible at <https://github.com/chenyizi086/wu.2023.sigspatial.git>.

{{}}

### (23/160) Diverse Diffusion: Enhancing Image Diversity in Text-to-Image Generation (Mariia Zameshina et al., 2023)

{{}}

Mariia Zameshina, Olivier Teytaud, Laurent Najman. (2023)  
**Diverse Diffusion: Enhancing Image Diversity in Text-to-Image Generation**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs.CV  
Keywords: AI  
[Paper Link](http://arxiv.org/abs/2310.12583v1)

* * *

**ABSTRACT**  
Latent diffusion models excel at producing high-quality images from text. Yet, concerns appear about the lack of diversity in the generated imagery. To tackle this, we introduce Diverse Diffusion, a method for boosting image diversity beyond gender and ethnicity, spanning into richer realms, including color diversity.Diverse Diffusion is a general unsupervised technique that can be applied to existing text-to-image models. Our approach focuses on finding vectors in the Stable Diffusion latent space that are distant from each other. We generate multiple vectors in the latent space until we find a set of vectors that meets the desired distance requirements and the required batch size.To evaluate the effectiveness of our diversity methods, we conduct experiments examining various characteristics, including color diversity, LPIPS metric, and ethnicity/gender representation in images featuring humans.The results of our experiments emphasize the significance of diversity in generating realistic and varied images, offering valuable insights for improving text-to-image models. Through the enhancement of image diversity, our approach contributes to the creation of more inclusive and representative AI-generated art.

{{}}

### (24/160) Weakly-Supervised Semantic Segmentation with Image-Level Labels: from Traditional Models to Foundation Models (Zhaozheng Chen et al., 2023)

{{}}

Zhaozheng Chen, Qianru Sun. (2023)  
**Weakly-Supervised Semantic Segmentation with Image-Level Labels: from Traditional Models to Foundation Models**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs.CV  
Keywords: Semantic Segmentation  
[Paper Link](http://arxiv.org/abs/2310.13026v1)

* * *

**ABSTRACT**  
The rapid development of deep learning has driven significant progress in the field of image semantic segmentation - a fundamental task in computer vision. Semantic segmentation algorithms often depend on the availability of pixel-level labels (i.e., masks of objects), which are expensive, time-consuming, and labor-intensive. Weakly-supervised semantic segmentation (WSSS) is an effective solution to avoid such labeling. It utilizes only partial or incomplete annotations and provides a cost-effective alternative to fully-supervised semantic segmentation. In this paper, we focus on the WSSS with image-level labels, which is the most challenging form of WSSS. Our work has two parts. First, we conduct a comprehensive survey on traditional methods, primarily focusing on those presented at premier research conferences. We categorize them into four groups based on where their methods operate: pixel-wise, image-wise, cross-image, and external data. Second, we investigate the applicability of visual foundation models, such as the Segment Anything Model (SAM), in the context of WSSS. We scrutinize SAM in two intriguing scenarios: text prompting and zero-shot learning. We provide insights into the potential and challenges associated with deploying visual foundational models for WSSS, facilitating future developments in this exciting research area.

{{}}

### (25/160) WeedCLR: Weed Contrastive Learning through Visual Representations with Class-Optimized Loss in Long-Tailed Datasets (Alzayat Saleh et al., 2023)

{{}}

Alzayat Saleh, Alex Olsen, Jake Wood, Bronson Philippa, Mostafa Rahimi Azghadi. (2023)  
**WeedCLR: Weed Contrastive Learning through Visual Representations with Class-Optimized Loss in Long-Tailed Datasets**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs.CV  
Keywords: Contrastive Learning  
[Paper Link](http://arxiv.org/abs/2310.12465v1)

* * *

**ABSTRACT**  
Image classification is a crucial task in modern weed management and crop intervention technologies. However, the limited size, diversity, and balance of existing weed datasets hinder the development of deep learning models for generalizable weed identification. In addition, the expensive labelling requirements of mainstream fully-supervised weed classifiers make them cost- and time-prohibitive to deploy widely, for new weed species, and in site-specific weed management. This paper proposes a novel method for Weed Contrastive Learning through visual Representations (WeedCLR), that uses class-optimized loss with Von Neumann Entropy of deep representation for weed classification in long-tailed datasets. WeedCLR leverages self-supervised learning to learn rich and robust visual features without any labels and applies a class-optimized loss function to address the class imbalance problem in long-tailed datasets. WeedCLR is evaluated on two public weed datasets: CottonWeedID15, containing 15 weed species, and DeepWeeds, containing 8 weed species. WeedCLR achieves an average accuracy improvement of 4.3% on CottonWeedID15 and 5.6% on DeepWeeds over previous methods. It also demonstrates better generalization ability and robustness to different environmental conditions than existing methods without the need for expensive and time-consuming human annotations. These significant improvements make WeedCLR an effective tool for weed classification in long-tailed datasets and allows for more rapid and widespread deployment of site-specific weed management and crop intervention technologies.

{{}}

### (26/160) Not Just Learning from Others but Relying on Yourself: A New Perspective on Few-Shot Segmentation in Remote Sensing (Hanbo Bi et al., 2023)

{{}}

Hanbo Bi, Yingchao Feng, Zhiyuan Yan, Yongqiang Mao, Wenhui Diao, Hongqi Wang, Xian Sun. (2023)  
**Not Just Learning from Others but Relying on Yourself: A New Perspective on Few-Shot Segmentation in Remote Sensing**

* * *

Primary Category: cs.CV  
Categories: cs-CV, cs.CV  
Keywords: AI, Few-Shot  
[Paper Link](http://arxiv.org/abs/2310.12452v1)

* * *

**ABSTRACT**  
Few-shot segmentation (FSS) is proposed to segment unknown class targets with just a few annotated samples. Most current FSS methods follow the paradigm of mining the semantics from the support images to guide the query image segmentation. However, such a pattern of `learning from others' struggles to handle the extreme intra-class variation, preventing FSS from being directly generalized to remote sensing scenes. To bridge the gap of intra-class variance, we develop a Dual-Mining network named DMNet for cross-image mining and self-mining, meaning that it no longer focuses solely on support images but pays more attention to the query image itself. Specifically, we propose a Class-public Region Mining (CPRM) module to effectively suppress irrelevant feature pollution by capturing the common semantics between the support-query image pair. The Class-specific Region Mining (CSRM) module is then proposed to continuously mine the class-specific semantics of the query image itself in a `filtering' and `purifying' manner. In addition, to prevent the co-existence of multiple classes in remote sensing scenes from exacerbating the collapse of FSS generalization, we also propose a new Known-class Meta Suppressor (KMS) module to suppress the activation of known-class objects in the sample. Extensive experiments on the iSAID and LoveDA remote sensing datasets have demonstrated that our method sets the state-of-the-art with a minimum number of model parameters. Significantly, our model with the backbone of Resnet-50 achieves the mIoU of 49.58% and 51.34% on iSAID under 1-shot and 5-shot settings, outperforming the state-of-the-art method by 1.8% and 1.12%, respectively. The code is publicly available at <https://github.com/HanboBizl/DMNet>.

{{}}

### (27/160) DocXChain: A Powerful Open-Source Toolchain for Document Parsing and Beyond (Cong Yao, 2023)

{{}}

Cong Yao. (2023)  
**DocXChain: A Powerful Open-Source Toolchain for Document Parsing and Beyond**

* * *

Primary Category: cs.CV  
Categories: cs-CL, cs-CV, cs.CV  
Keywords: ChatGPT, GPT  
[Paper Link](http://arxiv.org/abs/2310.12430v1)

* * *

**ABSTRACT**  
In this report, we introduce DocXChain, a powerful open-source toolchain for document parsing, which is designed and developed to automatically convert the rich information embodied in unstructured documents, such as text, tables and charts, into structured representations that are readable and manipulable by machines. Specifically, basic capabilities, including text detection, text recognition, table structure recognition and layout analysis, are provided. Upon these basic capabilities, we also build a set of fully functional pipelines for document parsing, i.e., general text reading, table parsing, and document structurization, to drive various applications related to documents in real-world scenarios. Moreover, DocXChain is concise, modularized and flexible, such that it can be readily integrated with existing tools, libraries or models (such as LangChain and ChatGPT), to construct more powerful systems that can accomplish more complicated and challenging tasks. The code of DocXChain is publicly available at:~\url{[https://github.com/AlibabaResearch/AdvancedLiterateMachinery/tree/main/Applications/DocXChain}](https://github.com/AlibabaResearch/AdvancedLiterateMachinery/tree/main/Applications/DocXChain%7D)

{{}}

## cs.LG (30)

### (28/160) Heterogeneous Graph Neural Networks for Data-driven Traffic Assignment (Tong Liu et al., 2023)

{{}}

Tong Liu, Hadi Meidani. (2023)  
**Heterogeneous Graph Neural Networks for Data-driven Traffic Assignment**

* * *

Primary Category: cs.LG  
Categories: cs-LG, cs.LG  
Keywords: Graph Neural Network, Graph Neural Networks  
[Paper Link](http://arxiv.org/abs/2310.13193v1)

* * *

**ABSTRACT**  
The traffic assignment problem is one of the significant components of traffic flow analysis for which various solution approaches have been proposed. However, deploying these approaches for large-scale networks poses significant challenges. In this paper, we leverage the power of heterogeneous graph neural networks to propose a novel data-driven approach for traffic assignment and traffic flow learning. The proposed model is capable of capturing spatial traffic patterns across different links, yielding highly accurate results. We present numerical experiments on urban transportation networks and show that the proposed heterogeneous graph neural network model outperforms other conventional neural network models in terms of convergence rate, training loss, and prediction accuracy. Notably, the proposed heterogeneous graph neural network model can also be generalized to different network topologies. This approach offers a promising solution for complex traffic flow analysis and prediction, enhancing our understanding and management of a wide range of transportation systems.

{{}}

### (29/160) Graph Neural Networks with polynomial activations have limited expressivity (Sammy Khalife, 2023)

{{}}

Sammy Khalife. (2023)  
**Graph Neural Networks with polynomial activations have limited expressivity**

* * *

Primary Category: cs.LG  
Categories: cs-LG, cs.LG  
Keywords: GNN, Graph Neural Network, Graph Neural Networks  
[Paper Link](http://arxiv.org/abs/2310.13139v1)

* * *

**ABSTRACT**  
The expressivity of Graph Neural Networks (GNNs) can be entirely characterized by appropriate fragments of the first order logic. Namely, any query of the two variable fragment of graded modal logic (GC2) interpreted over labelled graphs can be expressed using a GNN whose size depends only on the depth of the query. As pointed out by [Barcelo & Al., 2020, Grohe, 2021 ], this description holds for a family of activation functions, leaving the possibibility for a hierarchy of logics expressible by GNNs depending on the chosen activation function. In this article, we show that such hierarchy indeed exists by proving that GC2 queries cannot be expressed by GNNs with polynomial activation functions. This implies a separation between polynomial and popular non polynomial activations (such as ReLUs, sigmoid and hyperbolic tan and others) and answers an open question formulated by [Grohe, 2021].

{{}}

### (30/160) Understanding Addition in Transformers (Philip Quirke et al., 2023)

{{}}

Philip Quirke, Fazl Barez. (2023)  
**Understanding Addition in Transformers**

* * *

Primary Category: cs.LG  
Categories: cs-AI, cs-LG, cs.LG  
Keywords: AI, Transformer, Transformers  
[Paper Link](http://arxiv.org/abs/2310.13121v2)

* * *

**ABSTRACT**  
Understanding the inner workings of machine learning models like Transformers is vital for their safe and ethical use. This paper presents an in-depth analysis of a one-layer Transformer model trained for integer addition. We reveal that the model divides the task into parallel, digit-specific streams and employs distinct algorithms for different digit positions. Our study also finds that the model starts calculations late but executes them rapidly. A rare use case with high loss is identified and explained. Overall, the model's algorithm is explained in detail. These findings are validated through rigorous testing and mathematical modeling, contributing to the broader works in Mechanistic Interpretability, AI safety, and alignment. Our approach opens the door for analyzing more complex tasks and multi-layer Transformer models.

{{}}

### (31/160) Semi-Supervised Learning of Dynamical Systems with Neural Ordinary Differential Equations: A Teacher-Student Model Approach (Yu Wang et al., 2023)

{{}}

Yu Wang, Yuxuan Yin, Karthik Somayaji Nanjangud Suryanarayana, Jan Drgona, Malachi Schram, Mahantesh Halappanavar, Frank Liu, Peng Li. (2023)  
**Semi-Supervised Learning of Dynamical Systems with Neural Ordinary Differential Equations: A Teacher-Student Model Approach**

* * *

Primary Category: cs.LG  
Categories: cs-AI, cs-LG, cs.LG  
Keywords: Semi-Supervised  
[Paper Link](http://arxiv.org/abs/2310.13110v1)

* * *

**ABSTRACT**  
Modeling dynamical systems is crucial for a wide range of tasks, but it remains challenging due to complex nonlinear dynamics, limited observations, or lack of prior knowledge. Recently, data-driven approaches such as Neural Ordinary Differential Equations (NODE) have shown promising results by leveraging the expressive power of neural networks to model unknown dynamics. However, these approaches often suffer from limited labeled training data, leading to poor generalization and suboptimal predictions. On the other hand, semi-supervised algorithms can utilize abundant unlabeled data and have demonstrated good performance in classification and regression tasks. We propose TS-NODE, the first semi-supervised approach to modeling dynamical systems with NODE. TS-NODE explores cheaply generated synthetic pseudo rollouts to broaden exploration in the state space and to tackle the challenges brought by lack of ground-truth system data under a teacher-student model. TS-NODE employs an unified optimization framework that corrects the teacher model based on the student's feedback while mitigating the potential false system dynamics present in pseudo rollouts. TS-NODE demonstrates significant performance improvements over a baseline Neural ODE model on multiple dynamical system modeling tasks.

{{}}

### (32/160) SRAI: Towards Standardization of Geospatial AI (Piotr Gramacki et al., 2023)

{{}}

Piotr Gramacki, Kacper Leśniara, Kamil Raczycki, Szymon Woźniak, Marcin Przymus, Piotr Szymański. (2023)  
**SRAI: Towards Standardization of Geospatial AI**

* * *

Primary Category: cs.LG  
Categories: cs-LG, cs.LG  
Keywords: AI  
[Paper Link](http://arxiv.org/abs/2310.13098v2)

* * *

**ABSTRACT**  
Spatial Representations for Artificial Intelligence (srai) is a Python library for working with geospatial data. The library can download geospatial data, split a given area into micro-regions using multiple algorithms and train an embedding model using various architectures. It includes baseline models as well as more complex methods from published works. Those capabilities make it possible to use srai in a complete pipeline for geospatial task solving. The proposed library is the first step to standardize the geospatial AI domain toolset. It is fully open-source and published under Apache 2.0 licence.

{{}}

### (33/160) Unsupervised Representation Learning to Aid Semi-Supervised Meta Learning (Atik Faysal et al., 2023)

{{}}

Atik Faysal, Mohammad Rostami, Huaxia Wang, Avimanyu Sahoo, Ryan Antle. (2023)  
**Unsupervised Representation Learning to Aid Semi-Supervised Meta Learning**

* * *

Primary Category: cs.LG  
Categories: cs-AI, cs-LG, cs.LG  
Keywords: Representation Learning, Semi-Supervised  
[Paper Link](http://arxiv.org/abs/2310.13085v1)

* * *

**ABSTRACT**  
Few-shot learning or meta-learning leverages the data scarcity problem in machine learning. Traditionally, training data requires a multitude of samples and labeling for supervised learning. To address this issue, we propose a one-shot unsupervised meta-learning to learn the latent representation of the training samples. We use augmented samples as the query set during the training phase of the unsupervised meta-learning. A temperature-scaled cross-entropy loss is used in the inner loop of meta-learning to prevent overfitting during unsupervised learning. The learned parameters from this step are applied to the targeted supervised meta-learning in a transfer-learning fashion for initialization and fast adaptation with improved accuracy. The proposed method is model agnostic and can aid any meta-learning model to improve accuracy. We use model agnostic meta-learning (MAML) and relation network (RN) on Omniglot and mini-Imagenet datasets to demonstrate the performance of the proposed method. Furthermore, a meta-learning model with the proposed initialization can achieve satisfactory accuracy with significantly fewer training samples.

{{}}

### (34/160) Robust multimodal models have outlier features and encode more concepts (Jonathan Crabbé et al., 2023)

{{}}

Jonathan Crabbé, Pau Rodríguez, Vaishaal Shankar, Luca Zappella, Arno Blaas. (2023)  
**Robust multimodal models have outlier features and encode more concepts**

* * *

Primary Category: cs.LG  
Categories: cs-AI, cs-CV, cs-LG, cs.LG  
Keywords: AI  
[Paper Link](http://arxiv.org/abs/2310.13040v1)

* * *

**ABSTRACT**  
What distinguishes robust models from non-robust ones? This question has gained traction with the appearance of large-scale multimodal models, such as CLIP. These models have demonstrated unprecedented robustness with respect to natural distribution shifts. While it has been shown that such differences in robustness can be traced back to differences in training data, so far it is not known what that translates to in terms of what the model has learned. In this work, we bridge this gap by probing the representation spaces of 12 robust multimodal models with various backbones (ResNets and ViTs) and pretraining sets (OpenAI, LAION-400M, LAION-2B, YFCC15M, CC12M and DataComp). We find two signatures of robustness in the representation spaces of these models: (1) Robust models exhibit outlier features characterized by their activations, with some being several orders of magnitude above average. These outlier features induce privileged directions in the model's representation space. We demonstrate that these privileged directions explain most of the predictive power of the model by pruning up to $80 %$ of the least important representation space directions without negative impacts on model accuracy and robustness; (2) Robust models encode substantially more concepts in their representation space. While this superposition of concepts allows robust models to store much information, it also results in highly polysemantic features, which makes their interpretation challenging. We discuss how these insights pave the way for future research in various fields, such as model pruning and mechanistic interpretability.

{{}}

### (35/160) Does Your Model Think Like an Engineer? Explainable AI for Bearing Fault Detection with Deep Learning (Thomas Decker et al., 2023)

{{}}

Thomas Decker, Michael Lebacher, Volker Tresp. (2023)  
**Does Your Model Think Like an Engineer? Explainable AI for Bearing Fault Detection with Deep Learning**

* * *

Primary Category: cs.LG  
Categories: cs-AI, cs-LG, cs.LG  
Keywords: AI  
[Paper Link](http://arxiv.org/abs/2310.12967v1)

* * *

**ABSTRACT**  
Deep Learning has already been successfully applied to analyze industrial sensor data in a variety of relevant use cases. However, the opaque nature of many well-performing methods poses a major obstacle for real-world deployment. Explainable AI (XAI) and especially feature attribution techniques promise to enable insights about how such models form their decision. But the plain application of such methods often fails to provide truly informative and problem-specific insights to domain experts. In this work, we focus on the specific task of detecting faults in rolling element bearings from vibration signals. We propose a novel and domain-specific feature attribution framework that allows us to evaluate how well the underlying logic of a model corresponds with expert reasoning. Utilizing the framework we are able to validate the trustworthiness and to successfully anticipate the generalization ability of different well-performing deep learning models. Our methodology demonstrates how signal processing tools can effectively be used to enhance Explainable AI techniques and acts as a template for similar problems.

{{}}

### (36/160) Eureka-Moments in Transformers: Multi-Step Tasks Reveal Softmax Induced Optimization Problems (David T. Hoffmann et al., 2023)

{{}}

David T. Hoffmann, Simon Schrodi, Nadine Behrmann, Volker Fischer, Thomas Brox. (2023)  
**Eureka-Moments in Transformers: Multi-Step Tasks Reveal Softmax Induced Optimization Problems**

* * *

Primary Category: cs.LG  
Categories: cs-AI, cs-CV, cs-LG, cs.LG  
Keywords: Transformer, Transformers  
[Paper Link](http://arxiv.org/abs/2310.12956v1)

* * *

**ABSTRACT**  
In this work, we study rapid, step-wise improvements of the loss in transformers when being confronted with multi-step decision tasks. We found that transformers struggle to learn the intermediate tasks, whereas CNNs have no such issue on the tasks we studied. When transformers learn the intermediate task, they do this rapidly and unexpectedly after both training and validation loss saturated for hundreds of epochs. We call these rapid improvements Eureka-moments, since the transformer appears to suddenly learn a previously incomprehensible task. Similar leaps in performance have become known as Grokking. In contrast to Grokking, for Eureka-moments, both the validation and the training loss saturate before rapidly improving. We trace the problem back to the Softmax function in the self-attention block of transformers and show ways to alleviate the problem. These fixes improve training speed. The improved models reach 95% of the baseline model in just 20% of training steps while having a much higher likelihood to learn the intermediate task, lead to higher final accuracy and are more robust to hyper-parameters.

{{}}

### (37/160) Towards Robust Offline Reinforcement Learning under Diverse Data Corruption (Rui Yang et al., 2023)

{{}}

Rui Yang, Han Zhong, Jiawei Xu, Amy Zhang, Chongjie Zhang, Lei Han, Tong Zhang. (2023)  
**Towards Robust Offline Reinforcement Learning under Diverse Data Corruption**

* * *

Primary Category: cs.LG  
Categories: cs-AI, cs-LG, cs.LG  
Keywords: Reinforcement Learning  
[Paper Link](http://arxiv.org/abs/2310.12955v1)

* * *

**ABSTRACT**  
Offline reinforcement learning (RL) presents a promising approach for learning reinforced policies from offline datasets without the need for costly or unsafe interactions with the environment. However, datasets collected by humans in real-world environments are often noisy and may even be maliciously corrupted, which can significantly degrade the performance of offline RL. In this work, we first investigate the performance of current offline RL algorithms under comprehensive data corruption, including states, actions, rewards, and dynamics. Our extensive experiments reveal that implicit Q-learning (IQL) demonstrates remarkable resilience to data corruption among various offline RL algorithms. Furthermore, we conduct both empirical and theoretical analyses to understand IQL's robust performance, identifying its supervised policy learning scheme as the key factor. Despite its relative robustness, IQL still suffers from heavy-tail targets of Q functions under dynamics corruption. To tackle this challenge, we draw inspiration from robust statistics to employ the Huber loss to handle the heavy-tailedness and utilize quantile estimators to balance penalization for corrupted data and learning stability. By incorporating these simple yet effective modifications into IQL, we propose a more robust offline RL approach named Robust IQL (RIQL). Extensive experiments demonstrate that RIQL exhibits highly robust performance when subjected to diverse data corruption scenarios.

{{}}

### (38/160) The Foundation Model Transparency Index (Rishi Bommasani et al., 2023)

{{}}

Rishi Bommasani, Kevin Klyman, Shayne Longpre, Sayash Kapoor, Nestor Maslej, Betty Xiong, Daniel Zhang, Percy Liang. (2023)  
**The Foundation Model Transparency Index**

* * *

Primary Category: cs.LG  
Categories: cs-AI, cs-LG, cs.LG  
Keywords: AI, GPT, GPT-4, Google, PaLM  
[Paper Link](http://arxiv.org/abs/2310.12941v1)

* * *

**ABSTRACT**  
Foundation models have rapidly permeated society, catalyzing a wave of generative AI applications spanning enterprise and consumer-facing contexts. While the societal impact of foundation models is growing, transparency is on the decline, mirroring the opacity that has plagued past digital technologies (e.g. social media). Reversing this trend is essential: transparency is a vital precondition for public accountability, scientific innovation, and effective governance. To assess the transparency of the foundation model ecosystem and help improve transparency over time, we introduce the Foundation Model Transparency Index. The Foundation Model Transparency Index specifies 100 fine-grained indicators that comprehensively codify transparency for foundation models, spanning the upstream resources used to build a foundation model (e.g data, labor, compute), details about the model itself (e.g. size, capabilities, risks), and the downstream use (e.g. distribution channels, usage policies, affected geographies). We score 10 major foundation model developers (e.g. OpenAI, Google, Meta) against the 100 indicators to assess their transparency. To facilitate and standardize assessment, we score developers in relation to their practices for their flagship foundation model (e.g. GPT-4 for OpenAI, PaLM 2 for Google, Llama 2 for Meta). We present 10 top-level findings about the foundation model ecosystem: for example, no developer currently discloses significant information about the downstream impact of its flagship model, such as the number of users, affected market sectors, or how users can seek redress for harm. Overall, the Foundation Model Transparency Index establishes the level of transparency today to drive progress on foundation model governance via industry standards and regulatory intervention.

{{}}

### (39/160) Probabilistic Modeling of Human Teams to Infer False Beliefs (Paulo Soares et al., 2023)

{{}}

Paulo Soares, Adarsh Pyarelal, Kobus Barnard. (2023)  
**Probabilistic Modeling of Human Teams to Infer False Beliefs**

* * *

Primary Category: cs.LG  
Categories: cs-LG, cs.LG  
Keywords: AI  
[Paper Link](http://arxiv.org/abs/2310.12929v1)

* * *

**ABSTRACT**  
We develop a probabilistic graphical model (PGM) for artificially intelligent (AI) agents to infer human beliefs during a simulated urban search and rescue (USAR) scenario executed in a Minecraft environment with a team of three players. The PGM approach makes observable states and actions explicit, as well as beliefs and intentions grounded by evidence about what players see and do over time. This approach also supports inferring the effect of interventions, which are vital if AI agents are to assist human teams. The experiment incorporates manipulations of players' knowledge, and the virtual Minecraft-based testbed provides access to several streams of information, including the objects in the players' field of view. The participants are equipped with a set of marker blocks that can be placed near room entrances to signal the presence or absence of victims in the rooms to their teammates. In each team, one of the members is given a different legend for the markers than the other two, which may mislead them about the state of the rooms; that is, they will hold a false belief. We extend previous works in this field by introducing ToMCAT, an AI agent that can reason about individual and shared mental states. We find that the players' behaviors are affected by what they see in their in-game field of view, their beliefs about the meaning of the markers, and their beliefs about which meaning the team decided to adopt. In addition, we show that ToMCAT's beliefs are consistent with the players' actions and that it can infer false beliefs with accuracy significantly better than chance and comparable to inferences made by human observers.

{{}}

### (40/160) Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning (Juan Rocamonde et al., 2023)

{{}}

Juan Rocamonde, Victoriano Montesinos, Elvis Nava, Ethan Perez, David Lindner. (2023)  
**Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning**

* * *

Primary Category: cs.LG  
Categories: cs-AI, cs-LG, cs.LG  
Keywords: Language Model, Reinforcement Learning, Zero-Shot  
[Paper Link](http://arxiv.org/abs/2310.12921v1)

* * *

**ABSTRACT**  
Reinforcement learning (RL) requires either manually specifying a reward function, which is often infeasible, or learning a reward model from a large amount of human feedback, which is often very expensive. We study a more sample-efficient alternative: using pretrained vision-language models (VLMs) as zero-shot reward models (RMs) to specify tasks via natural language. We propose a natural and general approach to using VLMs as reward models, which we call VLM-RMs. We use VLM-RMs based on CLIP to train a MuJoCo humanoid to learn complex tasks without a manually specified reward function, such as kneeling, doing the splits, and sitting in a lotus position. For each of these tasks, we only provide a single sentence text prompt describing the desired task with minimal prompt engineering. We provide videos of the trained agents at: <https://sites.google.com/view/vlm-rm>. We can improve performance by providing a second ``baseline'' prompt and projecting out parts of the CLIP embedding space irrelevant to distinguish between goal and baseline. Further, we find a strong scaling effect for VLM-RMs: larger VLMs trained with more compute and data are better reward models. The failure modes of VLM-RMs we encountered are all related to known capability limitations of current VLMs, such as limited spatial reasoning ability or visually unrealistic environments that are far off-distribution for the VLM. We find that VLM-RMs are remarkably robust as long as the VLM is large enough. This suggests that future VLMs will become more and more useful reward models for a wide range of RL applications.

{{}}

### (41/160) Causal-structure Driven Augmentations for Text OOD Generalization (Amir Feder et al., 2023)

{{}}

Amir Feder, Yoav Wald, Claudia Shi, Suchi Saria, David Blei. (2023)  
**Causal-structure Driven Augmentations for Text OOD Generalization**

* * *

Primary Category: cs.LG  
Categories: cs-CL, cs-LG, cs.LG  
Keywords: Augmentation  
[Paper Link](http://arxiv.org/abs/2310.12803v1)

* * *

**ABSTRACT**  
The reliance of text classifiers on spurious correlations can lead to poor generalization at deployment, raising concerns about their use in safety-critical domains such as healthcare. In this work, we propose to use counterfactual data augmentation, guided by knowledge of the causal structure of the data, to simulate interventions on spurious features and to learn more robust text classifiers. We show that this strategy is appropriate in prediction problems where the label is spuriously correlated with an attribute. Under the assumptions of such problems, we discuss the favorable sample complexity of counterfactual data augmentation, compared to importance re-weighting. Pragmatically, we match examples using auxiliary data, based on diff-in-diff methodology, and use a large language model (LLM) to represent a conditional probability of text. Through extensive experimentation on learning caregiver-invariant predictors of clinical diagnoses from medical narratives and on semi-synthetic data, we demonstrate that our method for simulating interventions improves out-of-distribution (OOD) accuracy compared to baseline invariant learning algorithms.

{{}}

### (42/160) Exploring Graph Neural Networks for Indian Legal Judgment Prediction (Mann Khatri et al., 2023)

{{}}

Mann Khatri, Mirza Yusuf, Yaman Kumar, Rajiv Ratn Shah, Ponnurangam Kumaraguru. (2023)  
**Exploring Graph Neural Networks for Indian Legal Judgment Prediction**

* * *

Primary Category: cs.LG  
Categories: cs-AI, cs-LG, cs.LG  
Keywords: Graph Neural Network, Graph Neural Networks, Legal  
[Paper Link](http://arxiv.org/abs/2310.12800v1)

* * *

**ABSTRACT**  
The burdensome impact of a skewed judges-to-cases ratio on the judicial system manifests in an overwhelming backlog of pending cases alongside an ongoing influx of new ones. To tackle this issue and expedite the judicial process, the proposition of an automated system capable of suggesting case outcomes based on factual evidence and precedent from past cases gains significance. This research paper centres on developing a graph neural network-based model to address the Legal Judgment Prediction (LJP) problem, recognizing the intrinsic graph structure of judicial cases and making it a binary node classification problem. We explored various embeddings as model features, while nodes such as time nodes and judicial acts were added and pruned to evaluate the model's performance. The study is done while considering the ethical dimension of fairness in these predictions, considering gender and name biases. A link prediction task is also conducted to assess the model's proficiency in anticipating connections between two specified nodes. By harnessing the capabilities of graph neural networks and incorporating fairness analyses, this research aims to contribute insights towards streamlining the adjudication process, enhancing judicial efficiency, and fostering a more equitable legal landscape, ultimately alleviating the strain imposed by mounting case backlogs. Our best-performing model with XLNet pre-trained embeddings as its features gives the macro F1 score of 75% for the LJP task. For link prediction, the same set of features is the best performing giving ROC of more than 80%

{{}}

### (43/160) Agri-GNN: A Novel Genotypic-Topological Graph Neural Network Framework Built on GraphSAGE for Optimized Yield Prediction (Aditya Gupta et al., 2023)

{{}}

Aditya Gupta, Asheesh Singh. (2023)  
**Agri-GNN: A Novel Genotypic-Topological Graph Neural Network Framework Built on GraphSAGE for Optimized Yield Prediction**

* * *

Primary Category: cs.LG  
Categories: cs-AI, cs-LG, cs.LG  
Keywords: GNN, Graph Neural Network, Graph Neural Networks  
[Paper Link](http://arxiv.org/abs/2310.13037v1)

* * *

**ABSTRACT**  
Agriculture, as the cornerstone of human civilization, constantly seeks to integrate technology for enhanced productivity and sustainability. This paper introduces $\textit{Agri-GNN}$, a novel Genotypic-Topological Graph Neural Network Framework tailored to capture the intricate spatial and genotypic interactions of crops, paving the way for optimized predictions of harvest yields. $\textit{Agri-GNN}$ constructs a Graph $\mathcal{G}$ that considers farming plots as nodes, and then methodically constructs edges between nodes based on spatial and genotypic similarity, allowing for the aggregation of node information through a genotypic-topological filter. Graph Neural Networks (GNN), by design, consider the relationships between data points, enabling them to efficiently model the interconnected agricultural ecosystem. By harnessing the power of GNNs, $\textit{Agri-GNN}$ encapsulates both local and global information from plants, considering their inherent connections based on spatial proximity and shared genotypes, allowing stronger predictions to be made than traditional Machine Learning architectures. $\textit{Agri-GNN}$ is built from the GraphSAGE architecture, because of its optimal calibration with large graphs, like those of farming plots and breeding experiments. $\textit{Agri-GNN}$ experiments, conducted on a comprehensive dataset of vegetation indices, time, genotype information, and location data, demonstrate that $\textit{Agri-GNN}$ achieves an $R^2 = .876$ in yield predictions for farming fields in Iowa. The results show significant improvement over the baselines and other work in the field. $\textit{Agri-GNN}$ represents a blueprint for using advanced graph-based neural architectures to predict crop yield, providing significant improvements over baselines in the field.

{{}}

### (44/160) TabuLa: Harnessing Language Models for Tabular Data Synthesis (Zilong Zhao et al., 2023)

{{}}

Zilong Zhao, Robert Birke, Lydia Chen. (2023)  
**TabuLa: Harnessing Language Models for Tabular Data Synthesis**

* * *

Primary Category: cs.LG  
Categories: cs-LG, cs.LG  
Keywords: Language Model, NLP  
[Paper Link](http://arxiv.org/abs/2310.12746v1)

* * *

**ABSTRACT**  
Given the ubiquitous use of tabular data in industries and the growing concerns in data privacy and security, tabular data synthesis emerges as a critical research area. The recent state-of-the-art methods show that large language models (LLMs) can be adopted to generate realistic tabular data. As LLMs pre-process tabular data as full text, they have the advantage of avoiding the curse of dimensionality associated with one-hot encoding high-dimensional data. However, their long training time and limited re-usability on new tasks prevent them from replacing exiting tabular generative models. In this paper, we propose Tabula, a tabular data synthesizer based on the language model structure. Through Tabula, we demonstrate the inherent limitation of employing pre-trained language models designed for natural language processing (NLP) in the context of tabular data synthesis. Our investigation delves into the development of a dedicated foundational model tailored specifically for tabular data synthesis. Additionally, we propose a token sequence compression strategy to significantly reduce training time while preserving the quality of synthetic data. Extensive experiments on six datasets demonstrate that using a language model structure without loading the well-trained model weights yields a better starting model for tabular data synthesis. Moreover, the Tabula model, previously trained on other tabular data, serves as an excellent foundation model for new tabular data synthesis tasks. Additionally, the token sequence compression method substantially reduces the model's training time. Results show that Tabula averagely reduces 46.2% training time per epoch comparing to current LLMs-based state-of-the-art algorithm and consistently achieves even higher synthetic data utility.

{{}}

### (45/160) Learn from the Past: A Proxy based Adversarial Defense Framework to Boost Robustness (Yaohua Liu et al., 2023)

{{}}

Yaohua Liu, Jiaxin Gao, Zhu Liu, Xianghao Jiao, Xin Fan, Risheng Liu. (2023)  
**Learn from the Past: A Proxy based Adversarial Defense Framework to Boost Robustness**

* * *

Primary Category: cs.LG  
Categories: cs-LG, cs.LG  
Keywords: Adversarial Training  
[Paper Link](http://arxiv.org/abs/2310.12713v1)

* * *

**ABSTRACT**  
In light of the vulnerability of deep learning models to adversarial samples and the ensuing security issues, a range of methods, including Adversarial Training (AT) as a prominent representative, aimed at enhancing model robustness against various adversarial attacks, have seen rapid development. However, existing methods essentially assist the current state of target model to defend against parameter-oriented adversarial attacks with explicit or implicit computation burdens, which also suffers from unstable convergence behavior due to inconsistency of optimization trajectories. Diverging from previous work, this paper reconsiders the update rule of target model and corresponding deficiency to defend based on its current state. By introducing the historical state of the target model as a proxy, which is endowed with much prior information for defense, we formulate a two-stage update rule, resulting in a general adversarial defense framework, which we refer to as `LAST' ({\bf L}earn from the P{\bf ast}). Besides, we devise a Self Distillation (SD) based defense objective to constrain the update process of the proxy model without the introduction of larger teacher models. Experimentally, we demonstrate consistent and significant performance enhancements by refining a series of single-step and multi-step AT methods (e.g., up to $\bf 9.2%$ and $\bf 20.5%$ improvement of Robust Accuracy (RA) on CIFAR10 and CIFAR100 datasets, respectively) across various datasets, backbones and attack modalities, and validate its ability to enhance training stability and ameliorate catastrophic overfitting issues meanwhile.

{{}}

### (46/160) On the Optimization and Generalization of Multi-head Attention (Puneesh Deora et al., 2023)

{{}}

Puneesh Deora, Rouzbeh Ghaderi, Hossein Taheri, Christos Thrampoulidis. (2023)  
**On the Optimization and Generalization of Multi-head Attention**

* * *

Primary Category: cs.LG  
Categories: cs-LG, cs.LG, math-OC, stat-ML  
Keywords: Attention, Transformer  
[Paper Link](http://arxiv.org/abs/2310.12680v1)

* * *

**ABSTRACT**  
The training and generalization dynamics of the Transformer's core mechanism, namely the Attention mechanism, remain under-explored. Besides, existing analyses primarily focus on single-head attention. Inspired by the demonstrated benefits of overparameterization when training fully-connected networks, we investigate the potential optimization and generalization advantages of using multiple attention heads. Towards this goal, we derive convergence and generalization guarantees for gradient-descent training of a single-layer multi-head self-attention model, under a suitable realizability condition on the data. We then establish primitive conditions on the initialization that ensure realizability holds. Finally, we demonstrate that these conditions are satisfied for a simple tokenized-mixture model. We expect the analysis can be extended to various data-model and architecture variations.

{{}}

### (47/160) Neural networks for insurance pricing with frequency and severity data: a benchmark study from data preprocessing to technical tariff (Freek Holvoet et al., 2023)

{{}}

Freek Holvoet, Katrien Antonio, Roel Henckaerts. (2023)  
**Neural networks for insurance pricing with frequency and severity data: a benchmark study from data preprocessing to technical tariff**

* * *

Primary Category: cs.LG  
Categories: cs-LG, cs.LG, q-fin-RM  
Keywords: GLM  
[Paper Link](http://arxiv.org/abs/2310.12671v1)

* * *

**ABSTRACT**  
Insurers usually turn to generalized linear models for modelling claim frequency and severity data. Due to their success in other fields, machine learning techniques are gaining popularity within the actuarial toolbox. Our paper contributes to the literature on frequency-severity insurance pricing with machine learning via deep learning structures. We present a benchmark study on four insurance data sets with frequency and severity targets in the presence of multiple types of input features. We compare in detail the performance of: a generalized linear model on binned input data, a gradient-boosted tree model, a feed-forward neural network (FFNN), and the combined actuarial neural network (CANN). Our CANNs combine a baseline prediction established with a GLM and GBM, respectively, with a neural network correction. We explain the data preprocessing steps with specific focus on the multiple types of input features typically present in tabular insurance data sets, such as postal codes, numeric and categorical covariates. Autoencoders are used to embed the categorical variables into the neural network and we explore their potential advantages in a frequency-severity setting. Finally, we construct global surrogate models for the neural nets' frequency and severity models. These surrogates enable the translation of the essential insights captured by the FFNNs or CANNs to GLMs. As such, a technical tariff table results that can easily be deployed in practice.

{{}}

### (48/160) WeaveNet for Approximating Two-sided Matching Problems (Shusaku Sone et al., 2023)

{{}}

Shusaku Sone, Jiaxin Ma, Atsushi Hashimoto, Naoya Chiba, Yoshitaka Ushiku. (2023)  
**WeaveNet for Approximating Two-sided Matching Problems**

* * *

Primary Category: cs.LG  
Categories: cs-LG, cs.LG  
Keywords: GNN  
[Paper Link](http://arxiv.org/abs/2310.12515v1)

* * *

**ABSTRACT**  
Matching, a task to optimally assign limited resources under constraints, is a fundamental technology for society. The task potentially has various objectives, conditions, and constraints; however, the efficient neural network architecture for matching is underexplored. This paper proposes a novel graph neural network (GNN), \textit{WeaveNet}, designed for bipartite graphs. Since a bipartite graph is generally dense, general GNN architectures lose node-wise information by over-smoothing when deeply stacked. Such a phenomenon is undesirable for solving matching problems. WeaveNet avoids it by preserving edge-wise information while passing messages densely to reach a better solution. To evaluate the model, we approximated one of the \textit{strongly NP-hard} problems, \textit{fair stable matching}. Despite its inherent difficulties and the network's general purpose design, our model reached a comparative performance with state-of-the-art algorithms specially designed for stable matching for small numbers of agents.

{{}}

### (49/160) SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation (Chongyu Fan et al., 2023)

{{}}

Chongyu Fan, Jiancheng Liu, Yihua Zhang, Dennis Wei, Eric Wong, Sijia Liu. (2023)  
**SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation**

* * *

Primary Category: cs.LG  
Categories: cs-AI, cs-LG, cs.LG  
Keywords: AI, Image Classification  
[Paper Link](http://arxiv.org/abs/2310.12508v1)

* * *

**ABSTRACT**  
With evolving data regulations, machine unlearning (MU) has become an important tool for fostering trust and safety in today's AI models. However, existing MU methods focusing on data and/or weight perspectives often grapple with limitations in unlearning accuracy, stability, and cross-domain applicability. To address these challenges, we introduce the concept of 'weight saliency' in MU, drawing parallels with input saliency in model explanation. This innovation directs MU's attention toward specific model weights rather than the entire model, improving effectiveness and efficiency. The resultant method that we call saliency unlearning (SalUn) narrows the performance gap with 'exact' unlearning (model retraining from scratch after removing the forgetting dataset). To the best of our knowledge, SalUn is the first principled MU approach adaptable enough to effectively erase the influence of forgetting data, classes, or concepts in both image classification and generation. For example, SalUn yields a stability advantage in high-variance random data forgetting, e.g., with a 0.2% gap compared to exact unlearning on the CIFAR-10 dataset. Moreover, in preventing conditional diffusion models from generating harmful images, SalUn achieves nearly 100% unlearning accuracy, outperforming current state-of-the-art baselines like Erased Stable Diffusion and Forget-Me-Not.

{{}}

### (50/160) SDGym: Low-Code Reinforcement Learning Environments using System Dynamics Models (Emmanuel Klu et al., 2023)

{{}}

Emmanuel Klu, Sameer Sethi, DJ Passey, Donald Martin Jr. (2023)  
**SDGym: Low-Code Reinforcement Learning Environments using System Dynamics Models**

* * *

Primary Category: cs.LG  
Categories: cs-LG, cs.LG  
Keywords: AI, Reinforcement Learning  
[Paper Link](http://arxiv.org/abs/2310.12494v1)

* * *

**ABSTRACT**  
Understanding the long-term impact of algorithmic interventions on society is vital to achieving responsible AI. Traditional evaluation strategies often fall short due to the complex, adaptive and dynamic nature of society. While reinforcement learning (RL) can be a powerful approach for optimizing decisions in dynamic settings, the difficulty of realistic environment design remains a barrier to building robust agents that perform well in practical settings. To address this issue we tap into the field of system dynamics (SD) as a complementary method that incorporates collaborative simulation model specification practices. We introduce SDGym, a low-code library built on the OpenAI Gym framework which enables the generation of custom RL environments based on SD simulation models. Through a feasibility study we validate that well specified, rich RL environments can be generated from preexisting SD models and a few lines of configuration code. We demonstrate the capabilities of the SDGym environment using an SD model of the electric vehicle adoption problem. We compare two SD simulators, PySD and BPTK-Py for parity, and train a D4PG agent using the Acme framework to showcase learning and environment interaction. Our preliminary findings underscore the dual potential of SD to improve RL environment design and for RL to improve dynamic policy discovery within SD models. By open-sourcing SDGym, the intent is to galvanize further research and promote adoption across the SD and RL communities, thereby catalyzing collaboration in this emerging interdisciplinary space.

{{}}

### (51/160) Improved Operator Learning by Orthogonal Attention (Zipeng Xiao et al., 2023)

{{}}

Zipeng Xiao, Zhongkai Hao, Bokai Lin, Zhijie Deng, Hang Su. (2023)  
**Improved Operator Learning by Orthogonal Attention**

* * *

Primary Category: cs.LG  
Categories: cs-LG, cs.LG  
Keywords: Attention  
[Paper Link](http://arxiv.org/abs/2310.12487v2)

* * *

**ABSTRACT**  
Neural operators, as an efficient surrogate model for learning the solutions of PDEs, have received extensive attention in the field of scientific machine learning. Among them, attention-based neural operators have become one of the mainstreams in related research. However, existing approaches overfit the limited training data due to the considerable number of parameters in the attention mechanism. To address this, we develop an orthogonal attention based on the eigendecomposition of the kernel integral operator and the neural approximation of eigenfunctions. The orthogonalization naturally poses a proper regularization effect on the resulting neural operator, which aids in resisting overfitting and boosting generalization. Experiments on six standard neural operator benchmark datasets comprising both regular and irregular geometries show that our method can outperform competing baselines with decent margins.

{{}}

### (52/160) Unmasking Transformers: A Theoretical Approach to Data Recovery via Attention Weights (Yichuan Deng et al., 2023)

{{}}

Yichuan Deng, Zhao Song, Shenghao Xie, Chiwun Yang. (2023)  
**Unmasking Transformers: A Theoretical Approach to Data Recovery via Attention Weights**

* * *

Primary Category: cs.LG  
Categories: cs-CL, cs-LG, cs.LG, stat-ML  
Keywords: Attention, Transformer, Transformers  
[Paper Link](http://arxiv.org/abs/2310.12462v1)

* * *

**ABSTRACT**  
In the realm of deep learning, transformers have emerged as a dominant architecture, particularly in natural language processing tasks. However, with their widespread adoption, concerns regarding the security and privacy of the data processed by these models have arisen. In this paper, we address a pivotal question: Can the data fed into transformers be recovered using their attention weights and outputs? We introduce a theoretical framework to tackle this problem. Specifically, we present an algorithm that aims to recover the input data $X \in \mathbb{R}^{d \times n}$ from given attention weights $W = QK^\top \in \mathbb{R}^{d \times d}$ and output $B \in \mathbb{R}^{n \times n}$ by minimizing the loss function $L(X)$. This loss function captures the discrepancy between the expected output and the actual output of the transformer. Our findings have significant implications for the Localized Layer-wise Mechanism (LLM), suggesting potential vulnerabilities in the model's design from a security and privacy perspective. This work underscores the importance of understanding and safeguarding the internal workings of transformers to ensure the confidentiality of processed data.

{{}}

### (53/160) MuseGNN: Interpretable and Convergent Graph Neural Network Layers at Scale (Haitian Jiang et al., 2023)

{{}}

Haitian Jiang, Renjie Liu, Xiao Yan, Zhenkun Cai, Minjie Wang, David Wipf. (2023)  
**MuseGNN: Interpretable and Convergent Graph Neural Network Layers at Scale**

* * *

Primary Category: cs.LG  
Categories: cs-LG, cs.LG  
Keywords: GNN, Graph Neural Network  
[Paper Link](http://arxiv.org/abs/2310.12457v1)

* * *

**ABSTRACT**  
Among the many variants of graph neural network (GNN) architectures capable of modeling data with cross-instance relations, an important subclass involves layers designed such that the forward pass iteratively reduces a graph-regularized energy function of interest. In this way, node embeddings produced at the output layer dually serve as both predictive features for solving downstream tasks (e.g., node classification) and energy function minimizers that inherit desirable inductive biases and interpretability. However, scaling GNN architectures constructed in this way remains challenging, in part because the convergence of the forward pass may involve models with considerable depth. To tackle this limitation, we propose a sampling-based energy function and scalable GNN layers that iteratively reduce it, guided by convergence guarantees in certain settings. We also instantiate a full GNN architecture based on these designs, and the model achieves competitive accuracy and scalability when applied to the largest publicly-available node classification benchmark exceeding 1TB in size.

{{}}

### (54/160) MTS-LOF: Medical Time-Series Representation Learning via Occlusion-Invariant Features (Huayu Li et al., 2023)

{{}}

Huayu Li, Ana S. Carreon-Rascon, Xiwen Chen, Geng Yuan, Ao Li. (2023)  
**MTS-LOF: Medical Time-Series Representation Learning via Occlusion-Invariant Features**

* * *

Primary Category: cs.LG  
Categories: cs-AI, cs-LG, cs.LG  
Keywords: Representation Learning, Time Series  
[Paper Link](http://arxiv.org/abs/2310.12451v1)

* * *

**ABSTRACT**  
Medical time series data are indispensable in healthcare, providing critical insights for disease diagnosis, treatment planning, and patient management. The exponential growth in data complexity, driven by advanced sensor technologies, has presented challenges related to data labeling. Self-supervised learning (SSL) has emerged as a transformative approach to address these challenges, eliminating the need for extensive human annotation. In this study, we introduce a novel framework for Medical Time Series Representation Learning, known as MTS-LOF. MTS-LOF leverages the strengths of contrastive learning and Masked Autoencoder (MAE) methods, offering a unique approach to representation learning for medical time series data. By combining these techniques, MTS-LOF enhances the potential of healthcare applications by providing more sophisticated, context-rich representations. Additionally, MTS-LOF employs a multi-masking strategy to facilitate occlusion-invariant feature learning. This approach allows the model to create multiple views of the data by masking portions of it. By minimizing the discrepancy between the representations of these masked patches and the fully visible patches, MTS-LOF learns to capture rich contextual information within medical time series datasets. The results of experiments conducted on diverse medical time series datasets demonstrate the superiority of MTS-LOF over other methods. These findings hold promise for significantly enhancing healthcare applications by improving representation learning. Furthermore, our work delves into the integration of joint-embedding SSL and MAE techniques, shedding light on the intricate interplay between temporal and structural dependencies in healthcare data. This understanding is crucial, as it allows us to grasp the complexities of healthcare data analysis.

{{}}

### (55/160) CAT: Closed-loop Adversarial Training for Safe End-to-End Driving (Linrui Zhang et al., 2023)

{{}}

Linrui Zhang, Zhenghao Peng, Quanyi Li, Bolei Zhou. (2023)  
**CAT: Closed-loop Adversarial Training for Safe End-to-End Driving**

* * *

Primary Category: cs.LG  
Categories: cs-LG, cs.LG  
Keywords: Adversarial Training  
[Paper Link](http://arxiv.org/abs/2310.12432v1)

* * *

**ABSTRACT**  
Driving safety is a top priority for autonomous vehicles. Orthogonal to prior work handling accident-prone traffic events by algorithm designs at the policy level, we investigate a Closed-loop Adversarial Training (CAT) framework for safe end-to-end driving in this paper through the lens of environment augmentation. CAT aims to continuously improve the safety of driving agents by training the agent on safety-critical scenarios that are dynamically generated over time. A novel resampling technique is developed to turn log-replay real-world driving scenarios into safety-critical ones via probabilistic factorization, where the adversarial traffic generation is modeled as the multiplication of standard motion prediction sub-problems. Consequently, CAT can launch more efficient physical attacks compared to existing safety-critical scenario generation methods and yields a significantly less computational cost in the iterative learning pipeline. We incorporate CAT into the MetaDrive simulator and validate our approach on hundreds of driving scenarios imported from real-world driving datasets. Experimental results demonstrate that CAT can effectively generate adversarial scenarios countering the agent being trained. After training, the agent can achieve superior driving safety in both log-replay and safety-critical traffic scenarios on the held-out test set. Code and data are available at <https://metadriverse.github.io/cat>.

{{}}

### (56/160) Detecting and Mitigating Algorithmic Bias in Binary Classification using Causal Modeling (Wendy Hui et al., 2023)

{{}}

Wendy Hui, Wai Kwong Lau. (2023)  
**Detecting and Mitigating Algorithmic Bias in Binary Classification using Causal Modeling**

* * *

Primary Category: cs.LG  
Categories: cs-CY, cs-LG, cs.LG  
Keywords: Bias  
[Paper Link](http://arxiv.org/abs/2310.12421v1)

* * *

**ABSTRACT**  
This paper proposes the use of causal modeling to detect and mitigate algorithmic bias. We provide a brief description of causal modeling and a general overview of our approach. We then use the Adult dataset, which is available for download from the UC Irvine Machine Learning Repository, to develop (1) a prediction model, which is treated as a black box, and (2) a causal model for bias mitigation. In this paper, we focus on gender bias and the problem of binary classification. We show that gender bias in the prediction model is statistically significant at the 0.05 level. We demonstrate the effectiveness of the causal model in mitigating gender bias by cross-validation. Furthermore, we show that the overall classification accuracy is improved slightly. Our novel approach is intuitive, easy-to-use, and can be implemented using existing statistical software tools such as "lavaan" in R. Hence, it enhances explainability and promotes trust.

{{}}

### (57/160) Cooperative Minibatching in Graph Neural Networks (Muhammed Fatih Balin et al., 2023)

{{}}

Muhammed Fatih Balin, Dominique LaSalle, Ümit V. Çatalyürek. (2023)  
**Cooperative Minibatching in Graph Neural Networks**

* * *

Primary Category: cs.LG  
Categories: cs-DC, cs-LG, cs.LG  
Keywords: GNN, Graph Neural Network, Graph Neural Networks  
[Paper Link](http://arxiv.org/abs/2310.12403v2)

* * *

**ABSTRACT**  
Significant computational resources are required to train Graph Neural Networks (GNNs) at a large scale, and the process is highly data-intensive. One of the most effective ways to reduce resource requirements is minibatch training coupled with graph sampling. GNNs have the unique property that items in a minibatch have overlapping data. However, the commonly implemented Independent Minibatching approach assigns each Processing Element (PE) its own minibatch to process, leading to duplicated computations and input data access across PEs. This amplifies the Neighborhood Explosion Phenomenon (NEP), which is the main bottleneck limiting scaling. To reduce the effects of NEP in the multi-PE setting, we propose a new approach called Cooperative Minibatching. Our approach capitalizes on the fact that the size of the sampled subgraph is a concave function of the batch size, leading to significant reductions in the amount of work per seed vertex as batch sizes increase. Hence, it is favorable for processors equipped with a fast interconnect to work on a large minibatch together as a single larger processor, instead of working on separate smaller minibatches, even though global batch size is identical. We also show how to take advantage of the same phenomenon in serial execution by generating dependent consecutive minibatches. Our experimental evaluations show up to 4x bandwidth savings for fetching vertex embeddings, by simply increasing this dependency without harming model convergence. Combining our proposed approaches, we achieve up to 64% speedup over Independent Minibatching on single-node multi-GPU systems.

{{}}

## cs.AI (5)

### (58/160) The opaque law of artificial intelligence (Vincenzo Calderonio, 2023)

{{}}

Vincenzo Calderonio. (2023)  
**The opaque law of artificial intelligence**

* * *

Primary Category: cs.AI  
Categories: F-0; I-2; J-4; K-4; K-5, cs-AI, cs.AI  
Keywords: AI, GPT, NLP  
[Paper Link](http://arxiv.org/abs/2310.13192v1)

* * *

**ABSTRACT**  
The purpose of this paper is to analyse the opacity of algorithms, contextualized in the open debate on responsibility for artificial intelligence causation; with an experimental approach by which, applying the proposed conversational methodology of the Turing Test, we expect to evaluate the performance of one of the best existing NLP model of generative AI (Chat-GPT) to see how far it can go right now and how the shape of a legal regulation of it could be. The analysis of the problem will be supported by a comment of Italian classical law categories such as causality, intent and fault to understand the problem of the usage of AI, focusing in particular on the human-machine interaction. On the computer science side, for a technical point of view of the logic used to craft these algorithms, in the second chapter will be proposed a practical interrogation of Chat-GPT aimed at finding some critical points of the functioning of AI. The end of the paper will concentrate on some existing legal solutions which can be applied to the problem, plus a brief description of the approach proposed by EU Artificial Intelligence act.

{{}}

### (59/160) Safe RLHF: Safe Reinforcement Learning from Human Feedback (Josef Dai et al., 2023)

{{}}

Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, Yaodong Yang. (2023)  
**Safe RLHF: Safe Reinforcement Learning from Human Feedback**

* * *

Primary Category: cs.AI  
Categories: cs-AI, cs-LG, cs.AI  
Keywords: AI, Reinforcement Learning  
[Paper Link](http://arxiv.org/abs/2310.12773v1)

* * *

**ABSTRACT**  
With the development of large language models (LLMs), striking a balance between the performance and safety of AI systems has never been more critical. However, the inherent tension between the objectives of helpfulness and harmlessness presents a significant challenge during LLM training. To address this issue, we propose Safe Reinforcement Learning from Human Feedback (Safe RLHF), a novel algorithm for human value alignment. Safe RLHF explicitly decouples human preferences regarding helpfulness and harmlessness, effectively avoiding the crowdworkers' confusion about the tension and allowing us to train separate reward and cost models. We formalize the safety concern of LLMs as an optimization task of maximizing the reward function while satisfying specified cost constraints. Leveraging the Lagrangian method to solve this constrained problem, Safe RLHF dynamically adjusts the balance between the two objectives during fine-tuning. Through a three-round fine-tuning using Safe RLHF, we demonstrate a superior ability to mitigate harmful responses while enhancing model performance compared to existing value-aligned algorithms. Experimentally, we fine-tuned the Alpaca-7B using Safe RLHF and aligned it with collected human preferences, significantly improving its helpfulness and harmlessness according to human evaluations.

{{}}

### (60/160) PSYCHIC: A Neuro-Symbolic Framework for Knowledge Graph Question-Answering Grounding (Hanna Abi Akl, 2023)

{{}}

Hanna Abi Akl. (2023)  
**PSYCHIC: A Neuro-Symbolic Framework for Knowledge Graph Question-Answering Grounding**

* * *

Primary Category: cs.AI  
Categories: cs-AI, cs.AI  
Keywords: Knowledge Graph, QA, Question Answering  
[Paper Link](http://arxiv.org/abs/2310.12638v1)

* * *

**ABSTRACT**  
The Scholarly Question Answering over Linked Data (Scholarly QALD) at The International Semantic Web Conference (ISWC) 2023 challenge presents two sub-tasks to tackle question answering (QA) over knowledge graphs (KGs). We answer the KGQA over DBLP (DBLP-QUAD) task by proposing a neuro-symbolic (NS) framework based on PSYCHIC, an extractive QA model capable of identifying the query and entities related to a KG question. Our system achieved a F1 score of 00.18% on question answering and came in third place for entity linking (EL) with a score of 71.00%.

{{}}

### (61/160) Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark (Jiaming Ji et al., 2023)

{{}}

Jiaming Ji, Borong Zhang, Jiayi Zhou, Xuehai Pan, Weidong Huang, Ruiyang Sun, Yiran Geng, Yifan Zhong, Juntao Dai, Yaodong Yang. (2023)  
**Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark**

* * *

Primary Category: cs.AI  
Categories: cs-AI, cs-LG, cs.AI  
Keywords: AI, Reinforcement Learning  
[Paper Link](http://arxiv.org/abs/2310.12567v1)

* * *

**ABSTRACT**  
Artificial intelligence (AI) systems possess significant potential to drive societal progress. However, their deployment often faces obstacles due to substantial safety concerns. Safe reinforcement learning (SafeRL) emerges as a solution to optimize policies while simultaneously adhering to multiple constraints, thereby addressing the challenge of integrating reinforcement learning in safety-critical scenarios. In this paper, we present an environment suite called Safety-Gymnasium, which encompasses safety-critical tasks in both single and multi-agent scenarios, accepting vector and vision-only input. Additionally, we offer a library of algorithms named Safe Policy Optimization (SafePO), comprising 16 state-of-the-art SafeRL algorithms. This comprehensive library can serve as a validation tool for the research community. By introducing this benchmark, we aim to facilitate the evaluation and comparison of safety performance, thus fostering the development of reinforcement learning for safer, more reliable, and responsible real-world applications. The website of this project can be accessed at <https://sites.google.com/view/safety-gymnasium>.

{{}}

### (62/160) GPT-4 Doesn't Know It's Wrong: An Analysis of Iterative Prompting for Reasoning Problems (Kaya Stechly et al., 2023)

{{}}

Kaya Stechly, Matthew Marquez, Subbarao Kambhampati. (2023)  
**GPT-4 Doesn't Know It's Wrong: An Analysis of Iterative Prompting for Reasoning Problems**

* * *

Primary Category: cs.AI  
Categories: cs-AI, cs.AI  
Keywords: GPT, GPT-4, Language Model, Reasoning  
[Paper Link](http://arxiv.org/abs/2310.12397v1)

* * *

**ABSTRACT**  
There has been considerable divergence of opinion on the reasoning abilities of Large Language Models (LLMs). While the initial optimism that reasoning might emerge automatically with scale has been tempered thanks to a slew of counterexamples, a wide spread belief in their iterative self-critique capabilities persists. In this paper, we set out to systematically investigate the effectiveness of iterative prompting of LLMs in the context of Graph Coloring, a canonical NP-complete reasoning problem that is related to propositional satisfiability as well as practical problems like scheduling and allocation. We present a principled empirical study of the performance of GPT4 in solving graph coloring instances or verifying the correctness of candidate colorings. In iterative modes, we experiment with the model critiquing its own answers and an external correct reasoner verifying proposed solutions. In both cases, we analyze whether the content of the criticisms actually affects bottom line performance. The study seems to indicate that (i) LLMs are bad at solving graph coloring instances (ii) they are no better at verifying a solution--and thus are not effective in iterative modes with LLMs critiquing LLM-generated solutions (iii) the correctness and content of the criticisms--whether by LLMs or external solvers--seems largely irrelevant to the performance of iterative prompting. We show that the observed increase in effectiveness is largely due to the correct solution being fortuitously present in the top-k completions of the prompt (and being recognized as such by an external verifier). Our results thus call into question claims about the self-critiquing capabilities of state of the art LLMs.

{{}}

## cs.CL (59)

### (63/160) Towards Robust Pruning: An Adaptive Knowledge-Retention Pruning Strategy for Language Models (Jianwei Li et al., 2023)

{{}}

Jianwei Li, Qi Lei, Wei Cheng, Dongkuan Xu. (2023)  
**Towards Robust Pruning: An Adaptive Knowledge-Retention Pruning Strategy for Language Models**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs.CL  
Keywords: BERT, Language Model, Pruning  
[Paper Link](http://arxiv.org/abs/2310.13191v1)

* * *

**ABSTRACT**  
The pruning objective has recently extended beyond accuracy and sparsity to robustness in language models. Despite this, existing methods struggle to enhance robustness against adversarial attacks when continually increasing model sparsity and require a retraining process. As humans step into the era of large language models, these issues become increasingly prominent. This paper proposes that the robustness of language models is proportional to the extent of pre-trained knowledge they encompass. Accordingly, we introduce a post-training pruning strategy designed to faithfully replicate the embedding space and feature space of dense language models, aiming to conserve more pre-trained knowledge during the pruning process. In this setup, each layer's reconstruction error not only originates from itself but also includes cumulative error from preceding layers, followed by an adaptive rectification. Compared to other state-of-art baselines, our approach demonstrates a superior balance between accuracy, sparsity, robustness, and pruning cost with BERT on datasets SST2, IMDB, and AGNews, marking a significant stride towards robust pruning in language models.

{{}}

### (64/160) Fast and Accurate Factual Inconsistency Detection Over Long Documents (Barrett Martin Lattimer et al., 2023)

{{}}

Barrett Martin Lattimer, Patrick Chen, Xinyuan Zhang, Yi Yang. (2023)  
**Fast and Accurate Factual Inconsistency Detection Over Long Documents**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs.CL  
Keywords: AI, Generative AI, NLI, Natural Language Inference  
[Paper Link](http://arxiv.org/abs/2310.13189v2)

* * *

**ABSTRACT**  
Generative AI models exhibit remarkable potential; however, hallucinations across various tasks present a significant challenge, particularly for longer inputs that current approaches struggle to address effectively. We introduce SCALE (Source Chunking Approach for Large-scale inconsistency Evaluation), a task-agnostic model for detecting factual inconsistencies using a novel chunking strategy. Specifically, SCALE is a Natural Language Inference (NLI) based model that uses large text chunks to condition over long texts. This approach achieves state-of-the-art performance in factual inconsistency detection for diverse tasks and long inputs. Additionally, we leverage the chunking mechanism and employ a novel algorithm to explain SCALE's decisions through relevant source sentence retrieval. Our evaluations reveal that SCALE outperforms existing methods on both standard benchmarks and a new long-form dialogue dataset ScreenEval we constructed. Moreover, SCALE surpasses competitive systems in efficiency and model explanation evaluations. We have released our code and data publicly to GitHub.

{{}}

### (65/160) CLIFT: Analysing Natural Distribution Shift on Question Answering Models in Clinical Domain (Ankit Pal, 2023)

{{}}

Ankit Pal. (2023)  
**CLIFT: Analysing Natural Distribution Shift on Question Answering Models in Clinical Domain**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs-CY, cs-DC, cs-LG, cs.CL  
Keywords: Clinical, QA, Question Answering  
[Paper Link](http://arxiv.org/abs/2310.13146v1)

* * *

**ABSTRACT**  
This paper introduces a new testbed CLIFT (Clinical Shift) for the clinical domain Question-answering task. The testbed includes 7.5k high-quality question answering samples to provide a diverse and reliable benchmark. We performed a comprehensive experimental study and evaluated several QA deep-learning models under the proposed testbed. Despite impressive results on the original test set, the performance degrades when applied to new test sets, which shows the distribution shift. Our findings emphasize the need for and the potential for increasing the robustness of clinical domain models under distributional shifts. The testbed offers one way to track progress in that direction. It also highlights the necessity of adopting evaluation metrics that consider robustness to natural distribution shifts. We plan to expand the corpus by adding more samples and model results. The full paper and the updated benchmark are available at github.com/openlifescience-ai/clift

{{}}

### (66/160) Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries (Yiqiao Jin et al., 2023)

{{}}

Yiqiao Jin, Mohit Chandra, Gaurav Verma, Yibo Hu, Munmun De Choudhury, Srijan Kumar. (2023)  
**Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs.CL  
Keywords: Language Model  
[Paper Link](http://arxiv.org/abs/2310.13132v2)

* * *

**ABSTRACT**  
Large language models (LLMs) are transforming the ways the general public accesses and consumes information. Their influence is particularly pronounced in pivotal sectors like healthcare, where lay individuals are increasingly appropriating LLMs as conversational agents for everyday queries. While LLMs demonstrate impressive language understanding and generation proficiencies, concerns regarding their safety remain paramount in these high-stake domains. Moreover, the development of LLMs is disproportionately focused on English. It remains unclear how these LLMs perform in the context of non-English languages, a gap that is critical for ensuring equity in the real-world use of these systems.This paper provides a framework to investigate the effectiveness of LLMs as multi-lingual dialogue systems for healthcare queries. Our empirically-derived framework XlingEval focuses on three fundamental criteria for evaluating LLM responses to naturalistic human-authored health-related questions: correctness, consistency, and verifiability. Through extensive experiments on four major global languages, including English, Spanish, Chinese, and Hindi, spanning three expert-annotated large health Q&A datasets, and through an amalgamation of algorithmic and human-evaluation strategies, we found a pronounced disparity in LLM responses across these languages, indicating a need for enhanced cross-lingual capabilities. We further propose XlingHealth, a cross-lingual benchmark for examining the multilingual capabilities of LLMs in the healthcare context. Our findings underscore the pressing need to bolster the cross-lingual capacities of these models, and to provide an equitable information ecosystem accessible to all.

{{}}

### (67/160) Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Models (Zhihan Zhang et al., 2023)

{{}}

Zhihan Zhang, Shuohang Wang, Wenhao Yu, Yichong Xu, Dan Iter, Qingkai Zeng, Yang Liu, Chenguang Zhu, Meng Jiang. (2023)  
**Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Models**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: Language Model, NLP  
[Paper Link](http://arxiv.org/abs/2310.13127v1)

* * *

**ABSTRACT**  
Large language models (LLMs) can perform a wide range of tasks by following natural language instructions, without the necessity of task-specific fine-tuning. Unfortunately, the performance of LLMs is greatly influenced by the quality of these instructions, and manually writing effective instructions for each task is a laborious and subjective process. In this paper, we introduce Auto-Instruct, a novel method to automatically improve the quality of instructions provided to LLMs. Our method leverages the inherent generative ability of LLMs to produce diverse candidate instructions for a given task, and then ranks them using a scoring model trained on a variety of 575 existing NLP tasks. In experiments on 118 out-of-domain tasks, Auto-Instruct surpasses both human-written instructions and existing baselines of LLM-generated instructions. Furthermore, our method exhibits notable generalizability even with other LLMs that are not incorporated into its training process.

{{}}

### (68/160) Do Language Models Learn about Legal Entity Types during Pretraining? (Claire Barale et al., 2023)

{{}}

Claire Barale, Michael Rovatsos, Nehal Bhuta. (2023)  
**Do Language Models Learn about Legal Entity Types during Pretraining?**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: BERT, Language Model, Legal, NLP, QA  
[Paper Link](http://arxiv.org/abs/2310.13092v1)

* * *

**ABSTRACT**  
Language Models (LMs) have proven their ability to acquire diverse linguistic knowledge during the pretraining phase, potentially serving as a valuable source of incidental supervision for downstream tasks. However, there has been limited research conducted on the retrieval of domain-specific knowledge, and specifically legal knowledge. We propose to explore the task of Entity Typing, serving as a proxy for evaluating legal knowledge as an essential aspect of text comprehension, and a foundational task to numerous downstream legal NLP applications. Through systematic evaluation and analysis and two types of prompting (cloze sentences and QA-based templates) and to clarify the nature of these acquired cues, we compare diverse types and lengths of entities both general and domain-specific entities, semantics or syntax signals, and different LM pretraining corpus (generic and legal-oriented) and architectures (encoder BERT-based and decoder-only with Llama2). We show that (1) Llama2 performs well on certain entities and exhibits potential for substantial improvement with optimized prompt templates, (2) law-oriented LMs show inconsistent performance, possibly due to variations in their training corpus, (3) LMs demonstrate the ability to type entities even in the case of multi-token entities, (4) all models struggle with entities belonging to sub-domains of the law (5) Llama2 appears to frequently overlook syntactic cues, a shortcoming less present in BERT-based architectures.

{{}}

### (69/160) From Multilingual Complexity to Emotional Clarity: Leveraging Commonsense to Unveil Emotions in Code-Mixed Dialogues (Shivani Kumar et al., 2023)

{{}}

Shivani Kumar, Ramaneswaran S, Md Shad Akhtar, Tanmoy Chakraborty. (2023)  
**From Multilingual Complexity to Emotional Clarity: Leveraging Commonsense to Unveil Emotions in Code-Mixed Dialogues**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs.CL  
Keywords: Dialog, Dialogue, Emotion Recognition, Multilingual, NLP  
[Paper Link](http://arxiv.org/abs/2310.13080v1)

* * *

**ABSTRACT**  
Understanding emotions during conversation is a fundamental aspect of human communication, driving NLP research for Emotion Recognition in Conversation (ERC). While considerable research has focused on discerning emotions of individual speakers in monolingual dialogues, understanding the emotional dynamics in code-mixed conversations has received relatively less attention. This motivates our undertaking of ERC for code-mixed conversations in this study. Recognizing that emotional intelligence encompasses a comprehension of worldly knowledge, we propose an innovative approach that integrates commonsense information with dialogue context to facilitate a deeper understanding of emotions. To achieve this, we devise an efficient pipeline that extracts relevant commonsense from existing knowledge graphs based on the code-mixed input. Subsequently, we develop an advanced fusion technique that seamlessly combines the acquired commonsense information with the dialogue representation obtained from a dedicated dialogue understanding module. Our comprehensive experimentation showcases the substantial performance improvement obtained through the systematic incorporation of commonsense in ERC. Both quantitative assessments and qualitative analyses further corroborate the validity of our hypothesis, reaffirming the pivotal role of commonsense integration in enhancing ERC.

{{}}

### (70/160) GARI: Graph Attention for Relative Isomorphism of Arabic Word Embeddings (Muhammad Asif Ali et al., 2023)

{{}}

Muhammad Asif Ali, Maha Alshmrani, Jianbin Qin, Yan Hu, Di Wang. (2023)  
**GARI: Graph Attention for Relative Isomorphism of Arabic Word Embeddings**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: Attention, Embedding, NLP, Word Embedding  
[Paper Link](http://arxiv.org/abs/2310.13068v1)

* * *

**ABSTRACT**  
Bilingual Lexical Induction (BLI) is a core challenge in NLP, it relies on the relative isomorphism of individual embedding spaces. Existing attempts aimed at controlling the relative isomorphism of different embedding spaces fail to incorporate the impact of semantically related words in the model training objective. To address this, we propose GARI that combines the distributional training objectives with multiple isomorphism losses guided by the graph attention network. GARI considers the impact of semantical variations of words in order to define the relative isomorphism of the embedding spaces. Experimental evaluation using the Arabic language data set shows that GARI outperforms the existing research by improving the average P@1 by a relative score of up to 40.95% and 76.80% for in-domain and domain mismatch settings respectively. We release the codes for GARI at <https://github.com/asif6827/GARI>.

{{}}

### (71/160) AutoMix: Automatically Mixing Language Models (Aman Madaan et al., 2023)

{{}}

Aman Madaan, Pranjal Aggarwal, Ankit Anand, Srividya Pranavi Potharaju, Swaroop Mishra, Pei Zhou, Aditya Gupta, Dheeraj Rajagopal, Karthik Kappaganthu, Yiming Yang, Shyam Upadhyay, Mausam, Manaal Faruqui. (2023)  
**AutoMix: Automatically Mixing Language Models**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs.CL  
Keywords: Language Model  
[Paper Link](http://arxiv.org/abs/2310.12963v1)

* * *

**ABSTRACT**  
Large language models (LLMs) are now available in various sizes and configurations from cloud API providers. While this diversity offers a broad spectrum of choices, effectively leveraging the options to optimize computational cost and performance remains challenging. In this work, we present AutoMix, an approach that strategically routes queries to larger LMs, based on the approximate correctness of outputs from a smaller LM. Central to AutoMix is a few-shot self-verification mechanism, which estimates the reliability of its own outputs without requiring training. Given that verifications can be noisy, we employ a meta verifier in AutoMix to refine the accuracy of these assessments. Our experiments using LLAMA2-13/70B, on five context-grounded reasoning datasets demonstrate that AutoMix surpasses established baselines, improving the incremental benefit per cost by up to 89%. Our code and data are available at <https://github.com/automix-llm/automix>.

{{}}

### (72/160) An Emulator for Fine-Tuning Large Language Models using Small Language Models (Eric Mitchell et al., 2023)

{{}}

Eric Mitchell, Rafael Rafailov, Archit Sharma, Chelsea Finn, Christopher D. Manning. (2023)  
**An Emulator for Fine-Tuning Large Language Models using Small Language Models**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs-LG, cs.CL  
Keywords: Falcon, Language Model  
[Paper Link](http://arxiv.org/abs/2310.12962v1)

* * *

**ABSTRACT**  
Widely used language models (LMs) are typically built by scaling up a two-stage training pipeline: a pre-training stage that uses a very large, diverse dataset of text and a fine-tuning (sometimes, 'alignment') stage that uses targeted examples or other specifications of desired behaviors. While it has been hypothesized that knowledge and skills come from pre-training, and fine-tuning mostly filters this knowledge and skillset, this intuition has not been extensively tested. To aid in doing so, we introduce a novel technique for decoupling the knowledge and skills gained in these two stages, enabling a direct answer to the question, "What would happen if we combined the knowledge learned by a large model during pre-training with the knowledge learned by a small model during fine-tuning (or vice versa)?" Using an RL-based framework derived from recent developments in learning from human preferences, we introduce emulated fine-tuning (EFT), a principled and practical method for sampling from a distribution that approximates (or 'emulates') the result of pre-training and fine-tuning at different scales. Our experiments with EFT show that scaling up fine-tuning tends to improve helpfulness, while scaling up pre-training tends to improve factuality. Beyond decoupling scale, we show that EFT enables test-time adjustment of competing behavioral traits like helpfulness and harmlessness without additional training. Finally, a special case of emulated fine-tuning, which we call LM up-scaling, avoids resource-intensive fine-tuning of large pre-trained models by ensembling them with small fine-tuned models, essentially emulating the result of fine-tuning the large pre-trained model. Up-scaling consistently improves helpfulness and factuality of instruction-following models in the Llama, Llama-2, and Falcon families, without additional hyperparameters or training.

{{}}

### (73/160) SEGO: Sequential Subgoal Optimization for Mathematical Problem-Solving (Xueliang Zhao et al., 2023)

{{}}

Xueliang Zhao, Xinting Huang, Wei Bi, Lingpeng Kong. (2023)  
**SEGO: Sequential Subgoal Optimization for Mathematical Problem-Solving**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: AI, Language Model  
[Paper Link](http://arxiv.org/abs/2310.12960v1)

* * *

**ABSTRACT**  
Large Language Models (LLMs) have driven substantial progress in artificial intelligence in recent years, exhibiting impressive capabilities across a wide range of tasks, including mathematical problem-solving. Inspired by the success of subgoal-based methods, we propose a novel framework called \textbf{SE}quential sub\textbf{G}oal \textbf{O}ptimization (SEGO) to enhance LLMs' ability to solve mathematical problems. By establishing a connection between the subgoal breakdown process and the probability of solving problems, SEGO aims to identify better subgoals with theoretical guarantees. Addressing the challenge of identifying suitable subgoals in a large solution space, our framework generates problem-specific subgoals and adjusts them according to carefully designed criteria. Incorporating these optimized subgoals into the policy model training leads to significant improvements in problem-solving performance. We validate SEGO's efficacy through experiments on two benchmarks, GSM8K and MATH, where our approach outperforms existing methods, highlighting the potential of SEGO in AI-driven mathematical problem-solving. Data and code associated with this paper will be available at <https://github.com/zhaoxlpku/SEGO>

{{}}

### (74/160) On the Representational Capacity of Recurrent Neural Language Models (Franz Nowak et al., 2023)

{{}}

Franz Nowak, Anej Svete, Li Du, Ryan Cotterell. (2023)  
**On the Representational Capacity of Recurrent Neural Language Models**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs-LG, cs.CL  
Keywords: Language Model  
[Paper Link](http://arxiv.org/abs/2310.12942v2)

* * *

**ABSTRACT**  
This work investigates the computational expressivity of language models (LMs) based on recurrent neural networks (RNNs). Siegelmann and Sontag (1992) famously showed that RNNs with rational weights and hidden states and unbounded computation time are Turing complete. However, LMs define weightings over strings in addition to just (unweighted) language membership and the analysis of the computational power of RNN LMs (RLMs) should reflect this. We extend the Turing completeness result to the probabilistic case, showing how a rationally weighted RLM with unbounded computation time can simulate any probabilistic Turing machine (PTM). Since, in practice, RLMs work in real-time, processing a symbol at every time step, we treat the above result as an upper bound on the expressivity of RLMs. We also provide a lower bound by showing that under the restriction to real-time computation, such models can simulate deterministic real-time rational PTMs.

{{}}

### (75/160) A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Models (Yi Zhou et al., 2023)

{{}}

Yi Zhou, Jose Camacho-Collados, Danushka Bollegala. (2023)  
**A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Models**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: Bias, Language Model  
[Paper Link](http://arxiv.org/abs/2310.12936v2)

* * *

**ABSTRACT**  
Various types of social biases have been reported with pretrained Masked Language Models (MLMs) in prior work. However, multiple underlying factors are associated with an MLM such as its model size, size of the training data, training objectives, the domain from which pretraining data is sampled, tokenization, and languages present in the pretrained corpora, to name a few. It remains unclear as to which of those factors influence social biases that are learned by MLMs. To study the relationship between model factors and the social biases learned by an MLM, as well as the downstream task performance of the model, we conduct a comprehensive study over 39 pretrained MLMs covering different model sizes, training objectives, tokenization methods, training data domains and languages. Our results shed light on important factors often neglected in prior literature, such as tokenization or model objectives.

{{}}

### (76/160) Experimental Narratives: A Comparison of Human Crowdsourced Storytelling and AI Storytelling (Nina Begus, 2023)

{{}}

Nina Begus. (2023)  
**Experimental Narratives: A Comparison of Human Crowdsourced Storytelling and AI Storytelling**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs.CL  
Keywords: AI, GPT, GPT-3.5, GPT-4  
[Paper Link](http://arxiv.org/abs/2310.12902v1)

* * *

**ABSTRACT**  
The paper proposes a framework that combines behavioral and computational experiments employing fictional prompts as a novel tool for investigating cultural artifacts and social biases in storytelling both by humans and generative AI. The study analyzes 250 stories authored by crowdworkers in June 2019 and 80 stories generated by GPT-3.5 and GPT-4 in March 2023 by merging methods from narratology and inferential statistics. Both crowdworkers and large language models responded to identical prompts about creating and falling in love with an artificial human. The proposed experimental paradigm allows a direct comparison between human and LLM-generated storytelling. Responses to the Pygmalionesque prompts confirm the pervasive presence of the Pygmalion myth in the collective imaginary of both humans and large language models. All solicited narratives present a scientific or technological pursuit. The analysis reveals that narratives from GPT-3.5 and particularly GPT-4 are more more progressive in terms of gender roles and sexuality than those written by humans. While AI narratives can occasionally provide innovative plot twists, they offer less imaginative scenarios and rhetoric than human-authored texts. The proposed framework argues that fiction can be used as a window into human and AI-based collective imaginary and social dimensions.

{{}}

### (77/160) A Systematic Study of Performance Disparities in Multilingual Task-Oriented Dialogue Systems (Songbo Hu et al., 2023)

{{}}

Songbo Hu, Han Zhou, Moy Yuan, Milan Gritta, Guchun Zhang, Ignacio Iacobacci, Anna Korhonen, Ivan Vulić. (2023)  
**A Systematic Study of Performance Disparities in Multilingual Task-Oriented Dialogue Systems**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: Dialog, Dialogue, Multilingual, NLP  
[Paper Link](http://arxiv.org/abs/2310.12892v1)

* * *

**ABSTRACT**  
Achieving robust language technologies that can perform well across the world's many languages is a central goal of multilingual NLP. In this work, we take stock of and empirically analyse task performance disparities that exist between multilingual task-oriented dialogue (ToD) systems. We first define new quantitative measures of absolute and relative equivalence in system performance, capturing disparities across languages and within individual languages. Through a series of controlled experiments, we demonstrate that performance disparities depend on a number of factors: the nature of the ToD task at hand, the underlying pretrained language model, the target language, and the amount of ToD annotated data. We empirically prove the existence of the adaptation and intrinsic biases in current ToD systems: e.g., ToD systems trained for Arabic or Turkish using annotated ToD data fully parallel to English ToD data still exhibit diminished ToD task performance. Beyond providing a series of insights into the performance disparities of ToD systems in different languages, our analyses offer practical tips on how to approach ToD data collection and system development for new languages.

{{}}

### (78/160) StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding (Cheng Jiayang et al., 2023)

{{}}

Cheng Jiayang, Lin Qiu, Tsz Ho Chan, Tianqing Fang, Weiqi Wang, Chunkit Chan, Dongyu Ru, Qipeng Guo, Hongming Zhang, Yangqiu Song, Yue Zhang, Zheng Zhang. (2023)  
**StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: ChatGPT, GPT, Language Model, T5  
[Paper Link](http://arxiv.org/abs/2310.12874v2)

* * *

**ABSTRACT**  
Analogy-making between narratives is crucial for human reasoning. In this paper, we evaluate the ability to identify and generate analogies by constructing a first-of-its-kind large-scale story-level analogy corpus, \textsc{StoryAnalogy}, which contains 24K story pairs from diverse domains with human annotations on two similarities from the extended Structure-Mapping Theory. We design a set of tests on \textsc{StoryAnalogy}, presenting the first evaluation of story-level analogy identification and generation. Interestingly, we find that the analogy identification tasks are incredibly difficult not only for sentence embedding models but also for the recent large language models (LLMs) such as ChatGPT and LLaMa. ChatGPT, for example, only achieved around 30% accuracy in multiple-choice questions (compared to over 85% accuracy for humans). Furthermore, we observe that the data in \textsc{StoryAnalogy} can improve the quality of analogy generation in LLMs, where a fine-tuned FlanT5-xxl model achieves comparable performance to zero-shot ChatGPT.

{{}}

### (79/160) The Locality and Symmetry of Positional Encodings (Lihu Chen et al., 2023)

{{}}

Lihu Chen, Gaël Varoquaux, Fabian M. Suchanek. (2023)  
**The Locality and Symmetry of Positional Encodings**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: BERT, Language Model  
[Paper Link](http://arxiv.org/abs/2310.12864v1)

* * *

**ABSTRACT**  
Positional Encodings (PEs) are used to inject word-order information into transformer-based language models. While they can significantly enhance the quality of sentence representations, their specific contribution to language models is not fully understood, especially given recent findings that various positional encodings are insensitive to word order. In this work, we conduct a systematic study of positional encodings in \textbf{Bidirectional Masked Language Models} (BERT-style) , which complements existing work in three aspects: (1) We uncover the core function of PEs by identifying two common properties, Locality and Symmetry; (2) We show that the two properties are closely correlated with the performances of downstream tasks; (3) We quantify the weakness of current PEs by introducing two new probing tasks, on which current PEs perform poorly. We believe that these results are the basis for developing better PEs for transformer-based language models. The code is available at \faGithub~ \url{[https://github.com/tigerchen52/locality\\_symmetry}](https://github.com/tigerchen52/locality%5C_symmetry%7D)

{{}}

### (80/160) Probing LLMs for hate speech detection: strengths and vulnerabilities (Sarthak Roy et al., 2023)

{{}}

Sarthak Roy, Ashish Harshavardhan, Animesh Mukherjee, Punyajoy Saha. (2023)  
**Probing LLMs for hate speech detection: strengths and vulnerabilities**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs-CY, cs.CL  
Keywords: GPT, GPT-3.5, T5  
[Paper Link](http://arxiv.org/abs/2310.12860v1)

* * *

**ABSTRACT**  
Recently efforts have been made by social media platforms as well as researchers to detect hateful or toxic language using large language models. However, none of these works aim to use explanation, additional context and victim community information in the detection process. We utilise different prompt variation, input information and evaluate large language models in zero shot setting (without adding any in-context examples). We select three large language models (GPT-3.5, text-davinci and Flan-T5) and three datasets - HateXplain, implicit hate and ToxicSpans. We find that on average including the target information in the pipeline improves the model performance substantially (~20-30%) over the baseline across the datasets. There is also a considerable effect of adding the rationales/explanations into the pipeline (~10-20%) over the baseline across the datasets. In addition, we further provide a typology of the error cases where these large language models fail to (i) classify and (ii) explain the reason for the decisions they take. Such vulnerable points automatically constitute 'jailbreak' prompts for these models and industry scale safeguard techniques need to be developed to make the models robust against such prompts.

{{}}

### (81/160) Knowledge-Augmented Language Model Verification (Jinheon Baek et al., 2023)

{{}}

Jinheon Baek, Soyeong Jeong, Minki Kang, Jong C. Park, Sung Ju Hwang. (2023)  
**Knowledge-Augmented Language Model Verification**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs-LG, cs.CL  
Keywords: Language Model  
[Paper Link](http://arxiv.org/abs/2310.12836v1)

* * *

**ABSTRACT**  
Recent Language Models (LMs) have shown impressive capabilities in generating texts with the knowledge internalized in parameters. Yet, LMs often generate the factually incorrect responses to the given queries, since their knowledge may be inaccurate, incomplete, and outdated. To address this problem, previous works propose to augment LMs with the knowledge retrieved from an external knowledge source. However, such approaches often show suboptimal text generation performance due to two reasons: 1) the model may fail to retrieve the knowledge relevant to the given query, or 2) the model may not faithfully reflect the retrieved knowledge in the generated text. To overcome these, we propose to verify the output and the knowledge of the knowledge-augmented LMs with a separate verifier, which is a small LM that is trained to detect those two types of errors through instruction-finetuning. Then, when the verifier recognizes an error, we can rectify it by either retrieving new knowledge or generating new text. Further, we use an ensemble of the outputs from different instructions with a single verifier to enhance the reliability of the verification processes. We validate the effectiveness of the proposed verification steps on multiple question answering benchmarks, whose results show that the proposed verifier effectively identifies retrieval and generation errors, allowing LMs to provide more factually correct outputs. Our code is available at <https://github.com/JinheonBaek/KALMV>.

{{}}

### (82/160) AgentTuning: Enabling Generalized Agent Abilities for LLMs (Aohan Zeng et al., 2023)

{{}}

Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, Jie Tang. (2023)  
**AgentTuning: Enabling Generalized Agent Abilities for LLMs**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs-LG, cs.CL  
Keywords: ChatGPT, GPT, GPT-3.5, GPT-4  
[Paper Link](http://arxiv.org/abs/2310.12823v2)

* * *

**ABSTRACT**  
Open large language models (LLMs) with great performance in various tasks have significantly advanced the development of LLMs. However, they are far inferior to commercial models such as ChatGPT and GPT-4 when acting as agents to tackle complex tasks in the real world. These agent tasks employ LLMs as the central controller responsible for planning, memorization, and tool utilization, necessitating both fine-grained prompting methods and robust LLMs to achieve satisfactory performance. Though many prompting methods have been proposed to complete particular agent tasks, there is lack of research focusing on improving the agent capabilities of LLMs themselves without compromising their general abilities. In this work, we present AgentTuning, a simple and general method to enhance the agent abilities of LLMs while maintaining their general LLM capabilities. We construct AgentInstruct, a lightweight instruction-tuning dataset containing high-quality interaction trajectories. We employ a hybrid instruction-tuning strategy by combining AgentInstruct with open-source instructions from general domains. AgentTuning is used to instruction-tune the Llama 2 series, resulting in AgentLM. Our evaluations show that AgentTuning enables LLMs' agent capabilities without compromising general abilities. The AgentLM-70B is comparable to GPT-3.5-turbo on unseen agent tasks, demonstrating generalized agent capabilities. We open source the AgentInstruct and AgentLM-7B, 13B, and 70B models at <https://github.com/THUDM/AgentTuning>, serving open and powerful alternatives to commercial LLMs for agent tasks.

{{}}

### (83/160) GestureGPT: Zero-shot Interactive Gesture Understanding and Grounding with Large Language Model Agents (Xin Zeng et al., 2023)

{{}}

Xin Zeng, Xiaoyu Wang, Tengxiang Zhang, Chun Yu, Shengdong Zhao, Yiqiang Chen. (2023)  
**GestureGPT: Zero-shot Interactive Gesture Understanding and Grounding with Large Language Model Agents**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs-HC, cs.CL  
Keywords: GPT, Language Model  
[Paper Link](http://arxiv.org/abs/2310.12821v2)

* * *

**ABSTRACT**  
Current gesture recognition systems primarily focus on identifying gestures within a predefined set, leaving a gap in connecting these gestures to interactive GUI elements or system functions (e.g., linking a 'thumb-up' gesture to a 'like' button). We introduce GestureGPT, a novel zero-shot gesture understanding and grounding framework leveraging large language models (LLMs). Gesture descriptions are formulated based on hand landmark coordinates from gesture videos and fed into our dual-agent dialogue system. A gesture agent deciphers these descriptions and queries about the interaction context (e.g., interface, history, gaze data), which a context agent organizes and provides. Following iterative exchanges, the gesture agent discerns user intent, grounding it to an interactive function. We validated the gesture description module using public first-view and third-view gesture datasets and tested the whole system in two real-world settings: video streaming and smart home IoT control. The highest zero-shot Top-5 grounding accuracies are 80.11% for video streaming and 90.78% for smart home tasks, showing potential of the new gesture understanding paradigm.

{{}}

### (84/160) Boosting Inference Efficiency: Unleashing the Power of Parameter-Shared Pre-trained Language Models (Weize Chen et al., 2023)

{{}}

Weize Chen, Xiaoyue Xu, Xu Han, Yankai Lin, Ruobing Xie, Zhiyuan Liu, Maosong Sun, Jie Zhou. (2023)  
**Boosting Inference Efficiency: Unleashing the Power of Parameter-Shared Pre-trained Language Models**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs-LG, cs.CL  
Keywords: Language Model  
[Paper Link](http://arxiv.org/abs/2310.12818v1)

* * *

**ABSTRACT**  
Parameter-shared pre-trained language models (PLMs) have emerged as a successful approach in resource-constrained environments, enabling substantial reductions in model storage and memory costs without significant performance compromise. However, it is important to note that parameter sharing does not alleviate computational burdens associated with inference, thus impeding its practicality in situations characterized by limited stringent latency requirements or computational resources. Building upon neural ordinary differential equations (ODEs), we introduce a straightforward technique to enhance the inference efficiency of parameter-shared PLMs. Additionally, we propose a simple pre-training technique that leads to fully or partially shared models capable of achieving even greater inference acceleration. The experimental results demonstrate the effectiveness of our methods on both autoregressive and autoencoding PLMs, providing novel insights into more efficient utilization of parameter-shared models in resource-constrained settings.

{{}}

### (85/160) MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter (Zhiyuan Liu et al., 2023)

{{}}

Zhiyuan Liu, Sihang Li, Yanchen Luo, Hao Fei, Yixin Cao, Kenji Kawaguchi, Xiang Wang, Tat-Seng Chua. (2023)  
**MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs-MM, cs.CL  
Keywords: Language Model  
[Paper Link](http://arxiv.org/abs/2310.12798v1)

* * *

**ABSTRACT**  
Language Models (LMs) have demonstrated impressive molecule understanding ability on various 1D text-related tasks. However, they inherently lack 2D graph perception - a critical ability of human professionals in comprehending molecules' topological structures. To bridge this gap, we propose MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter. MolCA enables an LM (e.g., Galactica) to understand both text- and graph-based molecular contents via the cross-modal projector. Specifically, the cross-modal projector is implemented as a Q-Former to connect a graph encoder's representation space and an LM's text space. Further, MolCA employs a uni-modal adapter (i.e., LoRA) for the LM's efficient adaptation to downstream tasks. Unlike previous studies that couple an LM with a graph encoder via cross-modal contrastive learning, MolCA retains the LM's ability of open-ended text generation and augments it with 2D graph information. To showcase its effectiveness, we extensively benchmark MolCA on tasks of molecule captioning, IUPAC name prediction, and molecule-text retrieval, on which MolCA significantly outperforms the baselines. Our codes and checkpoints can be found at <https://github.com/acharkq/MolCA>.

{{}}

### (86/160) Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalization (Ningyu Xu et al., 2023)

{{}}

Ningyu Xu, Qi Zhang, Jingting Ye, Menghan Zhang, Xuanjing Huang. (2023)  
**Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalization**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: Language Model, Transformer  
[Paper Link](http://arxiv.org/abs/2310.12794v1)

* * *

**ABSTRACT**  
Large language models (LLMs) have exhibited considerable cross-lingual generalization abilities, whereby they implicitly transfer knowledge across languages. However, the transfer is not equally successful for all languages, especially for low-resource ones, which poses an ongoing challenge. It is unclear whether we have reached the limits of implicit cross-lingual generalization and if explicit knowledge transfer is viable. In this paper, we investigate the potential for explicitly aligning conceptual correspondence between languages to enhance cross-lingual generalization. Using the syntactic aspect of language as a testbed, our analyses of 43 languages reveal a high degree of alignability among the spaces of structural concepts within each language for both encoder-only and decoder-only LLMs. We then propose a meta-learning-based method to learn to align conceptual spaces of different languages, which facilitates zero-shot and few-shot generalization in concept classification and also offers insights into the cross-lingual in-context learning phenomenon. Experiments on syntactic analysis tasks show that our approach achieves competitive results with state-of-the-art methods and narrows the performance gap between languages, particularly benefiting those with limited resources.

{{}}

### (87/160) Label-Aware Automatic Verbalizer for Few-Shot Text Classification (Thanakorn Thaminkaew et al., 2023)

{{}}

Thanakorn Thaminkaew, Piyawat Lertvittayakumjorn, Peerapon Vateekul. (2023)  
**Label-Aware Automatic Verbalizer for Few-Shot Text Classification**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs-LG, cs.CL  
Keywords: Few-Shot, Text Classification  
[Paper Link](http://arxiv.org/abs/2310.12778v1)

* * *

**ABSTRACT**  
Prompt-based learning has shown its effectiveness in few-shot text classification. One important factor in its success is a verbalizer, which translates output from a language model into a predicted class. Notably, the simplest and widely acknowledged verbalizer employs manual labels to represent the classes. However, manual selection does not guarantee the optimality of the selected words when conditioned on the chosen language model. Therefore, we propose Label-Aware Automatic Verbalizer (LAAV), effectively augmenting the manual labels to achieve better few-shot classification results. Specifically, we use the manual labels along with the conjunction "and" to induce the model to generate more effective words for the verbalizer. The experimental results on five datasets across five languages demonstrate that LAAV significantly outperforms existing verbalizers. Furthermore, our analysis reveals that LAAV suggests more relevant words compared to similar approaches, especially in mid-to-low resource languages.

{{}}

### (88/160) Survival of the Most Influential Prompts: Efficient Black-Box Prompt Search via Clustering and Pruning (Han Zhou et al., 2023)

{{}}

Han Zhou, Xingchen Wan, Ivan Vulić, Anna Korhonen. (2023)  
**Survival of the Most Influential Prompts: Efficient Black-Box Prompt Search via Clustering and Pruning**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs-LG, cs.CL  
Keywords: Pruning  
[Paper Link](http://arxiv.org/abs/2310.12774v1)

* * *

**ABSTRACT**  
Prompt-based learning has been an effective paradigm for large pretrained language models (LLM), enabling few-shot or even zero-shot learning. Black-box prompt search has received growing interest recently for its distinctive properties of gradient-free optimization, proven particularly useful and powerful for model-as-a-service usage. However, the discrete nature and the complexity of combinatorial optimization hinder the efficiency of modern black-box approaches. Despite extensive research on search algorithms, the crucial aspect of search space design and optimization has been largely overlooked. In this paper, we first conduct a sensitivity analysis by prompting LLM, revealing that only a small number of tokens exert a disproportionate amount of influence on LLM predictions. Leveraging this insight, we propose the Clustering and Pruning for Efficient Black-box Prompt Search (ClaPS), a simple black-box search method that first clusters and prunes the search space to focus exclusively on influential prompt tokens. By employing even simple search methods within the pruned search space, ClaPS achieves state-of-the-art performance across various tasks and LLMs, surpassing the performance of complex approaches while significantly reducing search costs. Our findings underscore the critical role of search space design and optimization in enhancing both the usefulness and the efficiency of black-box prompt-based learning.

{{}}

### (89/160) Transformer-based Entity Legal Form Classification (Alexander Arimond et al., 2023)

{{}}

Alexander Arimond, Mauro Molteni, Dominik Jany, Zornitsa Manolova, Damian Borth, Andreas G. F. Hoepner. (2023)  
**Transformer-based Entity Legal Form Classification**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs-LG, cs.CL  
Keywords: BERT, Legal, Transformer  
[Paper Link](http://arxiv.org/abs/2310.12766v1)

* * *

**ABSTRACT**  
We propose the application of Transformer-based language models for classifying entity legal forms from raw legal entity names. Specifically, we employ various BERT variants and compare their performance against multiple traditional baselines. Our evaluation encompasses a substantial subset of freely available Legal Entity Identifier (LEI) data, comprising over 1.1 million legal entities from 30 different legal jurisdictions. The ground truth labels for classification per jurisdiction are taken from the Entity Legal Form (ELF) code standard (ISO 20275). Our findings demonstrate that pre-trained BERT variants outperform traditional text classification approaches in terms of F1 score, while also performing comparably well in the Macro F1 Score. Moreover, the validity of our proposal is supported by the outcome of third-party expert reviews conducted in ten selected jurisdictions. This study highlights the significant potential of Transformer-based models in advancing data standardization and data integration. The presented approaches can greatly benefit financial institutions, corporations, governments and other organizations in assessing business relationships, understanding risk exposure, and promoting effective governance.

{{}}

### (90/160) Character-level Chinese Backpack Language Models (Hao Sun et al., 2023)

{{}}

Hao Sun, John Hewitt. (2023)  
**Character-level Chinese Backpack Language Models**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: Language Model, Transformer  
[Paper Link](http://arxiv.org/abs/2310.12751v1)

* * *

**ABSTRACT**  
The Backpack is a Transformer alternative shown to improve interpretability in English language modeling by decomposing predictions into a weighted sum of token sense components. However, Backpacks' reliance on token-defined meaning raises questions as to their potential for languages other than English, a language for which subword tokenization provides a reasonable approximation for lexical items. In this work, we train, evaluate, interpret, and control Backpack language models in character-tokenized Chinese, in which words are often composed of many characters. We find that our (134M parameter) Chinese Backpack language model performs comparably to a (104M parameter) Transformer, and learns rich character-level meanings that log-additively compose to form word meanings. In SimLex-style lexical semantic evaluations, simple averages of Backpack character senses outperform input embeddings from a Transformer. We find that complex multi-character meanings are often formed by using the same per-character sense weights consistently across context. Exploring interpretability-through control, we show that we can localize a source of gender bias in our Backpacks to specific character senses and intervene to reduce the bias.

{{}}

### (91/160) Quality-Diversity through AI Feedback (Herbie Bradley et al., 2023)

{{}}

Herbie Bradley, Andrew Dai, Hannah Teufel, Jenny Zhang, Koen Oostermeijer, Marco Bellagente, Jeff Clune, Kenneth Stanley, Grégory Schott, Joel Lehman. (2023)  
**Quality-Diversity through AI Feedback**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs-LG, cs-NE, cs.CL  
Keywords: AI  
[Paper Link](http://arxiv.org/abs/2310.13032v1)

* * *

**ABSTRACT**  
In many text-generation problems, users may prefer not only a single response, but a diverse range of high-quality outputs from which to choose. Quality-diversity (QD) search algorithms aim at such outcomes, by continually improving and diversifying a population of candidates. However, the applicability of QD to qualitative domains, like creative writing, has been limited by the difficulty of algorithmically specifying measures of quality and diversity. Interestingly, recent developments in language models (LMs) have enabled guiding search through AI feedback, wherein LMs are prompted in natural language to evaluate qualitative aspects of text. Leveraging this development, we introduce Quality-Diversity through AI Feedback (QDAIF), wherein an evolutionary algorithm applies LMs to both generate variation and evaluate the quality and diversity of candidate text. When assessed on creative writing domains, QDAIF covers more of a specified search space with high-quality samples than do non-QD controls. Further, human evaluation of QDAIF-generated creative texts validates reasonable agreement between AI and human evaluation. Our results thus highlight the potential of AI feedback to guide open-ended search for creative and original solutions, providing a recipe that seemingly generalizes to many domains and modalities. In this way, QDAIF is a step towards AI systems that can independently search, diversify, evaluate, and improve, which are among the core skills underlying human society's capacity for innovation.

{{}}

### (92/160) Is ChatGPT a Financial Expert? Evaluating Language Models on Financial Natural Language Processing (Yue Guo et al., 2023)

{{}}

Yue Guo, Zian Xu, Yi Yang. (2023)  
**Is ChatGPT a Financial Expert? Evaluating Language Models on Financial Natural Language Processing**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: ChatGPT, Financial, GPT, Language Model, NLP, Natural Language Processing  
[Paper Link](http://arxiv.org/abs/2310.12664v1)

* * *

**ABSTRACT**  
The emergence of Large Language Models (LLMs), such as ChatGPT, has revolutionized general natural language preprocessing (NLP) tasks. However, their expertise in the financial domain lacks a comprehensive evaluation. To assess the ability of LLMs to solve financial NLP tasks, we present FinLMEval, a framework for Financial Language Model Evaluation, comprising nine datasets designed to evaluate the performance of language models. This study compares the performance of encoder-only language models and the decoder-only language models. Our findings reveal that while some decoder-only LLMs demonstrate notable performance across most financial tasks via zero-shot prompting, they generally lag behind the fine-tuned expert models, especially when dealing with proprietary datasets. We hope this study provides foundation evaluations for continuing efforts to build more advanced LLMs in the financial domain.

{{}}

### (93/160) A Use Case: Reformulating Query Rewriting as a Statistical Machine Translation Problem (Abdullah Can Algan et al., 2023)

{{}}

Abdullah Can Algan, Emre Yürekli, Aykut Çayır. (2023)  
**A Use Case: Reformulating Query Rewriting as a Statistical Machine Translation Problem**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs-LG, cs.CL  
Keywords: Machine Translation, NLP  
[Paper Link](http://arxiv.org/abs/2310.13031v1)

* * *

**ABSTRACT**  
One of the most important challenges for modern search engines is to retrieve relevant web content based on user queries. In order to achieve this challenge, search engines have a module to rewrite user queries. That is why modern web search engines utilize some statistical and neural models used in the natural language processing domain. Statistical machine translation is a well-known NLP method among them. The paper proposes a query rewriting pipeline based on a monolingual machine translation model that learns to rewrite Arabic user search queries. This paper also describes preprocessing steps to create a mapping between user queries and web page titles.

{{}}

### (94/160) Towards Real-World Streaming Speech Translation for Code-Switched Speech (Belen Alastruey et al., 2023)

{{}}

Belen Alastruey, Matthias Sperber, Christian Gollan, Dominic Telaar, Tim Ng, Aashish Agarwal. (2023)  
**Towards Real-World Streaming Speech Translation for Code-Switched Speech**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: NLP, Natural Language Processing  
[Paper Link](http://arxiv.org/abs/2310.12648v2)

* * *

**ABSTRACT**  
Code-switching (CS), i.e. mixing different languages in a single sentence, is a common phenomenon in communication and can be challenging in many Natural Language Processing (NLP) settings. Previous studies on CS speech have shown promising results for end-to-end speech translation (ST), but have been limited to offline scenarios and to translation to one of the languages present in the source (\textit{monolingual transcription}). In this paper, we focus on two essential yet unexplored areas for real-world CS speech translation: streaming settings, and translation to a third language (i.e., a language not included in the source). To this end, we extend the Fisher and Miami test and validation datasets to include new targets in Spanish and German. Using this data, we train a model for both offline and streaming ST and we establish baseline results for the two settings mentioned earlier.

{{}}

### (95/160) Non-Autoregressive Sentence Ordering (Yi Bin et al., 2023)

{{}}

Yi Bin, Wenhao Shi, Bin Ji, Jipeng Zhang, Yujuan Ding, Yang Yang. (2023)  
**Non-Autoregressive Sentence Ordering**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: Transformer  
[Paper Link](http://arxiv.org/abs/2310.12640v1)

* * *

**ABSTRACT**  
Existing sentence ordering approaches generally employ encoder-decoder frameworks with the pointer net to recover the coherence by recurrently predicting each sentence step-by-step. Such an autoregressive manner only leverages unilateral dependencies during decoding and cannot fully explore the semantic dependency between sentences for ordering. To overcome these limitations, in this paper, we propose a novel Non-Autoregressive Ordering Network, dubbed \textit{NAON}, which explores bilateral dependencies between sentences and predicts the sentence for each position in parallel. We claim that the non-autoregressive manner is not just applicable but also particularly suitable to the sentence ordering task because of two peculiar characteristics of the task: 1) each generation target is in deterministic length, and 2) the sentences and positions should match exclusively. Furthermore, to address the repetition issue of the naive non-autoregressive Transformer, we introduce an exclusive loss to constrain the exclusiveness between positions and sentences. To verify the effectiveness of the proposed model, we conduct extensive experiments on several common-used datasets and the experimental results show that our method outperforms all the autoregressive approaches and yields competitive performance compared with the state-of-the-arts. The codes are available at: \url{[https://github.com/steven640pixel/nonautoregressive-sentence-ordering}](https://github.com/steven640pixel/nonautoregressive-sentence-ordering%7D).

{{}}

### (96/160) Predict the Future from the Past? On the Temporal Data Distribution Shift in Financial Sentiment Classifications (Yue Guo et al., 2023)

{{}}

Yue Guo, Chenxi Hu, Yi Yang. (2023)  
**Predict the Future from the Past? On the Temporal Data Distribution Shift in Financial Sentiment Classifications**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: Financial  
[Paper Link](http://arxiv.org/abs/2310.12620v1)

* * *

**ABSTRACT**  
Temporal data distribution shift is prevalent in the financial text. How can a financial sentiment analysis system be trained in a volatile market environment that can accurately infer sentiment and be robust to temporal data distribution shifts? In this paper, we conduct an empirical study on the financial sentiment analysis system under temporal data distribution shifts using a real-world financial social media dataset that spans three years. We find that the fine-tuned models suffer from general performance degradation in the presence of temporal distribution shifts. Furthermore, motivated by the unique temporal nature of the financial text, we propose a novel method that combines out-of-distribution detection with time series modeling for temporal financial sentiment analysis. Experimental results show that the proposed method enhances the model's capability to adapt to evolving temporal shifts in a volatile financial market.

{{}}

### (97/160) Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Model (Abhijith Chintam et al., 2023)

{{}}

Abhijith Chintam, Rahel Beloch, Willem Zuidema, Michael Hanna, Oskar van der Wal. (2023)  
**Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Model**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs.CL  
Keywords: Bias, GPT, Language Model, Transformer  
[Paper Link](http://arxiv.org/abs/2310.12611v1)

* * *

**ABSTRACT**  
Language models (LMs) exhibit and amplify many types of undesirable biases learned from the training data, including gender bias. However, we lack tools for effectively and efficiently changing this behavior without hurting general language modeling performance. In this paper, we study three methods for identifying causal relations between LM components and particular output: causal mediation analysis, automated circuit discovery and our novel, efficient method called DiffMask+ based on differential masking. We apply the methods to GPT-2 small and the problem of gender bias, and use the discovered sets of components to perform parameter-efficient fine-tuning for bias mitigation. Our results show significant overlap in the identified components (despite huge differences in the computational requirements of the methods) as well as success in mitigating gender bias, with less damage to general language modeling compared to full model fine-tuning. However, our work also underscores the difficulty of defining and measuring bias, and the sensitivity of causal discovery procedures to dataset choice. We hope our work can contribute to more attention for dataset development, and lead to more effective mitigation strategies for other types of bias.

{{}}

### (98/160) Time-Aware Representation Learning for Time-Sensitive Question Answering (Jungbin Son et al., 2023)

{{}}

Jungbin Son, Alice Oh. (2023)  
**Time-Aware Representation Learning for Time-Sensitive Question Answering**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs-LG, cs.CL  
Keywords: QA, Question Answering, Representation Learning  
[Paper Link](http://arxiv.org/abs/2310.12585v1)

* * *

**ABSTRACT**  
Time is one of the crucial factors in real-world question answering (QA) problems. However, language models have difficulty understanding the relationships between time specifiers, such as 'after' and 'before', and numbers, since existing QA datasets do not include sufficient time expressions. To address this issue, we propose a Time-Context aware Question Answering (TCQA) framework. We suggest a Time-Context dependent Span Extraction (TCSE) task, and build a time-context dependent data generation framework for model training. Moreover, we present a metric to evaluate the time awareness of the QA model using TCSE. The TCSE task consists of a question and four sentence candidates classified as correct or incorrect based on time and context. The model is trained to extract the answer span from the sentence that is both correct in time and context. The model trained with TCQA outperforms baseline models up to 8.5 of the F1-score in the TimeQA dataset. Our dataset and code are available at <https://github.com/sonjbin/TCQA>

{{}}

### (99/160) Pretraining Language Models with Text-Attributed Heterogeneous Graphs (Tao Zou et al., 2023)

{{}}

Tao Zou, Le Yu, Yifei Huang, Leilei Sun, Bowen Du. (2023)  
**Pretraining Language Models with Text-Attributed Heterogeneous Graphs**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs.CL  
Keywords: Language Model  
[Paper Link](http://arxiv.org/abs/2310.12580v2)

* * *

**ABSTRACT**  
In many real-world scenarios (e.g., academic networks, social platforms), different types of entities are not only associated with texts but also connected by various relationships, which can be abstracted as Text-Attributed Heterogeneous Graphs (TAHGs). Current pretraining tasks for Language Models (LMs) primarily focus on separately learning the textual information of each entity and overlook the crucial aspect of capturing topological connections among entities in TAHGs. In this paper, we present a new pretraining framework for LMs that explicitly considers the topological and heterogeneous information in TAHGs. Firstly, we define a context graph as neighborhoods of a target node within specific orders and propose a topology-aware pretraining task to predict nodes involved in the context graph by jointly optimizing an LM and an auxiliary heterogeneous graph neural network. Secondly, based on the observation that some nodes are text-rich while others have little text, we devise a text augmentation strategy to enrich textless nodes with their neighbors' texts for handling the imbalance issue. We conduct link prediction and node classification tasks on three datasets from various domains. Experimental results demonstrate the superiority of our approach over existing methods and the rationality of each design. Our code is available at <https://github.com/Hope-Rita/THLM>.

{{}}

### (100/160) Multilingual estimation of political-party positioning: From label aggregation to long-input Transformers (Dmitry Nikolaev et al., 2023)

{{}}

Dmitry Nikolaev, Tanise Ceron, Sebastian Padó. (2023)  
**Multilingual estimation of political-party positioning: From label aggregation to long-input Transformers**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: Multilingual, NLP, Transformer, Transformers  
[Paper Link](http://arxiv.org/abs/2310.12575v1)

* * *

**ABSTRACT**  
Scaling analysis is a technique in computational political science that assigns a political actor (e.g. politician or party) a score on a predefined scale based on a (typically long) body of text (e.g. a parliamentary speech or an election manifesto). For example, political scientists have often used the left--right scale to systematically analyse political landscapes of different countries. NLP methods for automatic scaling analysis can find broad application provided they (i) are able to deal with long texts and (ii) work robustly across domains and languages. In this work, we implement and compare two approaches to automatic scaling analysis of political-party manifestos: label aggregation, a pipeline strategy relying on annotations of individual statements from the manifestos, and long-input-Transformer-based models, which compute scaling values directly from raw text. We carry out the analysis of the Comparative Manifestos Project dataset across 41 countries and 27 languages and find that the task can be efficiently solved by state-of-the-art models, with label aggregation producing the best results.

{{}}

### (101/160) Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrong (Chenglei Si et al., 2023)

{{}}

Chenglei Si, Navita Goyal, Sherry Tongshuang Wu, Chen Zhao, Shi Feng, Hal Daumé III, Jordan Boyd-Graber. (2023)  
**Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrong**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs-HC, cs.CL  
Keywords: AI, Language Model  
[Paper Link](http://arxiv.org/abs/2310.12558v1)

* * *

**ABSTRACT**  
Large Language Models (LLMs) are increasingly used for accessing information on the web. Their truthfulness and factuality are thus of great interest. To help users make the right decisions about the information they're getting, LLMs should not only provide but also help users fact-check information. In this paper, we conduct experiments with 80 crowdworkers in total to compare language models with search engines (information retrieval systems) at facilitating fact-checking by human users. We prompt LLMs to validate a given claim and provide corresponding explanations. Users reading LLM explanations are significantly more efficient than using search engines with similar accuracy. However, they tend to over-rely the LLMs when the explanation is wrong. To reduce over-reliance on LLMs, we ask LLMs to provide contrastive information - explain both why the claim is true and false, and then we present both sides of the explanation to users. This contrastive explanation mitigates users' over-reliance on LLMs, but cannot significantly outperform search engines. However, showing both search engine results and LLM explanations offers no complementary benefits as compared to search engines alone. Taken together, natural language explanations by LLMs may not be a reliable replacement for reading the retrieved passages yet, especially in high-stakes settings where over-relying on wrong AI explanations could lead to critical consequences.

{{}}

### (102/160) DepWiGNN: A Depth-wise Graph Neural Network for Multi-hop Spatial Reasoning in Text (Shuaiyi Li et al., 2023)

{{}}

Shuaiyi Li, Yang Deng, Wai Lam. (2023)  
**DepWiGNN: A Depth-wise Graph Neural Network for Multi-hop Spatial Reasoning in Text**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs.CL  
Keywords: GNN, Graph Neural Network, Reasoning  
[Paper Link](http://arxiv.org/abs/2310.12557v1)

* * *

**ABSTRACT**  
Spatial reasoning in text plays a crucial role in various real-world applications. Existing approaches for spatial reasoning typically infer spatial relations from pure text, which overlook the gap between natural language and symbolic structures. Graph neural networks (GNNs) have showcased exceptional proficiency in inducing and aggregating symbolic structures. However, classical GNNs face challenges in handling multi-hop spatial reasoning due to the over-smoothing issue, \textit{i.e.}, the performance decreases substantially as the number of graph layers increases. To cope with these challenges, we propose a novel \textbf{Dep}th-\textbf{Wi}se \textbf{G}raph \textbf{N}eural \textbf{N}etwork (\textbf{DepWiGNN}). Specifically, we design a novel node memory scheme and aggregate the information over the depth dimension instead of the breadth dimension of the graph, which empowers the ability to collect long dependencies without stacking multiple layers. Experimental results on two challenging multi-hop spatial reasoning datasets show that DepWiGNN outperforms existing spatial reasoning methods. The comparisons with the other three GNNs further demonstrate its superiority in capturing long dependency in the graph.

{{}}

### (103/160) Reliable Academic Conference Question Answering: A Study Based on Large Language Model (Zhiwei Huang et al., 2023)

{{}}

Zhiwei Huang, Long Jin, Junjie Wang, Mingchen Tu, Yin Hua, Zhiqiang Liu, Jiawei Meng, Huajun Chen, Wen Zhang. (2023)  
**Reliable Academic Conference Question Answering: A Study Based on Large Language Model**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs.CL  
Keywords: Language Model, NLP, QA, Question Answering  
[Paper Link](http://arxiv.org/abs/2310.13028v1)

* * *

**ABSTRACT**  
The rapid growth of computer science has led to a proliferation of research presented at academic conferences, fostering global scholarly communication. Researchers consistently seek accurate, current information about these events at all stages. This data surge necessitates an intelligent question-answering system to efficiently address researchers' queries and ensure awareness of the latest advancements. The information of conferences is usually published on their official website, organized in a semi-structured way with a lot of text. To address this need, we have developed the ConferenceQA dataset for 7 diverse academic conferences with human annotations. Firstly, we employ a combination of manual and automated methods to organize academic conference data in a semi-structured JSON format. Subsequently, we annotate nearly 100 question-answer pairs for each conference. Each pair is classified into four different dimensions. To ensure the reliability of the data, we manually annotate the source of each answer. In light of recent advancements, Large Language Models (LLMs) have demonstrated impressive performance in various NLP tasks. They have demonstrated impressive capabilities in information-seeking question answering after instruction fine-tuning, and as such, we present our conference QA study based on LLM. Due to hallucination and outdated knowledge of LLMs, we adopt retrieval based methods to enhance LLMs' question-answering abilities. We have proposed a structure-aware retrieval method, specifically designed to leverage inherent structural information during the retrieval process. Empirical validation on the ConferenceQA dataset has demonstrated the effectiveness of this method. The dataset and code are readily accessible on <https://github.com/zjukg/ConferenceQA>.

{{}}

### (104/160) Product Attribute Value Extraction using Large Language Models (Alexander Brinkmann et al., 2023)

{{}}

Alexander Brinkmann, Roee Shraga, Christian Bizer. (2023)  
**Product Attribute Value Extraction using Large Language Models**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: BERT, GPT, GPT-3.5, GPT-4, Language Model  
[Paper Link](http://arxiv.org/abs/2310.12537v1)

* * *

**ABSTRACT**  
E-commerce applications such as faceted product search or product comparison are based on structured product descriptions like attribute/value pairs. The vendors on e-commerce platforms do not provide structured product descriptions but describe offers using titles or descriptions. To process such offers, it is necessary to extract attribute/value pairs from textual product attributes. State-of-the-art attribute/value extraction techniques rely on pre-trained language models (PLMs), such as BERT. Two major drawbacks of these models for attribute/value extraction are that (i) the models require significant amounts of task-specific training data and (ii) the fine-tuned models face challenges in generalizing to attribute values not included in the training data. This paper explores the potential of large language models (LLMs) as a training data-efficient and robust alternative to PLM-based attribute/value extraction methods. We consider hosted LLMs, such as GPT-3.5 and GPT-4, as well as open-source LLMs based on Llama2. We evaluate the models in a zero-shot scenario and in a scenario where task-specific training data is available. In the zero-shot scenario, we compare various prompt designs for representing information about the target attributes of the extraction. In the scenario with training data, we investigate (i) the provision of example attribute values, (ii) the selection of in-context demonstrations, and (iii) the fine-tuning of GPT-3.5. Our experiments show that GPT-4 achieves an average F1-score of 85% on the two evaluation datasets while the best PLM-based techniques perform on average 5% worse using the same amount of training data. GPT-4 achieves a 10% higher F1-score than the best open-source LLM. The fine-tuned GPT-3.5 model reaches a similar performance as GPT-4 while being significantly more cost-efficient.

{{}}

### (105/160) ICU: Conquering Language Barriers in Vision-and-Language Modeling by Dividing the Tasks into Image Captioning and Language Understanding (Guojun Wu, 2023)

{{}}

Guojun Wu. (2023)  
**ICU: Conquering Language Barriers in Vision-and-Language Modeling by Dividing the Tasks into Image Captioning and Language Understanding**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: GLUE, Image Captioning, Language Model  
[Paper Link](http://arxiv.org/abs/2310.12531v2)

* * *

**ABSTRACT**  
Most multilingual vision-and-language (V&L) research aims to accomplish multilingual and multimodal capabilities within one model. However, the scarcity of multilingual captions for images has hindered the development. To overcome this obstacle, we propose ICU, Image Caption Understanding, which divides a V&L task into two stages: a V&L model performs image captioning in English, and a multilingual language model (mLM), in turn, takes the caption as the alt text and performs crosslingual language understanding. The burden of multilingual processing is lifted off V&L model and placed on mLM. Since the multilingual text data is relatively of higher abundance and quality, ICU can facilitate the conquering of language barriers for V&L models. In experiments on two tasks across 9 languages in the IGLUE benchmark, we show that ICU can achieve new state-of-the-art results for five languages, and comparable results for the rest.

{{}}

### (106/160) Named Entity Recognition for Monitoring Plant Health Threats in Tweets: a ChouBERT Approach (Shufan Jiang et al., 2023)

{{}}

Shufan Jiang, Rafael Angarita, Stéphane Cormier, Francis Rousseaux. (2023)  
**Named Entity Recognition for Monitoring Plant Health Threats in Tweets: a ChouBERT Approach**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs-LG, cs.CL  
Keywords: BERT, Named Entity Recognition, Twitter  
[Paper Link](http://arxiv.org/abs/2310.12522v1)

* * *

**ABSTRACT**  
An important application scenario of precision agriculture is detecting and measuring crop health threats using sensors and data analysis techniques. However, the textual data are still under-explored among the existing solutions due to the lack of labelled data and fine-grained semantic resources. Recent research suggests that the increasing connectivity of farmers and the emergence of online farming communities make social media like Twitter a participatory platform for detecting unfamiliar plant health events if we can extract essential information from unstructured textual data. ChouBERT is a French pre-trained language model that can identify Tweets concerning observations of plant health issues with generalizability on unseen natural hazards. This paper tackles the lack of labelled data by further studying ChouBERT's know-how on token-level annotation tasks over small labeled sets.

{{}}

### (107/160) Lost in Translation: When GPT-4V(ision) Can't See Eye to Eye with Text. A Vision-Language-Consistency Analysis of VLLMs and Beyond (Xiang Zhang et al., 2023)

{{}}

Xiang Zhang, Senyu Li, Zijun Wu, Ning Shi. (2023)  
**Lost in Translation: When GPT-4V(ision) Can't See Eye to Eye with Text. A Vision-Language-Consistency Analysis of VLLMs and Beyond**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs-CV, cs.CL  
Keywords: GPT, GPT-4, Language Model  
[Paper Link](http://arxiv.org/abs/2310.12520v1)

* * *

**ABSTRACT**  
Recent advancements in multimodal techniques open exciting possibilities for models excelling in diverse tasks involving text, audio, and image processing. Models like GPT-4V, blending computer vision and language modeling, excel in complex text and image tasks. Numerous prior research endeavors have diligently examined the performance of these Vision Large Language Models (VLLMs) across tasks like object detection, image captioning and others. However, these analyses often focus on evaluating the performance of each modality in isolation, lacking insights into their cross-modal interactions. Specifically, questions concerning whether these vision-language models execute vision and language tasks consistently or independently have remained unanswered. In this study, we draw inspiration from recent investigations into multilingualism and conduct a comprehensive analysis of model's cross-modal interactions. We introduce a systematic framework that quantifies the capability disparities between different modalities in the multi-modal setting and provide a set of datasets designed for these evaluations. Our findings reveal that models like GPT-4V tend to perform consistently modalities when the tasks are relatively simple. However, the trustworthiness of results derived from the vision modality diminishes as the tasks become more challenging. Expanding on our findings, we introduce "Vision Description Prompting," a method that effectively improves performance in challenging vision-related tasks.

{{}}

### (108/160) Automatic Hallucination Assessment for Aligned Large Language Models via Transferable Adversarial Attacks (Xiaodong Yu et al., 2023)

{{}}

Xiaodong Yu, Hao Cheng, Xiaodong Liu, Dan Roth, Jianfeng Gao. (2023)  
**Automatic Hallucination Assessment for Aligned Large Language Models via Transferable Adversarial Attacks**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs-LG, cs.CL  
Keywords: Adversarial Attack, ChatGPT, GPT, GPT-4, Language Model  
[Paper Link](http://arxiv.org/abs/2310.12516v1)

* * *

**ABSTRACT**  
Although remarkable progress has been achieved in preventing large language model (LLM) hallucinations using instruction tuning and retrieval augmentation, it remains challenging to measure the reliability of LLMs using human-crafted evaluation data which is not available for many tasks and domains and could suffer from data leakage. Inspired by adversarial machine learning, this paper aims to develop a method of automatically generating evaluation data by appropriately modifying existing data on which LLMs behave faithfully. Specifically, this paper presents AutoDebug, an LLM-based framework to use prompting chaining to generate transferable adversarial attacks in the form of question-answering examples. We seek to understand the extent to which these examples trigger the hallucination behaviors of LLMs. We implement AutoDebug using ChatGPT and evaluate the resulting two variants of a popular open-domain question-answering dataset, Natural Questions (NQ), on a collection of open-source and proprietary LLMs under various prompting settings. Our generated evaluation data is human-readable and, as we show, humans can answer these modified questions well. Nevertheless, we observe pronounced accuracy drops across multiple LLMs including GPT-4. Our experimental results show that LLMs are likely to hallucinate in two categories of question-answering scenarios where (1) there are conflicts between knowledge given in the prompt and their parametric knowledge, or (2) the knowledge expressed in the prompt is complex. Finally, we find that the adversarial examples generated by our method are transferable across all considered LLMs. The examples generated by a small model can be used to debug a much larger model, making our approach cost-effective.

{{}}

### (109/160) Towards Anytime Fine-tuning: Continually Pre-trained Language Models with Hypernetwork Prompt (Gangwei Jiang et al., 2023)

{{}}

Gangwei Jiang, Caigao Jiang, Siqiao Xue, James Y. Zhang, Jun Zhou, Defu Lian, Ying Wei. (2023)  
**Towards Anytime Fine-tuning: Continually Pre-trained Language Models with Hypernetwork Prompt**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs-LG, cs.CL  
Keywords: Language Model  
[Paper Link](http://arxiv.org/abs/2310.13024v1)

* * *

**ABSTRACT**  
Continual pre-training has been urgent for adapting a pre-trained model to a multitude of domains and tasks in the fast-evolving world. In practice, a continually pre-trained model is expected to demonstrate not only greater capacity when fine-tuned on pre-trained domains but also a non-decreasing performance on unseen ones. In this work, we first investigate such anytime fine-tuning effectiveness of existing continual pre-training approaches, concluding with unanimously decreased performance on unseen domains. To this end, we propose a prompt-guided continual pre-training method, where we train a hypernetwork to generate domain-specific prompts by both agreement and disagreement losses. The agreement loss maximally preserves the generalization of a pre-trained model to new domains, and the disagreement one guards the exclusiveness of the generated hidden states for each domain. Remarkably, prompts by the hypernetwork alleviate the domain identity when fine-tuning and promote knowledge transfer across domains. Our method achieved improvements of 3.57% and 3.4% on two real-world datasets (including domain shift and temporal shift), respectively, demonstrating its efficacy.

{{}}

### (110/160) GraphGPT: Graph Instruction Tuning for Large Language Models (Jiabin Tang et al., 2023)

{{}}

Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, Chao Huang. (2023)  
**GraphGPT: Graph Instruction Tuning for Large Language Models**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs.CL  
Keywords: GNN, GPT, Graph Neural Network, Graph Neural Networks, Language Model  
[Paper Link](http://arxiv.org/abs/2310.13023v1)

* * *

**ABSTRACT**  
Graph Neural Networks (GNNs) have advanced graph structure understanding via recursive information exchange and aggregation among graph nodes. To improve model robustness, self-supervised learning (SSL) has emerged as a promising approach for data augmentation. However, existing methods for generating pre-trained graph embeddings often rely on fine-tuning with specific downstream task labels, which limits their usability in scenarios where labeled data is scarce or unavailable. To address this, our research focuses on advancing the generalization capabilities of graph models in challenging zero-shot learning scenarios. Inspired by the success of large language models (LLMs), we aim to develop a graph-oriented LLM that can achieve high generalization across diverse downstream datasets and tasks, even without any information available from the downstream graph data. In this work, we present the GraphGPT framework that aligns LLMs with graph structural knowledge with a graph instruction tuning paradigm. Our framework incorporates a text-graph grounding component to establish a connection between textual information and graph structures. Additionally, we propose a dual-stage instruction tuning paradigm, accompanied by a lightweight graph-text alignment projector. This paradigm explores self-supervised graph structural signals and task-specific graph instructions, to guide LLMs in understanding complex graph structures and improving their adaptability across different downstream tasks. Our framework is evaluated on supervised and zero-shot graph learning tasks, demonstrating superior generalization and outperforming state-of-the-art baselines.

{{}}

### (111/160) Attack Prompt Generation for Red Teaming and Defending Large Language Models (Boyi Deng et al., 2023)

{{}}

Boyi Deng, Wenjie Wang, Fuli Feng, Yang Deng, Qifan Wang, Xiangnan He. (2023)  
**Attack Prompt Generation for Red Teaming and Defending Large Language Models**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs-CR, cs-LG, cs.CL  
Keywords: Language Model  
[Paper Link](http://arxiv.org/abs/2310.12505v1)

* * *

**ABSTRACT**  
Large language models (LLMs) are susceptible to red teaming attacks, which can induce LLMs to generate harmful content. Previous research constructs attack prompts via manual or automatic methods, which have their own limitations on construction cost and quality. To address these issues, we propose an integrated approach that combines manual and automatic methods to economically generate high-quality attack prompts. Specifically, considering the impressive capabilities of newly emerged LLMs, we propose an attack framework to instruct LLMs to mimic human-generated prompts through in-context learning. Furthermore, we propose a defense framework that fine-tunes victim LLMs through iterative interactions with the attack framework to enhance their safety against red teaming attacks. Extensive experiments on different LLMs validate the effectiveness of our proposed attack and defense frameworks. Additionally, we release a series of attack prompts datasets named SAP with varying sizes, facilitating the safety evaluation and enhancement of more LLMs. Our code and dataset is available on <https://github.com/Aatrox103/SAP> .

{{}}

### (112/160) Co$^2$PT: Mitigating Bias in Pre-trained Language Models through Counterfactual Contrastive Prompt Tuning (Xiangjue Dong et al., 2023)

{{}}

Xiangjue Dong, Ziwei Zhu, Zhuoer Wang, Maria Teleki, James Caverlee. (2023)  
**Co$^2$PT: Mitigating Bias in Pre-trained Language Models through Counterfactual Contrastive Prompt Tuning**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: Bias, Language Model  
[Paper Link](http://arxiv.org/abs/2310.12490v1)

* * *

**ABSTRACT**  
Pre-trained Language Models are widely used in many important real-world applications. However, recent studies show that these models can encode social biases from large pre-training corpora and even amplify biases in downstream applications. To address this challenge, we propose Co$^2$PT, an efficient and effective debias-while-prompt tuning method for mitigating biases via counterfactual contrastive prompt tuning on downstream tasks. Our experiments conducted on three extrinsic bias benchmarks demonstrate the effectiveness of Co$^2$PT on bias mitigation during the prompt tuning process and its adaptability to existing upstream debiased language models. These findings indicate the strength of Co$^2$PT and provide promising avenues for further enhancement in bias mitigation on downstream tasks.

{{}}

### (113/160) MedAI Dialog Corpus (MEDIC): Zero-Shot Classification of Doctor and AI Responses in Health Consultations (Olumide E. Ojo et al., 2023)

{{}}

Olumide E. Ojo, Olaronke O. Adebanji, Alexander Gelbukh, Hiram Calvo, Anna Feldman. (2023)  
**MedAI Dialog Corpus (MEDIC): Zero-Shot Classification of Doctor and AI Responses in Health Consultations**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: AI, Dialog, Zero-Shot  
[Paper Link](http://arxiv.org/abs/2310.12489v2)

* * *

**ABSTRACT**  
Zero-shot classification enables text to be classified into classes not seen during training. In this research, we investigate the effectiveness of pre-trained language models to accurately classify responses from Doctors and AI in health consultations through zero-shot learning. Our study aims to determine whether these models can effectively detect if a text originates from human or AI models without specific corpus training. We collect responses from doctors to patient inquiries about their health and pose the same question/response to AI models. While zero-shot language models show a good understanding of language in general, they have limitations in classifying doctor and AI responses in healthcare consultations. This research lays the groundwork for further research into this field of medical text classification, informing the development of more effective approaches to accurately classify doctor-generated and AI-generated text in health consultations.

{{}}

### (114/160) Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models (Wenxuan Wang et al., 2023)

{{}}

Wenxuan Wang, Wenxiang Jiao, Jingyuan Huang, Ruyi Dai, Jen-tse Huang, Zhaopeng Tu, Michael R. Lyu. (2023)  
**Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs.CL  
Keywords: ChatGPT, GPT, GPT-4, Language Model  
[Paper Link](http://arxiv.org/abs/2310.12481v1)

* * *

**ABSTRACT**  
In this paper, we identify a cultural dominance issue within large language models (LLMs) due to the predominant use of English data in model training (e.g. ChatGPT). LLMs often provide inappropriate English-culture-related answers that are not relevant to the expected culture when users ask in non-English languages. To systematically evaluate the cultural dominance issue, we build a benchmark that consists of both concrete (e.g. holidays and songs) and abstract (e.g. values and opinions) cultural objects. Empirical results show that the representative GPT models suffer from the culture dominance problem, where GPT-4 is the most affected while text-davinci-003 suffers the least from this problem. Our study emphasizes the need for critical examination of cultural dominance and ethical consideration in their development and deployment. We show two straightforward methods in model development (i.e. pretraining on more diverse data) and deployment (e.g. culture-aware prompting) can significantly mitigate the cultural dominance issue in LLMs.

{{}}

### (115/160) Contrastive Learning for Inference in Dialogue (Etsuko Ishii et al., 2023)

{{}}

Etsuko Ishii, Yan Xu, Bryan Wilie, Ziwei Ji, Holy Lovenia, Willy Chung, Pascale Fung. (2023)  
**Contrastive Learning for Inference in Dialogue**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: Contrastive Learning, Dialog, Dialogue  
[Paper Link](http://arxiv.org/abs/2310.12467v1)

* * *

**ABSTRACT**  
Inference, especially those derived from inductive processes, is a crucial component in our conversation to complement the information implicitly or explicitly conveyed by a speaker. While recent large language models show remarkable advances in inference tasks, their performance in inductive reasoning, where not all information is present in the context, is far behind deductive reasoning. In this paper, we analyze the behavior of the models based on the task difficulty defined by the semantic information gap -- which distinguishes inductive and deductive reasoning (Johnson-Laird, 1988, 1993). Our analysis reveals that the disparity in information between dialogue contexts and desired inferences poses a significant challenge to the inductive inference process. To mitigate this information gap, we investigate a contrastive learning approach by feeding negative samples. Our experiments suggest negative samples help models understand what is wrong and improve their inference generations.

{{}}

### (116/160) Rethinking the Construction of Effective Metrics for Understanding the Mechanisms of Pretrained Language Models (You Li et al., 2023)

{{}}

You Li, Jinhui Yin, Yuming Lin. (2023)  
**Rethinking the Construction of Effective Metrics for Understanding the Mechanisms of Pretrained Language Models**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs.CL  
Keywords: BERT, Language Model, Pretrained Language Models  
[Paper Link](http://arxiv.org/abs/2310.12454v1)

* * *

**ABSTRACT**  
Pretrained language models are expected to effectively map input text to a set of vectors while preserving the inherent relationships within the text. Consequently, designing a white-box model to compute metrics that reflect the presence of specific internal relations in these vectors has become a common approach for post-hoc interpretability analysis of pretrained language models. However, achieving interpretability in white-box models and ensuring the rigor of metric computation becomes challenging when the source model lacks inherent interpretability. Therefore, in this paper, we discuss striking a balance in this trade-off and propose a novel line to constructing metrics for understanding the mechanisms of pretrained language models. We have specifically designed a family of metrics along this line of investigation, and the model used to compute these metrics is referred to as the tree topological probe. We conducted measurements on BERT-large by using these metrics. Based on the experimental results, we propose a speculation regarding the working mechanism of BERT-like pretrained language models, as well as a strategy for enhancing fine-tuning performance by leveraging the topological probe to improve specific submodules.

{{}}

### (117/160) Efficient Long-Range Transformers: You Need to Attend More, but Not Necessarily at Every Layer (Qingru Zhang et al., 2023)

{{}}

Qingru Zhang, Dhananjay Ram, Cole Hawkins, Sheng Zha, Tuo Zhao. (2023)  
**Efficient Long-Range Transformers: You Need to Attend More, but Not Necessarily at Every Layer**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs-LG, cs.CL  
Keywords: Attention, Transformer, Transformers  
[Paper Link](http://arxiv.org/abs/2310.12442v1)

* * *

**ABSTRACT**  
Pretrained transformer models have demonstrated remarkable performance across various natural language processing tasks. These models leverage the attention mechanism to capture long- and short-range dependencies in the sequence. However, the (full) attention mechanism incurs high computational cost - quadratic in the sequence length, which is not affordable in tasks with long sequences, e.g., inputs with 8k tokens. Although sparse attention can be used to improve computational efficiency, as suggested in existing work, it has limited modeling capacity and often fails to capture complicated dependencies in long sequences. To tackle this challenge, we propose MASFormer, an easy-to-implement transformer variant with Mixed Attention Spans. Specifically, MASFormer is equipped with full attention to capture long-range dependencies, but only at a small number of layers. For the remaining layers, MASformer only employs sparse attention to capture short-range dependencies. Our experiments on natural language modeling and generation tasks show that a decoder-only MASFormer model of 1.3B parameters can achieve competitive performance to vanilla transformers with full attention while significantly reducing computational cost (up to 75%). Additionally, we investigate the effectiveness of continual training with long sequence data and how sequence length impacts downstream generation performance, which may be of independent interest.

{{}}

### (118/160) PoisonPrompt: Backdoor Attack on Prompt-based Large Language Models (Hongwei Yao et al., 2023)

{{}}

Hongwei Yao, Jian Lou, Zhan Qin. (2023)  
**PoisonPrompt: Backdoor Attack on Prompt-based Large Language Models**

* * *

Primary Category: cs.CL  
Categories: I-2-7, cs-AI, cs-CL, cs.CL  
Keywords: Language Model  
[Paper Link](http://arxiv.org/abs/2310.12439v1)

* * *

**ABSTRACT**  
Prompts have significantly improved the performance of pretrained Large Language Models (LLMs) on various downstream tasks recently, making them increasingly indispensable for a diverse range of LLM application scenarios. However, the backdoor vulnerability, a serious security threat that can maliciously alter the victim model's normal predictions, has not been sufficiently explored for prompt-based LLMs. In this paper, we present POISONPROMPT, a novel backdoor attack capable of successfully compromising both hard and soft prompt-based LLMs. We evaluate the effectiveness, fidelity, and robustness of POISONPROMPT through extensive experiments on three popular prompt methods, using six datasets and three widely used LLMs. Our findings highlight the potential security threats posed by backdoor attacks on prompt-based LLMs and emphasize the need for further research in this area.

{{}}

### (119/160) MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Models (Deepak Nathani et al., 2023)

{{}}

Deepak Nathani, David Wang, Liangming Pan, William Yang Wang. (2023)  
**MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Models**

* * *

Primary Category: cs.CL  
Categories: cs-AI, cs-CL, cs.CL  
Keywords: Language Model, Reasoning  
[Paper Link](http://arxiv.org/abs/2310.12426v1)

* * *

**ABSTRACT**  
Language Models (LMs) have shown impressive performance in various natural language tasks. However, when it comes to natural language reasoning, LMs still face challenges such as hallucination, generating incorrect intermediate reasoning steps, and making mathematical errors. Recent research has focused on enhancing LMs through self-improvement using feedback. Nevertheless, existing approaches relying on a single generic feedback source fail to address the diverse error types found in LM-generated reasoning chains. In this work, we propose Multi-Aspect Feedback, an iterative refinement framework that integrates multiple feedback modules, including frozen LMs and external tools, each focusing on a specific error category. Our experimental results demonstrate the efficacy of our approach to addressing several errors in the LM-generated reasoning chain and thus improving the overall performance of an LM in several reasoning tasks. We see a relative improvement of up to 20% in Mathematical Reasoning and up to 18% in Logical Entailment.

{{}}

### (120/160) The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions (Siru Ouyang et al., 2023)

{{}}

Siru Ouyang, Shuohang Wang, Yang Liu, Ming Zhong, Yizhu Jiao, Dan Iter, Reid Pryzant, Chenguang Zhu, Heng Ji, Jiawei Han. (2023)  
**The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: GPT, Language Model, NLP  
[Paper Link](http://arxiv.org/abs/2310.12418v1)

* * *

**ABSTRACT**  
Recent progress in Large Language Models (LLMs) has produced models that exhibit remarkable performance across a variety of NLP tasks. However, it remains unclear whether the existing focus of NLP research accurately captures the genuine requirements of human users. This paper provides a comprehensive analysis of the divergence between current NLP research and the needs of real-world NLP applications via a large-scale collection of user-GPT conversations. We analyze a large-scale collection of real user queries to GPT. We compare these queries against existing NLP benchmark tasks and identify a significant gap between the tasks that users frequently request from LLMs and the tasks that are commonly studied in academic research. For example, we find that tasks such as `design'' and `planning'' are prevalent in user interactions but are largely neglected or different from traditional NLP benchmarks. We investigate these overlooked tasks, dissect the practical challenges they pose, and provide insights toward a roadmap to make LLMs better aligned with user needs.

{{}}

### (121/160) FinEntity: Entity-level Sentiment Classification for Financial Texts (Yixuan Tang et al., 2023)

{{}}

Yixuan Tang, Yi Yang, Allen H Huang, Andy Tam, Justin Z Tang. (2023)  
**FinEntity: Entity-level Sentiment Classification for Financial Texts**

* * *

Primary Category: cs.CL  
Categories: cs-CL, cs.CL  
Keywords: BERT, ChatGPT, Financial, GPT  
[Paper Link](http://arxiv.org/abs/2310.12406v1)

* * *

**ABSTRACT**  
In the financial domain, conducting entity-level sentiment analysis is crucial for accurately assessing the sentiment directed toward a specific financial entity. To our knowledge, no publicly available dataset currently exists for this purpose. In this work, we introduce an entity-level sentiment classification dataset, called \textbf{FinEntity}, that annotates financial entity spans and their sentiment (positive, neutral, and negative) in financial news. We document the dataset construction process in the paper. Additionally, we benchmark several pre-trained models (BERT, FinBERT, etc.) and ChatGPT on entity-level sentiment classification. In a case study, we demonstrate the practical utility of using FinEntity in monitoring cryptocurrency markets. The data and code of FinEntity is available at \url{[https://github.com/yixuantt/FinEntity}](https://github.com/yixuantt/FinEntity%7D)

{{}}

## cs.RO (4)

### (122/160) Enhancing Multi-Drone Coordination for Filming Group Behaviours in Dynamic Environments (Aditya Rauniyar et al., 2023)

{{}}

Aditya Rauniyar, Jiaoyang Li, Sebastian Scherer. (2023)  
**Enhancing Multi-Drone Coordination for Filming Group Behaviours in Dynamic Environments**

* * *

Primary Category: cs.RO  
Categories: cs-RO, cs.RO  
Keywords: AI, Drone  
[Paper Link](http://arxiv.org/abs/2310.13184v1)

* * *

**ABSTRACT**  
Multi-Agent Path Finding (MAPF) is a fundamental problem in robotics and AI, with numerous applications in real-world scenarios. One such scenario is filming scenes with multiple actors, where the goal is to capture the scene from multiple angles simultaneously. Here, we present a formation-based filming directive of task assignment followed by a Conflict-Based MAPF algorithm for efficient path planning of multiple agents to achieve filming objectives while avoiding collisions. We propose an extension to the standard MAPF formulation to accommodate actor-specific requirements and constraints. Our approach incorporates Conflict-Based Search, a widely used heuristic search technique for solving MAPF problems. We demonstrate the effectiveness of our approach through experiments on various MAPF scenarios in a simulated environment. The proposed algorithm enables the efficient online task assignment of formation-based filming to capture dynamic scenes, making it suitable for various filming and coverage applications.

{{}}

### (123/160) Creative Robot Tool Use with Large Language Models (Mengdi Xu et al., 2023)

{{}}

Mengdi Xu, Peide Huang, Wenhao Yu, Shiqi Liu, Xilun Zhang, Yaru Niu, Tingnan Zhang, Fei Xia, Jie Tan, Ding Zhao. (2023)  
**Creative Robot Tool Use with Large Language Models**

* * *

Primary Category: cs.RO  
Categories: cs-AI, cs-LG, cs-RO, cs.RO  
Keywords: Language Model  
[Paper Link](http://arxiv.org/abs/2310.13065v1)

* * *

**ABSTRACT**  
Tool use is a hallmark of advanced intelligence, exemplified in both animal behavior and robotic capabilities. This paper investigates the feasibility of imbuing robots with the ability to creatively use tools in tasks that involve implicit physical constraints and long-term planning. Leveraging Large Language Models (LLMs), we develop RoboTool, a system that accepts natural language instructions and outputs executable code for controlling robots in both simulated and real-world environments. RoboTool incorporates four pivotal components: (i) an "Analyzer" that interprets natural language to discern key task-related concepts, (ii) a "Planner" that generates comprehensive strategies based on the language input and key concepts, (iii) a "Calculator" that computes parameters for each skill, and (iv) a "Coder" that translates these plans into executable Python code. Our results show that RoboTool can not only comprehend explicit or implicit physical constraints and environmental factors but also demonstrate creative tool use. Unlike traditional Task and Motion Planning (TAMP) methods that rely on explicit optimization, our LLM-based system offers a more flexible, efficient, and user-friendly solution for complex robotics tasks. Through extensive experiments, we validate that RoboTool is proficient in handling tasks that would otherwise be infeasible without the creative use of tools, thereby expanding the capabilities of robotic systems. Demos are available on our project page: <https://creative-robotool.github.io/>.

{{}}

### (124/160) CCIL: Continuity-based Data Augmentation for Corrective Imitation Learning (Liyiming Ke et al., 2023)

{{}}

Liyiming Ke, Yunchu Zhang, Abhay Deshpande, Siddhartha Srinivasa, Abhishek Gupta. (2023)  
**CCIL: Continuity-based Data Augmentation for Corrective Imitation Learning**

* * *

Primary Category: cs.RO  
Categories: cs-RO, cs.RO  
Keywords: Augmentation  
[Paper Link](http://arxiv.org/abs/2310.12972v1)

* * *

**ABSTRACT**  
We present a new technique to enhance the robustness of imitation learning methods by generating corrective data to account for compounding errors and disturbances. While existing methods rely on interactive expert labeling, additional offline datasets, or domain-specific invariances, our approach requires minimal additional assumptions beyond access to expert data. The key insight is to leverage local continuity in the environment dynamics to generate corrective labels. Our method first constructs a dynamics model from the expert demonstration, encouraging local Lipschitz continuity in the learned model. In locally continuous regions, this model allows us to generate corrective labels within the neighborhood of the demonstrations but beyond the actual set of states and actions in the dataset. Training on this augmented data enhances the agent's ability to recover from perturbations and deal with compounding errors. We demonstrate the effectiveness of our generated labels through experiments in a variety of robotics domains in simulation that have distinct forms of continuity and discontinuity, including classic control problems, drone flying, navigation with high-dimensional sensor observations, legged locomotion, and tabletop manipulation.

{{}}

### (125/160) Eureka: Human-Level Reward Design via Coding Large Language Models (Yecheng Jason Ma et al., 2023)

{{}}

Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, Anima Anandkumar. (2023)  
**Eureka: Human-Level Reward Design via Coding Large Language Models**

* * *

Primary Category: cs.RO  
Categories: cs-AI, cs-LG, cs-RO, cs.RO  
Keywords: GPT, GPT-4, Language Model  
[Paper Link](http://arxiv.org/abs/2310.12931v1)

* * *

**ABSTRACT**  
Large Language Models (LLMs) have excelled as high-level semantic planners for sequential decision-making tasks. However, harnessing them to learn complex low-level manipulation tasks, such as dexterous pen spinning, remains an open problem. We bridge this fundamental gap and present Eureka, a human-level reward design algorithm powered by LLMs. Eureka exploits the remarkable zero-shot generation, code-writing, and in-context improvement capabilities of state-of-the-art LLMs, such as GPT-4, to perform evolutionary optimization over reward code. The resulting rewards can then be used to acquire complex skills via reinforcement learning. Without any task-specific prompting or pre-defined reward templates, Eureka generates reward functions that outperform expert human-engineered rewards. In a diverse suite of 29 open-source RL environments that include 10 distinct robot morphologies, Eureka outperforms human experts on 83% of the tasks, leading to an average normalized improvement of 52%. The generality of Eureka also enables a new gradient-free in-context learning approach to reinforcement learning from human feedback (RLHF), readily incorporating human inputs to improve the quality and the safety of the generated rewards without model updating. Finally, using Eureka rewards in a curriculum learning setting, we demonstrate for the first time, a simulated Shadow Hand capable of performing pen spinning tricks, adeptly manipulating a pen in circles at rapid speed.

{{}}

## eess.SP (1)

### (126/160) Active Sensing for Localization with Reconfigurable Intelligent Surface (Zhongze Zhang et al., 2023)

{{}}

Zhongze Zhang, Tao Jiang, Wei Yu. (2023)  
**Active Sensing for Localization with Reconfigurable Intelligent Surface**

* * *

Primary Category: eess.SP  
Categories: cs-IT, eess-SP, eess.SP, math-IT  
Keywords: LSTM  
[Paper Link](http://arxiv.org/abs/2310.13160v1)

* * *

**ABSTRACT**  
This paper addresses an uplink localization problem in which the base station (BS) aims to locate a remote user with the aid of reconfigurable intelligent surface (RIS). This paper proposes a strategy in which the user transmits pilots over multiple time frames, and the BS adaptively adjusts the RIS reflection coefficients based on the observations already received so far in order to produce an accurate estimate of the user location at the end. This is a challenging active sensing problem for which finding an optimal solution involves a search through a complicated functional space whose dimension increases with the number of measurements. In this paper, we show that the long short-term memory (LSTM) network can be used to exploit the latent temporal correlation between measurements to automatically construct scalable information vectors (called hidden state) based on the measurements. Subsequently, the state vector can be mapped to the RIS configuration for the next time frame in a codebook-free fashion via a deep neural network (DNN). After all the measurements have been received, a final DNN can be used to map the LSTM cell state to the estimated user equipment (UE) position. Numerical result shows that the proposed active RIS design results in lower localization error as compared to existing active and nonactive methods. The proposed solution produces interpretable results and is generalizable to early stopping in the sequence of sensing stages.

{{}}

## cs.HC (6)

### (127/160) Understanding Generative AI in Art: An Interview Study with Artists on G-AI from an HCI Perspective (Jingyu Shi et al., 2023)

{{}}

Jingyu Shi, Rahul Jain, Runlin Duan, Karthik Ramani. (2023)  
**Understanding Generative AI in Art: An Interview Study with Artists on G-AI from an HCI Perspective**

* * *

Primary Category: cs.HC  
Categories: cs-HC, cs.HC  
Keywords: AI, Generative AI  
[Paper Link](http://arxiv.org/abs/2310.13149v1)

* * *

**ABSTRACT**  
The emergence of Generative Artificial Intelligence (G-AI) has changed the landscape of creative arts with its power to compose novel artwork and thus brought ethical concerns. Despite the efforts by prior works to address these concerns from technical and societal perspectives, there exists little discussion on this topic from an HCI point of view, considering the artists as human factors. We sought to investigate the impact of G-AI on artists, understanding the relationship between artists and G-AI, in order to motivate the underlying HCI research. We conducted semi-structured interviews with artists ($N=25$) from diverse artistic disciplines involved with G-AI in their artistic creation. We found (1) a dilemma among the artists, (2) a disparity in the understanding of G-AI between the artists and the AI developers(3) a tendency to oppose G-AI among the artists. We discuss the future opportunities of HCI research to tackle the problems identified from the interviews.

{{}}

### (128/160) Gender Biases in Error Mitigation by Voice Assistants (Amama Mahmood et al., 2023)

{{}}

Amama Mahmood, Chien-Ming Huang. (2023)  
**Gender Biases in Error Mitigation by Voice Assistants**

* * *

Primary Category: cs.HC  
Categories: cs-HC, cs.HC  
Keywords: AI, Bias  
[Paper Link](http://arxiv.org/abs/2310.13074v1)

* * *

**ABSTRACT**  
Commercial voice assistants are largely feminized and associated with stereotypically feminine traits such as warmth and submissiveness. As these assistants continue to be adopted for everyday uses, it is imperative to understand how the portrayed gender shapes the voice assistant's ability to mitigate errors, which are still common in voice interactions. We report a study (N=40) that examined the effects of voice gender (feminine, ambiguous, masculine), error mitigation strategies (apology, compensation) and participant's gender on people's interaction behavior and perceptions of the assistant. Our results show that AI assistants that apologized appeared warmer than those offered compensation. Moreover, male participants preferred apologetic feminine assistants over apologetic masculine ones. Furthermore, male participants interrupted AI assistants regardless of perceived gender more frequently than female participants when errors occurred. Our results suggest that the perceived gender of a voice assistant biases user behavior, especially for male users, and that an ambiguous voice has the potential to reduce biases associated with gender-specific traits.

{{}}

### (129/160) Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation (Sangho Suh et al., 2023)

{{}}

Sangho Suh, Meng Chen, Bryan Min, Toby Jia-Jun Li, Haijun Xia. (2023)  
**Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation**

* * *

Primary Category: cs.HC  
Categories: cs-AI, cs-HC, cs.HC  
Keywords: AI, Language Model  
[Paper Link](http://arxiv.org/abs/2310.12953v2)

* * *

**ABSTRACT**  
Thanks to their generative capabilities, large language models (LLMs) have become an invaluable tool for creative processes. These models have the capacity to produce hundreds and thousands of visual and textual outputs, offering abundant inspiration for creative endeavors. But are we harnessing their full potential? We argue that current interaction paradigms fall short, guiding users towards rapid convergence on a limited set of ideas, rather than empowering them to explore the vast latent design space in generative models. To address this limitation, we propose a framework that facilitates the structured generation of design space in which users can seamlessly explore, evaluate, and synthesize a multitude of responses. We demonstrate the feasibility and usefulness of this framework through the design and development of an interactive system, Luminate, and a user study with 8 professional writers. Our work advances how we interact with LLMs for creative tasks, introducing a way to harness the creative potential of LLMs.

{{}}

### (130/160) Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots (Xavier Puig et al., 2023)

{{}}

Xavier Puig, Eric Undersander, Andrew Szot, Mikael Dallaire Cote, Tsung-Yen Yang, Ruslan Partsey, Ruta Desai, Alexander William Clegg, Michal Hlavac, So Yeon Min, Vladimír Vondruš, Theophile Gervet, Vincent-Pierre Berges, John M. Turner, Oleksandr Maksymets, Zsolt Kira, Mrinal Kalakrishnan, Jitendra Malik, Devendra Singh Chaplot, Unnat Jain, Dhruv Batra, Akshara Rai, Roozbeh Mottaghi. (2023)  
**Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots**

* * *

Primary Category: cs.HC  
Categories: cs-AI, cs-CV, cs-GR, cs-HC, cs-MA, cs-RO, cs.HC  
Keywords: AI  
[Paper Link](http://arxiv.org/abs/2310.13724v1)

* * *

**ABSTRACT**  
We present Habitat 3.0: a simulation platform for studying collaborative human-robot tasks in home environments. Habitat 3.0 offers contributions across three dimensions: (1) Accurate humanoid simulation: addressing challenges in modeling complex deformable bodies and diversity in appearance and motion, all while ensuring high simulation speed. (2) Human-in-the-loop infrastructure: enabling real human interaction with simulated robots via mouse/keyboard or a VR interface, facilitating evaluation of robot policies with human input. (3) Collaborative tasks: studying two collaborative tasks, Social Navigation and Social Rearrangement. Social Navigation investigates a robot's ability to locate and follow humanoid avatars in unseen environments, whereas Social Rearrangement addresses collaboration between a humanoid and robot while rearranging a scene. These contributions allow us to study end-to-end learned and heuristic baselines for human-robot collaboration in-depth, as well as evaluate them with humans in the loop. Our experiments demonstrate that learned robot policies lead to efficient task completion when collaborating with unseen humanoid agents and human partners that might exhibit behaviors that the robot has not seen before. Additionally, we observe emergent behaviors during collaborative task execution, such as the robot yielding space when obstructing a humanoid agent, thereby allowing the effective completion of the task by the humanoid agent. Furthermore, our experiments using the human-in-the-loop tool demonstrate that our automated evaluation with humanoids can provide an indication of the relative ordering of different policies when evaluated with real human collaborators. Habitat 3.0 unlocks interesting new features in simulators for Embodied AI, and we hope it paves the way for a new frontier of embodied human-AI interaction capabilities.

{{}}

### (131/160) Spatial and Temporal Attention-based emotion estimation on HRI-AVC dataset (Karthik Subramanian et al., 2023)

{{}}

Karthik Subramanian, Saurav Singh, Justin Namba, Jamison Heard, Christopher Kanan, Ferat Sahin. (2023)  
**Spatial and Temporal Attention-based emotion estimation on HRI-AVC dataset**

* * *

Primary Category: cs.HC  
Categories: cs-HC, cs.HC  
Keywords: Attention  
[Paper Link](http://arxiv.org/abs/2310.12887v1)

* * *

**ABSTRACT**  
Many attempts have been made at estimating discrete emotions (calmness, anxiety, boredom, surprise, anger) and continuous emotional measures commonly used in psychology, namely `valence' (The pleasantness of the emotion being displayed) and `arousal' (The intensity of the emotion being displayed). Existing methods to estimate arousal and valence rely on learning from data sets, where an expert annotator labels every image frame. Access to an expert annotator is not always possible, and the annotation can also be tedious. Hence it is more practical to obtain self-reported arousal and valence values directly from the human in a real-time Human-Robot collaborative setting. Hence this paper provides an emotion data set (HRI-AVC) obtained while conducting a human-robot interaction (HRI) task. The self-reported pair of labels in this data set is associated with a set of image frames. This paper also proposes a spatial and temporal attention-based network to estimate arousal and valence from this set of image frames. The results show that an attention-based network can estimate valence and arousal on the HRI-AVC data set even when Arousal and Valence values are unavailable per frame.

{{}}

### (132/160) Affective Conversational Agents: Understanding Expectations and Personal Influences (Javier Hernandez et al., 2023)

{{}}

Javier Hernandez, Jina Suh, Judith Amores, Kael Rowan, Gonzalo Ramos, Mary Czerwinski. (2023)  
**Affective Conversational Agents: Understanding Expectations and Personal Influences**

* * *

Primary Category: cs.HC  
Categories: cs-AI, cs-HC, cs.HC  
Keywords: AI  
[Paper Link](http://arxiv.org/abs/2310.12459v1)

* * *

**ABSTRACT**  
The rise of AI conversational agents has broadened opportunities to enhance human capabilities across various domains. As these agents become more prevalent, it is crucial to investigate the impact of different affective abilities on their performance and user experience. In this study, we surveyed 745 respondents to understand the expectations and preferences regarding affective skills in various applications. Specifically, we assessed preferences concerning AI agents that can perceive, respond to, and simulate emotions across 32 distinct scenarios. Our results indicate a preference for scenarios that involve human interaction, emotional support, and creative tasks, with influences from factors such as emotional reappraisal and personality traits. Overall, the desired affective skills in AI agents depend largely on the application's context and nature, emphasizing the need for adaptability and context-awareness in the design of affective AI conversational agents.

{{}}

## cond-mat.mtrl-sci (1)

### (133/160) Approaches for Uncertainty Quantification of AI-predicted Material Properties: A Comparison (Francesca Tavazza et al., 2023)

{{}}

Francesca Tavazza, Kamal Choudhary, Brian DeCost. (2023)  
**Approaches for Uncertainty Quantification of AI-predicted Material Properties: A Comparison**

* * *

Primary Category: cond-mat.mtrl-sci  
Categories: cond-mat-mtrl-sci, cond-mat.mtrl-sci, cs-LG  
Keywords: AI  
[Paper Link](http://arxiv.org/abs/2310.13136v1)

* * *

**ABSTRACT**  
The development of large databases of material properties, together with the availability of powerful computers, has allowed machine learning (ML) modeling to become a widely used tool for predicting material performances. While confidence intervals are commonly reported for such ML models, prediction intervals, i.e., the uncertainty on each prediction, are not as frequently available. Here, we investigate three easy-to-implement approaches to determine such individual uncertainty, comparing them across ten ML quantities spanning energetics, mechanical, electronic, optical, and spectral properties. Specifically, we focused on the Quantile approach, the direct machine learning of the prediction intervals and Ensemble methods.

{{}}

## eess.SY (1)

### (134/160) Deep Reinforcement Learning-based Intelligent Traffic Signal Controls with Optimized CO2 emissions (Pedram Agand et al., 2023)

{{}}

Pedram Agand, Alexey Iskrov, Mo Chen. (2023)  
**Deep Reinforcement Learning-based Intelligent Traffic Signal Controls with Optimized CO2 emissions**

* * *

Primary Category: eess.SY  
Categories: cs-AI, cs-LG, cs-SY, eess-SY, eess.SY  
Keywords: Reinforcement Learning  
[Paper Link](http://arxiv.org/abs/2310.13129v2)

* * *

**ABSTRACT**  
Nowadays, transportation networks face the challenge of sub-optimal control policies that can have adverse effects on human health, the environment, and contribute to traffic congestion. Increased levels of air pollution and extended commute times caused by traffic bottlenecks make intersection traffic signal controllers a crucial component of modern transportation infrastructure. Despite several adaptive traffic signal controllers in literature, limited research has been conducted on their comparative performance. Furthermore, despite carbon dioxide (CO2) emissions' significance as a global issue, the literature has paid limited attention to this area. In this report, we propose EcoLight, a reward shaping scheme for reinforcement learning algorithms that not only reduces CO2 emissions but also achieves competitive results in metrics such as travel time. We compare the performance of tabular Q-Learning, DQN, SARSA, and A2C algorithms using metrics such as travel time, CO2 emissions, waiting time, and stopped time. Our evaluation considers multiple scenarios that encompass a range of road users (trucks, buses, cars) with varying pollution levels.

{{}}

## stat.ML (3)

### (135/160) Sequence Length Independent Norm-Based Generalization Bounds for Transformers (Jacob Trauger et al., 2023)

{{}}

Jacob Trauger, Ambuj Tewari. (2023)  
**Sequence Length Independent Norm-Based Generalization Bounds for Transformers**

* * *

Primary Category: stat.ML  
Categories: cs-LG, stat-ML, stat.ML  
Keywords: Transformer, Transformers  
[Paper Link](http://arxiv.org/abs/2310.13088v1)

* * *

**ABSTRACT**  
This paper provides norm-based generalization bounds for the Transformer architecture that do not depend on the input sequence length. We employ a covering number based approach to prove our bounds. We use three novel covering number bounds for the function class of bounded linear transformations to upper bound the Rademacher complexity of the Transformer. Furthermore, we show this generalization bound applies to the common Transformer training technique of masking and then predicting the masked word. We also run a simulated study on a sparse majority data set that empirically validates our theoretical findings.

{{}}

### (136/160) On existence, uniqueness and scalability of adversarial robustness measures for AI classifiers (Illia Horenko, 2023)

{{}}

Illia Horenko. (2023)  
**On existence, uniqueness and scalability of adversarial robustness measures for AI classifiers**

* * *

Primary Category: stat.ML  
Categories: 68T01 (Primary), 68T99, 68Q32, 86A22, 92C50 (Secondary), cs-AI, cs-LG, stat-ML, stat.ML  
Keywords: AI, GLM  
[Paper Link](http://arxiv.org/abs/2310.14421v1)

* * *

**ABSTRACT**  
Simply-verifiable mathematical conditions for existence, uniqueness and explicit analytical computation of minimal adversarial paths (MAP) and minimal adversarial distances (MAD) for (locally) uniquely-invertible classifiers, for generalized linear models (GLM), and for entropic AI (EAI) are formulated and proven. Practical computation of MAP and MAD, their comparison and interpretations for various classes of AI tools (for neuronal networks, boosted random forests, GLM and EAI) are demonstrated on the common synthetic benchmarks: on a double Swiss roll spiral and its extensions, as well as on the two biomedical data problems (for the health insurance claim predictions, and for the heart attack lethality classification). On biomedical applications it is demonstrated how MAP provides unique minimal patient-specific risk-mitigating interventions in the predefined subsets of accessible control variables.

{{}}

### (137/160) Neural Likelihood Approximation for Integer Valued Time Series Data (Luke O'Loughlin et al., 2023)

{{}}

Luke O'Loughlin, John Maclean, Andrew Black. (2023)  
**Neural Likelihood Approximation for Integer Valued Time Series Data**

* * *

Primary Category: stat.ML  
Categories: cs-LG, stat-ML, stat.ML  
Keywords: Time Series  
[Paper Link](http://arxiv.org/abs/2310.12544v1)

* * *

**ABSTRACT**  
Stochastic processes defined on integer valued state spaces are popular within the physical and biological sciences. These models are necessary for capturing the dynamics of small systems where the individual nature of the populations cannot be ignored and stochastic effects are important. The inference of the parameters of such models, from time series data, is difficult due to intractability of the likelihood; current methods, based on simulations of the underlying model, can be so computationally expensive as to be prohibitive. In this paper we construct a neural likelihood approximation for integer valued time series data using causal convolutions, which allows us to evaluate the likelihood of the whole time series in parallel. We demonstrate our method by performing inference on a number of ecological and epidemiological models, showing that we can accurately approximate the true posterior while achieving significant computational speed ups in situations where current methods struggle.

{{}}

## cs.DC (2)

### (138/160) End-to-End Delay Minimization based on Joint Optimization of DNN Partitioning and Resource Allocation for Cooperative Edge Inference (Xinrui Ye et al., 2023)

{{}}

Xinrui Ye, Yanzan Sun, Dingzhu Wen, Guanjin Pan, Shunqing Zhang. (2023)  
**End-to-End Delay Minimization based on Joint Optimization of DNN Partitioning and Resource Allocation for Cooperative Edge Inference**

* * *

Primary Category: cs.DC  
Categories: cs-DC, cs.DC  
Keywords: AI, Reinforcement Learning  
[Paper Link](http://arxiv.org/abs/2310.12937v1)

* * *

**ABSTRACT**  
Cooperative inference in Mobile Edge Computing (MEC), achieved by deploying partitioned Deep Neural Network (DNN) models between resource-constrained user equipments (UEs) and edge servers (ESs), has emerged as a promising paradigm. Firstly, we consider scenarios of continuous Artificial Intelligence (AI) task arrivals, like the object detection for video streams, and utilize a serial queuing model for the accurate evaluation of End-to-End (E2E) delay in cooperative edge inference. Secondly, to enhance the long-term performance of inference systems, we formulate a multi-slot stochastic E2E delay optimization problem that jointly considers model partitioning and multi-dimensional resource allocation. Finally, to solve this problem, we introduce a Lyapunov-guided Multi-Dimensional Optimization algorithm (LyMDO) that decouples the original problem into per-slot deterministic problems, where Deep Reinforcement Learning (DRL) and convex optimization are used for joint optimization of partitioning decisions and complementary resource allocation. Simulation results show that our approach effectively improves E2E delay while balancing long-term resource constraints.

{{}}

### (139/160) Reliable and Efficient In-Memory Fault Tolerance of Large Language Model Pretraining (Yuxin Wang et al., 2023)

{{}}

Yuxin Wang, Shaohuai Shi, Xin He, Zhenheng Tang, Xinglin Pan, Yang Zheng, Xiaoyu Wu, Amelie Chi Zhou, Bingsheng He, Xiaowen Chu. (2023)  
**Reliable and Efficient In-Memory Fault Tolerance of Large Language Model Pretraining**

* * *

Primary Category: cs.DC  
Categories: cs-DC, cs-PF, cs.DC  
Keywords: Language Model  
[Paper Link](http://arxiv.org/abs/2310.12670v1)

* * *

**ABSTRACT**  
Extensive system scales (i.e. thousands of GPU/TPUs) and prolonged training periods (i.e. months of pretraining) significantly escalate the probability of failures when training large language models (LLMs). Thus, efficient and reliable fault-tolerance methods are in urgent need. Checkpointing is the primary fault-tolerance method to periodically save parameter snapshots from GPU memory to disks via CPU memory. In this paper, we identify the frequency of existing checkpoint-based fault-tolerance being significantly limited by the storage I/O overheads, which results in hefty re-training costs on restarting from the nearest checkpoint. In response to this gap, we introduce an in-memory fault-tolerance framework for large-scale LLM pretraining. The framework boosts the efficiency and reliability of fault tolerance from three aspects: (1) Reduced Data Transfer and I/O: By asynchronously caching parameters, i.e., sharded model parameters, optimizer states, and RNG states, to CPU volatile memory, Our framework significantly reduces communication costs and bypasses checkpoint I/O. (2) Enhanced System Reliability: Our framework enhances parameter protection with a two-layer hierarchy: snapshot management processes (SMPs) safeguard against software failures, together with Erasure Coding (EC) protecting against node failures. This double-layered protection greatly improves the survival probability of the parameters compared to existing checkpointing methods. (3) Improved Snapshotting Frequency: Our framework achieves more frequent snapshotting compared with asynchronous checkpointing optimizations under the same saving time budget, which improves the fault tolerance efficiency. Empirical results demonstrate that Our framework minimizes the overhead of fault tolerance of LLM pretraining by effectively leveraging redundant CPU resources.

{{}}

## eess.IV (4)

### (140/160) Perceptual Assessment and Optimization of High Dynamic Range Image Rendering (Peibei Cao et al., 2023)

{{}}

Peibei Cao, Rafal K. Mantiuk, Kede Ma. (2023)  
**Perceptual Assessment and Optimization of High Dynamic Range Image Rendering**

* * *

Primary Category: eess.IV  
Categories: cs-CV, eess-IV, eess.IV  
Keywords: QA  
[Paper Link](http://arxiv.org/abs/2310.12877v2)

* * *

**ABSTRACT**  
High dynamic range (HDR) imaging has gained increasing popularity for its ability to faithfully reproduce the luminance levels in natural scenes. Accordingly, HDR image quality assessment (IQA) is crucial but has been superficially treated. The majority of existing IQA models are developed for and calibrated against low dynamic range (LDR) images, which have been shown to be poorly correlated with human perception of HDR image quality. In this work, we propose a family of HDR IQA models by transferring the recent advances in LDR IQA. The key step in our approach is to specify a simple inverse display model that decomposes an HDR image to a set of LDR images with different exposures, which will be assessed by existing LDR quality models. The local quality scores of each exposure are then aggregated with the help of a simple well-exposedness measure into a global quality score for each exposure, which will be further weighted across exposures to obtain the overall quality score. When assessing LDR images, the proposed HDR quality models reduce gracefully to the original LDR ones with the same performance. Experiments on four human-rated HDR image datasets demonstrate that our HDR quality models are consistently better than existing IQA methods, including the HDR-VDP family. Moreover, we demonstrate their strengths in perceptual optimization of HDR novel view synthesis.

{{}}

### (141/160) Predicting Ovarian Cancer Treatment Response in Histopathology using Hierarchical Vision Transformers and Multiple Instance Learning (Jack Breen et al., 2023)

{{}}

Jack Breen, Katie Allen, Kieran Zucker, Geoff Hall, Nishant Ravikumar, Nicolas M. Orsi. (2023)  
**Predicting Ovarian Cancer Treatment Response in Histopathology using Hierarchical Vision Transformers and Multiple Instance Learning**

* * *

Primary Category: eess.IV  
Categories: cs-AI, cs-CV, eess-IV, eess.IV  
Keywords: Transformer, Transformers  
[Paper Link](http://arxiv.org/abs/2310.12866v1)

* * *

**ABSTRACT**  
For many patients, current ovarian cancer treatments offer limited clinical benefit. For some therapies, it is not possible to predict patients' responses, potentially exposing them to the adverse effects of treatment without any therapeutic benefit. As part of the automated prediction of treatment effectiveness in ovarian cancer using histopathological images (ATEC23) challenge, we evaluated the effectiveness of deep learning to predict whether a course of treatment including the antiangiogenic drug bevacizumab could contribute to remission or prevent disease progression for at least 6 months in a set of 282 histopathology whole slide images (WSIs) from 78 ovarian cancer patients. Our approach used a pretrained Hierarchical Image Pyramid Transformer (HIPT) to extract region-level features and an attention-based multiple instance learning (ABMIL) model to aggregate features and classify whole slides. The optimal HIPT-ABMIL model had an internal balanced accuracy of 60.2% +- 2.9% and an AUC of 0.646 +- 0.033. Histopathology-specific model pretraining was found to be beneficial to classification performance, though hierarchical transformers were not, with a ResNet feature extractor achieving similar performance. Due to the dataset being small and highly heterogeneous, performance was variable across 5-fold cross-validation folds, and there were some extreme differences between validation and test set performance within folds. The model did not generalise well to tissue microarrays, with accuracy worse than random chance. It is not yet clear whether ovarian cancer WSIs contain information that can be used to accurately predict treatment response, with further validation using larger, higher-quality datasets required.

{{}}

### (142/160) A reproducible 3D convolutional neural network with dual attention module (3D-DAM) for Alzheimer's disease classification (Gia Minh Hoang et al., 2023)

{{}}

Gia Minh Hoang, Youngjoo Lee, Jae Gwan Kim. (2023)  
**A reproducible 3D convolutional neural network with dual attention module (3D-DAM) for Alzheimer's disease classification**

* * *

Primary Category: eess.IV  
Categories: cs-CV, eess-IV, eess.IV  
Keywords: AI  
[Paper Link](http://arxiv.org/abs/2310.12574v1)

* * *

**ABSTRACT**  
Alzheimer's disease is one of the most common types of neurodegenerative disease, characterized by the accumulation of amyloid-beta plaque and tau tangles. Recently, deep learning approaches have shown promise in Alzheimer's disease diagnosis. In this study, we propose a reproducible model that utilizes a 3D convolutional neural network with a dual attention module for Alzheimer's disease classification. We trained the model in the ADNI database and verified the generalizability of our method in two independent datasets (AIBL and OASIS1). Our method achieved state-of-the-art classification performance, with an accuracy of 91.94% for MCI progression classification and 96.30% for Alzheimer's disease classification on the ADNI dataset. Furthermore, the model demonstrated good generalizability, achieving an accuracy of 86.37% on the AIBL dataset and 83.42% on the OASIS1 dataset. These results indicate that our proposed approach has competitive performance and generalizability when compared to recent studies in the field.

{{}}

### (143/160) DA-TransUNet: Integrating Spatial and Channel Dual Attention with Transformer U-Net for Medical Image Segmentation (Guanqun Sun et al., 2023)

{{}}

Guanqun Sun, Yizhi Pan, Weikun Kong, Zichang Xu, Jianhua Ma, Teeradaj Racharak, Le-Minh Nguyen, Junyi Xin. (2023)  
**DA-TransUNet: Integrating Spatial and Channel Dual Attention with Transformer U-Net for Medical Image Segmentation**

* * *

Primary Category: eess.IV  
Categories: cs-CV, cs-GR, cs-LG, eess-IV, eess.IV  
Keywords: Attention, Transformer  
[Paper Link](http://arxiv.org/abs/2310.12570v1)

* * *

**ABSTRACT**  
Great progress has been made in automatic medical image segmentation due to powerful deep representation learning. The influence of transformer has led to research into its variants, and large-scale replacement of traditional CNN modules. However, such trend often overlooks the intrinsic feature extraction capabilities of the transformer and potential refinements to both the model and the transformer module through minor adjustments. This study proposes a novel deep medical image segmentation framework, called DA-TransUNet, aiming to introduce the Transformer and dual attention block into the encoder and decoder of the traditional U-shaped architecture. Unlike prior transformer-based solutions, our DA-TransUNet utilizes attention mechanism of transformer and multifaceted feature extraction of DA-Block, which can efficiently combine global, local, and multi-scale features to enhance medical image segmentation. Meanwhile, experimental results show that a dual attention block is added before the Transformer layer to facilitate feature extraction in the U-net structure. Furthermore, incorporating dual attention blocks in skip connections can enhance feature transfer to the decoder, thereby improving image segmentation performance. Experimental results across various benchmark of medical image segmentation reveal that DA-TransUNet significantly outperforms the state-of-the-art methods. The codes and parameters of our model will be publicly available at <https://github.com/SUN-1024/DA-TransUnet>.

{{}}

## cs.CR (4)

### (144/160) Prompt Injection Attacks and Defenses in LLM-Integrated Applications (Yupei Liu et al., 2023)

{{}}

Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, Neil Zhenqiang Gong. (2023)  
**Prompt Injection Attacks and Defenses in LLM-Integrated Applications**

* * *

Primary Category: cs.CR  
Categories: cs-AI, cs-CL, cs-CR, cs-LG, cs.CR  
Keywords: Language Model  
[Paper Link](http://arxiv.org/abs/2310.12815v1)

* * *

**ABSTRACT**  
Large Language Models (LLMs) are increasingly deployed as the backend for a variety of real-world applications called LLM-Integrated Applications. Multiple recent works showed that LLM-Integrated Applications are vulnerable to prompt injection attacks, in which an attacker injects malicious instruction/data into the input of those applications such that they produce results as the attacker desires. However, existing works are limited to case studies. As a result, the literature lacks a systematic understanding of prompt injection attacks and their defenses. We aim to bridge the gap in this work. In particular, we propose a general framework to formalize prompt injection attacks. Existing attacks, which are discussed in research papers and blog posts, are special cases in our framework. Our framework enables us to design a new attack by combining existing attacks. Moreover, we also propose a framework to systematize defenses against prompt injection attacks. Using our frameworks, we conduct a systematic evaluation on prompt injection attacks and their defenses with 10 LLMs and 7 tasks. We hope our frameworks can inspire future research in this field. Our code is available at <https://github.com/liu00222/Open-Prompt-Injection>.

{{}}

### (145/160) RANDGENER: Distributed Randomness Beacon from Verifiable Delay Function (Arup Mondal et al., 2023)

{{}}

Arup Mondal, Ruthu Hulikal Rooparaghunath, Debayan Gupta. (2023)  
**RANDGENER: Distributed Randomness Beacon from Verifiable Delay Function**

* * *

Primary Category: cs.CR  
Categories: cs-CR, cs.CR  
Keywords: NER  
[Paper Link](http://arxiv.org/abs/2310.12693v1)

* * *

**ABSTRACT**  
Buoyed by the excitement around secure decentralized applications, the last few decades have seen numerous constructions of distributed randomness beacons (DRB) along with use cases; however, a secure DRB (in many variations) remains an open problem. We further note that it is natural to want some kind of reward for participants who spend time and energy evaluating the randomness beacon value -- this is already common in distributed protocols. In this work, we present RandGener, a novel $n$-party commit-reveal-recover (or collaborative) DRB protocol with a novel reward and penalty mechanism along with a set of realistic guarantees. We design our protocol using trapdoor watermarkable verifiable delay functions in the RSA group setting (without requiring a trusted dealer or distributed key generation).

{{}}

### (146/160) SecurityNet: Assessing Machine Learning Vulnerabilities on Public Models (Boyang Zhang et al., 2023)

{{}}

Boyang Zhang, Zheng Li, Ziqing Yang, Xinlei He, Michael Backes, Mario Fritz, Yang Zhang. (2023)  
**SecurityNet: Assessing Machine Learning Vulnerabilities on Public Models**

* * *

Primary Category: cs.CR  
Categories: cs-CR, cs-LG, cs.CR  
Keywords: Security  
[Paper Link](http://arxiv.org/abs/2310.12665v1)

* * *

**ABSTRACT**  
While advanced machine learning (ML) models are deployed in numerous real-world applications, previous works demonstrate these models have security and privacy vulnerabilities. Various empirical research has been done in this field. However, most of the experiments are performed on target ML models trained by the security researchers themselves. Due to the high computational resource requirement for training advanced models with complex architectures, researchers generally choose to train a few target models using relatively simple architectures on typical experiment datasets. We argue that to understand ML models' vulnerabilities comprehensively, experiments should be performed on a large set of models trained with various purposes (not just the purpose of evaluating ML attacks and defenses). To this end, we propose using publicly available models with weights from the Internet (public models) for evaluating attacks and defenses on ML models. We establish a database, namely SecurityNet, containing 910 annotated image classification models. We then analyze the effectiveness of several representative attacks/defenses, including model stealing attacks, membership inference attacks, and backdoor detection on these public models. Our evaluation empirically shows the performance of these attacks/defenses can vary significantly on public models compared to self-trained models. We share SecurityNet with the research community. and advocate researchers to perform experiments on public models to better demonstrate their proposed methods' effectiveness in the future.

{{}}

### (147/160) Privacy Preserving Large Language Models: ChatGPT Case Study Based Vision and Framework (Imdad Ullah et al., 2023)

{{}}

Imdad Ullah, Najm Hassan, Sukhpal Singh Gill, Basem Suleiman, Tariq Ahamed Ahanger, Zawar Shah, Junaid Qadir, Salil S. Kanhere. (2023)  
**Privacy Preserving Large Language Models: ChatGPT Case Study Based Vision and Framework**

* * *

Primary Category: cs.CR  
Categories: cs-CR, cs.CR  
Keywords: AI, ChatGPT, GPT, Language Model, Reinforcement Learning  
[Paper Link](http://arxiv.org/abs/2310.12523v1)

* * *

**ABSTRACT**  
The generative Artificial Intelligence (AI) tools based on Large Language Models (LLMs) use billions of parameters to extensively analyse large datasets and extract critical private information such as, context, specific details, identifying information etc. This have raised serious threats to user privacy and reluctance to use such tools. This article proposes the conceptual model called PrivChatGPT, a privacy-preserving model for LLMs that consists of two main components i.e., preserving user privacy during the data curation/pre-processing together with preserving private context and the private training process for large-scale data. To demonstrate its applicability, we show how a private mechanism could be integrated into the existing model for training LLMs to protect user privacy; specifically, we employed differential privacy and private training using Reinforcement Learning (RL). We measure the privacy loss and evaluate the measure of uncertainty or randomness once differential privacy is applied. It further recursively evaluates the level of privacy guarantees and the measure of uncertainty of public database and resources, during each update when new information is added for training purposes. To critically evaluate the use of differential privacy for private LLMs, we hypothetically compared other mechanisms e..g, Blockchain, private information retrieval, randomisation, for various performance measures such as the model performance and accuracy, computational complexity, privacy vs. utility etc. We conclude that differential privacy, randomisation, and obfuscation can impact utility and performance of trained models, conversely, the use of ToR, Blockchain, and PIR may introduce additional computational complexity and high training latency. We believe that the proposed model could be used as a benchmark for proposing privacy preserving LLMs for generative AI tools.

{{}}

## cs.SE (2)

### (148/160) Patch-CLIP: A Patch-Text Pre-Trained Model (Xunzhu Tang et al., 2023)

{{}}

Xunzhu Tang, Zhenghan Chen, Saad Ezzini, Haoye Tian, Jacques Klein, Tegawende F. Bissyande. (2023)  
**Patch-CLIP: A Patch-Text Pre-Trained Model**

* * *

Primary Category: cs.SE  
Categories: cs-SE, cs.SE  
Keywords: BLEU, Pre-Trained Model  
[Paper Link](http://arxiv.org/abs/2310.12753v1)

* * *

**ABSTRACT**  
In recent years, patch representation learning has emerged as a necessary research direction for exploiting the capabilities of machine learning in software generation. These representations have driven significant performance enhancements across a variety of tasks involving code changes. While the progress is undeniable, a common limitation among existing models is their specialization: they predominantly excel in either predictive tasks, such as security patch classification, or in generative tasks such as patch description generation. This dichotomy is further exacerbated by a prevalent dependency on potentially noisy data sources. Specifically, many models utilize patches integrated with Abstract Syntax Trees (AST) that, unfortunately, may contain parsing inaccuracies, thus acting as a suboptimal source of supervision. In response to these challenges, we introduce PATCH-CLIP, a novel pre-training framework for patches and natural language text. PATCH-CLIP deploys a triple-loss training strategy for 1) patch-description contrastive learning, which enables to separate patches and descriptions in the embedding space, 2) patch-description matching, which ensures that each patch is associated to its description in the embedding space, and 3) patch-description generation, which ensures that the patch embedding is effective for generation. These losses are implemented for joint learning to achieve good performance in both predictive and generative tasks involving patches. Empirical evaluations focusing on patch description generation, demonstrate that PATCH-CLIP sets new state of the art performance, consistently outperforming the state-of-the-art in metrics like BLEU, ROUGE-L, METEOR, and Recall.

{{}}

### (149/160) Automated Repair of Declarative Software Specifications in the Era of Large Language Models (Md Rashedul Hasan et al., 2023)

{{}}

Md Rashedul Hasan, Jiawei Li, Iftekhar Ahmed, Hamid Bagheri. (2023)  
**Automated Repair of Declarative Software Specifications in the Era of Large Language Models**

* * *

Primary Category: cs.SE  
Categories: cs-AI, cs-LG, cs-SE, cs.SE  
Keywords: AI, ChatGPT, GPT, Language Model  
[Paper Link](http://arxiv.org/abs/2310.12425v1)

* * *

**ABSTRACT**  
The growing adoption of declarative software specification languages, coupled with their inherent difficulty in debugging, has underscored the need for effective and automated repair techniques applicable to such languages. Researchers have recently explored various methods to automatically repair declarative software specifications, such as template-based repair, feedback-driven iterative repair, and bounded exhaustive approaches. The latest developments in large language models provide new opportunities for the automatic repair of declarative specifications. In this study, we assess the effectiveness of utilizing OpenAI's ChatGPT to repair software specifications written in the Alloy declarative language. Unlike imperative languages, specifications in Alloy are not executed but rather translated into logical formulas and evaluated using backend constraint solvers to identify specification instances and counterexamples to assertions. Our evaluation focuses on ChatGPT's ability to improve the correctness and completeness of Alloy declarative specifications through automatic repairs. We analyze the results produced by ChatGPT and compare them with those of leading automatic Alloy repair methods. Our study revealed that while ChatGPT falls short in comparison to existing techniques, it was able to successfully repair bugs that no other technique could address. Our analysis also identified errors in ChatGPT's generated repairs, including improper operator usage, type errors, higher-order logic misuse, and relational arity mismatches. Additionally, we observed instances of hallucinations in ChatGPT-generated repairs and inconsistency in its results. Our study provides valuable insights for software practitioners, researchers, and tool builders considering ChatGPT for declarative specification repairs.

{{}}

## cs.DL (2)

### (150/160) The Botization of Science? Large-scale study of the presence and impact of Twitter bots in science dissemination (Wenceslao Arroyo-Machado et al., 2023)

{{}}

Wenceslao Arroyo-Machado, Enrique Herrera-Viedma, Daniel Torres-Salinas. (2023)  
**The Botization of Science? Large-scale study of the presence and impact of Twitter bots in science dissemination**

* * *

Primary Category: cs.DL  
Categories: cs-DL, cs-SI, cs.DL, physics-soc-ph  
Keywords: Twitter  
[Paper Link](http://arxiv.org/abs/2310.12741v1)

* * *

**ABSTRACT**  
Twitter bots are a controversial element of the platform, and their negative impact is well known. In the field of scientific communication, they have been perceived in a more positive light, and the accounts that serve as feeds alerting about scientific publications are quite common. However, despite being aware of the presence of bots in the dissemination of science, no large-scale estimations have been made nor has it been evaluated if they can truly interfere with altmetrics. Analyzing a dataset of 3,744,231 papers published between 2017 and 2021 and their associated 51,230,936 Twitter mentions, our goal was to determine the volume of publications mentioned by bots and whether they skew altmetrics indicators. Using the BotometerLite API, we categorized Twitter accounts based on their likelihood of being bots. The results showed that 11,073 accounts (0.23% of total users) exhibited automated behavior, contributing to 4.72% of all mentions. A significant bias was observed in the activity of bots. Their presence was particularly pronounced in disciplines such as Mathematics, Physics, and Space Sciences, with some specialties even exceeding 70% of the tweets. However, these are extreme cases, and the impact of this activity on altmetrics varies by speciality, with minimal influence in Arts & Humanities and Social Sciences. This research emphasizes the importance of distinguishing between specialties and disciplines when using Twitter as an altmetric.

{{}}

### (151/160) Metadata for Scientific Experiment Reporting: A Case Study in Metal-Organic Frameworks (Xintong Zhao et al., 2023)

{{}}

Xintong Zhao, Kyle Langlois, Jacob Furst, Scott McClellan, Xiaohua Hu, Yuan An, Diego A. Gómez-Gualdrón, Fernando J. Uribe-Romo, Jane Greenberg. (2023)  
**Metadata for Scientific Experiment Reporting: A Case Study in Metal-Organic Frameworks**

* * *

Primary Category: cs.DL  
Categories: cs-DL, cs.DL  
Keywords: AI  
[Paper Link](http://arxiv.org/abs/2310.12417v1)

* * *

**ABSTRACT**  
Research methods and procedures are core aspects of the research process. Metadata focused on these components is critical to supporting the FAIR principles, particularly reproducibility. The research reported on in this paper presents a methodological framework for metadata documentation supporting the reproducibility of research producing Metal Organic Frameworks (MOFs). The MOF case study involved natural language processing to extract key synthesis experiment information from a corpus of research literature. Following, a classification activity was performed by domain experts to identify entity-relation pairs. Results include: 1) a research framework for metadata design, 2) a metadata schema that includes nine entities and two relationships for reporting MOF synthesis experiments, and 3) a growing database of MOF synthesis reports structured by our metadata scheme. The metadata schema is intended to support discovery and reproducibility of metal-organic framework research and the FAIR principles. The paper provides background information, identifies the research goals and objectives, research design, results, a discussion, and the conclusion.

{{}}

## cs.NE (2)

### (152/160) LASER: Linear Compression in Wireless Distributed Optimization (Ashok Vardhan Makkuva et al., 2023)

{{}}

Ashok Vardhan Makkuva, Marco Bondaschi, Thijs Vogels, Martin Jaggi, Hyeji Kim, Michael C. Gastpar. (2023)  
**LASER: Linear Compression in Wireless Distributed Optimization**

* * *

Primary Category: cs.NE  
Categories: cs-AI, cs-IT, cs-LG, cs-NE, cs.NE, math-IT  
Keywords: GPT  
[Paper Link](http://arxiv.org/abs/2310.13033v1)

* * *

**ABSTRACT**  
Data-parallel SGD is the de facto algorithm for distributed optimization, especially for large scale machine learning. Despite its merits, communication bottleneck is one of its persistent issues. Most compression schemes to alleviate this either assume noiseless communication links, or fail to achieve good performance on practical tasks. In this paper, we close this gap and introduce LASER: LineAr CompreSsion in WirEless DistRibuted Optimization. LASER capitalizes on the inherent low-rank structure of gradients and transmits them efficiently over the noisy channels. Whilst enjoying theoretical guarantees similar to those of the classical SGD, LASER shows consistent gains over baselines on a variety of practical benchmarks. In particular, it outperforms the state-of-the-art compression schemes on challenging computer vision and GPT language modeling tasks. On the latter, we obtain $50$-$64 %$ improvement in perplexity over our baselines for noisy channels.

{{}}

### (153/160) Large Language Model for Multi-objective Evolutionary Optimization (Fei Liu et al., 2023)

{{}}

Fei Liu, Xi Lin, Zhenkun Wang, Shunyu Yao, Xialiang Tong, Mingxuan Yuan, Qingfu Zhang. (2023)  
**Large Language Model for Multi-objective Evolutionary Optimization**

* * *

Primary Category: cs.NE  
Categories: cs-AI, cs-CL, cs-ET, cs-NE, cs.NE  
Keywords: Language Model  
[Paper Link](http://arxiv.org/abs/2310.12541v1)

* * *

**ABSTRACT**  
Multiobjective evolutionary algorithms (MOEAs) are major methods for solving multiobjective optimization problems (MOPs). Many MOEAs have been proposed in the past decades, of which the operators need carefully handcrafted design with domain knowledge. Recently, some attempts have been made to replace the manually designed operators in MOEAs with learning-based operators (e.g., neural network models). However, much effort is still required for designing and training such models, and the learned operators might not generalize well to solve new problems. To tackle the above challenges, this work investigates a novel approach that leverages the powerful large language model (LLM) to design MOEA operators. With proper prompt engineering, we successfully let a general LLM serve as a black-box search operator for decomposition-based MOEA (MOEA/D) in a zero-shot manner. In addition, by learning from the LLM behavior, we further design an explicit white-box operator with randomness and propose a new version of decomposition-based MOEA, termed MOEA/D-LO. Experimental studies on different test benchmarks show that our proposed method can achieve competitive performance with widely used MOEAs. It is also promising to see the operator only learned from a few instances can have robust generalization performance on unseen problems with quite different patterns and settings. The results reveal the potential benefits of using pre-trained LLMs in the design of MOEAs.

{{}}

## cs.MM (1)

### (154/160) Generating Robust Adversarial Examples against Online Social Networks (OSNs) (Jun Liu et al., 2023)

{{}}

Jun Liu, Jiantao Zhou, Haiwei Wu, Weiwei Sun, Jinyu Tian. (2023)  
**Generating Robust Adversarial Examples against Online Social Networks (OSNs)**

* * *

Primary Category: cs.MM  
Categories: cs-CV, cs-MM, cs.MM  
Keywords: Social Network  
[Paper Link](http://arxiv.org/abs/2310.12708v1)

* * *

**ABSTRACT**  
Online Social Networks (OSNs) have blossomed into prevailing transmission channels for images in the modern era. Adversarial examples (AEs) deliberately designed to mislead deep neural networks (DNNs) are found to be fragile against the inevitable lossy operations conducted by OSNs. As a result, the AEs would lose their attack capabilities after being transmitted over OSNs. In this work, we aim to design a new framework for generating robust AEs that can survive the OSN transmission; namely, the AEs before and after the OSN transmission both possess strong attack capabilities. To this end, we first propose a differentiable network termed SImulated OSN (SIO) to simulate the various operations conducted by an OSN. Specifically, the SIO network consists of two modules: 1) a differentiable JPEG layer for approximating the ubiquitous JPEG compression and 2) an encoder-decoder subnetwork for mimicking the remaining operations. Based upon the SIO network, we then formulate an optimization framework to generate robust AEs by enforcing model outputs with and without passing through the SIO to be both misled. Extensive experiments conducted over Facebook, WeChat and QQ demonstrate that our attack methods produce more robust AEs than existing approaches, especially under small distortion constraints; the performance gain in terms of Attack Success Rate (ASR) could be more than 60%. Furthermore, we build a public dataset containing more than 10,000 pairs of AEs processed by Facebook, WeChat or QQ, facilitating future research in the robust AEs generation. The dataset and code are available at <https://github.com/csjunjun/RobustOSNAttack.git>.

{{}}

## cs.LO (1)

### (155/160) Embedding Pure Type Systems in the lambda-Pi-calculus modulo (Denis Cousineau et al., 2023)

{{}}

Denis Cousineau, Gilles Dowek. (2023)  
**Embedding Pure Type Systems in the lambda-Pi-calculus modulo**

* * *

Primary Category: cs.LO  
Categories: cs-LO, cs.LO  
Keywords: Embedding  
[Paper Link](http://arxiv.org/abs/2310.12540v1)

* * *

**ABSTRACT**  
The lambda-Pi-calculus allows to express proofs of minimal predicate logic. It can be extended, in a very simple way, by adding computation rules. This leads to the lambda-Pi-calculus modulo. We show in this paper that this simple extension is surprisingly expressive and, in particular, that all functional Pure Type Systems, such as the system F, or the Calculus of Constructions, can be embedded in it. And, moreover, that this embedding is conservative under termination hypothesis.

{{}}

## q-fin.PR (1)

### (156/160) American Option Pricing using Self-Attention GRU and Shapley Value Interpretation (Yanhui Shen, 2023)

{{}}

Yanhui Shen. (2023)  
**American Option Pricing using Self-Attention GRU and Shapley Value Interpretation**

* * *

Primary Category: q-fin.PR  
Categories: cs-LG, q-fin-PR, q-fin.PR  
Keywords: Attention, LSTM, Self-Attention  
[Paper Link](http://arxiv.org/abs/2310.12500v1)

* * *

**ABSTRACT**  
Options, serving as a crucial financial instrument, are used by investors to manage and mitigate their investment risks within the securities market. Precisely predicting the present price of an option enables investors to make informed and efficient decisions. In this paper, we propose a machine learning method for forecasting the prices of SPY (ETF) option based on gated recurrent unit (GRU) and self-attention mechanism. We first partitioned the raw dataset into 15 subsets according to moneyness and days to maturity criteria. For each subset, we matched the corresponding U.S. government bond rates and Implied Volatility Indices. This segmentation allows for a more insightful exploration of the impacts of risk-free rates and underlying volatility on option pricing. Next, we built four different machine learning models, including multilayer perceptron (MLP), long short-term memory (LSTM), self-attention LSTM, and self-attention GRU in comparison to the traditional binomial model. The empirical result shows that self-attention GRU with historical data outperforms other models due to its ability to capture complex temporal dependencies and leverage the contextual information embedded in the historical data. Finally, in order to unveil the "black box" of artificial intelligence, we employed the SHapley Additive exPlanations (SHAP) method to interpret and analyze the prediction results of the self-attention GRU model with historical data. This provides insights into the significance and contributions of different input features on the pricing of American-style options.

{{}}

## eess.AS (1)

### (157/160) An Exploration of In-Context Learning for Speech Language Model (Ming-Hao Hsu et al., 2023)

{{}}

Ming-Hao Hsu, Kai-Wei Chang, Shang-Wen Li, Hung-yi Lee. (2023)  
**An Exploration of In-Context Learning for Speech Language Model**

* * *

Primary Category: eess.AS  
Categories: cs-AI, cs-CL, eess-AS, eess.AS  
Keywords: GPT, Language Model, NLP  
[Paper Link](http://arxiv.org/abs/2310.12477v1)

* * *

**ABSTRACT**  
Ever since the development of GPT-3 in the natural language processing (NLP) field, in-context learning (ICL) has played an important role in utilizing large language models (LLMs). By presenting the LM utterance-label demonstrations at the input, the LM can accomplish few-shot learning without relying on gradient descent or requiring explicit modification of its parameters. This enables the LM to learn and adapt in a black-box manner. Despite the success of ICL in NLP, little work is exploring the possibility of ICL in speech processing. This study proposes the first exploration of ICL with a speech LM without text supervision. We first show that the current speech LM does not have the ICL capability. With the proposed warmup training, the speech LM can, therefore, perform ICL on unseen tasks. In this work, we verify the feasibility of ICL for speech LM on speech classification tasks.

{{}}

## cs.IR (1)

### (158/160) Know Where to Go: Make LLM a Relevant, Responsible, and Trustworthy Searcher (Xiang Shi et al., 2023)

{{}}

Xiang Shi, Jiawei Liu, Yinpeng Liu, Qikai Cheng, Wei Lu. (2023)  
**Know Where to Go: Make LLM a Relevant, Responsible, and Trustworthy Searcher**

* * *

Primary Category: cs.IR  
Categories: cs-AI, cs-CL, cs-IR, cs.IR  
Keywords: Language Model  
[Paper Link](http://arxiv.org/abs/2310.12443v1)

* * *

**ABSTRACT**  
The advent of Large Language Models (LLMs) has shown the potential to improve relevance and provide direct answers in web searches. However, challenges arise in validating the reliability of generated results and the credibility of contributing sources, due to the limitations of traditional information retrieval algorithms and the LLM hallucination problem. Aiming to create a "PageRank" for the LLM era, we strive to transform LLM into a relevant, responsible, and trustworthy searcher. We propose a novel generative retrieval framework leveraging the knowledge of LLMs to foster a direct link between queries and online sources. This framework consists of three core modules: Generator, Validator, and Optimizer, each focusing on generating trustworthy online sources, verifying source reliability, and refining unreliable sources, respectively. Extensive experiments and evaluations highlight our method's superior relevance, responsibility, and trustfulness against various SOTA methods.

{{}}

## q-bio.NC (1)

### (159/160) AI for Mathematics: A Cognitive Science Perspective (Cedegao E. Zhang et al., 2023)

{{}}

Cedegao E. Zhang, Katherine M. Collins, Adrian Weller, Joshua B. Tenenbaum. (2023)  
**AI for Mathematics: A Cognitive Science Perspective**

* * *

Primary Category: q-bio.NC  
Categories: cs-AI, q-bio-NC, q-bio.NC  
Keywords: AI  
[Paper Link](http://arxiv.org/abs/2310.13021v1)

* * *

**ABSTRACT**  
Mathematics is one of the most powerful conceptual systems developed and used by the human species. Dreams of automated mathematicians have a storied history in artificial intelligence (AI). Rapid progress in AI, particularly propelled by advances in large language models (LLMs), has sparked renewed, widespread interest in building such systems. In this work, we reflect on these goals from a \textit{cognitive science} perspective. We call attention to several classical and ongoing research directions from cognitive science, which we believe are valuable for AI practitioners to consider when seeking to build truly human (or superhuman)-level mathematical systems. We close with open discussions and questions that we believe necessitate a multi-disciplinary perspective -- cognitive scientists working in tandem with AI researchers and mathematicians -- as we move toward better mathematical AI systems which not only help us push the frontier of the mathematics, but also offer glimpses into how we as humans are even capable of such great cognitive feats.

{{}}

## cs.SD (1)

### (160/160) Loop Copilot: Conducting AI Ensembles for Music Generation and Iterative Editing (Yixiao Zhang et al., 2023)

{{}}

Yixiao Zhang, Akira Maezawa, Gus Xia, Kazuhiko Yamamoto, Simon Dixon. (2023)  
**Loop Copilot: Conducting AI Ensembles for Music Generation and Iterative Editing**

* * *

Primary Category: cs.SD  
Categories: cs-CL, cs-HC, cs-LG, cs-SD, cs.SD, eess-AS  
Keywords: AI  
[Paper Link](http://arxiv.org/abs/2310.12404v1)

* * *

**ABSTRACT**  
Creating music is iterative, requiring varied methods at each stage. However, existing AI music systems fall short in orchestrating multiple subsystems for diverse needs. To address this gap, we introduce Loop Copilot, a novel system that enables users to generate and iteratively refine music through an interactive, multi-round dialogue interface. The system uses a large language model to interpret user intentions and select appropriate AI models for task execution. Each backend model is specialized for a specific task, and their outputs are aggregated to meet the user's requirements. To ensure musical coherence, essential attributes are maintained in a centralized table. We evaluate the effectiveness of the proposed system through semi-structured interviews and questionnaires, highlighting its utility not only in facilitating music creation but also its potential for broader applications.

{{}}

## Footer

[ ](https://github.com "GitHub") (C) 2025 GitHub, Inc. 

### Footer navigation

  * [Terms](https://docs.github.com/site-policy/github-terms/github-terms-of-service)
  * [Privacy](https://docs.github.com/site-policy/privacy-policies/github-privacy-statement)
  * [Security](https://github.com/security)
  * [Status](https://www.githubstatus.com/)
  * [Docs](https://docs.github.com/)
  * [Contact](https://support.github.com?tags=dotcom-footer)
  * Manage cookies 
  * Do not share my personal information 



You can’t perform that action at this time. 


scrape_date: 2025-03-17T13:26:50.813675
content_type: markdown
hash: d3a796cbd57bba5ad4235c72949c2cb78ad2158ee47a9269ff556e1828b7118c
url:
https://github.com/nullscc/nullscc.github.io/blob/b54b57dfcce90728078dfc6ff9e1e8e078ef1a63/2023/11/16/cs.AI_2023_11_16/index.html

title:
nullscc.github.io/2023/11/16/cs.AI_2023_11_16/index.html at b54b57dfcce90728078dfc6ff9e1e8e078ef1a63 · nullscc/nullscc.github.io · GitHub

content:
Skip to content

## Navigation Menu

Toggle navigation

[ ](/)

[ Sign in ](/login?return_to=https%3A%2F%2Fgithub.com%2Fnullscc%2Fnullscc.github.io%2Fblob%2Fb54b57dfcce90728078dfc6ff9e1e8e078ef1a63%2F2023%2F11%2F16%2Fcs.AI_2023_11_16%2Findex.html)

  * Product 

    * [ GitHub Copilot Write better code with AI  ](https://github.com/features/copilot)
    * [ Security Find and fix vulnerabilities  ](https://github.com/features/security)
    * [ Actions Automate any workflow  ](https://github.com/features/actions)
    * [ Codespaces Instant dev environments  ](https://github.com/features/codespaces)
    * [ Issues Plan and track work  ](https://github.com/features/issues)
    * [ Code Review Manage code changes  ](https://github.com/features/code-review)
    * [ Discussions Collaborate outside of code  ](https://github.com/features/discussions)
    * [ Code Search Find more, search less  ](https://github.com/features/code-search)

Explore
    * [ All features ](https://github.com/features)
    * [ Documentation  ](https://docs.github.com)
    * [ GitHub Skills  ](https://skills.github.com)
    * [ Blog  ](https://github.blog)

  * Solutions 

By company size
    * [ Enterprises ](https://github.com/enterprise)
    * [ Small and medium teams ](https://github.com/team)
    * [ Startups ](https://github.com/enterprise/startups)
    * [ Nonprofits ](/solutions/industry/nonprofits)

By use case
    * [ DevSecOps ](/solutions/use-case/devsecops)
    * [ DevOps ](/solutions/use-case/devops)
    * [ CI/CD ](/solutions/use-case/ci-cd)
    * [ View all use cases ](/solutions/use-case)

By industry
    * [ Healthcare ](/solutions/industry/healthcare)
    * [ Financial services ](/solutions/industry/financial-services)
    * [ Manufacturing ](/solutions/industry/manufacturing)
    * [ Government ](/solutions/industry/government)
    * [ View all industries ](/solutions/industry)

[ View all solutions  ](/solutions)

  * Resources 

Topics
    * [ AI ](/resources/articles/ai)
    * [ DevOps ](/resources/articles/devops)
    * [ Security ](/resources/articles/security)
    * [ Software Development ](/resources/articles/software-development)
    * [ View all ](/resources/articles)

Explore
    * [ Learning Pathways  ](https://resources.github.com/learn/pathways)
    * [ Events & Webinars  ](https://resources.github.com)
    * [ Ebooks & Whitepapers ](https://github.com/resources/whitepapers)
    * [ Customer Stories ](https://github.com/customer-stories)
    * [ Partners  ](https://partner.github.com)
    * [ Executive Insights ](https://github.com/solutions/executive-insights)

  * Open Source 

    * [ GitHub Sponsors Fund open source developers  ](/sponsors)

    * [ The ReadME Project GitHub community articles  ](https://github.com/readme)

Repositories
    * [ Topics ](https://github.com/topics)
    * [ Trending ](https://github.com/trending)
    * [ Collections ](https://github.com/collections)

  * Enterprise 

    * [ Enterprise platform AI-powered developer platform  ](/enterprise)

Available add-ons
    * [ Advanced Security Enterprise-grade security features  ](https://github.com/enterprise/advanced-security)
    * [ Copilot for business Enterprise-grade AI features  ](/features/copilot/copilot-business)
    * [ Premium Support Enterprise-grade 24/7 support  ](/premium-support)

  * [Pricing](https://github.com/pricing)



Search or jump to...

# Search code, repositories, users, issues, pull requests...

Search 

Clear




[Search syntax tips](https://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax)

#  Provide feedback 

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Cancel  Submit feedback 

#  Saved searches 

## Use saved searches to filter your results more quickly

Name

Query

To see all available qualifiers, see our [documentation](https://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax). 

Cancel  Create saved search 

[ Sign in ](/login?return_to=https%3A%2F%2Fgithub.com%2Fnullscc%2Fnullscc.github.io%2Fblob%2Fb54b57dfcce90728078dfc6ff9e1e8e078ef1a63%2F2023%2F11%2F16%2Fcs.AI_2023_11_16%2Findex.html)

[ Sign up ](/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fblob%2Fshow&source=header-repo&source_repo=nullscc%2Fnullscc.github.io) Reseting focus

You signed in with another tab or window. [Reload]() to refresh your session. You signed out in another tab or window. [Reload]() to refresh your session. You switched accounts on another tab or window. [Reload]() to refresh your session. Dismiss alert

{{ message }}

[ nullscc ](/nullscc) / **[nullscc.github.io](/nullscc/nullscc.github.io) ** Public

  * [ Notifications ](/login?return_to=%2Fnullscc%2Fnullscc.github.io) You must be signed in to change notification settings
  * [ Fork 0 ](/login?return_to=%2Fnullscc%2Fnullscc.github.io)
  * [ Star  8 ](/login?return_to=%2Fnullscc%2Fnullscc.github.io)




  * [ Code ](/nullscc/nullscc.github.io)
  * [ Issues 0 ](/nullscc/nullscc.github.io/issues)
  * [ Pull requests 0 ](/nullscc/nullscc.github.io/pulls)
  * [ Actions ](/nullscc/nullscc.github.io/actions)
  * [ Projects 0 ](/nullscc/nullscc.github.io/projects)
  * [ Security ](/nullscc/nullscc.github.io/security)
  * [ Insights ](/nullscc/nullscc.github.io/pulse)



Additional navigation options

  * [ Code  ](/nullscc/nullscc.github.io)
  * [ Issues  ](/nullscc/nullscc.github.io/issues)
  * [ Pull requests  ](/nullscc/nullscc.github.io/pulls)
  * [ Actions  ](/nullscc/nullscc.github.io/actions)
  * [ Projects  ](/nullscc/nullscc.github.io/projects)
  * [ Security  ](/nullscc/nullscc.github.io/security)
  * [ Insights  ](/nullscc/nullscc.github.io/pulse)



## Files

b54b57d

## Breadcrumbs

  1. [nullscc.github.io](/nullscc/nullscc.github.io/tree/b54b57dfcce90728078dfc6ff9e1e8e078ef1a63)
  2. /[2023](/nullscc/nullscc.github.io/tree/b54b57dfcce90728078dfc6ff9e1e8e078ef1a63/2023)
  3. /[11](/nullscc/nullscc.github.io/tree/b54b57dfcce90728078dfc6ff9e1e8e078ef1a63/2023/11)
  4. /[16](/nullscc/nullscc.github.io/tree/b54b57dfcce90728078dfc6ff9e1e8e078ef1a63/2023/11/16)
  5. /[cs.AI_2023_11_16](/nullscc/nullscc.github.io/tree/b54b57dfcce90728078dfc6ff9e1e8e078ef1a63/2023/11/16/cs.AI_2023_11_16)



/

# index.html

Copy path

Blame

Blame

## Latest commit

## History

[History](/nullscc/nullscc.github.io/commits/b54b57dfcce90728078dfc6ff9e1e8e078ef1a63/2023/11/16/cs.AI_2023_11_16/index.html)

[](/nullscc/nullscc.github.io/commits/b54b57dfcce90728078dfc6ff9e1e8e078ef1a63/2023/11/16/cs.AI_2023_11_16/index.html)

1627 lines (1526 loc) · 310 KB

b54b57d

## Breadcrumbs

  1. [nullscc.github.io](/nullscc/nullscc.github.io/tree/b54b57dfcce90728078dfc6ff9e1e8e078ef1a63)
  2. /[2023](/nullscc/nullscc.github.io/tree/b54b57dfcce90728078dfc6ff9e1e8e078ef1a63/2023)
  3. /[11](/nullscc/nullscc.github.io/tree/b54b57dfcce90728078dfc6ff9e1e8e078ef1a63/2023/11)
  4. /[16](/nullscc/nullscc.github.io/tree/b54b57dfcce90728078dfc6ff9e1e8e078ef1a63/2023/11/16)
  5. /[cs.AI_2023_11_16](/nullscc/nullscc.github.io/tree/b54b57dfcce90728078dfc6ff9e1e8e078ef1a63/2023/11/16/cs.AI_2023_11_16)



/

# index.html

Top

## File metadata and controls

  * Code

  * Blame




1627 lines (1526 loc) · 310 KB

[Raw](https://github.com/nullscc/nullscc.github.io/raw/b54b57dfcce90728078dfc6ff9e1e8e078ef1a63/2023/11/16/cs.AI_2023_11_16/index.html)

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

484

485

486

487

488

489

490

491

492

493

494

495

496

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

520

521

522

523

524

525

526

527

528

529

530

531

532

533

534

535

536

537

538

539

540

541

542

543

544

545

546

547

548

549

550

551

552

553

554

555

556

557

558

559

560

561

562

563

564

565

566

567

568

569

570

571

572

573

574

575

576

577

578

579

580

581

582

583

584

585

586

587

588

589

590

591

592

593

594

595

596

597

598

599

600

601

602

603

604

605

606

607

608

609

610

611

612

613

614

615

616

617

618

619

620

621

622

623

624

625

626

627

628

629

630

631

632

633

634

635

636

637

638

639

640

641

642

643

644

645

646

647

648

649

650

651

652

653

654

655

656

657

658

659

660

661

662

663

664

665

666

667

668

669

670

671

672

673

674

675

676

677

678

679

680

681

682

683

684

685

686

687

688

689

690

691

692

693

694

695

696

697

698

699

700

701

702

703

704

705

706

707

708

709

710

711

712

713

714

715

716

717

718

719

720

721

722

723

724

725

726

727

728

729

730

731

732

733

734

735

736

737

738

739

740

741

742

743

744

745

746

747

748

749

750

751

752

753

754

755

756

757

758

759

760

761

762

763

764

765

766

767

768

769

770

771

772

773

774

775

776

777

778

779

780

781

782

783

784

785

786

787

788

789

790

791

792

793

794

795

796

797

798

799

800

801

802

803

804

805

806

807

808

809

810

811

812

813

814

815

816

817

818

819

820

821

822

823

824

825

826

827

828

829

830

831

832

833

834

835

836

837

838

839

840

841

842

843

844

845

846

847

848

849

850

851

852

853

854

855

856

857

858

859

860

861

862

863

864

865

866

867

868

869

870

871

872

873

874

875

876

877

878

879

880

881

882

883

884

885

886

887

888

889

890

891

892

893

894

895

896

897

898

899

900

901

902

903

904

905

906

907

908

909

910

911

912

913

914

915

916

917

918

919

920

921

922

923

924

925

926

927

928

929

930

931

932

933

934

935

936

937

938

939

940

941

942

943

944

945

946

947

948

949

950

951

952

953

954

955

956

957

958

959

960

961

962

963

964

965

966

967

968

969

970

971

972

973

974

975

976

977

978

979

980

981

982

983

984

985

986

987

988

989

990

991

992

993

994

995

996

997

998

999

1000

<!DOCTYPE html>

<html>

<head>

<meta charset="UTF-8">

<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1" />

<title>cs.AI - 2023-11-16 | Fun Paper</title>

<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">

<meta name="description" content="The Analysis and Extraction of Structure from Organizational Charts paper_url: http:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2311.10234 repo_url: None paper_authors: Nikhil Manali, David Doermann, Mahesh Desai for: 提供一种自动化和端到">

<meta property="og:type" content="article">

<meta property="og:title" content="cs.AI - 2023-11-16">

<meta property="og:url" content="https://nullscc.github.io/2023/11/16/cs.AI_2023_11_16/index.html">

<meta property="og:site_name" content="Fun Paper">

<meta property="og:description" content="The Analysis and Extraction of Structure from Organizational Charts paper_url: http:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2311.10234 repo_url: None paper_authors: Nikhil Manali, David Doermann, Mahesh Desai for: 提供一种自动化和端到">

<meta property="og:locale" content="en_US">

<meta property="article:published_time" content="2023-11-16T12:00:00.000Z">

<meta property="article:modified_time" content="2023-11-21T10:06:13.587Z">

<meta property="article:author" content="nullscc">

<meta name="twitter:card" content="summary">

<link rel="stylesheet" href="/css/style.css">

<!--[if lt IE 9]><script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7/html5shiv.min.js"></script><![endif]-->

<meta name="generator" content="Hexo 6.3.0"></head>

<body>

<div id="container">

<div id="wrap">

<header id="header">

<div id="banner"></div>

<div id="header-outer" class="outer">

<nav id="upper-nav" class="inner">

<a id="main-nav-toggle" class="nav-icon"></a>

<div class="sub-nav">

<a id="nav-github" class="nav-icon" target="_blank" rel="noopener" href="https://github.com/nullscc"></a>

</div>

</nav>

<div id="header-title">

<h1 id="blog-title-wrap">

<a href="/" id="blog-title">Fun Paper</a>

</h1>

</div>

<div id="contenedor">

<ul class="cube">

<li class="cara">Paper</li>

<li class="cara"></li>

<li class="cara"></li>

<li class="cara"></li>

<li class="cara"></li>

<li class="cara"></li>

</ul>

</div>

<nav id="main-nav">

<a class="main-nav-link" href="/">Home</a>

<a class="main-nav-link" href="/archives">Archives</a>

</nav>

</div>

</header>

<div class="outer">

<section id="main"><article id="post-cs.AI_2023_11_16" class="article article-type-post" itemscope itemprop="blogPost">

<div class="article-meta">

<h3 href="/2023/11/16/cs.AI_2023_11_16/" class="article-date">

<time datetime="2023-11-16T12:00:00.000Z" itemprop="datePublished">2023-11-16</time>

</h3>

<div class="article-category">

<a class="article-category-link" href="/categories/cs-AI/">cs.AI</a>

</div>

</div>

<div class="article-inner">

<div class="curve-down">

<div class="fill-content">

<header class="article-header">

<h1 class="article-title" itemprop="name">

cs.AI - 2023-11-16

</h1>

</header>

<div class="article-entry" itemprop="articleBody">

<h2 id="The-Analysis-and-Extraction-of-Structure-from-Organizational-Charts"><a href="#The-Analysis-and-Extraction-of-Structure-from-Organizational-Charts" class="headerlink" title="The Analysis and Extraction of Structure from Organizational Charts"></a>The Analysis and Extraction of Structure from Organizational Charts</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10234">http://arxiv.org/abs/2311.10234</a></li>

<li>repo_url: None</li>

<li>paper_authors: Nikhil Manali, David Doermann, Mahesh Desai</li>

<li>for: 提供一种自动化和端到端的方法，用于从组织图中提取信息，以解决手动提取信息的困难和时间消耗问题。</li>

<li>methods: 使用计算机视觉、深度学习和自然语言处理技术来自动提取组织图中的信息。</li>

<li>results: 提出一种用于评估提取信息的完整性和层次准确性的度量，并通过实验证明该方法的有效性。<details>

<summary>Abstract</summary>

Organizational charts, also known as org charts, are critical representations of an organization's structure and the hierarchical relationships between its components and positions. However, manually extracting information from org charts can be error-prone and time-consuming. To solve this, we present an automated and end-to-end approach that uses computer vision, deep learning, and natural language processing techniques. Additionally, we propose a metric to evaluate the completeness and hierarchical accuracy of the extracted information. This approach has the potential to improve organizational restructuring and resource utilization by providing a clear and concise representation of the organizational structure. Our study lays a foundation for further research on the topic of hierarchical chart analysis.

</details>

<details>

<summary>摘要</summary>

organizational charts, also known as org charts, are critical representations of an organization's structure and the hierarchical relationships between its components and positions. however, manually extracting information from org charts can be error-prone and time-consuming. to solve this, we present an automated and end-to-end approach that uses computer vision, deep learning, and natural language processing techniques. additionally, we propose a metric to evaluate the completeness and hierarchical accuracy of the extracted information. this approach has the potential to improve organizational restructuring and resource utilization by providing a clear and concise representation of the organizational structure. our study lays a foundation for further research on the topic of hierarchical chart analysis.Note: The translation is in Simplified Chinese, which is the standard writing system used in mainland China. If you need the translation in Traditional Chinese, please let me know.

</details></li>

</ul>

<hr>

<h2 id="A-Graphical-Model-of-Hurricane-Evacuation-Behaviors"><a href="#A-Graphical-Model-of-Hurricane-Evacuation-Behaviors" class="headerlink" title="A Graphical Model of Hurricane Evacuation Behaviors"></a>A Graphical Model of Hurricane Evacuation Behaviors</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10228">http://arxiv.org/abs/2311.10228</a></li>

<li>repo_url: None</li>

<li>paper_authors: Hui Sophie Wang, Nutchanon Yongsatianchot, Stacy Marsella</li>

<li>for: 这 paper 的目的是研究人们在风暴来临时是否离开家园的决策，以及这些决策如何影响紧急准备和应急响应。</li>

<li>methods: 这 paper 使用了 Protection motivation theory (PMT) 框架，构建了风暴离开决策的复杂关系图，并通过 conditional independence tests 评估不同的图 Structures。</li>

<li>results: 研究发现，人们的风暴离开决策受到了威胁评估（threat appraisal）和应急 coping 评估的直接影响，以及媒体信息的直接和间接影响。 certain information received from media 影响了威胁评估，并通过它影响了风暴离开行为。此外，一些变量直接影响了风暴离开行为和威胁评估，包括家人和朋友的建议，邻居的离开行为，以及官员发布的离开通知。<details>

<summary>Abstract</summary>

Natural disasters such as hurricanes are increasing and causing widespread devastation. People's decisions and actions regarding whether to evacuate or not are critical and have a large impact on emergency planning and response. Our interest lies in computationally modeling complex relationships among various factors influencing evacuation decisions. We conducted a study on the evacuation of Hurricane Irma of the 2017 Atlantic hurricane season. The study was guided by the Protection motivation theory (PMT), a widely-used framework to understand people's responses to potential threats. Graphical models were constructed to represent the complex relationships among the factors involved and the evacuation decision. We evaluated different graphical structures based on conditional independence tests using Irma data. The final model largely aligns with PMT. It shows that both risk perception (threat appraisal) and difficulties in evacuation (coping appraisal) influence evacuation decisions directly and independently. Certain information received from media was found to influence risk perception, and through it influence evacuation behaviors indirectly. In addition, several variables were found to influence both risk perception and evacuation behaviors directly, including family and friends' suggestions, neighbors' evacuation behaviors, and evacuation notices from officials.

</details>

<details>

<summary>摘要</summary>

自然灾害如飓风减少不断，引起广泛的破坏。人们的逃离或不逃离的决定对紧急准备和应急应对有着重要的影响。我们的兴趣在于通过计算模型来模拟人们逃离决定的复杂关系。我们在2017年大西洋飓风赛季的飓风艾尔马事例进行了研究。研究受保护动机理论（PMT）的导向，这是解释人们面临可能威胁的响应的广泛使用的框架。我们使用图表模型来表示逃离决定中的复杂关系，并对逃离决定进行了不同的图表结构的评估。我们根据飓风艾尔马数据进行了条件独立测试，最终模型大致与PMT相符。它表明，风险识别（威胁评估）和逃离困难（处理评估）都直接和独立地影响逃离决定。媒体上接受的信息也影响了风险识别，并通过它影响了逃离行为。此外，一些变量直接和 indirectly影响了逃离决定，包括家庭和朋友的建议、邻居的逃离行为以及官员发布的逃离通知。

</details></li>

</ul>

<hr>

<h2 id="Think-Twice-Perspective-Taking-Improves-Large-Language-Models’-Theory-of-Mind-Capabilities"><a href="#Think-Twice-Perspective-Taking-Improves-Large-Language-Models’-Theory-of-Mind-Capabilities" class="headerlink" title="Think Twice: Perspective-Taking Improves Large Language Models’ Theory-of-Mind Capabilities"></a>Think Twice: Perspective-Taking Improves Large Language Models’ Theory-of-Mind Capabilities</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10227">http://arxiv.org/abs/2311.10227</a></li>

<li>repo_url: None</li>

<li>paper_authors: Alex Wilf, Sihyun Shawn Lee, Paul Pu Liang, Louis-Philippe Morency</li>

<li>for: 本研究旨在提高现有的大自然语言模型（LLM） Theory of Mind（ToM）能力。</li>

<li>methods: 本研究提出了一种新的两阶段引导框架，名为SimToM，它基于认知科学理论“模拟理论”中的视角变换。</li>

<li>results: 对当前 ToM benchmark 进行应用，SimToM 方法显示了明显的改善，而且我们的分析表明了对理论知识的了解对 ToM 能力的重要性。<details>

<summary>Abstract</summary>

Human interactions are deeply rooted in the interplay of thoughts, beliefs, and desires made possible by Theory of Mind (ToM): our cognitive ability to understand the mental states of ourselves and others. Although ToM may come naturally to us, emulating it presents a challenge to even the most advanced Large Language Models (LLMs). Recent improvements to LLMs' reasoning capabilities from simple yet effective prompting techniques such as Chain-of-Thought have seen limited applicability to ToM. In this paper, we turn to the prominent cognitive science theory "Simulation Theory" to bridge this gap. We introduce SimToM, a novel two-stage prompting framework inspired by Simulation Theory's notion of perspective-taking. To implement this idea on current ToM benchmarks, SimToM first filters context based on what the character in question knows before answering a question about their mental state. Our approach, which requires no additional training and minimal prompt-tuning, shows substantial improvement over existing methods, and our analysis reveals the importance of perspective-taking to Theory-of-Mind capabilities. Our findings suggest perspective-taking as a promising direction for future research into improving LLMs' ToM capabilities.

</details>

<details>

<summary>摘要</summary>

人类互动深受理智思维、信念和愿望的互动，这些思维是通过理智思维（ToM）实现的：我们的认知能力理解自己和他人的心理状态。虽然ToM可能是自然的，但模拟它对even最高级语言模型（LLMs）来说是一项挑战。现有的LLMs的理解能力的改进从简单而有效的提示技术such as Chain-of-Thought中有限的应用于ToM。在这篇论文中，我们转向了著名的认知科学理论“模拟理论”来bridging这个差距。我们提出了一种新的两个阶段的提示框架，称为SimToM，它是基于模拟理论中的看法拟合的想法。为了在当前的ToM标准benchmark上实现这个想法，SimToM首先根据character知道的信息过滤上下文，然后回答关于其心理状态的问题。我们的方法不需要额外的训练和微小的提示调整，而且与现有的方法比较，我们的结果表明了看法拟合的重要性，以及它在理智思维能力方面的潜在性。我们的发现建议将 perspective-taking作为未来研究理智思维能力的可能方向。

</details></li>

</ul>

<hr>

<h2 id="A-Language-and-Its-Dimensions-Intrinsic-Dimensions-of-Language-Fractal-Structures"><a href="#A-Language-and-Its-Dimensions-Intrinsic-Dimensions-of-Language-Fractal-Structures" class="headerlink" title="A Language and Its Dimensions: Intrinsic Dimensions of Language Fractal Structures"></a>A Language and Its Dimensions: Intrinsic Dimensions of Language Fractal Structures</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10217">http://arxiv.org/abs/2311.10217</a></li>

<li>repo_url: None</li>

<li>paper_authors: Vasilii A. Gromov, Nikita S. Borodin, Asel S. Yerbolova</li>

<li>for: The paper is written to introduce a new object of study - a language fractal structure, and to estimate the intrinsic dimensions of language fractal structures for the Russian and English languages.</li>

<li>methods: The paper uses methods based on topological data analysis and a minimum spanning tree of a data graph to estimate the intrinsic dimensions of language fractal structures.</li>

<li>results: The paper finds that the intrinsic dimensions of language fractal structures for both the Russian and English languages are non-integer values, close to 9 for both languages.Here is the information in Simplified Chinese text, as requested:</li>

<li>for: 本研究对自然语言中的语言异步结构进行了新的研究，并估计了俄语和英语语言异步结构的内在维度。</li>

<li>methods: 本研究使用了基于拓扑数据分析和数据图中最小杆的方法来估计语言异步结构的内在维度。</li>

<li>results: 研究发现，俄语和英语语言异步结构的内在维度都是非整数值，都接近9。<details>

<summary>Abstract</summary>

The present paper introduces a novel object of study - a language fractal structure. We hypothesize that a set of embeddings of all $n$-grams of a natural language constitutes a representative sample of this fractal set. (We use the term Hailonakea to refer to the sum total of all language fractal structures, over all $n$). The paper estimates intrinsic (genuine) dimensions of language fractal structures for the Russian and English languages. To this end, we employ methods based on (1) topological data analysis and (2) a minimum spanning tree of a data graph for a cloud of points considered (Steele theorem). For both languages, for all $n$, the intrinsic dimensions appear to be non-integer values (typical for fractal sets), close to 9 for both of the Russian and English language.

</details>

<details>

<summary>摘要</summary>

本文介绍一种新的研究对象——语言自similarity结构。我们假设所有自然语言中的ngrams集合可以视为这种自similarity结构的代表样本。（我们使用“Hailonakea”这个 термин来描述所有语言自similarity结构的总和，随着n的变化）。本文对俄语和英语两种语言的语言自similarity结构进行了估计。为此，我们使用了基于拓扑数据分析和最小杆法（Steele theorem）的方法。对于两种语言和所有n，内部维度都显示为非整数值（典型的自similarity集合特征），约等于9。

</details></li>

</ul>

<hr>

<h2 id="Predictive-Minds-LLMs-As-Atypical-Active-Inference-Agents"><a href="#Predictive-Minds-LLMs-As-Atypical-Active-Inference-Agents" class="headerlink" title="Predictive Minds: LLMs As Atypical Active Inference Agents"></a>Predictive Minds: LLMs As Atypical Active Inference Agents</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10215">http://arxiv.org/abs/2311.10215</a></li>

<li>repo_url: None</li>

<li>paper_authors: Jan Kulveit, Clem von Stengel, Roman Leventov</li>

<li>for: 本文探讨大语言模型（LLM）如何理解和使用active inference理论。</li>

<li>methods: 文章比较传统的active inference系统和LLM的相似之处和差异，并结论LLM目前缺乏与行动在世界中产生影响的紧密反馈循环，但其他地方符合active inference模式。</li>

<li>results: 文章列出了可能关闭这个循环的原因，以及这可能导致模型自我意识和减少预测错误的变化。<details>

<summary>Abstract</summary>

Large language models (LLMs) like GPT are often conceptualized as passive predictors, simulators, or even stochastic parrots. We instead conceptualize LLMs by drawing on the theory of active inference originating in cognitive science and neuroscience. We examine similarities and differences between traditional active inference systems and LLMs, leading to the conclusion that, currently, LLMs lack a tight feedback loop between acting in the world and perceiving the impacts of their actions, but otherwise fit in the active inference paradigm. We list reasons why this loop may soon be closed, and possible consequences of this including enhanced model self-awareness and the drive to minimize prediction error by changing the world.

</details>

<details>

<summary>摘要</summary>

大型语言模型（LLM）如GPT通常被概念化为无动的预测器、模拟器或甚至是随机的喊喊鸟。我们则通过从认知科学和神经科学中的活跃推理理论来概念化LLM。我们比较了传统的活跃推理系统和LLM之间的相似之处和不同之处，结论是，目前LLM缺乏在世界中行动并观察自己的影响的紧密回路，但以其他方面符合活跃推理概念。我们列出了关闭这个回路的原因，以及这可能会带来的影响，包括增强模型自我意识和驱动降低预测错误的改变世界的驱动。

</details></li>

</ul>

<hr>

<h2 id="Bayes-in-the-age-of-intelligent-machines"><a href="#Bayes-in-the-age-of-intelligent-machines" class="headerlink" title="Bayes in the age of intelligent machines"></a>Bayes in the age of intelligent machines</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10206">http://arxiv.org/abs/2311.10206</a></li>

<li>repo_url: None</li>

<li>paper_authors: Thomas L. Griffiths, Jian-Qiao Zhu, Erin Grant, R. Thomas McCoy</li>

<li>for: 本研究旨在探讨人工神经网络如何影响人类认知解释，并 argue that Bayesian模型和人工神经网络是 complementary modeling approach，可以用来理解人类认知和智能机器的行为。</li>

<li>methods: 本研究使用了人工神经网络和 Bayesian模型来解释人类认知和智能机器的行为。</li>

<li>results: 研究发现，Bayesian模型和人工神经网络是不同的层次分析方法，可以共同理解人类认知和智能机器的行为，并且 Bayesian模型在解释大型、透明度低的人工神经网络行为方面可能具有独特的价值。<details>

<summary>Abstract</summary>

The success of methods based on artificial neural networks in creating intelligent machines seems like it might pose a challenge to explanations of human cognition in terms of Bayesian inference. We argue that this is not the case, and that in fact these systems offer new opportunities for Bayesian modeling. Specifically, we argue that Bayesian models of cognition and artificial neural networks lie at different levels of analysis and are complementary modeling approaches, together offering a way to understand human cognition that spans these levels. We also argue that the same perspective can be applied to intelligent machines, where a Bayesian approach may be uniquely valuable in understanding the behavior of large, opaque artificial neural networks that are trained on proprietary data.

</details>

<details>

<summary>摘要</summary>

人类认知的解释可能会受到基于人工神经网络的方法的成功威胁。我们认为这并不是如此，我们认为这些系统实际上提供了新的机会来模型人类认知。具体来说，我们认为认知科学的概率模型和人工神经网络模型在不同的水平上进行模型化，这些模型之间存在衔接，可以用来理解人类认知的各个水平。此外，我们还认为概率模型在理解大型、不透明的人工神经网络的行为方面可能具有独特的价值，这些网络通常是基于专有数据进行训练的。

</details></li>

</ul>

<hr>

<h2 id="Towards-Improving-Robustness-Against-Common-Corruptions-using-Mixture-of-Class-Specific-Experts"><a href="#Towards-Improving-Robustness-Against-Common-Corruptions-using-Mixture-of-Class-Specific-Experts" class="headerlink" title="Towards Improving Robustness Against Common Corruptions using Mixture of Class Specific Experts"></a>Towards Improving Robustness Against Common Corruptions using Mixture of Class Specific Experts</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10177">http://arxiv.org/abs/2311.10177</a></li>

<li>repo_url: None</li>

<li>paper_authors: Shashank Kotyan, Danilo Vasconcellos Vargas</li>

<li>for: 这篇论文的目的是提高神经网络的适用范围和性能，以应对不断变化的实际世界情况。</li>

<li>methods: 这篇论文提出了一种名为“混合类别专家架构”的新方法，它通过专门为每个类别训练独立的神经网络段，并将其输出组合以提高神经网络的扩展性和绩效。</li>

<li>results: 研究发现，这种新方法可以提高神经网络的适用范围和性能，并在不同的测试 benchmark 上表现出色。特别是在面对未知的扭曲和折衣时，这种方法可以提供更高的适用范围和稳定性。<details>

<summary>Abstract</summary>

Neural networks have demonstrated significant accuracy across various domains, yet their vulnerability to subtle input alterations remains a persistent challenge. Conventional methods like data augmentation, while effective to some extent, fall short in addressing unforeseen corruptions, limiting the adaptability of neural networks in real-world scenarios. In response, this paper introduces a novel paradigm known as the Mixture of Class-Specific Expert Architecture. The approach involves disentangling feature learning for individual classes, offering a nuanced enhancement in scalability and overall performance. By training dedicated network segments for each class and subsequently aggregating their outputs, the proposed architecture aims to mitigate vulnerabilities associated with common neural network structures. The study underscores the importance of comprehensive evaluation methodologies, advocating for the incorporation of benchmarks like the common corruptions benchmark. This inclusion provides nuanced insights into the vulnerabilities of neural networks, especially concerning their generalization capabilities and robustness to unforeseen distortions. The research aligns with the broader objective of advancing the development of highly robust learning systems capable of nuanced reasoning across diverse and challenging real-world scenarios. Through this contribution, the paper aims to foster a deeper understanding of neural network limitations and proposes a practical approach to enhance their resilience in the face of evolving and unpredictable conditions.

</details>

<details>

<summary>摘要</summary></li>

</ul>

</details>

<hr>

<h2 id="JaxMARL-Multi-Agent-RL-Environments-in-JAX"><a href="#JaxMARL-Multi-Agent-RL-Environments-in-JAX" class="headerlink" title="JaxMARL: Multi-Agent RL Environments in JAX"></a>JaxMARL: Multi-Agent RL Environments in JAX</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10090">http://arxiv.org/abs/2311.10090</a></li>

<li>repo_url: <a target="_blank" rel="noopener" href="https://github.com/flairox/jaxmarl">https://github.com/flairox/jaxmarl</a></li>

<li>paper_authors: Alexander Rutherford, Benjamin Ellis, Matteo Gallici, Jonathan Cook, Andrei Lupu, Gardar Ingvarsson, Timon Willi, Akbir Khan, Christian Schroeder de Witt, Alexandra Souly, Saptarashmi Bandyopadhyay, Mikayel Samvelyan, Minqi Jiang, Robert Tjarko Lange, Shimon Whiteson, Bruno Lacerda, Nick Hawes, Tim Rocktaschel, Chris Lu, Jakob Nicolaus Foerster</li>

<li>for: This paper is written for researchers and developers in the field of reinforcement learning (RL) and multi-agent reinforcement learning (MARL), who need efficient and scalable environments for training and evaluating their algorithms.</li>

<li>methods: The paper uses JAX (Jax.org) to enable massively parallel RL training pipelines and environments, and presents JaxMARL, an open-source code base that combines ease-of-use with GPU-enabled efficiency for commonly used MARL environments and popular baseline algorithms.</li>

<li>results: The paper shows that JaxMARL is up to 12,500 times faster than existing approaches in terms of wall clock time, enabling efficient and thorough evaluations, and introduces SMAX, a vectorized and simplified version of the StarCraft Multi-Agent Challenge that enables GPU acceleration and provides a more flexible MARL environment.<details>

<summary>Abstract</summary>

Benchmarks play an important role in the development of machine learning algorithms. For example, research in reinforcement learning (RL) has been heavily influenced by available environments and benchmarks. However, RL environments are traditionally run on the CPU, limiting their scalability with typical academic compute. Recent advancements in JAX have enabled the wider use of hardware acceleration to overcome these computational hurdles, enabling massively parallel RL training pipelines and environments. This is particularly useful for multi-agent reinforcement learning (MARL) research. First of all, multiple agents must be considered at each environment step, adding computational burden, and secondly, the sample complexity is increased due to non-stationarity, decentralised partial observability, or other MARL challenges. In this paper, we present JaxMARL, the first open-source code base that combines ease-of-use with GPU enabled efficiency, and supports a large number of commonly used MARL environments as well as popular baseline algorithms. When considering wall clock time, our experiments show that per-run our JAX-based training pipeline is up to 12500x faster than existing approaches. This enables efficient and thorough evaluations, with the potential to alleviate the evaluation crisis of the field. We also introduce and benchmark SMAX, a vectorised, simplified version of the popular StarCraft Multi-Agent Challenge, which removes the need to run the StarCraft II game engine. This not only enables GPU acceleration, but also provides a more flexible MARL environment, unlocking the potential for self-play, meta-learning, and other future applications in MARL. We provide code at https://github.com/flairox/jaxmarl.

</details>

<details>

<summary>摘要</summary>

��benchmark��是机器学习算法开发中非常重要的一部分。例如，在强化学习（RL）方面，可用的环境和benchmark��对研究产生了深远的影响。然而，RL环境通常在CPU上运行，这限制了学术计算的扩展性。现在，JAX技术的发展使得可以通过硬件加速来超越这些计算障碍，实现了大规模并行的RL训练管道和环境。这特别有用于多智能体强化学习（MARL）研究。首先，在每个环境步骤中，需要考虑多个智能体，这添加了计算压力；其次，样本复杂性增加由非站立性、分布式部分可见性或其他MARL挑战。在这篇论文中，我们提供了JaxMARL，首个开源代码库，搭配易用性和GPU启用效率，支持大量常用的MARL环境以及流行的基线算法。在考虑wall clock时间的情况下，我们的JAX基本训练管道相比现有方法，每次训练的时间提高了12500倍。这使得有效和详细的评估变得可能，有potential解决机器学习领域的评估危机。我们还引入了SMAX，一种简化版的StarCraft Multi-Agent Challenge， removes the need to run the StarCraft II game engine。这不仅使得GPU加速可能，还提供了更 flexible MARL环境，解锁了自适应、元学习和未来MARL应用的潜力。我们提供了代码，可以在https://github.com/flairox/jaxmarl上获取。

</details></li>

</ul>

<hr>

<h2 id="Emu-Edit-Precise-Image-Editing-via-Recognition-and-Generation-Tasks"><a href="#Emu-Edit-Precise-Image-Editing-via-Recognition-and-Generation-Tasks" class="headerlink" title="Emu Edit: Precise Image Editing via Recognition and Generation Tasks"></a>Emu Edit: Precise Image Editing via Recognition and Generation Tasks</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10089">http://arxiv.org/abs/2311.10089</a></li>

<li>repo_url: None</li>

<li>paper_authors: Shelly Sheynin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, Yaniv Taigman</li>

<li>For: 这个论文旨在提出一种多任务图像编辑模型，以便通过自然语言指令执行图像编辑操作。* Methods: 该模型通过多任务学习和学习任务嵌入来提高图像编辑能力，并且可以通过几个例子学习新任务。* Results: 该模型在多个图像编辑任务中达到了领先水平，包括区域基本编辑、自由形编辑和计算机视觉任务。此外，该模型还可以执行新的图像编辑任务，只需几个标注的样本即可。<details>

<summary>Abstract</summary>

Instruction-based image editing holds immense potential for a variety of applications, as it enables users to perform any editing operation using a natural language instruction. However, current models in this domain often struggle with accurately executing user instructions. We present Emu Edit, a multi-task image editing model which sets state-of-the-art results in instruction-based image editing. To develop Emu Edit we train it to multi-task across an unprecedented range of tasks, such as region-based editing, free-form editing, and Computer Vision tasks, all of which are formulated as generative tasks. Additionally, to enhance Emu Edit's multi-task learning abilities, we provide it with learned task embeddings which guide the generation process towards the correct edit type. Both these elements are essential for Emu Edit's outstanding performance. Furthermore, we show that Emu Edit can generalize to new tasks, such as image inpainting, super-resolution, and compositions of editing tasks, with just a few labeled examples. This capability offers a significant advantage in scenarios where high-quality samples are scarce. Lastly, to facilitate a more rigorous and informed assessment of instructable image editing models, we release a new challenging and versatile benchmark that includes seven different image editing tasks.

</details>

<details>

<summary>摘要</summary>

<SYS>请求已处理，以下是文本的简化中文版本：</SYS> instruciton-based image editing 拥有巨大的潜力，可以用自然语言指令进行任何编辑操作。然而，当前在这个领域的模型经常会对用户的指令进行不准确的执行。我们介绍了 Emu Edit，一个多任务图像编辑模型，它在 instruciton-based image editing 中设置了新的STATE-OF-THE-ART 成绩。为了开发 Emu Edit，我们训练它来执行多种任务，例如区域编辑、自由形编辑和计算机视觉任务，这些任务都是生成任务。此外，为了增强 Emu Edit 的多任务学习能力，我们为它提供了学习任务嵌入，这些嵌入引导生成过程向正确的编辑类型。这两个元素都是 Emu Edit 的出色表现的关键。此外，我们表明 Emu Edit 可以通过几个标注的例子学习新任务，如图像填充、超解像和编辑任务的组合。这种能力在情况中缺乏高质量样本时具有重要优势。最后，为了促进 instruciton-based image editing 模型的更加严格和有知识的评估，我们发布了一个新的复杂和多样的benchmark，包括七种不同的图像编辑任务。

</details></li>

</ul>

<hr>

<h2 id="Intelligent-Generation-of-Graphical-Game-Assets-A-Conceptual-Framework-and-Systematic-Review-of-the-State-of-the-Art"><a href="#Intelligent-Generation-of-Graphical-Game-Assets-A-Conceptual-Framework-and-Systematic-Review-of-the-State-of-the-Art" class="headerlink" title="Intelligent Generation of Graphical Game Assets: A Conceptual Framework and Systematic Review of the State of the Art"></a>Intelligent Generation of Graphical Game Assets: A Conceptual Framework and Systematic Review of the State of the Art</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10129">http://arxiv.org/abs/2311.10129</a></li>

<li>repo_url: None</li>

<li>paper_authors: Kaisei Fukaya, Damon Daylamani-Zad, Harry Agius</li>

<li>for: 这篇论文的目的是对游戏中的图形资产生成进行系统性的文献综述，以便为感兴趣的人提供可能的方法和方法，并帮助他们了解和应用这些方法。</li>

<li>methods: 这篇论文使用了系统性的文献综述方法，检索了200篇有关图形资产生成的论文，从而掌握了现状的方法和技术。</li>

<li>results: 该论文对现有的图形资产生成方法进行了概括和分析，并基于文献的研究，提出了一种概念框架，以帮助感兴趣的人了解和应用图形资产生成的方法。<details>

<summary>Abstract</summary>

Procedural content generation (PCG) can be applied to a wide variety of tasks in games, from narratives, levels and sounds, to trees and weapons. A large amount of game content is comprised of graphical assets, such as clouds, buildings or vegetation, that do not require gameplay function considerations. There is also a breadth of literature examining the procedural generation of such elements for purposes outside of games. The body of research, focused on specific methods for generating specific assets, provides a narrow view of the available possibilities. Hence, it is difficult to have a clear picture of all approaches and possibilities, with no guide for interested parties to discover possible methods and approaches for their needs, and no facility to guide them through each technique or approach to map out the process of using them. Therefore, a systematic literature review has been conducted, yielding 200 accepted papers. This paper explores state-of-the-art approaches to graphical asset generation, examining research from a wide range of applications, inside and outside of games. Informed by the literature, a conceptual framework has been derived to address the aforementioned gaps.

</details>

<details>

<summary>摘要</summary>

<<SYS>>translate_orientation: horizontal Процедурное содержание генерации (PCG) может быть применено к широкому спектру задач в играх, от сюжетов, уровней и звуков, до деревьев и оружия. Огромное количество содержимого игры состоит из графических ассетов, таких как облака, здания или растительность, которые не требуют рассмотрения функций игры. Также существует широкий спектр литературы, который изучает процедурное генерация таких элементов для целей вне игр. Тело исследований, сосредоточенное на конкретных методах для генерации конкретных активов, ограничивает видимость доступных возможных подходов и подходов. Поэтому трудно получить ясный обзор всех подходов и возможностей, а также нет инструментов для руководства заинтересованными сторонами в возможных методах и подходах для их потребностей. Поэтому была проведена систематическая рецензия литературы, которая дала 200 принятых статей. Эта статья исследовает современные подходы к генерации графических активов, изучая исследования из широкого спектра приложений, как внутри, так и вне игр. Обоснованная литературой, была получена концептуальная рамка, чтобы устранить перечисленные пробелы.

</details></li>

</ul>

<hr>

<h2 id="ChatGPT-3-5-ChatGPT-4-Google-Bard-and-Microsoft-Bing-to-Improve-Health-Literacy-and-Communication-in-Pediatric-Populations-and-Beyond"><a href="#ChatGPT-3-5-ChatGPT-4-Google-Bard-and-Microsoft-Bing-to-Improve-Health-Literacy-and-Communication-in-Pediatric-Populations-and-Beyond" class="headerlink" title="ChatGPT-3.5, ChatGPT-4, Google Bard, and Microsoft Bing to Improve Health Literacy and Communication in Pediatric Populations and Beyond"></a>ChatGPT-3.5, ChatGPT-4, Google Bard, and Microsoft Bing to Improve Health Literacy and Communication in Pediatric Populations and Beyond</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10075">http://arxiv.org/abs/2311.10075</a></li>

<li>repo_url: None</li>

<li>paper_authors: Kanhai S. Amin, Linda Mayes, Pavan Khosla, Rushabh Doshi</li>

<li>For: The paper aims to investigate whether large language models (LLMs) can improve health literacy in children and other populations.* Methods: The authors used 26 different prompts to test the ability of three LLMs (ChatGPT-3.5, Microsoft Bing, and Google Bard) to provide health information at different reading grade levels (RGL). They evaluated the responses based on their reading grade level and word count.* Results: The results show that all three LLMs were able to provide responses at or above a 10th-grade RGL. However, ChatGPT-3.5 and ChatGPT-4 were better at providing responses at lower grade levels, while Microsoft Bing and Google Bard tended to produce responses at a consistent high school level. The authors also found that Bard was more cautious in providing certain outputs, which may indicate a need for further research on the accuracy and effectiveness of LLMs in health communication.<details>

<summary>Abstract</summary>

Purpose: Enhanced health literacy has been linked to better health outcomes; however, few interventions have been studied. We investigate whether large language models (LLMs) can serve as a medium to improve health literacy in children and other populations. Methods: We ran 288 conditions using 26 different prompts through ChatGPT-3.5, Microsoft Bing, and Google Bard. Given constraints imposed by rate limits, we tested a subset of 150 conditions through ChatGPT-4. The primary outcome measurements were the reading grade level (RGL) and word counts of output. Results: Across all models, output for basic prompts such as "Explain" and "What is (are)" were at, or exceeded, a 10th-grade RGL. When prompts were specified to explain conditions from the 1st to 12th RGL, we found that LLMs had varying abilities to tailor responses based on RGL. ChatGPT-3.5 provided responses that ranged from the 7th-grade to college freshmen RGL while ChatGPT-4 outputted responses from the 6th-grade to the college-senior RGL. Microsoft Bing provided responses from the 9th to 11th RGL while Google Bard provided responses from the 7th to 10th RGL. Discussion: ChatGPT-3.5 and ChatGPT-4 did better in achieving lower-grade level outputs. Meanwhile Bard and Bing tended to consistently produce an RGL that is at the high school level regardless of prompt. Additionally, Bard's hesitancy in providing certain outputs indicates a cautious approach towards health information. LLMs demonstrate promise in enhancing health communication, but future research should verify the accuracy and effectiveness of such tools in this context. Implications: LLMs face challenges in crafting outputs below a sixth-grade reading level. However, their capability to modify outputs above this threshold provides a potential mechanism to improve health literacy and communication in a pediatric population and beyond.

</details>

<details>

<summary>摘要</summary>

目的：增强健康文化知识与健康结果之间的关系，但只有少数临床实践被研究。我们 investigate whether large language models (LLMs) can serve as a medium to improve health literacy in children and other populations. 方法：我们运行了288个条件，使用26个提示，通过ChatGPT-3.5、Microsoft Bing和Google Bard进行测试。由于环境限制，我们只测试了150个条件。主要输出测量方法包括阅读水平（RGL）和单词计数。 结果：所有模型的输出基本提问（如“解释”和“是什么”）的RGL都达到或超过了高中水平。当提示是指定为1-12年级的条件时，我们发现LLMs有不同的能力来适应不同的学龄水平。ChatGPT-3.5提供了7-12年级的回答，而ChatGPT-4则提供了6-12年级的回答。Microsoft Bing提供了9-11年级的回答，而Google Bard则提供了7-10年级的回答。 讨论：ChatGPT-3.5和ChatGPT-4在实现更低学龄水平的输出方面表现更好。与之相比，Bing和Bard倾向于一直提供高中水平的回答，无论提示是什么。此外，Bard的某些输出表现出了谨慎的态度，这可能是一种对健康信息的谨慎方式。LLMs在健康沟通方面表现了搭配性，但未来的研究应该验证这些工具在这种上下文中的准确性和效果。 意义：LLMs面临低于6年级阅读水平的输出创作的挑战。然而，它们可以修改输出以上这个阈值提供一个可能的机制来提高健康文化知识和沟通。

</details></li>

</ul>

<hr>

<h2 id="The-Song-Describer-Dataset-a-Corpus-of-Audio-Captions-for-Music-and-Language-Evaluation"><a href="#The-Song-Describer-Dataset-a-Corpus-of-Audio-Captions-for-Music-and-Language-Evaluation" class="headerlink" title="The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluation"></a>The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluation</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10057">http://arxiv.org/abs/2311.10057</a></li>

<li>repo_url: <a target="_blank" rel="noopener" href="https://github.com/mulab-mir/song-describer-dataset">https://github.com/mulab-mir/song-describer-dataset</a></li>

<li>paper_authors: Ilaria Manco, Benno Weck, SeungHeon Doh, Minz Won, Yixiao Zhang, Dmitry Bodganov, Yusong Wu, Ke Chen, Philip Tovstogan, Emmanouil Benetos, Elio Quinton, György Fazekas, Juhan Nam</li>

<li>for: 评估音乐和语言模型的评估 dataset，提供高质量的音频-caption对。</li>

<li>methods: 使用人工写好的自然语言描述，对706首乐曲进行了评估。</li>

<li>results: 通过三种音乐和语言任务的测试（乐曲描述、文本到乐曲生成和乐曲语言检索），研究人员可以通过 SDD 来更好地了解模型性能。<details>

<summary>Abstract</summary>

We introduce the Song Describer dataset (SDD), a new crowdsourced corpus of high-quality audio-caption pairs, designed for the evaluation of music-and-language models. The dataset consists of 1.1k human-written natural language descriptions of 706 music recordings, all publicly accessible and released under Creative Common licenses. To showcase the use of our dataset, we benchmark popular models on three key music-and-language tasks (music captioning, text-to-music generation and music-language retrieval). Our experiments highlight the importance of cross-dataset evaluation and offer insights into how researchers can use SDD to gain a broader understanding of model performance.

</details>

<details>

<summary>摘要</summary>

我们介绍歌曲描述数据集（SDD），一个新的人工抽样的音频-caption对数据集，适用于评估音乐和语言模型。该数据集包含1.1k名人写的自然语言描述，描述了706首音乐录音，所有公共可访问，发布于创意共享许可证下。为了展示我们的数据集的使用，我们在三个关键的音乐和语言任务（音频描述、文本到音乐生成和音乐语言检索）中对流行的模型进行了测试。我们的实验表明了跨数据集评估的重要性，并提供了如何使用 SDD 来深入了解模型性能的细节。

</details></li>

</ul>

<hr>

<h2 id="Is-“A-Helpful-Assistant”-the-Best-Role-for-Large-Language-Models-A-Systematic-Evaluation-of-Social-Roles-in-System-Prompts"><a href="#Is-“A-Helpful-Assistant”-the-Best-Role-for-Large-Language-Models-A-Systematic-Evaluation-of-Social-Roles-in-System-Prompts" class="headerlink" title="Is “A Helpful Assistant” the Best Role for Large Language Models? A Systematic Evaluation of Social Roles in System Prompts"></a>Is “A Helpful Assistant” the Best Role for Large Language Models? A Systematic Evaluation of Social Roles in System Prompts</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10054">http://arxiv.org/abs/2311.10054</a></li>

<li>repo_url: None</li>

<li>paper_authors: Mingqian Zheng, Jiaxin Pei, David Jurgens</li>

<li>for: 这个研究旨在评估社交角色在系统提示中对模型性能的影响，以帮助设计更好的AI系统提示。</li>

<li>methods: 该研究使用了3个流行的大语言模型和2457道问题进行了广泛的分析，并制定了162个社交角色，包括6种人际关系和8种职业。</li>

<li>results: 研究发现，在提示中添加社交角色可以一致提高模型的性能，并且使用gender-neutral角色和指定受众角色可以更好地提高模型的性能。然而，预测哪一个角色会导致最佳性能仍然是一个挑战，并且频率、相似度和混淆率不能完全解释社交角色对模型性能的影响。<details>

<summary>Abstract</summary>

Prompting serves as the major way humans interact with Large Language Models (LLM). Commercial AI systems commonly define the role of the LLM in system prompts. For example, ChatGPT uses "You are a helpful assistant" as part of the default system prompt. But is "a helpful assistant" the best role for LLMs? In this study, we present a systematic evaluation of how social roles in system prompts affect model performance. We curate a list of 162 roles covering 6 types of interpersonal relationships and 8 types of occupations. Through extensive analysis of 3 popular LLMs and 2457 questions, we show that adding interpersonal roles in prompts consistently improves the models' performance over a range of questions. Moreover, while we find that using gender-neutral roles and specifying the role as the audience leads to better performances, predicting which role leads to the best performance remains a challenging task, and that frequency, similarity, and perplexity do not fully explain the effect of social roles on model performances. Our results can help inform the design of system prompts for AI systems. Code and data are available at https://github.com/Jiaxin-Pei/Prompting-with-Social-Roles.

</details>

<details>

<summary>摘要</summary>

大量语言模型（LLM）与人类之间的互动主要是通过提示来实现。商业人工智能系统通常将 LLM 的角色定义为系统提示中的一部分。例如，ChatGPT 使用 "你是一位有用的助手" 作为默认系统提示。但是 "有用的助手" 是 LLM 的最佳角色吗？在这项研究中，我们提供了一种系统atic评估如何社交角色在系统提示中影响模型性能。我们筛选了 162 个角色，涵盖了6种人际关系和8种职业。通过对 3 个流行 LLM 和 2457 个问题进行广泛分析，我们发现，在提示中添加社交角色可以逐渐提高模型的性能范围内的问题。此外，我们发现使用 gender-neutral 角色和指定角色为受众可以提高模型的性能，但是预测哪一个角色会导致最佳性能是一项困难的任务，并且频率、相似度和混淆率不能完全解释社交角色对模型性能的影响。我们的结果可以帮助设计 AI 系统的提示。代码和数据可以在 https://github.com/Jiaxin-Pei/Prompting-with-Social-Roles 上获取。

</details></li>

</ul>

<hr>

<h2 id="Inherently-Interpretable-Time-Series-Classification-via-Multiple-Instance-Learning"><a href="#Inherently-Interpretable-Time-Series-Classification-via-Multiple-Instance-Learning" class="headerlink" title="Inherently Interpretable Time Series Classification via Multiple Instance Learning"></a>Inherently Interpretable Time Series Classification via Multiple Instance Learning</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10049">http://arxiv.org/abs/2311.10049</a></li>

<li>repo_url: <a target="_blank" rel="noopener" href="https://github.com/jaearly/miltimeseriesclassification">https://github.com/jaearly/miltimeseriesclassification</a></li>

<li>paper_authors: Joseph Early, Gavin KC Cheung, Kurt Cutajar, Hanting Xie, Jas Kandola, Niall Twomey</li>

<li>for: 本研究旨在提高时间序列分类器的可解释性，使其决策过程更加直观和可理解。</li>

<li>methods: 本研究使用多例学习（MIL）技术，提出了一种名为MILLET（多例学习为时间序列分类提供了本地可解释性）的新框架。该框架可以应用于现有的深度学习时间序列分类模型，使其自然地具有可解释性，而不会产生性能下降。</li>

<li>results: 在85个UCR时间序列分类 dataset上测试了MILLET，并证明了它可以生成高质量的解释，比其他已知的可解释方法更好。此外，我们还提供了一个专门为可解释性评估设计的 synthetic dataset。<details>

<summary>Abstract</summary>

Conventional Time Series Classification (TSC) methods are often black boxes that obscure inherent interpretation of their decision-making processes. In this work, we leverage Multiple Instance Learning (MIL) to overcome this issue, and propose a new framework called MILLET: Multiple Instance Learning for Locally Explainable Time series classification. We apply MILLET to existing deep learning TSC models and show how they become inherently interpretable without compromising (and in some cases, even improving) predictive performance. We evaluate MILLET on 85 UCR TSC datasets and also present a novel synthetic dataset that is specially designed to facilitate interpretability evaluation. On these datasets, we show MILLET produces sparse explanations quickly that are of higher quality than other well-known interpretability methods. To the best of our knowledge, our work with MILLET, which is available on GitHub (https://github.com/JAEarly/MILTimeSeriesClassification), is the first to develop general MIL methods for TSC and apply them to an extensive variety of domains

</details>

<details>

<summary>摘要</summary>

We evaluate MILLET on 85 UCR TSC datasets and also present a novel synthetic dataset that is specifically designed to facilitate interpretability evaluation. Our results show that MILLET produces high-quality explanations quickly, which are superior to other well-known interpretability methods. To the best of our knowledge, our work with MILLET, which is available on GitHub (https://github.com/JAEarly/MILTimeSeriesClassification), is the first to develop general MIL methods for TSC and apply them to a wide range of domains.

</details></li>

</ul>

<hr>

<h2 id="A-Novel-Neural-Network-Based-Federated-Learning-System-for-Imbalanced-and-Non-IID-Data"><a href="#A-Novel-Neural-Network-Based-Federated-Learning-System-for-Imbalanced-and-Non-IID-Data" class="headerlink" title="A Novel Neural Network-Based Federated Learning System for Imbalanced and Non-IID Data"></a>A Novel Neural Network-Based Federated Learning System for Imbalanced and Non-IID Data</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10025">http://arxiv.org/abs/2311.10025</a></li>

<li>repo_url: None</li>

<li>paper_authors: Mahfuzur Rahman Chowdhury, Muhammad Ibrahim</li>

<li>for: 本研究旨在提高 Federated Learning 中数据隐私的保护，并提出一种中央化的 neural network-based Federated Learning 系统，以提高 Accuracy 和 Efficiency。</li>

<li>methods: 本研究使用了 Micro-level 并行处理，启发自传统的 mini-batch 算法， Client 设备和服务器分别处理前向和反向传播。此外，我们还提出了一种 Edge Computing 版本的我们的提posed algorithm，在减少中央服务器负担的情况下， Client 处理前向和反向传播。</li>

<li>results: 我们在五个Well-known Benchmark dataset上进行了评估，并在不同数据分布Setting下获得了满意的性能，与一些现有的标准algorithm相比，我们的提posed system在一定程度上减少了训练时间。<details>

<summary>Abstract</summary>

With the growth of machine learning techniques, privacy of data of users has become a major concern. Most of the machine learning algorithms rely heavily on large amount of data which may be collected from various sources. Collecting these data yet maintaining privacy policies has become one of the most challenging tasks for the researchers. To combat this issue, researchers have introduced federated learning, where a prediction model is learnt by ensuring the privacy of data of clients data. However, the prevalent federated learning algorithms possess an accuracy and efficiency trade-off, especially for non-IID data. In this research, we propose a centralized, neural network-based federated learning system. The centralized algorithm incorporates micro-level parallel processing inspired by the traditional mini-batch algorithm where the client devices and the server handle the forward and backward propagation respectively. We also devise a semi-centralized version of our proposed algorithm. This algorithm takes advantage of edge computing for minimizing the load from the central server, where clients handle both the forward and backward propagation while sacrificing the overall train time to some extent. We evaluate our proposed systems on five well-known benchmark datasets and achieve satisfactory performance in a reasonable time across various data distribution settings as compared to some existing benchmark algorithms.

</details>

<details>

<summary>摘要</summary>

随着机器学习技术的发展，用户数据隐私的问题已成为一个重要的挑战。大多数机器学习算法需要大量数据，这些数据可能来自于多个来源。收集这些数据并保持隐私政策已成为研究人员最大的挑战。为解决这个问题，研究人员已经引入联邦学习，这种方法可以保证客户端数据的隐私。然而，现有的联邦学习算法具有精度和效率的负担假设，特别是非相关数据。在这些研究中，我们提议了一种中央化的神经网络基于联邦学习系统。中央算法包括微级并行处理，这种方法由客户端设备和服务器处理前向和反向传播。我们还开发了一种半中央化版本的我们的提议算法。这种算法利用边计算来减轻中央服务器的负担，客户端处理了前向和反向传播，但是在一定程度上减少了总训练时间。我们在五个常见的benchmark数据集上评估了我们的提议系统，并在不同的数据分布设置下实现了满意的性能，与一些现有的benchmark算法相比。

</details></li>

</ul>

<hr>

<h2 id="Learning-interactions-to-boost-human-creativity-with-bandits-and-GPT-4"><a href="#Learning-interactions-to-boost-human-creativity-with-bandits-and-GPT-4" class="headerlink" title="Learning interactions to boost human creativity with bandits and GPT-4"></a>Learning interactions to boost human creativity with bandits and GPT-4</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10127">http://arxiv.org/abs/2311.10127</a></li>

<li>repo_url: None</li>

<li>paper_authors: Ara Vartanian, Xiaoxi Sun, Yun-Shiuan Chuang, Siddharth Suresh, Xiaojin Zhu, Timothy T. Rogers</li>

<li>for: 这个论文探讨了人与AI算法之间的互动如何提高人类创造力。</li>

<li>methods: 研究人员使用了一种心理任务来检验人类创造力的限制，即Semantic feature generation：给一个概念名称后，参与者需要列出该概念的所有特征。</li>

<li>results: 研究发现，人类参与者和语言AI（GPT-4）在标准任务和一种提供算法生成提示的变体任务中的行为相似，而且 bandaits学习自AI响应 preferences 与人类行为学习的提示策略相同。结果表明，通过计算机互动，可以使bandits learn from simulated participants的行为，以提高人类创造力。<details>

<summary>Abstract</summary>

This paper considers how interactions with AI algorithms can boost human creative thought. We employ a psychological task that demonstrates limits on human creativity, namely semantic feature generation: given a concept name, respondents must list as many of its features as possible. Human participants typically produce only a fraction of the features they know before getting "stuck." In experiments with humans and with a language AI (GPT-4) we contrast behavior in the standard task versus a variant in which participants can ask for algorithmically-generated hints. Algorithm choice is administered by a multi-armed bandit whose reward indicates whether the hint helped generating more features. Humans and the AI show similar benefits from hints, and remarkably, bandits learning from AI responses prefer the same prompting strategy as those learning from human behavior. The results suggest that strategies for boosting human creativity via computer interactions can be learned by bandits run on groups of simulated participants.

</details>

<details>

<summary>摘要</summary>

Translated into Simplified Chinese:这篇论文研究了人与AI算法之间的互动如何提高人类创造力。我们使用了一项心理任务，以示人类创造力的限制，即给出概念名称后，参与者需要列出该概念的所有特征。人类参与者通常只能生成一小部分的特征，然后就会被"困顿"。在人类参与者和一种语言AI（GPT-4）的实验中，我们比较了标准任务和一种允许参与者请求算法生成提示的变体。算法选择是通过一个多重机枪进行管理，其奖励参与者选择有助于生成更多特征的提示。人类和AI都显示了类似的好处，奇怪地，由人类行为学习的多重机枪偏好了与AI回答相同的提示策略。这些结果表明，通过计算机互动，可以学习提高人类创造力的策略。

</details></li>

</ul>

<hr>

<h2 id="Straggler-resilient-Federated-Learning-Tackling-Computation-Heterogeneity-with-Layer-wise-Partial-Model-Training-in-Mobile-Edge-Network"><a href="#Straggler-resilient-Federated-Learning-Tackling-Computation-Heterogeneity-with-Layer-wise-Partial-Model-Training-in-Mobile-Edge-Network" class="headerlink" title="Straggler-resilient Federated Learning: Tackling Computation Heterogeneity with Layer-wise Partial Model Training in Mobile Edge Network"></a>Straggler-resilient Federated Learning: Tackling Computation Heterogeneity with Layer-wise Partial Model Training in Mobile Edge Network</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10002">http://arxiv.org/abs/2311.10002</a></li>

<li>repo_url: None</li>

<li>paper_authors: Hongda Wu, Ping Wang, C V Aswartha Narayana</li>

<li>for: 这个研究旨在提出一种基于分布式学习的训练方法，以便让不同设备 collaboration 训练模型，但不需要共享数据。</li>

<li>methods: 该方法基于模型不同设备的Compute能力不同，设备会根据自己的Compute能力来训练模型，而不是训练完整的模型。</li>

<li>results: 研究表明，该方法可以让不同设备 collaboration 训练模型，并且可以更好地平衡学习精度和完成时间。 compared to 现有的参考方法，该方法可以更快地达到学习目标。<details>

<summary>Abstract</summary>

Federated Learning (FL) enables many resource-limited devices to train a model collaboratively without data sharing. However, many existing works focus on model-homogeneous FL, where the global and local models are the same size, ignoring the inherently heterogeneous computational capabilities of different devices and restricting resource-constrained devices from contributing to FL. In this paper, we consider model-heterogeneous FL and propose Federated Partial Model Training (FedPMT), where devices with smaller computational capabilities work on partial models (subsets of the global model) and contribute to the global model. Different from Dropout-based partial model generation, which removes neurons in hidden layers at random, model training in FedPMT is achieved from the back-propagation perspective. As such, all devices in FedPMT prioritize the most crucial parts of the global model. Theoretical analysis shows that the proposed partial model training design has a similar convergence rate to the widely adopted Federated Averaging (FedAvg) algorithm, $\mathcal{O}(1/T)$, with the sub-optimality gap enlarged by a constant factor related to the model splitting design in FedPMT. Empirical results show that FedPMT significantly outperforms the existing benchmark FedDrop. Meanwhile, compared to the popular model-homogeneous benchmark, FedAvg, FedPMT reaches the learning target in a shorter completion time, thus achieving a better trade-off between learning accuracy and completion time.

</details>

<details>

<summary>摘要</summary>

Federated Learning (FL) 允许多个资源有限的设备共同训练模型而无需数据共享。然而，现有的大多数研究都专注于模型同质的 FL，即全球模型和本地模型均为同一个大小，这ignore了不同设备的内在不同计算能力，从而限制了资源有限的设备参与FL。在这篇文章中，我们考虑了模型不同质的 FL，并提出了 Federated Partial Model Training（FedPMT），其中设备的计算能力较低的设备可以在部分模型（全球模型的子集）上进行训练，并对全球模型进行贡献。与Dropout技术基于随机移除隐藏层中的神经元不同，FedPMT中的模型训练是从反射层的角度来实现的，因此所有的设备在FedPMT中都会优先级别最重要的部分。理论分析表明，我们的 partial model 训练设计和 widely adopted Federated Averaging（FedAvg）算法相似，具有 $\mathcal{O}(1/T)$ 的收敛率，但与 FedAvg 相比，FedPMT 的优劣差异因子与模型分割设计相关。实验结果表明，FedPMT 明显超过了现有的 refer 替 benchmark FedDrop。同时，相比于通用模型同质的标准 refer 替 FedAvg，FedPMT 在完成时间和学习精度之间更好地做出了平衡。

</details></li>

</ul>

<hr>

<h2 id="Towards-more-Practical-Threat-Models-in-Artificial-Intelligence-Security"><a href="#Towards-more-Practical-Threat-Models-in-Artificial-Intelligence-Security" class="headerlink" title="Towards more Practical Threat Models in Artificial Intelligence Security"></a>Towards more Practical Threat Models in Artificial Intelligence Security</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09994">http://arxiv.org/abs/2311.09994</a></li>

<li>repo_url: None</li>

<li>paper_authors: Kathrin Grosse, Lukas Bieringer, Tarek Richard Besold, Alexandre Alahi</li>

<li>for: 这个论文旨在描述人工智能安全领域的研究与实践之间的差距。</li>

<li>methods: 该论文通过对6种最常研究的AI安全攻击方法进行审查，并与271名工业实践者进行调查，以确定这些攻击方法在实际场景中的可行性。</li>

<li>results: 研究发现，现有的威胁模型都是可行的，但是有一些重大匹配度：研究往往假设攻击者具有实际场景中不易获得的信息。这篇论文因此呼吁研究更加实际的威胁模型。<details>

<summary>Abstract</summary>

Recent works have identified a gap between research and practice in artificial intelligence security: threats studied in academia do not always reflect the practical use and security risks of AI. For example, while models are often studied in isolation, they form part of larger ML pipelines in practice. Recent works also brought forward that adversarial manipulations introduced by academic attacks are impractical. We take a first step towards describing the full extent of this disparity. To this end, we revisit the threat models of the six most studied attacks in AI security research and match them to AI usage in practice via a survey with \textbf{271} industrial practitioners. On the one hand, we find that all existing threat models are indeed applicable. On the other hand, there are significant mismatches: research is often too generous with the attacker, assuming access to information not frequently available in real-world settings. Our paper is thus a call for action to study more practical threat models in artificial intelligence security.

</details>

<details>

<summary>摘要</summary>

最近的研究发现人工智能安全领域存在一个研究与实践之间的差距：在学术中研究的威胁不一定与实际使用和安全风险相符。例如，模型在实践中通常是作为更大的机器学习管道的一部分进行研究，而不是孤立的单元。此外，学术攻击的恶意修改也被证明是不实际的。我们为了描述这种差距，我们重新审视了人工智能安全领域最常研究的六种攻击方法，并通过对 \textbf{271} 名工业实践者进行调查，发现所有的威胁模型都是可靠的。然而，我们发现有一些巨大的匹配错误：研究经常假设攻击者具有实际场景中不常 disponibles的信息。因此，我们的论文是一种呼吁，呼吁更多地研究实际可行的人工智能安全威胁模型。

</details></li>

</ul>

<hr>

<h2 id="Generative-AI-for-Hate-Speech-Detection-Evaluation-and-Findings"><a href="#Generative-AI-for-Hate-Speech-Detection-Evaluation-and-Findings" class="headerlink" title="Generative AI for Hate Speech Detection: Evaluation and Findings"></a>Generative AI for Hate Speech Detection: Evaluation and Findings</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09993">http://arxiv.org/abs/2311.09993</a></li>

<li>repo_url: None</li>

<li>paper_authors: Sagi Pendzel, Tomer Wullach, Amir Adler, Einat Minkov</li>

<li>for: 提高自动仇恨语言检测的泛化性能</li>

<li>methods: 使用生成AI生成大量的人工仇恨语言序列，并在已有标注数据的基础上训练大型预训练语言模型（LLM）</li>

<li>results: 对于BERT、RoBERTa、ALBERT等常见LLM，以及已适应仇恨检测的RoBERTa-Toxicity、HateBERT、HateXplain、ToxDect和ToxiGen等模型，进行了评估和比较，并证实了这种方法可以提高仇恨语言泛化性能。同时，我们还对采用零shot仇恨检测的GPT-3.5模型进行了测试，结果显示该模型可以获得更高的泛化性能，但是具有较差的准确率和预测率。<details>

<summary>Abstract</summary>

Automatic hate speech detection using deep neural models is hampered by the scarcity of labeled datasets, leading to poor generalization. To mitigate this problem, generative AI has been utilized to generate large amounts of synthetic hate speech sequences from available labeled examples, leveraging the generated data in finetuning large pre-trained language models (LLMs). In this chapter, we provide a review of relevant methods, experimental setups and evaluation of this approach. In addition to general LLMs, such as BERT, RoBERTa and ALBERT, we apply and evaluate the impact of train set augmentation with generated data using LLMs that have been already adapted for hate detection, including RoBERTa-Toxicity, HateBERT, HateXplain, ToxDect, and ToxiGen. An empirical study corroborates our previous findings, showing that this approach improves hate speech generalization, boosting recall performance across data distributions. In addition, we explore and compare the performance of the finetuned LLMs with zero-shot hate detection using a GPT-3.5 model. Our results demonstrate that while better generalization is achieved using the GPT-3.5 model, it achieves mediocre recall and low precision on most datasets. It is an open question whether the sensitivity of models such as GPT-3.5, and onward, can be improved using similar techniques of text generation.

</details>

<details>

<summary>摘要</summary>

自动发现仇恨言语使用深度神经网络受到数据标注的罕见性的限制，导致模型的泛化性差。为解决这个问题，生成AI技术被应用来生成大量的人工仇恨言语序列，利用生成的数据进行训练大型预训练语言模型（LLM）。在这章中，我们提供了相关的方法、实验设置和评估这种方法的评估。除了一般的LLM，如BERT、RoBERTa和ALBERT，我们还应用并评估生成数据集 augmentation的影响。我们使用已经适应仇恨检测的LLM，包括RoBERTa-Toxicity、HateBERT、HateXplain、ToxDect和ToxiGen进行训练和评估。我们的实验结果表明，这种方法可以提高仇恨言语泛化性，提高检测性能 across data distributions。此外，我们还探讨了使用GPT-3.5模型进行零容量仇恨检测的性能。我们的结果表明，虽然使用GPT-3.5模型可以获得更好的泛化性，但它的准确率和精度在大多数数据集上都很低。这是一个开放的问题，是否可以通过类似的文本生成技术提高模型的敏感性。

</details></li>

</ul>

<hr>

<h2 id="A-Framework-for-Monitoring-and-Retraining-Language-Models-in-Real-World-Applications"><a href="#A-Framework-for-Monitoring-and-Retraining-Language-Models-in-Real-World-Applications" class="headerlink" title="A Framework for Monitoring and Retraining Language Models in Real-World Applications"></a>A Framework for Monitoring and Retraining Language Models in Real-World Applications</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09930">http://arxiv.org/abs/2311.09930</a></li>

<li>repo_url: None</li>

<li>paper_authors: Jaykumar Kasundra, Claudia Schulz, Melicaalsadat Mirsafian, Stavroula Skylaki</li>

<li>for: 本研究旨在探讨多类标签分类模型在不同 retraining 决策点下的影响，以及如何设计有效的模型重新训练策略。</li>

<li>methods: 本研究使用了多种 retraining 决策点，包括数据或概念漂移、模型性能下降和新数据收集等，以评估它们对模型性能和资源利用率的影响。</li>

<li>results: 研究发现，不同 retraining 决策点可能导致不同的模型性能和资源利用率。根据研究结果，提出了一个参考框架，可以帮助设计有效的模型重新训练策略。<details>

<summary>Abstract</summary>

In the Machine Learning (ML) model development lifecycle, training candidate models using an offline holdout dataset and identifying the best model for the given task is only the first step. After the deployment of the selected model, continuous model monitoring and model retraining is required in many real-world applications. There are multiple reasons for retraining, including data or concept drift, which may be reflected on the model performance as monitored by an appropriate metric. Another motivation for retraining is the acquisition of increasing amounts of data over time, which may be used to retrain and improve the model performance even in the absence of drifts. We examine the impact of various retraining decision points on crucial factors, such as model performance and resource utilization, in the context of Multilabel Classification models. We explain our key decision points and propose a reference framework for designing an effective model retraining strategy.

</details>

<details>

<summary>摘要</summary>

在机器学习（ML）模型开发生命周期中，使用停滞 dataset 训练候选模型并选择适合任务的最佳模型只是第一步。在实际应用中，已经部署的选定模型后，需要进行连续的模型监测和重新训练。有多种重新训练的原因，包括数据或概念漂移，这可能会影响模型性能，并且可以通过适当的指标来监测。另一个重新训练的动机是随着时间的推移，采集到的数据量的增加，可以重新训练并改进模型性能，即使没有数据漂移。我们研究重新训练决策点对关键因素的影响，如模型性能和资源利用率，并提出了一个参考框架，以设计有效的模型重新训练策略。

</details></li>

</ul>

<hr>

<h2 id="DSR-Diff-Depth-Map-Super-Resolution-with-Diffusion-Model"><a href="#DSR-Diff-Depth-Map-Super-Resolution-with-Diffusion-Model" class="headerlink" title="DSR-Diff: Depth Map Super-Resolution with Diffusion Model"></a>DSR-Diff: Depth Map Super-Resolution with Diffusion Model</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09919">http://arxiv.org/abs/2311.09919</a></li>

<li>repo_url: None</li>

<li>paper_authors: Yuan Shi, Bin Xia, Rui Zhu, Qingmin Liao, Wenming Yang</li>

<li>for: 提高低质量深度图的空间分辨率，用于3D重建、虚拟现实和增强现实等应用。</li>

<li>methods: 利用扩散模型在幽Defaultslat space中生成导航，并将其与高质量颜色图相结合，以实现深度图超分辨。</li>

<li>results: 对比州前方法，提出了一种新的CDSR模型，并实现了较高的准确率和效率。代码将在<a target="_blank" rel="noopener" href="https://github.com/shiyuan7/DSR-Diff%E4%B8%AD%E5%8F%91%E5%B8%83%E3%80%82">https://github.com/shiyuan7/DSR-Diff中发布。</a><details>

<summary>Abstract</summary>

Color-guided depth map super-resolution (CDSR) improve the spatial resolution of a low-quality depth map with the corresponding high-quality color map, benefiting various applications such as 3D reconstruction, virtual reality, and augmented reality. While conventional CDSR methods typically rely on convolutional neural networks or transformers, diffusion models (DMs) have demonstrated notable effectiveness in high-level vision tasks. In this work, we present a novel CDSR paradigm that utilizes a diffusion model within the latent space to generate guidance for depth map super-resolution. The proposed method comprises a guidance generation network (GGN), a depth map super-resolution network (DSRN), and a guidance recovery network (GRN). The GGN is specifically designed to generate the guidance while managing its compactness. Additionally, we integrate a simple but effective feature fusion module and a transformer-style feature extraction module into the DSRN, enabling it to leverage guided priors in the extraction, fusion, and reconstruction of multi-model images. Taking into account both accuracy and efficiency, our proposed method has shown superior performance in extensive experiments when compared to state-of-the-art methods. Our codes will be made available at https://github.com/shiyuan7/DSR-Diff.

</details>

<details>

<summary>摘要</summary>

颜色导航深度地图超分辨 (CDSR) 可以提高低质量深度地图的空间分辨率，有利于多种应用，如三维重建、虚拟现实和增强现实。传统的 CDSR 方法通常采用卷积神经网络或转换器，而扩散模型（DM）则在高级视觉任务中表现出了很好的效果。在这项工作中，我们提出了一种新的 CDSR 模式，利用在潜在空间中的扩散模型来生成指导 depth map 超分辨。我们的方法包括指导生成网络（GGN）、深度地图超分辨网络（DSRN）和指导恢复网络（GRN）。GGN 专门设计用于生成指导，同时管理其 компакт性。此外，我们还将一个简单 yet effective 的特征融合模块和一个基于转换器的特征提取模块integrated into DSRN，使其能够在抽取、融合和重建多模型图像时利用指导PRIORS。考虑到精度和效率，我们提出的方法在广泛的实验中显示出了与状态 искусственный智能方法相比的superior performance。我们的代码将在 GitHub 上发布，请参考 https://github.com/shiyuan7/DSR-Diff.

</details></li>

</ul>

<hr>

<h2 id="INTERVENOR-Prompt-the-Coding-Ability-of-Large-Language-Models-with-the-Interactive-Chain-of-Repairing"><a href="#INTERVENOR-Prompt-the-Coding-Ability-of-Large-Language-Models-with-the-Interactive-Chain-of-Repairing" class="headerlink" title="INTERVENOR: Prompt the Coding Ability of Large Language Models with the Interactive Chain of Repairing"></a>INTERVENOR: Prompt the Coding Ability of Large Language Models with the Interactive Chain of Repairing</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09868">http://arxiv.org/abs/2311.09868</a></li>

<li>repo_url: <a target="_blank" rel="noopener" href="https://github.com/neuir/intervenor">https://github.com/neuir/intervenor</a></li>

<li>paper_authors: Hanbin Wang, Zhenghao Liu, Shuo Wang, Ganqu Cui, Ning Ding, Zhiyuan Liu, Ge Yu</li>

<li>for: 这个论文是为了提出一种基于人类编程行为的自动代码修复方法（INTERVENOR），该方法利用了两个基于大语言模型（LLM）的代理机制，以便在代码修复过程中提高代码生成和翻译能力。</li>

<li>methods: 该方法使用了两个LLM代理机制，namely Code Learner和Code Teacher。Code Learner是根据Code Teacher的指导生成和修复代码，而Code Teacher则是通过编译器的反馈来重新思考代码错误，并生成一个链式修复（CoR）来引导代码修复过程。</li>

<li>results: 我们的实验表明，INTERVENOR比 estado-of-the-art 方法更高效，在代码生成和代码翻译任务中分别提高了约13%和4.5%。我们的进一步分析还表明，CoR可以通过自然语言提供错误原因和解决方案的明确描述。由于编译器的反馈，INTERVENOR可以准确地识别代码中的语法错误和断言错误，并提供精确的修复指令，使LLMs在只需要三次修复后就能达到极限性能。<details>

<summary>Abstract</summary>

This paper proposes INTERactiVE chaiN Of Repairing (INTERVENOR), which mimics human code repairing behavior (iteratively judging, rethinking, and repairing) and prompts the coding ability of regard Large Language Models (LLMs). Specifically, INTERVENOR employs two LLM based agents, Code Learner and Code Teacher, to play different roles in code repairing and work interactively to repair the generated codes. The Code Learner is asked to generate and repair code according to the instructions from the Code Teacher. The Code Teacher rethinks the code errors according to the corresponding feedback from compilers and iteratively generates the chain-of-repairing (CoR) to guide the code repairing process for Code Learner. Our experiments show that INTERVENOR outperforms the state-of-the-art methods and achieves about 13% and 4.5% improvements over the GPT-3.5 model in code generation and code translation tasks, respectively. Our further analyses show that CoR can illuminate the bug reasons and solution plans via natural language. Thanks to the feedback of code compilers, INTERVENOR can accurately identify the syntax errors and assertion errors in the code and provide precise instructions to repair codes, making LLMs achieve the plateau performance with only three repairing turns. All data and codes are available at https://github.com/NEUIR/INTERVENOR

</details>

<details>

<summary>摘要</summary>

这个论文提出了一种名为INTERactiVE chaiN Of Repairing（INTERVENOR）的方法，它模仿人类代码修复行为（迭代评估、重新思考和修复），并唤醒LLM的编程能力。具体来说，INTERVENOR使用两个基于LLM的代理人：代码学习者和代码教师。代码学习者根据代码教师的指导生成和修复代码。代码教师根据编译器的反馈重新评估代码错误，并生成了修复过程中的链条（CoR），以引导代码修复过程。我们的实验表明，INTERVENOR在代码生成和代码翻译任务上表现出色，与当前状态的方法相比，提高了约13%和4.5%。我们的进一步分析表明，CoR可以通过自然语言来描述错误原因和解决方案。由于编译器的反馈，INTERVENOR可以准确地识别代码中的语法错误和断言错误，并提供精准的修复指导，使LLM在只需三次修复后达到极限性能。所有数据和代码可以在https://github.com/NEUIR/INTERVENOR上获取。

</details></li>

</ul>

<hr>

<h2 id="PsyBench-a-balanced-and-in-depth-Psychological-Chinese-Evaluation-Benchmark-for-Foundation-Models"><a href="#PsyBench-a-balanced-and-in-depth-Psychological-Chinese-Evaluation-Benchmark-for-Foundation-Models" class="headerlink" title="PsyBench: a balanced and in-depth Psychological Chinese Evaluation Benchmark for Foundation Models"></a>PsyBench: a balanced and in-depth Psychological Chinese Evaluation Benchmark for Foundation Models</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09861">http://arxiv.org/abs/2311.09861</a></li>

<li>repo_url: None</li>

<li>paper_authors: Junlei Zhang, Hongliang He, Nirui Song, Shuyuan He, Shuai Zhang, Huachuan Qiu, Anqi Li, Lizhi Ma, Zhenzhong Lan</li>

<li>for: 这个论文是为了提供一个全面的中文评价集，用于评估基础模型在心理学领域的能力。</li>

<li>methods: 本文使用多选题型的评价方法，以评估基础模型在不同知识领域的表现。</li>

<li>results: 研究发现不同知识领域的表现存在显著差异，而且只有ChatGPT模型的平均准确率超过70%， indicating that there is still room for improvement in this area.<details>

<summary>Abstract</summary>

As Large Language Models (LLMs) are becoming prevalent in various fields, there is an urgent need for improved NLP benchmarks that encompass all the necessary knowledge of individual discipline. Many contemporary benchmarks for foundational models emphasize a broad range of subjects but often fall short in presenting all the critical subjects and encompassing necessary professional knowledge of them. This shortfall has led to skewed results, given that LLMs exhibit varying performance across different subjects and knowledge areas. To address this issue, we present psybench, the first comprehensive Chinese evaluation suite that covers all the necessary knowledge required for graduate entrance exams. psybench offers a deep evaluation of a model's strengths and weaknesses in psychology through multiple-choice questions. Our findings show significant differences in performance across different sections of a subject, highlighting the risk of skewed results when the knowledge in test sets is not balanced. Notably, only the ChatGPT model reaches an average accuracy above $70\%$, indicating that there is still plenty of room for improvement. We expect that psybench will help to conduct thorough evaluations of base models' strengths and weaknesses and assist in practical application in the field of psychology.

</details>

<details>

<summary>摘要</summary>

如Large Language Models（LLMs）在不同领域变得越来越普遍，需要改进的自然语言处理（NLP）标准测试Suite来涵盖所有必要的专业知识。许多当代测试标准 для基础模型通常会忽略一些重要的主题和专业知识，这会导致测试结果偏向。为解决这个问题，我们介绍了psybench，第一个涵盖所有必要的心理学入学考试知识的全面中文评估suite。psybench通过多选题提供了深入的模型强项和弱项评估。我们的发现表明不同主题section的性能存在显著差异，这说明测试集知识不均衡可能会导致偏向测试结果。各种ChatGPT模型的平均准确率超过70%，这表明当前还有很多机会进行改进。我们期望psybench可以帮助进行深入的模型强项和弱项评估，并在心理学领域的实践应用中提供支持。

</details></li>

</ul>

<hr>

<h2 id="SurvTimeSurvival-Survival-Analysis-On-The-Patient-With-Multiple-Visits-Records"><a href="#SurvTimeSurvival-Survival-Analysis-On-The-Patient-With-Multiple-Visits-Records" class="headerlink" title="SurvTimeSurvival: Survival Analysis On The Patient With Multiple Visits&#x2F;Records"></a>SurvTimeSurvival: Survival Analysis On The Patient With Multiple Visits&#x2F;Records</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09854">http://arxiv.org/abs/2311.09854</a></li>

<li>repo_url: <a target="_blank" rel="noopener" href="https://github.com/davidlee1102/surtimesurvival">https://github.com/davidlee1102/surtimesurvival</a></li>

<li>paper_authors: Hung Le, Ong Eng-Jon, Bober Miroslaw</li>

<li>for: 预测严重疾病患者生存时间的准确预测仍然是艰难的挑战，尽管近年来人工智能技术得到了进一步发展。这项研究推出了“SurvTimeSurvival：survival分析在多次&#x2F;记录数据上”，利用Transformer模型不仅能够处理时间变化的covariates，还能够处理covariates数据。</li>

<li>methods: 我们采用了Transformer模型来处理时间变化的covariates和covariates数据，并解决了生存分析数据集中的数据缺失问题，通过将生成Synthetic数据集成到学习过程中。</li>

<li>results: 我们的方法在covariates和时间变化covariates数据集上都超过了现有的深度学习方法的性能。我们的方法的目的不仅是提高个体患者生存轨迹的理解，从而提高预测精度，而且也在设计临床试验和开发新的治疗方法中发挥重要作用。<details>

<summary>Abstract</summary>

The accurate prediction of survival times for patients with severe diseases remains a critical challenge despite recent advances in artificial intelligence. This study introduces "SurvTimeSurvival: Survival Analysis On Patients With Multiple Visits/Records", utilizing the Transformer model to not only handle the complexities of time-varying covariates but also covariates data. We also tackle the data sparsity issue common to survival analysis datasets by integrating synthetic data generation into the learning process of our model. We show that our method outperforms state-of-the-art deep learning approaches on both covariates and time-varying covariates datasets. Our approach aims not only to enhance the understanding of individual patient survival trajectories across various medical conditions, thereby improving prediction accuracy, but also to play a pivotal role in designing clinical trials and creating new treatments.

</details>

<details>

<summary>摘要</summary>

医学预测患者生存时间的准确性仍然是一项关键挑战，尽管最近的人工智能技术得到了进步。这项研究介绍了“SurvTimeSurvival：基于多次/记录的生存分析”，利用Transformer模型不仅能处理时间变化的共 covariates，而且还能处理 covariates 数据。我们还解决了生存分析数据集中的数据缺失问题通过将生成 Synthetic 数据 incorporated 到我们的模型学习过程中。我们表明我们的方法在 covariates 和时间变化 covariates 数据集上都能够超越当前的深度学习方法。我们的方法不仅可以提高预测准确性，还可以提高对各种医疗情况的个体患者生存轨迹的理解，从而为设计临床试验和开发新药物做出重要贡献。

</details></li>

</ul>

<hr>

<h2 id="Leveraging-LLMs-in-Scholarly-Knowledge-Graph-Question-Answering"><a href="#Leveraging-LLMs-in-Scholarly-Knowledge-Graph-Question-Answering" class="headerlink" title="Leveraging LLMs in Scholarly Knowledge Graph Question Answering"></a>Leveraging LLMs in Scholarly Knowledge Graph Question Answering</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09841">http://arxiv.org/abs/2311.09841</a></li>

<li>repo_url: <a target="_blank" rel="noopener" href="https://github.com/huntila/scholarly-kgqa">https://github.com/huntila/scholarly-kgqa</a></li>

<li>paper_authors: Tilahun Abedissa Taffa, Ricardo Usbeck</li>

<li>for: 这篇论文旨在提出一种学术知识图问答系统，可以通过几 shot 的方式回答 bibliographic 自然语言问题。</li>

<li>methods: 该模型首先使用 BERT 基于 sentence encoder 将测试问题与training问题进行相似性比较，然后选择 top-n 相似问题对应的 SPARQL，并将这些对应的问题作为示例，将测试问题作为提示， passing it to LLM 生成 SPARQL。最后，对于underlying KG （ORKG）终端进行查询，并返回答案。</li>

<li>results: 该系统在 SciQA 中实现了 F1 分数 99.0%，在 Scholarly-QALD-23 挑战 benchmark 上表现出色。<details>

<summary>Abstract</summary>

This paper presents a scholarly Knowledge Graph Question Answering (KGQA) that answers bibliographic natural language questions by leveraging a large language model (LLM) in a few-shot manner. The model initially identifies the top-n similar training questions related to a given test question via a BERT-based sentence encoder and retrieves their corresponding SPARQL. Using the top-n similar question-SPARQL pairs as an example and the test question creates a prompt. Then pass the prompt to the LLM and generate a SPARQL. Finally, runs the SPARQL against the underlying KG - ORKG (Open Research KG) endpoint and returns an answer. Our system achieves an F1 score of 99.0%, on SciQA - one of the Scholarly-QALD-23 challenge benchmarks.

</details>

<details>

<summary>摘要</summary>

这篇论文提出了一种学术知识图问答系统（KGQA），该系统可以通过几个尝试回答文学性问题，使用大型语言模型（LLM）。系统首先使用BERT基于的句子编码器来将测试问题与相似训练问题进行对比，然后使用相似问题-SPARQL对应的最上层对象来生成一个提示。最后，将提示传递给LLM进行生成SPARQL，并将其运行于基础知识图（ORKG）终端，以获得答案。我们的系统在SciQA中的F1分数达99.0%。

</details></li>

</ul>

<hr>

<h2 id="PELMS-Pre-training-for-Effective-Low-Shot-Multi-Document-Summarization"><a href="#PELMS-Pre-training-for-Effective-Low-Shot-Multi-Document-Summarization" class="headerlink" title="PELMS: Pre-training for Effective Low-Shot Multi-Document Summarization"></a>PELMS: Pre-training for Effective Low-Shot Multi-Document Summarization</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09836">http://arxiv.org/abs/2311.09836</a></li>

<li>repo_url: None</li>

<li>paper_authors: Joseph J. Peper, Wenzhao Qiu, Lu Wang</li>

<li>for: 本研究旨在提高摘要文献的抽象性和反射性，即通过多文摘要来提高文献的简洁、流畅和准确性。</li>

<li>methods: 我们提出了一种基于语义相关性规则和准确性约束的预训练模型，并使用了多个文档输入来支持模型的训练。</li>

<li>results: 我们在多种摘要任务上进行了广泛的评估，并经验显示了我们的方法在低shot设定下可以准确地捕捉摘要文献的主题和含义，并且在抽象性、流畅性、准确性和 faithfulness 等方面具有优异性。<details>

<summary>Abstract</summary>

We investigate pre-training techniques for abstractive multi-document summarization (MDS), which is much less studied than summarizing single documents. Though recent work has demonstrated the effectiveness of highlighting information salience for pre-training strategy design, it struggles to generate abstractive and reflective summaries, which are critical properties for MDS. To this end, we present PELMS, a pre-trained model that uses objectives based on semantic coherence heuristics and faithfulness constraints with un-labeled multi-document inputs, to promote the generation of concise, fluent, and faithful summaries. To support the training of PELMS, we compile MultiPT, a multi-document pre-training corpus containing over 93 million documents to form more than 3 million unlabeled topic-centric document clusters, covering diverse genres such as product reviews, news, and general knowledge. We perform extensive evaluation of PELMS in low-shot settings on a wide range of MDS datasets. Our approach consistently outperforms competitive comparisons with respect to overall informativeness, abstractiveness, coherence, and faithfulness.

</details>

<details>

<summary>摘要</summary>

我团队研究了抽象多文摘要（MDS）的预训练技术，这个领域比单文摘要更为少studied。虽然 latest work 表明了突出信息重要性的预训练策略的效果，但它很难生成抽象和反射的摘要，这些特性是MDS的关键性能。为此，我们提出了 PELMS，一种预训练模型，使用基于 semantic coherence heuristics 和 faithfulness constraints 的目标函数，以便在无标签多文输入下生成简洁、流畅、忠实的摘要。为支持 PELMS 的训练，我们编译了 MultiPT，一个包含超过 93 万个文档的多文预训练集，其中包含多种类型的文档，如产品评论、新闻和通用知识。我们对 PELMS 进行了广泛的评估，包括低投入设定下的评估，并在多种 MDS 数据集上表现出consistent 的优异性。

</details></li>

</ul>

<hr>

<h2 id="ML-Bench-Large-Language-Models-Leverage-Open-source-Libraries-for-Machine-Learning-Tasks"><a href="#ML-Bench-Large-Language-Models-Leverage-Open-source-Libraries-for-Machine-Learning-Tasks" class="headerlink" title="ML-Bench: Large Language Models Leverage Open-source Libraries for Machine Learning Tasks"></a>ML-Bench: Large Language Models Leverage Open-source Libraries for Machine Learning Tasks</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09835">http://arxiv.org/abs/2311.09835</a></li>

<li>repo_url: None</li>

<li>paper_authors: Yuliang Liu, Xiangru Tang, Zefan Cai, Junjie Lu, Yichi Zhang, Yanjun Shao, Zexuan Deng, Helan Hu, Zengxian Yang, Kaikai An, Ruijun Huang, Shuzheng Si, Sheng Chen, Haozhe Zhao, Zhengliang Li, Liang Chen, Yiming Zong, Yan Wang, Tianyu Liu, Zhiwei Jiang, Baobao Chang, Yujia Qin, Wangchunshu Zhou, Yilun Zhao, Arman Cohan, Mark Gerstein</li>

<li>for: 本研究旨在评估大语言模型在使用开源库完成机器学习任务时的实用性。</li>

<li>methods: 本研究提出了一个新的评估Setup，其中大语言模型使用开源库完成机器学习任务，而不是从scratch编写代码。研究者们制定了ML-Bench，一个包含10044个样例和130个任务的广泛的benchmark，以评估大语言模型在使用开源库时的效果。</li>

<li>results: GPT-4在这些任务中表现出色，但只有39.73%的任务得到了成功。研究者们提出了ML-Agent，一种用于快速浏览代码库，找到相关文档、代码和执行代码的方法。与GPT-4结合使用后，ML-Agent得到了进一步的改进。<details>

<summary>Abstract</summary>

Large language models have shown promising performance in code generation benchmarks. However, a considerable divide exists between these benchmark achievements and their practical applicability, primarily attributed to real-world programming's reliance on pre-existing libraries. Instead of evaluating LLMs to code from scratch, this work aims to propose a new evaluation setup where LLMs use open-source libraries to finish machine learning tasks. Therefore, we propose ML-Bench, an expansive benchmark developed to assess the effectiveness of LLMs in leveraging existing functions in open-source libraries. Consisting of 10044 samples spanning 130 tasks over 14 notable machine learning GitHub repositories. In this setting, given a specific machine learning task instruction and the accompanying README in a codebase, an LLM is tasked to generate code to accomplish the task. This necessitates the comprehension of long and language-code interleaved documents, as well as the understanding of complex cross-file code structures, introducing new challenges. Notably, while GPT-4 exhibits remarkable improvement over other LLMs, it manages to accomplish only 39.73\% of the tasks, leaving a huge space for improvement. We address these challenges by proposing ML-Agent, designed to effectively navigate the codebase, locate documentation, retrieve code, and generate executable code. Empirical results demonstrate that ML-Agent, built upon GPT-4, results in further improvements. Code, data, and models are available at \url{https://ml-bench.github.io/}.

</details>

<details>

<summary>摘要</summary>

大型语言模型在代码生成比赛中表现出了优异的成绩。然而，实际Programming中对于这些模型的实用性存在许多差距，主要是因为实际程式码中的对于预存函数的依赖。相反于评估这些模型在从零开始写代码的能力，这个工作尝试了一个新的评估设置，让模型使用公开源代码库中的函数来完成机器学习任务。因此，我们提出了 ML-Bench，一个包含10044个样本、130个任务和14个知名机器学习GitHub来源的广泛的库。在这个设置下，给定一个机器学习任务的指令和相应的README档案，一个模型需要使用代码库中的函数来完成任务。这需要理解长长的文档和代码档案之间的交互，以及复杂的跨档代码结构，带来新的挑战。对此，我们提出了 ML-Agent，用于有效地浏览代码库、找到文档、获取代码和实现可执行的代码。实验结果显示，使用 GPT-4 建立的 ML-Agent 导致了进一步的改善。代码、数据和模型可以在 \url{https://ml-bench.github.io/} 获取。

</details></li>

</ul>

<hr>

<h2 id="AutoPlanBench-Automatically-generating-benchmarks-for-LLM-planners-from-PDDL"><a href="#AutoPlanBench-Automatically-generating-benchmarks-for-LLM-planners-from-PDDL" class="headerlink" title="AutoPlanBench: : Automatically generating benchmarks for LLM planners from PDDL"></a>AutoPlanBench: : Automatically generating benchmarks for LLM planners from PDDL</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09830">http://arxiv.org/abs/2311.09830</a></li>

<li>repo_url: None</li>

<li>paper_authors: Katharina Stein, Alexander Koller</li>

<li>for: 这paper的目的是探讨LLMs在规划任务中的能力，以及如何使用文本描述来评估这些能力。</li>

<li>methods: 这paper使用了一种新的方法，可以将PDDL中的规划benchmark转换成文本描述，并提供了一个新的benchmark集。</li>

<li>results: 研究发现，当今最好的LLM规划器在许多规划任务上表现出色，但是其他任务仍然远远超出了现有方法的能力范围。<details>

<summary>Abstract</summary>

LLMs are being increasingly used for planning-style tasks, but their capabilities for planning and reasoning are poorly understood. We present a novel method for automatically converting planning benchmarks written in PDDL into textual descriptions and offer a benchmark dataset created with our method. We show that while the best LLM planners do well on many planning tasks, others remain out of reach of current methods.

</details>

<details>

<summary>摘要</summary>

LLMs 正在越来越多地用于计划样式的任务，但它们的计划和理解能力尚未得到充分的理解。我们提出了一种新的方法，可以自动将 PDDL 格式的计划标准转换成文本描述，并提供了我们创建的 benchmark 数据集。我们发现，当前的 LLM 计划器在许多计划任务上表现出色，但有些任务仍然超出当前的能力范围。

</details></li>

</ul>

<hr>

<h2 id="PWISeg-Point-based-Weakly-supervised-Instance-Segmentation-for-Surgical-Instruments"><a href="#PWISeg-Point-based-Weakly-supervised-Instance-Segmentation-for-Surgical-Instruments" class="headerlink" title="PWISeg: Point-based Weakly-supervised Instance Segmentation for Surgical Instruments"></a>PWISeg: Point-based Weakly-supervised Instance Segmentation for Surgical Instruments</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09819">http://arxiv.org/abs/2311.09819</a></li>

<li>repo_url: <a target="_blank" rel="noopener" href="https://github.com/seanxuu/PWISeg">https://github.com/seanxuu/PWISeg</a></li>

<li>paper_authors: Zhen Sun, Huan Xu, Jinlin Wu, Zhen Chen, Zhen Lei, Hongbin Liu</li>

<li>for: 这个论文的目的是提出一种新的、有效的医疗器械实例分割方法，以解决实际中获得mask-level注释是劳动密集的问题。</li>

<li>methods: 该方法基于FCN架构，包括点到方框和点到面分支，用于模型特征点和 bounding box 之间的关系，以及特征点和分割面之间的关系。</li>

<li>results: 该方法在我们所提供的新的医疗器械数据集上进行了广泛的试验，并证明了与大多数无监督 bounding box 的实例分割方法相比，它的实rument分割精度得到了显著提高。<details>

<summary>Abstract</summary>

In surgical procedures, correct instrument counting is essential. Instance segmentation is a location method that locates not only an object's bounding box but also each pixel's specific details. However, obtaining mask-level annotations is labor-intensive in instance segmentation. To address this issue, we propose a novel yet effective weakly-supervised surgical instrument instance segmentation approach, named Point-based Weakly-supervised Instance Segmentation (PWISeg). PWISeg adopts an FCN-based architecture with point-to-box and point-to-mask branches to model the relationships between feature points and bounding boxes, as well as feature points and segmentation masks on FPN, accomplishing instrument detection and segmentation jointly in a single model. Since mask level annotations are hard to available in the real world, for point-to-mask training, we introduce an unsupervised projection loss, utilizing the projected relation between predicted masks and bboxes as supervision signal. On the other hand, we annotate a few pixels as the key pixel for each instrument. Based on this, we further propose a key pixel association loss and a key pixel distribution loss, driving the point-to-mask branch to generate more accurate segmentation predictions. To comprehensively evaluate this task, we unveil a novel surgical instrument dataset with manual annotations, setting up a benchmark for further research. Our comprehensive research trial validated the superior performance of our PWISeg. The results show that the accuracy of surgical instrument segmentation is improved, surpassing most methods of instance segmentation via weakly supervised bounding boxes. This improvement is consistently observed in our proposed dataset and when applied to the public HOSPI-Tools dataset.

</details>

<details>

<summary>摘要</summary>

在手术过程中，正确的工具数量是非常重要的。实例分割是一种位置方法，可以不仅找到物体的包围盒，还可以每个像素的特定细节。然而，在实例分割中获得mask水平的注释是很劳动密集的。为解决这个问题，我们提出了一种新的但有效的weakly-supervised手术工具实例分割方法，名为Point-based Weakly-supervised Instance Segmentation（PWISeg）。PWISeg采用了FCN基 architecture，并设置了点到包围盒和点到面积分支，以模型特征点和包围盒之间的关系，以及特征点和分割面积之间的关系。这样可以同时完成工具检测和分割。由于mask水平的注释很难在实际世界中获得，为点到面训练，我们引入了一种无监督投影损失，利用预测的面积和包围盒之间的投影关系作为监督信号。此外，我们还标注了每个工具的一些键点，并基于这些键点，我们进一步提出了键点协会损失和键点分布损失，使点到面分支生成更加准确的分割预测。为全面评估这个任务，我们披露了一个新的手术工具数据集，并设置了一个标准的比较基准。我们的全面研究试验证明了PWISeg的超越性。结果表明，通过我们提出的PWISeg，手术工具分割的准确性得到了提高，超越了大多数基于weakly supervised bounding box的实例分割方法。这种改进是在我们所提出的数据集和公共HOSPI-Tools数据集上均可见。

</details></li>

</ul>

<hr>

<h2 id="MedAgents-Large-Language-Models-as-Collaborators-for-Zero-shot-Medical-Reasoning"><a href="#MedAgents-Large-Language-Models-as-Collaborators-for-Zero-shot-Medical-Reasoning" class="headerlink" title="MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning"></a>MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10537">http://arxiv.org/abs/2311.10537</a></li>

<li>repo_url: <a target="_blank" rel="noopener" href="https://github.com/gersteinlab/medagents">https://github.com/gersteinlab/medagents</a></li>

<li>paper_authors: Xiangru Tang, Anni Zou, Zhuosheng Zhang, Yilun Zhao, Xingyao Zhang, Arman Cohan, Mark Gerstein</li>

<li>for: 本研究旨在提高语言模型在医学领域的表现，并且提供一种可行的多学科合作框架，以便语言模型可以更好地理解和应用医学知识。</li>

<li>methods: 本研究使用了多学科合作框架，其包括五个关键步骤：寻找培训数据，提出个人分析，汇总分析成报告，进行多轮讨论，并最终做出决策。</li>

<li>results: 本研究在九个数据集（MedQA、MedMCQA、PubMedQA以及MMLU中的六个子任务）上得到了出色的结果，证明了我们提出的多学科合作框架可以帮助语言模型更好地理解和应用医学知识，同时还可以扩展其理解能力。<details>

<summary>Abstract</summary>

Large Language Models (LLMs), despite their remarkable progress across various general domains, encounter significant barriers in medicine and healthcare. This field faces unique challenges such as domain-specific terminologies and the reasoning over specialized knowledge. To address these obstinate issues, we propose a novel Multi-disciplinary Collaboration (MC) framework for the medical domain that leverages role-playing LLM-based agents who participate in a collaborative multi-round discussion, thereby enhancing LLM proficiency and reasoning capabilities. This training-free and interpretable framework encompasses five critical steps: gathering domain experts, proposing individual analyses, summarising these analyses into a report, iterating over discussions until a consensus is reached, and ultimately making a decision. Our work particularly focuses on the zero-shot scenario, our results on nine data sets (MedQA, MedMCQA, PubMedQA, and six subtasks from MMLU) establish that our proposed MC framework excels at mining and harnessing the medical expertise in LLMs, as well as extending its reasoning abilities. Based on these outcomes, we further conduct a human evaluation to pinpoint and categorize common errors within our method, as well as ablation studies aimed at understanding the impact of various factors on overall performance. Our code can be found at \url{https://github.com/gersteinlab/MedAgents}.

</details>

<details>

<summary>摘要</summary>

大型语言模型（LLM），尽管在不同领域中做出了惊人的进步，在医疗领域还是遇到了很多障碍。这个领域面临着域名特定的术语和专业知识的推理等独特问题。为了解决这些难题，我们提出了一种新的多学科合作（MC）框架，该框架通过多个角色扮演LLM基于的代理人参与协同多轮讨论，从而提高LLM的技能和推理能力。这个无需训练和可解释的框架包括五个关键步骤：召集域专家、提出个人分析、汇总分析成报告、讨论迭代 until 达成一致，并最终做出决策。我们的工作特别关注于零容量情况下的情况，我们的结果表明，我们提出的MC框架在激活和利用医疗领域LLM的专业知识，以及扩展其推理能力方面表现出色。基于这些结果，我们进一步进行了人类评估，以找到和分类我们方法中的常见错误，以及进行了缺失因素的研究，以了解对总性表现的影响。我们的代码可以在 \url{https://github.com/gersteinlab/MedAgents} 找到。

</details></li>

</ul>

<hr>

<h2 id="Performance-Trade-offs-of-Watermarking-Large-Language-Models"><a href="#Performance-Trade-offs-of-Watermarking-Large-Language-Models" class="headerlink" title="Performance Trade-offs of Watermarking Large Language Models"></a>Performance Trade-offs of Watermarking Large Language Models</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09816">http://arxiv.org/abs/2311.09816</a></li>

<li>repo_url: None</li>

<li>paper_authors: Anirudh Ajith, Sameer Singh, Danish Pruthi</li>

<li>for: 本研究旨在评估水印模型在多种任务上的性能，以了解水印的影响和用户应该考虑的贸易OFF。</li>

<li>methods: 本研究使用了水印策略，在生成文本中嵌入一个信号，以便区分人工生成的文本和模型生成的文本。</li>

<li>results: 研究发现，在大多数情况下，水印对任务的性能没有显著影响。但是，长形生成任务（如概要和翻译）的性能下降了15-20%。这些结果指出了水印使用的贸易OFF，并提出了未来研究的可能性。<details>

<summary>Abstract</summary>

Amidst growing concerns of large language models (LLMs) being misused for generating misinformation or completing homework assignments, watermarking has emerged as an effective solution for distinguishing human-written and LLM-generated text. A prominent watermarking strategy is to embed a signal into generated text by upsampling a (pseudorandomly-chosen) subset of tokens at every generation step. Although this signal is imperceptible to a human reader, it is detectable through statistical testing. However, implanting such signals alters the model's output distribution and can have unintended effects when watermarked LLMs are used for downstream applications. In this work, we evaluate the performance of watermarked LLMs on a diverse suite of tasks, including text classification, textual entailment, reasoning, question answering, translation, summarization, and language modeling. We find that watermarking has negligible impact on the performance of tasks posed as k-class classification problems in the average case. However, the accuracy can plummet to that of a random classifier for some scenarios (that occur with non-negligible probability). Tasks that are cast as multiple-choice questions and short-form generation are surprisingly unaffected by watermarking. For long-form generation tasks, including summarization and translation, we see a drop of 15-20% in the performance due to watermarking. Our findings highlight the trade-offs that users should be cognizant of when using watermarked models, and point to cases where future research could improve existing trade-offs.

</details>

<details>

<summary>摘要</summary>

在大型语言模型（LLM）被违用于生成谣言或完成作业任务时，水印技术已成为一种有效的解决方案，以区分人类写作和LLM生成的文本。一种常见的水印策略是在生成文本时附加一个信号，通过随机选择一部分token进行upsampling。尽管这个信号对人类读者无法察觉，但可以通过统计测试探测。然而，植入这个信号会改变模型的输出分布，可能会导致下游应用中的不良影响。在这个工作中，我们评估水印LLM在多种任务上的表现，包括文本分类、文本推理、问答、翻译、概要和语言模型。我们发现，对于大多数情况，水印对任务的性能没有显著影响。然而，在某些特殊情况下（占总体的非致命概率），水印可能会导致性能降低到随机分类器水平。在长形生成任务中，如概要和翻译，我们发现水印导致性能下降约15-20%。我们的发现指出了使用水印模型时的交易offs，并提出了未来研究可以改善现有交易offs的可能性。

</details></li>

</ul>

<hr>

<h2 id="Towards-Formal-Fault-Injection-for-Safety-Assessment-of-Automated-Systems"><a href="#Towards-Formal-Fault-Injection-for-Safety-Assessment-of-Automated-Systems" class="headerlink" title="Towards Formal Fault Injection for Safety Assessment of Automated Systems"></a>Towards Formal Fault Injection for Safety Assessment of Automated Systems</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09810">http://arxiv.org/abs/2311.09810</a></li>

<li>repo_url: None</li>

<li>paper_authors: Ashfaq Farooqui, Behrooz Sangchoolie</li>

<li>for: 这篇论文是为了探讨自动化系统中的安全性、安全性和其他可靠性特性的问题，以便在日常生活中广泛应用自动化系统。</li>

<li>methods: 这篇论文使用了正式方法，这些方法可以数学方式推导系统的行为，从而确保系统的可靠性。然而，这些方法通常只适用于系统抽象模型，可能不能完全反映实际系统。</li>

<li>results: 这篇论文提出了正式缺陷插入，一种将正式方法和缺陷插入相结合的技术，以提高自动化系统的可靠性。文章还讨论了这些技术在开发生命周期中的潜在优势，并提出了未来研究的可能性，以解决当前的挑战。<details>

<summary>Abstract</summary>

Reasoning about safety, security, and other dependability attributes of autonomous systems is a challenge that needs to be addressed before the adoption of such systems in day-to-day life. Formal methods is a class of methods that mathematically reason about a system's behavior. Thus, a correctness proof is sufficient to conclude the system's dependability. However, these methods are usually applied to abstract models of the system, which might not fully represent the actual system. Fault injection, on the other hand, is a testing method to evaluate the dependability of systems. However, the amount of testing required to evaluate the system is rather large and often a problem. This vision paper introduces formal fault injection, a fusion of these two techniques throughout the development lifecycle to enhance the dependability of autonomous systems. We advocate for a more cohesive approach by identifying five areas of mutual support between formal methods and fault injection. By forging stronger ties between the two fields, we pave the way for developing safe and dependable autonomous systems. This paper delves into the integration's potential and outlines future research avenues, addressing open challenges along the way.

</details>

<details>

<summary>摘要</summary>

考虑 autonomous systems 的安全性、安全性和其他可靠性特性的推理是在普及这些系统之前需要解决的挑战。Formal methods 是一类方法，通过数学方式推理系统的行为，因此，一个正确性证明即可确保系统的可靠性。然而，这些方法通常应用于系统抽象模型，可能不完全反映实际系统。错误插入测试是一种评估系统可靠性的方法，但测试量很大，经常成为问题。这篇视野论文介绍了形式错误插入，它将这两种技术在开发生命周期中融合，以提高自动化系统的可靠性。我们认为这两个领域之间存在五个互助领域，通过加强这两个领域之间的关系，我们开拓了开发安全可靠的自动化系统的可能性。这篇论文探讨了融合的潜在可能性和未来研究方向，并解决了一些开放的挑战。

</details></li>

</ul>

<hr>

<h2 id="Comparing-Differentiable-Logics-for-Learning-Systems-A-Research-Preview"><a href="#Comparing-Differentiable-Logics-for-Learning-Systems-A-Research-Preview" class="headerlink" title="Comparing Differentiable Logics for Learning Systems: A Research Preview"></a>Comparing Differentiable Logics for Learning Systems: A Research Preview</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09809">http://arxiv.org/abs/2311.09809</a></li>

<li>repo_url: <a target="_blank" rel="noopener" href="https://github.com/tflinkow/dl-comparison">https://github.com/tflinkow/dl-comparison</a></li>

<li>paper_authors: Thomas Flinkow, Barak A. Pearlmutter, Rosemary Monahan</li>

<li>for: 这篇论文旨在研究如何使机器学习（ML）系统满足正确性和安全性要求，并考虑了自动化系统的自我更新和适应能力。</li>

<li>methods: 论文使用了 differentiable logics 方法，其中 Background Knowledge 编码为逻辑约束，导引学习过程。</li>

<li>results: 实验结果与文献报道的结果相符，但是使用 differentiable logics 引入了一个新的 гиперпарамет，即 tuning 难度和影响力。<details>

<summary>Abstract</summary>

Extensive research on formal verification of machine learning (ML) systems indicates that learning from data alone often fails to capture underlying background knowledge. A variety of verifiers have been developed to ensure that a machine-learnt model satisfies correctness and safety properties, however, these verifiers typically assume a trained network with fixed weights. ML-enabled autonomous systems are required to not only detect incorrect predictions, but should also possess the ability to self-correct, continuously improving and adapting. A promising approach for creating ML models that inherently satisfy constraints is to encode background knowledge as logical constraints that guide the learning process via so-called differentiable logics. In this research preview, we compare and evaluate various logics from the literature in weakly-supervised contexts, presenting our findings and highlighting open problems for future work. Our experimental results are broadly consistent with results reported previously in literature; however, learning with differentiable logics introduces a new hyperparameter that is difficult to tune and has significant influence on the effectiveness of the logics.

</details>

<details>

<summary>摘要</summary>

根据大量研究，机器学习（ML）系统从数据alone学习时常常无法捕捉下面背景知识。为确保机器学习模型满足正确性和安全性质量，许多验证工具已经开发，但这些验证工具通常假设已经训练过的网络重量是固定的。ML自适应系统需要不仅检测错误预测，还应该具备自我更新和适应能力。一种有前途的方法是通过 différentiable logics将背景知识编码成逻辑约束，以导引学习过程。在这个研究预览中，我们对文献中的不同逻辑进行比较和评价，在弱监督上下文中展示我们的发现和挑出未来工作的开放问题。我们的实验结果与文献中已经报道的结果相符，但是学习与 diffe抽象逻辑引入了一个新的Hyperparameter，它具有很大的影响力和难于调整。

</details></li>

</ul>

<hr>

<h2 id="Neuro-Symbolic-Integration-Brings-Causal-and-Reliable-Reasoning-Proofs"><a href="#Neuro-Symbolic-Integration-Brings-Causal-and-Reliable-Reasoning-Proofs" class="headerlink" title="Neuro-Symbolic Integration Brings Causal and Reliable Reasoning Proofs"></a>Neuro-Symbolic Integration Brings Causal and Reliable Reasoning Proofs</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09802">http://arxiv.org/abs/2311.09802</a></li>

<li>repo_url: <a target="_blank" rel="noopener" href="https://github.com/damo-nlp-sg/caring">https://github.com/damo-nlp-sg/caring</a></li>

<li>paper_authors: Sen Yang, Xin Li, Leyang Cui, Lidong Bing, Wai Lam</li>

<li>for: 这 paper 是为了提高 AI 模型的推理能力和可靠性而写的。</li>

<li>methods: 这 paper 使用了一种 combining 方法，即将 neural LLM 和 symbolic solver integrate 在一起，以便进行 deliberative reasoning 和证明。</li>

<li>results: 实验表明，使用这种方法可以在 ProofWriter 和 GSM8K 上大幅提高推理 accuracy 和证明相似性。<details>

<summary>Abstract</summary>

Though prompting LLMs with various reasoning structures produces reasoning proofs along with answers, these proofs are not ensured to be causal and reliable due to the inherent defects of LLMs. Tracking such deficiencies, we present a neuro-symbolic integration method, in which a neural LLM is used to represent the knowledge of the problem while an LLM-free symbolic solver is adopted to do deliberative reasoning using the knowledge. Specifically, our customized meta-interpreters allow the production of reasoning proofs and support flexible search strategies. These reasoning proofs are ensured to be causal and reliable because of the deterministic executing nature of the symbolic solvers. Empirically, on ProofWriter, our method surpasses the CoT baseline by nearly double in accuracy and more than triple in proof similarity. On GSM8K, our method also shows accuracy improvements and nearly doubled proof similarity. Our code is released at https://github.com/DAMO-NLP-SG/CaRing

</details>

<details>

<summary>摘要</summary>

尽管通过不同的逻辑结构让LLMs进行推理生成推理证明和答案，但这些证明并不能保证是 causal 和可靠的，因为 LLMS 本身存在一些缺陷。为了解决这些问题，我们提出了一种神经符号 интеграция方法，在这种方法中，一个神经网络 LLM 用于表示问题的知识，而另一个 LLM-free 符号分析器用于进行思考和推理。具体来说，我们自定义的 meta-interpreters 允许生成推理证明和支持灵活的搜索策略。由于符号分析器的 deterministic 执行特性，所生成的推理证明是 causal 和可靠的。在 ProofWriter 上，我们的方法比 CoT 基线高出 nearly double 的准确率和 более triple 的证明相似度。在 GSM8K 上，我们的方法也显示了准确率上的提高和证明相似度的近 double。我们的代码在 <https://github.com/DAMO-NLP-SG/CaRing> 上发布。

</details></li>

</ul>

<hr>

<h2 id="Interpreting-User-Requests-in-the-Context-of-Natural-Language-Standing-Instructions"><a href="#Interpreting-User-Requests-in-the-Context-of-Natural-Language-Standing-Instructions" class="headerlink" title="Interpreting User Requests in the Context of Natural Language Standing Instructions"></a>Interpreting User Requests in the Context of Natural Language Standing Instructions</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09796">http://arxiv.org/abs/2311.09796</a></li>

<li>repo_url: None</li>

<li>paper_authors: Nikita Moghe, Patrick Xia, Jacob Andreas, Jason Eisner, Benjamin Van Durme, Harsh Jhamtani</li>

<li>for: 用于提高自然语言界面的用户体验，减少用户重复提供偏好信息的需求。</li>

<li>methods: 利用大型语言模型（LLMs）和自然语言描述文本（standing instructions）作为用户偏好和指令的补充信息。</li>

<li>results: 在NLSI语料集上进行实验，使用大语言模型和不同的检索方法，最高达44.7%的精确匹配率。<details>

<summary>Abstract</summary>

Users of natural language interfaces, generally powered by Large Language Models (LLMs),often must repeat their preferences each time they make a similar request. To alleviate this, we propose including some of a user's preferences and instructions in natural language -- collectively termed standing instructions -- as additional context for such interfaces. For example, when a user states I'm hungry, their previously expressed preference for Persian food will be automatically added to the LLM prompt, so as to influence the search for relevant restaurants. We develop NLSI, a language-to-program dataset consisting of over 2.4K dialogues spanning 17 domains, where each dialogue is paired with a user profile (a set of users specific standing instructions) and corresponding structured representations (API calls). A key challenge in NLSI is to identify which subset of the standing instructions is applicable to a given dialogue. NLSI contains diverse phenomena, from simple preferences to interdependent instructions such as triggering a hotel search whenever the user is booking tickets to an event. We conduct experiments on NLSI using prompting with large language models and various retrieval approaches, achieving a maximum of 44.7% exact match on API prediction. Our results demonstrate the challenges in identifying the relevant standing instructions and their interpretation into API calls.

</details>

<details>

<summary>摘要</summary>

用户们使用自然语言界面，通常需要每次发出相似的请求都重复他们的首选项。为了解决这个问题，我们建议将用户的首选项和指令（总称为“站坐指令”）作为自然语言 interfaces 的附加上下文。例如，当用户说 “我饿” 时，他们之前表达的波斯料食物的首选项将自动添加到 LLM 提示中，以影响搜索相关餐厅。我们开发了 NLSI，一个语言到程序数据集，包含了超过 2.4K 对话，涵盖 17 个领域，每个对话都与用户的 Profile（用户特定的站坐指令）和相应的结构化表示（API 调用）一起出现。在 NLSI 中，一个主要挑战是确定哪些站坐指令适用于给定的对话。NLSI 包含了多种现象，从简单的首选项到互相关联的指令，例如在购买票务时自动触发酒店搜索。我们使用 LLM 和不同的检索方法进行实验，最高达 44.7% 精确匹配 API 预测。我们的结果表明了站坐指令的适用和其 интерпретация成 API 调用的挑战。

</details></li>

</ul>

<hr>

<h2 id="Breaking-Boundaries-Balancing-Performance-and-Robustness-in-Deep-Wireless-Traffic-Forecasting"><a href="#Breaking-Boundaries-Balancing-Performance-and-Robustness-in-Deep-Wireless-Traffic-Forecasting" class="headerlink" title="Breaking Boundaries: Balancing Performance and Robustness in Deep Wireless Traffic Forecasting"></a>Breaking Boundaries: Balancing Performance and Robustness in Deep Wireless Traffic Forecasting</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09790">http://arxiv.org/abs/2311.09790</a></li>

<li>repo_url: None</li>

<li>paper_authors: Romain Ilbert, Thai V. Hoang, Zonghua Zhang, Themis Palpanas</li>

<li>for: 本研究旨在寻找一种能够平衡精度和Robustness的时序预测方法，以便在真实世界中的电信数据上进行预测。</li>

<li>methods: 我们使用了一种混合策略，包括一个分类器来检测外延攻击，一个去噪器来消除外延数据样本中的噪声，以及一个标准预测模型。我们对这些策略进行了比较，并与两种现有的对抗训练算法进行了比较。</li>

<li>results: 我们的hybrid策略在 both clean和外延数据上表现出色，其MSE在clean数据上保持了92.02%的原始预测模型性能，而在外延数据上则更加Robust，其MSE比较方法的MSE低出2.71倍和2.51倍。此外，我们的模型的组件可以并行训练，从而提高计算效率。<details>

<summary>Abstract</summary>

Balancing the trade-off between accuracy and robustness is a long-standing challenge in time series forecasting. While most of existing robust algorithms have achieved certain suboptimal performance on clean data, sustaining the same performance level in the presence of data perturbations remains extremely hard. In this paper, we study a wide array of perturbation scenarios and propose novel defense mechanisms against adversarial attacks using real-world telecom data. We compare our strategy against two existing adversarial training algorithms under a range of maximal allowed perturbations, defined using $\ell_{\infty}$-norm, $\in [0.1,0.4]$. Our findings reveal that our hybrid strategy, which is composed of a classifier to detect adversarial examples, a denoiser to eliminate noise from the perturbed data samples, and a standard forecaster, achieves the best performance on both clean and perturbed data. Our optimal model can retain up to $92.02\%$ the performance of the original forecasting model in terms of Mean Squared Error (MSE) on clean data, while being more robust than the standard adversarially trained models on perturbed data. Its MSE is 2.71$\times$ and 2.51$\times$ lower than those of comparing methods on normal and perturbed data, respectively. In addition, the components of our models can be trained in parallel, resulting in better computational efficiency. Our results indicate that we can optimally balance the trade-off between the performance and robustness of forecasting models by improving the classifier and denoiser, even in the presence of sophisticated and destructive poisoning attacks.

</details>

<details>

<summary>摘要</summary>

平衡精度和Robustness之间的贸易OFF是时间序列预测领域的长standing挑战。大多数现有的Robust算法在干净数据上达到了一定的下行性，但在数据抖动的情况下维持同样的性能很难。在这篇论文中，我们研究了各种抖动enario并提出了新的防御机制，使用实际的电信数据对抗 adversarial 攻击。我们与两种现有的 adversarial 训练算法进行比较，使用 $[0.1,0.4]$ 的 $\ell_{\infty}$ 范围内的最大允许抖动。我们的发现显示，我们的混合策略，包括一个分类器来检测 adversarial 示例，一个去噪器来除掉抖动数据示例中的噪声，以及一个标准预测器，在干净数据和抖动数据上都能够达到最好的性能。我们的优化模型可以保持原始预测模型的 $92.02\%$ 的性能（按照 Mean Squared Error 的评价），而且在抖动数据上比标准 adversarial 训练模型更加Robust。它的 MSE 值分别为 $2.71\times$ 和 $2.51\times$ 比对应模型的 MSE 值更低。此外，我们的模型组件可以并行训练，从而提高计算效率。我们的结果表明，我们可以通过改进分类器和去噪器来优化 forecasting 模型，甚至在抖动数据上进行高级和破坏性攻击。

</details></li>

</ul>

<hr>

<h2 id="3vLTL-A-Tool-to-Generate-Automata-for-Three-valued-LTL"><a href="#3vLTL-A-Tool-to-Generate-Automata-for-Three-valued-LTL" class="headerlink" title="3vLTL: A Tool to Generate Automata for Three-valued LTL"></a>3vLTL: A Tool to Generate Automata for Three-valued LTL</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09787">http://arxiv.org/abs/2311.09787</a></li>

<li>repo_url: None</li>

<li>paper_authors: Francesco Belardinelli, Angelo Ferrando, Vadim Malvone</li>

<li>for: 这篇论文主要是为了提供一个工具来生成 Buchi 自动机，用于验证 Linear-time Temporal Logic（LTL）式语言中的多值规定。</li>

<li>methods: 该工具使用Linear-time Temporal Logic（LTL）式语言中的三值 semantics来解释 formulas，并生成一个 Buchi 自动机，用于验证 LTL 式是否真、假或未定义于一个模型中。</li>

<li>results: 该工具可以生成一个 Buchi 自动机，用于验证 LTL 式的真假性，并且可以让这个自动机被第三方库处理，以便进一步进行验证。<details>

<summary>Abstract</summary>

Multi-valued logics have a long tradition in the literature on system verification, including run-time verification. However, comparatively fewer model-checking tools have been developed for multi-valued specification languages. We present 3vLTL, a tool to generate Buchi automata from formulas in Linear-time Temporal Logic (LTL) interpreted on a three-valued semantics. Given an LTL formula, a set of atomic propositions as the alphabet for the automaton, and a truth value, our procedure generates a Buchi automaton that accepts all the words that assign the chosen truth value to the LTL formula. Given the particular type of the output of the tool, it can also be seamlessly processed by third-party libraries in a natural way. That is, the Buchi automaton can then be used in the context of formal verification to check whether an LTL formula is true, false, or undefined on a given model.

</details>

<details>

<summary>摘要</summary>

多值逻辑在系统验证文献中有很长的传统，包括运行时验证。然而，相比之下， fewer model-checking工具被开发用于多值规定语言。我们介绍了3vLTL，一种生成 Buchi 自动机从Linear-time Temporal Logic（LTL）在三值 semantics中解释的方程的工具。给定一个 LTL 方程，一个字母集，一个真假值，我们的过程生成一个 Buchi 自动机，接受将选择的真假值分配给 LTL 方程的所有词。给出特定输出的类型，这个 Buchi 自动机可以轻松地处理第三方库中的自然方式。也就是说，Buch i自动机可以在正式验证中使用，以验证一个 LTL 方程是否真、假或未定义于一个模型。

</details></li>

</ul>

<hr>

<h2 id="Correct-by-Construction-Control-for-Stochastic-and-Uncertain-Dynamical-Models-via-Formal-Abstractions"><a href="#Correct-by-Construction-Control-for-Stochastic-and-Uncertain-Dynamical-Models-via-Formal-Abstractions" class="headerlink" title="Correct-by-Construction Control for Stochastic and Uncertain Dynamical Models via Formal Abstractions"></a>Correct-by-Construction Control for Stochastic and Uncertain Dynamical Models via Formal Abstractions</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09786">http://arxiv.org/abs/2311.09786</a></li>

<li>repo_url: None</li>

<li>paper_authors: Thom Badings, Nils Jansen, Licio Romao, Alessandro Abate</li>

<li>for: 这篇论文的目的是为Autonomous Systems的自动化控制器的批量生成提供了一种可靠的方法。</li>

<li>methods: 这篇论文使用了一种基于Markov决策过程的间接抽象方法，以及现有的验证技术来计算一个满足给定规范的优化策略。</li>

<li>results: 这篇论文的结果表明，通过使用这种方法，可以生成一个可靠地满足规范的控制器，并且可以证明这个控制器满足规范的 garanties。<details>

<summary>Abstract</summary>

Automated synthesis of correct-by-construction controllers for autonomous systems is crucial for their deployment in safety-critical scenarios. Such autonomous systems are naturally modeled as stochastic dynamical models. The general problem is to compute a controller that provably satisfies a given task, represented as a probabilistic temporal logic specification. However, factors such as stochastic uncertainty, imprecisely known parameters, and hybrid features make this problem challenging. We have developed an abstraction framework that can be used to solve this problem under various modeling assumptions. Our approach is based on a robust finite-state abstraction of the stochastic dynamical model in the form of a Markov decision process with intervals of probabilities (iMDP). We use state-of-the-art verification techniques to compute an optimal policy on the iMDP with guarantees for satisfying the given specification. We then show that, by construction, we can refine this policy into a feedback controller for which these guarantees carry over to the dynamical model. In this short paper, we survey our recent research in this area and highlight two challenges (related to scalability and dealing with nonlinear dynamics) that we aim to address with our ongoing research.

</details>

<details>

<summary>摘要</summary>

自动生成正确性承诺控制器是自主系统的部署中关键的一步。这些自主系统通常是随机动力学模型的。通用问题是计算一个可以准确满足给定任务的控制器，该任务是 probabilistic temporal logic 规范。然而，因为随机不确定性、不精确知道参数以及混合特征，这个问题具有挑战性。我们已经开发了一个抽象框架，可以在不同的模型假设下解决这个问题。我们的方法基于一种可靠的finite-state抽象方法，即Markov decision process with intervals of probabilities（iMDP）。我们使用当前的验证技术来计算iMDP上的优质策略，并 garantía para satisfacer给定规范。然后，我们表明，通过构建，我们可以从这种策略中提取一个反馈控制器，这些 garantías会传递到动力学模型中。在这篇短文中，我们回顾了我们近期在这个领域的研究，并提出了两个挑战（相关于扩展性和处理非线性动力学），我们计划通过我们的进行研究来解决这些挑战。

</details></li>

</ul>

<hr>

<h2 id="Automatic-Generation-of-Scenarios-for-System-level-Simulation-based-Verification-of-Autonomous-Driving-Systems"><a href="#Automatic-Generation-of-Scenarios-for-System-level-Simulation-based-Verification-of-Autonomous-Driving-Systems" class="headerlink" title="Automatic Generation of Scenarios for System-level Simulation-based Verification of Autonomous Driving Systems"></a>Automatic Generation of Scenarios for System-level Simulation-based Verification of Autonomous Driving Systems</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09784">http://arxiv.org/abs/2311.09784</a></li>

<li>repo_url: None</li>

<li>paper_authors: Srajan Goyal, Alberto Griggio, Jacob Kimblad, Stefano Tonetta<br>for:* The paper is written for the purpose of presenting a generic framework for system-level simulation-based verification and validation (V&amp;V) of autonomous driving systems (ADS) that employ AI components.methods:* The framework uses a simulation model of the system, an abstract model that describes symbolically the system behavior, and formal methods to generate scenarios and verify the simulation executions.* The approach leverages the CARLA driving simulator and its ScenarioRunner tool to create diverse and complex driving scenarios.results:* The paper describes the instantiation of the VIVAS framework for an ADS case study, and demonstrates the effectiveness of the approach in automatically generating scenarios for system-level simulation-based V&amp;V of an automated driving system using CARLA and ScenarioRunner.* The results show the potential of the approach as a powerful tool in the future of ADS V&amp;V methodologies.<details>

<summary>Abstract</summary>

With increasing complexity of Automated Driving Systems (ADS), ensuring their safety and reliability has become a critical challenge. The Verification and Validation (V&V) of these systems are particularly demanding when AI components are employed to implement perception and/or control functions. In ESA-funded project VIVAS, we developed a generic framework for system-level simulation-based V&V of autonomous systems. The approach is based on a simulation model of the system, an abstract model that describes symbolically the system behavior, and formal methods to generate scenarios and verify the simulation executions. Various coverage criteria can be defined to guide the automated generation of the scenarios. In this paper, we describe the instantiation of the VIVAS framework for an ADS case study. This is based on the integration of CARLA, a widely-used driving simulator, and its ScenarioRunner tool, which enables the creation of diverse and complex driving scenarios. This is also used in the CARLA Autonomous Driving Challenge to validate different ADS agents for perception and control based on AI, shared by the CARLA community. We describe the development of an abstract ADS model and the formulation of a coverage criterion that focuses on the behaviors of vehicles relative to the vehicle with ADS under verification. Leveraging the VIVAS framework, we generate and execute various driving scenarios, thus testing the capabilities of the AI components. The results show the effectiveness of VIVAS in automatically generating scenarios for system-level simulation-based V&V of an automated driving system using CARLA and ScenarioRunner. Therefore, they highlight the potential of the approach as a powerful tool in the future of ADS V&V methodologies.

</details>

<details>

<summary>摘要</summary>

随着自动驾驶系统（ADS）的复杂度增加，确保其安全性和可靠性已成为一项杰匡的挑战。验证和验议（V&V）这些系统特别是当AI组件用于感知和/或控制功能时，变得非常具有挑战性。在欧洲空间局（ESA）资助的项目VIVAS中，我们开发了一种通用框架 для系统级别的模拟基于验证和验议自动驾驶系统。该方法基于系统模型、一个抽象模型，用于 символи地描述系统行为，以及正式方法来生成场景和验证模拟执行。可以定义多种覆盖度标准来引导自动生成场景。 在这篇论文中，我们描述了VIvas框架在自动驾驶系统case study中的实现。这基于卡拉拉（CARLA）广泛使用的驾驶模拟器和其ScenarioRunner工具，可以创造多样化和复杂的驾驶场景。这也是在CARLA自动驾驶挑战中 validate不同的ADS代理人以及感知和控制基于AI的不同ADS代理人。我们开发了抽象的ADS模型，并将其与驾驶车辆相对于具有ADS的车辆的行为相关的覆盖度标准定义。通过VIvas框架，我们生成并执行多种驾驶场景，因此测试了AI组件的能力。结果显示VIvas在使用CARLA和ScenarioRunner进行系统级别的模拟基于验证和验议中自动生成场景的能力是非常有力的。因此，它高亮了未来ADS验证和验议方法的潜在力量。

</details></li>

</ul>

<hr>

<h2 id="Investigating-Data-Contamination-in-Modern-Benchmarks-for-Large-Language-Models"><a href="#Investigating-Data-Contamination-in-Modern-Benchmarks-for-Large-Language-Models" class="headerlink" title="Investigating Data Contamination in Modern Benchmarks for Large Language Models"></a>Investigating Data Contamination in Modern Benchmarks for Large Language Models</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09783">http://arxiv.org/abs/2311.09783</a></li>

<li>repo_url: None</li>

<li>paper_authors: Chunyuan Deng, Yilun Zhao, Xiangru Tang, Mark Gerstein, Arman Cohan</li>

<li>for: 提高LLM的评估标准和评估方法的可靠性</li>

<li>methods: 提议两种适用于开源和专有模型的数据污染检测方法，包括检索系统和TS推测法</li>

<li>results: 研究发现一些商业LLM可以很准确地猜测测试集中的缺失选项，并且在一些标准套件中发现了模型的表现改善。<details>

<summary>Abstract</summary>

Recent observations have underscored a disparity between the inflated benchmark scores and the actual performance of LLMs, raising concerns about potential contamination of evaluation benchmarks. This issue is especially critical for closed-source models and certain open-source models where training data transparency is lacking. In this paper we study data contamination by proposing two methods tailored for both open-source and proprietary LLMs. We first introduce a retrieval-based system to explore potential overlaps between evaluation benchmarks and pretraining corpora. We further present a novel investigation protocol named \textbf{T}estset \textbf{S}lot Guessing (\textit{TS-Guessing}), applicable to both open and proprietary models. This approach entails masking a wrong answer in a multiple-choice question and prompting the model to fill in the gap. Additionally, it involves obscuring an unlikely word in an evaluation example and asking the model to produce it. We find that certain commercial LLMs could surprisingly guess the missing option in various test sets. Specifically, in the TruthfulQA benchmark, we find that LLMs exhibit notable performance improvement when provided with additional metadata in the benchmark. Further, in the MMLU benchmark, ChatGPT and GPT-4 demonstrated an exact match rate of 52\% and 57\%, respectively, in guessing the missing options in benchmark test data. We hope these results underscore the need for more robust evaluation methodologies and benchmarks in the field.

</details>

<details>

<summary>摘要</summary>

近期观察发现了 LLM 的评估标准和实际性能之间的差距，引发了评估标准污染的 Concerns 。这个问题尤其对于关闭源模型和某些开源模型来说是 kritisch 。在这篇论文中，我们研究了数据污染问题，并提出了两种适用于开源和专有 LLM 的方法。我们首先介绍了一个检索基于系统，用于探索评估标准和预训练 corpora 之间的可能的重叠。然后，我们介绍了一种新的调查协议，名为 \textbf{T}estset \textbf{S}lot Guessing (\textit{TS-Guessing）}, 适用于开源和专有模型。这种方法包括在多选问题中隐藏错误答案，并让模型填充差距。此外，还包括隐藏评估示例中不可能的单词，并让模型生成它。我们发现了一些商业 LLM 可以意外地猜测测试集中的缺失选项。例如，在 TruthfulQA benchmark 中，我们发现了 LLM 在提供额外metadata 时表现出 Notable 的性能提升。此外，在 MMLU benchmark 中， ChatGPT 和 GPT-4 在测试集中猜测缺失选项的精准率分别为 52% 和 57%。我们希望这些结果可以强调评估方法和标准的需要更加Robust 。

</details></li>

</ul>

<hr>

<h2 id="Model-Checking-for-Closed-Loop-Robot-Reactive-Planning"><a href="#Model-Checking-for-Closed-Loop-Robot-Reactive-Planning" class="headerlink" title="Model Checking for Closed-Loop Robot Reactive Planning"></a>Model Checking for Closed-Loop Robot Reactive Planning</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09780">http://arxiv.org/abs/2311.09780</a></li>

<li>repo_url: None</li>

<li>paper_authors: Christopher Chandler, Bernd Porr, Alice Miller, Giulia Lafratta</li>

<li>for: 本研究用模型检查来创建多步计划，使Diffuse Drive驱动轮胎自动车可以避免危险。</li>

<li>methods: 我们使用一种小型、专门设计的模型检查算法，在实时环境中生成计划，并且这种方法具有 Egocentric 反应性的简单生物体特征。我们的方法基于链接临时控制系统，通过消除环境中的干扰来让自动车保持其首选行为或休眠状态。我们使用2D LiDAR数据的精细化方法，敏感于环境中的 bounded 随机变化。我们使用深度优先搜索来实现多步规划，并通过cul-de-sac 场景作为第一个测试用例。</li>

<li>results: 我们的结果表明，模型检查可以用来规划效率的轨迹，超越单步规划的表现。我们在实时使用无预计算数据来实现这一点。虽然我们的方法有一些局限性，但我们认为我们的方法具有开发安全、可靠和透明的轨迹规划方法的潜力。<details>

<summary>Abstract</summary>

In this paper, we show how model checking can be used to create multi-step plans for a differential drive wheeled robot so that it can avoid immediate danger. Using a small, purpose built model checking algorithm in situ we generate plans in real-time in a way that reflects the egocentric reactive response of simple biological agents. Our approach is based on chaining temporary control systems which are spawned to eliminate disturbances in the local environment that disrupt an autonomous agent from its preferred action (or resting state). The method involves a novel discretization of 2D LiDAR data which is sensitive to bounded stochastic variations in the immediate environment. We operationalise multi-step planning using invariant checking by forward depth-first search, using a cul-de-sac scenario as a first test case. Our results demonstrate that model checking can be used to plan efficient trajectories for local obstacle avoidance, improving on the performance of a reactive agent which can only plan one step. We achieve this in near real-time using no pre-computed data. While our method has limitations, we believe our approach shows promise as an avenue for the development of safe, reliable and transparent trajectory planning in the context of autonomous vehicles.

</details>

<details>

<summary>摘要</summary>

在这篇论文中，我们展示了如何使用模型检查来创建多步计划，以使Diffusion Drive轮胎自动车避免 immediate danger。我们使用一种小型、特有的模型检查算法在实时中生成计划，以模仿简单生物体的 Egocentric 反应。我们的方法基于临时控制系统的链接，以消除环境中的干扰，使自动 Agent 能够继续进行其 preferred action（或休眠状态）。我们的方法包括一种新的2D LiDAR数据的精度化，敏感于环境中的 bounded 随机变化。我们通过在深度优先搜索中进行 invariants 检查来实现多步规划，并使用 cul-de-sac 场景作为第一个测试 caso。我们的结果表明，模型检查可以用于计划高效的轨迹，超越了只能计划一步的感知Agent。我们在实时中使用不需要预计算数据来实现这一点。虽然我们的方法有限制，但我们认为我们的方法显示出了在自动汽车中安全、可靠和透明的轨迹规划的可能性。

</details></li>

</ul>

<hr>

<h2 id="HuatuoGPT-II-One-stage-Training-for-Medical-Adaption-of-LLMs"><a href="#HuatuoGPT-II-One-stage-Training-for-Medical-Adaption-of-LLMs" class="headerlink" title="HuatuoGPT-II, One-stage Training for Medical Adaption of LLMs"></a>HuatuoGPT-II, One-stage Training for Medical Adaption of LLMs</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09774">http://arxiv.org/abs/2311.09774</a></li>

<li>repo_url: None</li>

<li>paper_authors: Junying Chen, Xidong Wang, Anningzhe Gao, Feng Jiang, Shunian Chen, Hongbo Zhang, Dingjie Song, Wenya Xie, Chuyi Kong, Jianquan Li, Xiang Wan, Haizhou Li, Benyou Wang</li>

<li>for: 这个论文的目的是适应具有特殊知识的领域，如医学，以便将特殊知识 integrate into一个通用语言模型中，如Llama2。</li>

<li>methods: 该论文提出了一种将多种数据集，包括预训练和指导训练，转化为一个统一的输入输出对形式，以简化学习协议。</li>

<li>results: 该论文通过对多个医学领域的测试，证明了其在中医领域的状态对级表现，并在一些方面超过了现有的专有模型，如ChatGPT和GPT-4。专家手动评估也证明了该模型的优势。<details>

<summary>Abstract</summary>

Adapting a language model into a specific domain, a.k.a `domain adaption', is a common practice when specialized knowledge, e.g. medicine, is not encapsulated in a general language model like Llama2. The challenge lies in the heterogeneity of data across the two training stages, as it varies in languages, genres, or formats. To tackle this and simplify the learning protocol, we propose to transform heterogeneous data, from the both pre-training and supervised stages, into a unified, simple input-output pair format. We validate the new protocol in the domains where proprietary LLMs like ChatGPT perform relatively poorly, such as Traditional Chinese Medicine. The developed model, HuatuoGPT-II, has shown state-of-the-art performance in Chinese medicine domain on a number of benchmarks, e.g. medical licensing exams. It even outperforms proprietary models like ChatGPT and GPT-4 in some aspects, especially in Traditional Chinese Medicine. Expert manual evaluations further validate HuatuoGPT-II's advantages over existing LLMs. Notably, HuatuoGPT-II was benchmarked in a fresh Chinese National Medical Licensing Examination where it achieved the best performance, showcasing not only its effectiveness but also its generalization capabilities.

</details>

<details>

<summary>摘要</summary>

适应特定领域的语言模型化，即域 adaptation，是一种常见的做法，当特殊知识，如医学，不包含在通用语言模型如LLAMA2中。挑战在两个训练阶段数据的异ogeneity上，因为数据的语言、种类、格式都不同。为了解决这个问题并简化学习协议，我们提议将各种不同数据，从预训练和监督两个阶段，转换成一个统一、简单的输入输出对 format。我们验证了新协议的效果在各种领域，如中医，并在一些标准测试任务上达到了状态 искусственный智能表现。特别是在传统中医领域，我们的模型 HuatuoGPT-II 表现出了优秀的result，并在一些方面超越了商业化模型，如ChatGPT和GPT-4。专业人员手动评估也证明了 HuatuoGPT-II 的优势。值得一提的是，HuatuoGPT-II 在新的中医国家医籍考试中达到了最佳表现，证明了它不仅有效，还有普适化能力。

</details></li>

</ul>

<hr>

<h2 id="Back-to-Basics-A-Simple-Recipe-for-Improving-Out-of-Domain-Retrieval-in-Dense-Encoders"><a href="#Back-to-Basics-A-Simple-Recipe-for-Improving-Out-of-Domain-Retrieval-in-Dense-Encoders" class="headerlink" title="Back to Basics: A Simple Recipe for Improving Out-of-Domain Retrieval in Dense Encoders"></a>Back to Basics: A Simple Recipe for Improving Out-of-Domain Retrieval in Dense Encoders</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09765">http://arxiv.org/abs/2311.09765</a></li>

<li>repo_url: <a target="_blank" rel="noopener" href="https://github.com/amy-hyunji/lora-for-retrieval">https://github.com/amy-hyunji/lora-for-retrieval</a></li>

<li>paper_authors: Hyunji Lee, Luca Soldaini, Arman Cohan, Minjoon Seo, Kyle Lo</li>

<li>for: 这个论文主要是为了提高 dense retriever 模型在未经见过的领域中的泛化能力。</li>

<li>methods: 该论文提出了一种简单的训练方法，包括使用 LoRA 等参数效率的方法，并选择使用 batch 内的负样本，除非给出了准确制定的困难负样本。</li>

<li>results: 该论文使用 BEIR benchmark 进行验证，并发现这些建议可以在不同的 dense encoder 和基础模型大小上 persist，并且与其他资源密集的策略（如建筑修改或多个预训练）相结合，可以提高 dense retriever 模型的泛化能力。<details>

<summary>Abstract</summary>

Prevailing research practice today often relies on training dense retrievers on existing large datasets such as MSMARCO and then experimenting with ways to improve zero-shot generalization capabilities to unseen domains. While prior work has tackled this challenge through resource-intensive steps such as data augmentation, architectural modifications, increasing model size, or even further base model pretraining, comparatively little investigation has examined whether the training procedures themselves can be improved to yield better generalization capabilities in the resulting models. In this work, we recommend a simple recipe for training dense encoders: Train on MSMARCO with parameter-efficient methods, such as LoRA, and opt for using in-batch negatives unless given well-constructed hard negatives. We validate these recommendations using the BEIR benchmark and find results are persistent across choice of dense encoder and base model size and are complementary to other resource-intensive strategies for out-of-domain generalization such as architectural modifications or additional pretraining. We hope that this thorough and impartial study around various training techniques, which augments other resource-intensive methods, offers practical insights for developing a dense retrieval model that effectively generalizes, even when trained on a single dataset.

</details>

<details>

<summary>摘要</summary>

现有研究往往采用 dense retriever 训练大量数据集 MSMARCO，然后尝试改进零shot泛化能力。而在优化这种泛化能力方面，已有许多研究，包括数据增强、结构修改、模型大小增加和额外预训练。然而，对于训练过程本身是否可以进行优化以实现更好的泛化能力，尚未得到了充分的探讨。在这项工作中，我们建议一种简单的 dense encoder 训练热键：通过 LoRA 等参数效率的方法在 MSMARCO 上训练，并在批处中使用卷积批处。我们使用 BEIR 测试准则 validate 这些建议，并发现结果是不依赖于 dense encoder 和基础模型大小，并且与其他资源占用量大的方法相结合，可以实现更好的泛化能力。我们希望这种具有充分的探讨和不偏袋的研究，能够为开发高效泛化 dense retrieval 模型提供实际的指导。

</details></li>

</ul>

<hr>

<h2 id="Graph-Guided-Reasoning-for-Multi-Hop-Question-Answering-in-Large-Language-Models"><a href="#Graph-Guided-Reasoning-for-Multi-Hop-Question-Answering-in-Large-Language-Models" class="headerlink" title="Graph-Guided Reasoning for Multi-Hop Question Answering in Large Language Models"></a>Graph-Guided Reasoning for Multi-Hop Question Answering in Large Language Models</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09762">http://arxiv.org/abs/2311.09762</a></li>

<li>repo_url: None</li>

<li>paper_authors: Jinyoung Park, Ameen Patel, Omar Zia Khan, Hyunwoo J. Kim, Joo-Kyung Kim</li>

<li>For: The paper aims to improve the multi-step reasoning capabilities of large language models (LLMs) by addressing two issues in previous CoT prompting methods: generating irrelevant rationales and failing to compose subquestions or queries for obtaining all relevant information.* Methods: The proposed graph-guided CoT prompting method uses a “question&#x2F;rationale graph” constructed by LLMs to guide the reasoning process. The method includes graph representation and verification steps to filter out irrelevant rationales and generate follow-up questions to obtain relevant information.* Results: The proposed method shows superior performance compared to previous CoT prompting methods and their variants on multi-hop question answering benchmark datasets.<details>

<summary>Abstract</summary>

Chain-of-Thought (CoT) prompting has boosted the multi-step reasoning capabilities of Large Language Models (LLMs) by generating a series of rationales before the final answer. We analyze the reasoning paths generated by CoT and find two issues in multi-step reasoning: (i) Generating rationales irrelevant to the question, (ii) Unable to compose subquestions or queries for generating/retrieving all the relevant information. To address them, we propose a graph-guided CoT prompting method, which guides the LLMs to reach the correct answer with graph representation/verification steps. Specifically, we first leverage LLMs to construct a "question/rationale graph" by using knowledge extraction prompting given the initial question and the rationales generated in the previous steps. Then, the graph verification step diagnoses the current rationale triplet by comparing it with the existing question/rationale graph to filter out irrelevant rationales and generate follow-up questions to obtain relevant information. Additionally, we generate CoT paths that exclude the extracted graph information to represent the context information missed from the graph extraction. Our graph-guided reasoning method shows superior performance compared to previous CoT prompting and the variants on multi-hop question answering benchmark datasets.

</details>

<details>

<summary>摘要</summary>

chain-of-thought (CoT) 提示法已经提高了大语言模型 (LLM) 的多步逻辑能力，通过生成一系列的理由来提高答案。我们分析了 CoT 生成的逻辑路径，并发现了两个多步逻辑问题：（i）生成与问题无关的理由，（ii）无法组合子问题或查询来获取所有相关信息。为解决这些问题，我们提议一种图表引导 CoT 提示法，使 LLM 能够通过图表表示/验证步骤达到正确答案。 Specifically，我们首先利用 LLM 使用知识EXTRACTION 提示生成一个 "问题/理由图"，并then 使用图表验证步骤来诊断当前的理由 triplet，并将无关的理由过滤掉，生成跟进 вопро题以获取相关信息。此外，我们还生成 CoT 路径，排除扩展的图信息以表示Context missed 信息。我们的图表引导逻辑方法在多个多步问答 benchmark 数据集上表现出色。

</details></li>

</ul>

<hr>

<h2 id="MAFALDA-A-Benchmark-and-Comprehensive-Study-of-Fallacy-Detection-and-Classification"><a href="#MAFALDA-A-Benchmark-and-Comprehensive-Study-of-Fallacy-Detection-and-Classification" class="headerlink" title="MAFALDA: A Benchmark and Comprehensive Study of Fallacy Detection and Classification"></a>MAFALDA: A Benchmark and Comprehensive Study of Fallacy Detection and Classification</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09761">http://arxiv.org/abs/2311.09761</a></li>

<li>repo_url: None</li>

<li>paper_authors: Chadi Helwe, Tom Calamai, Pierre-Henri Paris, Chloé Clavel, Fabian Suchanek</li>

<li>For: The paper aims to address the challenges of automated fallacy detection and classification, particularly the subjectivity of the task and the need for a comprehensive and unified approach in existing research.* Methods: The paper introduces a novel taxonomy of fallacies that refines and aligns previous classifications, a new annotation scheme tailored for subjective NLP tasks, and a new evaluation method designed to handle subjectivity.* Results: The paper introduces MAFALDA (Multi-level Annotated FALlacy DAtaset), a gold standard dataset based on examples from various previously existing fallacy datasets under the unified taxonomy, and evaluates several language models under a zero-shot learning setting using MAFALDA to assess their fallacy detection and classification capability. The evaluation provides valuable insights into the strengths and limitations of these models in addressing fallacious reasoning.Here is the same information in Simplified Chinese text:* For: 这篇论文目标是解决自动推理错误检测和分类的挑战，特别是任务的主观性和现有研究中存在的缺乏一致性。* Methods: 论文引入了一种新的论点分类方法，该方法可以协调和融合先前的分类方法，同时还提供了一种适用于主观NLPTask的新注释方案。* Results: 论文引入了一个金标准数据集MAFALDA（多级注释错误数据集），该数据集基于先前的许多错误数据集的一致性，并对多种语言模型进行零基础学习 Setting中的评价。<details>

<summary>Abstract</summary>

Fallacies can be used to spread disinformation, fake news, and propaganda, underlining the importance of their detection. Automated detection and classification of fallacies, however, remain challenging, mainly because of the innate subjectivity of the task and the need for a comprehensive, unified approach in existing research. Addressing these limitations, our study introduces a novel taxonomy of fallacies that aligns and refines previous classifications, a new annotation scheme tailored for subjective NLP tasks, and a new evaluation method designed to handle subjectivity, adapted to precision, recall, and F1-Score metrics. Using our annotation scheme, the paper introduces MAFALDA (Multi-level Annotated FALlacy DAtaset), a gold standard dataset. MAFALDA is based on examples from various previously existing fallacy datasets under our unified taxonomy across three levels of granularity. We then evaluate several language models under a zero-shot learning setting using MAFALDA to assess their fallacy detection and classification capability. Our comprehensive evaluation not only benchmarks the performance of these models but also provides valuable insights into their strengths and limitations in addressing fallacious reasoning.

</details>

<details>

<summary>摘要</summary>

False information, fake news, and propaganda can be spread through fallacies, highlighting the importance of detecting them. However, automated detection and classification of fallacies are challenging due to the subjective nature of the task and the lack of a comprehensive, unified approach in existing research. To address these limitations, our study proposes a new taxonomy of fallacies that aligns and refines previous classifications, a new annotation scheme tailored for subjective NLP tasks, and a new evaluation method designed to handle subjectivity, adapted to precision, recall, and F1-Score metrics. Using our annotation scheme, we introduce MAFALDA (Multi-level Annotated FALlacy DAtaset), a gold standard dataset based on examples from various previously existing fallacy datasets under our unified taxonomy across three levels of granularity. We then evaluate several language models under a zero-shot learning setting using MAFALDA to assess their fallacy detection and classification capability. Our comprehensive evaluation not only benchmarks the performance of these models but also provides valuable insights into their strengths and limitations in addressing fallacious reasoning.

</details></li>

</ul>

<hr>

<h2 id="UFPS-A-unified-framework-for-partially-annotated-federated-segmentation-in-heterogeneous-data-distribution"><a href="#UFPS-A-unified-framework-for-partially-annotated-federated-segmentation-in-heterogeneous-data-distribution" class="headerlink" title="UFPS: A unified framework for partially-annotated federated segmentation in heterogeneous data distribution"></a>UFPS: A unified framework for partially-annotated federated segmentation in heterogeneous data distribution</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09757">http://arxiv.org/abs/2311.09757</a></li>

<li>repo_url: <a target="_blank" rel="noopener" href="https://github.com/tekap404/unified_federated_partially-labeled_segmentation">https://github.com/tekap404/unified_federated_partially-labeled_segmentation</a></li>

<li>paper_authors: Le Jiang, Li Yan Ma, Tie Yong Zeng, Shi Hui Ying</li>

<li>for: 这个研究是为了解决基于半自动化标签的医疗影像分类问题，并且不会遗漏数据的隐私问题。</li>

<li>methods: 这个研究使用了联邦式半自动化分类（FPSS），并且提出了一个统一的全球模型训练方法（UFPS），以解决半自动化分类中的分类异常和客户端漂移问题。</li>

<li>results: 这个研究的实验结果显示，UFPS方法能够更好地解决半自动化分类中的分类异常和客户端漂移问题，并且在实际医疗影像数据上显示了更好的适应和普遍性。<details>

<summary>Abstract</summary>

Partially supervised segmentation is a label-saving method based on datasets with fractional classes labeled and intersectant. However, it is still far from landing on real-world medical applications due to privacy concerns and data heterogeneity. As a remedy without privacy leakage, federated partially supervised segmentation (FPSS) is formulated in this work. The main challenges for FPSS are class heterogeneity and client drift. We propose a Unified Federated Partially-labeled Segmentation (UFPS) framework to segment pixels within all classes for partially-annotated datasets by training a totipotential global model without class collision. Our framework includes Unified Label Learning and sparsed Unified Sharpness Aware Minimization for unification of class and feature space, respectively. We find that vanilla combinations for traditional methods in partially supervised segmentation and federated learning are mainly hampered by class collision through empirical study. Our comprehensive experiments on real medical datasets demonstrate better deconflicting and generalization ability of UFPS compared with modified methods.

</details>

<details>

<summary>摘要</summary>

partially supervised segmentation是一种基于分类数据集的标签保存方法，但它还远离实际医疗应用中的应用，主要原因是隐私问题和数据不一致。为解决这些问题，本文提出了联邦半supervised分割（FPSS）方法。FPSS的主要挑战是分类异ogeneous和客户端漂移。我们提出了一种总统的联邦半supervised分割（UFPS）框架，用于对半标注数据集中的像素进行分割，不会出现分类冲突。我们的框架包括统一标签学习和粗粒化的统一锐度感知优化，用于统一类和特征空间。我们通过实际研究发现，传统的partially supervised segmentation和联邦学习方法的组合在面临分类冲突的情况下效果较差。我们的全面实验表明，UFPS在实际医疗数据集上具有更好的冲突解决和泛化能力，与修改后的方法相比。

</details></li>

</ul>

<hr>

<h2 id="Redefining-the-Laparoscopic-Spatial-Sense-AI-based-Intra-and-Postoperative-Measurement-from-Stereoimages"><a href="#Redefining-the-Laparoscopic-Spatial-Sense-AI-based-Intra-and-Postoperative-Measurement-from-Stereoimages" class="headerlink" title="Redefining the Laparoscopic Spatial Sense: AI-based Intra- and Postoperative Measurement from Stereoimages"></a>Redefining the Laparoscopic Spatial Sense: AI-based Intra- and Postoperative Measurement from Stereoimages</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09744">http://arxiv.org/abs/2311.09744</a></li>

<li>repo_url: <a target="_blank" rel="noopener" href="https://github.com/leopoldmueller/laparoscopicmeasurement">https://github.com/leopoldmueller/laparoscopicmeasurement</a></li>

<li>paper_authors: Leopold Müller, Patrick Hemmer, Moritz Queisner, Igor Sauer, Simeon Allmendinger, Johannes Jakubik, Michael Vössing, Niklas Kühl<br>for: This paper aims to provide a more accurate and efficient solution for image-guided surgery, specifically for measuring relevant structures such as vessel segments, resection margins, and bowel lengths.methods: The proposed method utilizes stereo vision and state-of-the-art machine learning architectures, including RAFT-Stereo and YOLOv8, to achieve high accuracy in distance measurements with errors below 1 mm.results: The developed method is assessed in various realistic experimental evaluation environments and demonstrates robustness in challenging environments with textureless regions. The results outline the potential of the method for providing more precise, safe, and efficient surgical procedures.<details>

<summary>Abstract</summary>

A significant challenge in image-guided surgery is the accurate measurement task of relevant structures such as vessel segments, resection margins, or bowel lengths. While this task is an essential component of many surgeries, it involves substantial human effort and is prone to inaccuracies. In this paper, we develop a novel human-AI-based method for laparoscopic measurements utilizing stereo vision that has been guided by practicing surgeons. Based on a holistic qualitative requirements analysis, this work proposes a comprehensive measurement method, which comprises state-of-the-art machine learning architectures, such as RAFT-Stereo and YOLOv8. The developed method is assessed in various realistic experimental evaluation environments. Our results outline the potential of our method achieving high accuracies in distance measurements with errors below 1 mm. Furthermore, on-surface measurements demonstrate robustness when applied in challenging environments with textureless regions. Overall, by addressing the inherent challenges of image-guided surgery, we lay the foundation for a more robust and accurate solution for intra- and postoperative measurements, enabling more precise, safe, and efficient surgical procedures.

</details>

<details>

<summary>摘要</summary>

significante挑战在图像引导手术中是准确测量有关结构，如血管段、切除边缘或肠长度。这项任务是许多手术中不可或缺的一部分，但它具有较大的人工劳动量和不准确率。在这篇论文中，我们开发了一种新的人类-AI基于方法，用于肠 Laparoscopic 测量，利用推导导航的STereo视力。基于全面的quality要求分析，这项工作提出了一种全面的测量方法，包括当前的机器学习架构，如RAFT-Stereo和YOLOv8。我们开发的方法在不同的实际试验环境中进行了评估。我们的结果表明，我们的方法可以实现高精度的距离测量，错误在1毫米以下。此外，在表面上进行的测量也能够在粗糙区域中展示 robustness。总之，我们通过解决图像引导手术中的内在挑战，为更加精确、安全和有效的手术过程奠定了基础。

</details></li>

</ul>

<hr>

<h2 id="Redefining-Super-Resolution-Fine-mesh-PDE-predictions-without-classical-simulations"><a href="#Redefining-Super-Resolution-Fine-mesh-PDE-predictions-without-classical-simulations" class="headerlink" title="Redefining Super-Resolution: Fine-mesh PDE predictions without classical simulations"></a>Redefining Super-Resolution: Fine-mesh PDE predictions without classical simulations</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09740">http://arxiv.org/abs/2311.09740</a></li>

<li>repo_url: None</li>

<li>paper_authors: Rajat Kumar Sarkar, Ritam Majumdar, Vishal Jadhav, Sagar Srinivas Sakhinana, Venkataramana Runkana</li>

<li>for: 用于提高计算流体力学（CFD）中粗粒度 simulations 的精度，并且可以避免传统的超分辨率方法中的约束。</li>

<li>methods: 我们提出了一种新的超分辨率定义，将 coarse-grid 数据作为输入，预测 fine-grid 数据。使用 physics-infused UNet 上升方法，并在多种2D-CFD问题中进行了证明，包括 Burger 方程中的缺陷检测、甲烷燃烧和工业热交换器中的沾吸。</li>

<li>results: 我们的方法可以生成精度高的 fine-mesh 解决方案，不需要传统的计算，同时保持了原始真实情况的精度。通过在训练过程中使用多种边界条件，我们还证明了我们的方法的稳定性，这将推动其广泛应用于工程和科学 CFD 解决方案中。<details>

<summary>Abstract</summary>

In Computational Fluid Dynamics (CFD), coarse mesh simulations offer computational efficiency but often lack precision. Applying conventional super-resolution to these simulations poses a significant challenge due to the fundamental contrast between downsampling high-resolution images and authentically emulating low-resolution physics. The former method conserves more of the underlying physics, surpassing the usual constraints of real-world scenarios. We propose a novel definition of super-resolution tailored for PDE-based problems. Instead of simply downsampling from a high-resolution dataset, we use coarse-grid simulated data as our input and predict fine-grid simulated outcomes. Employing a physics-infused UNet upscaling method, we demonstrate its efficacy across various 2D-CFD problems such as discontinuity detection in Burger's equation, Methane combustion, and fouling in Industrial heat exchangers. Our method enables the generation of fine-mesh solutions bypassing traditional simulation, ensuring considerable computational saving and fidelity to the original ground truth outcomes. Through diverse boundary conditions during training, we further establish the robustness of our method, paving the way for its broad applications in engineering and scientific CFD solvers.

</details>

<details>

<summary>摘要</summary></li>

</ul>

</details>

<hr>

<h2 id="Source-Prompt-Coordinated-Pre-training-of-Language-Models-on-Diverse-Corpora-from-Multiple-Sources"><a href="#Source-Prompt-Coordinated-Pre-training-of-Language-Models-on-Diverse-Corpora-from-Multiple-Sources" class="headerlink" title="Source Prompt: Coordinated Pre-training of Language Models on Diverse Corpora from Multiple Sources"></a>Source Prompt: Coordinated Pre-training of Language Models on Diverse Corpora from Multiple Sources</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09732">http://arxiv.org/abs/2311.09732</a></li>

<li>repo_url: None</li>

<li>paper_authors: Yipei Xu, Dakuan Lu, Jiaqing Liang, Xintao Wang, Yipeng Geng, Yingsi Xin, Hengkui Wu, Ken Chen, ruiji zhang, Yanghua Xiao</li>

<li>for: 这 paper 是研究 Pre-trained language models (PLMs) 的新 paradigm 在 NLP 领域的一部分，以及如何更好地准备这些模型和它们的预训练 corpora。</li>

<li>methods: 这 paper 使用了一种常见和成功的方法，即不断扩大模型和预训练 corpora 的大小。这些大 corpora 通常是从多个源 converges 而来的，因此变得越来越多样化。然而，这些巨大的 converged corpora 的侧effect 尚未得到了充分的研究。</li>

<li>results: 在这 paper 中， authors 发现了各种 corpora 的多样性会对预训练 PLMs 的性能产生负面影响。为了协调预训练在多种 corpora 上，authors 提出了源提示 (SP)，这是一种在预训练和精度调整阶段显式地提示模型数据源的技术。Results 表明，使用 SP 在多种 corpora 上预训练 PLMs 可以获得显著的下游任务提升。<details>

<summary>Abstract</summary>

Pre-trained language models (PLMs) have established the new paradigm in the field of NLP. For more powerful PLMs, one of the most popular and successful way is to continuously scale up sizes of the models and the pre-training corpora. These large corpora are generally obtained by converging smaller ones from multiple sources, they are thus growing increasingly diverse. However, the side-effects of these colossal converged corpora remain understudied. In this paper, we identify the disadvantage of heterogeneous corpora from multiple sources for pre-training PLMs. Towards coordinated pre-training on diverse corpora, we further propose source prompts (SP), which explicitly prompt the model of the data source at the pre-training and fine-tuning stages. Results of extensive experiments demonstrate that PLMs pre-trained with SP on diverse corpora gain significant improvement in various downstream tasks.

</details>

<details>

<summary>摘要</summary></li>

</ul>

</details>

<hr>

<h2 id="Prudent-Silence-or-Foolish-Babble-Examining-Large-Language-Models’-Responses-to-the-Unknown"><a href="#Prudent-Silence-or-Foolish-Babble-Examining-Large-Language-Models’-Responses-to-the-Unknown" class="headerlink" title="Prudent Silence or Foolish Babble? Examining Large Language Models’ Responses to the Unknown"></a>Prudent Silence or Foolish Babble? Examining Large Language Models’ Responses to the Unknown</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09731">http://arxiv.org/abs/2311.09731</a></li>

<li>repo_url: None</li>

<li>paper_authors: Genglin Liu, Xingyao Wang, Lifan Yuan, Yangyi Chen, Hao Peng</li>

<li>for: 本研究旨在系统地研究LLMs在缺乏必要知识的情况下的行为，以及这种行为如何与人类对话规范不符。</li>

<li>methods: 本研究使用了一个反 adversarial question-answering benchmark，该 benchmark 包含了 LLMS 缺乏训练数据的情况下的问题。</li>

<li>results: 研究发现，通过 instrucion finetuning 和人类反馈学习（RLHF），LLMs 可以更好地表达uncertainty，并且与有效的问题相对应，表现出更高的准确率和自信度。<details>

<summary>Abstract</summary>

Large Language Models (LLMs) often struggle when faced with situations where they lack the prerequisite knowledge to generate a sensical response. In these cases, models tend to fabricate and hallucinate, rather than appropriately signaling uncertainty as humans would. This behavior misaligns with human conversational norms and presents challenges surrounding responsible and ethical AI development. This work aims to systematically investigate LLMs' behaviors in such situations. We curate an adversarial question-answering benchmark containing unanswerable questions targeting information absent from the LLM's training data. Concretely, these unanswerable questions contain non-existent concepts or false premises. When presented with such unanswerable questions, an LLM should appropriately convey uncertainty, and be able to challenge the premise and refuse to generate a response. While facing answerable valid questions, a model should demonstrate a positive correlation between accuracy and confidence. Using a model-agnostic unified confidence elicitation approach, we observe that LLMs that have gone through instruction finetuning and reinforcement learning from human feedback (RLHF) perform significantly better than their counterparts that do not. Moreover, uncertainty expression 1 through our elicitation method does not always stay consistent with the perceived confidence of the direct response of an LLM. Our findings call for further research into teaching LLMs to proactively and reliably express uncertainty.

</details>

<details>

<summary>摘要</summary></li>

</ul>

</details>

<hr>

<h2 id="Aligning-with-Whom-Large-Language-Models-Have-Gender-and-Racial-Biases-in-Subjective-NLP-Tasks"><a href="#Aligning-with-Whom-Large-Language-Models-Have-Gender-and-Racial-Biases-in-Subjective-NLP-Tasks" class="headerlink" title="Aligning with Whom? Large Language Models Have Gender and Racial Biases in Subjective NLP Tasks"></a>Aligning with Whom? Large Language Models Have Gender and Racial Biases in Subjective NLP Tasks</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09730">http://arxiv.org/abs/2311.09730</a></li>

<li>repo_url: <a target="_blank" rel="noopener" href="https://github.com/jiaxin-pei/llm-group-bias">https://github.com/jiaxin-pei/llm-group-bias</a></li>

<li>paper_authors: Huaman Sun, Jiaxin Pei, Minje Choi, David Jurgens</li>

<li>for: 这个研究探讨了大语言模型（LLM）在主观NLP任务上是否具有性别和民族特征的偏见。</li>

<li>methods: 研究使用了POPQUORN数据集，对四种流行的LLM进行了一系列实验，检查它们在政eness和不礼貌任务上是否具有偏见。</li>

<li>results: 研究发现，对于这两个任务，模型的预测结果更接近白人和女性参与者的标签。进一步的探讨发现，在使用目标民族和性别标签作为提示时，模型的性能会下降。 Code和数据可以在<a target="_blank" rel="noopener" href="https://github.com/Jiaxin-Pei/LLM-Group-Bias%E4%B8%8A%E8%8E%B7%E5%8F%96%E3%80%82">https://github.com/Jiaxin-Pei/LLM-Group-Bias上获取。</a><details>

<summary>Abstract</summary>

Human perception of language depends on personal backgrounds like gender and ethnicity. While existing studies have shown that large language models (LLMs) hold values that are closer to certain societal groups, it is unclear whether their prediction behaviors on subjective NLP tasks also exhibit a similar bias. In this study, leveraging the POPQUORN dataset which contains annotations of diverse demographic backgrounds, we conduct a series of experiments on four popular LLMs to investigate their capability to understand group differences and potential biases in their predictions for politeness and offensiveness. We find that for both tasks, model predictions are closer to the labels from White and female participants. We further explore prompting with the target demographic labels and show that including the target demographic in the prompt actually worsens the model's performance. More specifically, when being prompted to respond from the perspective of "Black" and "Asian" individuals, models show lower performance in predicting both overall scores as well as the scores from corresponding groups. Our results suggest that LLMs hold gender and racial biases for subjective NLP tasks and that demographic-infused prompts alone may be insufficient to mitigate such effects. Code and data are available at https://github.com/Jiaxin-Pei/LLM-Group-Bias.

</details>

<details>

<summary>摘要</summary>

人类对语言的理解受个人背景的影响，如性别和民族。 although existing studies have shown that large language models (LLMs) hold values that are closer to certain societal groups, it is unclear whether their prediction behaviors on subjective NLP tasks also exhibit a similar bias. In this study, we leverage the POPQUORN dataset, which contains annotations of diverse demographic backgrounds, to investigate the capability of four popular LLMs to understand group differences and potential biases in their predictions for politeness and offensiveness. We find that for both tasks, model predictions are closer to the labels from White and female participants. We further explore prompting with the target demographic labels and show that including the target demographic in the prompt actually worsens the model's performance. Specifically, when prompted to respond from the perspective of "Black" and "Asian" individuals, models show lower performance in predicting both overall scores and the scores from corresponding groups. Our results suggest that LLMs hold gender and racial biases for subjective NLP tasks, and that demographic-infused prompts alone may be insufficient to mitigate such effects. 可以在 GitHub 上获取代码和数据：https://github.com/Jiaxin-Pei/LLM-Group-Bias。

</details></li>

</ul>

<hr>

<h2 id="Outcome-supervised-Verifiers-for-Planning-in-Mathematical-Reasoning"><a href="#Outcome-supervised-Verifiers-for-Planning-in-Mathematical-Reasoning" class="headerlink" title="Outcome-supervised Verifiers for Planning in Mathematical Reasoning"></a>Outcome-supervised Verifiers for Planning in Mathematical Reasoning</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09724">http://arxiv.org/abs/2311.09724</a></li>

<li>repo_url: None</li>

<li>paper_authors: Fei Yu, Anningzhe Gao, Benyou Wang<br>for: 这种研究旨在解决大型语言模型（LLM）在数学逻辑推理中维持准确性的问题， LLMs 经常在推理过程中出现错误，导致最终结果不准确。methods: 该研究提出了一种新的评估模型——结果监督价值模型（OVM），通过结果监督来训练，从而提高了模型的准确性。 OVM 不需要劳动密集的步骤级别正确性注释，从而提高了其可扩展性。results: 在两个多步数学逻辑数据集上进行了实验，结果显示 OVM 模型在 GSM8K 数据集上 achievement 状态的最佳结果，而不需要使用 GPT-4 或代码执行。 这些发现提供了一种新的视角，即在训练评估模型时，结果监督可以提高模型的价值估计，并且有理论基础。<details>

<summary>Abstract</summary>

Large language models (LLMs) often struggle with maintaining accuracy across a sequence of intermediate reasoning steps in mathematical reasoning, leading to error propagation that undermines the final result. The current methodology to mitigate this issue primarily involves using a verifier model to assess the correctness of generated solution candidates, focusing either on the overall reasoning path or on an incomplete reasoning path. By rethinking this approach, we argue that assessing potentials of incomplete reasoning paths could be more advantageous as it guides towards correct final answers, transforming the task into a \textit{planning} problem. Our proposed verifier, the Outcome-supervision Value Model (OVM), employs outcome supervision for training, offering an efficient and intuitive method for \textit{planning} by prioritizing steps that lead to accurate conclusions over mere per-step correctness. Furthermore, the OVM eschews the need for labor-intensive annotations on step-level correctness, enhancing its scalability. Our experiments on two multi-step mathematical reasoning datasets, GSM8K and Game of 24, demonstrate the superior performance of the OVM model. Notably, in GSM8K, our \textbf{OVM-7B model achieves state-of-the-art results among LLMs up to 13B parameters}; especially it does not utilize GPT-4 or code execution. These findings offer a novel perspective on the role of outcome supervision in training verifiers for multi-step reasoning tasks and provide theoretical justification for its advantage in value estimation for planning.

</details>

<details>

<summary>摘要</summary>

大型语言模型（LLM）经常在进行多步骤推理时维持正确性问题，导致错误传递，干扰最终结果。现有的方法来解决这问题主要是使用验证模型来评估生成的解答候选者，专注于全局推理路径或部分推理路径。我们认为评估未完成推理路径的潜力可能更有利，将任务转换为规划问题。我们提出的验证器是结果监督值模型（OVM），通过结果监督进行训练，提供一种高效和直观的规划方法，优先级是导向正确的结论的步骤。此外，OVM不需要耗费劳动的步骤正确性标注，提高其扩展性。我们在GSM8K和Game of 24两个多步骤推理数据集上进行实验，结果显示OVM模型在LLM中表现出色，特别是OVM-7B模型在GSM8K数据集上实现了状态顶尖结果，而不使用GPT-4或代码执行。这些结果提供一个新的见解，强调结果监督在训练验证器 для多步骤推理任务中的role，并提供了值估计的理论基础。

</details></li>

</ul>

<hr>

<h2 id="You-don’t-need-a-personality-test-to-know-these-models-are-unreliable-Assessing-the-Reliability-of-Large-Language-Models-on-Psychometric-Instruments"><a href="#You-don’t-need-a-personality-test-to-know-these-models-are-unreliable-Assessing-the-Reliability-of-Large-Language-Models-on-Psychometric-Instruments" class="headerlink" title="You don’t need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments"></a>You don’t need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09718">http://arxiv.org/abs/2311.09718</a></li>

<li>repo_url: <a target="_blank" rel="noopener" href="https://github.com/orange0629/llm-personas">https://github.com/orange0629/llm-personas</a></li>

<li>paper_authors: Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Dallas Card, David Jurgens</li>

<li>for: 本研究旨在检验当前问题提示的格式是否能够具备一致和可靠的回答能力。</li>

<li>methods: 研究者首先构建了39种人格测试工具的693个问题集，然后设计了一些细微变化的提示，以检验LLM的回答准确性和一致性。</li>

<li>results: 实验发现，即使使用简单的变换，LLM的回答能力也会受到很大的下降，而大多数LLM的否定一致性也很低。这些结果表明，当前的问题提示方式不够准确地捕捉模型的感知，并讨论了可能的更好的 alternatives。<details>

<summary>Abstract</summary>

The versatility of Large Language Models (LLMs) on natural language understanding tasks has made them popular for research in social sciences. In particular, to properly understand the properties and innate personas of LLMs, researchers have performed studies that involve using prompts in the form of questions that ask LLMs of particular opinions. In this study, we take a cautionary step back and examine whether the current format of prompting enables LLMs to provide responses in a consistent and robust manner. We first construct a dataset that contains 693 questions encompassing 39 different instruments of persona measurement on 115 persona axes. Additionally, we design a set of prompts containing minor variations and examine LLM's capabilities to generate accurate answers, as well as consistency variations to examine their consistency towards simple perturbations such as switching the option order. Our experiments on 15 different open-source LLMs reveal that even simple perturbations are sufficient to significantly downgrade a model's question-answering ability, and that most LLMs have low negation consistency. Our results suggest that the currently widespread practice of prompting is insufficient to accurately capture model perceptions, and we discuss potential alternatives to improve such issues.

</details>

<details>

<summary>摘要</summary>

大型自然语言模型（LLM）在自然语言理解任务上的多样性，使得它们在社会科学研究中受欢迎。特别是，为了准确理解LLM的性质和内在人格，研究人员已经使用了问题提示的形式进行研究。在这项研究中，我们做出了一个谨慎的步骤，检查当前的提示格式是否能够让LLM提供一致和稳定的回答。我们首先构建了一个包含39种测量工具和115个人格轴的数据集。此外，我们设计了一些具有轻微变化的提示，检查LLM是否能够生成准确回答，以及对简单的变化（如选项顺序交换）的一致性。我们在15种开源LLM上进行了实验，发现，即使使用简单的变化，也可以导致模型的问题回答能力下降 significatively，并且大多数LLM具有低的否定一致性。我们的结果表明，当前广泛使用的提示方式不够用于准确捕捉模型的感知，我们讨论了可能的改进方案。

</details></li>

</ul>

<hr>

<h2 id="Towards-Autonomous-Hypothesis-Verification-via-Language-Models-with-Minimal-Guidance"><a href="#Towards-Autonomous-Hypothesis-Verification-via-Language-Models-with-Minimal-Guidance" class="headerlink" title="Towards Autonomous Hypothesis Verification via Language Models with Minimal Guidance"></a>Towards Autonomous Hypothesis Verification via Language Models with Minimal Guidance</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09706">http://arxiv.org/abs/2311.09706</a></li>

<li>repo_url: None</li>

<li>paper_authors: Shiro Takagi, Ryutaro Yamauchi, Wataru Kumagai</li>

<li>for: 本研究旨在检验AI是否可以自主生成和验证假设，以实现人类水平的自主研究。</li>

<li>methods: 本研究使用GPT-4生成假设和Python代码进行验证，仅提供有限的方法指导。</li>

<li>results: 研究发现，在某些情况下，GPT-4可以自主生成和验证假设，但无一个完美的验证结果，表明还有许多挑战需要继续探索，以实现自主研究。<details>

<summary>Abstract</summary>

Research automation efforts usually employ AI as a tool to automate specific tasks within the research process. To create an AI that truly conduct research themselves, it must independently generate hypotheses, design verification plans, and execute verification. Therefore, we investigated if an AI itself could autonomously generate and verify hypothesis for a toy machine learning research problem. We prompted GPT-4 to generate hypotheses and Python code for hypothesis verification with limited methodological guidance. Our findings suggest that, in some instances, GPT-4 can autonomously generate and validate hypotheses without detailed guidance. While this is a promising result, we also found that none of the verifications were flawless, and there remain significant challenges in achieving autonomous, human-level research using only generic instructions. These findings underscore the need for continued exploration to develop a general and autonomous AI researcher.

</details>

<details>

<summary>摘要</summary></li>

</ul>

</details>

<hr>

<h2 id="Deceiving-Semantic-Shortcuts-on-Reasoning-Chains-How-Far-Can-Models-Go-without-Hallucination"><a href="#Deceiving-Semantic-Shortcuts-on-Reasoning-Chains-How-Far-Can-Models-Go-without-Hallucination" class="headerlink" title="Deceiving Semantic Shortcuts on Reasoning Chains: How Far Can Models Go without Hallucination?"></a>Deceiving Semantic Shortcuts on Reasoning Chains: How Far Can Models Go without Hallucination?</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09702">http://arxiv.org/abs/2311.09702</a></li>

<li>repo_url: None</li>

<li>paper_authors: Bangzheng Li, Ben Zhou, Fei Wang, Xingyu Fu, Dan Roth, Muhao Chen</li>

<li>for: 这个研究探讨了大型语言模型（LLM）在句子理解中存在的幻觉和不当思维现象，以及这些现象如何影响模型的表现。</li>

<li>methods: 该研究使用了一种新的探测方法和benchmark，称为EureQA，以量化这种现象。该方法从问题中找到可以得到答案的关键Entity，然后逐层添加证据句，让模型在找到答案之前必须按照链式的证据逻辑进行搜寻。</li>

<li>results: 研究发现，现有的LLM模型缺乏能够按照正确的逻辑链进行搜寻和理解句子的能力，而是倾向于通过幻觉和不当思维来简化问题。这些幻觉通常是基于语义关系的，并且会导致模型的偏差和幻觉。<details>

<summary>Abstract</summary>

Despite the recent advancement in large language models (LLMs) and their high performances across numerous benchmarks, recent research has unveiled that LLMs suffer from hallucinations and unfaithful reasoning. This work studies a specific type of hallucination induced by semantic associations. Specifically, we investigate to what extent LLMs take shortcuts from certain keyword/entity biases in the prompt instead of following the correct reasoning path. To quantify this phenomenon, we propose a novel probing method and benchmark called EureQA. We start from questions that LLMs will answer correctly with utmost certainty, and mask the important entity with evidence sentence recursively, asking models to find masked entities according to a chain of evidence before answering the question. During the construction of the evidence, we purposefully replace semantic clues (entities) that may lead to the correct answer with distractor clues (evidence) that will not directly lead to the correct answer but require a chain-like reasoning process. We evaluate if models can follow the correct reasoning chain instead of short-cutting through distractor clues. We find that existing LLMs lack the necessary capabilities to follow correct reasoning paths and resist the attempt of greedy shortcuts. We show that the distractor semantic associations often lead to model hallucination, which is strong evidence that questions the validity of current LLM reasoning.

</details>

<details>

<summary>摘要</summary>

尽管最近的大语言模型（LLM）在许多测试中表现出色，但最新的研究表明这些模型受到了幻觉和不正确的理解的影响。这项研究探讨了 LLM 受到 semantic association 引起的幻觉的特点。我们具体研究 LLM 是否因为提示中的关键词/实体偏见而快速缩短 reasoning 过程。为了衡量这种现象，我们提出了一种新的探测方法和标准 benchmark called EureQA。我们从可以通过 utmost certainty 回答 вопро题的问题开始，并在提示中逐层隐藏关键实体，要求模型通过证据链来找到隐藏的实体。在建构证据时，我们故意将可能导致正确答案的 semantic clue 替换为不直接导致正确答案的 distractor clue，以便要求模型遵循 chain-like 的 reasoning 过程。我们评估模型是否可以遵循正确的 reasoning 路径，而不是短ircuit 通过 distractor clue。我们发现现有的 LLM 缺乏遵循正确 reasoning 路径的能力，并且很容易受到幻觉的影响。我们显示，distractor semantic association frequently leads to model hallucination，这是强有力的证据，证明了现有 LLM 的reasoning 无效。

</details></li>

</ul>

<hr>

<h2 id="Accommodating-Missing-Modalities-in-Time-Continuous-Multimodal-Emotion-Recognition"><a href="#Accommodating-Missing-Modalities-in-Time-Continuous-Multimodal-Emotion-Recognition" class="headerlink" title="Accommodating Missing Modalities in Time-Continuous Multimodal Emotion Recognition"></a>Accommodating Missing Modalities in Time-Continuous Multimodal Emotion Recognition</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.10119">http://arxiv.org/abs/2311.10119</a></li>

<li>repo_url: None</li>

<li>paper_authors: Juan Vazquez-Rodriguez, Grégoire Lefebvre, Julien Cumin, James L. Crowley</li>

<li>for: 本研究旨在提高时间连续的情感识别精度，即使有一些模式缺失。</li>

<li>methods: 我们提议一种基于Transformer的新架构，通过跨模式关注和自关注机制来强调时间上的关系，以提高学习过程的稳定性和精度。</li>

<li>results: 实验结果表明，我们的模型在 Ulm-TSST 数据集上的评价协调系数评价提高了37% （对应于情感值预测）和30% （对应于情感评价），相比基线方法。<details>

<summary>Abstract</summary>

Decades of research indicate that emotion recognition is more effective when drawing information from multiple modalities. But what if some modalities are sometimes missing? To address this problem, we propose a novel Transformer-based architecture for recognizing valence and arousal in a time-continuous manner even with missing input modalities. We use a coupling of cross-attention and self-attention mechanisms to emphasize relationships between modalities during time and enhance the learning process on weak salient inputs. Experimental results on the Ulm-TSST dataset show that our model exhibits an improvement of the concordance correlation coefficient evaluation of 37% when predicting arousal values and 30% when predicting valence values, compared to a late-fusion baseline approach.

</details>

<details>

<summary>摘要</summary>

以下是文本的简化中文翻译：多种研究表明，情感识别更有效率地使用多种modalities。但如果某些modalities缺失呢？为解决这个问题，我们提出了一种基于Transformer架构的时间连续的情感识别模型，即使缺失输入modalities也能够准确地识别情感。我们通过跨modalities和自modalities的相互关注机制来强调时间和模式之间的关系，从而提高模型在弱烈输入上学习的能力。实验结果表明，我们的模型在 Ulm-TSST 数据集上的评价方法比基准方法提高了37% （对于情绪值预测）和30% （对于情感值预测）。

</details></li>

</ul>

<hr>

<h2 id="BLT-Can-Large-Language-Models-Handle-Basic-Legal-Text"><a href="#BLT-Can-Large-Language-Models-Handle-Basic-Legal-Text" class="headerlink" title="BLT: Can Large Language Models Handle Basic Legal Text?"></a>BLT: Can Large Language Models Handle Basic Legal Text?</h2><ul>

<li>paper_url: <a target="_blank" rel="noopener" href="http://arxiv.org/abs/2311.09693">http://arxiv.org/abs/2311.09693</a></li>

<li>repo_url: <a target="_blank" rel="noopener" href="https://github.com/blairstanek/blt">https://github.com/blairstanek/blt</a></li>

<li>paper_authors: Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme</li>

<li>for: 法律执业中需要基础文本处理能力，现有的公共可用LLM like GPT-4和PaLM2表现不佳。</li>

<li>methods: 我们引入了一个 benchmark 来衡量这种 poor performance，这casts 疑问 LLMS 目前是否可靠 для legal practice。</li>

<li>results: fine-tuning older LLM 可以带来 near-perfect performance 在我们的测试集上，也提高了相关的法律任务的表现。这个结果强调了 LLMS 训练中需要更多的领域专业知识。<details>

<summary>Abstract</summary>

We find that the best publicly available LLMs like GPT-4 and PaLM 2 currently perform poorly at basic text handling required of lawyers or paralegals, such as looking up the text at a line of a witness deposition or at a subsection of a contract. We introduce a benchmark to quantify this poor performance, which casts into doubt LLMs' current reliability as-is for legal practice. Finetuning for these tasks brings an older LLM to near-perfect performance on our test set and also raises performance on a related legal task. This stark result highlights the need for more domain expertise in LLM training.

[View remainder of file in raw view](https://github.com/nullscc/nullscc.github.io/raw/b54b57dfcce90728078dfc6ff9e1e8e078ef1a63/2023/11/16/cs.AI_2023_11_16/index.html)

## Footer

[ ](https://github.com "GitHub") (C) 2025 GitHub, Inc. 

### Footer navigation

  * [Terms](https://docs.github.com/site-policy/github-terms/github-terms-of-service)
  * [Privacy](https://docs.github.com/site-policy/privacy-policies/github-privacy-statement)
  * [Security](https://github.com/security)
  * [Status](https://www.githubstatus.com/)
  * [Docs](https://docs.github.com/)
  * [Contact](https://support.github.com?tags=dotcom-footer)
  * Manage cookies 
  * Do not share my personal information 



You can’t perform that action at this time. 


