The Remarkable Robustness of LLMs: Stages of Inference?

We find that deleting and swapping interventions retain 72-95% of the original model’s prediction accuracy without fine-tuning, and hypothesize the existence of four universal stages of inference across eight different models.

<span title='2024-06-27 00:00:00 +0000 UTC'>June 2024</span>&nbsp;&middot;&nbsp;Vedang Lad, Wes Gurnee, Max Tegmark

Estimating label quality and errors in semantic segmentation data via any model

The soft-minimum of the model-estimated likelihoods of each pixel’s annotated class – that is particularly effective to identify images that are mislabeled, across multiple types of annotation error

<span title='2023-07-11 00:00:00 +0000 UTC'>July 2023</span>&nbsp;&middot;&nbsp;Vedang Lad, Jonas Mueller

CleaNLP: Detecting Label Errors in General NLP Tasks

This paper presents a method for identifying label errors in natural language processing (NLP) datasets using the T5 model.

<span title='2022-12-11 00:00:00 +0000 UTC'>December 2022</span>&nbsp;&middot;&nbsp;Vedang Lad, Ryan Wilson, Alex Wang

Building an Efficient Poker Agent Using RL

In this paper, we apply variations of Deep Q-learning (DQN) and Proximal Policy Optimization (PPO) to learn the game of heads-up no-limit Texas Hold’em.

<span title='2022-05-11 00:00:00 +0000 UTC'>May 2022</span>&nbsp;&middot;&nbsp;Alex Kashi, Vedang Lad, Hakon Grini

GRUNet: A Novel Bi-directional RNN for VQA

We propose a new framework which we call GRUNet. GRUNet is a novel Bi-Directional RNN architecture that combines GRU + RNN + ResNet to effectively combine text and im- age input to answer VQA questions.

<span title='2021-05-11 00:00:00 +0000 UTC'>May 2021</span>&nbsp;&middot;&nbsp;Vedang Lad, Lowell Hensgen