About Me

I'm Andy Zhou and I'm the founder of Intology, a lab studying the science of intelligence. Check out our work on Zochi, the world's first Artificial Scientist with peer-reviewed publications.

I previously founded Lapis Labs, a student-led research group. We published more than 20 papers in top AI conferences and collaborated with many groups, including DeepMind, the Center for AI Safety, and IBM Research.

Email  /  GitHub  /  Google Scholar  /  Twitter

Research

My research focuses on advancing artificial intelligence with an emphasis on:

• AI agents and autonomous systems
• AI safety and alignment
• Scientific discovery through AI

I'm particularly interested in developing AI systems that can reason, learn, and discover in ways that augment human capabilities and advance scientific knowledge. Check out my papers here.

News

Selected Research

Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks

We propose a defense objective for defending LLMs against jailbreaking and an algorithm to generate trigger tokens that enforce harmless behavior, improving robustness across jailbreaks and models.

AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policies

We present a safety benchmark based on risk categories from regulations and policies with 5,694 diverse prompts based on 16 company policies and 8 government policies mapped into 314 risk categories.

KnowGraph: Knowledge-Enabled Anomaly Detection via Logical Reasoning on Graph Data

We propose a logical reasoning framework for anomaly detection on graph data that uses domain knowledge to organize the predictions of a collection of specialized models.

Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models

We propose the first search algorithm for LM agents which draws upon aspects of reasoning and acting prompting methods to improve decision-making. We achieve SOTA on HumanEval with a Pass@1 rate of 94.4%

Distilling Out-of-Distribution Robustness from Vision-Language Foundation Models

We propose a data augmentation and knowledge distillation objective that uses teacher gradients to generate diverse samples, improving out-of-distribution robustness. We distill from CLIP to train the most robust ResNet34 and ResNet50 on OOD generalization.

Research

RedCode: Risky Code Execution and Generation Benchmark for Code Agents

We propose a benchmark for evaluating the safety and reliability of code agents on executing and generating malicious code.

Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characters

We propose an attack on moderation guardrails that use cipher characters to detect harmful content and a benchmark for evaluating LLM guardrails.

Tamper-Resistant Safeguards for Open-Weight LLMs

We present a method for improving safeguard robustness against finetuning attacks.

AI Risk Categorization Decoded: From Corporate Policies to Government Regulations

We present a comprehensive AI risk taxonomy derived from eight government policies from the European Union, United States, and China and 16 company policies worldwide, making a significant step towards establishing a unified language for generative AI safety evaluation.

Towards Robust Unlearning in LLMs

We outline the setting of robust machine unlearning in LLMs for reliably removing unwanted knowledge.

FedSelect: Personalized Federated Learning with Customized Selection of Parameters for Fine-Tuning

We propose a federated-learning algorithm based on selecting which parameters to use for fine-tuning and which to make global updates.

GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Models

We propose a framework to generate semantic jailbreaks from human safety guidelines using syntatic parsing organized into knowledge graphs and LM optimization. Jailbreaks are SOTA for success rate and work on VLMS.

YouTubePD: A Multimodal Benchmark for Parkinson’s Disease Analysis

We propose the first public benchmark for automated Parkinson’s disease analysis. We explore three tasks–facial-expression-based PD classification, multimodal PD classification, and PD progression synthesis–and show models trained on YouTubePD generalize to real clinical data.

FedSelect: Customized Selection of Parameters for Fine-Tuning during Personalized Federated Learning

We propose a federated-learning algorithm based on selecting which parameters to use for fine-tuning and which to make global updates.

A Sentence Speaks a Thousand Images: Domain Generalization through Distilling CLIP with Language Guidance

We propose a distillation objective based on CLIP text representations to improve domain generalization.

Misc

Design and source code from Jon Barron's website