🌐Anthropic 为部分研究人员提供有帮助的访问权限及免费 API 访问，还为一些研究者提供 API 信用额度，并为 Ryan Greenblatt 提供深度模型访问，但除 Ryan 外无其他精细调整或深度访问，且提供外部指导

💻Google DeepMind 分享部分模型评估用于危险能力研究，发布 Gemma SAE 和权重、Embeddings API，提供外部指导，但对前沿模型无精细调整或深度访问

🤖OpenAI 为一些 OP 受助人提供更好的 API 访问，对 GPT - 3.5 进行精细调整，实验性地为学术研究者提供 GPT - 4 精细调整访问，提供早期访问及 API 相关功能

Published on July 31, 2024 7:30 PM GMT

Frontier AI labs can boost external safety researchers by

Sharing better access to powerful models (early access, fine-tuning, helpful-only,^[1] filters/moderation-off, logprobs, activations)^[2]Releasing research artifacts besides modelsPublishing (transparent, reproducible) safety researchGiving API creditsMentoring

Here's what the labs have done. Let me know if anything is missing/wrong.

Anthropic:

Giving helpful-only access to some researchers developing evalsGiving free API access to some OP grantees and giving some researchers $1K in API credits(Giving deep model access to Ryan Greenblatt)(External mentoring, in particular via MATS)[No fine-tuning or deep access, except for Ryan]

Google DeepMind:

Sharing some of their model evals for dangerous capabilitiesReleasing Gemma SAEsReleasing Gemma weightsEmbeddings API(External mentoring, in particular via MATS)(Grants programs — details private)[No fine-tuning or deep access to frontier models]

OpenAI:

Maybe giving better API access to some OP granteesFine-tuning GPT-3.5 (and "GPT-4 fine-tuning is in experimental access"; OpenAI shared GPT-4 fine-tuning access with academic researchers including Jacob Steinhardt and Daniel Kang in 2023)Early access: shared GPT-4 with a few safety researchers including Rachel Freedman before releaseAPI gives top 5 logprobsEmbeddings API

Meta AI:

Releasing Llama weights

Microsoft:

[Nothing]

Somewhat-related papers:

Structured access for third-party research on frontier AI models (Bucknall and Trager 2024)Black-Box Access is Insufficient for Rigorous AI Audits (Casper et al. 2024)
(The paper is about audits, like for risk assessment and oversight; this post is about research)
A Safe Harbor for AI Evaluation and Red Teaming (Longpre et al. 2024)

1. ^{^}

   "Helpful-only" refers to the version of the model RLHFed/RLAIFed/finetuned/whatever for helpfulness but not harmlessness.

2. ^{^}

   Releasing model weights will likely be dangerous once models are more powerful, but all past releases seem fine, but e.g. Meta's poor risk assessment and lack of a plan to make release decisions conditional on risk assessment is concerning.