本文介绍了GitHub Models，一个为开源项目解决AI推理瓶颈的解决方案。传统的AI项目常常因需要用户自行配置付费API密钥或本地部署大型语言模型（LLM）而阻碍了用户和贡献者的采用。GitHub Models提供了一个免费、OpenAI兼容的推理API，用户无需新密钥或SDK即可使用。文章详细阐述了如何将GitHub Models集成到项目中，包括在CI/CD环境中的应用，以及如何通过付费选项扩展吞吐量和上下文窗口。通过消除AI集成障碍，GitHub Models旨在加速开源AI的普及和发展。

💰 **免费且易于集成的AI推理**：GitHub Models提供了一个与OpenAI兼容的免费推理API，消除了开源项目在集成AI功能时常见的障碍，如要求用户提供付费API密钥或自行部署大型语言模型。这使得开发者可以轻松地将AI能力添加到他们的项目中，并降低了用户和贡献者的入门门槛。

🚀 **无缝集成与广泛兼容性**：由于其OpenAI兼容的RESTful API设计，GitHub Models可以与现有的OpenAI SDK（如OpenAI-JS、OpenAI Python、LangChain等）以及其他LLM工具无缝集成，无需修改现有代码。这极大地简化了AI功能的引入和应用。

⚙️ **CI/CD的零配置体验**：对于在GitHub Actions中运行的项目，GitHub Models允许通过简单的`models:read`权限配置，使用内置的`GITHUB_TOKEN`进行推理调用，无需用户额外配置任何Secrets。这使得构建一键安装的AI驱动型GitHub Actions成为可能，例如自动化代码审查、问题标签处理等。

📈 **可扩展的付费选项**：除了免费额度，GitHub Models还提供付费层级，解锁更高的请求速率限制（RPM）、更大的上下文窗口（最高144k token）以及更低的延迟，以满足项目增长和大规模应用的需求。用户可以通过组织或企业设置启用付费使用。

🤝 **推动开源AI普及**：通过提供免费、便捷的AI推理能力，GitHub Models旨在降低开源AI项目的采用门槛，吸引更多用户和贡献者参与，从而加速AI技术在开源社区的普及和创新。

AI features can make an open source project shine. At least, until setup asks for a paid inference API key.  Requiring contributors or even casual users to bring their own large language model (LLM) key stops adoption in its tracks:

$ my-cool-ai-toolError: OPENAI_API_KEY not found

Developers may not want to buy a paid plan just to try out your tool, and self hosting a model can be too heavy for laptops or GitHub Actions runners. 

GitHub Models solves that friction with a free, OpenAI-compatible inference API that every GitHub account can use with no new keys, consoles, or SDKs required. In this article, we’ll show you how to drop it into your project, run it in CI/CD, and scale when your community takes off.

Let’s jump in.

The hidden cost of “just add AI”

AI features feel ubiquitous today, but getting them running locally is still a challenge for a few reasons:

Paid APIs: The simplest path is to ask users for an OpenAI or Anthropic key. That’s a non-starter for many hobbyists and students because paid APIs are too expensive.Local models: Running a 2 B-parameter LLM can work for lightweight tasks, but anything that requires more intelligence will quickly blow past typical laptop memory — let alone the 14 GB container that backs a GitHub Actions runner.Docker images and weights: You can bundle a model with your app, but distributing multi-gigabyte weights balloons install size and slows CI.

Every additional requirement filters out potential users and contributors. What you need is an inference endpoint that’s:

Free for public projectsCompatible with existing OpenAI SDKsAvailable wherever your code runs, like your laptop, server, or Actions runner

That’s what GitHub Models provides.

GitHub Models in a nutshell

What it is: A REST endpoint that speaks the chat/completions spec you already know.What you get: A curated set of models (GPT-4o, DeepSeek-R1, Llama 3, and more) hosted by GitHub.Who can call it: Anyone with a GitHub Personal Access Token (PAT), or a repository’s built-in GITHUB_TOKEN when you opt-in via permissions.How much it costs: Free tier for all personal accounts and OSS orgs; metered paid tier unlocks higher throughput and larger context windows.

Because the API mirrors OpenAI’s, any client that accepts a baseURL will work without code changes. This includes OpenAI-JS, OpenAI Python, LangChain, llamacpp, or your own curl script.

How to get started with GitHub Models

Since GitHub Models is compatible with the OpenAI chat/completions API, almost every inference SDK can use it. To get started, you can use the OpenAI SDK:

import OpenAI from "openai";const openai = new OpenAI({  baseURL: "https://models.github.ai/inference/chat/completions",  apiKey: process.env.GITHUB_TOKEN  // or any PAT with models:read});const res = await openai.chat.completions.create({  model: "openai/gpt-4o",  messages: [{ role: "user", content: "Hi!" }]});console.log(res.choices[0].message.content);

If you write your AI open source software with GitHub Models as an inference provider, all GitHub users will be able to get up and running with it just by supplying a GitHub Personal Access Token (PAT).

And if your software runs in GitHub Actions, your users won’t even need to supply a PAT. By requesting the models: read permission in your workflow file, the built-in GitHub token will have permissions to make inference requests to GitHub Models. This means you can build a whole array of AI-powered Actions that can be shared and installed with a single click. For instance:

Code review or PR triage botsSmart issue tagging workflowsWeekly repository activity report generatorsAnd anything else that a GitHub Action can do

Plus, using GitHub Models makes it easy for your users to set up AI inference. And that has another positive effect: it’s easier for your contributors to set up AI inference as well. When anyone with a GitHub account can run your code end to end, you’ll be able to get contributions from the whole range of GitHub users, not just the ones with an OpenAI key.

Zero-configuration CI with GitHub Actions

Publishing an Action that relies on AI used to require users to add their inference API key as a GitHub Actions secret. Now you can ship a one-click install:

yaml # .github/workflows/triage.ymlpermissions:  contents: read  issues: write  models: read   # 👈 unlocks GitHub Models for the GITHUB_TOKENjobs:  triage:    runs-on: ubuntu-latest    steps:      - uses: actions/checkout@v4      - name: Smart issue triage        run: node scripts/triage.js

The runner’s GITHUB_TOKEN carries the models:read scope, so your Action can call any model without extra setup. This makes it well suited for:

Automated pull request summariesIssue deduplication and taggingWeekly repository digestsAnything else you can script in an Action

Scaling when your project takes off

The GitHub Models inference API is free for everyone. But if you or your users want to do more inference than the free rate limits allow, you can turn on paid inference in your settings for significantly larger context windows and higher requests-per-minute. 

When your community grows, so will traffic. So it’s important to consider the following: 

Requests per minute (RPM): While the free tier offers default limits, the paid tier offers multiples higher.Context window: Free tier tops out at standard model limits; paid enables 128k tokens on supported models.Latency: The paid tier runs in its own separate deployment, so you’re not in the same queue as free tier users.

To get started, you can enable paid usage in Settings > Models for your org or enterprise. Your existing clients and tokens will keep working (but they’ll be faster and support bigger contexts).

Take this with you

LLMs are transforming how developers build and ship software, but requiring users to supply their own paid API key can be a barrier to entry. The magic only happens when the first npm install, cargo run, or go test just works.

If you maintain an AI-powered open source codebase, you should consider adding GitHub Models as a default inference provider. Your users already have free AI inference via GitHub, so there’s little downside to letting them use it with your code. That’s doubly true if your project is able to run in GitHub Actions. The best API key is no API key!

By making high-quality inference a free default for every developer on GitHub, GitHub Models gets rid of the biggest blocker to OSS AI adoption. And that opens the door to more contributions, faster onboarding, and happier users.

Want to give it a try? Check out the GitHub Models documentation or jump straight into the API reference and start shipping AI features that just work today.

The post Solving the inference problem for open source AI projects with GitHub Models appeared first on The GitHub Blog.