API Auth Is Deeper Than It Looks

If you want to integrate your product (or your AI agent) with other products, the first thing you need to do is authenticate with the external API.

We implemented auth for over 400 APIs.

We thought it would take a few weeks. It took three years.

Why? Not because the problem is inherently complicated, but because it is deceptively deep.

Here is what we learned.

OAuth: Not a Protocol

Famously, OAuth is not really a protocol.

We wrote a whole blog post on this issue: Why is OAuth still so hard?

Quick recap:

• The OAuth standard is large and sprawling, but it leaves many implementation details up to each API.
• As a result, every API has non-standard extensions, parameters, custom scope separators, and other quirks. If you're lucky, they are at least documented.
• Error messages are theoretically standardized, but they are rarely useful. invalid_grant is often the only clue you get. Debugging OAuth can feel like finding a needle in a haystack.

There are many OAuth libraries available, but most are designed for login flows, not API access. These are very different use cases.

API Keys: Great for You, Terrible for Users

API keys seem simple, until a non-technical user needs to paste one into your app.

That is when the real pain begins:

• Users cannot find their keys. You need to create detailed guides with screenshots and maintain them regularly.
• Many systems assign scopes or permissions to API keys, which adds another element to explain, document, and validate.
• Validation is necessary. Users often enter incorrect keys, miss scopes, or copy extra whitespace. It is not difficult to build validation, but it adds another flow you must design, debug, and support (and hope your users understand the error messages).
• Some systems expire API keys, for example, when the user who created them is deleted. This can be hard to detect unless you scan API responses and build logic to identify broken keys.

APIs vary widely in how they expect API keys to be passed. Basic Auth and the Authorization header are common, but we have also seen custom headers, query parameters, and even requests to pass them in the body of POST calls.

Some APIs require both an API key and OAuth credentials in every request. It gets messy fast.

Custom Auth Flows (GitHub App, etc.)

OAuth has its flaws, so some teams built their own alternatives.

This list includesGitHub, Stripe, Bill.com, Sage, Jazz, Shopify, Tableau, and many others.

These flows often resemble OAuth in structure. You ask the user for secrets, exchange them for an access token, and sometimes get a refresh token too. Each varies slightly, so you will end up with many conditional blocks scattered across your codebase.

Platforms like Slack and Shopify also require support for flows triggered by an "Install" button in their app stores. These installations often send a webhook to a pre-defined endpoint on your side. Sounds simple, until you need to assign this "random" webhook to one of your user's accounts.

Surprisingly, custom auth flows are becoming more common. GitHub and Stripe introduced theirs within the past three years.

One Product, Many APIs

You might assume one product offers one API.

Not quite. Shopify offers three APIs for different use cases, all with different auth flows.

GitHub supports at least four authentication methods, each designed for a specific purpose.

Sometimes, only specific endpoints support certain auth methods.

Token Refreshes

Most OAuth 2 APIs now issue short-lived access tokens.

Alongside the access token, you usually receive a refresh token. This lets you renew the access token once it expires. It improves security and minimizes the damage of token leakage.

However, this also increases the complexity of your implementation:

• Race conditions. Many APIs revoke the current access token as soon as a refresh starts. If any requests go out during this time, they will fail.
• Token rotation. Some APIs return a new refresh token during each refresh, while others do not. You need to store the latest version, or you will lose access.
• Expiration ambiguity. Some APIs (like Salesforce) do not provide the access token’s expiration time. You must query a separate endpoint to get it.
• Refresh windows. Some APIs revoke refresh tokens that are unused for a period of time. You should run a periodic job to keep them fresh. Be mindful of rate limits and concurrent requests when doing this.

Revoked Credentials and Re-Auth

Access tokens, refresh tokens, and API keys can all be revoked.

APIs rarely explain why a token was revoked. Once it is revoked, there is no way to recover it. The user must go through the original authentication flow again.

Detecting revoked credentials can be simple or very complex:

• For short-lived access tokens, a refresh failure usually provides a clear signal.
• For API keys or long-lived access tokens, detection is trickier. Few APIs offer a validation endpoint. Your best option is to scan responses for 401 and 403 status codes. These codes are often overloaded, so you will need a classification system to interpret them correctly.

Re-authentication happens often. Make sure the experience is smooth and clear for your users.

Additional Required Parameters

Account ID, realmId, team ID, datacenter, hosting region, subdomain—the names vary, but many APIs require additional context besides the access token to make authenticated requests.

How you obtain these parameters varies across APIs:

• Some platforms (like Intercom or Datadog) let users choose their hosting region. These regions may host separate API instances. You will need to ask users for this info during the auth flow.
• Some products use customer-specific subdomains. Zendesk makes this easier by including the subdomain with the access token. Others, like Shopify, require the subdomain as part of the initial auth request.
• Jira (or more accurately, Atlassian) is particularly confusing. The subdomain is optional during the auth flow, but if you omit it, Atlassian will randomly pick one of the user’s subdomains, without giving the user control.
• Some required parameters are passed once during the OAuth callback and cannot be retrieved later.
• Others require you to make follow-up API calls after the auth flow to discover which accounts, projects, or teams were authorized.

We recommend using a plug-in-style function that runs immediately after auth succeeds to gather any required information.

Also, centralize the logic that builds requests to external APIs. That way, you can insert required parameters in one place.

Permissions and Scopes

Scopes and permissions are complicated enough for their own article.

Here are a few important points:

• Always store the scopes that were granted alongside the access token. Be careful to store what was actually granted, not just what you requested.
• If your app's scopes change, users must re-authenticate. Track who has done so.
• Some APIs have different types of scopes. For example, Slack has bot-level and user-level scopes, both granted through the same flow.
• API keys can also have scopes. Be sure to validate them.
• Some APIs require you to pre-register your scopes.
• Some let you mark scopes as required or optional.
• Some (like Gmail and Gusto) require a security review if you request certain scopes.

After years of building Nango, we still have not found a universal abstraction for scopes. Our advice: understand them as needed, track what was granted, and do not try to over-generalize.

Security

Your users’ credentials are literally the keys to their systems.

Handle them with the appropriate security measures:

• Encrypt credentials in transit and at rest (in your database).
• Use trusted cryptographic libraries and modern ciphers.
• Use per-customer encryption keys.
• Rotate encryption keys regularly, and make sure your system supports this.
• Never expose credentials to the frontend.
• Limit exposure to refresh tokens. They are only needed for access token renewal.
• Conduct regular security reviews and penetration tests (performed by professionals, not just automated tools).

Open Source Libraries and Other Solutions

We spent more time digging through OAuth libraries on GitHub than we would like to admit.

Most focus on user login. The few that go beyond that only handle standard OAuth flows. None address the quirks of accessing data across hundreds of APIs.

Also, auth is stateful. You need background jobs to refresh tokens and manage their lifecycle.

We could not find a library that met our needs, so we built our own and open-sourced it:

• GitHub repository
• Free to self-host all core auth features
• Cloud-hosted version with usage-based pricing

We hope this helps your team avoid the months of pain we went through while building API auth.