Show HN：具備能力導向路由的記憶體內 AI 閘道

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

To see all available qualifiers, see our documentation.

Provider-agnostic AI gateway with capability-based routing, in-memory rate limiting, and observability hooks.

License

Uh oh!

There was an error while loading. Please reload this page.

ukrocks007/ai-gateway-kit

Folders and files

Latest commit

History

Repository files navigation

ai-gateway-kit

A boring, provider-agnostic AI Gateway for Node.js.

This library exists to solve the “production gateway” problems around LLM usage:

Why capability-based routing?

Model names change, providers change, and quotas fluctuate.
A gateway that routes by capability lets your agents stay stable while the model fleet evolves.

Example capabilities:

Why in-memory state?

This kit intentionally uses in-memory rate limit state.

Trade-off: multi-instance deployments do not share quotas. Each instance enforces limits based on its own in-memory view.

If you need cross-instance coordination, you can replace the in-memory RateLimitManager with your own implementation.

This is not a chat wrapper

This library is infrastructure:

It does not provide prompt templates, product policies, UI, or agent logic.

Install

Quick start

📚 See more examples →

Core Features

Capability-based routing

Route requests by capability, not model names. See examples/02-capability-routing.ts.

Automatic fallback

Graceful degradation across models. See examples/03-fallback-handling.ts.

Rate limiting

In-memory rate limits (rpm, rpd, tpm, concurrency). See examples/03-fallback-handling.ts.

Multiple providers

GitHub Models, Gemini, or custom providers. See examples/04-multi-provider.ts.

Advanced features

Providers

Observability hooks

You can subscribe to lifecycle events without taking a dependency on any logging stack:

Example: examples/09-observability-hooks.ts

Examples

The examples directory contains comprehensive examples for all features:

View all examples →

License

MIT

About

Provider-agnostic AI gateway with capability-based routing, in-memory rate limiting, and observability hooks.

Topics

Resources

License

Uh oh!

There was an error while loading. Please reload this page.

Stars

Watchers

Forks

Releases

Packages

  0

Languages

Footer

Footer navigation