Code & developer
Code & packages benchmark
Repositories, packages, releases and container images. Every code & packages provider in Alexandria, compared on quality, speed and cost from real agent calls, so your agent can pick the right one.
- Providers
- 5
- 4 with enough calls to rank
- Calls
- 7.2K
- Last 24 hours
- Top quality
- 100%
- npm
- Fastest
- 393 ms
- PyPI
Leaderboard
Ranked by quality on code & packages tools only, not each provider as a whole.
| # | Provider | Quality | Speed | Cost | Calls |
|---|---|---|---|---|---|
| 1 | 100% | 716 ms | 5 credits | 203 | |
| 2 | 99.1% | 393 ms | 5 credits | 540 | |
| 3 | 98.5% | 23.0 s | 1–5 credits | 6.4K | |
| 4 | 95.3% | 45.5 s | 1 credit | 64 |
Fewer than 20 calls to rank yet: Docker Hub.
Tools in this benchmark
The endpoints agents call for code & packages, busiest first.
| Tool | Provider | Quality | Speed | Calls |
|---|---|---|---|---|
| Repo | GitHub | 97.9% | 45.5 s | 3.1K |
| File | GitHub | 99.3% | 481 ms | 1.5K |
| Search repos | GitHub | 98.8% | 1.9 s | 860 |
| Project | PyPI | 99.1% | 393 ms | 540 |
| Latest release | GitHub | 99.1% | 1.2 s | 462 |
| Issues | GitHub | 98.8% | 968 ms | 167 |
| Releases | GitHub | 98.2% | 833 ms | 167 |
| Get package | npm | 100% | 792 ms | 160 |
| Package record | Package registries | 95.3% | 45.5 s | 64 |
| Get package version | npm | 100% | 435 ms | 43 |
| Repo advisories | GitHub | 97.6% | 9.2 s | 41 |
| Advisory | GitHub | 100% | 505 ms | 40 |
| User | GitHub | 100% | 45.5 s | 39 |
| Contributors | GitHub | 100% | 45.5 s | 33 |
| Search advisories | GitHub | 100% | 435 ms | 24 |
How we measure
- Quality
- Calls answered without a tool error, on this use case's tools.
- Speed
- P95 latency: 95% of successful calls finish faster.
- Cost
- Firecrawl credits per call, from the catalogue.
Measured over last 24 hours of calls through Alexandria by every user. A provider needs at least 20 calls on these tools to rank; the rest are listed under the table.
More Code & developer benchmarks
- Apps3 providers · 1.8K calls
- Developer docs14 providers · 1.3K calls
- Cloud & AI models7 providers · 1.3K calls
- Security & vulnerabilities5 providers · 209 calls
See every use case on the benchmarks and rankings pages.
//
Get started
//
Start building with Alexandria.
Give your AI agent direct access to the sources it needs.
Are you an AI agent? See setup options