Research & reference
Research papers benchmark
Academic papers, preprints and publications. Every research papers provider in Alexandria, compared on quality, speed and cost from real agent calls, so your agent can pick the right one.
- Providers
- 6
- 5 with enough calls to rank
- Calls
- 5.9K
- Last 24 hours
- Top quality
- 100%
- Nature
- Fastest
- 1.4 s
- Nature
Leaderboard
Ranked by quality on research papers tools only, not each provider as a whole.
| # | Provider | Quality | Speed | Cost | Calls |
|---|---|---|---|---|---|
| 1 | 100% | 1.4 s | 5 credits | 23 | |
| 2 |  Research Index | 99.9% | 31.6 s | 0 credits | 2K |
| 3 | 98.9% | 3.2 s | 5 credits | 471 | |
| 4 | 98.8% | 9.7 s | 5 credits | 3.4K | |
| 5 | 98.2% | 2.6 s | 5 credits | 55 |
Fewer than 20 calls to rank yet: Reactome: biological pathways and pathway enrichment.
Tools in this benchmark
The endpoints agents call for research papers, busiest first.
| Tool | Provider | Quality | Speed | Calls |
|---|---|---|---|---|
| Article | PubMed and PMC | 98.8% | 1.4 s | 2K |
| Search | PubMed and PMC | 99% | 25.0 s | 1.1K |
| Read paper passages | Research Index | 100% | 52.8 s | 956 |
| Inspect paper | Research Index | 100% | 195 ms | 528 |
| Search papers | Research Index | 99.5% | 17.6 s | 383 |
| Search | Europe PMC | 98.7% | 4.3 s | 298 |
| Full text | PubMed and PMC | 98.3% | 6.5 s | 180 |
| Find related papers | Research Index | 100% | 41.2 s | 167 |
| Get | Europe PMC | 100% | 1.0 s | 131 |
| Full text | Europe PMC | 97.6% | 1.9 s | 42 |
| Search submissions | OpenReview | 97% | 2.9 s | 33 |
| Summaries | PubMed and PMC | 100% | 531 ms | 26 |
| List journal articles | Nature | 100% | 1.4 s | 23 |
| Search replies | OpenReview | 100% | 2.3 s | 22 |
How we measure
- Quality
- Calls answered without a tool error, on this use case's tools.
- Speed
- P95 latency: 95% of successful calls finish faster.
- Cost
- Firecrawl credits per call, from the catalogue.
Measured over last 24 hours of calls through Alexandria by every user. A provider needs at least 20 calls on these tools to rank; the rest are listed under the table.
More Research & reference benchmarks
See every use case on the benchmarks and rankings pages.
//
Get started
//
Start building with Alexandria.
Give your AI agent direct access to the sources it needs.
Are you an AI agent? See setup options