Substitution Across LLM API Products in the OpenRouter Market

• Read the paper
• NYU Stern Glucksman archive
• Project Github repo

This was my NYU Stern MBA research project through the Glucksman Institute for Research in Securities Markets, advised by Professor Felix Montag.

The question was practical: as developers get access to many LLMs through a single routing layer, do API products start to behave like commodities, or does market structure still show up in how users substitute across products?

My answer is narrower, but more useful: in the OpenRouter-routed market, competition does not look like one flat pool of interchangeable models. Substitution appears much stronger within the open-source/proprietary divide than across it.

Research Question

LLM APIs are often discussed as if they are moving toward commodity infrastructure: cheaper tokens, many available models, and lower switching costs for developers who already route calls through an aggregator.

That intuition is partly right, but it leaves out the competitive question. When usage shifts, does it shift across the whole market, or mostly among products that users already treat as close substitutes?

I focused on OpenRouter because it gives that question a concrete empirical setting. Many competing models are available through one routed API marketplace, prices are visible, and switching friction is lower than in a world where every provider has its own separate integration path.

Why OpenRouter

OpenRouter is not the entire LLM inference market. It is a routed API marketplace, and the sample should be read that way.

That narrower scope is also what makes it useful. The setting puts open-source and proprietary models beside each other in a shared distribution layer. It makes model availability, routing, prices, and usage visible enough to study market structure rather than only benchmark performance or product announcements.

The raw weekly panel covers 53 models, 11 providers, and 27 weeks from August 2025 through February 2026. Total observed routed usage in the raw panel is about 154.5 trillion tokens.

What I Built

I built the data pipeline and empirical panel behind the paper: OpenRouter-routed model activity, prices, model metadata, LM Arena-style quality signals, manual alias reconciliation, and final product labels.

The data work included scraping and normalizing OpenRouter model activity, converting daily observations into a weekly model panel, joining price and quality signals, manually resolving model aliases, and labeling models by provider, open-source/proprietary status, generation status, and product bucket.

The final estimation panel aggregates model-level observations to provider x bucket composite products. That aggregation matters: individual model prices barely moved during the sample, so the demand estimates are not estimates for individual named models. They are estimates for composite products such as a provider’s flagship, balanced, or value bucket.

Main Result

The preferred specification is a nested-logit IV demand model with open-source and proprietary nests. The empirical object is substitution across provider x bucket products inside the OpenRouter-routed market.

The headline result is segmentation: substitution is much stronger within the OSS/proprietary divide than across it. The preferred model implies that within-group substitution is roughly 14.9 times stronger than across-group substitution.

I read the exact elasticity magnitude cautiously, but the structured substitution pattern is the more important result. The segmentation finding remains stable under nearby outside-good calibrations and a stable-roster robustness check.

Why It Matters

For AI infrastructure, the result pushes against the simplest commodity story. Lower switching costs and shared routing do not necessarily erase product segmentation.

For product strategy, it suggests that OSS/proprietary status is not just metadata. It may bundle differences in trust, deployment preference, performance expectations, cost structure, buyer taste, or organizational constraints that shape substitution even when users can access models through the same marketplace.

For economics, the project is a small example of how AI marketplace data can be used to study competition in a market that changes faster than traditional industry datasets usually capture.

Limits

Scope matters. This is a study of OpenRouter-routed API usage, not all LLM inference.

The estimates are for provider x bucket composite products, not individual named models. The outside option is calibrated rather than directly observed. Exact elasticity levels should be read with care, especially because the market was evolving quickly during the sample.

The main takeaway is the pattern: within this routed marketplace, substitution appears structured by the OSS/proprietary divide rather than behaving like one undifferentiated pool of API products.

What I Learned

The project became a lesson in how much empirical AI work depends on the boring, load-bearing parts of data infrastructure.

The hardest part was not running the final demand model. It was building a reliable path from visible marketplace activity to an analysis-ready panel: collecting usage signals, handling dynamic model pages, reconciling aliases, deciding when two names referred to the same underlying product, and preserving enough metadata to make the final econometrics interpretable.

Felix was especially helpful in pushing the project toward the right industrial organization toolkit. His feedback helped me move from descriptive market tracking toward demand estimation, nesting structures, outside-good assumptions, and IV-style identification questions that made the substitution result more economically meaningful.

The most useful product lesson was that routing layers do not automatically flatten markets. Even in a shared API marketplace, product identity still matters. The open-source/proprietary divide behaved less like a label and more like a market boundary.

Links

• Read the paper
• NYU Stern Glucksman student research archive
• Project Github repo