← Practice · August 31, 2026

How to Organize 10,000 Pages in Astro Without Breaking the Build

Why a large reference site uses a static site generator, how to structure the content, and what exactly breaks at ten thousand pages: builds, search, the sitemap, and deployment.

8 min readwebsites

Ten pages and ten thousand pages are two different crafts. With ten, you can write the markup by hand. With ten thousand, every decision is multiplied by ten thousand: an extra database query, an extra image, an extra second of build time.

I am writing from experience with a live project: a reference site built with Astro. It currently has three and a half thousand pages and continues to grow. Everything below is something I have stumbled over myself.

Why a Static Site Generator Rather Than a CMS

For a reference site—a catalog, dictionary, collection of summaries, or product cards without a shopping cart—content changes rarely but is read often. This is exactly where a CMS works against you: every visit starts PHP, queries the database, and assembles a page that has not changed in six months. Then you start treating the symptoms: caching, then caching the cache, then “why has our cache broken again?”

A static site generator builds the same pages once and stores the finished HTML. The server serves a file. There is no database in production, nothing to hack, nothing to fail under load, and hosting is as cheap as it gets.

A CMS clearly wins in two cases: several people edit the content without repository access, or the pages are personalized (user accounts, shopping carts, user-specific prices). If neither applies, use static pages.

Astro is convenient here because it ships zero kilobytes of JavaScript by default, while components can be written like ordinary HTML. On a large reference site, this matters more than fashionable features: the page should open instantly on a phone in the subway.

How to Structure the Content

The main rule: a page is data plus a template, not a file someone wrote by hand. Nobody can maintain ten thousand .astro files.

Keep the data separately: JSON, YAML, or Markdown with fields. Then a single dynamic route expands it into pages:

src/
  pages/
    произведения/[work].astro     ← one template for all entries
    авторы/[author].astro
    index.astro
  data/
    работы/*.json                 ← data, one file per entity

// src/pages/произведения/[work].astro
export async function getStaticPaths() {
  const работы = await Astro.glob('../../data/работы/*.json');
  return работы.map((р) => ({
    params: { work: р.адрес },
    props: { работа: р },
  }));
}
const { работа } = Astro.props;

There are three things worth deciding once and never changing:

URLs. Latin characters, transliteration, no dates in the path. A page URL is permanent: renaming it a year later means redirects, lost links, and lower search rankings.

Flat or nested structure. For ten thousand pages, a flat structure with meaningful sections (/произведения/имя/, /авторы/имя/) is better than a five-level hierarchy. Depth complicates navigation, breaks breadcrumbs, and makes URLs long.

What is computed during the build and what is stored in the data. Anything that can be calculated in advance—counters, relationships, “related content”—should be calculated once by a script and stored in the data. Otherwise, ten thousand pages will recalculate the same thing during every build.

What Breaks at Scale

Build time grows linearly, but memory usage does not. With a couple of hundred pages, the build takes seconds. With thousands, it takes minutes, and the main resource hog is not HTML but image processing. If every page has an image passed through the built-in optimizer, every build will perform ten thousand image operations. The fix is simple: process media once with a separate script, put the finished files alongside the content, and include them as static assets. The rule is that the build should contain nothing that can be done in advance.

Do not load the entire dataset on every page. Running Astro.glob over ten thousand files inside a page template means ten thousand directory reads. Collect shared data once in getStaticPaths and pass only what the specific page needs through props.

Lists need pagination. A catalog with ten thousand links on a single page means megabytes of HTML that nobody will scroll through. Astro’s built-in pagination handles this in a few lines:

export async function getStaticPaths({ paginate }) {
  const все = await получитьВсе();
  return paginate(все, { pageSize: 50 });
}

You run out of disk space before patience. My build directory for three and a half thousand pages takes up 1.2 gigabytes, mostly because of media. On a small VPS, that is half the available space. Keep media separate from the build and do not upload it again every time.

Search

Client-side search across ten thousand pages is another trap. The naive approach—“let’s build one JSON file containing all the text”—produces an index tens of megabytes in size, which the browser dutifully downloads on a phone.

There are two workable options.

A segmented index. Keep only titles, keywords, and URLs in the index, without the full text. This is usually enough for a reference site: people search for a title, not a phrase from the middle.

Pagefind. A tool designed specifically for static sites: after the build, it scans the finished HTML and splits the index into small chunks, so the browser loads only what it needs. Add it as a post-build step:

npx pagefind --site dist

Whatever you choose, check the size of what the browser downloads. That is the only metric that matters here.

Sitemap

The standard does not allow more than 50,000 URLs in a single file. Ten thousand still fits, but it is better to install the official plugin from the start—it will split the sitemap into parts and create an index file automatically:

// astro.config.mjs
import sitemap from '@astrojs/sitemap';
export default defineConfig({
  site: 'https://example.ru',
  integrations: [sitemap()],
});

Do not forget site—without it, the sitemap URLs will be relative, and the search engine will reject it.

Deployment

This is where most people run into trouble. A built site consists of tens of thousands of small files. Uploading all of them to conventional hosting over SFTP takes an hour and carries a high chance that the connection will drop halfway through.

Upload only what changed. If SSH is available, use rsync -a --delete dist/ сервер:/путь/. If the hosting provider offers only SFTP, maintain a manifest: store file checksums after each deployment and upload only files whose checksums have changed. My deployment script accepts a list of paths and uploads exactly those files—a routine edit affects three or four files instead of ten thousand.

Do not touch media. Images and audio are uploaded once and left in place. Reuploading them with every build is the most common reason why “deployment takes forty minutes.”

Keep in mind that some artifacts do not rebuild themselves. A large site almost always accumulates things that are generated separately: the search index, static listings, and the sitemap. Create a deployment checklist and follow it—otherwise, a month later you will discover that the latest three hundred pages are missing from search.

When Astro Is Not the Best Choice

If you have not ten thousand pages but a hundred thousand or more, consider Hugo: it builds that volume in seconds because it is written in Go, and the difference becomes decisive. The tradeoff is a less pleasant templating language.

If the content changes every hour and waiting for a build is not an option, you need on-demand rendering with caching—Next.js and its incremental regeneration.

If the content is edited by non-developers, add a headless CMS to the static setup but keep the build static: an editor writes in the admin interface, the build starts at the press of a button, and the finished HTML is still what gets served publicly.

In every other case, static sites are boring, cheap, and keep working for years without your involvement. For a reference site with ten thousand pages, that is exactly what you need.