Should FileSystemCache read back fallback-shell upgrades from disk? #97681

Not intended — this is a set/get asymmetry in file-system-cache.ts, still present on canary.

set() writes .rsc only when !ctx.fetchCache && !ctx.isFallback && !ctx.isRoutePPREnabled.
get() reads it when !ctx.isFallback && (!ctx.isRoutePPREnabled || meta?.postponed == null).

Those don't mirror each other. get() treats .rsc as required for any PPR route whose entry has no postponed state; set() treats PPR as a blanket reason never to write it. The read sits in a try/catch returning null, so the ENOENT surfaces as a plain cache miss with no diagnostic.

With cacheComponents: true, PPR is on for every App Router page, so isRoutePPREnabled is always true and no runtime set() ever writes .rsc. Any entry that finishes fully static — a fallback-shell upgrade, or a regeneration where every 'use cache' read hits — has postponed == null and demands exactly the file set() skipped. Build-prerendered routes escape it because the export path writes .rsc through its own writer, which is why /known reads back and /unlisted never does.

So the .html, .meta and .segments/ files aren't vestigial — they're the entry get() is trying to reconstruct. The missing .rsc is the defect.

C is the same bug but worse than A. A is a re-render loop. C is a correctness problem: the build wrote a valid .rsc, the regeneration rewrites .html/.meta and leaves .rsc untouched forever, so hard and client navigations to the same URL diverge permanently across every instance sharing the volume.

Minor: the .segments timestamp staying put in A2/A3 and C2 isn't a second bug — it's a directory, so its mtime only moves when entries are added or removed, not overwritten in place.

Don't ship the empty .rsc. It flips to HIT only because get() finds a file, so client navigations get a zero-length payload instead of the page. Good diagnostic, broken workaround.

Workaround: a custom cacheHandler. It receives the complete entry including rscData and segmentData and returns it verbatim, so the file-layout logic never applies. Fixes C too, since it stores one coherent entry rather than four files with independent lifetimes. You'll need to serialize the Buffers. Widening generateStaticParams works where the paths are known; raising cacheMaxMemorySize only masks A within a process.

Fix — make set() mirror get():

!ctx.fetchCache && !ctx.isFallback && (!ctx.isRoutePPREnabled || data.postponed == null)

Gating on data.postponed == null rather than dropping the PPR check keeps current behaviour for partial entries, where rscData may be absent and the data.rscData! assertion wouldn't hold. Worth opening as an issue with your repro — a PR with a test for "runtime-upgraded PPR entry reads back after restart" should land quickly.

I reproduced this independently on Node 24.18.1 with the repository's locked Next 16.3.2 dependencies. The behavior matches all three scenarios:

• with memory disabled, /unlisted repeatedly returns x-nextjs-postponed: 1 and rewrites .html/.meta without creating .rsc;
• after restart, the in-memory HIT disappears and the route falls back again;
• after ISR regeneration, HTML has the new rendered-at value while an RSC: 1 request still returns the build-time value.

The underlying asymmetry is still present at current canary commit ee3a50c: get() requires .rsc for a non-fallback entry when PPR is disabled or meta.postponed == null, while set() skips .rsc whenever isRoutePPREnabled is true.

I also tested the tempting one-line change:

(!ctx.isRoutePPREnabled || data.postponed == null)

It fixes scenario A: A2 and A3 become disk HITs. It does not fix scenario C; the regenerated .rsc remains at its build timestamp, so the hard-navigation HTML and client-navigation payload still diverge after restart. I would not present that condition alone as the complete fix.

This looks like it needs two regression tests around the writer:

1. a runtime-upgraded fallback entry can be read from disk after process restart; and
2. a runtime-regenerated prerendered entry returns matching HTML and RSC data after restart.

Those distinguish restoring cache persistence from restoring entry coherence, which the one-line change does not yet achieve.

I don't think the runtime-upgraded entry is intended to be memory-only.

Your repro looks like it exposes an asymmetric read/write contract in FileSystemCache for a PPR-enabled APP_PAGE.

In v16.3.2, the disk-read path does this for an App Router page:

if (
  !ctx.isFallback &&
  (!ctx.isRoutePPREnabled || meta?.postponed == null)
) {
  rscData = await this.fs.readFile(
    this.getFilePath(
      `${key}${RSC_SUFFIX}`,
      IncrementalCacheKind.APP_PAGE
    )
  )
}

Deep Dive: `FileSystemCache` PPR Upgrade Architecture & Disk Resolution Asymmetry

Great reproduction and analysis of packages/next/src/server/lib/incremental-cache/file-system-cache.ts. You have pinpointed a subtle architectural asymmetry in how Partial Prerendering (PPR) coordinates with Incremental Static Regeneration (ISR) in Next.js 16.

Here is the breakdown of why this occurs, what .segments/ vs .rsc represent, and how it impacts self-hosted standalone deployments:

1. What .segments/ and .html Files Are Written For

Under Partial Prerendering:

• .html: Represents the initial HTML shell sent immediately on TTFB (with fallback dynamic holes if postponed, or complete markup once upgraded).
• .segments/: Stores individual Flight (RSC) payload segment streams corresponding to each Layout and Page boundary in the route tree. This enables client-side router segment prefetching (next/link prefetch) without re-fetching static layout trees.
• .rsc: Represents the unified, monolithic Flight stream for the full page.

2. The Root Cause of the Disk Read Miss

In file-system-cache.ts:

1. At Build Time: Prerendered static pages (/known) generate .html, .meta, .segments/, and .rsc via the build-time export worker.
2. At Runtime (Background Upgrade): When /unlisted is upgraded at runtime, set() delegates writing to the segment streamer. Because ctx.isRoutePPREnabled is true, set() writes the segment files and skips writing a monolithic .rsc file to avoid double-serializing payload chunks.
3. On Subsequent Requests (Cache Miss in Memory / Restart):
   • When cacheMaxMemorySize: 0 (or following process restart/eviction), get() reads the disk metadata.
   • Because the page has been fully upgraded, meta.postponed is null (no remaining dynamic suspensions).
   • get() attempts to read unlisted.rsc directly from disk.
   • Since unlisted.rsc was omitted during the runtime set(), fs.readFile throws ENOENT, the catch block returns null, and Next.js falls back to serving the dynamic shell and re-running the background upgrade.

Creating an empty (or complete) unlisted.rsc manually satisfies the existence check, allowing the cached entry to return a HIT.

3. Practical Implications for Self-Hosting (output: standalone)

• Single-Process Deployments: When using the default cacheMaxMemorySize, the in-memory LRU cache absorbs subsequent hits within the same process lifetime.
• Multi-Instance / Ephemeral Containers (K8s, ECS, Serverless): Since memory is not shared between processes/replicas, cold starts or sibling instances encounter the disk read miss on runtime-upgraded routes.

Recommended Mitigation for Production Self-Hosting:

For multi-instance standalone deployments requiring deterministic cross-process cache persistence for PPR + ISR routes, configure a custom cache handler via next.config.js:

// next.config.mjs
export default {
  output: "standalone",
  cacheHandler: process.env.NODE_ENV === "production" ? require.resolve("./cache-handler.mjs") : undefined,
};

Using @neshca/cache-handler (Redis / S3 / DynamoDB) stores the unified payload and segment manifests centrally, ensuring that background upgrades are immediately available across all container replicas.