Skip to content

Upstream proxy mode

Upstream proxy mode turns Pier into a transparent mirror of registry.npmjs.org (or any npm-compatible upstream). The whole team uses one .npmrc URL and Pier handles the rest — packuments cached and revalidated, tarballs pulled lazily on first install, GC keeps disk usage under a cap.

  • One URL, no scope routing. Private and public packages live behind the same https://your-pier-host/registry/npm/. No more @org:registry=… plus a default registry — one line.
  • Installs survive npmjs.org outages. Anything you’ve installed before is in the cache. New transitive deps fail only on first miss.
  • Audit visibility. The Mirror tab lists every public package the team has installed through Pier, with size on disk and last-fetched time.
  • Compliance. If you need to vet which versions reach your codebase, the proxy is the chokepoint.
Client Pier Upstream
│ │ │
│ GET /react │ │
├───────────────────►│ cache miss / TTL expired │
│ ├───────────────────────────►│
│ │ 200 packument │
│ │◄───────────────────────────┤
│ packument │ cache + URL-rewrite │
│◄───────────────────┤ dist.tarball → /registry/ │
│ │ │
│ GET /react/-/.tgz │ │
├───────────────────►│ cache miss │
│ ├───────────────────────────►│
│ │ tarball bytes │
│ │◄───────────────────────────┤
│ tarball │ write FS + DB row │
│◄───────────────────┤ │
  • Packument cache lives in SQLite as a single JSON blob per package (npm_packages.upstream_packument_json). One row per package — even for next (3769 versions) that’s still one row.
  • Tarball cache lives on disk at data/registry/{package}/{file}.tgz. Only versions clients actually installed land here.
  • TTL revalidation — when a cached packument is older than the configured TTL, Pier sends If-None-Match: <stored-etag> upstream. A 304 short-circuits without payload; a 200 replaces the blob.

Packages → Upstream proxy has four settings:

Setting What it does Reasonable values
Enable upstream proxy Master toggle Off by default; flip on
Upstream URL Where to fetch from https://registry.npmjs.org (default)
Packument TTL (seconds) How often to revalidate metadata 600 (10 min) default; 3600 if you want less chatter
Max cache size (MiB) Hard cap on tarball storage. 0 = unlimited See sizing below

The packument cache is small (~1–50 KB per package, all in SQLite — counts toward the DB file, not the on-disk cap). Tarballs dominate.

Scenario Recommended cap
Single dev box 500–1000 MiB
Small team (3–5 devs) 2–5 GiB
Shared CI mirror 20+ GiB
Server with plenty of disk 0 (unlimited)

When the cap is exceeded, the background LRU GC (runs every 10 minutes) drops the oldest tarballs until the total fits. Evicted tarballs are silently re-fetched on next install — the user sees no error, just an extra network round-trip.

The Mirror tab in Packages lists every cached public package:

  • Versions cached — count from the upstream packument
  • Tarballs on disk — sum of actually-downloaded versions
  • Last fetched — when the packument was last refreshed
  • ★ — pin the packages you care about, then filter with Pinned only

Click a row to open the package detail. Cached versions show their size, integrity hash, and a Deprecate / Unpublish button (operator-level operations).

When a proxy package has metadata but no tarballs on disk (post-GC, or a cold cache), the detail page shows a Download latest (X.Y.Z) button. Click it to eagerly pull that version’s tarball through Pier without running npm install locally — useful for warming the cache or smoke-testing a new mirror.

Toggle Enable upstream proxy off and save. Cached packages stay in the DB and on disk; new requests for uncached packages return 404. Re-enable to resume. Cache is never auto-cleared on disable.

If upstream is unreachable on a TTL revalidation, Pier serves the cached copy and logs a warning. The proxy never blocks reads on upstream availability.

If upstream returns 404 on a refresh of a previously-cached package, Pier keeps serving the cached copy — explicit unpublish from your registry is a separate, more careful operation.