Skip to content
[ PIER ]

Upstream proxy mode

Upstream proxy mode turns Pier into a transparent mirror of registry.npmjs.org (or any npm-compatible upstream). The whole team uses one .npmrc URL and Pier handles the rest — packuments cached and revalidated, tarballs pulled lazily on first install, GC keeps disk usage under a cap.

Why use it

Section titled “Why use it”
  • One URL, no scope routing. Private and public packages live behind the same https://your-pier-host/registry/npm/. No more @org:registry=… plus a default registry — one line.
  • Installs survive npmjs.org outages. Anything you’ve installed before is in the cache. New transitive deps fail only on first miss.
  • Audit visibility. The Mirror tab lists every public package the team has installed through Pier, with size on disk and last-fetched time.
  • Compliance. If you need to vet which versions reach your codebase, the proxy is the chokepoint.

How it works

Section titled “How it works”
Client                Pier                       Upstream
  │                    │                            │
  │  GET /react        │                            │
  ├───────────────────►│  cache miss / TTL expired  │
  │                    ├───────────────────────────►│
  │                    │   200 packument            │
  │                    │◄───────────────────────────┤
  │   packument        │  cache + URL-rewrite       │
  │◄───────────────────┤  dist.tarball → /registry/ │
  │                    │                            │
  │  GET /react/-/.tgz │                            │
  ├───────────────────►│  cache miss                │
  │                    ├───────────────────────────►│
  │                    │   tarball bytes            │
  │                    │◄───────────────────────────┤
  │   tarball          │  write FS + DB row         │
  │◄───────────────────┤                            │
  • Packument cache lives in SQLite as a single JSON blob per package (npm_packages.upstream_packument_json). One row per package — even for next (3769 versions) that’s still one row.
  • Tarball cache lives on disk at data/registry/{package}/{file}.tgz. Only versions clients actually installed land here.
  • TTL revalidation — when a cached packument is older than the configured TTL, Pier sends If-None-Match: <stored-etag> upstream. A 304 short-circuits without payload; a 200 replaces the blob.

Configure

Section titled “Configure”

Packages → Upstream proxy has four settings:

SettingWhat it doesReasonable values
Enable upstream proxyMaster toggleOff by default; flip on
Upstream URLWhere to fetch fromhttps://registry.npmjs.org (default)
Packument TTL (seconds)How often to revalidate metadata600 (10 min) default; 3600 if you want less chatter
Max cache size (MiB)Hard cap on tarball storage. 0 = unlimitedSee sizing below

Sizing the cache

Section titled “Sizing the cache”

The packument cache is small (~1–50 KB per package, all in SQLite — counts toward the DB file, not the on-disk cap). Tarballs dominate.

ScenarioRecommended cap
Single dev box500–1000 MiB
Small team (3–5 devs)2–5 GiB
Shared CI mirror20+ GiB
Server with plenty of disk0 (unlimited)

When the cap is exceeded, the background LRU GC (runs every 10 minutes) drops the oldest tarballs until the total fits. Evicted tarballs are silently re-fetched on next install — the user sees no error, just an extra network round-trip.

Inspect what’s cached

Section titled “Inspect what’s cached”

The Mirror tab in Packages lists every cached public package:

  • Versions cached — count from the upstream packument
  • Tarballs on disk — sum of actually-downloaded versions
  • Last fetched — when the packument was last refreshed
  • ★ — pin the packages you care about, then filter with Pinned only

Click a row to open the package detail. Cached versions show their size, integrity hash, and a Deprecate / Unpublish button (operator-level operations).

Pre-fetch a version

Section titled “Pre-fetch a version”

When a proxy package has metadata but no tarballs on disk (post-GC, or a cold cache), the detail page shows a Download latest (X.Y.Z) button. Click it to eagerly pull that version’s tarball through Pier without running npm install locally — useful for warming the cache or smoke-testing a new mirror.

Disable temporarily

Section titled “Disable temporarily”

Toggle Enable upstream proxy off and save. Cached packages stay in the DB and on disk; new requests for uncached packages return 404. Re-enable to resume. Cache is never auto-cleared on disable.

Failure handling

Section titled “Failure handling”

If upstream is unreachable on a TTL revalidation, Pier serves the cached copy and logs a warning. The proxy never blocks reads on upstream availability.

If upstream returns 404 on a refresh of a previously-cached package, Pier keeps serving the cached copy — explicit unpublish from your registry is a separate, more careful operation.

Section titled “Related”