# Evidence

> Every claim nextship makes, the measurement behind it, and where that measurement was taken.

Source: https://nextship.doodlebytestudio.in/docs/evidence

Everything below was produced by running something, not by reading code. Each row says
where it came from, so you can disagree with the method rather than with the number. The
last section is the honest list of what has not been proven.

## The adapter compatibility suite

The official Next.js adapter test suite is the arbiter of whether an adapter works. It
builds a real container per test file and deploys it.

| Result | Value |
|---|---|
| Suites passing | 1123 of 1123 |
| Assertions passing | 3599 of 3599, the measure Next.js's adapters support page publishes |
| Retries | None: every suite passed on its first attempt |
| Skipped by our list | 9 tests that assert what Vercel's CDN or proxy does, each with its reason |
| Next.js version | 16.4.0-canary.22 |
| Mode | `deploy`, a real container per test file |
| Reproduced | Two runs, the second on a later commit, matching suite by suite in passed, failed and skipped tests |
| Stable Next.js | On 16.3.5, 1108 of 1108 suites and 3489 of 3489 assertions, against that release's own, smaller test corpus |
| Re-run for 1.1.0 | Both again on the release's code, after the server target merged, with the same results |

1.0.1's run passed 1051 of 1115 suites. Researching all 64 failures found most in our test
harness rather than in nextship: the Docker build never received the variables a test sets
or the flags Next.js's own deploy path passes, packages fixtures commit under `node_modules`
were deleted before the build, and the container sat behind a remapped port. Two were real
nextship defects, both fixed: a Next.js build cache shared between projects, and route module
contexts missing from the image. The nine skipped tests, and why each cannot pass off
Vercel, are in the
[results](https://github.com/gowtham472/nextship#compatibility-suite-results).

A full run now takes 319 runner-minutes against 788, in the same 32 groups.

The suite runs from GitHub Actions in this repository, weekly and on demand, so a
regression shows up without anyone remembering to look.

## Streaming

A buffering proxy leaves Suspense and partial prerendering apparently working while
delivering none of the benefit: the page arrives complete, at the end. So streaming is a
test rather than an assumption. A fixture renders its shell at once and its tail two
seconds later, and the check fails unless the shell arrives first.

| Where | First byte | Last byte |
|---|---|---|
| Local, Next.js 16.3.4 feature app | 27 ms | 2.02 s |
| CI, on every push and pull request | 96 ms | 2063 ms |
| A live App Platform app, 2026-09-11 | 230 to 448 ms | 2.2 to 2.4 s |

The live run passed three times out of three, sent chunked through DigitalOcean's
Cloudflare edge with the cache bypassed.

The check is also known to fail when it should: it was run against a server that buffers,
one that never answers, and one that drops the connection part way, and it reported each.

## Edge runtime

A route handler and a page declaring `runtime = 'edge'` both served from a nextship image,
and the code in each saw the `EdgeRuntime` global and no Node.js version: Next.js runs it
in its Edge runtime inside the server, not on the Node.js runtime. Measured on 2026-09-13
with nextship 1.0.0 and Next.js 16.3.4 in a local container, and checked on every push
since, because the prune that builds the image could otherwise drop the Edge bundles
without any other test noticing.

## The runtime image

Measured on a production project: Next.js 16.2.9, pnpm, 21 routes, `next/image` in 21
files.

| | Before | After |
|---|---|---|
| Image | 1.13 GB | 591 MB |
| `node_modules` inside it | 469 MB | 58 MB |

`@next/swc`, 125 MB of compiler that no production server calls, is absent. Of the 591 MB
that remain, 332 MB is the `node:24-slim` base image, which a node pulls once and shares
across deployments.

Alpine was measured rather than assumed, and rejected: it saves 97 MB, and `sharp` resizes
take 91.8 ms on it against 37.1 ms on slim, a median of 15 runs on identical libvips.

## Against a live account

Every day two command was verified against a real DigitalOcean account rather than a
fixture:

- **Deploy and rollback**, including two rollbacks in opposite directions.
- **Custom domains:** certificate issued by Google Trust Services, TLS 1.3 negotiated,
  TLS 1.1 refused, plain HTTP redirected, every route 200.
- **Image retention:** a prune took the registry from 390.2 MiB to 286.1 MiB, freeing
  104.2 MiB.
- **Runtime logs**, both the snapshot and `--follow`.
- **Destroy**, which refuses unless you name the app.

Before 1.1.0 was tagged, the same commands ran again against a throwaway app built with the
release's code: deploy, a second deploy, rollback with no build or push, `logs` and
`logs --follow`, `env push` read by the running app about a minute later, `domain` and
`images`. The app and its registry were removed afterwards.

## Your own server

The server target, new in 1.1.0, ran on DigitalOcean Droplets between 2026-09-20 and
2026-09-23 (`s-1vcpu-2gb-70gb`, amd64, Ubuntu 24.04), after a local test server and a
Proxmox VM. Ragul D ran these, apart from the last two rows, which Gowtham ran on 2026-09-24
while recording the README's demo and testing the fix.

| Checked on a Droplet | Result |
|---|---|
| `server add` on a clean machine | Every step applied, swap included; a second run changed nothing |
| `deploy`, straight to the Droplet | 110 ms to first byte, 176 ms total |
| Streaming through Caddy | First byte 260 ms against a last byte of 2137 ms |
| Requests across a deployment | 579 of 579 succeeded, and 51 of 51 streams |
| A deployment whose server exits on start | Refused; 558 of 558 requests to the previous one succeeded |
| `server reboot`, a real kernel | Every app healthy again in 37 s |
| A hung app process | Restarted by the watchdog, serving again 4 min 52 s after it hung |
| `deploy` on a nearly full disk | Refused at 2.4 GB free, before building; the app kept serving |
| `server move` to a fresh Droplet | 1 min 34 s including setup, env file and Server Actions key identical |
| `conformance/vm/e2e.sh` in full | 8 min 6 s, including an ISR page and an optimized image surviving a restart |
| `deploy` two seconds after `server add`, two fresh Droplets, 2026-09-24 | One lost its build session mid-build ([issue #5](https://github.com/gowtham472/nextship/issues/5)) and deployed in 75 s when run again; the other deployed first time |
| A build with its connection cut four seconds in, a third Droplet | Same error as issue #5; built once more and deployed. A page that did not compile failed once in 14 s, not retried |

## Compared with Coolify

Coolify is the tool most often suggested for self-hosting Next.js, so we deployed the same
app with both, on 2026-09-26: two identical Droplets (`s-2vcpu-4gb`, 2 vCPU and 4 GB,
Ubuntu 24.04, `blr1`), one running Coolify 4.3.23 with its own installer and deploying to
itself, the other set up by nextship 1.1.2 with `server add`. The app was the streaming
fixture in `conformance/streaming/app`, Next.js 16.3.4, from the same commit. Gowtham ran
both. Every Coolify setting stayed at its default unless a row says otherwise.

| | Coolify 4.3.23 | nextship 1.1.2 |
|---|---|---|
| Server setup | 117 s, the installer | 41 s, `server add` |
| First deploy, cold | 170 s | 107 s |
| A rebuild with a warm cache | 59 s, a forced rebuild | 55 s, after a one-line change |
| App image, as `docker images` reports it | 987 MB, built by Railpack | 407 MB |
| Memory used by the app | 74 MB | 48 MB |
| Memory used by the tool itself | 460 to 520 MB for its dashboard, database, cache, realtime server, proxy and monitor, plus 296 to 341 MB for a build container that stayed running | 10 MB, Caddy |
| Streaming through the proxy | Passed three runs of three, first byte 167 to 651 ms | Passed three runs of three, first byte 108 to 186 ms |
| Edge runtime route | Served | Served |
| Requests during a redeploy, polled every 100 ms | 418 of 420 succeeded; 2 answered 502 | 356 of 356 succeeded |
| The same with Coolify's health check turned on (`/`, port 3000) | 1007 of 1008 succeeded; 1 got no response | Not applicable: the health check always runs |
| An app that never starts | Reported the deployment as completed while the container restarted in a loop | Refuses the deployment and keeps the previous one serving (above) |

How to read it:

- **The health check is what separates the redeploy rows.** Coolify leaves it off by
  default; with it on, one request failed in 1008, in a single run that cannot tell a
  failure during the container switch from one on the network between us and the
  Droplet. nextship always waits for the new container to pass before Caddy switches.
- **The redeploys were not identical.** nextship's rebuilt the app after a change;
  Coolify's reused its image, because a redeploy of an unchanged commit skips the build.
- **The app had no `start` script**, since nextship runs Next.js's server itself. Coolify's
  default build pack needs one, so its first deploy started an empty shell: we set the
  start command to `npx next start -p 3000` and forced a rebuild. An app made with
  `create-next-app` has the script and would not hit this.
- **One small app, one run of each.** Image size and memory will differ for your app;
  the difference in what each tool runs beside it will not change much.
- **Coolify does far more:** a dashboard, databases, any kind of app, many servers. This
  compares one thing, deploying a Next.js app to one server, and nothing else.

## An end-to-end test on macOS

On 2026-09-11 we installed nextship 0.4.4 from npm on a MacBook Pro with an M3 Max,
deployed a Next.js 16.3.4 app to App Platform, and measured it against the live URL. The
app was built to be awkward: a streaming Suspense boundary, an ISR route, a Server Action,
`next/image` on a raster source, a route reading a runtime environment variable, and a
dynamic homepage.

| Checked live | Result |
|---|---|
| Streaming | First byte 21 ms against a 1.52 s page |
| Server Actions | A form POST changed server state, and the response carried it |
| ISR | `s-maxage=30` with a correct stale-while-revalidate cycle |
| Image optimization | PNG to WebP at 640x480, 8593 bytes to 626, then `X-Nextjs-Cache: HIT` |
| Runtime environment | `env push` stored the value as a runtime secret, read at request time |
| Cross-architecture build | Built on darwin-arm64; the image carries `@img/sharp-linux-x64` |
| Secrets | No env file in the final image |
| TLS | HTTP/2, a valid certificate, correct 404 handling |
| Image size | 2181 traced files, a 107 MB runtime image |

**The first deploy failed.** Its dynamic homepage made nextship choose `/_global-error` as
the health check path, which answers 500, and the deployment never went live. 1.0.1 fixes
it; the second and third deploys went live. The test found eight things in all, and 1.0.1
fixes, pins with a test, or documents every one, as the changelog records.

## Supply chain

- Published with **provenance**, so the bytes on npm are linked to this repository and the
  exact commit that produced them, and anyone can verify it.
- Every release is **staged and then approved with two factor authentication**, which no
  workflow can do. A compromised workflow can stage a release; it cannot ship one.
- **No npm token exists**, in the repository or on a laptop.

## What the CLI can reach

- **One outbound destination per target:** your own cloud provider's API, or your own
  server over your SSH. There is no nextship server, no telemetry endpoint, no analytics,
  no update check and no crash reporting.
- The registry credential is written to a temporary directory at mode `0600` and removed
  afterwards, rather than passed on a command line where any local process could read it.
- `shell: true` appears nowhere in the codebase.

This site counts page views with Cloudflare Web Analytics, which sets no cookies. The CLI
sends nothing.

## What has not been proven

- **Middleware, Cache Components and PPR are proven by the compatibility suite alone.** In
  the full run, 481 assertions pass in suites named for middleware and 122 in suites for
  Cache Components, in containers on the CI runner. Deploy mode skips most PPR tests,
  leaving 10. None has been exercised on a live App Platform app.
- **Why a build session can drop right after `server add` is not known.** It happened once
  in three fresh servers, and is tracked in
  [issue #6](https://github.com/gowtham472/nextship/issues/6). The retry that answers it, in
  1.1.2, has run on a live server against a connection cut on purpose, not against the
  natural drop.
- **The server target has not run on Hetzner, on an arm64 machine, on Debian 12, with a
  certificate for a real domain, or from the GitHub Actions workflow in its guide.** Every
  real run so far was amd64, on DigitalOcean or Proxmox. Those are the next release's
  checklist.
- **Multi-instance behaviour is not supported.** One instance is correct; a shared cache
  is the next major version's work.
- **No third party penetration test**, and no dependency vulnerability scan in CI.
- **No outside test yet.** Every verification above, the macOS run included, was carried
  out by the people who wrote nextship. If you try it and something breaks, that is the
  most useful thing anyone can currently contribute.