Starting point: three repos, one product surface
The iOS app, the Mainline CI/CD platform, and a user-analytics pipeline started life in three separate repositories. They ship one product family, so the split cost us the usual things: cross-repo changes needed coordinated PRs, a shared Go helper had to be vendored or duplicated, and CI in each repo had its own build conventions. We consolidated in two moves — the Mainline backend stack first, then the analytics pipeline three months later — and the second move was deliberately cheaper because the conventions were already in place.
Boundaries first: code merges, runtime does not
The rule we held both times: merging code into the monorepo says nothing about sharing runtime resources. The repository is the source of truth for source and build config; deployment topology is a separate decision.
Mainline landed under Server/mainline/, sitting next to Server/aio-server/ (Widget management) and the Next.js frontends Server/mainline-web/ and Server/aio-server-web/. The first consolidation commit moved 2050 files under Server/ — 1165 for mainline, 353 for aio-server, plus the two web apps and shared libs — establishing the whole backend tree in one shot.
The analytics service kept the boundary explicit in its own CLAUDE.md:
Deployment resource isolation: zero sharing with the AIO business backend
(aio-server / mainline). Separate VLAN, separate VM, separate PG / Redis / MQ.
Code merges into AIO, runtime resources do not.
So the analytics five-stage pipeline (Collector → Redpanda → Worker → ClickHouse → Query → Dashboard) lives in Server/analytics-service/ and Server/analytics-web/, but it runs on its own infrastructure. The migration commit only merged the code layer; deploy and migrate jobs were left as TODO stubs until the isolated infra was ready. Co-locating source while isolating runtime is the trade-off that let the merge happen without a coupled deployment.
Dependency handling: one Go workspace, one pnpm workspace
Go modules are stitched together with a root go.work. Each merge appends one line:
go 1.25.5
use (
./Server/libs/go
./Server/libs/proto
./Server/aio-server
./Server/analytics-service
./Server/mainline
)
Shared Go code lives in Server/libs/go, and every service imports it under the same module path prefix — github.com/anthropics/mainline/libs/go/{logger,apperror,hmac,tracectx,jwt,apikey,ratelimit}. The analytics service kept its original module path, github.com/anthropics/mainline/services/analytics-service, so internal imports didn't have to be rewritten.
The interesting friction was the shared library. The analytics service depended on helpers AIO didn't have (apikey, ratelimit) and on slightly different shapes of ones it did. Rather than fork, we extended libs/go additively: new jwt.GenericManager (UUID-keyed, alongside the existing user-system manager), hmac.SignSimple (no timestamp), apperror.NotFoundMessage. New files (jwt/generic.go, hmac/simple.go, apperror/extra.go) rather than edits to existing ones, so the established services kept their behavior untouched.
Bazel is the only build system, so third-party Go deps go through Bzlmod. The analytics merge added exactly what the pipeline needed to MODULE.bazel:
+ "com_github_clickhouse_clickhouse_go_v2",
+ "com_github_pashagolub_pgxmock_v4",
+ "com_github_twmb_franz_go",
Frontends share a single dependency tree. analytics-web was trimmed on the way in — its standalone recharts / date-fns / clsx were dropped in favor of plain SVG sparklines, and it joined the root pnpm-workspace.yaml to share the top-level package.json and node_modules:
packages:
- "Server/mainline-web"
- "Server/aio-server-web"
- "Server/portal-web"
- "Server/analytics-web"
- "Server/libs/ts/*"
One thing the merge does break is hard-coded paths that assumed a repo root. Mainline searched for its LLM config at ./dev/mainline/llm.yaml; under the monorepo that path no longer exists, and the fix repointed it to ./DevOps/Development/Server/mainline/llm.yaml. Path constants that encode repository layout are exactly what shifts when a repo becomes a subtree, so they're worth grepping for up front.
Port isolation: hashing the worktree name
Multiple worktrees on one machine each want to run the full backend stack — Postgres, Redis, ClickHouse, Redpanda, MinIO, the services themselves. Fixed ports would collide the moment two worktrees both ran bazel run. The scheme is a deterministic offset derived from the worktree name:
calculate_port_offset() {
local name="$1"
local hash=$(echo -n "$name" | md5sum | cut -c1-6) # md5 on macOS
echo $(( 16#$hash % 500 ))
}
Six hex digits of the MD5, mod 500, yields an offset in [0, 499] that's stable for a given worktree. Each service then derives concrete ports from its own base. Mainline spreads its offset by 10:
SERVER_PORT=$((8080 + offset * 10)) # ≤ 13070
WEB_PORT=$((3000 + offset * 10)) # ≤ 7990
GRPC_PORT=$((9090 + offset * 10)) # ≤ 14080
The analytics stack was added later, so it had to route around the ranges Mainline, aio-server, and portal already claim. It starts every range at 15000 and steps by 1000 — comfortably above the others' ceilings, and with an inter-range gap (1000) far larger than the maximum offset (499), so ranges can't bleed into each other:
POSTGRES_PORT=$((15000 + offset))
REDIS_PORT=$((16000 + offset))
CLICKHOUSE_HTTP_PORT=$((17000 + offset))
CLICKHOUSE_NATIVE_PORT=$((18000 + offset))
REDPANDA_KAFKA_PORT=$((19000 + offset))
# ... up to COLLECTOR_PORT=$((24000 + offset))
Two escape hatches matter in practice. PORT_OFFSET can be set explicitly when the MD5-derived value happens to clash with something unrelated. And CI passes WORKTREE_NAME=ci-job-$CI_JOB_ID, so concurrent CI jobs on a shared runner get distinct offsets the same way local worktrees do — the docker compose project name (analytics-e2e-${WORKTREE_NAME}) is keyed off it too, keeping stacks fully separate.
The full E2E flow this enables — bring up the stack, migrate, seed, run tests, tear down — reported 28/28 passing in about 15 seconds inside a worktree, alongside 15/15 Bazel unit tests and 2/2 integration tests.
Constraints worth carrying forward
- Separate "code lives here" from "it runs here." Co-locating source in a monorepo is independent of sharing infrastructure; state the runtime isolation explicitly (we put it in
CLAUDE.md) so nobody assumes a shared database came along for the ride. - Extend shared libraries additively. New files and new entry points let an incoming service reuse common code without disturbing the services already depending on it.
- A later arrival routes around incumbents. When a stack joins last, its port ranges must dodge whatever's already allocated; an explicit base far above the existing ceilings is cheaper than reshuffling everyone.
- Grep for repository-root assumptions. Config search paths and any constant that encodes layout will break the moment a repo becomes a subtree.