Skip to main content
  1. Tech Blog: AI, Security, Infrastructure & Open Source/

walgit — Shopify's CEO Open-Sources an S3-Native Git Server

·1239 words·6 mins
Osmond van Hemert
Author
Osmond van Hemert
Open Source Chronicles - This article is part of a series.
Part : This Article

On August 24, Shopify CEO Tobias Lütke dropped walgit on GitHub: a Git server, written in Rust, MIT-licensed, and built — by his own account — over a single weekend out of frustration with Shopify’s internal Git infrastructure. Within 24 hours it had roughly 1,500 stars, a front-page Hacker News thread, and a running side-debate about how much of it he typed himself. The more interesting question is architectural: walgit is an open-source implementation of the design Cursor described in its Git at any scale post — the proprietary system Cursor calls Continuity — and it makes a bet that could reshape how we host repositories: the bucket is the repository, and every server is just a cache.

What walgit Actually Is
#

The README pitches it in one line: “a git server that is one binary in front of an object store.” You point a single binary at an S3-compatible bucket or GCS, and you get smart HTTP (v0/v2) fetch and push, Git LFS, a browsing web UI, a JSON API with a dependency-free SDK, per-repository push policies, webhooks, and OIDC authentication. There is no relational database, no leader node, and — the phrase that matters — no local state that matters. Push to a new name and the repository auto-creates. Add more machines pointed at the same bucket and they serve the same repositories with nothing to coordinate. Kill them all and, in the project’s own words, “you lose warmth, nothing else.”

It supports both SHA-1 and SHA-256 repositories, atomic receive-pack with push options, and repos namespaced as <owner>/<repo>. Upstream git does the pack-level heavy lifting (upload-pack, repack, bundle); walgit owns receive-pack, the write-ahead log, and the plumbing.

The Architecture: CAS Is the Consensus
#

To understand why this is interesting, you have to understand what it’s replacing. Git hosting is miserable at scale because of packfiles: every Git operation is a random walk over gigabytes of compressed binary, which is why “just put the repos on NFS” failed at every large host that tried it. The design that survived — GitHub’s Spokes — keeps real repositories on local NVMe and replicates at the packfile level with strict consistency, paid for with three-phase commit across a fixed replica set, a database mapping every repository to its machines, and a fleet of pets.

Continuity’s insight, which walgit takes as-is, changes the economics: make a write-ahead log in object storage the source of truth, and every on-disk repository a rebuildable cache. A push is indexed, checked for connectivity and policy, uploaded as immutable content-addressed objects, and becomes visible only when a tiny manifest is rewritten with a compare-and-swap. That CAS is the consensus — no election, no quorum, no primary. Any instance may accept a push; two racing instances cannot both win, and the loser re-reads, re-validates every ref, and retries. Reads stay consistent without coordination because every read starts with a conditional GET of the manifest — usually a cheap 304. Compaction is done once by whoever holds a lease and published into the log, so replicas download compacted packs instead of repacking. And because the WAL is the truth, you get complete provenance for free: every push and every repack, replayable to any point.

Built for Monorepos on Small Machines
#

Where walgit goes beyond the Continuity paper is the part aimed at Shopify’s actual problem — serving a monorepo from machines smaller than the repository:

  • Remote reader — refs, web pages, and API answers for a repository whose packs will never fit on the instance are served over HTTP range requests straight from the bucket.
  • History pack — commits and trees stay local while blobs stay in the bucket, so log and blame work without downloading the world.
  • bundle-uri — fresh clones and catch-ups are served as static bundle chains (weekly fulls, chained dailies and hourlies, cut as a pure function of the WAL) handed out by the bucket or a CDN. Clone traffic leaves the server entirely.

Maintenance — checkpoints, bundle builds, geometric compaction, connectivity audits, repairs — runs as one loop that computes desired state from (config, WAL) each pass and does one bounded unit of the most important missing work. An outage leaves no holes; a deleted artifact is simply “missing” and gets rebuilt identically. Self-healing by construction.

Why Now: The Forge Is Being Unbundled
#

The timing is not subtle. walgit landed exactly one week after Cursor opened the beta of Origin, its agent-native git forge — a launch that coincided, awkwardly for GitHub, with a multi-hour outage affecting Actions, the API, and Copilot. Lütke has been explicit that Cursor’s Continuity post was the inspiration; Cursor kept its engine proprietary, and walgit is the unlicensed, build-it-yourself version of the same idea. Coming days after the SpaceX–Cursor acquisition reframed who owns the AI coding stack, the message from two very different companies is the same: the teams building agentic development tools no longer trust classic Git hosting to survive the write volume that agent-based systems are about to produce. GitHub’s own response has been to evolve the workflow layer — stacked pull requests being the latest example — while Cursor and now Lütke are attacking the storage layer underneath it.

The Honest Caveats
#

A few things worth flagging before you point your monorepo at a bucket:

  • It’s a weekend project, however impressive. The design docs are unusually thorough (docs/CONTRACT.md, docs/INTEGRITY.md, docs/ROUNDTRIPS.md), the test suite is real, and there’s a CI pipeline — but this is days old, single-author code. The bus factor is a CEO with a day job.
  • Your consistency is your object store’s consistency. The entire design rests on compare-and-swap semantics being exactly as strong as advertised. That’s solid on S3 and GCS today, but “S3-compatible” covers a lot of implementations, and the failure modes of a weak CAS here are lost or forked refs.
  • It’s a Git server, not a forge. No pull requests, no issues, no code review. walgit replaces the storage and serving layer — the thing under GitHub, not GitHub itself. You’d pair it with a review tool, or run it as the resilient backend underneath one.
  • The authorship debate is a distraction, but not an irrelevant one. Whether a CEO can ship this in a weekend is exactly the kind of question AI-assisted development has made unanswerable from the outside — and the answer matters less than the fact that the architecture is documented well enough to audit either way.

My Take
#

walgit matters less as a product than as a proof that the Continuity architecture — object storage as source of truth, servers as disposable caches, CAS as consensus — is now reproducible by one motivated person in a weekend. That’s a genuine shift: the hardest part of Git hosting used to be the distributed-systems plumbing that only GitHub, GitLab, and a handful of large companies could afford to operate, and it’s collapsing into “a binary and a bucket.” Don’t rip out your Git host; do clone the repo and read docs/reference/cursor-git-at-any-scale.md alongside the CONTRACT doc, because this storage pattern is going to show up everywhere agents write code faster than packfile-era infrastructure can absorb. The open question I keep coming back to: if the repository is just a log in a bucket, the forge — pull requests, review, permissions — becomes a replaceable view layer on top. GitHub’s moat was never Git. walgit is a weekend project that makes that uncomfortably concrete.

Open Source Chronicles - This article is part of a series.
Part : This Article

Related