--- type: "article" title: "WAL Git" newsletter: "Jordi Villar" newsletter_handle: "jordivillarcom" newsletter_url: "https://usecommune.com/n/jordivillarcom" author: "Jordi Villar (@jordi)" published: "2026-09-15T20:33:35.204Z" canonical_url: "https://usecommune.com/n/jordivillarcom/a/wal-git" markdown_url: "https://usecommune.com/n/jordivillarcom/a/wal-git.md" chat_url: "https://usecommune.com/n/jordivillarcom/a/wal-git/chat" source_url: "https://jordivillar.com/blog/walgit/" body_source: "imported" likes: 0 replies: 0 body_words: 2567 --- # WAL Git Read the full article in your browser [here](https://jordivillar.com/blog/walgit/) # WAL Git A few days ago I read [Git at any scale](https://cursor.com/blog/git-at-any-scale). Although I managed to follow the architecture and the ideas behind it, I couldn’t stop thinking about how difficult it would be to come up with a solution like this one. Whenever I come across an interesting article like this one, I need to understand it better. And the only way that works for me is implementing it and writing about the process. I have [followed](https://jordivillar.com//blog/the-algorithm-behind-shazam) [this method](https://jordivillar.com//blog/lazy-loading-images) [a few times](https://jordivillar.com//blog/memory-alignment) already. If you’ve already read the original article and understood the concepts behind it, feel free to skip this post. It adds no new ideas, only the real requests and objects behind them. ### The Prompt These days there is always a prompt. I have never shared this approach before, but this is how I work now. Instead of building everything from scratch and spending a lot of time on stitching everything together, I ask the following to my trusted LLM: > Hey Claude, I have read this blog: [https://cursor.com/blog/git-at-any-scale](https://cursor.com/blog/git-at-any-scale). I think I have followed it properly but I want to reproduce what they describe in there. I think the best way to learn everything is to you to prepare something like a coding interview where I have to implement some functions and fill the blanks. It’ll be ideal if at any point I can run some tests or even run a real git command to see how things start working together. > > The most interesting parts to me are how the WAL and the CAS work with S3 but also the compaction algorithm implementation. What do you think? Can you prepare something? The output was perfect. I just had to go through the code and fill in the blanks.[1](#user-content-fn-1) The generated tests are the perfect example of our starting point and how the programming exercises are organized. ``` Permissions Size User Date Modified Name .rw-r--r-- 5.7k jordi 29 Aug 10:17 conftest.py .rw-r--r-- 3.2k jordi 29 Aug 10:18 test_00_preflight.py .rw-r--r-- 4.6k jordi 30 Aug 12:20 test_01_objects.py .rw-r--r-- 8.7k jordi 29 Aug 10:19 test_02_wal.py .rw-r--r-- 9.4k jordi 29 Aug 10:20 test_03_compaction.py .rw-r--r-- 6.5k jordi 29 Aug 10:22 test_04_serve.py ``` The objects module is just an interface to interact with S3, nothing super interesting for our purposes but still nice to go through it. Especially these days, when we don’t write much code anymore. Let’s first try to understand how things are put together by looking at how the push process works. ### The Push A walgit node is split in two: processes and a local cache on the node, and the repository itself in S3. This is how a push moves between them: *Every push runs in the same order. The server first syncs the warm repo from S3 (a conditional GET of the index, then any packs it is missing) and only then runs git http-backend on it, all while holding `walgit.lock`. Everything inside the dashed box is a cache, and can be deleted without losing a single commit. The bucket is the repository.* I call the node’s local copy of the repository the “warm cache”. It’s no more than a local copy, but it acts as a cache: the ground truth lives on S3, and the copy can be regenerated from it whenever needed. To understand what happens during a push, let’s use an example where one or more commits are pushed to a repository with `git push http://.../ main`. This is what the person pushing sees: ``` git push http://.../ main Enumerating objects: 3, done. Counting objects: 100% (3/3), done. Writing objects: 100% (3/3), 206 bytes | 206.00 KiB/s, done. Total 3 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) remote: walgit: seq=3 entry=543d651c6f7a pack=sha256:71972b034287 261B index-version=3 To http://.../ 9965c5c..4e5911a main -> main ``` The `git` client makes two calls to the server: `GET //info/refs?service=git-receive-pack` No body. Git sends this on every push to get the repository’s refs before pushing any data. | \# | Who | What happens | | --- | --- | --- | | 1 | `server.py` | Calls `sync()` [(more details)](#the-sync) and recreates the repository from S3 into a warm local cache. | | 2 | `git http-backend`[2](#user-content-fn-2) | Is called with the `GET` request. ``` REQUEST_METHOD=GET PATH_INFO=/.git/info/refs QUERY_STRING=service=git-receive-pack CONTENT_TYPE="" CONTENT_LENGTH=0 GIT_PROJECT_ROOT= REMOTE_USER=anonymous stdin: ``` | | 3 | `git http-backend` | Prints the response to stdout. ``` Expires: Fri, 01 Jan 1980 00:00:00 GMT Pragma: no-cache Cache-Control: no-cache, max-age=0, must-revalidate Content-Type: application/x-git-receive-pack-advertisement ``` | | 4 | `server.py` | Turns the output into an HTTP response: `200 OK`. | Then comes the real push. `POST //git-receive-pack` | \# | Who | What happens | | --- | --- | --- | | 1 | `git` | Sends the ref update, then a packfile. ``` User-Agent git/2.55.0 Content-Type application/x-git-receive-pack-request Accept application/x-git-receive-pack-result Content-Length 446 <446 bytes: ref update + pack> ``` | | 2 | `server.py` | Takes the `walgit.lock`. Nobody else can perform a sync in this node until the current push is complete. | | 3 | `server.py` | Calls `sync()`. This time the warm cache is already built, but it still needs to verify if there are any changes to the repository. | | 4 | `git http-backend` | Is called with the `POST` request. ``` REQUEST_METHOD=POST PATH_INFO=/.git/git-receive-pack QUERY_STRING="" CONTENT_TYPE=application/x-git-receive-pack-request CONTENT_LENGTH=446 GIT_PROJECT_ROOT= REMOTE_USER=anonymous stdin: ``` | | 5 | `git http-backend` | Starts `git receive-pack`: it reads the ref update and the pack from stdin, indexes the pack into a quarantine directory, then runs the `pre-receive` hook. | | 6 | `pre-receive` | Reads one line on stdin containing the ref update. ``` GIT_QUARANTINE_PATH=/.git/./objects/tmp_objdir-incoming-1dWDqU GIT_DIR=. stdin: 9965c5c 4e5911a refs/heads/main ``` | | 7 | `pre-receive` | Also finds the packfile inside the quarantine directory. | | 8 | `pre-receive` | Proceeds to update the WAL [(more details)](#the-wal). | | 9 | `git` | Moves the quarantined objects into the repository and updates `refs/heads/main`. | | 10 | `post-receive` | Repoints HEAD and records the new ETag into `walgit-state.json`. | | 11 | `git http-backend` | Prints the response to stdout. ``` Expires: Fri, 01 Jan 1980 00:00:00 GMT Pragma: no-cache Cache-Control: no-cache, max-age=0, must-revalidate Content-Type: application/x-git-receive-pack-result ``` | | 12 | `server.py` | Turns the output into an HTTP response: `200 OK`. | The part I had the most trouble understanding was how the server interacts with the client. But once you understand how `git http-backend` and the hooks work, everything starts to make sense. It’s a clever trick that allows the server to act as a proxy without needing to implement a custom server that speaks the Git protocol. The most important part after seeing how everything is wired together is of course the WAL, the central idea behind Cursor’s post. ### The WAL The WAL (Write-Ahead Log) is formed by three different objects in S3: the packfile, the entry, and the index. Only the latter is mutable, and it’s the one that makes the other two visible. Writing a pack or an entry changes nothing observable. They sit in the bucket, correct and unreferenced, and no reader has any way to find them. The visibility is managed in a single place, a pointer appearing inside `index.json`. That’s why the only operation that needs to be atomic is a single small PUT to that file. S3 makes it atomic with a conditional write, a compare-and-swap (CAS): the PUT only succeeds if the index still has the ETag you read. *Steps 1 and 2 are unconditional, invisible writes. Step 3 is the only atomic operation in the system, and it is what makes the chain `index → entry → pack` reachable. If something fails between 2 and 3, the first two writes are just garbage the garbage collector (GC) will clean later.* Let’s follow the same approach we have taken while analyzing [the push](#the-push). Once the `pre-receive` hook is triggered and detects a packfile, it follows these steps: | \# | Who | What happens | | --- | --- | --- | | 1 | `pre-receive` | Puts the pack to S3 `objects/sha256/…/71972b0342…` ``` call PutObject key repos//objects/sha256/71/97/71972b0342872cb8e0cf3d7587c37270339950a5fbca645acdac5cec5278b7f6 conditions If-None-Match: * ``` | | 2 | `pre-receive` | Puts the entry to S3 `wal/543d651c6f….json` ``` call PutObject key repos//wal/543d651c6f7a4cb185ec39e39bf1dc42.json conditions none ``` | | 3 | `pre-receive` | Gets the index object from S3. ``` call GetObject key repos//index.json conditions none ETag returned "ee28db1389164078d8f3873e9b3641c1" ``` | | 4 | `pre-receive` | Calls `apply_push` to check each ref and apply the changes, adding the new entry to the index. | | 5 | `pre-receive` | Puts the index object to S3. ``` call PutObject key repos//index.json conditions If-Match: "ee28db1389164078d8f3873e9b3641c1" ETag returned "a0ef3e595968827261a3dbfb659e4183" ``` | | 6 | `post-receive` | Records the new index ETag in `walgit-state.json`, so this node’s next sync gets a 304 instead of re-downloading the index. | The index will look like this after the push: ``` // repos//index.json, ETag: "a0ef3e595968827261a3dbfb659e4183" { "repo_id": "", "version": 3, "next_seq": 4, "base": null, "entries": [ // seq 1 and 2, from earlier pushes { "seq": 3, "entry_id": "543d651c6f7a4cb185ec39e39bf1dc42", "key": "repos//wal/543d651c6f7a4cb185ec39e39bf1dc42.json", "pack": "sha256:71972b0342872cb8e0cf3d7587c37270339950a5fbca645acdac5cec5278b7f6", "pack_size": 261 } ], "refs": { "refs/heads/main": "4e5911a3f5c75f56ef21ab108a638e6f20337514" } } ``` As always, this is the happy path. When updating the WAL we can find two main errors: - `CasConflict`: The index.json has been modified by another process, and ETag doesn’t match anymore. We need to re-download the index, apply the changes, and upload it again. - `RefConflict`: The ref has been moved to a value the client didn’t anticipate. We can’t recover from this, so we need to fail. The client can then fetch, rebase, and push again. It’s easier to understand with an example. Imagine two walgit nodes serving two different clients. Both read the same ETag and both push to `refs/heads/main`. The loser hits both conflicts on the way down. *The `CasConflict` after losing isn't the interesting part. The re-validation after it is. B's retry re-runs `apply_push` against the new index, discovers that `main` is no longer pointing to the expected commit, and stops. Without that second check B would append happily and A's commit would vanish.* Note that files uploaded by node B are left behind. They are not a problem since they are not visible to anybody. They will be cleaned up by the garbage collector. You might be wondering why retry after a `CasConflict` at all if it ends in a `RefConflict`. The case here is that both nodes were trying to apply changes on the same ref (`main`), but if one had been pushing to `main` and the other to a feature branch, the retry would have succeeded. It’d look something like this: *Same race, but B pushes to a different branch. The `CasConflict` still happens, but when B's retry re-runs `apply_push` against the new index, the ref it updates hasn't moved, so its second compare-and-swap succeeds with `seq=4`.* Now that we understand how the WAL works and how it’s updated while being durable and consistent, let’s dive into the other key part of the described solution, the sync process. ### The Sync The sync process is responsible for rebuilding the repository from the WAL and the files stored in S3. Without it, the repository would be either centralized in a single node or distributed without any consistency and synchronization. *Packs travel left to right, one at a time: S3, a temporary file, then git's own pack directory (3, 4). Refs travel separately, straight from `index.json`, and only once every pack is in (5). walgit never writes into `objects/pack` itself: git does, and git decides the file names.* Back to the original push: its first request, the `GET info/refs` from [the push](#the-push), makes `server.py` call `sync()`. When the node has no warm cache, it must rebuild the repository entirely from the WAL. | \# | Who | What happens | | --- | --- | --- | | 1 | `server.py` | Runs `git init --bare` to create an empty repository. | | 2 | `server.py` | Makes sure the hooks (`pre-receive` & `post-receive`) are set up. | | 3 | `server.py` | Loads `walgit-state.json` to get the ETag. Empty in this case. | | 4 | `server.py` | Gets `index.json` from S3. ``` call GetObject key repos//index.json conditions none ETag returned "ee28db1389164078d8f3873e9b3641c1" ``` | | 5 | `server.py` | The index lists two packs and ref main to `9965c5c`. ``` new ETag "ee28db1389164078d8f3873e9b3641c1" refs refs/heads/main = 9965c5c2d0824edd316c301f33507d26cfab9da2 packs sha256:3ae211fdb444946d43f0aa61c323368122271d2f04a708a3272da13b6a13ddca sha256:2389c2c6eb6f9654570f8e24b31d1d22616acbeeea8aca6f8525f5710119667d ``` | | 6 | `server.py` | Creates a temporary directory to store the packfiles. | | 7 | `server.py` | Gets object `3ae211fdb444…` from S3. ``` call GetObject key repos//objects/sha256/3a/e2/3ae211fdb444946d43f0aa61c323368122271d2f04a708a3272da13b6a13ddca conditions none to /tmp/walgit-pack-nbtvqnif/3ae211fdb444946d43f0aa61c323368122271d2f04a708a3272da13b6a13ddca.pack ``` | | 8 | `git index-pack` | Indexes the pack into the warm cache. ``` command git index-pack --stdin --fix-thin (run inside the repository) git printed packb36ce02f48e09f4027c45765fb0336d7540144ff files written pack-b36ce02f48e09f4027c45765fb0336d7540144ff.idx 1156 B pack-b36ce02f48e09f4027c45765fb0336d7540144ff.pack 219 B pack-b36ce02f48e09f4027c45765fb0336d7540144ff.rev 64 B ``` | | 9 | `server.py` | Same with the second pack. | | 10 | `server.py` | Temporary folder gets removed. | | 11 | `git update-ref` | Points `refs/heads/main` → `9965c5c`. | | 12 | `git symbolic-ref` | Points `HEAD` → `refs/heads/main`. | | 13 | `server.py` | Writes to `walgit-state.json` with `ETag="ee28db1389164078d8f3873e9b3641c1"`. | Once you have the warm cache, checking for updates is a single conditional GET: the node asks for `index.json` with `If-None-Match` set to the ETag it stored in `walgit-state.json`. A 304 means nothing changed, so it serves straight from disk. A 200 means someone pushed, so it downloads the missing packs, moves the refs, and saves the new ETag. ### Closing Remarks After reading Cursor’s post I thought I understood it. Implementing it showed me what I had missed: how `git http-backend` and the hooks let a plain HTTP server take pushes, and why a `CasConflict` is worth retrying while a `RefConflict` is not. The whole design fits in two ideas. The bucket is the repository, and nodes are caches that can disappear at any moment. And every push is ordered by one conditional PUT on one small JSON object. Everything else is arranged so that losing that race is cheap. There are a few things pending before I can call this project done. But they are secondary and won’t add much value to understanding how the general approach works. I leave the compaction and other mechanisms as something to be implemented later. [The code is on GitHub](https://github.com/jrdi/playground/tree/main/walgit) if you want to try the exercises yourself. ## Footnotes 1. Models evolve fast. Last time I followed this approach the output was a project with blanks to be filled. This time the model generated a fully functional version, a green test suite, removed some parts that I will need to implement myself with a guided approach, and a tool to peek into the working solution if I’m stuck. [See the initial version](https://github.com/jrdi/playground/tree/f2666b898d389ec97b76df5e4bcf13f117ea99e2/walgit). [↩](#user-content-fnref-1) 2. A program that ships with `git` and speaks CGI. It felt like ancient technology to me, so here’s how it works: for each request, `server.py` starts it, describes the request in environment variables, pipes the body to its stdin, and forwards whatever it prints. It reads `PATH_INFO` to get the repository and the service, and runs git’s server program on that directory. [↩](#user-content-fnref-2) *** ## Discussion No replies yet.