How Cursor beat Git's scalability shortcomings

Developers recently burned by GitHub’s system outages should take note that there are other ways to manage Git at scale.

One approach, recently put into action by Cursor, is to build the distributed version control system on object storage.

A recent post from Cursor principal systems engineer Vicent Martí explains how the SpaceX subsidiary worked through its scaling issues with the notoriously fickle Git distributed version control system.

The post explains how Cursor arrived at an architecture for its own Git-based repository service called Origin, which is powered by an internal engine called Continuity.

A beta of the service is available with paid Cursor plans.

“Agents have fundamentally changed the way we work with software, and in many ways they've made this situation worse.

More code, more PRs, more CI runs.

Version control is at the core of all of this, and it is possibly the hardest thing to change overnight,” Martí wrote.

Martí speaks from experience, having worked at GitHub through much of the last decade when the company arrived at its own current architecture for managing Git.

Synchronization is a bitch Git creator Linus Torvalds designed his software to work as a content-addressable data store, where all the objects are stored and indexed by the SHA-1 hash of their contents.

Git sees a repository as a directed acyclic graph (DAG), with each commit being a node in a graph of nodes all connected by pointers.

A Git server can look up an object directly by its SHA, but if it doesn’t have the SHA, then it “must actually walk the DAG step by step,” Martí noted.

A client may just want to fetch or clone a consolidated packfile of the repository, or even get a list of recent changes, but to fulfill these requests, the server must traverse the entire graph only to assemble the necessary objects.

Now, imagine providing such a service for over 400 million repositories, and you’ll get an idea of the scale at which GitHub operates (or struggles to do).