Jonathan Creamer

How a Small Team Shrank a Microsoft Monorepo by 94%

A Microsoft monorepo ballooned to 150GB. The culprit wasn't code, but a Git hashing bug from 2006 that saw different files as identical.

How a Small Team Shrank a Microsoft Monorepo by 94%
#1about 2 minutes

The scale of Microsoft's monorepo problem

A monorepo with 20 million lines of code grew from a manageable 2GB to an unworkable 150GB, prompting an investigation into its exponential growth.

#2about 3 minutes

How automated changelog tooling bloated the repository

The versioning tool Beach Ball generated thousands of changelog files, causing a separate versioning branch to swell to an enormous 130GB.

#3about 9 minutes

Discovering a Git hashing algorithm bug from 2006

A Git expert found that an old hashing algorithm only used the last 16 characters of a filename, causing collisions that prevented proper diffing of changelog files.

#4about 4 minutes

Implementing the new path walk algorithm to fix Git

The solution was a new "Path Walk" algorithm for `git push` and `git repack` that uses the full file path to avoid hash collisions and ensure correct diffing.

#5about 2 minutes

Applying the fix with new Git config and repack commands

Developers can enable the new algorithm for pushes via a `git config` setting and shrink local clones using the `git repack --use-path-walk` command.

#6about 2 minutes

Using the new `git survey` command to find large files

A new built-in command, `git survey`, was created to help developers identify large files, blobs, and binaries in their repository history.

#7about 3 minutes

Best practices for managing large repositories

Beyond the specific fix, general best practices like not checking in binaries and avoiding thousands of files in a single folder are crucial for repository health.

#8about 6 minutes

The broader impact on the open source community

The new algorithm has shown significant size reductions for other large monorepos like Chromium, and the fix is being upstreamed to benefit the entire Git community.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

Moving beyond headcount to solve business problems
04:27 MIN

Moving beyond headcount to solve business problems

What 2025 Taught Us: A Year-End Special with Hung Lee

Breaking down silos between HR, tech, and business
03:39 MIN

Breaking down silos between HR, tech, and business

What 2025 Taught Us: A Year-End Special with Hung Lee

Why corporate AI adoption lags behind the hype
03:28 MIN

Why corporate AI adoption lags behind the hype

What 2025 Taught Us: A Year-End Special with Hung Lee

Why HR struggles with technology implementation and adoption
04:22 MIN

Why HR struggles with technology implementation and adoption

What 2025 Taught Us: A Year-End Special with Hung Lee

Rapid-fire thoughts on the future of work
02:44 MIN

Rapid-fire thoughts on the future of work

What 2025 Taught Us: A Year-End Special with Hung Lee

Balancing the trade-off between efficiency and resilience
03:38 MIN

Balancing the trade-off between efficiency and resilience

What 2025 Taught Us: A Year-End Special with Hung Lee

Incentivizing automation with a 'keep what you kill' policy
05:18 MIN

Incentivizing automation with a 'keep what you kill' policy

What 2025 Taught Us: A Year-End Special with Hung Lee

Developing resilience by expanding your capacity for failure
04:57 MIN

Developing resilience by expanding your capacity for failure

What 2025 Taught Us: A Year-End Special with Hung Lee

Featured Partners

Related Videos

See all videos
Coffee With Developers - Kyle Daigle, COO of GitHub42:13

Coffee With Developers - Kyle Daigle, COO of GitHub

Kyle Daigle

Coffee with Developers - Scott Chacon on growing GitButler and the future of version control37:05

Coffee with Developers - Scott Chacon on growing GitButler and the future of version control

Scott Chacon

Git for Code Reviews24:19

Git for Code Reviews

Johannes Haux

GitLab CI pipelines for a whole company32:29

GitLab CI pipelines for a whole company

Martin Beránek

Reusing apps between teams and environments through Containers43:39

Reusing apps between teams and environments through Containers

Adrian Kosmaczewski

GitOps keeps focus on apps, not on infrastructure36:09

GitOps keeps focus on apps, not on infrastructure

Chad Carlson

Improving Developer Happiness with GitOps42:29

Improving Developer Happiness with GitOps

Lars Hesel Christensen & Basil Brunner

How to Optimize Non-Coding Time26:35

How to Optimize Non-Coding Time

Yishai Beeri

Related Articles

View all articles
Dev Digest 108 - Git off my cloud!
Welcome to another edition of the WeAreDevelopers Dev Digest. This time we have an interview with Sead Ahmetovic, CEO of of WeAreDevelopers and Scott Chacon, co-Founder of GitHub. They talk about careers, early coding days, developer communities, eva...
Dev Digest 108 - Git off my cloud!

From learning to earning

Jobs that call for the skills explored in this talk.