AdvancedLesson 21 of 306 min read

How Git Stores Things: Objects, Packs and Reachability

Look inside the .git folder: four kinds of object, why identical text is stored once, what a pack is, and what dangling really means.

In this lesson you will

  • Name the four kinds of Git object
  • Explain why identical files are stored once
  • Say what git gc does
  • Explain reachable and dangling objects

About 30 minutes

“Does Git copy everything every time?”

Dev asks Nia a fair question: if the garden project has five hundred commits, is Git keeping five hundred copies of every file? If so, the folder should be enormous, and it is not. The answer explains why Git is fast, why committed work is so hard to lose, and why some things, such as a leaked secret, are so hard to take back. At the Advanced level it pays to look inside the box once.

Four kinds of object

Everything Git stores in .git is an object, named by a fingerprint of its contents, as lesson 3 showed. There are four kinds. A blob holds the contents of one file, and nothing else, not even its name. A tree is a folder: a list of names, each pointing at a blob or at another tree. A commit points at one tree, at its parent commits, and carries the author and message. An annotated tag points at a commit and adds a message of its own. That is all. The whole history is a web of these four things.

The same text is stored once

Because an object is named by its contents, two files with the same text have the same name, so Git keeps only one. The first session creates three files, two of them identical. The listing prints each file with the short code of its blob: the two copies of the plan show the same code. The final count of stored files is 4: one commit, one tree, and two blobs, not three.

$ mkdir garden-notes
$ cd garden-notes
$ git init
Initialized empty Git repository in /home/nia/garden-notes/.git/
$ echo "Beans: row 1" > plan.txt
$ echo "Beans: row 1" > copy.txt
$ echo "Mon: Nia" > rota.txt
$ git add .
$ git commit -m "Add plan, copy and rota"
[main (root-commit) 27bb66f] Add plan, copy and rota
 3 files changed, 3 insertions(+)
 create mode 100644 copy.txt
 create mode 100644 plan.txt
 create mode 100644 rota.txt
$ git ls-tree --abbrev -r HEAD
100644 blob d0f2908	copy.txt
100644 blob d0f2908	plan.txt
100644 blob 024937f	rota.txt
$ find .git/objects -type f | wc -l
4

This is why a rename costs almost nothing, and why moving a file in a commit does not duplicate it. It is also why history is cheap. A new commit creates a new commit object and a new tree, and new blobs only for files whose text actually changed. Everything else is pointed at, not copied.

Loose objects and packs

Each new object starts life as a small compressed file under .git/objects. These are called loose objects. After two more commits the session counts 10 of them. Then git gc, short for garbage collection, runs. It gathers loose objects into a single pack, storing similar versions of a file as differences from each other, and builds an index so that any object can be found fast. Git also runs this for you now and then, so you rarely need to. The count-objects command now reports 10 objects in one pack and none loose.

$ echo "Tue: Dev" >> rota.txt
$ git commit -am "Add Tuesday"
[main 6d486e1] Add Tuesday
 1 file changed, 1 insertion(+)
$ echo "Wed: Asha" >> rota.txt
$ git commit -am "Add Wednesday"
[main ea70ef0] Add Wednesday
 1 file changed, 1 insertion(+)
$ find .git/objects -type f | wc -l
10
$ git gc -q
$ ls .git/objects/pack | wc -l
3
$ git count-objects -v
count: 0
size: 0
in-pack: 10
packs: 1
size-pack: 2
prune-packable: 0
garbage: 0
size-garbage: 0

The pack folder holds 3 files, an index, the pack itself, and a helper index. Their long names are fingerprints, so the lesson prints only how many there are. Nothing was lost: the log, the files and every old version still read back exactly the same. Only the packaging changed.

Reachable and unreachable

An object is reachable if you can get to it by starting at a branch, a tag or another reference and following pointers. Everything else is unreachable. When you reset a branch back one commit, the commit you left behind becomes unreachable, but it is not deleted. It stays in the store until a cleanup removes it, which by default only happens to unreachable objects after a grace period of a couple of weeks. The reflog, from lesson 5, also keeps pointing at recent positions. That double safety net is why committed work is so hard to lose by accident.

$ git reset -q --hard HEAD~1
$ git fsck --no-reflogs | cut -d' ' -f1-2
dangling commit
$ git fsck
$ git log --oneline
6d486e1 Add Tuesday
27bb66f Add plan, copy and rota

The check command with the no-reflogs option lists objects that nothing points at. It found one dangling commit: the Wednesday commit that the reset left behind. The ordinary check, which counts the reflog as a pointer, found nothing wrong. Only the commit is listed, because its tree and file are still reached through it.

Common mistake: “I deleted it, so it is gone”

Believing a deleted file or commit is gone for good. Until old objects are pruned, and until every copy of the repository has been cleaned, the data is still stored.

Your counts may differ

The files are made up. File counts, pack file counts and sizes depend on your Git version and settings, so your numbers may differ. The idea does not: content is stored by fingerprint, and unreachable objects survive for a while.

In a practice repository, make two commits, count the files under the objects folder, run gc, and count again. Then reset back one commit and find the dangling commit with the check command.

Practice for this lesson

Check what you learned

Five quick questions. Pick an answer and you will see straight away whether it is right and why.

What are the four kinds of Git object?

  1. File, folder, branch and remote
  2. Blob, tree, commit and annotated tag
  3. Add, commit, push and pull
  4. Index, log, stash and hook

A blob holds file contents, a tree is a folder, a commit points at a tree, and a tag labels a commit.

Why were only four objects stored for three files in the first commit?

  1. Git ignores small files
  2. Git stores only two files per commit
  3. The commit was compressed to nothing
  4. Two files had identical text, so they share one blob

Objects are named by their contents, so identical text is stored once.

What does git gc do to loose objects?

  1. Gathers them into a pack, storing similar versions as differences
  2. Deletes them
  3. Uploads them
  4. Renames them

The loose count went to none and the pack held every object.

What is an unreachable object?

  1. One that is encrypted
  2. One on a remote
  3. One that no branch, tag or other reference can reach
  4. One that is too large

The commit left behind by the reset was unreachable.

Why is committed work hard to lose by accident?

  1. Git copies it to GitHub
  2. Git locks the folder
  3. Commits cannot be changed
  4. Old objects and the reflog stay around for a while after a reset

The dangling commit was still in the store after the reset.

Summary

  • Git stores blobs, trees, commits and tags, each named by a fingerprint of its contents.
  • Identical text is stored once, so history is cheap.
  • git gc packs loose objects and stores similar versions as differences.
  • Unreachable objects survive for a grace period, so committed work is hard to lose.

Next lesson

Lesson 22 shows two power tools: cherry-pick and bisect.

Questions

Do I ever need to run git gc myself?

Rarely. Git runs it automatically when it judges that a repository needs it. Running it by hand is mainly useful for learning, or for a large repository you want to compact.

Is a dangling commit a problem?

No. It is a commit nothing points at. It is normal after a reset, and it is also how you find work you thought you had lost.

Why do my object counts differ from the lesson?

Counts depend on your Git version, settings and history. The ideas are the same.

Progress is saved in this browser.