1
////////////////////////////////////////////////////////////////
3
GIT - the stupid content tracker
5
////////////////////////////////////////////////////////////////
7
"git" can mean anything, depending on your mood.
9
- random three-letter combination that is pronounceable, and not
10
actually used by any common UNIX command. The fact that it is a
11
mispronunciation of "get" may or may not be relevant.
12
- stupid. contemptible and despicable. simple. Take your pick from the
14
- "global information tracker": you're in a good mood, and it actually
15
works for you. Angels sing, and a light suddenly fills the room.
16
- "goddamn idiotic truckload of sh*t": when it breaks
18
This is a (not so) stupid but extremely fast directory content manager.
19
It doesn't do a whole lot at its core, but what it 'does' do is track
20
directory contents efficiently.
22
There are two object abstractions: the "object database", and the
23
"current directory cache" aka "index".
27
The object database is literally just a content-addressable collection
28
of objects. All objects are named by their content, which is
29
approximated by the SHA1 hash of the object itself. Objects may refer
30
to other objects (by referencing their SHA1 hash), and so you can
31
build up a hierarchy of objects.
33
All objects have a statically determined "type" aka "tag", which is
34
determined at object creation time, and which identifies the format of
35
the object (i.e. how it is used, and how it can refer to other
36
objects). There are currently four different object types: "blob",
37
"tree", "commit" and "tag".
39
A "blob" object cannot refer to any other object, and is, like the type
40
implies, a pure storage object containing some user data. It is used to
41
actually store the file data, i.e. a blob object is associated with some
42
particular version of some file.
44
A "tree" object is an object that ties one or more "blob" objects into a
45
directory structure. In addition, a tree object can refer to other tree
46
objects, thus creating a directory hierarchy.
48
A "commit" object ties such directory hierarchies together into
49
a DAG of revisions - each "commit" is associated with exactly one tree
50
(the directory hierarchy at the time of the commit). In addition, a
51
"commit" refers to one or more "parent" commit objects that describe the
52
history of how we arrived at that directory hierarchy.
54
As a special case, a commit object with no parents is called the "root"
55
object, and is the point of an initial project commit. Each project
56
must have at least one root, and while you can tie several different
57
root objects together into one project by creating a commit object which
58
has two or more separate roots as its ultimate parents, that's probably
59
just going to confuse people. So aim for the notion of "one root object
60
per project", even if git itself does not enforce that.
62
A "tag" object symbolically identifies and can be used to sign other
63
objects. It contains the identifier and type of another object, a
64
symbolic name (of course!) and, optionally, a signature.
66
Regardless of object type, all objects share the following
67
characteristics: they are all deflated with zlib, and have a header
68
that not only specifies their type, but also provides size information
69
about the data in the object. It's worth noting that the SHA1 hash
70
that is used to name the object is the hash of the original data
71
plus this header, so `sha1sum` 'file' does not match the object name
73
(Historical note: in the dawn of the age of git the hash
74
was the sha1 of the 'compressed' object.)
76
As a result, the general consistency of an object can always be tested
77
independently of the contents or the type of the object: all objects can
78
be validated by verifying that (a) their hashes match the content of the
79
file and (b) the object successfully inflates to a stream of bytes that
80
forms a sequence of <ascii type without space> + <space> + <ascii decimal
81
size> + <byte\0> + <binary object data>.
83
The structured objects can further have their structure and
84
connectivity to other objects verified. This is generally done with
85
the `git-fsck` program, which generates a full dependency graph
86
of all objects, and verifies their internal consistency (in addition
87
to just verifying their superficial consistency through the hash).
89
The object types in some more detail:
93
A "blob" object is nothing but a binary blob of data, and doesn't
94
refer to anything else. There is no signature or any other
95
verification of the data, so while the object is consistent (it 'is'
96
indexed by its sha1 hash, so the data itself is certainly correct), it
97
has absolutely no other attributes. No name associations, no
98
permissions. It is purely a blob of data (i.e. normally "file
101
In particular, since the blob is entirely defined by its data, if two
102
files in a directory tree (or in multiple different versions of the
103
repository) have the same contents, they will share the same blob
104
object. The object is totally independent of its location in the
105
directory tree, and renaming a file does not change the object that
106
file is associated with in any way.
108
A blob is typically created when gitlink:git-update-index[1]
109
(or gitlink:git-add[1]) is run, and its data can be accessed by
110
gitlink:git-cat-file[1].
114
The next hierarchical object type is the "tree" object. A tree object
115
is a list of mode/name/blob data, sorted by name. Alternatively, the
116
mode data may specify a directory mode, in which case instead of
117
naming a blob, that name is associated with another TREE object.
119
Like the "blob" object, a tree object is uniquely determined by the
120
set contents, and so two separate but identical trees will always
121
share the exact same object. This is true at all levels, i.e. it's
122
true for a "leaf" tree (which does not refer to any other trees, only
123
blobs) as well as for a whole subdirectory.
125
For that reason a "tree" object is just a pure data abstraction: it
126
has no history, no signatures, no verification of validity, except
127
that since the contents are again protected by the hash itself, we can
128
trust that the tree is immutable and its contents never change.
130
So you can trust the contents of a tree to be valid, the same way you
131
can trust the contents of a blob, but you don't know where those
132
contents 'came' from.
134
Side note on trees: since a "tree" object is a sorted list of
135
"filename+content", you can create a diff between two trees without
136
actually having to unpack two trees. Just ignore all common parts,
137
and your diff will look right. In other words, you can effectively
138
(and efficiently) tell the difference between any two random trees by
139
O(n) where "n" is the size of the difference, rather than the size of
142
Side note 2 on trees: since the name of a "blob" depends entirely and
143
exclusively on its contents (i.e. there are no names or permissions
144
involved), you can see trivial renames or permission changes by
145
noticing that the blob stayed the same. However, renames with data
146
changes need a smarter "diff" implementation.
148
A tree is created with gitlink:git-write-tree[1] and
149
its data can be accessed by gitlink:git-ls-tree[1].
150
Two trees can be compared with gitlink:git-diff-tree[1].
154
The "commit" object is an object that introduces the notion of
155
history into the picture. In contrast to the other objects, it
156
doesn't just describe the physical state of a tree, it describes how
157
we got there, and why.
159
A "commit" is defined by the tree-object that it results in, the
160
parent commits (zero, one or more) that led up to that point, and a
161
comment on what happened. Again, a commit is not trusted per se:
162
the contents are well-defined and "safe" due to the cryptographically
163
strong signatures at all levels, but there is no reason to believe
164
that the tree is "good" or that the merge information makes sense.
165
The parents do not have to actually have any relationship with the
168
Note on commits: unlike real SCM's, commits do not contain
169
rename information or file mode change information. All of that is
170
implicit in the trees involved (the result tree, and the result trees
171
of the parents), and describing that makes no sense in this idiotic
174
A commit is created with gitlink:git-commit-tree[1] and
175
its data can be accessed by gitlink:git-cat-file[1].
179
An aside on the notion of "trust". Trust is really outside the scope
180
of "git", but it's worth noting a few things. First off, since
181
everything is hashed with SHA1, you 'can' trust that an object is
182
intact and has not been messed with by external sources. So the name
183
of an object uniquely identifies a known state - just not a state that
184
you may want to trust.
186
Furthermore, since the SHA1 signature of a commit refers to the
187
SHA1 signatures of the tree it is associated with and the signatures
188
of the parent, a single named commit specifies uniquely a whole set
189
of history, with full contents. You can't later fake any step of the
190
way once you have the name of a commit.
192
So to introduce some real trust in the system, the only thing you need
193
to do is to digitally sign just 'one' special note, which includes the
194
name of a top-level commit. Your digital signature shows others
195
that you trust that commit, and the immutability of the history of
196
commits tells others that they can trust the whole history.
198
In other words, you can easily validate a whole archive by just
199
sending out a single email that tells the people the name (SHA1 hash)
200
of the top commit, and digitally sign that email using something
203
To assist in this, git also provides the tag object...
207
Git provides the "tag" object to simplify creating, managing and
208
exchanging symbolic and signed tokens. The "tag" object at its
209
simplest simply symbolically identifies another object by containing
210
the sha1, type and symbolic name.
212
However it can optionally contain additional signature information
213
(which git doesn't care about as long as there's less than 8k of
214
it). This can then be verified externally to git.
216
Note that despite the tag features, "git" itself only handles content
217
integrity; the trust framework (and signature provision and
218
verification) has to come from outside.
220
A tag is created with gitlink:git-mktag[1],
221
its data can be accessed by gitlink:git-cat-file[1],
222
and the signature can be verified by
223
gitlink:git-verify-tag[1].
226
The "index" aka "Current Directory Cache"
227
-----------------------------------------
228
The index is a simple binary file, which contains an efficient
229
representation of a virtual directory content at some random time. It
230
does so by a simple array that associates a set of names, dates,
231
permissions and content (aka "blob") objects together. The cache is
232
always kept ordered by name, and names are unique (with a few very
233
specific rules) at any point in time, but the cache has no long-term
234
meaning, and can be partially updated at any time.
236
In particular, the index certainly does not need to be consistent with
237
the current directory contents (in fact, most operations will depend on
238
different ways to make the index 'not' be consistent with the directory
239
hierarchy), but it has three very important attributes:
241
'(a) it can re-generate the full state it caches (not just the
242
directory structure: it contains pointers to the "blob" objects so
243
that it can regenerate the data too)'
245
As a special case, there is a clear and unambiguous one-way mapping
246
from a current directory cache to a "tree object", which can be
247
efficiently created from just the current directory cache without
248
actually looking at any other data. So a directory cache at any one
249
time uniquely specifies one and only one "tree" object (but has
250
additional data to make it easy to match up that tree object with what
251
has happened in the directory)
253
'(b) it has efficient methods for finding inconsistencies between that
254
cached state ("tree object waiting to be instantiated") and the
257
'(c) it can additionally efficiently represent information about merge
258
conflicts between different tree objects, allowing each pathname to be
259
associated with sufficient information about the trees involved that
260
you can create a three-way merge between them.'
262
Those are the three ONLY things that the directory cache does. It's a
263
cache, and the normal operation is to re-generate it completely from a
264
known tree object, or update/compare it with a live tree that is being
265
developed. If you blow the directory cache away entirely, you generally
266
haven't lost any information as long as you have the name of the tree
269
At the same time, the index is at the same time also the
270
staging area for creating new trees, and creating a new tree always
271
involves a controlled modification of the index file. In particular,
272
the index file can have the representation of an intermediate tree that
273
has not yet been instantiated. So the index can be thought of as a
274
write-back cache, which can contain dirty information that has not yet
275
been written back to the backing store.
281
Generally, all "git" operations work on the index file. Some operations
282
work *purely* on the index file (showing the current state of the
283
index), but most operations move data to and from the index file. Either
284
from the database or from the working directory. Thus there are four
287
1) working directory -> index
288
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
290
You update the index with information from the working directory with
291
the gitlink:git-update-index[1] command. You
292
generally update the index information by just specifying the filename
293
you want to update, like so:
295
git-update-index filename
297
but to avoid common mistakes with filename globbing etc, the command
298
will not normally add totally new entries or remove old entries,
299
i.e. it will normally just update existing cache entries.
301
To tell git that yes, you really do realize that certain files no
302
longer exist, or that new files should be added, you
303
should use the `--remove` and `--add` flags respectively.
305
NOTE! A `--remove` flag does 'not' mean that subsequent filenames will
306
necessarily be removed: if the files still exist in your directory
307
structure, the index will be updated with their new status, not
308
removed. The only thing `--remove` means is that update-cache will be
309
considering a removed file to be a valid thing, and if the file really
310
does not exist any more, it will update the index accordingly.
312
As a special case, you can also do `git-update-index --refresh`, which
313
will refresh the "stat" information of each index to match the current
314
stat information. It will 'not' update the object status itself, and
315
it will only update the fields that are used to quickly test whether
316
an object still matches its old backing store object.
318
2) index -> object database
319
~~~~~~~~~~~~~~~~~~~~~~~~~~~
321
You write your current index file to a "tree" object with the program
325
that doesn't come with any options - it will just write out the
326
current index into the set of tree objects that describe that state,
327
and it will return the name of the resulting top-level tree. You can
328
use that tree to re-generate the index at any time by going in the
331
3) object database -> index
332
~~~~~~~~~~~~~~~~~~~~~~~~~~~
334
You read a "tree" file from the object database, and use that to
335
populate (and overwrite - don't do this if your index contains any
336
unsaved state that you might want to restore later!) your current
337
index. Normal operation is just
339
git-read-tree <sha1 of tree>
341
and your index file will now be equivalent to the tree that you saved
342
earlier. However, that is only your 'index' file: your working
343
directory contents have not been modified.
345
4) index -> working directory
346
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
348
You update your working directory from the index by "checking out"
349
files. This is not a very common operation, since normally you'd just
350
keep your files updated, and rather than write to your working
351
directory, you'd tell the index files about the changes in your
352
working directory (i.e. `git-update-index`).
354
However, if you decide to jump to a new version, or check out somebody
355
else's version, or just restore a previous tree, you'd populate your
356
index file with read-tree, and then you need to check out the result
359
git-checkout-index filename
361
or, if you want to check out all of the index, use `-a`.
363
NOTE! git-checkout-index normally refuses to overwrite old files, so
364
if you have an old version of the tree already checked out, you will
365
need to use the "-f" flag ('before' the "-a" flag or the filename) to
366
'force' the checkout.
369
Finally, there are a few odds and ends which are not purely moving
370
from one representation to the other:
372
5) Tying it all together
373
~~~~~~~~~~~~~~~~~~~~~~~~
374
To commit a tree you have instantiated with "git-write-tree", you'd
375
create a "commit" object that refers to that tree and the history
376
behind it - most notably the "parent" commits that preceded it in
379
Normally a "commit" has one parent: the previous state of the tree
380
before a certain change was made. However, sometimes it can have two
381
or more parent commits, in which case we call it a "merge", due to the
382
fact that such a commit brings together ("merges") two or more
383
previous states represented by other commits.
385
In other words, while a "tree" represents a particular directory state
386
of a working directory, a "commit" represents that state in "time",
387
and explains how we got there.
389
You create a commit object by giving it the tree that describes the
390
state at the time of the commit, and a list of parents:
392
git-commit-tree <tree> -p <parent> [-p <parent2> ..]
394
and then giving the reason for the commit on stdin (either through
395
redirection from a pipe or file, or by just typing it at the tty).
397
git-commit-tree will return the name of the object that represents
398
that commit, and you should save it away for later use. Normally,
399
you'd commit a new `HEAD` state, and while git doesn't care where you
400
save the note about that state, in practice we tend to just write the
401
result to the file pointed at by `.git/HEAD`, so that we can always see
402
what the last committed state was.
404
Here is an ASCII art by Jon Loeliger that illustrates how
405
various pieces fit together.
433
checkout-index -u | | checkout-index
444
6) Examining the data
445
~~~~~~~~~~~~~~~~~~~~~
447
You can examine the data represented in the object database and the
448
index with various helper tools. For every object, you can use
449
gitlink:git-cat-file[1] to examine details about the
452
git-cat-file -t <objectname>
454
shows the type of the object, and once you have the type (which is
455
usually implicit in where you find the object), you can use
457
git-cat-file blob|tree|commit|tag <objectname>
459
to show its contents. NOTE! Trees have binary content, and as a result
460
there is a special helper for showing that content, called
461
`git-ls-tree`, which turns the binary content into a more easily
464
It's especially instructive to look at "commit" objects, since those
465
tend to be small and fairly self-explanatory. In particular, if you
466
follow the convention of having the top commit name in `.git/HEAD`,
469
git-cat-file commit HEAD
471
to see what the top commit was.
473
7) Merging multiple trees
474
~~~~~~~~~~~~~~~~~~~~~~~~~
476
Git helps you do a three-way merge, which you can expand to n-way by
477
repeating the merge procedure arbitrary times until you finally
478
"commit" the state. The normal situation is that you'd only do one
479
three-way merge (two parents), and commit it, but if you like to, you
480
can do multiple parents in one go.
482
To do a three-way merge, you need the two sets of "commit" objects
483
that you want to merge, use those to find the closest common parent (a
484
third "commit" object), and then use those commit objects to find the
485
state of the directory ("tree" object) at these points.
487
To get the "base" for the merge, you first look up the common parent
490
git-merge-base <commit1> <commit2>
492
which will return you the commit they are both based on. You should
493
now look up the "tree" objects of those commits, which you can easily
494
do with (for example)
496
git-cat-file commit <commitname> | head -1
498
since the tree object information is always the first line in a commit
501
Once you know the three trees you are going to merge (the one
502
"original" tree, aka the common case, and the two "result" trees, aka
503
the branches you want to merge), you do a "merge" read into the
504
index. This will complain if it has to throw away your old index contents, so you should
505
make sure that you've committed those - in fact you would normally
506
always do a merge against your last commit (which should thus match
507
what you have in your current index anyway).
511
git-read-tree -m -u <origtree> <yourtree> <targettree>
513
which will do all trivial merge operations for you directly in the
514
index file, and you can just write the result out with
517
Historical note. We did not have `-u` facility when this
518
section was first written, so we used to warn that
519
the merge is done in the index file, not in your
520
working tree, and your working tree will not match your
521
index after this step.
522
This is no longer true. The above command, thanks to `-u`
523
option, updates your working tree with the merge results for
524
paths that have been trivially merged.
527
8) Merging multiple trees, continued
528
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
530
Sadly, many merges aren't trivial. If there are files that have
531
been added.moved or removed, or if both branches have modified the
532
same file, you will be left with an index tree that contains "merge
533
entries" in it. Such an index tree can 'NOT' be written out to a tree
534
object, and you will have to resolve any such merge clashes using
535
other tools before you can write out the result.
537
You can examine such index state with `git-ls-files --unmerged`
540
------------------------------------------------
541
$ git-read-tree -m $orig HEAD $target
542
$ git-ls-files --unmerged
543
100644 263414f423d0e4d70dae8fe53fa34614ff3e2860 1 hello.c
544
100644 06fa6a24256dc7e560efa5687fa84b51f0263c3a 2 hello.c
545
100644 cc44c73eb783565da5831b4d820c962954019b69 3 hello.c
546
------------------------------------------------
548
Each line of the `git-ls-files --unmerged` output begins with
549
the blob mode bits, blob SHA1, 'stage number', and the
550
filename. The 'stage number' is git's way to say which tree it
551
came from: stage 1 corresponds to `$orig` tree, stage 2 `HEAD`
552
tree, and stage3 `$target` tree.
554
Earlier we said that trivial merges are done inside
555
`git-read-tree -m`. For example, if the file did not change
556
from `$orig` to `HEAD` nor `$target`, or if the file changed
557
from `$orig` to `HEAD` and `$orig` to `$target` the same way,
558
obviously the final outcome is what is in `HEAD`. What the
559
above example shows is that file `hello.c` was changed from
560
`$orig` to `HEAD` and `$orig` to `$target` in a different way.
561
You could resolve this by running your favorite 3-way merge
562
program, e.g. `diff3` or `merge`, on the blob objects from
563
these three stages yourself, like this:
565
------------------------------------------------
566
$ git-cat-file blob 263414f... >hello.c~1
567
$ git-cat-file blob 06fa6a2... >hello.c~2
568
$ git-cat-file blob cc44c73... >hello.c~3
569
$ merge hello.c~2 hello.c~1 hello.c~3
570
------------------------------------------------
572
This would leave the merge result in `hello.c~2` file, along
573
with conflict markers if there are conflicts. After verifying
574
the merge result makes sense, you can tell git what the final
575
merge result for this file is by:
577
mv -f hello.c~2 hello.c
578
git-update-index hello.c
580
When a path is in unmerged state, running `git-update-index` for
581
that path tells git to mark the path resolved.
583
The above is the description of a git merge at the lowest level,
584
to help you understand what conceptually happens under the hood.
585
In practice, nobody, not even git itself, uses three `git-cat-file`
586
for this. There is `git-merge-index` program that extracts the
587
stages to temporary files and calls a "merge" script on it:
589
git-merge-index git-merge-one-file hello.c
591
and that is what higher level `git merge -s resolve` is implemented
added
removed
Documentation/core-intro.txt
