1
"FSFS" is the name of a Subversion filesystem implementation, an
2
alternative to the original Berkeley DB-based implementation. See
3
http://subversion.tigris.org/ for information about Subversion. This
4
is a propaganda document for FSFS, to help people determine if they
5
should be interested in using it instead of the BDB filesystem.
10
* Write access not required for read operations
12
To perform a checkout, update, or similar operation on an FSFS
13
repository requires no write access to any part of the repository.
15
* Little or no need for recovery
17
An svn process which terminates improperly will not generally cause
18
the repository to wedge. (See "Note: Recovery" below for a more
19
in-depth discussion of what could conceivably go wrong.)
21
* Smaller repositories
23
An FSFS repository is smaller than a BDB repository. Generally, the
24
space savings are on the order of 10-20%, but if you do a lot of work
25
on branches, the savings could be much higher, due to the way FSFS
26
stores deltas. Also, if you have many small repositories, the
27
overhead of FSFS is much smaller than the overhead of the BDB
30
* Platform-independent
32
The format of an FSFS repository is platform-independent, whereas a
33
BDB repository will generally require recovery (or a dump and load)
34
before it can be accessed with a different operating system, hardware
35
platform, or BDB version.
37
* Can host on network filesystem
39
FSFS repositories can be hosted on network filesystems, just as CVS
40
repositories can. (See "Note: Locking" for caveats about
45
FSFS is careful to match the permissions of new revision files to the
46
permissions of the previous most-recent revision, so there is no need
47
to worry about a committer's umask rendering part of the repository
48
inaccessible to other users. (You must still set the g+s bit on the
49
db directories on most Unix platforms other than the *BSDs.)
51
* Standard backup software
53
An FSFS repository can be backed up with standard backup software.
54
Since old revision files don't change, incremental backups with
55
standard backup software are efficient. (See "Note: Backups" for
58
(BDB repositories can be backed up using "svnadmin hotcopy" and can be
59
backed up incrementally using "svnadmin dump". FSFS just makes it
62
* Can split up repository across multiple spools
64
If an FSFS repository is outgrowing the filesystem it lives on, you
65
can symlink old revisions off to another filesystem.
67
* More easily understood repository layout
69
If something goes wrong and you need to examine your repository, it
70
may be easier to do so with the FSFS format than with the BDB format.
71
(To be fair, both of them are difficult to extract file contents from
72
by hand, because they use delta storage, and "db_dump" makes it
73
possible to analyze a BDB repository.)
75
* Faster handling of directories with many files
77
If you are importing a tree which has directories with many files in
78
it, the BDB repository must, by design, rewrite the directory once for
79
each file, which is O(n^2) work. FSFS appends an entry to the
80
directory file for each change and then collapses the changes at the
81
end of the commit, so it can do the import with O(n) work.
83
Purely as a matter of implementation, FSFS also performs better
84
caching, so that iterations over large directories are much faster for
85
both read and write operations. Some of those caching changes could
86
be ported to BDB without changing the schema.
88
* (Fine point) Fast "svn log -v" over big revisions
90
In the BDB filesystem, if you do a large import and then do "svn log
91
-v", the server has to crawl the database for each changed path to
92
find the copyfrom information, which can take a minute or two of high
93
server load. FSFS stores the copyfrom information along with the
94
changed-path information, so the same operation takes just a few
97
* (Marginal) Can give insert-only access to revs subdir for commits
99
In some filesystems such as AFS, it is possible to give insert-only
100
write access to a directory. If you can do this, you can give people
101
commit access to an FSFS repository without allowing them to modify
102
old revisions, without using a server.
104
(The Unix sticky bit comes close, but people would still have
105
permission to modify their own old revisions, which, because of delta
106
storage, might allow them to influence the contents of other people's
107
more recent revisions.)
112
Most of the downsides of FSFS are more theoretical than practical, but
113
for the sake of completeness, here are all the ones I know about:
115
* More server work for head checkout
117
Because of the way FSFS stores deltas, it takes more work to derive
118
the contents of the head revision than it does in a BDB filesystem.
119
Measurements suggest that in a typical workload, the server has to do
120
about twice as much work (computation and file access) to check out
121
the head. From the client's perspective, with network and working
122
copy overhead added in, the extra time required for a checkout
123
operation is minimal, but if server resources are scarce, FSFS might
124
not be the best choice for a repository with many readers.
128
Although FSFS commits are generally faster than BDB commits, more of
129
the work of an FSFS commit is deferred until the final step. For a
130
very large commit (tens of thousands of files), the final step may
131
involve a delay of over a minute. There is no user feedback during
132
the final phase of a commit, which can lead to impatience and, in
133
really bad cases, HTTP client timeouts.
135
* Lower commit throughput
137
Because of the greater amount of work done during the final phase of a
138
commit, if there are many commits to an FSFS repository, they may
139
stack up behind each other waiting for the write lock, whereas in a
140
BDB repository they would be able to do more of their work in
145
FSFS was only recently implemented. It is new in the Subversion 1.1
146
release, and has received only a moderate amount of field testing.
148
* Big directories full of revision files
150
Each revision in an FSFS repository corresponds to a file in the
151
db/revs directory and another one in the db/rev-props directory. If
152
you have many revisions, this means there are two directories each
153
containing many files. Though some modern filesystems perform well on
154
directories containing many files (even if they require a linear
155
search for files within a directory, they may do well on repeated
156
accesses using an in-memory hash of the directory), some do not.
158
Subversion 1.2 may address this issue by optionally organizing
159
revision files into subdirectories.
161
* (Developers) More difficult to index
163
Every so often, people propose new Subversion features which require
164
adding new indexing to the repository in order to implement
165
efficiently. Here's a little picture showing where FSFS lies on the
166
indexing difficulty axis:
168
Ease of adding new indexing
169
harder <----------------------------------> easier
172
With a hypothetical SQL database implementation, new indexes could be
173
added easily. In the BDB implementation, it is necessary to write
174
code to maintain the index, but transactions and tables make that code
175
relatively straightforward to write. In a dedicated format like FSFS,
176
particularly with its "old revisions never change" constraint, adding
177
new indexing features would generally require a careful design
183
FSFS support is new in Subversion 1.1. If you are running a
184
Subversion 1.0.x release, you will need to upgrade the server (but not
185
the client, unless you are using file:/// access).
187
Once you've gotten that out of the way, using FSFS is simple: just
188
create your repositories with "svnadmin create --fs-type=fsfs PATH".
189
Or, build Subversion without Berkeley DB support, and repositories
190
will be created with FSFS by default.
195
If a process terminates abnormally during a read operation, it should
196
leave behind no traces in the repository, since read operations do not
197
modify the repository in any way.
199
If a process terminates abnormally during a commit operation, it will
200
leave behind a stale transaction, which will not interfere with
201
operation and which can be removed with a normal recursive delete
204
If a process terminates abnormally during the final phase of a commit
205
operation, it may be holding the write lock. The way locking is
206
currently implemented, a dead process should not be able to hold a
207
lock, but over a remote filesystem that guarantee may not apply.
208
Also, in the future, FSFS may have optional support for
209
NFSv2-compatible locking which would allow for the possibility of
210
stale locks. In either case, the write-lock file can simply be
211
removed to unblock commits, and read operations will remain
217
Locking is currently implemented using the apr_file_lock() function,
218
which on Unix uses fcntl() locking, and on Windows uses LockFile().
219
Modern remote filesystem implementations should support these
220
operations, but may not do so perfectly, and NFSv2 servers may not
223
It is possible to do exclusive locking under basic NFSv2 using a
224
complicated dance involving link(). It's possible that FSFS will
225
evolve to allow NFSv2-compatible locking, or perhaps just basic O_EXCL
226
locking, as a repository configuration option.
231
Naively copying an FSFS repository while a commit is taking place
232
could result in an easily-repaired inconsistency in the backed-up
233
repository. The backed-up "current" file could wind up referring to a
234
new revision which wasn't copied, or which was only partially
235
populated when it was copied.
237
The "svnadmin hotcopy" command avoids this problem by copying the
238
"current" file before copying the revision files. But a backup using
239
the hotcopy command isn't as efficient as a straight incremental
240
backup. FSFS may evolve so that "svnadmin recover" (currently a
241
no-op) knows how to recover from the inconsistency which might result
244
Naively copying an FSFS repository might also copy in-progress
245
transactions, which would become stale and take up extra room until
246
manually removed. "svnadmin hotcopy" does not copy in-progress
247
transactions from an FSFS repository, although that might need to
248
change if Subversion starts making use of long-lived transactions.
250
So, if you are using standard backup tools to make backups of an FSFS
251
repository, configure the software to copy the "current" file before
252
the numbered revision files, if possible, and configure it not to copy
253
the "transactions" directory. If you can't do those things, use
254
"svnadmin hotcopy", or be prepared to cope with the very occasional
255
need for manual repair of the repository upon restoring it from
added
removed
notes/fsfs
