fsdb

What is fsdb?

fsdb is a small collection of programs that speed up the creation of secure locate file indexes to the point where a completely up to date database is possible.

Where can I download fsdb?

fsdb is available for download from the SourceForge project page. Installation instructions are included with the distribution.

How does it work?

fsdb is split up into a four different parts (see diagram):

1. A kernel patch to add a hook into the dnotify mechanism.
2. A loadable kernel module to take advantage of the dnotify_hook by keeping a list of changed directory inodes. The kernel module exports a character device interface. The minor number is dynamically assigned by the "misc" driver (see /proc/misc). The major number is 10. If devfs is used the device file should automatically appear as /dev/misc/smartupdate.
3. A daemon (called fsupdate) which connects to the device file and reads the inodes from the kernel. It publishes a unix socket interface (/var/lib/fsdb/fsupdate.sock) and keeps the inodes stored in /var/lib/fsupdate.txt for persistent storage.
4. A program to generate file system databases, fsdb. This program takes many of the arguments that updatedb (slocate) does, specifically it process /etc/updatedb.conf and also understands the prune paths (-e) and prune filesystems (-f) switches. Additionally fsdb can produce an slocate database for use with slocate for high performance searching.

Additionally some scripts are provided for easy integration:

• cron job to update the database. I schedule it one every hour.
• init script to load the kernel module.
• ilocate script performs a fast update of the database before invoking locate

How much overhead is fsdb?

Computation

• The kernel module is very simple and is unlikely to add more overhead than a mouse driver, depending on how fast directories are changing in your system.
• The fsupdate daemon reads from the kernel modules, so it will take up some memory. On my system it takes up total size of 2252 kb with 456 kb resident. This memory usage could be reduced by rewriting fsupdate in C (it is currently a C++) program.
• On my system, the cron job takes about 1-3 seconds to update the database:

  # time /etc/cron.hourly/fsdb.cron
  real    0m2.357s
  user    0m1.860s
  sys     0m0.440s
  

• Intermediate locates are fast:

  $ time ilocate slocate.txt
  /home/jak/cvs/fsdb/doc/slocate.txt
  /home/jak/devel/fsdb.orig/doc/slocate.txt
  
  real    0m0.172s
  user    0m0.150s
  sys     0m0.020s
  

Disk Space

• The trade off is in the larger disk space usage. Fsdb's database has more information and is not as well compressed as slocates:

  # ls -l /var/lib/fsdb
  total 16412
  -rw-rw----    1 jak      jak      12170326 Jan 18 11:29 fs.db
  srwxrwx---    1 root     root            0 Jan 18 10:52 fsupdate.sock
  -rw-rw----    1 root     root           48 Jan 18 11:35 fsupdate.txt
  -rw-rw----    1 root     slocate       547 Jan 18 11:33 recent.db
  -rw-rw----    1 root     slocate   4595252 Jan 18 11:29 slocate.db
  

  As you can see, fsdb's database is nearly 3 times as big as slocates.

John Knottenbelt Home Page Last modified: Fri Jan 17 15:19:57 PST 2003