TODO.txt

Optional:
  - Think about a cool name & rebrand everything?

Milestone 0.1.0:
  - create a README with some information and usage notes.
    (This should contain everything which is currently spread over USAGE,
     NOTES, and TODO, though pure TODOs can stay in this file.)
     - document build depencies (FUSE 2.6, SQLCipher version X?)

Milestone 1.0.0:
  - add a license (BSD-style, like SQLcipher? Check github for recommendation.)
    (would require copyright release!)
  - implement 'check' command (verify that it works on read only files)
    (once implemented, add it to teardown() in sqlfuse_test to verify that each
    test case leaves the filesystem in a consistent state, without dangling
    pointers and so on).

Milestone 1.1.0:
  - Support for symlinks.
  - Support for hardlinks.

Maybe later:
  - Generate a man-page.
  - Port to FreeBSD/OpenBSD.
  - Add a flag to override the cipher_page_size? (Would apply to all commands.)
  - Add a benchmark test
      - Read/write perfomance test (random/sequential reads/writes in a file)
      - A separate test which generates a "random" filesystem?
  - Use benchmark to decide on optimal block_size and cipher_page_size
    (it doesn't make sense to bump the schema version, since we cannot open the
     database with the wrong cipher_page_size, so we cannot auto-upgrade.)
  - Consider calling sqlfs_purge_all() in destroy rather than purging each open
    inode individually. (It would have the benefit that sqlfs_purge_all() is
    more efficient because it can use an index to find unlinked entries.)
  - Transparent compression of file data? Or maybe this can be implemented on a
    lower level (compressing the database file itself)?
  - Consider running `PRAGMA optimize` before/after `VACUUM` during compaction.
  - During compaction, consider rewriting large files using a larger blocksize.
    This could reduce the filesize overhead from ~13% with 4 KiB blocks to ~3%
    with 40 KiB blocks.

Decisions to document somewhere:
  - No multithreading support. It complicates everything, is not immediately
    necessary, and would not work well with SQLite (since SQLite serializes
    transactions by default, and we even open the database in exclusive mode,
    which prevents concurrent access).
  - ctime and atime are not stored. (They're not very useful.)
  - Each file has an associated block size (which can vary between files, though
    it's 4 KiB by default). File data is stored as a series of blocks, indexed
    from 0 to N. Blocks 0 through N - 1 have size `blocksize`; the N-th block
    is shorter if the file size is not a multiple of the blocksize. Blocks are
    never empty.
  - Sparse files are currently not supported. Maybe later!
  - For performance reasons, the inode PRIMARY KEY in the metadata table is not
    declared as AUTOINCREMENT. Because we always set the `generation` field to a
    fixed value, that means the filesystem cannot be exported over NFS! An
    alternative solution would be to store the creation timestamp in the
    metadata table, and use that as the generation number.
  - Directory reading: we use "next_name" instead of "last_name" as the
    continuation token (and "SELECT ... WHERE name >= next_name", instead of
    "WHERE name > last_name") to avoid having to copy strings in
    sqlfuse_readdir() to remember the last entry name.
  - We don't support the FUSE flush() function because all writes commit to the
    database anyway (they are flushed implicitly, though changes are not durable
    until fsync() is called).
  - Permission bits sticky/setuid/setgid (mask 07000) are currently not
    supported. Maybe later! (Careful: we should clear setuid/setgid bits when
    the owner/group of a file is changed.)
  - No tests for setuid()/setgid() because that (probably?) requires root
    privileges to test.
  - No implementation for FUSE open()/release(), because we implement stateless
    file I/O instead.
  - Permission checking is deferred to libfuse. It might not be safe to share
    the filesystem with others (i.e. using the -o allow_others mount option).
  - There is no separate command to inspect/dump the database. Just use the
    sqlcipher CLI for debugging.

Things I learned about fuse:
  - direct_io mode is not available in low-level mode. Without it (by default),
    file access goes through the kernel's page cache, which makes testing more
    difficult, but it has the benefit that reads are aligned to page boundaries.

    (The underlying sqlfs library would support direct I/O and unaligned read()/
    write() calls.)

  - The keep_cache flag allows the kernel to cache file data. The kernel will
    invalidate the cache automatically. This mode is applicable to filesystems
    where the underlying data cannot change except through the kernel API, which
    is true for us (but wouldn't be true for e.g. a network filesystem that can
    be modified by multiple users at once).

    (How does this relate to the kernel_cache mount option?)

  - TODO: go over other mount options? (For example, we do support returning
    inos in readdir().)

  - Lookup accounting: I/O may be performed on files and directories after they
    are deleted. To track which entries are still active, the kernel module
    maintains a lookup count per inode number.

      - The root inode (number 1) starts with lookup count 1.
      - Each time an inode number is sent to the kernel with fuse_reply_entry()
        or fuse_reply_create(), its lookup count increases by 1. (We don't
        implement create(), so we can ignore the second function.)
      - Each time an inode number is passed to forget(), its lookup count
        decreases by 1.
      - When an inode number's lookup count reaches zero, and its link count is
        zero, then it may be permanently deleted. (We implement this in
        sqlfs_purge(), which removes a file but only if its link count is zero.)
      - It's not guaranteed that all inode numbers are forgotten before
        destroy() is called (when the filesystem is unmounted), so on destroy,
        we should purge all entries with positive lookup count.