[DEPRECATED] archive-format

car - Certified ARchives

IMPORTANT! This work has been superseded by https://github.com/ipld/specs/blob/a3c982518232b79123af2a2cf5e8642162c62524/block-layer/content-addressable-archives.md

(WIP -- quick thought dump)

A car for your data!

car is an archive format, like tar. It is designed to improve upon some deficiencies in tar as well as add fundamental improvements. The deatils of car are still being worked on, but the design space is already well scoped. car is a synthesis of good ideas. Many other archive formats introduced some of its improvements. car is different as it was introduced mainly to work with IPFS and related protocols, like IPLD. But car is useful beyond IPFS.

This is a discussion repo. That means that all of the work gets done in the issues.

Table of Contents

• File System Improvements:
• Fundamentally New Features:
• Whishlist:
• Related Work:
• Contribute
  • Want to hack on IPFS?
• License

File System Improvements:

• Seeking Optimized: finding object within a car file should be easy and fast. We can do this via an index with in-file byte offsets.
• UNIX/POSIX files: full unix + posix files compliance.
• Extended Attributes: support for xattrs and other extensions.
• Compression Optimized: support for object-level and archive-level compression. Modular to support new compression protocols.
• Encryption Optimized: support for object-level and archive-level encryption (through AEADs / NaCl/Box)

Fundamentally New Features:

• Data Structures: support for archiving more than files-- arbitrary datastructures (first-class JSON, XML, IPLD support)
• Merkle-Linked: all objects in a car file are merkle-linked -- hash-linked -- together. this means the whole car file is a merkle-dag.
• Digital Signatures: car has first-class support for Digital Signatures, to verify the authorship and certification of its content.

Whishlist:

• Repo Friendliness: make it easy, efficient, and useful to use car archives as on-disk repositories for other tools.
• Authenticated Datastructures: define all car operations as an authenticated datastructure, to have secure, untrusted computing. Merkle-linking and Digital Signatures get us most of the way there.

Related Work:

• continuity: https://github.com/stevvooe/continuity
• asar: https://github.com/electron/asar
• IPFS: https://github.com/ipfs/ipfs
• IPLD: https://github.com/ipfs/specs/tree/master/merkledag/ipld.md
• ipfs/unixfs: https://github.com/ipfs/go-ipfs/tree/master/unixfs
• lambda-auth: http://amiller.github.io/lambda-auth/

Contribute

Feel free to join in. All welcome.

Want to hack on IPFS?

License

MIT