v2.6

Note!

This release bumps the on-disk minor version of littlefs from lfs2.0 -> lfs2.1.

This change is backwards-compatible, but after the first write with the new version, the image on disk will no longer be mountable by older versions of littlefs.

Read-only operations will not change the on-disk version, and the on-disk version can be explicitly bumped with lfs_fs_mkconsistent if desired.

This is the first on-disk minor version bump for littlefs, and is untested waters, so do open a GitHub issue if this creates any problems.

What's new?

• FCRCs - forward-looking erase-state CRCs (#497)

  Found by @pjsg, littlefs was relying on a bad assumption about the order/atomicity of prog operations to detect incomplete metadata commits. In the case of power-failure, this could lead to littlefs progging the same location multiple times without an erase, leading to best case fixable corruption, worst case significantly reduced data retention.

  The solution implemented here is to store a checksum of the next erased prog-size in every commit. This way any incomplete commits that modify the erased state can be detected.

  In theory this could be implemented in a forward-compatible way, but littlefs currently treats all CRC tags as commit-ending CRC tags, so only updated versions of littlefs will be able to read metadata blocks containing FCRCs. This is the reason for the on-disk minor version bump. On the bright-side, future CRC-related extensions will be forward-compatible.

• A complete overhaul of the testing/bench framework (#752)

  This does not affect users, but involves a long list of needed changes to make it easier to develop/bench littlefs:

  • Tests/benches now run in parallel, and much faster thanks to shared processes.
  • Tests are more reproducible, and narrow bugs can be reproduced locally repeatedly without leaving GDB.
  • More options for power-loss testing. Exhaustive testing of all 1 and 2-deep powerlosses have been added to CI.
  • More scripts related to perf measurements, plotting, tracing, etc.
  • A make help rule to help use/document the Makefile, which is sort of the entry point for littlefs development.
  • Added a CI rule to have @geky-bot comment on each PR with CI measurements (examples here). We'll see if this is an improvement over the GitHub statuses.

  The increased power-loss coverage has already found a couple of difficult to hit bugs (eba5553 and 0b11ce0), and the bench runner will be useful for evaluating littlefs's performance moving forward.

• Added a couple more asserts to help common implementation issues (#801, #802)

• Adopted Brent's algorithm for cycle detection (#809)

  This should significantly speed up cycle detection in the case the metadata linked-list contains a cycle.

• Added lfs_fs_mkconsistent as proposed by @kasper0 (#812)

  lfs_fs_mkconsistent allows users to front-load the internal consistency checks littlefs needs to run before it can write to the filesystem.

  Conveniently this also allows a way to force littlefs to write the new on-disk minor version if that is desired.

• A number of bug fixes, thanks to @ghost, @ajaybhargav, @colin-foster-in-advantage, @rvanschoren, @hgspbs, @sosthene-nitrokey, @kasper0, @Ldd309, and others (#800, #805, #811):

  • #800 - Fixed issues with truncate around block-aligned boundaries
  • #805 - Fixed issue where seeking to end-of-directory return LFS_ERR_INVAL
  • #811 - Fixed issue where lfs_fs_deorphan may run more than needed
  • eba5553 - Fixed issue where half-orphans can hide full orphans in nested power-loss situations
  • 0b11ce0 - Fixed incorrect calculation of extra space needed in mdir blocks
  • ba1c764 - Fixed issue where deorphan could get stuck circling between two half-orpans in power-loss situations

• And a number of links to useful community projects related to littlefs, thanks to @tniessen, @yomimono, @elpiel (#572, #718, #807):

  • littlefs-disk-img-viewer
  • chamelon
  • littlefs2-rust

  If you have a littlefs-related project or a project that uses littlefs, consider opening a PR. Eventually my plan is to move these to a user-contributable wiki.

Changes

	Code	Stack	Structs		Coverage
Default	16556 B	1432 B	772 B	Lines	2289/2469 lines
Readonly	5990 B	448 B	772 B	Branches	1174/1496 branches
Threadsafe	17362 B	1432 B	780 B		Benchmarks
Migrate	18240 B	1736 B	776 B	Readed	29369693876 B
Error-asserts	17192 B	1424 B	772 B	Proged	1482874766 B
				Erased	1568888832 B

259535e Added lfs_fs_mkconsistent
94d9e09 Fixed issue where lfs_fs_deorphan may run more than needed
363a8b5 Tweaked wording of littlefs2-rust link in README.md
e43d381 chore: add littlefs2 crate to README
3e25dfc Added FCRC tags and an explanation of how FCRCs work to SPEC.md
9e28c75 Bumped minor version to v2.6 and on-disk minor version to lfs2.1
4c93600 Added ability to bump on-disk minor version
ca0da3d Added compatibility testing on pull-request to GitHub test action
116332d Added tests for forwards and backwards disk compatibility
f0cc1db Tweaked changeprefix.py to not rename dir component in paths
bf045dd Tweaked link to littlefs-disk-img-viewer to go to github repo
b33a5b3 Fixed issue where seeking to end-of-directory return LFS_ERR_INVAL
384a498 Extend dir seek tests to include seeking to end of directory
b0a4a44 Added explicit assert for minimum block size of 128 bytes
aae897f Added an assert for truthy-preserving bool conversions
e57402c Added ability to revert to inline file in lfs_file_truncate
6dc18c3 Fixed block-boundary truncate issue
d5dc487 Expanded truncate tests to test more corner cases
24795e6 Add missing iterations in tests
7b151e1 Add test scenario for truncating to a block size
ba1c764 Fixed issue where deorphan could get stuck circling between two half-orphans
d1b254d Reverted removal of 1-bit counter threaded through tags
2f26966 Continued implementation of forward-crcs, adopted new test runners
b4091c6 Switched to separate-tag encoding of forward-looking CRCs
91ad673 Cleaned up a few additional commit corner cases
52dd830 Initial implementation of forward-looking erase-state CRCs
1278ec1 Adopted Brent's algorithm for cycle detection
c2147c4 Added --gdb-pl to test.py for breaking on specific powerlosses
801cf27 Tweaked/fixed a number of small runner things after a bit of use
1f37eb5 Adopted --subplot* in plot.py
cfd4e60 Added --subplot* to plotmpl.py
2d2dd8b Added plotmpl.py --github flag to match the website's foreground/background
b0382fa Added BENCH/TEST_PRNG, replacing other ad-hoc sources of randomness
d8e7ffb Changed lfs_emubd_get* -> lfs_emubd_*
cda2f6f Changed test_runner to run with -Pnone,linear by default
9b687dd Added make benchmarks/testmarks rules
c4b3e9d A couple of script changes after CI integration
9990342 Fixed Clang testing in CI, removed override vars in Makefile
4a20934 Fixed bench workflow + changeprefix issue in prefix releases
a659c02 Added a bot-generated PR-comment with a simple status table
397aa27 Removed unnecessarily heavy RAM usage from logs in bench/test.py
65923cd Adopted script changes in GitHub Actions
387cf6f Fixed a couple corner cases in scripts when fields are empty
0b11ce0 Fixed incorrect calculation of extra space needed in mdir blocks
eba5553 Fixed hidden orphans by separating deorphan search into two passes
f89d758 Fixed test out-of-space issues with powerloss testing
6c18b4d Added a simple help rule to the Makefile
f734941 Changed default build target lfs.a -> liblfs.a
bcc88f5 A couple Makefile-related tweaks
e35e078 Renamed prefix.py -> changeprefix.py and updated to use argparse
1a07c2c A number of small script fixes/tweaks from usage
6fce9e5 Changed plotmpl.py/plot.py to not treat missing values as discontinuities
559e174 Added plotmpl.py for creating svg/png plots with matplotlib
b2a2cc9 Added teepipe.py and watch.py
3a33c37 Added perfbd.py and block device performance sampling in bench-runner
29cbafe Renamed coverage.py -> cov.py
df283ae Added recursive results to perf.py
490e1c4 Added perf.py a wrapper around Linux's perf tool for perf sampling
ca66993 Tweaked scripts to share more code, added coverage calls/hits
296c5af Renamed bench_read/prog/erased -> bench_readed/proged/erased
274222b Added some automatic sizing for field-names in scripts/runners
a2fb708 Added stddev/gmean/gstddev to summary.py
9507e62 Several tweaks to script flags
42d889e Reworked/simplified tracebd.py a bit
fb58148 Consistent handling of by/field arguments for plot.py and summary.py
7591d9c Added plot.py for in-terminal plotting
9a0e3be Added a quick trie to avoid running redundant test/bench permutations
4fe0738 Added bench.py and bench_runner.c for benchmarking
20ec0be Cleaned up a number of small tweaks in the scripts
11d6d12 Dropped namespacing of test cases
1fcd82d Made test.py output parsable by summary.py
acdea18 Made summary.py more powerful, dropped -m from size scripts
23fba40 Added option for updating a CSV file with test results
03c1a4e Added permutations and ranges to test defines
bfbe44e Dropped permutation number for full leb16-encoded defines
5a2ff17 Changed test identifier separator # -> :
c7f7094 Several tweaks to test.py and test runner
a208d84 Reworked test defines a bit to use one common array layout
91200e6 Added tracebd.py, a script for rendering block device operations
c9a6e3a Added tailpipe.py and improved redirecting test trace/log output over fifos
5279fc6 Implemented exhaustive testing of n nested powerlosses
552336e Added optional read/prog/erase delays to testbd
3f4f859 Readded support for mirror writes to a file in testbd
4689678 Added --color to test.py, fixed some terminal-clobbering issues
61455b6 Added back heuristic-based power-loss testing
01b11da Added a simple test that the block device works
a368d3a Moved emulation of erase values up into lfs_testbd
b08463f Reworked scripts/pretty_asserts.py a bit
92eee8e Removed some prefixes from Makefile variables where not necessary
d9333ec Add "chamelon" to the related projects section.
46cc6d4 Added support for annotated source in coverage.py
5b0a6d4 Reworked scripts to move field details into classes
4a7e94f Reimplemented coverage.py, using only gcov and with line+branch coverage
2b11f2b Tweaked generation of .cgi files, error code for recursion in stack.py
1616115 Fix test.py hang on ctrl-C, cleanup TODOs
4a42326 Moved test suites into custom linker section
0781f50 Ported tests to new framework
d679fbb In ./scripts/test.py, readded external commands, tweaked subprocesses
5a572ce Reworked how test defines are implemented to support recursion
be0e6ad More progress toward test-runner feature parity
4962829 Continued progress toward feature parity with new test-runner
5ee4b05 Misc test-runner improvements
5812d2b Reworked how multi-layered defines work in the test-runner
6443693 Putting together rewritten test.py script
92a600a Added trace and persist flags to test_runner
9281ce2 More test_runner progress
4b0aa62 Some more minor improvements to the test_runner
d683f1c Reintroduced test-defines into the new test_runner
56a9903 Created new test_runner.c and test_.py
3ae87f4 Add littlefs-disk-img-viewer to README