FreeBSD 13.0-RELEASE based experimental 0.7.0 build

[probonopd]

https://github.com/helloSystem/ISO/releases/tag/experimental-13.0

Showstoppers:

helloSystem/ISO#198

If we cannot solve these in time, 0.7.0 will continue to be 12.2 based.

[grahamperrin]

Spun off from helloSystem/ISO#198 (comment)

0G35 cannot boot on non-EFI systems: …

hello-0.7.0_0G35-FreeBSD-13.0-amd64.iso booting without EFI:

Desktop environment without EFI:

[grahamperrin]

Spun off from helloSystem/ISO#198 (comment)

Why is the 0.7.0 build ~500 MB larger with FreeBSD 13? …

Accessing .uzip file content : freebsd – I'm confused by uzip

[grahamperrin]

My previous attempt omitted geom_uzip.

# kldload geom_uzip
# cd /Volumes/t500/VirtualBox/BSD/helloSystem/helloSystem\ 0.6.0/
# mdconfig -o readonly -f system.uzip -u 6
# cd ../helloSystem\ 0.7.0/
# mdconfig -o readonly -f system.uzip -u 7
# mdconfig -l
md6 md7 
# zpool import 
   pool: furybsd
     id: 11467834712836977993
  state: ONLINE
status: Some supported features are not enabled on the pool.
        (Note that they may be intentionally disabled if the
        'compatibility' property is set.)
 action: The pool can be imported using its name or numeric identifier, though
        some features will not be available without an explicit 'zpool upgrade'.
 config:

        furybsd       ONLINE
          md6.uzipp1  ONLINE

   pool: furybsd
     id: 5751110427541885720
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:

        furybsd       ONLINE
          md7.uzipp1  ONLINE
# mkdir -p /tmp/0.6.0
# mkdir -p /tmp/0.7.0
# zpool import -R /tmp/0.6.0 -o readonly=on 11467834712836977993 h060
# zpool import -R /tmp/0.7.0 -o readonly=on 5751110427541885720 h070
# zpool status h060
  pool: h060
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
        The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
config:

        NAME          STATE     READ WRITE CKSUM
        h060          ONLINE       0     0     0
          md6.uzipp1  ONLINE       0     0     0

errors: No known data errors
# zpool status h070
  pool: h070
 state: ONLINE
config:

        NAME          STATE     READ WRITE CKSUM
        h070          ONLINE       0     0     0
          md7.uzipp1  ONLINE       0     0     0

errors: No known data errors
# time du -hs /tmp/0.6.0
1.5G    /tmp/0.6.0
       17.12 real         0.12 user         1.30 sys
# time du -hs /tmp/0.7.0
1.4G    /tmp/0.7.0
       25.08 real         0.07 user         1.42 sys
# ls -hl ../helloSystem\ 0.6.0/system.uzip 
-rwxr-xr-x  1 grahamperrin  grahamperrin   1.4G Sep 30 09:54 ../helloSystem 0.6.0/system.uzip
# du -h ../helloSystem\ 0.6.0/system.uzip
1.4G    ../helloSystem 0.6.0/system.uzip
# ls -hl system.uzip
-rwxr-xr-x  1 grahamperrin  grahamperrin   1.9G Oct 24 04:18 system.uzip
# du -h system.uzip
1.9G    system.uzip
# 

Whilst one of the two system.uzip files is considerably larger, for the associated file systems the sums of sizes (measured by Filelight) barely differ. 2.7 GiB in both cases:

Housekeeping, after browsing the contents of the pools:

# zpool export h060
# zpool export h070
# mdconfig -d -u md6
# mdconfig -d -u md7
# 

[probonopd]

Very interesting investigation! What could be causing this difference?

[grahamperrin]

An uneducated guess: with FreeBSD 13.0⋯, maybe change your use of mkuzip(8) to zstd-19.

NB:

• https://www.freebsd.org/cgi/man.cgi?query=mkuzip&sektion=8&manpath=FreeBSD#IMPLEMENTATION_NOTES
• https://www.freebsd.org/cgi/man.cgi?query=geom_uzip&sektion=4&manpath=FreeBSD#HISTORY

geom_uzip(4)

---

root@mowa219-gjp4-8570p-freebsd:~ # zfs get compression h060 h070
NAME  PROPERTY     VALUE           SOURCE
h060  compression  gzip            local
h070  compression  zstd-9          local
root@mowa219-gjp4-8570p-freebsd:~ # zfs get logicalused,logicalreferenced,refcompressratio,written,used,referenced,compressratio h060
NAME  PROPERTY           VALUE     SOURCE
h060  logicalused        4.46G     -
h060  logicalreferenced  4.46G     -
h060  refcompressratio   3.17x     -
h060  written            1.43G     -
h060  used               1.43G     -
h060  referenced         1.43G     -
h060  compressratio      3.17x     -
root@mowa219-gjp4-8570p-freebsd:~ # zfs get logicalused,logicalreferenced,refcompressratio,written,used,referenced,compressratio h070
NAME  PROPERTY           VALUE     SOURCE
h070  logicalused        4.57G     -
h070  logicalreferenced  4.57G     -
h070  refcompressratio   3.33x     -
h070  written            1.39G     -
h070  used               1.39G     -
h070  referenced         1.39G     -
h070  compressratio      3.33x     -
root@mowa219-gjp4-8570p-freebsd:~ # zpool export h060 h070
root@mowa219-gjp4-8570p-freebsd:~ # mdconfig -d -u md6
root@mowa219-gjp4-8570p-freebsd:~ # mdconfig -d -u md7
root@mowa219-gjp4-8570p-freebsd:~ # mdconfig -l
root@mowa219-gjp4-8570p-freebsd:~ # exit
logout
% 

[probonopd]

Am I reading it wrong? h070 has a better compression ration and fewer GB written. Why is the image larger then? Is h070 something you made with special parameters?

[probonopd]

Will be moving to an entirely different methodology for the Live system, let's look at the sizes again once it is implemented.

[probonopd]

helloSystem/ISO@c97e25e

When using the same compression as with FreeBSD 12.x, then the FreeBSD 13.0 based ISO becomes even larger, exceeding 2 GB:

+ THRESHOLD_BYTES=2147483647
+ stat -f%z /tmp/cirrus-ci-build/artifacts/hello-0.7.0_0G39-FreeBSD-13.0-amd64.iso
+ ISO_SIZE=2148382720
+ [ 2148382720 -gt 2147483647 ]
+ echo 'Size exceeds GitHub Releases file size limit; splitting the ISO'

https://cirrus-ci.com/task/6308002190852096

For some reason I don't quite understand, our weird zfs+zfscompression+uzip combination apparently performs worse under FreeBSD 13 than under 12.x?

[probonopd]

The FreeBSD 13 based experimental image is unbootable after installing:
#242

No clue why.

[Daasin]

Would there be Beta's & Release Candidates in the works as we get closer to 1.0?

[probonopd]

Sure. We generally put out Experimental builds, and once one of those has been tested satisfactorily, it will be released. As we approach 1.0 we might add a beta test and/or RC phase.