v0.8.57

Full Changelog: v0.8.56...v0.8.57

v0.8.56

Full Changelog: v0.8.55...v0.8.56

v0.8.55

Full Changelog: v0.8.54...v0.8.55

v0.8.54

What's Changed

• Basic DNS fallback by @yzqzss in #55

Full Changelog: v0.8.53...v0.8.54

v0.8.53

What's Changed

• Correctly closing records when errors occur before sending to channel by @NGTmeaty in #54

Full Changelog: v0.8.52...v0.8.53

v0.8.52

What's Changed

• Update default size threshold to 2048 bytes by @NGTmeaty in #53

Full Changelog: v0.8.51...v0.8.52

v0.8.51

What's Changed

• Fix DNS query string by @NGTmeaty in #52
• Stop writing revisit records for 3I42H by @NGTmeaty in #51

Full Changelog: v0.8.50...v0.8.51

v0.8.50

What's Changed

• Add custom DNS resolver & DNS archiving by @CorentinB in #49
• Add AnyIP capabilities to randomly bind to any IPv6 in the assigned prefix by @equals215 in #48

New Contributors

• @equals215 made their first contribution in #48

Full Changelog: v0.8.49...v0.8.50

v0.8.49

Add DisableIPv4 & DisableIPv6 (#47)

* add: DisableIPv4, DisableIPv6

* remove: unrelated broken test

* add: Payload-Digest checks for DisableIPv4 / IPv6 tests

v0.8.48

ZSTD Dictionary support (#41)

* Add example dictionary generated from test WARCs with `zstd --train`

* Update .gitignore to include zstd warcs

* feat: update packages and run tidy.

* fix: use 'proper' dictionary that decodes properly.

* feat: initial commit of zstd dictionary support.

This commit allows an external ZSTD generated dictionary to be used in the compression process. This implementation will be spec complaint against the IIPC spec and currently works with all known ZSTD WARC tools. It is currently a WIP and needs additional testing and validation to ensure everything is working correctly.

* feat: ensure TLS handshake time is being respected

* fix: run fieldalignment

warc/client.go:15:25: struct of size 176 could be 144
warc/client.go:31:23: struct of size 232 could be 216
warc/dedupe.go:23:20: struct with 32 pointer bytes could be 24
warc/dedupe.go:31:20: struct with 48 pointer bytes could be 40
warc/dialer.go:24:19: struct with 168 pointer bytes could be 160
warc/random_local_ip.go:16:19: struct with 24 pointer bytes could be 8
warc/spooled.go:40:22: struct of size 80 could be 72
warc.go:15:22: struct with 72 pointer bytes could be 64
write.go:19:13: struct with 64 pointer bytes could be 56
warc/write.go:32:18: struct with 40 pointer bytes could be 32

* fix: add comments back

* fix: run fieldalignment (again?)

warc/client.go:15:25: struct with 96 pointer bytes could be 88
warc/client.go:31:23: struct with 176 pointer bytes could be 168