support checksum newData,diffData,oldData; support sf_diff lib call b…

…y Java code(jni) on Windows&Linux; support long path on windows; recode & update libs;
sisong · Dec 24, 2023 · 980701d · 980701d
1 parent 596e751
commit 980701d
Show file tree

Hide file tree

Showing 3 changed files with 52 additions and 24 deletions.
diff --git a/LICENSE b/LICENSE
@@ -1,6 +1,6 @@
 sfpatcher: 
 Copyright © 2020-2023 housisong. All rights reserved.
-sfpatcher为商业软件，使用前必须获得合法授权。对外发布的可执行程序，用于小范围测试评估目的不受限制。
+sfpatcher为商业软件，使用前必须获得合法授权。对外发布的可执行程序软件，非商业授权用户仅可用于小范围内部测试评估目的。
 
 -----------------------------------------------------------------------------------------
 
@@ -182,6 +182,35 @@ for any purpose, commercial or non-commercial, and by any means.
 
 -----------------------------------------------------------------------------------------
 
+xxHash Library
+Copyright (c) 2012-2021 Yann Collet
+All rights reserved.
+
+BSD 2-Clause License (https://www.opensource.org/licenses/bsd-license.php)
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+* Redistributions of source code must retain the above copyright notice, this
+  list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright notice, this
+  list of conditions and the following disclaimer in the documentation and/or
+  other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+-----------------------------------------------------------------------------------------
+
 This program, "bzip2", the associated library "libbzip2", and all
 documentation, are copyright (C) 1996-2019 Julian R Seward.  All
 rights reserved.
@@ -220,4 +249,4 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 Julian Seward, [email protected]
 bzip2/libbzip2 version 1.0.8 of 13 July 2019
 
------------------------------------------------------------------------------------------
+-----------------------------------------------------------------------------------------
diff --git a/README.md b/README.md
@@ -1,5 +1,5 @@
 # sfpatcher：针对应用商店的apk增量算法
-**v1.0.15 已正式上线**，为亿级手机终端用户提供更新服务，当前最新版本 v1.1.3   
+**v1.0.15 已正式上线**，为亿级手机终端用户提供更新服务，当前最新版本 v1.2.0   
 [**sfpatcher** 命令行工具下载](https://github.com/sisong/sfpatcher/releases)（支持Windows、Linux、MacOS），
 [命令行使用说明](https://github.com/sisong/sfpatcher/blob/master/cmdline_doc.md)   
 需要商业授权(含源代码&培训)，请联系作者： <[email protected]>   
@@ -118,7 +118,6 @@
 
 # 测试条件
 在一台笔记本PC上对比测试：Windows11, CPU R9-7945HX, SSD PCIe4.0x4 4T, DDR5 5200MHz 32Gx2   
-测试时关闭了HDiffPatch和sfpatcher在diff时的多线程，而开启多线程时一般可以成倍的提高diff速度。   
 patch时标注tmpf表示使用了临时文件来储存中间数据；mem表示在内存中执行不使用临时文件；limit表示使用限制内存占用的模式执行；而标注MT表示开启了多线程(8个)并行。   
 **BsDiff** v4.3 还是保持着使用bzip2算法压缩补丁。   
 **xdelta** v3.1.0 使用`-e -n -f -s`来创建补丁, 而用`-d -f -s`参数来执行的patch。   
@@ -251,19 +250,19 @@ sfpatcher补充测试了用ApkNormalized(ApkDiffPatch方案)处理过的apk文
 
 | 方案|平均压缩率|
 |:----|----:|
-|sfpatcher-0 lzma2|50.8%|
-|sfpatcher-1 lzma2|31.5%|
-|sfpatcher-2 lzma2|29.3%|
-|sfpatcher-3 lzma2|26.7%|
-|sfpatcher-2pre lzma2|81.9%|
-|sfpatcher-3pre lzma2|76.6%|
+|hdiffz lzma2|50.8%|
+|sfpatcher -1 lzma2|31.5%|
+|sfpatcher -2 lzma2|29.3%|
+|sfpatcher -3 lzma2|26.7%|
+|sfpatcher -2 pre lzma2|81.9%|
+|sfpatcher -3 pre lzma2|76.6%|
 ||
-|sfpatcher-0 zstd|50.9%|
-|sfpatcher-1 zstd|32.6%|
-|sfpatcher-2 zstd|30.7%|
-|sfpatcher-3 zstd|28.3%|
-|sfpatcher-2pre zstd|86.3%|
-|sfpatcher-3pre zstd|82.3%|
+|hdiffz zstd|50.9%|
+|sfpatcher -1 zstd|32.6%|
+|sfpatcher -2 zstd|30.7%|
+|sfpatcher -3 zstd|28.3%|
+|sfpatcher -2 pre zstd|86.3%|
+|sfpatcher -3 pre zstd|82.3%|
 
 # 节省CDN带宽费用估算(仅供参考)
 单个apk一次升级节省的流量估算：用户的安卓手机现在经常使用的应用apk一般都越来越大(游戏平均更大)，假设按平均100MB算。   

diff --git a/cmdline_doc.md b/cmdline_doc.md
@@ -29,6 +29,8 @@ options:
   -HD   create compressed diffFile compatible with hdiffz
   -SD   create single compressed diffFile compatible with hdiffz -SD
   -BSD  create diffFile compatible with bsdiff
+  -m-matchScore
+      matchScore>=0, DEFAULT -m-2, recommended: 0--6 etc...
   -c-compressType[-compressLevel]
       set outDiffFile Compress type & level, DEFAULT uncompress;
       for resave diffFile,recompress diffFile to outDiffFile by new set;
@@ -41,13 +43,11 @@ options:
             support run by 2-thread parallel.
         -c-lzma2[-{0..9}[-dictSize]]        DEFAULT level 7
             dictSize can like 4096 or 4k or 4m or 128m etc..., DEFAULT 16m
-            WARNING: code not compatible with it compressed by -c-lzma!
+            NOTE: code not compatible with it compressed by -c-lzma!
             support run by multi-thread parallel, fast!
         -c-zstd[-{0..22}[-dictBits]]        DEFAULT level 20
             dictBits can 10--31, DEFAULT 24.
             support run by multi-thread parallel, fast!
-  -m-matchScore
-      matchScore>=0, DEFAULT -m-1, recommended: 0--6 etc...
   -block-fastMatchBlockSize
       set block match befor slow byte-by-byte match, DEFAULT -block-4k;
       if set -block-0, means don't use block match;
@@ -121,7 +121,7 @@ options:
 比如使用zstd，使用16MB的字典，该算法压缩率不错，解压非常快：   
 `$ sf_diff "old.apk" "new.apk" "diff.pat" -o-2 -c-zstd-21-24`   
 
-* **-m 选项**：设置匹配最小分数(>=0)，默认值1；少量影响补丁包输出大小；一般输入数据可压缩性越大，这个值就可以设得越大。
+* **-m 选项**：设置匹配最小分数(>=0)，默认值2；少量影响补丁包输出大小；一般输入数据可压缩性越大，这个值就可以设得越大。
 
 * **-block 选项**：设置块匹配，在较慢的逐字节匹配之前使用基于块的快速匹配，默认-block-4k；如果设置为-block-0，意思是关闭基于块的提前匹配；块匹配大小fastMatchBlockSize推荐256,1k,64k,1m等; 如果新版本和旧版本相同数据比较多,那diff速度就会比较快,并且减少内存占用，但有很小的可能补丁包会变大。
 
@@ -151,9 +151,9 @@ options:
 
 # PC上打补丁的工具sf_patch
 patch端一般在安卓手机上运行，提供了NDK编译出的.so库文件和其对应java代码（即SDK，使用需要获得商业授权）。   
-*安卓库 libsfpatch.so (v1.0.16) 文件大小参考：   
- arm64-v8a 静态库 298KB (zip压缩后 144KB), 动态库 230KB (zip压缩后 119KB);   
- armeabi-v7a 静态库 217KB (zip压缩后 135KB), 动态库 189KB (zip压缩后 122KB).*   
+*安卓库 libsfpatch.so (v1.2.0) 文件大小参考：   
+ arm64-v8a 静态库 275KB (zip压缩后 134KB), 动态库 207KB (zip压缩后 110KB);   
+ armeabi-v7a 静态库 198KB (zip压缩后 123KB), 动态库 170KB (zip压缩后 110KB).*   
 
 直接在PC上运行sf_patch命令会得到如下所示的帮助信息输出： 
 ```
@@ -214,7 +214,7 @@ options:
 
 # 推荐做法：
 * 获得较小的优化补丁，并有优化的patch速度：   
-`$ sf_diff "old.apk" "new.apk" "diff.pat" -o-1 -c-zstd-21-23 -m-1 -step-3m -lp-512k -cache`   
+`$ sf_diff "old.apk" "new.apk" "diff.pat" -o-1 -c-zstd-21-23 -step-3m -lp-512k -cache`   
 （patch时内存占用估算：8m+3m+0.5m，最大约20MB左右。 diff时的-p并行线程数量根据机器情况设置。）   
 patch端建议参数：`$ sf_patch "old.apk" "diff.pat" "out_new.apk" -lp -p-6`