Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace hashlib.md5() with hashlib.blake2b() #2318

Merged
merged 1 commit into from
Sep 19, 2023
Merged

Conversation

zas
Copy link
Collaborator

@zas zas commented Sep 19, 2023

It is faster, and more secure.

Summary

  • This is a…
    • Bug fix
    • Feature addition
    • Refactoring
    • Minor / simple change (like a typo)
    • Other
  • Describe this change in 1-2 sentences:

Problem

  • JIRA ticket (optional): PICARD-XXX

hashlib.md5() is now quite old, and it raises few warnings (even though our use isn't risky).
Since a while hashlib.blake2b() is a good alternative, it is even slightly faster than md5()

Solution

Action

@zas zas requested a review from phw September 19, 2023 12:03
@zas
Copy link
Collaborator Author

zas commented Sep 19, 2023

Small script used to compare performance:

import hashlib
import time

for h in sorted(hashlib.algorithms_guaranteed):
    try:
        f = hashlib.__get_builtin_constructor(h)
        start = time.time()
        loops = 100
        for i in range(loops):
            data = chr(i).encode() + b'a' * 1000000
            hsh = f(data).hexdigest()
        print("%s: %f %s" % (h, time.time() - start, hsh))
    except Exception:
        pass

Output:

blake2b: 0.124802 fdda09ac7b46e01056a5811109c3f1c4989f0eb32b6786d25f03900330c39f412004d0211847d9023a68201fdccc799b47a04dec29cb6297780f0f6fb62f3532
blake2s: 0.173296 266f333067f0961bc6a8025c02ccb2fb045add0f9ecd0eb542aee980c4f2ee92
md5: 0.133047 4beec58efaff1d9adcc267e7bcbc8c50
sha1: 0.330689 f0a7f8a2163c4f3a6c04368c35f1bc4666eb980c
sha224: 0.323075 0299eefa571ea59cb9c0eee607f7dbcca2398dcb353994b200c27cb9
sha256: 0.322453 11af12821c7a80314a31c0d7b029f223d0e589ef8f8fb0bf162645c2a5a66967
sha384: 0.212472 bb66398710b30adbfe5e406da61b544653b36da7e1cb5cbbf829eb9ab9e182e4fb4782d10ef373f255bb3e34d032c1ba
sha3_224: 0.212290 69e9c9e65d2f8d0a4adfb640489b56a4e0ba2a07cc38b33bc48c7f9a
sha3_256: 0.221512 ac5fdac6d921e782b6b1e8e3c624f4a9e1bd5ce9f9248671a788bd8dafa5d0f4
sha3_384: 0.286031 bf273a93627f71c03fd9939cb1a148718e54c960711d316241a87626c7c5c5659ae1c2da5eccb77d9fe208019c18b0b4
sha3_512: 0.410632 973104f1f7703f13ad2618ccb9af616504d1820feec1a45f5fa77277c8631b88c5114cbcd5a5c44a812c6a290a8d3f308370bfb5a6d330ca1f9b0fc37146dc77
sha512: 0.210969 49725f9092f137af10aeb38f928297ebc4e0a4cc452a379c3870db8887410bc64bf22fa49495bbc847c027484a6151649e3893bd2960fd8bd9f6b3153f80507c

phw
phw previously approved these changes Sep 19, 2023
Copy link
Member

@phw phw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a very good move

@zas zas force-pushed the byebyemd5 branch 2 times, most recently from 9a680ee to a9384f1 Compare September 19, 2023 15:15
It is faster, and more secure.
@zas zas marked this pull request as ready for review September 19, 2023 23:54
@zas zas merged commit 3018d52 into metabrainz:master Sep 19, 2023
69 checks passed
@zas zas deleted the byebyemd5 branch September 19, 2023 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants