[cryptotest] Add test coverage of HMAC streaming API

[RyanTorok]

Adds test coverage in the cryptotest framework for the HMAC streaming API added in #23335. The testing is modeled after how the streaming coverage was done for SHA-2 in #21281.

The second commit uncomments the hash algorithm tests in the BUILD file, which were accidentally commented out in a previous commit.

See also: #23471

[jwnrt]

Rust LGTM, thanks

[ballifatih]

LGTM, thanks @RyanTorok.

nit: Digest/tag sizes are exposed through cryptolib-API's hash.h now, which is included in mac.h. I recently refactored some other tests to reuse these exposed enums. E.g. kOtcryptoHmacTagBytesSha256 -> kSha256DigestWords.

Because of the amount of vectors in SHA-2, I cannot get past 2h timeout on FPGA. Am I missing another timeout parameters besides the one defined below?

opentitan/sw/device/tests/crypto/cryptotest/cryptotest.bzl

Line 39 in ec14811

timeout = "long",

[RyanTorok]

The test should respect the timeout that's there. If you're timing out, you can try changing long to eternal and see if it works. I don't recall it taking longer than 2 hours to run on my CW310, however.

[RyanTorok]

LGTM, thanks @RyanTorok.

nit: Digest/tag sizes are exposed through cryptolib-API's hash.h now, which is included in mac.h. I recently refactored some other tests to reuse these exposed enums. E.g. kOtcryptoHmacTagBytesSha256 -> kSha256DigestWords.

Because of the amount of vectors in SHA-2, I cannot get past 2h timeout on FPGA. Am I missing another timeout parameters besides the one defined below?

opentitan/sw/device/tests/crypto/cryptotest/cryptotest.bzl

Line 39 in ec14811

timeout = "long",

Refactored to use the constants in hash.h.

[ballifatih]

I am setting timeout="eternal", but for some reason it times out at 2h, which I found surprising. As for hmac_kat, I was able to run it without timeout. The reason is takes so long could be because previous OTBN-based implementation was faster, and the current HMAC driver needs a bit of improvement.

[jadephilipoom]

I find the timeout very surprising; HMAC should be fast. I'm worried this means we're missing a performance issue in the C code. How many test vectors are there exactly? Is the test progressing through them at a steady rate or taking much longer on some than others?

[ballifatih]

I find the timeout very surprising; HMAC should be fast. I'm worried this means we're missing a performance issue in the C code. How many test vectors are there exactly? Is the test progressing through them at a steady rate or taking much longer on some than others?

The test timeouts (at 2h mark) at the following test:

[2024-06-04T17:13:20.803Z INFO  harness] Test counter: 632
[2024-06-04T17:13:20.803Z INFO  harness] vendor: nist, algorithm: sha-512, test case: 117
[2024-06-04T17:13:20.803Z INFO  opentitanlib::test_utils::rpc] Sending: "Hash"
[2024-06-04T17:13:20.810Z INFO  opentitanlib::test_utils::rpc] Sending: "Sha512"
[2024-06-04T17:13:20.819Z INFO  opentitanlib::test_utils::rpc] Sending: {"length":64}

There are two immediate things to improve in the driver that I am aware of:

• Message bytes are written byte at a time, and needs to be optimized to word-writes as much as possible
• One-shot calls are actually broken down into 2 streaming calls, so there is a cost of storing/restoring context even for one-shot calls.

However I am not sure these alone should justify the speed. I will investigate to see why it's taking so long.

[RyanTorok]

I find the timeout very surprising; HMAC should be fast. I'm worried this means we're missing a performance issue in the C code. How many test vectors are there exactly? Is the test progressing through them at a steady rate or taking much longer on some than others?

There's a lot of hash test vectors, and quite a few of them are intentionally very long messages that take on the order of seconds to hash on a CW310. I recall the entire job taking over an hour for me, the last time I ran the whole thing.

Regarding your concern about the performance of HMAC, recall that the hash test job also includes SHA-3 and SHAKE, which are currently software backed, I think. One practical option would be to split the job up by high-level algorithm, i.e. one job for SHA-2, one for SHA-3, and one for SHAKE.

[moidx]

Regarding your concern about the performance of HMAC, recall that the hash test job also includes SHA-3 and SHAKE, which are currently software backed, I think. One practical option would be to split the job up by high-level algorithm, i.e. one job for SHA-2, one for SHA-3, and one for SHAKE.

We should make this change so that we can include the SHA2 and HMAC vectors as part of the hmac sign-off. I would prefer if there is a separate target per configuration: e.g. hash_sha2_256_kat, rather than bundling everything up. You can still have a test suite aggregating all tests as hmac_kat.

Let's get this PR in, and then we can work on the required refactoring.

[jadephilipoom]

Regarding your concern about the performance of HMAC, recall that the hash test job also includes SHA-3 and SHAKE, which are currently software backed, I think. One practical option would be to split the job up by high-level algorithm, i.e. one job for SHA-2, one for SHA-3, and one for SHAKE.

But according to the printout, it's timing out after 632 tests. I'd expect a hardware HMAC to get through a lot more than that in 2h on an FPGA. I'm still a little worried that we're missing a performance issue here. But it shouldn't block the PR; either the issue is in the test code (in which case it's not a huge deal) or it's not being introduced here, just detected.

[ballifatih]

But according to the printout, it's timing out after 632 tests. I'd expect a hardware HMAC to get through a lot more than that in 2h on an FPGA. I'm still a little worried that we're missing a performance issue here. But it shouldn't block the PR; either the issue is in the test code (in which case it's not a huge deal) or it's not being introduced here, just detected.

Yes, it timeouts before SHAKE/SHA3 vectors. I think the number of tests do not mean much in this case. The message length seems to be throttling the speed.

For comparison, hmac_kat passes in ~19 mins with 1422 tests. In any case, it's worth looking into. It should not block this PR.

[ballifatih]

I ran a simple experiment that runs 100 SHA-512 operations with 4KByte message blocks, by modifying sw/device/tests/crypto/hmac_functest.c:

• Define an array of 512-bit x_i variables for i in [0, ..., 100]. Set x_0 = 0.
• For i in [0, ..., 99] compute:
  • x_{i+1} = SHA512(x_i || x_i || … || x_i) where the input message is 4KByte. Each x_i is 64 bytes, so the message is concatenation of 64 copies of x_i.
• Check that x_100 matches known digest value.

I get the following time measurements on CW340.

• When SHA-512 calls are streamed with 64 update calls:
  • The complete test ends in 8.3s.
  • Each SHA-512 computation takes “314447 cycles or 54768 us @ 24 MHz” as reported by LOG_INFO.
• When SHA-512 calls are one-shot:
  • The complete test ends in 3.1s.
  • Each SHA-512 computation takes “29327 cycles or 1221 us @ 24 MHz.” as reported by LOG_INFO.

I suspect that the cryptotest and communicating with FPGA is the bottleneck for tests that have large message inputs.

[jadephilipoom]

Thanks for the experiments, @ballifatih ! I agree that given your results (reasonably fast tests even with large, streamed input) it seems likely that communication with the FPGA could be the culprit in terms of performance. It'd still be good to resolve, but that's much better than the bottleneck being in the actual library 🙂

[RyanTorok]

Rebased to master to resolve merge conflict with #23518