[kmac] Rework masked SHA3 core and switch to Trivium-based PRNG implementation

[vogelpi]

This PR consists of several commits required to improve the masking in the KMAC block and to prevent brute-forcing attacks on the PRNG state. This PR resolves #20828.

The commits can be grouped into 3 groups:

• The first commits simplify/cleanup/improve the I/O mux control and PRNG inside the masked SHA3 core. Ultimately, this allows preventing glitches on the mux control signals for the inputs and outputs of the non-linear layer (the DOM multipliers) needing the entropy. This helps improving the masking and comes basically for free in terms of area and timing.
• One commit adds 800-bit buffer stage between the output of the PRNG and the masked non-linear layer / the DOM multipliers to prevent glitches occurring inside the PRNG from propagating into the masked SHA3 core. For the old PRNG architecture (limited glitching activity in output due to 4-bit S-Boxes), this helps improving the masking whereas for the new architecture this is absolutely needed. This adds around 7 kGE to the overall KMAC area (+3.8%), no timing impact.
• One commit to replace the LFSR-based PRNG architecture potentially vulnerable to brute-forcing attacks with a Trivium-based implementation (similar to what we've already done for AES, see #[aes,rtl] Switch to Bivium-based masking PRNG implementation #20852) . This adds around 13 kGE to the overall KMAC area (20 kGE with the buffer stage, +11%) and reduced the max clock frequency from 617 MHz to 480 MHz (all numbers generated with the Yosys + nangate45 flow, so use with care). Due to the heavy unrolling of the primitive (we need it to generate 800 bits per clock cycle from a 288-bit state) the critical path inside KMAC is now withing the PRNG. However, the 480 MHz is still a lot faster than other parts / accelerators in the design. So, I don't think this should be a problem.

From an SCA perspective, I have the following results so far:

• The PROLEAD evaluation indicates a slight improvement but the results are still indecisive (test results seem to oscillate around the threshold but the oscillation now starts later).
• The FPGA measurements look a lot better now. Previously we would see leakage starting to kick in at around 1 Mio traces. Sometimes sooner, sometimes later.
  • In SHA3 mode we now have: no 1st order leakage with 10 Mio traces, 2nd-order leakage only during the first 2 rounds, i.e., when the SHA3 state is not yet properly randomized, i.e., we have little noise in the linear layer. This won't be an issue for KMAC. If we want to get rid of this, we have to move the message masking to after the padding (see [kmac] Consider moving message masking to output of sha3pad #17209). This will come for free in terms of area and timing, so can be done in M3 if we really want it. Just for reference, I had to optimize the top-level design / capture setup quite a bit to get results but I am now very confident in these measurements. If masking is disabled via SW, we get strong 1st leakage with already 1000 traces (a big improvement to the past!).
  • Measurements for the KMAC mode are currently running. are now done as well: we have no 1st and 2nd order leakage with 10 Mio traces. Again a big improvement compared to before :-)

[nasahlpa]

Thanks @vogelpi, great job - LGTM! Splitting up this PR into multiple commits and the offline discussion really helped in reviewing these code changes.

[andreaskurth]

LGTM, thanks for this quite extensive change and improvement @vogelpi!

(I focused my review on the RTL code and didn't notice red flags. For the changes to the other code, I think CI checks and your experiments + local tests should suffice.)

[msfschaffner]

Thanks for the well structured PR @vogelpi! This is great!

[vogelpi]

Thanks everybody for your reviews, let's merge this!

[cdgori]

Continuing catch-up, this all LGTM