Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding NISTDSA API to support ML-DSA-44 and ML-DSA-87 #1949

Closed
wants to merge 11 commits into from

Conversation

jakemas
Copy link
Contributor

@jakemas jakemas commented Oct 25, 2024

Issues:

Resolves #CryptoAlg-2725

Description of changes:

This is a sizable PR, I've worked extremely hard to keep the size down in a single PR, but with ML-DSA touching X.509, PKEY, ASN1, there are a lot of changes that need to happen simultaneously to preserve library functionality, so I will document this PR well.

1. PKEY structure changes

This PR adds ML-DSA-44 and ML-DSA-87 to AWS-LC. Before this PR AWS-LC only supported ML-DSA-65, as such, we were utilizing the void *ptr of evp_pkey_st, rather than a distinct structure for ML-DSA.

This PR introduces the new structure NISTDSA_KEY * nistdsa_key; inside evp_pkey_st to support ML-DSA and any additional FIPS digital signature algorithms provided by NIST, should AWS-LC wish to include them in the library.

  union {
    void *ptr;
    RSA *rsa;
    DSA *dsa;
    DH *dh;
    EC_KEY *ec;
    KEM_KEY *kem_key;
    NISTDSA_KEY * nistdsa_key;
  } pkey;

While the structure DSA already exists it does not provide support for FIPS-like signature APIs (see more at NIST api conventions. Rather than create a PKEY struct of the form MLDSA_KEY that can only support ML-DSA, this is an opportunity to build support for future signature algorithms that have similar. calling structures, de-randomized testing modes, API converntions, etc. by making a more generic structure type -- much like we did for KEM_KEY. Under the design in this PR, adding SLH-DSA to the PKEY struct would be very simple, as it conforms to a NISTDSA_KEY in design. So too will be true of any additional signature algorithms produced by NISTs call for additional signature algorithms.

As, such, NISTDSA_KEY utilizes a new structure to hold public/secret keys, as well as a NISTDSA struct which defines signature algorithm specific information:

struct nistdsa_st {
  const NISTDSA *nistdsa;
  uint8_t *public_key;
  uint8_t *secret_key;
};
typedef struct {
  int nid;
  const uint8_t *oid;
  uint8_t oid_len;
  const char *comment;
  size_t public_key_len;
  size_t secret_key_len;
  size_t signature_len;
  size_t keygen_seed_len;
  size_t sign_seed_len;
  const NISTDSA_METHOD *method;
  const EVP_PKEY_ASN1_METHOD *asn1_method;
} NISTDSA;

This allows us to use a single PKEY structure for all NIST FIPS signature algorithms, much like the existing PKEY struct KEM_KEY *kem_key for KEMS. As a consequence, we are now able to define signature methods such as:

DEFINE_LOCAL_DATA(NISTDSA_METHOD, sig_ml_dsa_65_method) {
  out->keygen = ml_dsa_65_keypair;
  out->sign = ml_dsa_65_sign;
  out->verify = ml_dsa_65_verify;
}

DEFINE_LOCAL_DATA(NISTDSA, sig_ml_dsa_65) {
  out->nid = NID_MLDSA65;
  out->oid = kOIDMLDSA65;
  out->oid_len = sizeof(kOIDMLDSA65);
  out->comment = "MLDSA65 ";
  out->public_key_len = MLDSA65_PUBLIC_KEY_BYTES;
  out->secret_key_len = MLDSA65_PRIVATE_KEY_BYTES;
  out->signature_len = MLDSA65_SIGNATURE_BYTES;
  out->keygen_seed_len = MLDSA65_KEYGEN_SEED_BYTES;
  out->sign_seed_len = MLDSA65_SIGNATURE_SEED_BYTES;
  out->method = sig_ml_dsa_65_method();
  out->asn1_method = &nistdsa_asn1_meth;
}

much like the existing kem.c file.

These structures will be incredibly useful for subsequent PRs, in which we will be adding de-randomized APIs to support FIPS validation and KATs from seeds. In effect, we will be extending the method to include:

DEFINE_LOCAL_DATA(NISTDSA_METHOD, sig_ml_dsa_65_method) {
  out->keygen = ml_dsa_65_keypair;
  out->keygen_internal = ml_dsa_65_keypair_internal;
  out->sign = ml_dsa_65_sign;
  out->sign_internal = ml_dsa_65_sign_internal;
  out->verify = ml_dsa_65_verify;
}

The directory dilithium/p_dilithium3.c is used to house generic PKEY operations, as well as NISTDSA specific EVP functions such as EVP_PKEY_nistdsa_set_params, EVP_PKEY_nistdsa_set_params, which work in a similar way to p_kem.c functions EVP_PKEY_CTX_kem_set_params and EVP_PKEY_kem_set_params.

The directory dilithium/p_dilithium3_asn1.c is used to house generic asn.1 operations for NIST FIPS DSA algorithms.

Rather than rename/relocate these files in this PR (which will convolute the CR), I will move these files in a subsequent PR.

2. Internal Design

NISTDSA

We introduce new structures: NISTDSA_METHOD, NISTDSA, NISTDSA_KEY which are very similar to how both EC Crypto, and the KEM API is implemented. The figure above shows the newly proposed structures and their integration into the existing EVP structures.

  • NISTDSA_METHOD is a table of function pointers with 5 functions defined for a NISTDSA: key generation, key generation internal (for testing/FIPS), sign, sign internal (for testing/FIPS), and verify. Every NISTDSA implementation has to implement this API, for example sig_ml_dsa_44_method implements the three functions for ML-DSA-44 (analogous to EC_METHOD and for example, EC_GFp_nistz256_method).
  • NISTDSA is a structure that holds basic information about the DSA: the id, size of parameters, and the pointer to the implementation of the corresponding NISTDSA_METHOD.
  • NISTDSA_KEY structure is a helper structure that holds pointers to public and secret key and the pointer to the corresponding NISTDSA.
  • NISTDSA_PKEY_CTX is a helper structure used to store DSA parameters in an EVP_PKEY_CTX object (the same as EC_PKEY_CTX). Since NISTDSA has everything we need, that’s what we store in NISTDSA_PKEY_CTX.

3. OID changes

We now have real OIDs available from https://csrc.nist.gov/projects/computer-security-objects-register/algorithm-registration. These have been added to the obj files and automatically populated by go run objects.go. We include the following NIDs for ML-DSA:

//2.16.840.1.101.3.4.3.17
static const uint8_t kOIDMLDSA44[]  = {0x60, 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x03, 0x11};
//2.16.840.1.101.3.4.3.18
static const uint8_t kOIDMLDSA65[]  = {0x60, 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x03, 0x12};
//2.16.840.1.101.3.4.3.19
static const uint8_t kOIDMLDSA87[]  = {0x60, 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x03, 0x13};

and a "top-level" NID EVP_PKEY_NISTDSA (similar to EVP_PKEY_KEM to subset all NIST FIPS DSA algorithms. Much like for the KEM API, we include functions SIG_find_dsa_by_nid(int nid) to return actual algorithm specific methods.

4. Testing structure changes

Prior to this PR all testing for ML-DSA-65 was done as a series of g-tests. As we now have the ability to get algorithm/security level specific parameters from the NISTDSA struct, I have overhauled the testing suite to be parameterized over the currently supported signature algorithms:

static const struct ML_DSA parameterSet[] = {
  {"MLDSA44", NID_MLDSA44, 1312, 2560, 2420,  "dilithium/kat/mldsa44.txt", mldsa44kPublicKey, mldsa44kPublicKeySPKI, 1333},
  {"MLDSA65", NID_MLDSA65, 1952, 4032, 3309,  "dilithium/kat/mldsa65.txt", mldsa65kPublicKey, mldsa65kPublicKeySPKI, 1973},
  {"MLDSA87", NID_MLDSA87, 2592, 4896, 4627,  "dilithium/kat/mldsa87.txt", mldsa87kPublicKey, mldsa87kPublicKeySPKI, 2613},
};

For each of the above parameter sets we run:

TEST_P(MLDSAParameterTest, KAT)
TEST_P(MLDSAParameterTest, KeyGen)
TEST_P(MLDSAParameterTest, KeyCmp) 
TEST_P(MLDSAParameterTest, KeySize) 
TEST_P(MLDSAParameterTest, NewKeyFromBytes) 
TEST_P(MLDSAParameterTest, RawFunctions) 
TEST_P(MLDSAParameterTest, SIGOperations) 
TEST_P(MLDSAParameterTest, MarshalParse) 

5. X.509 changes

Changes to X.509 code have been minimized to only the required changes to support this PR. This is already a big PR and I want to avoid expansion wherever possible. For X.509 the changes predominantly are regarding the change of NIDs from the old Dilithium NIDs to the new NIST DSA NIDs/OIDs.

As the OIDs changed, so too much the test certificates kDilithium3Cert, kDilithium3CertNull, and kDilithium3CertParam. These have been regenerated using the following:

Tool used https://github.com/google/der-ascii

1. First generate a valid test certificate in PEM encoding, say cert.pem
2. Convert PEM to DER: openssl x509 -in cert.pem -out cert.der -outform DER
3. Convert DER to ASCII: der2ascii -i cert.der >> cert.txt
4. Make edits to cert.txt as desired, say; certnew.txt
5. Convert ASCII to DER: ascii2der -i certnew.txt >> certnew.der
6. Convert DER to PEM: openssl x509 -in certnew.der -inform DER -out certnew.pem -outform PEM

6. Speed Tool

ML-DSA-44 and ML-DSA-87 have been added to the speed tool. To facilitate the new APIs used in this PR, the API version has been incremented by 1. The speed tool now produces benchmarks for all three parameter levels, example output:

Did 4870 MLDSA44 keygen operations in 1048820us (4643.3 ops/sec)
Did 1230 MLDSA44 signing operations in 1012232us (1215.1 ops/sec)
Did 4840 MLDSA44 verify operations in 1093069us (4427.9 ops/sec)
Did 2871 MLDSA65 keygen operations in 1051869us (2729.4 ops/sec)
Did 774 MLDSA65 signing operations in 1039468us (744.6 ops/sec)
Did 2893 MLDSA65 verify operations in 1089059us (2656.4 ops/sec)
Did 1600 MLDSA87 keygen operations in 1008766us (1586.1 ops/sec)
Did 654 MLDSA87 signing operations in 1179044us (554.7 ops/sec)
Did 1650 MLDSA87 verify operations in 1049198us (1572.6 ops/sec)

Callouts:

  • dilithium/p_dilithium3.call dilithium3 pkey specific functionality is now generic
  • dilithium/p_dilithium3_asn1.c all dilithium3 asn1 specific functionality is now generic
  • dilithium/p_dilithium_test.cc all dilithium3 testing specific functionality is now generic
  • dilithium/internal.h is a new file, added to maintain consistency with fipsmodule/kem/internal.h

Testing:

See above for description of changes to testing framework.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and the ISC license.

@codecov-commenter
Copy link

codecov-commenter commented Oct 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.75%. Comparing base (b77a698) to head (465d53f).
Report is 7 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1949      +/-   ##
==========================================
+ Coverage   78.68%   78.75%   +0.06%     
==========================================
  Files         585      590       +5     
  Lines      100854   101428     +574     
  Branches    14298    14383      +85     
==========================================
+ Hits        79357    79876     +519     
- Misses      20863    20917      +54     
- Partials      634      635       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jakemas jakemas marked this pull request as ready for review October 25, 2024 20:05
@jakemas jakemas requested a review from a team as a code owner October 25, 2024 20:05
int (*sign)(uint8_t *sig, size_t *sig_len,
const uint8_t *message,
size_t message_len,
const uint8_t *ctx,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's ctx?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ctx was added to all NIST PQC candidates in the final version of the FIPS standards. It is a context string (byte string 255 or fewer bytes) that is used as additional input when signing. See page 17 of FIPS 204.

extern "C" {
#endif

// NISTDSA_METHOD structure and helper functions.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should change the name NISTDSA, maybe to NIST_PQDSA or something like that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to change the name -- originally wanted to use DSA but it was taken. I chose to add NIST because of the consistency between the FIPS 204, 205 standards, it seems as though any subsequent signature scheme standardizaed by NIST should follow these conventions. I didn't want to limit this to only signature schemes aimed towards PQ-secure use-cases.

size_t keygen_seed_len;
size_t sign_seed_len;
const NISTDSA_METHOD *method;
const EVP_PKEY_ASN1_METHOD *asn1_method;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you need the asn1 method within this struct?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, not used, fixed in 465d53f

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where are the KATs from?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These KATs are from the official upstream repository (https://github.com/pq-crystals/dilithium). They were generated the same way the ML-DSA-65 KATs were generated, from the reference.

if (key->priv == NULL) {
goto err;
}
static const uint8_t kOIDMLDSA44[] = {0x60, 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x03, 0x11};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add reference for these ids

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added in 465d53f

@jakemas
Copy link
Contributor Author

jakemas commented Nov 4, 2024

Splitting this PR up further, first part adds the functionality to support multiple parameter sets of ML-DSA and other NIST PQC DSA (#1963). The second PR will add ML-DSA-44 and ML-DSA-87.

@jakemas jakemas closed this Nov 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants