Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatic testing of tvla.py script #96

Open
abdullahvarici opened this issue Jul 4, 2022 · 1 comment
Open

Automatic testing of tvla.py script #96

abdullahvarici opened this issue Jul 4, 2022 · 1 comment

Comments

@abdullahvarici
Copy link

With the PR #95, we can now load histograms and use them for general test. This is done automatically when a PR is created but it does not cover all functionality of the tvla.py script. We should also add an automatic test which gets power measurements, computes their t-values and compares with the expected values.

In addition there may be some other functionalities of the tvla.py script that may be tested. Maybe @vrozic, @vogelpi and @andreaskurth has some ideas?

@andreaskurth
Copy link
Collaborator

Capturing the results of our discussion yesterday:

Highest priority among the tests to be added for TVLA are:

  • AES general and byte-based
  • Stepping
  • Filtering, not just starting from histograms

Furthermore, we should add tests that compare the actual numerical values (e.g., by comparing the t-test trace against expected data).

Test data should always be accompanied by a shell script containing the commands used to capture/generate the data. The primary purpose of that script is to make test data reproducible. This is required because some test data will have to be updated as we change the behavior of the capture and/or analysis code. If manual interaction with hardware peripherals is required to produce test data, the script does not have to be executable from start to beginning; rather, it should contain comments so that an engineer knows when and how to manually interact with hardware between running commands.

For some tests, it could be beneficial to generate minimal, artificial traces to be able to quickly pinpoint bugs. For example, one problem with real traces is that you cannot always know at which exact clock cycle leakage is expected.

Finally, we should work with compressed test data to save LFS space and bandwidth. Note that np.savez() creates uncompressed .npz archives; use np.savez_compressed() to create zipfile.ZIP_DEFLATED .npz archives.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants