Change: Introduce more flexible NVDResults class #936
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Conventional Commits Report
🚀 Conventional commits found. |
bjoernricks
force-pushed
the
nvd-api-rework
branch
from
d7f106e to
9d45110
Compare
Before all NVD API classes methods returned an async iterator. This didn't allow much control of what the user actually wants and how the requests are issued. To improve the situation a new NVDResults class is returned which itself is an async iterator so that the previous API is kept compatible. But additionally the NVDResults instance allows to get the plain JSON data, the number of available results and also to iterate over chunks of results (which the NVD API is always returning). Most important improvement the NVDResults instance keeps the state. That means if an http error occurs it is possible to request the same data again. With the old API the requests need to start from the beginning if something did go wrong. For example if we downloaded already 100k CVEs and a http error was raised we needed to start from CVE number 1 again. With the new implementation we can just continue with the last request again.
bjoernricks
force-pushed
the
nvd-api-rework
branch
from
9d45110 to
0b6b52b
Compare
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #936 +/- ##
==========================================
+ Coverage 90.38% 90.51% +0.13%
==========================================
Files 97 97
Lines 6778 6814 +36
Branches 976 973 -3
==========================================
+ Hits 6126 6168 +42
+ Misses 446 445 -1
+ Partials 206 201 -5 ☔ View full report in Codecov by Sentry. |
y0urself
approved these changes
Nov 30, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
Before all NVD API classes methods returned an async iterator. This didn't allow much control of what the user actually wants and how the requests are issued. To improve the situation a new NVDResults class is returned which itself is an async iterator so that the previous API is kept compatible.
But additionally the NVDResults instance allows to get the plain JSON data, the number of available results and also to iterate over chunks of results (which the NVD API is always returning).
Most important improvement the NVDResults instance keeps the state. That means if an http error occurs it is possible to request the same data again. With the old API the requests need to start from the beginning if something did go wrong. For example if we downloaded already 100k CVEs and a http error was raised we needed to start from CVE number 1 again. With the new implementation we can just continue with the last request again.
Why
Allow much more flexible queries to the NVD API and allow to resume on errors.
References
DEVOPS-472
Checklist