-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add rudimentary profiling to processing pipeline #288
Labels
Comments
Here is some manually extracted data from a full CC run on the server.
Some numbers:
The biggest culprits in the runtime are the OCR in our pdf to text conversion and the download from CC pages. |
J08nY
added a commit
to J08nY/sec-certs
that referenced
this issue
Jul 26, 2023
Merged
J08nY
added a commit
to J08nY/sec-certs
that referenced
this issue
Aug 24, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
From #275 (comment):
Log entries like this could be replaced with some elegant way of tracking how long these stages and steps of processing take. Like a context manager that:
The text was updated successfully, but these errors were encountered: