Skip to content
This repository has been archived by the owner on Jun 24, 2020. It is now read-only.

Runtime is ~1 second for 50mb files #6

Open
mdsauce opened this issue May 15, 2019 · 4 comments
Open

Runtime is ~1 second for 50mb files #6

mdsauce opened this issue May 15, 2019 · 4 comments
Labels
performance not a bug, not a required feature, but needed for overall usability

Comments

@mdsauce
Copy link
Owner

mdsauce commented May 15, 2019

Takes too long
@mdsauce
Copy link
Owner Author

mdsauce commented May 15, 2019

  • setup benchmarks

@mdsauce
Copy link
Owner Author

mdsauce commented May 15, 2019 •
edited
Loading

  • reduce time to process each line via go routines and wait.sync

@mdsauce mdsauce added the performance not a bug, not a required feature, but needed for overall usability label May 22, 2019
@mdsauce
Copy link
Owner Author

mdsauce commented Jun 18, 2019

Also, strings contain pointers, remove strings where possible and simplify maps by using non-strings as a key (if possible)
https://stephen.sh/posts/quick-go-performance-improvements

@mdsauce mdsauce changed the title Large Files take ~1 second for 50mb files Runtime is ~1 second for 50mb files Jul 8, 2019
@mdsauce
Copy link
Owner Author

mdsauce commented Aug 2, 2019

Use https://github.com/pkg/profile Dave Cheney's profile pkg. Really simple. Should be literally 1 line. This includes:

  • cpu profiling
  • tracing
  • memory profiling

Not all at once but still.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
performance not a bug, not a required feature, but needed for overall usability
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mdsauce