-
-
Notifications
You must be signed in to change notification settings - Fork 286
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimize the camera so that phones don't heat up and drain battery #1047
Comments
I just played around with hashing the image. Wikipedia:
The results weren't good, I only got the same results when putting the virtual device camera directly against the wall otherwise it would always be completely different. In addition to that after a few seconds of running the flutter process got autonomous and wasn't able to controll it in any way until after a while it crashed. |
A quick Google search leads to this library that we may be able to use to compress images before passing to ML Kit |
Maybe give this a shot, looks promising |
Looking back at it hashing doesn't preserve image delta. Perhaps the best way to do it is first compress the image and the use the image difference library proposed by @jasmeet0817 (looking at the code the plugin compares pixels one by one, you might aswell write the function yourself). Then if the difference percentage exceeds your threshold pass the new compressed image to ML Kit. Do your testing as compressing the image may reduce the barcode detection accuracy. |
Just checked your code, instead of feeding the _processCameraImage with the camera feed why not reducing the frame rate first? Let's say the camera feed produces 30 frames per second, you could process 1/10 of it arbitrarily and still have a quick response time. |
I tried around resizing the photo earlier and it turned out to be harder than it might seem But when we look at the Ml Kit documentation, they recommend a minimum resolution of 2MP (1.920 x 1.080) which is actually higher than the one we currently have. https://developers.google.com/ml-kit/vision/barcode-scanning/android
However, we could only scan the upper area which not covered by the cards, but when I look at the complexity of the pure resizing, I suspect that this is not going to be so easy. @PrimaelQuemerais , you said you also use ML kit for barcode scanning, I am curious did you take any steps to improve performance and @jasmeet0817 one thing which just came to my mind, can you maybe keep your stock camera on for a while and check how the temprature is evolving in there perhaps this will help us to further narrow down the performance factor |
you mean you want to check if basic camera usage heats up the phone? In that case, I can confirm that it doesn't, also no other app that uses the camera (example whatsapp, telegram etc) heats up the phone since they don't process each image. I just tried doing as you suggested with "QR Scanner" app, and it didn't heat up the phone either, but the app closed itself after a while on it's own. I think they realized the camera was idle and closed the app. Maybe we could do something similar but instead of closing we could just move to history page (or something similar) |
Here are my proposals on how we could improve battery usage. 1) Improve the frequency and how barcodes are detectedCurrently, in the app, we scan for a barcode every 10 frames. I think this approach can be improved:
What I suggest instead:
2) Only decode barcodes when necessaryIf the user has scrolled to an item in the carousel, we can still continue to show the camera feed (= to prevent an empty background). But at the same time, the barcode detection is paused: the visor is replaced by a "resume scanning" button. When this button is pressed, the scanning is restarted. This way, I think we won't impact the quality of the detection but will reduce instead the number of unnecessary computations. |
Heyyy @g123k good proposals, I especially like the idea of calculating some delta time for the frequency.
Already in place
The real detection is running native, so this is already on a different threat, but definitely makes sense for cropping.
We should do some more benchmarking before to analyze what needs the most computing power, the scanning, the camera maybe even the cropping. |
@g123k @M123-dev I remember working on the "half-cropping" for optimization, then said to @M123-dev now you can test/profile and got as a reply: "Actually I don't have a heat/drain problem". The key is: we need someone that actually experience that problem, for the test part, not just theory.
Is that a fact? If the device is constantly working and has no time to
I don't believe in
That's what I already suggested. Of course that would mean a slightly different UX, but I think that the "Scan now!/Resume scanning" button is an elegant solution. |
For the battery drain, on the OFF team, we have some iOS users complaining about this. One optimization I have also in mind -> switch to a JPEG format (instead of YUV), as it will require less memory and should also quick the cropping process. But in that case, I think some changes are needed in image's croppers. |
@g123k We would need iOS developers with battery drain, for tests during development. If the scan process causes the battery drain, an JPEG format could be a good idea - is that available on all platforms? Of course that would probably mean changes in the croppers. Or we could just ignore the crop process in the end, depending on the performances. But wait, the only problem is that we need developers with battery draining problems. Of course we can have good theoretical ideas regarding how to solve it, but currently we are playing rubik's cube blindfolded. I've wasted enough time coding the croppers only to realize that anyway no one I know had the battery draining problem. Could any tester with battery draining problems lend a device? |
Should we consider that this issue is now resolved? |
Sounds good performance isn't really a problem anymore |
What
Possible solutions
@PrimaelQuemerais
Part of
The text was updated successfully, but these errors were encountered: