Use low camera resolution while scanning

[jasmeet0817]

What

• Use low camera resolution while scanning
• Tested it out, seems to work fine.

Here's documentation on startImageStream:

When running continuously with [CameraPreview] widget, this function runs best with [ResolutionPreset.low]. Running on [ResolutionPreset.high] can have significant frame rate drops for [CameraPreview] on lower end devices.

[M123-dev]

Actually, I am not a fan of setting it to low. We somehow have to find a balance between performance on low-end devices and comfort on high-end ones. Setting it to low of course needs less performance but also lowers the performance on low-end devices.

[jasmeet0817]

but also lowers the performance on low-end devices.

Are you sure it lowers the performance? Doesn't feel like there's any performance lapse on my Pixel 4XL. Please let me know the results of your testing.

Yesterday I was at the supermarket scanning things and after about 3 minutes of usage the app started being super slow and unresponsive, and the phone felt hot. If this is on pixel 4xl, then i wonder how lower end devices might fare.

If performance is indeed a problem as you say, then we would need to find another solution at high priority (like not processing every image but every other image etc).

[M123-dev]

Oh I meant "lowers the comfort on high-end devices"

Lowering the preview just lowers the needed performance

[jasmeet0817]

Oh I meant "lowers the comfort on high-end devices"

Lowering the preview just lowers the needed performance

The thing is, I'm not sure if we even need high resolution for scanning barcodes.

[M123-dev]

If the barcode scanning is affected by this is another topic. But for me it's like, looking at the documentation:

low → const ResolutionPreset

352x288 on iOS, 240p (320x240) on Android and Web

I don't want so see a 300px image covering my whole screen, I don't expect this as long as I don't have a 10 year old phone, which of course isn't our target group. Looking at the docs I'd rather go with veryHigh (1080p) then just high (720p)

Also I just did a quick test myself (Samsung S21) and let the scanning run for around a little less then 10 minutes and of course the phone warmed up a bit and smoothie wasn't running at 60FPS after that. I know this can't be said for every phone but thats why we stop the camera when going away from the scanner. No user is going to spend that much continuous time with the camera actived.

[jasmeet0817]

If the barcode scanning is affected by this is another topic. But for me it's like, looking at the documentation:

low → const ResolutionPreset

352x288 on iOS, 240p (320x240) on Android and Web

I don't want so see a 300px image covering my whole screen, I don't expect this as long as I don't have a 10 year old phone, which of course isn't our target group. Looking at the docs I'd rather go with veryHigh (1080p) then just high (720p)

it's not thaaaat bad :)

High resolution =

Low resolution =

ok it's not that great either. Maybe there's a way of doing the preview in high but processing in low (sounds like wishful thinking) :)

and here's medium
MEDIUM =

Also I just did a quick test myself (Samsung S21) and let the scanning run for around a little less then 10 minutes and of course the phone warmed up a bit and smoothie wasn't running at 60FPS after that. I know this can't be said for every phone but thats why we stop the camera when going away from the scanner. No user is going to spend that much continuous time with the camera actived.

Samsung S21 is a very advanced device :) it's one of the newest phones available yet, my Pixel 4XL is 2 years old but very powerful and it only lasted about 3 minutes before heating up and slowing everything down.

We need to test on some Huawei devices. @alexgarel could you help?

[jasmeet0817]

It's also a matter of how much we care about showing high resolution camera stream to the users. @teolemon @stephanegigandet any comments ? Marvin does have a good point, but the cost of high resolution images is way too high AFAICT. If the functionality isn't affected by the resolution I wouldn't worry too much about it.

@alexgarel it would be great if you can test the latest dev version and tell us how it fares on Huawei.

[M123-dev]

Maybe there is a way to only send the scanning filed to the ml_kit but this won't save any recources in the current setup (as long as we can't check 30+ pictures per second) since we always send a new picture when the last is finished scanning

[jasmeet0817]

Indeed I had some issues scanning with low resolution just now :(

EDIT: Nope, not an issue with Resolution, sometimes ML scanning library just doesn't scan no matter the resolution, tested again and it scanned the products just fine this time.

[alexgarel]

@jasmeet0817 I think I have most of the things ready to test, I have flutter doctor ok :-)
But I don't know what to do from there !
I would be glad to help from monday if you redirect me to a tutorial on what to do and what should I do in this specific case.

[jasmeet0817]

@jasmeet0817 I think I have most of the things ready to test, I have flutter doctor ok :-) But I don't know what to do from there ! I would be glad to help from monday if you redirect me to a tutorial on what to do and what should I do in this specific case.

Do you have android studio installed and the git repo setup ? If so, then it's really easy. Just connect your phone and run "main.dart" with Flutter. So step 9 from this: https://www.dummies.com/article/technology/programming-web-design/app-development/running-your-first-flutter-app-in-android-studio-272214

[teolemon]

Reading MLKit's doc about implementation: https://developers.google.com/ml-kit/vision/barcode-scanning/android
Interesting read, especially the 2MB limit

[M123-dev]

@teolemon where did you read it?

But some other good information in there:

Don't capture input at the camera’s native resolution. On some devices, capturing input at the native resolution produces extremely large (10+ megapixels) images, which results in very poor latency with no benefit to accuracy. Instead, only request the size from the camera that's required for barcode detection, which is usually no more than 2 megapixels.

The only question is how we are going to scale down the image for ml-kit

[teolemon]

that's what I was alluding to as well, brain not working well :-)

[M123-dev]

Moved discussion to #1047