bug: iOS Corner points detection issue #126
Comments
|
Please create a screen recording and show the problem there. |
|
the startScan code: private async startScan(): Promise<void> {
document.querySelector('body')?.classList.add('barcode-scanning-active');
const options: StartScanOptions = {
formats: this.formats,
lensFacing: this.lensFacing
};
const squareElementBoundingClientRect =
this.squareElement?.nativeElement.getBoundingClientRect();
const scaledRect = squareElementBoundingClientRect
? {
left: squareElementBoundingClientRect.left * window.devicePixelRatio,
right: squareElementBoundingClientRect.right * window.devicePixelRatio,
top: squareElementBoundingClientRect.top * window.devicePixelRatio,
bottom: squareElementBoundingClientRect.bottom * window.devicePixelRatio,
width: squareElementBoundingClientRect.width * window.devicePixelRatio,
height: squareElementBoundingClientRect.height * window.devicePixelRatio
}
: undefined;
console.log('Scaled Rect:', scaledRect);
const detectionCornerPoints = scaledRect
? [
[scaledRect.left, scaledRect.top],
[scaledRect.left + scaledRect.width, scaledRect.top],
[
scaledRect.left + scaledRect.width,
scaledRect.top + scaledRect.height
],
[scaledRect.left, scaledRect.top + scaledRect.height]
]
: undefined;
const listener = await BarcodeScanner.addListener(
'barcodeScanned',
async (event) => {
this.ngZone.run(() => {
const cornerPoints = event.barcode.cornerPoints;
console.log('Scanned Corner Points:', cornerPoints);
console.log('Detection Corner Points:', detectionCornerPoints);
if (detectionCornerPoints && cornerPoints) {
if (
detectionCornerPoints[0][0] > cornerPoints[0][0] ||
detectionCornerPoints[0][1] > cornerPoints[0][1] ||
detectionCornerPoints[1][0] < cornerPoints[1][0] ||
detectionCornerPoints[1][1] > cornerPoints[1][1] ||
detectionCornerPoints[2][0] < cornerPoints[2][0] ||
detectionCornerPoints[2][1] < cornerPoints[2][1] ||
detectionCornerPoints[3][0] > cornerPoints[3][0] ||
detectionCornerPoints[3][1] < cornerPoints[3][1]
) {
console.log('Condition not met, returning.');
return;
}
}
listener.remove();
this.closeModal(event.barcode);
});
}
);
console.log('Starting Barcode Scan...');
await BarcodeScanner.startScan(options);
}Will that be enough? In the last seconds of video im getting success, because my phone was in vertical position, only in this case it goes through. It clearly see the barcode instantly but can get this scan as well from the scan box, my guess i scalling |
I meant a screen recording of the device. |
|
the screen is just scanning modal pointed on my barcode :D I just changed angle of camera. There's no info on the screen or something |
|
Do you need any screen record? I mean there's nothing to see there, just view of phone trying to scan barcode from different angle |
|
Yes, please upload a screen recording of the device. |
|
This is how it works right now, did not noticed that video bricked. scan-test.mov |
|
Thank you for the video! Can you share an image file with the barcode so I can test it myself? Have you tried holding the camera further away, regardless of the angle? |
Sure heres the img:
Ye i tired it from bigger distance and many other ways. The first vid that i send shows the diff in coords I was thinking of flat workaround, in theory we could set the Y axis to the middle of our modal and the gave him eg. 200 height from there (100 top, 100 bottom) with 100% width. Which would make a box in the center of page with 200x100% I think it would be nice to be able to set beacuse some devices may have problems with window scaling, it could prevent form scanning with whole modal when few barcodes would be in area, what do u think about it? |
I'm not quite sure what you mean. You can already limit the area via JS. Do you want this to happen on the native level or how? |
I wrote it a bit wrong, I meant to mark the scanning field in the center, without using corner points. flat area in js. |
|
There are users who want this area above or below. Not everyone wants it in the middle. |
Ye, and i respect that, but was asking certainly about my case, it could skip the problem with part where code does not meet conditions, which bricks my app now somehow, because it sees the barcode immediately but detectionCornerPoints && cornerPoints not match. if (detectionCornerPoints && cornerPoints) {
if (
detectionCornerPoints[0][0] > cornerPoints[0][0] ||
detectionCornerPoints[0][1] > cornerPoints[0][1] ||
detectionCornerPoints[1][0] < cornerPoints[1][0] ||
detectionCornerPoints[1][1] > cornerPoints[1][1] ||
detectionCornerPoints[2][0] < cornerPoints[2][0] ||
detectionCornerPoints[2][1] < cornerPoints[2][1] ||
detectionCornerPoints[3][0] > cornerPoints[3][0] ||
detectionCornerPoints[3][1] < cornerPoints[3][1]
) {
console.log('Condition not met, returning.'); <------- this part
return;
}
}
listener.remove();
this.closeModal(event.barcode); |
|
I just fixed some corner points calculation issues in #127. |
|
ill check it right away, thanks. |
|
Sorry for late response. {"barcode":{"bytes":[71,80,72,50,51,48,52,49,49,49,50,54],"rawValue":"GPH230411126","format":"CODE_128","displayValue":"GPH230411126","valueType":"TEXT","cornerPoints":[[257,886],[673,821],[684,918],[268,968]]}}
⚡️ [log] - Scanned Corner Points: [[257,886],[673,821],[684,918],[268,968]]
⚡️ [log] - Detection Corner Points: [[114,991.6085815429688],[714,991.6085815429688],[714,1241.608642578125],[114,1241.608642578125]]
⚡️ [log] - Condition not met, returning. |
|
In this case you need to debug it yourself. I cannot reproduce any issues with latest version. Feel free to create a PR with a fix. |
|
seems like detection corner points are rotated 90 degrees, placed them from 3 position on chart, any clue with that? |
|
Are you sure you use the latest version? |
@capacitor-mlkit/[email protected] this one |
|
Please give it a try with the latest version (5.4.0): |
Hm i tested it and it seems to work, iOS is still working poorly, u have to move it some much to catch it, which gives feeling like it slowed. When I hover over the barcode, I see the logs returning with results not being in scanning box, but after a while in the same position suddenly the code is treated as being in the field and the scan passes. Which is a bit strange. Whole process of scanning works bit slower overally on iOS and Android |
|
Thanks for testing. That's weird. It works without problems for me. I recently shared a screen recording on X: https://x.com/robin_genz/status/1758536783414071431?s=46&t=k9Xz6D9ZXLbWibNDwFk3nQ As I said, feel free to debug the issue on your end. PRs are welcome! |
|
ill test it more tomorrow, it may be affected by camera losing focus from the center, and the code may be unreadable? Is this topic in ur schedule #119
|
It's still on my todo list but I don't have any ETA. Next, I'm working on the Capacitor 6 migration. |
|
Almost got it works fine, but one thing is bugging me. The scanning box catches 40-50px below the box and the opposite is at the top, It kind of went downhill a bit for those (bottom part of scan box catches well, and little below him, top part is somehow turned off :D). Thats the last problem im facing, did u set centering mechanism, or added some values to this scan box part in 5.4? My html is little different, do you see any issue in mine? <ion-header class="shader">
<ion-toolbar class="ios">
<ion-icon
(click)="closeModal()"
class="td-cursor-pointer td-position-relative td-right-0 td-z-index ion-padding ion-margin"
name="close"
size="large"
slot="start"
color="light">
</ion-icon>
<ion-icon
(click)="toggleTorch()"
class="td-cursor-pointer td-position-relative td-z-index ion-padding ion-margin"
[name]="torchActive ? 'flashlight' : 'flashlight-outline'"
size="large"
slot="end"
[color]="torchActive ? 'success' : 'light'">
</ion-icon>
</ion-toolbar>
</ion-header>
<ion-content>
<div #scanBox class="scan-box">
<p></p>
</div>
<ion-fab slot="fixed" vertical="bottom" horizontal="end">
<ion-fab-button
class="in"
type="button"
size="large"
(click)="zoomIn()"
(touchstart)="startZoomIn()"
(touchend)="stopZoom()">
<ion-icon name="add"></ion-icon>
</ion-fab-button>
<ion-fab-button
class="out"
type="button"
size="large"
(click)="zoomOut()"
(touchstart)="startZoomOut()"
(touchend)="stopZoom()">
<ion-icon name="remove"></ion-icon>
</ion-fab-button>
</ion-fab>
<div class="zoom-ratio-wrapper">
<p class="ion-text-center app-mt-0">
{{ zoomRatio >= 1 ? Math.round(zoomRatio * 100) + '%' : '100%' }}
</p>
</div>
</ion-content>
`,
styles: [
`
ion-content {
--background: transparent;
}
ion-fab-button {
margin-bottom: 16px;
width: 78px;
height: 78px;
--box-shadow: 0px 1px 2px 0px rgba(0, 0, 0, 0.3), 0px 1px 3px 1px rgba(0, 0, 0, 0.15);
--color: #fff;
--background: var(--ion-color-dark) !important;
opacity: 0.6;
}
ion-modal {
--border-radius: 0;
}
.shader {
background-color: rgba(0, 0, 0, 0.5) !important;
box-shadow: none !important;
z-index: 0;
}
.ios ion-toolbar {
--background: transparent;
--ion-color-base: transparent !important;
--border-width: 0;
}
ion-button {
margin-inline: unset
}
.box {
box-shadow: 0 0 0 100vmax rgba(0, 0, 0, 0.6);
position: absolute;
width: 330px;
height: 100px;
left: 50%;
top: 50%;
transform: translate(-50%, -50%);
}
.scan-box:before, .scan-box:after, .scan-box > :first-child:before, .scan-box > :first-child:after {
position: absolute;
width: 22px;
height: 22px;
border-color: var(--ion-color-light);
border-style: solid;
content: '';
}
.scan-box:before {
top: -8px;
left: -8px;
border-width: 3px 0 0 3px
}
.scan-box:after {
z-index: -1;
top: -8px;
right: -8px;
border-width: 3px 3px 0 0
}
.scan-box > :first-child:before {
bottom: -8px;
right: -8px;
border-width: 0 3px 3px 0
}
.scan-box > :first-child:after {
bottom: -8px;
left: -8px;
border-width: 0 0 3px 3px
}
.zoom-ratio-wrapper {
position: absolute;
left: 50%;
bottom: 16px;
transform: translateX(-50%);
width: 50%;
} |
|
No, your html seems to be okay. |
|
ill record u tomorrow in the morning what it looks like in action, cant understand that. Maybe u might be able to localize the issue |
|
It looks like there hasn't been a reply in 30 days, so I'm closing this issue. |
Plugin(s)
Version
5.3.0-dev.966ebeb.1703101506
Platform(s)
Current behavior
I have problem trying to scan.
The corner point problem occurs only on iOS device in my case.
Can't understand why iOS coords are different on every angle im attempting to scan. Is there a way to lock it somehow?
I seems like i can get scan go through on lucky strike.
Android works fine for me.
my scan log:
{"rawValue":"ASDSADASDAS","bytes":[71,80,76,50,50,48,53,48,53,50,56,50,57],"cornerPoints":[[686,973],[681,1069],[165,1094],[174,1008]],"format":"CODE_128","valueType":"TEXT","displayValue":"ASDSADASDAS"} ⚡️ [log] - Detection Corner Points: [[57,519.7821655273438],[357,519.7821655273438],[357,644.7821655273438],[57,644.7821655273438]] ⚡️ [log] - Current Corner Points: [[686,973],[681,1069],[165,1094],[174,1008]] ⚡️ [log] - Conditions not met. Returning...It makes problem because as u can see in logs scan never getting the value and enter never ending loop, when im commenting corner detection scanner works fine.
Would be greatfull for some tips/help
similar case: there is opened issue on this case.
#115
Expected behavior
I expect the success scan when catch barcode in my scan box.
Reproduction
none
Steps to reproduce
Other information
No response
Capacitor doctor
Latest Dependencies:
@capacitor/cli: 5.7.0
@capacitor/core: 5.7.0
@capacitor/android: 5.7.0
@capacitor/ios: 5.7.0
Installed Dependencies:
@capacitor/core: 5.3.0
@capacitor/ios: 5.0.5
@capacitor/cli: 5.0.5
@capacitor/android: 5.3.0
[success] iOS looking great! 👌
Before submitting
The text was updated successfully, but these errors were encountered: