Description
What happened?
Turns out frame calculation of OCR for photos on IOS is messed up in @react-native-ml-kit/text-recognition.
Have found that the frame is misinterpreted and should be:
top: is misinterpreted as left
height and width are switched.
and
left: would be screen width - their top - their height.
Version
@react-native-ml-kit/barcode-scanning: version@react-native-ml-kit/face-detection: version@react-native-ml-kit/identify-languages: version@react-native-ml-kit/image-labeling: version@react-native-ml-kit/text-recognition: version@react-native-ml-kit/translate-text: version
Which ML Kit packages do you use?
- @react-native-ml-kit/barcode-scanning
- @react-native-ml-kit/face-detection
- @react-native-ml-kit/identify-languages
- @react-native-ml-kit/image-labeling
- @react-native-ml-kit/text-recognition
- @react-native-ml-kit/translate-text
What platforms are you seeing this issue on?
- Android
- iOS
System Information
RN 0.73.6
System:
OS: macOS 14.6.1
CPU: (8) arm64 Apple M3
Memory: 84.88 MB / 16.00 GB
Shell:
version: "5.9"
path: /bin/zsh
Binaries:
Node:
version: 18.20.0
path: ~/.nvm/versions/node/v18.20.0/bin/node
Yarn:
version: 1.22.22
path: /opt/homebrew/bin/yarn
npm:
version: 10.5.0
path: ~/.nvm/versions/node/v18.20.0/bin/npm
Watchman:
version: 2024.09.02.00
path: /opt/homebrew/bin/watchman
Managers:
CocoaPods:
version: 1.15.2
path: /opt/homebrew/bin/pod
SDKs:
iOS SDK:
Platforms:
- DriverKit 24.0
- iOS 18.0
- macOS 15.0
- tvOS 18.0
- visionOS 2.0
- watchOS 11.0
Android SDK:
API Levels:
- "28"
- "31"
- "33"
- "34"
Build Tools:
- 28.0.3
- 30.0.3
- 31.0.0
- 33.0.1
- 34.0.0
System Images:
- android-34 | Google APIs ARM 64 v8a
- android-34 | Google Play ARM 64 v8a
Android NDK: 25.1.8937393
IDEs:
Android Studio: 2024.1 AI-241.15989.150.2411.11948838
Xcode:
version: 16.0/16A242d
path: /usr/bin/xcodebuild
Languages:
Java:
version: 17.0.12
path: /opt/homebrew/bin/javac
Ruby:
version: 3.3.5
path: /opt/homebrew/opt/ruby/bin/ruby
npmPackages:
"@react-native-community/cli": Not Found
react:
installed: 18.2.0
wanted: 18.2.0
react-native:
installed: 0.73.6
wanted: 0.73.6
react-native-macos: Not Found
npmGlobalPackages:
"react-native": Not Found
Android:
hermesEnabled: true
newArchEnabled: false
iOS:
hermesEnabled: true
newArchEnabled: false
Steps to Reproduce
const {blocks} = await TextRecognition.recognize(`file://${photo.path}`);
expand lines then filter blocks somehow. (or don't)
use "react-native-fs"
to copy original image to somewhere on phone and save json encoded blocks to json file on phone.
Transfer files to PC.
Use small html/js/css file to position frames on top of original image.
badDiv.style.cssText = `top: ${value.frame.top}px; left: ${value.frame.left}px; height: ${value.frame.height}px; width: ${value.frame.width}px`;
goodDiv.style.cssText = `top: ${value.frame.left}px; right: ${value.frame.top}px; height: ${value.frame.width}px; width: ${value.frame.height}px`;
The red bordered blocks (badDiv) are messed up.
The green bordered blocks (goodDiv) with the properties switched around are in the correct positions.