[BUG]: Analyzer and Transpiler not working with arm64

[sujanchy]

Is there an existing issue for this?

• I have searched the existing issues

Category of Bug / Issue

Application crashed

Current Behavior

I am using Mac with apple chip (arm64) and getting error <Signals.SIGABRT: 6>. I have searched the error and it seems binary compatibility issue.

.databricks/labs/remorph-transpilers/bladebridge/lib/.venv/lib/python3.10/site-packages/databricks/labs/bladebridge/Converter/bin/MacOS/dbxconv: Mach-O 64-bit executable x86_64

Details Error

Analyzer:

ERROR [d.l.bladespector.analyzer] Analysis failed
Traceback (most recent call last):
File "/Users//.databricks/labs/lakebridge/state/venv/lib/python3.10/site-packages/databricks/labs/bladespector/analyzer.py", line 98, in _run_binary
raise subprocess.CalledProcessError(process.returncode, args)
subprocess.CalledProcessError: Command '['/Users//.databricks/labs/lakebridge/state/venv/lib/python3.10/site-packages/databricks/labs/bladespector/Analyzer/MacOS/analyzer', '-d', '/Users//gdw-migration/informatica', '-r', '/var/folders/h0/nby6qk911rb2gwjjkmdblktr0000gn/T/tmpxhwatahv/lakebridge-analyzer-results.xlsx', '-t', 'INFA']' died with <Signals.SIGABRT: 6>.
15:15:10 ERROR [d.l.lakebridge.analyze] RuntimeError: Analysis failed

Transpiler
TranspileError(code=FAILURE, kind=PARSING, severity=ERROR, path='informatica/output/', message='Command '['/Users//.databricks/labs/remorph-transpilers/bladebridge/lib/.venv/lib/python3.10/site-packages/databricks/labs/bladebridge/Converter/bin/MacOS/dbxconv', 'INFA', '-u', 'base_infapc2databricks_pyspark.json', '-n', 'transpiled', '-i', 'originals/', '-v', '-H', 'f20b266bb1a5f656b62f74a4a1ce22dd69649e55']' died with <Signals.SIGABRT: 6>.')

Expected Behavior

output excel and output sql files

lsp-server.log

Steps To Reproduce

databricks labs lakebridge transpile --input-source informatica --target databricks --input-source ./informatica --output-folder ./informatica/output

databricks labs lakebridge transpile --source-dialect "informatica (desktop edition)" --input-source ./informatica --output-folder ./informatica/output

databricks labs lakebridge analyze --source-directory /informatica --report-file /informatica/inventory.xlsx

Relevant log output or Exception details

I am using Mac with apple chip (arm64) and getting error <Signals.SIGABRT: 6>. I have searched the error and it seems binary compatibility issue. 

.databricks/labs/remorph-transpilers/bladebridge/lib/.venv/lib/python3.10/site-packages/databricks/labs/bladebridge/Converter/bin/MacOS/dbxconv: Mach-O 64-bit executable **x86_64**

Details Error

**Analyzer**:

ERROR [d.l.bladespector.analyzer] Analysis failed
Traceback (most recent call last):
  File "/Users/<myusername>/.databricks/labs/lakebridge/state/venv/lib/python3.10/site-packages/databricks/labs/bladespector/analyzer.py", line 98, in _run_binary
    raise subprocess.CalledProcessError(process.returncode, args)
subprocess.CalledProcessError: Command '['/Users/<myusername>/.databricks/labs/lakebridge/state/venv/lib/python3.10/site-packages/databricks/labs/bladespector/Analyzer/MacOS/analyzer', '-d', '/Users/<myusername>/gdw-migration/informatica', '-r', '/var/folders/h0/nby6qk911rb2gwjjkmdblktr0000gn/T/tmpxhwatahv/lakebridge-analyzer-results.xlsx', '-t', 'INFA']' died with <Signals.SIGABRT: 6>.
15:15:10    ERROR [d.l.lakebridge.analyze] RuntimeError: Analysis failed

**Transpiler**
TranspileError(code=FAILURE, kind=PARSING, severity=ERROR, path='informatica/output/<xml files>', message='Command '['/Users/<myusername>/.databricks/labs/remorph-transpilers/bladebridge/lib/.venv/lib/python3.10/site-packages/databricks/labs/bladebridge/Converter/bin/MacOS/dbxconv', 'INFA', '-u', 'base_infapc2databricks_pyspark.json', '-n', 'transpiled', '-i', 'originals/<xml files>', '-v', '-H', 'f20b266bb1a5f656b62f74a4a1ce22dd69649e55']' died with <Signals.SIGABRT: 6>.')

Logs Confirmation

• I ran the command line with --debug
• I have attached the lsp-server.log under USER_HOME/.databricks/labs/remorph-transpilers/<converter_name>/lib/lsp-server.log

Sample Query

Operating System

macOS

Version

latest via Databricks CLI

[simone-dbx-labs]

Hi @sujanchy
The analyzer and transpiler binaries are currently compiled for x86_64 (Intel) architecture. On Apple Silicon Macs (M1/M2/M3), these binaries require Rosetta 2 to run.
Could you please check if Rosetta 2 is installed?You can verify by running:
/usr/bin/pgrep -q oahd && echo "Rosetta 2 is installed" || echo "Rosetta 2 is not installed"
If Rosetta 2 is not installed, you can install it with:
softwareupdate --install-rosetta --agree-to-license
After installing Rosetta 2, please try running the analyzer and transpiler commands again.

If you already have Rosetta 2 installed and are still experiencing this issue, please let us know and we'll investigate further. There may be additional compatibility considerations we need to address.

[sujanchy]

Hi @simone-dbx-labs , thanks for your response, I did come across this solution and tried and still same error

I also tried open the terminal using arch -x86_64 zsh and checked, it become x86_64 instead of arm64, but still facing same issue

during transpiler installation, I saw an error (not sure, this is the reason), but later I fixed the SSL error and reinstall and still no change
" raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='repo.maven.apache.org', port=443): Max retries exceeded with url: /maven2/com/databricks/labs/databricks-morph-plugin/maven-metadata.xml (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1017)')))"