Pyodbc install erroring

[FabianDS2]

I am relatively new to Airflow, Docker, and Bitnami but I am having trouble getting pyodbc to be installed on the bitnami airflow containers. I want to be able to use Airflow on Azure for work projects so that's how I found out about the bitnami-docker-airflow project.

I have followed the directions from this page: https://github.com/bitnami/bitnami-docker-airflow/blob/master/README.md

I started with the curl -sSL https://raw.githubusercontent.com/bitnami/bitnami-docker-airflow/master/docker-compose.yml > docker-compose.yml and got the docker-compose.yaml file in my documents folder. I then went in to make some changes to mount a local folder with DAG files I wanted to use and mounted another folder that had a requirements.txt file in it.

The lines below in the docker-compose.yml are the ones I added for mounting. ./dags and ./packages are the folders that the DAG .py and requirement.txt files are in. My docker-compose file will be attached.

• ./dags:/opt/bitnami/airflow/dags # added this!
• ./packages:/bitnami/python/ # added this!

The requirements.txt has the text pyodbc===4.0.30 as the only content in the file.

I run docker-compose up to get the containers up and running.

The output shows me that pyodbc install is failing but I can't exactly figure out what the source of the error is and what could fix it. Will attach the copy and pasted output that shows the error, having a hard time interpreting it. I have tried the docker-compose without the requirements file mounting and airflow will start up and I can see my DAGs at localhost:8080 as I would expect. I want to be able to use pyodbc though in my DAGs

Please let me know if I can provide more context

docker compose errors.txt
docker-compose file.txt

[rafariossaa]

Hi,
The issue here is that our airflow image does not include gcc (C compiler) hence this error:

airflow-scheduler_1  |   unable to execute 'gcc': No such file or directory
airflow-scheduler_1  |   error: command 'gcc' failed with exit status 1

[FabianDS2]

Thank you @rafariossaa for the explanation. I am relatively new to Docker so is there a location in the Dockerfile/docker.compose.yaml that I should look at or do more research where I could add that dependency? For future planning, as I want to use the bitnami image for production workflow in Azure, is there a way to add that dependency on the Azure image? Maybe once it's set-up and deployed? Looking forward to hearing any guidance you can give, anything is helpful!

[rafariossaa]

Hi,
You can add build-essential to the list of packages here, and build your own image.
I am not very sure about your Azure question. Once you have you own image, push it to a repository (eg. docker hub) and you will be able to use from wherever you need.

[stale]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

[github-actions]

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.

[msmerc]

We also see this issue!

The airflow documentation says to do:

pip install apache-airflow[odbc]

http://airflow.apache.org/docs/apache-airflow-providers-odbc/stable/connections/odbc.html

But this leads to the same gcc error.

We'd expect this to work out-of-the-box given that all the other airflow providers do. Is it something that could be added do you think?

[rafariossaa]

Hi,
I am creating an internal task to evaluate and add the build tools to our container image.
We will come back as soon as we have news.

[msmerc]

@rafariossaa - thanks, appreciated.

If you do airflow info in this image once it is running the apache-airflow-providers-odbc is conspicuously absent. I'm guessing it's all related.

[rafariossaa]

Hi,
Yes, I guess it is as consequence of the compiling error.

[ShawnMcGough]

Bitnami has suggested here that this is by design.

This is how I solved it.
https://www.shawnmcgough.com/airflow-connect-to-sql-server-mssql/