Issues Creating a Dataset with a Firebird Database (No Schemas)

[saulfrr]

Hi everyone,

I’ve been running into problems when trying to create a dataset using a Firebird 2.5 database as the data source in Superset. According to the documentation, Firebird should be supported, and I’ve successfully connected my Firebird database. I can run SQL queries against it in Superset’s SQL Lab interface without any issue. However, when I attempt to create a new dataset, I’m stuck.

I'm currently just testing with Docker containers but following a Python installation, not using the Docker images.

The workflow to create a dataset in Superset requires selecting a schema before choosing a table. The problem is that Firebird databases don’t use schemas. As a result, the schema dropdown is empty and mandatory, which blocks me from proceeding.

What I’ve Tried:

1. Custom DB Engine Spec:

I created a custom FirebirdEngineSpec class that overrides get_schema_names and get_table_names to return an empty schema list and fetch tables without a schema. I followed the general advice from other discussions and the documentation on how to register a custom engine spec using CUSTOM_ENGINE_SPECS in superset_config.py.

from superset.db_engine_specs.base import BaseEngineSpec

class  FirebirdEngineSpec(BaseEngineSpec):
	engine = "firebird"

	@classmethod
	def  get_schema_names(cls, inspector):
		# Firebird does not have schemas, return an empty list
		return []
	
	@classmethod
	def  get_table_names(cls, inspector, schema=None):
		# Ignore the schema parameter since Firebird doesn't support schemas
		return  inspector.get_table_names()

2. Adjusting PYTHONPATH & Config:

I ensured that my custom engine spec file (firebird_engine_spec.py) and superset_config.py are both inside the Docker image and that PYTHONPATH is set to include /app. I can confirm that the file is present and the config is loaded since other changes in superset_config.py are reflected in the running instance.

3. Checking Superset & SQLAlchemy Versions:

I’m running a recent version of Superset (installed via pip install apache-superset) and have verified that the Firebird driver (fdb) is installed and functional.

Despite these attempts, the UI still insists on selecting a schema before listing tables. This makes it impossible to proceed with dataset creation for Firebird databases.

SQL Queries:

I can actually run queries on the Firebird DB

If I try to save this query as a Dataset then I get this error

The Dataset gets created (with errors)

Then when opening the Dataset it is broken

Questions:

• Has anyone successfully created a dataset for a Firebird database in Superset without schemas?
• Are there any known workarounds or approaches that don’t rely on modifying the frontend code?
• Is there a recommended pattern or a future enhancement planned in Superset to handle databases that do not support schemas?

Any guidance or insights would be greatly appreciated.

Thank you! 🫡

[dosubot[bot]]

Hey @saulfrr! I'm here to help you with any bugs, questions, or contributions you have in mind. Let's tackle this issue together!

To address the issue of creating a dataset in Superset with a Firebird database that does not use schemas, here are some suggestions and insights:

1. Custom DB Engine Spec: You've already taken a significant step by creating a custom FirebirdEngineSpec class. Ensure that your implementation of get_schema_names returns an empty list, as Firebird does not use schemas. Your get_table_names method should also correctly fetch tables without considering schemas, which seems to be correctly implemented in your code.

2. Superset Configuration: Make sure that your custom engine spec is correctly registered in superset_config.py using the CUSTOM_ENGINE_SPECS configuration. This should look something like this:

   CUSTOM_ENGINE_SPECS = {
       "firebird": "path.to.your.FirebirdEngineSpec",
   }

   Ensure that the path is correct and that the module is importable within the Superset environment.

3. Schema Selection Bypass: Unfortunately, there is no direct configuration in Superset to bypass the schema selection step for databases that do not use schemas. However, your custom engine spec should handle this by ensuring that the schema-related UI elements are not blocking. If the UI still insists on a schema, it might be due to a frontend validation that is not aware of your custom backend logic.

4. Frontend Adjustments: While you prefer not to modify the frontend, it might be necessary to ensure that the schema dropdown does not block the dataset creation process. This could involve customizing the frontend code to handle cases where the schema list is empty.

5. Future Enhancements: As of now, there is no specific documentation or planned enhancements in Superset to handle databases without schemas more effectively. This might be an area for future development or community contribution.

6. Workarounds: As a workaround, you might consider creating a virtual dataset directly from SQL Lab by saving a query as a dataset. This approach might bypass some of the schema-related issues, although you've mentioned encountering errors with this method.

If these steps do not resolve the issue, further investigation into the Superset codebase or community discussions might be necessary to identify any additional customizations or patches required.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[Physicalpixel]

@saulfrr Facing the same exact issue, did you find a solution to this? Thank you in advance for any information you can provide.