Description
Summary
We are currently utilizing Databend to create a stage using HDFS, as detailed in the Databend documentation. This implementation requires compiling Databend with Java and providing full support for the Hadoop package, which involves a highly complex configuration.
The troubleshooting process for any errors encountered during this setup has proven to be extremely intricate.
To simplify our operations, I propose replacing the current HDFS implementation with OpenDAL HDFS Native. This solution can streamline our processes, as outlined in the following resources:
OpenDAL HDFS Native Documentation
OpenDAL HDFS Native GitHub Repository
Adopting OpenDAL HDFS Native could reduce complexity and improve our efficiency in working with HDFS in the Databend | Databend Cloud environment.