Skip to content

Cannot find spark cluster log and create a table #1314

@LanitaGuan

Description

@LanitaGuan

Overwatch Version
0.8.2.0

Describe the bug
I understand that Spark related log information requires opening logging under the cluster and specifying the location where the logs are stored in order to load correctly.
But now I have opened the log of one of the all-purpose clusters and saved it in the catalog volumes. (The position of this volume is consistent with the ETL and consumer positions of Overwatch) When executing Overwatch, an error occurs: Event Logs DF is empty, Exiting. And 1006-Bronze_SparkEventLogs is a prerequisite table for all subsequent Spark analyses, so no Spark related tables have been created.
Some debug: deployment workspaces, the cluster running Overwatch has already set "Azure Storage Auth Config", and there is also relevant cluster data under the volume specified by the cluster in the target workspace

I found one related ticket here: 767 and 1055, - but seems "Azure Storage Auth Config" cannot help(ticket 767) since I already set and other table can generate successfully - rerun cannot work neither (ticket 1055)

Issue Reports
GenericTicket_Diagnostics

Metadata

Metadata

Assignees

No one assigned

    Labels

    user_questionQuestions coming from users

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions