Skip to content

feature-request: proceed with tuning even if ClearML (or other trackers) fail #653

@HarikrishnanBalagopal

Description

@HarikrishnanBalagopal

Is your feature request related to a problem? Please describe.

Proceed with tuning even if ClearML (or other trackers) fail.
Currently it crashes if ClearML tracker throws an exception/error.
We ran into this in a situation where the ClearML server's certificate had expired, it was blocking all tuning jobs.

Describe the solution you'd like

Do not crash. Print a warning/error and proceed with tuning.

Describe alternatives you've considered

This can be enabled by a flag if necessary. Example: --ignore-tracker-errors

Additional context

Add any other context about the feature request here.

Metadata

Metadata

Assignees

Labels

wontfixThis will not be worked on

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions