Meta model for Benchmarking Uncertaitny prediction of DNN using synthetic generated dataset

Uncertainty Benchmark

Question

• Uncertainty is a subjective metric.
• Can this subjective knowledge be written as some rules using some format?
• Use this rules to generate corresponding dataset for training and testing with corresponding ground truth of uncertainty.
• Now you have compare the expected and predicted uncertainty.

Deep learning Testing

• The test oracle problem for Vision-based AI systems

• Can we write the different testing rules :

  • Integration test
  • unit test

• all these require testing

• Can we se the same approach