Design: Validating user models and callables with data at `__init__` time

[jklaise]

This issue is to track discussions around what validation should happen at __init__ time, specifically related to validating inputs/outputs of user models and other callables. Initial discussion can be found here: #523 (comment)

To summarize, generally we get our hands on user data at explain time only by which time if there are issues with the model input/output an error may be raised. Ideally we would catch any issues at __init__ time, but this is hard unless the user gives us a sample of their data or tells us enough information about the input data so we can generate dummy data and put it through for validation purposes.