You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
write a layer that does an assessment to test that the content is equivalent between source input and processed enriched output
one way of doing this would be putting round trip it? source -> enriched/processed -> source
Can you generate the source again from the information that you have. See if the two differs or not.
Remember, the source may not be schemas. As in may not be structured schemas. It could be pydantic code bases
This is going to be a dynamical state, which is continuously evolving. You might have to set a cron jam that looks at them. And if Dandi has changed something in its code base, it will re-import at that point in time. But it's not like re-ingesting everything. Only update the relevant updates.