sometimes you may only want to index the data every n hours/days/weeks and we should allow you to do that through the yaml