Simplify paged queries - Python Generators

[santhoshramaraj]

Some client methods return paged responses. The user is required to handle the paged responses using continuation_token appropriately in the application.

Example - DataFrameClient().query_tables() , SpecClient().query_specs()

These paged responses can be simplified by using Generators in Python for easy use with loops and extracting the underlying paged data.

[ChrisStrykesAgain]

I think there could be value here to implementing a decorator and just using it any time there's an API call that uses a continuation token. It seems like that would be cleaner and reduce code duplication.

[santhoshramaraj]

The decorator concept satisfies the continuation token handling, but in most cases, users use these in some sort of loop, and making them a generator and exposing the underlying data intuitively is better than requiring the user to extract the data of interest.

For instance, query_tables() can directly return an iterable of TableMetadata instead of the user needing to invoke PagedTables.tables

[ChrisStrykesAgain]

I see your point; I was oversimplifying. I do feel there's some kind of reusable approach though. I'll think it through and post something more intelligent later if I can get my thoughts aligned.

[santhoshramaraj]

Raised this request on uplink as the development has resumed prkumar/uplink#322