How to iterate on same transform to fetch data in batches

[Vishusanap]

Hi, I am new to Apache Hop and evaluating Hop as a replacement for my current ETL tool.
I am using "Rest Client" Transform to fetch data from a rest service and able to fetch the data.

That rest service provides data in pages. I want to fetch all the data for that I need to pass parameter page=1 then page=2 and so on.
how to achieve this in the Apache Hop.

[dmainou]

Take a look at the samples. Specifically at the loops

[dmainou]

Also the documentation
https://hop.apache.org//manual/latest/best-practices/index.html#_loops

[CatalyseRichard]

Diego's answer is the correct one - but it points to several different methods by which you could achieve this.
We have had a similar requirement and used a pipeline executor to set up a repeating control loop. This contained some control logic and a pipeline that executed the REST API call and extracted not just the data set but also the page info and nextpage url. This is fed back as variables into the control loop to evaluate whether a further loop is required or not.

[Vishusanap]

Yes, We are also trying to develop the same flow. it will help if you could send the flow or image of your design flow.
Thanks!

[bamaer]

there is a "loops" folder in the samples project that contains the examples that are mentioned in the documentation page @dmainou referred to.

[fwsdataeng-1]

I know this is a bit late, you can do a loop using the repeat step on your main workflow, just call your module from there and define a loop terminator: