Skip to content

Incorrect handling of consecutive faulty items in chunk scanning #4370

Open
@fmbenhassine

Description

@fmbenhassine

As of v5.0.1, the FaultTolerantChunkProcessor is unable to skip two consecutive faulty items when scanning chunks.

Here is a failing test (currently disabled in FaultTolerantChunkProcessorTests):

@Test
void testWriteRetryOnTwoExceptions() throws Exception {
	SimpleRetryPolicy retryPolicy = new SimpleRetryPolicy();
	retryPolicy.setMaxAttempts(2);
	batchRetryTemplate.setRetryPolicy(retryPolicy);
	processor.setWriteSkipPolicy(new AlwaysSkipItemSkipPolicy());
	processor.setItemWriter(new ItemWriter<String>() {
		@Override
		public void write(Chunk<? extends String> chunk) throws Exception {
			if (chunk.getItems().contains("fail")) {
				throw new IllegalArgumentException("Expected Exception!");
			}
		}
	});
	Chunk<String> inputs = new Chunk<>(Arrays.asList("3", "fail", "fail", "4"));
	Exception exception = assertThrows(RuntimeException.class, () -> processor.process(contribution, inputs));
	assertEquals("Expected Exception!", exception.getMessage());
	// first retry
	exception = assertThrows(RuntimeException.class, () -> processor.process(contribution, inputs));
	assertEquals("Expected Exception!", exception.getMessage());
	// retry exhausted, now scanning
	processor.process(contribution, inputs);
	// skip on this attempt
	exception = assertThrows(RuntimeException.class, () -> processor.process(contribution, inputs));
	assertEquals("Expected Exception!", exception.getMessage());
	// 2nd exception detected
	exception = assertThrows(RuntimeException.class, () -> processor.process(contribution, inputs));
	assertEquals("Expected Exception!", exception.getMessage());
	// still scanning
	processor.process(contribution, inputs);
	assertEquals(2, contribution.getSkipCount());
	assertEquals(2, contribution.getWriteCount());
	assertEquals(0, contribution.getFilterCount());
}

This test started failing after resolving #4314.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions