Skip to content

KAFKA-10789: Streamlining Tests in ChangeLoggingKeyValueBytesStoreTest #18816

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: trunk
Choose a base branch
from

Conversation

leaf-soba
Copy link
Contributor

  1. use mocked InMemoryKeyValueStore to streamlined the unit test
  2. minor refactor including remove unnecessary public and rename the variable inside the method with a different name with outside of the method.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

JUnit5 test classes and methods should have default package visibility
Local variables should not shadow class fields
@github-actions github-actions bot added triage PRs from the community streams tests Test fixes (including flaky tests) labels Feb 6, 2025
@leaf-soba leaf-soba changed the title KAFKA-10789; Streamlining Tests in ChangeLoggingKeyValueBytesStoreTest KAFKA-10789: Streamlining Tests in ChangeLoggingKeyValueBytesStoreTest Feb 7, 2025
@leaf-soba
Copy link
Contributor Author

Hi @ijuma, @chia7712, @gongxuanzhang, could you please take a look?

@gongxuanzhang
Copy link
Collaborator

Pr needs a tag about Ci, I can't do it. sorry

Copy link

A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the triage label
should be removed to prevent this automation from happening again.

fix import static org.mockito.ArgumentMatchers.anyString;
Copy link

A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the triage label
should be removed to prevent this automation from happening again.

@leaf-soba
Copy link
Contributor Author

Hi @chia7712, in the CI process, there are three flaky tests that are unrelated to this PR. Could you let me know how to re-run the CI process to get them to pass?

Copy link

A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the triage label
should be removed to prevent this automation from happening again.

Copy link
Member

@mjsax mjsax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR. I just discovered it :)

Couple of comments/questions.

import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;

@SuppressWarnings("rawtypes")
@ExtendWith(MockitoExtension.class)
@MockitoSettings(strictness = Strictness.STRICT_STUBS)
public class ChangeLoggingKeyValueBytesStoreTest {
class ChangeLoggingKeyValueBytesStoreTest {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can a test class be package private?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In JUnit 5, test classes and methods can have default (package-private) visibility—they don’t need to be public to be recognized and executed. However, I noticed that most of the existing Kafka unit tests still use public, so I’ll follow the current convention and revert the change for consistency.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks to the info. Interesting. Did not know that. -- Guess we use public historically, and I also don't see much value to change it to package private -- sounds like a not of unnecessary noise on PRs, w/o any clear benefit.

mockMap.put(invocation.getArgument(0), invocation.getArgument(1));
StoreQueryUtils.updatePosition(innerMock.getPosition(), context);
return null;
}).when(innerMock).put(any(Bytes.class), any(byte[].class));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we use doAnswer(...).when(...) here?

In mockGet we use when(...).thanAnswer(...) what I find much easier to read.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the question!

We use doAnswer(...).when(...) here because put is a void method. Mockito requires this syntax for mocking void methods — when(...).thenAnswer(...) only works for methods that return a value.

In contrast, get returns a value, so we can use when(...).thenAnswer(...) there, which I agree is a bit more readable.

    @Override
    public synchronized void put(final Bytes key, final byte[] value) {
        putInternal(key, value);
    }

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah. My bad. Thanks.

private void mockPosition() {
when(innerMock.getPosition()).thenReturn(Position.emptyPosition());
}
private void mockGet(final Map<Bytes, byte[]> mockMap) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we using mockMap? It seems unnecessarily complex? -- It seems much more straightforward to just "expect" calls into the innerMock store?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing that out!

We're using mockMap here to simulate the internal state of the underlying store, since the tests in CachingInMemoryKeyValueStoreTest are being streamlined to rely on a mocked underlyingStore as per the KIP-614 review discussion.

Because innerMock is a mock and doesn’t retain state on its own, we use mockMap to mimic how the real store would behave across multiple interactions — especially for verifying behavior like reads after writes. This lets us preserve the semantics of the store while still keeping the actual store mocked, as requested.

That said, I'm open to simplifying it further if there's a cleaner way to preserve the same test coverage and behavior expectations.

https://issues.apache.org/jira/browse/KAFKA-10789

public void shouldDelegateInit() {
final InternalMockProcessorContext context = mockContext();
final KeyValueStore<Bytes, byte[]> innerMock = mock(InMemoryKeyValueStore.class);
void shouldDelegateInit() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question as above for the class: I thought test methods must be public?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In JUnit 5, test classes and methods can have default (package-private) visibility—they don’t need to be public to be recognized and executed. However, I noticed that most of the existing Kafka unit tests still use public, so I’ll follow the current convention and revert the change for consistency.

@mjsax mjsax removed triage PRs from the community needs-attention labels Apr 5, 2025
revert remove unnecessary public
 insert empty lines between methods
@leaf-soba leaf-soba requested a review from mjsax April 10, 2025 02:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-approved streams tests Test fixes (including flaky tests)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants