Skip to content

Tomandersen/handshake rewrite #6015

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 56 commits into
base: tomandersen/abortSnapshotListenersOnTerminate
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
2be4616
Clear Cache WIP
tom-andersen May 15, 2024
cbf5987
Clear Cache WIP
tom-andersen May 16, 2024
9f8cffb
Clear Cache WIP
tom-andersen May 16, 2024
40a182b
Clear Cache WIP
tom-andersen May 16, 2024
3d948a7
Merge remote-tracking branch 'origin/master' into tomandersen/cleanCache
tom-andersen May 16, 2024
28aaaad
Clear Cache WIP Test
tom-andersen May 17, 2024
b9d74fc
Merge remote-tracking branch 'origin/tomandersen/dbToken' into tomand…
tom-andersen May 17, 2024
25809f9
Fix
tom-andersen May 17, 2024
aaaba31
Merge branch 'tomandersen/dbToken' into tomandersen/cleanCache
tom-andersen May 17, 2024
d625571
Merge branch 'tomandersen/dbToken' into tomandersen/cleanCache
tom-andersen May 17, 2024
f982bbf
WIP
tom-andersen May 22, 2024
dbafb0e
Fix
tom-andersen May 17, 2024
b4c144a
Fix health metrics failures (#5969)
daymxn May 17, 2024
d6bc28e
WIP
tom-andersen May 22, 2024
0d0235c
Theoretical Implementation
tom-andersen May 22, 2024
bafd943
Merge remote-tracking branch 'origin/tomandersen/watchHandshake' into…
tom-andersen May 22, 2024
a3c7a61
Merge branch 'tomandersen/dbToken' into tomandersen/cleanCache
tom-andersen May 22, 2024
59e730a
Merge branch 'tomandersen/cleanCache' into tomandersen/watchHandshake
tom-andersen May 22, 2024
49f7d70
Fix spec tests
tom-andersen May 24, 2024
685a291
WIP
tom-andersen May 27, 2024
b853c4d
Merge remote-tracking branch 'origin/tomandersen/dbToken' into tomand…
tom-andersen May 27, 2024
bec7890
Merge branch 'tomandersen/dbToken' into tomandersen/cleanCache
tom-andersen May 27, 2024
386631c
Merge remote-tracking branch 'origin/tomandersen/cleanCache' into tom…
tom-andersen May 27, 2024
53a39f9
Rewrite
tom-andersen Jun 4, 2024
f2f70dd
Merge remote-tracking branch 'origin/master' into tomandersen/handsha…
tom-andersen Jun 4, 2024
1706efe
Cleanup
tom-andersen Jun 4, 2024
a3c17e6
Cleanup
tom-andersen Jun 4, 2024
0931087
Merge branch 'tomandersen/dbToken' into tomandersen/handshakeRewrite
tom-andersen Jun 5, 2024
4a99f58
Merge branch 'tomandersen/dbToken' into tomandersen/handshakeRewrite
tom-andersen Jun 5, 2024
203873d
Comments
tom-andersen Jun 5, 2024
030f892
Merge remote-tracking branch 'origin/tomandersen/handshakeRewrite' in…
tom-andersen Jun 5, 2024
d59c32f
Merge remote-tracking branch 'origin/tomandersen/streamRefactor' into…
tom-andersen Jun 5, 2024
9bc7692
Fix
tom-andersen Jun 6, 2024
6a0fd9a
Refactor parts back into FirebaseFirestore
tom-andersen Jun 6, 2024
45f22b4
Fix
tom-andersen Jun 6, 2024
6cba3f4
Cleanup
tom-andersen Jun 6, 2024
03c70b9
Cleanup
tom-andersen Jun 6, 2024
fdc425d
Add tests
tom-andersen Jun 11, 2024
d525b64
Merge branch 'tomandersen/streamRefactor' into tomandersen/handshakeR…
tom-andersen Jun 11, 2024
6b2d734
Merge branch 'tomandersen/streamRefactor' into tomandersen/handshakeR…
tom-andersen Jun 12, 2024
e58f8fc
Comments and tests
tom-andersen Jun 12, 2024
2262fdc
Simplify
tom-andersen Jun 12, 2024
2c678e3
Add tests
tom-andersen Jun 13, 2024
98112dd
Merge branch 'tomandersen/streamRefactor' into tomandersen/handshakeR…
tom-andersen Jun 14, 2024
dcba2f6
Merge remote-tracking branch 'origin/tomandersen/componentProviderRef…
tom-andersen Jun 17, 2024
7c049f0
Fix after merge
tom-andersen Jun 17, 2024
35c1090
Merge branch 'tomandersen/componentProviderRefactor' into tomandersen…
tom-andersen Jun 17, 2024
9f33949
Merge remote-tracking branch 'origin/tomandersen/handshakeTest' into …
tom-andersen Jun 17, 2024
92c73ee
fix after merge
tom-andersen Jun 17, 2024
586db20
Merge branch 'tomandersen/handshakeTest' into tomandersen/handshakeRe…
tom-andersen Jun 17, 2024
e4c9596
Merge remote-tracking branch 'origin/tomandersen/firestoreClientProvi…
tom-andersen Jun 19, 2024
e61b1ed
Fix after merge
tom-andersen Jun 19, 2024
3822e96
Merge remote-tracking branch 'origin/tomandersen/firestoreClientProvi…
tom-andersen Jul 29, 2024
f689342
Fix after merge
tom-andersen Jul 29, 2024
5746a58
Merge remote-tracking branch 'origin/tomandersen/abortSnapshotListene…
tom-andersen Jul 31, 2024
720da12
Merge branch 'tomandersen/abortSnapshotListenersOnTerminate' into tom…
tom-andersen Aug 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
package com.google.firebase.firestore;

import android.content.Context;
import androidx.core.util.Supplier;
import com.google.firebase.FirebaseApp;
import com.google.firebase.firestore.auth.CredentialsProvider;
import com.google.firebase.firestore.auth.User;
Expand All @@ -31,17 +32,17 @@ public static FirebaseFirestore newFirebaseFirestore(
Context context,
DatabaseId databaseId,
String persistenceKey,
CredentialsProvider<User> authProvider,
CredentialsProvider<String> appCheckProvider,
Supplier<CredentialsProvider<User>> authProviderFactory,
Supplier<CredentialsProvider<String>> appCheckTokenProviderFactory,
Function<FirebaseFirestoreSettings, ComponentProvider> componentProviderFactory,
FirebaseApp firebaseApp,
FirebaseFirestore.InstanceRegistry instanceRegistry) {
return new FirebaseFirestore(
context,
databaseId,
persistenceKey,
authProvider,
appCheckProvider,
authProviderFactory,
appCheckTokenProviderFactory,
componentProviderFactory,
firebaseApp,
instanceRegistry,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
package com.google.firebase.firestore;

import androidx.test.ext.junit.runners.AndroidJUnit4;
import org.junit.runner.RunWith;

@RunWith(AndroidJUnit4.class)
public class FirestoreClientProviderTest {

// TODO(requires backend/emulator support)

}
Original file line number Diff line number Diff line change
Expand Up @@ -38,12 +38,17 @@
import com.google.firebase.firestore.testutil.IntegrationTestUtil;
import com.google.firebase.firestore.util.AsyncQueue;
import com.google.firebase.firestore.util.AsyncQueue.TimerId;
import com.google.firestore.v1.InitResponse;
import com.google.protobuf.ByteString;
import io.grpc.Status;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.Semaphore;
import java.util.concurrent.TimeUnit;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.Timeout;
import org.junit.runner.RunWith;

class MockCredentialsProvider extends EmptyCredentialsProvider {
Expand All @@ -68,6 +73,9 @@ public List<String> observedStates() {

@RunWith(AndroidJUnit4.class)
public class StreamTest {

@Rule public Timeout timeout = new Timeout(10, TimeUnit.SECONDS);

/** Single mutation to send to the write stream. */
private static final List<Mutation> mutations =
Collections.singletonList(setMutation("foo/bar", map()));
Expand Down Expand Up @@ -96,7 +104,7 @@ public void onClose(Status status) {
}

@Override
public void onHandshakeComplete() {
public void onHandshake(InitResponse initResponse) {
handshakeSemaphore.release();
}

Expand Down Expand Up @@ -131,7 +139,7 @@ private void waitForWriteStreamOpen(
AsyncQueue testQueue, WriteStream writeStream, StreamStatusCallback callback) {
testQueue.enqueueAndForget(writeStream::start);
waitFor(callback.openSemaphore);
testQueue.enqueueAndForget(writeStream::writeHandshake);
testQueue.enqueueAndForget(() -> writeStream.sendHandshake(ByteString.EMPTY));
waitFor(callback.handshakeSemaphore);
}

Expand Down Expand Up @@ -180,9 +188,9 @@ public void testWriteStreamStopAfterHandshake() throws Exception {
StreamStatusCallback streamCallback =
new StreamStatusCallback() {
@Override
public void onHandshakeComplete() {
public void onHandshake(InitResponse initResponse) {
assertThat(writeStreamWrapper[0].getLastStreamToken()).isNotEmpty();
super.onHandshakeComplete();
super.onHandshake(initResponse);
}

@Override
Expand All @@ -202,7 +210,7 @@ public void onWriteResponse(
() -> assertThrows(Throwable.class, () -> writeStream.writeMutations(mutations)));

// Handshake should always be called
testQueue.enqueueAndForget(writeStream::writeHandshake);
testQueue.enqueueAndForget(() -> writeStream.sendHandshake(ByteString.EMPTY));
waitFor(streamCallback.handshakeSemaphore);

// Now writes should succeed
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,6 @@
import com.google.firebase.firestore.core.DatabaseInfo;
import com.google.firebase.firestore.model.DatabaseId;
import com.google.firebase.firestore.testutil.provider.FirestoreProvider;
import com.google.firebase.firestore.util.AsyncQueue;
import com.google.firebase.firestore.util.Listener;
import com.google.firebase.firestore.util.Logger;
import com.google.firebase.firestore.util.Logger.Level;
Expand Down Expand Up @@ -311,15 +310,13 @@ public static FirebaseFirestore testFirestore(

ensureStrictMode();

AsyncQueue asyncQueue = new AsyncQueue();

FirebaseFirestore firestore =
AccessHelper.newFirebaseFirestore(
context,
databaseId,
persistenceKey,
MockCredentialsProvider.instance(),
new EmptyAppCheckTokenProvider(),
MockCredentialsProvider::instance,
EmptyAppCheckTokenProvider::new,
ComponentProvider::defaultFactory,
/* firebaseApp= */ null,
/* instanceRegistry= */ (dbId) -> {});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,12 @@
import android.annotation.SuppressLint;
import android.app.Activity;
import android.content.Context;
import androidx.annotation.GuardedBy;
import androidx.annotation.Keep;
import androidx.annotation.NonNull;
import androidx.annotation.Nullable;
import androidx.annotation.VisibleForTesting;
import androidx.core.util.Supplier;
import com.google.android.gms.tasks.Task;
import com.google.android.gms.tasks.TaskCompletionSource;
import com.google.android.gms.tasks.Tasks;
Expand Down Expand Up @@ -56,6 +58,7 @@
import com.google.firebase.firestore.util.Logger.Level;
import com.google.firebase.firestore.util.Preconditions;
import com.google.firebase.inject.Deferred;
import com.google.protobuf.ByteString;
import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.nio.ByteBuffer;
Expand All @@ -76,6 +79,10 @@
public class FirebaseFirestore {

private final Function<FirebaseFirestoreSettings, ComponentProvider> componentProviderFactory;
private volatile ByteString sessionToken;

@GuardedBy("clientProvider")
private boolean networkEnabled;

/**
* Provides a registry management interface for {@code FirebaseFirestore} instances.
Expand All @@ -93,8 +100,8 @@ public interface InstanceRegistry {
// databaseId itself that needs locking; it just saves us creating a separate lock object.
private final DatabaseId databaseId;
private final String persistenceKey;
private final CredentialsProvider<User> authProvider;
private final CredentialsProvider<String> appCheckProvider;
private final Supplier<CredentialsProvider<User>> authProviderFactory;
private final Supplier<CredentialsProvider<String>> appCheckTokenProviderFactory;
private final FirebaseApp firebaseApp;
private final UserDataReader userDataReader;
// When user requests to terminate, use this to notify `FirestoreMultiDbComponent` to deregister
Expand All @@ -105,6 +112,8 @@ public interface InstanceRegistry {
final FirestoreClientProvider clientProvider;
private final GrpcMetadataProvider metadataProvider;

@VisibleForTesting Function<Executor, Task<Void>> clearPersistenceMethod;

@Nullable private PersistentCacheIndexManager persistentCacheIndexManager;

@NonNull
Expand Down Expand Up @@ -193,46 +202,44 @@ static FirebaseFirestore newInstance(
}
DatabaseId databaseId = DatabaseId.forDatabase(projectId, database);

CredentialsProvider<User> authProvider =
new FirebaseAuthCredentialsProvider(deferredAuthProvider);
CredentialsProvider<String> appCheckProvider =
new FirebaseAppCheckTokenProvider(deferredAppCheckTokenProvider);

// Firestore uses a different database for each app name. Note that we don't use
// app.getPersistenceKey() here because it includes the application ID which is related
// to the project ID. We already include the project ID when resolving the database,
// so there is no need to include it in the persistence key.
String persistenceKey = app.getName();

return new FirebaseFirestore(
context,
databaseId,
persistenceKey,
authProvider,
appCheckProvider,
ComponentProvider::defaultFactory,
app,
instanceRegistry,
metadataProvider);
FirebaseFirestore firestore =
new FirebaseFirestore(
context,
databaseId,
persistenceKey,
() -> new FirebaseAuthCredentialsProvider(deferredAuthProvider),
() -> new FirebaseAppCheckTokenProvider(deferredAppCheckTokenProvider),
ComponentProvider::defaultFactory,
app,
instanceRegistry,
metadataProvider);
return firestore;
}

@VisibleForTesting
FirebaseFirestore(
Context context,
DatabaseId databaseId,
String persistenceKey,
CredentialsProvider<User> authProvider,
CredentialsProvider<String> appCheckProvider,
@NonNull String persistenceKey,
@NonNull Supplier<CredentialsProvider<User>> authProviderFactory,
@NonNull Supplier<CredentialsProvider<String>> appCheckTokenProviderFactory,
@NonNull Function<FirebaseFirestoreSettings, ComponentProvider> componentProviderFactory,
@Nullable FirebaseApp firebaseApp,
InstanceRegistry instanceRegistry,
@Nullable GrpcMetadataProvider metadataProvider) {
this.networkEnabled = true;
this.context = checkNotNull(context);
this.databaseId = checkNotNull(checkNotNull(databaseId));
this.userDataReader = new UserDataReader(databaseId);
this.persistenceKey = checkNotNull(persistenceKey);
this.authProvider = checkNotNull(authProvider);
this.appCheckProvider = checkNotNull(appCheckProvider);
this.authProviderFactory = checkNotNull(authProviderFactory);
this.appCheckTokenProviderFactory = checkNotNull(appCheckTokenProviderFactory);
this.componentProviderFactory = checkNotNull(componentProviderFactory);
this.clientProvider = new FirestoreClientProvider(this::newClient);
// NOTE: We allow firebaseApp to be null in tests only.
Expand All @@ -241,6 +248,21 @@ static FirebaseFirestore newInstance(
this.metadataProvider = metadataProvider;

this.settings = new FirebaseFirestoreSettings.Builder().build();

this.clearPersistenceMethod =
executor -> {
final TaskCompletionSource<Void> source = new TaskCompletionSource<>();
executor.execute(
() -> {
try {
SQLitePersistence.clearPersistence(context, databaseId, persistenceKey);
source.setResult(null);
} catch (FirebaseFirestoreException e) {
source.setException(e);
}
});
return source.getTask();
};
}

/** Returns the settings used by this {@code FirebaseFirestore} object. */
Expand Down Expand Up @@ -296,14 +318,36 @@ private FirestoreClient newClient(AsyncQueue asyncQueue) {
DatabaseInfo databaseInfo =
new DatabaseInfo(databaseId, persistenceKey, settings.getHost(), settings.isSslEnabled());

return new FirestoreClient(
context,
databaseInfo,
authProvider,
appCheckProvider,
asyncQueue,
metadataProvider,
componentProviderFactory.apply(settings));
FirestoreClient client =
new FirestoreClient(
context,
databaseInfo,
authProviderFactory.get(),
appCheckTokenProviderFactory.get(),
asyncQueue,
metadataProvider,
componentProviderFactory.apply(settings));

client.setClearPersistenceCallback(
sessionToken -> {
synchronized (clientProvider) {
if (client.isTerminated()) return;
this.sessionToken = sessionToken;
clearPersistence();
}
});

// Session token must be set before we enable network, since it is part of stream handshake.
if (sessionToken != null) {
client.setSessionToken(sessionToken);
sessionToken = null;
}

if (networkEnabled) {
client.enableNetwork();
}

return client;
}
}

Expand Down Expand Up @@ -624,7 +668,14 @@ public Task<Void> waitForPendingWrites() {
*/
@NonNull
public Task<Void> enableNetwork() {
return clientProvider.call(FirestoreClient::enableNetwork);
synchronized (clientProvider) {
networkEnabled = true;
if (clientProvider.isConfigured()) {
return clientProvider.call(FirestoreClient::enableNetwork);
} else {
return Tasks.forResult(null);
}
}
}

/**
Expand All @@ -636,7 +687,14 @@ public Task<Void> enableNetwork() {
*/
@NonNull
public Task<Void> disableNetwork() {
return clientProvider.call(FirestoreClient::disableNetwork);
synchronized (clientProvider) {
networkEnabled = false;
if (clientProvider.isConfigured()) {
return clientProvider.call(FirestoreClient::disableNetwork);
} else {
return Tasks.forResult(null);
}
}
}

/** Globally enables / disables Cloud Firestore logging for the SDK. */
Expand Down Expand Up @@ -669,30 +727,14 @@ public static void setLoggingEnabled(boolean loggingEnabled) {
@NonNull
public Task<Void> clearPersistence() {
return clientProvider.executeIfShutdown(
this::clearPersistence,
clearPersistenceMethod,
executor ->
Tasks.forException(
new FirebaseFirestoreException(
"Persistence cannot be cleared while the firestore instance is running.",
FirebaseFirestoreException.Code.FAILED_PRECONDITION)));
}

@NonNull
private Task<Void> clearPersistence(Executor executor) {
final TaskCompletionSource<Void> source = new TaskCompletionSource<>();
executor.execute(
() -> {
try {
SQLitePersistence.clearPersistence(context, databaseId, persistenceKey);
source.setResult(null);
} catch (FirebaseFirestoreException e) {
source.setException(e);
}
});
return source.getTask();
}
;

/**
* Attaches a listener for a snapshots-in-sync event. The snapshots-in-sync event indicates that
* all listeners affected by a given change have fired, even if a single server-generated change
Expand Down
Loading
Loading