-
Notifications
You must be signed in to change notification settings - Fork 347
Description
Summary
The Spring Cloud GCP GcpDatastoreAutoConfiguration should support configurable timeout and retry settings through Spring Boot properties, similar to how other Spring Cloud GCP autoconfiguration classes (like Pub/Sub) expose configuration options.
Current Limitation
Currently, GcpDatastoreAutoConfiguration creates Datastore clients with default timeout and retry settings that cannot be customized without completely replacing the autoconfiguration. This forces developers to:
- Create custom
DatastoreProviderorDatastorebeans - Manually replicate the autoconfiguration logic
- Potentially break compatibility with future Spring Cloud GCP updates
Current Code (Spring Cloud GCP 5.x)
// From com.google.cloud.spring.autoconfigure.datastore.GcpDatastoreAutoConfiguration
private Datastore getDatastore(String namespace) {
DatastoreOptions.Builder builder = DatastoreOptions.newBuilder()
.setProjectId(this.projectId)
.setHeaderProvider(new UserAgentHeaderProvider(this.getClass()))
.setCredentials(this.credentials);
if (namespace != null) {
builder.setNamespace(namespace);
}
if (databaseId != null) {
builder.setDatabaseId(databaseId);
}
if (this.host != null) {
builder.setHost(this.host);
}
// ❌ NO WAY TO SET CUSTOM TIMEOUTS OR RETRY SETTINGS!
return builder.build().getService();
}What's Missing
The DatastoreOptions.Builder supports these configuration methods that are not exposed:
.setTransportOptions(HttpTransportOptions)- for connect and read timeouts.setRetrySettings(RetrySettings)- for retry behavior (max attempts, delays, multipliers)
Proposed Solution
Add properties to GcpDatastoreProperties similar to how GcpPubSubProperties handles configuration:
Proposed Property Structure
spring:
cloud:
gcp:
datastore:
# Existing properties
project-id: my-project
namespace: my-namespace
database-id: my-database
host: localhost:8081
# NEW: Timeout properties
timeout:
connect-timeout-ms: 1000 # HTTP connect timeout
read-timeout-ms: 2000 # HTTP read timeout
# NEW: Retry properties
retry:
total-timeout-ms: 3000 # Total time including retries
initial-rpc-timeout-ms: 1000 # First attempt timeout
max-rpc-timeout-ms: 2000 # Max timeout for any attempt
rpc-timeout-multiplier: 1.5 # Timeout increase per retry
max-attempts: 3 # Maximum retry attempts
initial-retry-delay-ms: 100 # Initial delay between retries
max-retry-delay-ms: 5000 # Max delay between retries
retry-delay-multiplier: 2.0 # Delay increase per retryProposed Code Changes
1. Update GcpDatastoreProperties
@ConfigurationProperties("spring.cloud.gcp.datastore")
public class GcpDatastoreProperties implements CredentialsSupplier {
// ...existing properties...
/** Timeout configuration for Datastore operations. */
private TimeoutProperties timeout = new TimeoutProperties();
/** Retry configuration for Datastore operations. */
private RetryProperties retry = new RetryProperties();
public TimeoutProperties getTimeout() {
return timeout;
}
public void setTimeout(TimeoutProperties timeout) {
this.timeout = timeout;
}
public RetryProperties getRetry() {
return retry;
}
public void setRetry(RetryProperties retry) {
this.retry = retry;
}
/** Timeout configuration properties. */
public static class TimeoutProperties {
/** HTTP connection timeout in milliseconds. Default: 20000ms (20s) */
private int connectTimeoutMs = 20000;
/** HTTP read timeout in milliseconds. Default: 20000ms (20s) */
private int readTimeoutMs = 20000;
// getters and setters...
}
/** Retry configuration properties. */
public static class RetryProperties {
/** Total timeout including retries in milliseconds. Default: 60000ms (60s) */
private long totalTimeoutMs = 60000;
/** Initial RPC timeout in milliseconds. Default: 5000ms (5s) */
private long initialRpcTimeoutMs = 5000;
/** Maximum RPC timeout in milliseconds. Default: 30000ms (30s) */
private long maxRpcTimeoutMs = 30000;
/** RPC timeout multiplier. Default: 1.3 */
private double rpcTimeoutMultiplier = 1.3;
/** Maximum number of attempts. Default: 6 */
private int maxAttempts = 6;
/** Initial retry delay in milliseconds. Default: 100ms */
private long initialRetryDelayMs = 100;
/** Maximum retry delay in milliseconds. Default: 5000ms (5s) */
private long maxRetryDelayMs = 5000;
/** Retry delay multiplier. Default: 1.3 */
private double retryDelayMultiplier = 1.3;
// getters and setters...
}
}2. Update GcpDatastoreAutoConfiguration
private Datastore getDatastore(String namespace) {
DatastoreOptions.Builder builder = DatastoreOptions.newBuilder()
.setProjectId(this.projectId)
.setHeaderProvider(new UserAgentHeaderProvider(this.getClass()))
.setCredentials(this.credentials);
// Apply timeout configuration
TimeoutProperties timeoutProps = gcpDatastoreProperties.getTimeout();
if (timeoutProps != null) {
HttpTransportOptions transportOptions = HttpTransportOptions.newBuilder()
.setConnectTimeout(timeoutProps.getConnectTimeoutMs())
.setReadTimeout(timeoutProps.getReadTimeoutMs())
.build();
builder.setTransportOptions(transportOptions);
}
// Apply retry configuration
RetryProperties retryProps = gcpDatastoreProperties.getRetry();
if (retryProps != null) {
RetrySettings retrySettings = RetrySettings.newBuilder()
.setTotalTimeout(Duration.ofMillis(retryProps.getTotalTimeoutMs()))
.setInitialRpcTimeout(Duration.ofMillis(retryProps.getInitialRpcTimeoutMs()))
.setMaxRpcTimeout(Duration.ofMillis(retryProps.getMaxRpcTimeoutMs()))
.setRpcTimeoutMultiplier(retryProps.getRpcTimeoutMultiplier())
.setMaxAttempts(retryProps.getMaxAttempts())
.setInitialRetryDelay(Duration.ofMillis(retryProps.getInitialRetryDelayMs()))
.setMaxRetryDelay(Duration.ofMillis(retryProps.getMaxRetryDelayMs()))
.setRetryDelayMultiplier(retryProps.getRetryDelayMultiplier())
.build();
builder.setRetrySettings(retrySettings);
}
if (namespace != null) {
builder.setNamespace(namespace);
}
if (databaseId != null) {
builder.setDatabaseId(databaseId);
}
if (this.host != null) {
builder.setHost(this.host);
}
return builder.build().getService();
}Use Cases
Use Case 1: Low-Latency Requirements
Applications with strict latency requirements need to fail fast rather than wait for default timeouts:
spring:
cloud:
gcp:
datastore:
timeout:
connect-timeout-ms: 500
read-timeout-ms: 1000
retry:
max-attempts: 1 # No retries - fail fastUse Case 2: High-Reliability Requirements
Applications that prioritize reliability over latency need more aggressive retry strategies:
spring:
cloud:
gcp:
datastore:
timeout:
connect-timeout-ms: 5000
read-timeout-ms: 10000
retry:
max-attempts: 5
total-timeout-ms: 30000
initial-retry-delay-ms: 500
retry-delay-multiplier: 2.0Use Case 3: Environment-Specific Configuration
Different environments (dev, staging, production) require different timeout profiles:
# application-prod.yaml
spring:
cloud:
gcp:
datastore:
timeout:
connect-timeout-ms: 2000
read-timeout-ms: 5000
retry:
max-attempts: 3
# application-dev.yaml
spring:
cloud:
gcp:
datastore:
timeout:
connect-timeout-ms: 10000
read-timeout-ms: 30000
retry:
max-attempts: 1 # Fail fast in dev for quicker feedbackCurrent Workaround
Developers currently must create custom configuration classes that replicate Spring Cloud GCP's logic:
@Configuration
@AutoConfigureBefore(GcpDatastoreAutoConfiguration.class)
public class CustomDatastoreTimeoutConfig {
@Bean
@ConditionalOnMissingBean
public DatastoreProvider datastoreProvider(
GcpProjectIdProvider projectIdProvider,
ObjectProvider<DatastoreNamespaceProvider> namespaceProvider,
CustomTimeoutProperties timeoutProperties) {
// Must manually replicate Spring Cloud GCP's namespace handling
// Must manually replicate credential handling
// Must manually replicate project ID resolution
// Brittle - breaks if Spring Cloud GCP changes its implementation
// ... custom implementation ...
}
}Problems with this approach:
- ❌ Code duplication
- ❌ Maintenance burden
- ❌ Risk of incompatibility with Spring Cloud GCP updates
- ❌ Loss of automatic features (namespace providers, credential providers)
- ❌ Requires deep understanding of Spring Cloud GCP internals
Benefits of This Feature
- Consistency: Aligns with how other Spring Cloud GCP components (Pub/Sub, Storage) expose configuration
- Simplicity: Developers can configure timeouts through standard Spring Boot properties
- Flexibility: Different environments can have different timeout profiles
- Maintainability: No need for custom autoconfiguration classes
- Best Practices: Encourages proper timeout configuration rather than using defaults
- Production Ready: Essential for production deployments with specific SLA requirements
Comparison with Other Spring Cloud GCP Components
Spring Cloud GCP Pub/Sub (Already Supports Configuration)
spring:
cloud:
gcp:
pubsub:
subscriber:
max-ack-extension-period: 0
parallel-pull-count: 1
pull-endpoint: localhost:8085
publisher:
batching:
element-count-threshold: 1
request-byte-threshold: 1000Spring Cloud GCP Storage (Already Supports Configuration)
spring:
cloud:
gcp:
storage:
project-id: my-project
credentials:
location: classpath:credentials.jsonSpring Cloud GCP Datastore (Currently Missing Configuration)
spring:
cloud:
gcp:
datastore:
project-id: my-project
namespace: my-namespace
# ❌ NO timeout configuration
# ❌ NO retry configurationImplementation Considerations
Backward Compatibility
- All new properties should have sensible defaults matching current behavior
- Existing applications without these properties should work unchanged
- Properties should be optional
Documentation
- Add to Spring Cloud GCP reference documentation
- Provide examples for common use cases
- Include migration guide for developers using custom workarounds
Testing
- Add integration tests with custom timeout configurations
- Add tests verifying backward compatibility
- Add tests for environment-specific configuration
Example Real-World Impact
Our production application requires:
- Connect timeout: 1000ms (must connect quickly or fail)
- Read timeout: 2000ms (must read quickly or fail)
- Max attempts: 1 (fail fast for load balancer to retry on different pod)
Without this feature, we had to:
- Create a custom
DatastoreProviderbean - Manually replicate namespace resolution logic
- Cache Datastore instances per namespace
- Maintain 150+ lines of configuration code
- Risk breakage with Spring Cloud GCP updates
With this feature, we could simply configure:
spring:
cloud:
gcp:
datastore:
timeout:
connect-timeout-ms: 1000
read-timeout-ms: 2000
retry:
max-attempts: 1References
- Google Cloud Datastore Client Library: https://cloud.google.com/java/docs/reference/google-cloud-datastore/latest
DatastoreOptions.BuilderAPI: https://cloud.google.com/java/docs/reference/google-cloud-datastore/latest/com.google.cloud.datastore.DatastoreOptions.BuilderRetrySettingsAPI: https://googleapis.dev/java/gax/latest/com/google/api/gax/retrying/RetrySettings.htmlHttpTransportOptionsAPI: https://googleapis.dev/java/google-http-client/latest/com/google/api/client/http/HttpTransport.html
Proposed Timeline
- Phase 1: Add timeout configuration properties
- Phase 2: Add retry configuration properties
- Phase 3: Add documentation and examples
- Phase 4: Add integration tests