Skip to content

Commit a623485

Browse files
damccormstankiewicz
authored andcommitted
Remove aws v1 from infrastructure (apache#33455)
* Remove aws v1 from infrastructure * Remove unused code
1 parent 5058e1a commit a623485

File tree

7 files changed

+39
-328
lines changed

7 files changed

+39
-328
lines changed

.github/workflows/README.md

-2
Original file line numberDiff line numberDiff line change
@@ -228,7 +228,6 @@ PreCommit Jobs run in a schedule and also get triggered in a PR if relevant sour
228228
| [ PreCommit Go ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Go.yml) | N/A |`Run Go PreCommit`| [![.github/workflows/beam_PreCommit_Go.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Go.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Go.yml?query=event%3Aschedule) |
229229
| [ PreCommit GoPortable ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_GoPortable.yml) | N/A |`Run GoPortable PreCommit`| [![.github/workflows/beam_PreCommit_GoPortable.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_GoPortable.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_GoPortable.yml?query=event%3Aschedule) |
230230
| [ PreCommit Java ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java.yml) | N/A |`Run Java PreCommit`| [![.github/workflows/beam_PreCommit_Java.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java.yml?query=event%3Aschedule) |
231-
| [ PreCommit Java Amazon Web Services IO Direct ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Amazon-Web-Services_IO_Direct.yml) | N/A |`Run Java_Amazon-Web-Services_IO_Direct PreCommit`| [![.github/workflows/beam_PreCommit_Java_Amazon-Web-Services_IO_Direct.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Amazon-Web-Services_IO_Direct.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Amazon-Web-Services_IO_Direct.yml?query=event%3Aschedule) |
232231
| [ PreCommit Java Amazon Web Services2 IO Direct ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Amazon-Web-Services2_IO_Direct.yml) | N/A |`Run Java_Amazon-Web-Services2_IO_Direct PreCommit`| [![.github/workflows/beam_PreCommit_Java_Amazon-Web-Services2_IO_Direct.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Amazon-Web-Services2_IO_Direct.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Amazon-Web-Services2_IO_Direct.yml?query=event%3Aschedule) |
233232
| [ PreCommit Java Amqp IO Direct ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Amqp_IO_Direct.yml) | N/A |`Run Java_Amqp_IO_Direct PreCommit`| [![.github/workflows/beam_PreCommit_Java_Amqp_IO_Direct.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Amqp_IO_Direct.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Amqp_IO_Direct.yml?query=event%3Aschedule) |
234233
| [ PreCommit Java Azure IO Direct ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Azure_IO_Direct.yml) | N/A |`Run Java_Azure_IO_Direct PreCommit`| [![.github/workflows/beam_PreCommit_Java_Azure_IO_Direct.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Azure_IO_Direct.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Azure_IO_Direct.yml?query=event%3Aschedule) |
@@ -252,7 +251,6 @@ PreCommit Jobs run in a schedule and also get triggered in a PR if relevant sour
252251
| [ PreCommit Java IOs Direct ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_IOs_Direct.yml) | N/A |`Run Java_IOs_Direct PreCommit`| N/A |
253252
| [ PreCommit Java JDBC IO Direct ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_JDBC_IO_Direct.yml) | N/A |`Run Java_JDBC_IO_Direct PreCommit`| [![.github/workflows/beam_PreCommit_Java_JDBC_IO_Direct.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_JDBC_IO_Direct.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_JDBC_IO_Direct.yml?query=event%3Aschedule) |
254253
| [ PreCommit Java Jms IO Direct ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Jms_IO_Direct.yml) | N/A |`Run Java_Jms_IO_Direct PreCommit`| [![.github/workflows/beam_PreCommit_Java_Jms_IO_Direct.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Jms_IO_Direct.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Jms_IO_Direct.yml?query=event%3Aschedule) |
255-
| [ PreCommit Java Kinesis IO Direct ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Kinesis_IO_Direct.yml) | N/A |`Run Java_Kinesis_IO_Direct PreCommit`| [![.github/workflows/beam_PreCommit_Java_Kinesis_IO_Direct.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Kinesis_IO_Direct.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Kinesis_IO_Direct.yml?query=event%3Aschedule) |
256254
| [ PreCommit Java Kudu IO Direct ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Kudu_IO_Direct.yml) | N/A |`Run Java_Kudu_IO_Direct PreCommit`| [![.github/workflows/beam_PreCommit_Java_Kudu_IO_Direct.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Kudu_IO_Direct.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Kudu_IO_Direct.yml?query=event%3Aschedule) |
257255
| [ PreCommit Java MongoDb IO Direct ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_MongoDb_IO_Direct.yml) | N/A |`Run Java_MongoDb_IO_Direct PreCommit`| [![.github/workflows/beam_PreCommit_Java_MongoDb_IO_Direct.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_MongoDb_IO_Direct.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_MongoDb_IO_Direct.yml?query=event%3Aschedule) |
258256
| [ PreCommit Java Mqtt IO Direct ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Mqtt_IO_Direct.yml) | N/A |`Run Java_Mqtt_IO_Direct PreCommit`| [![.github/workflows/beam_PreCommit_Java_Mqtt_IO_Direct.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Mqtt_IO_Direct.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Mqtt_IO_Direct.yml?query=event%3Aschedule) |

.github/workflows/beam_PreCommit_Java_Amazon-Web-Services_IO_Direct.yml

-145
This file was deleted.

.github/workflows/beam_PreCommit_Java_Kinesis_IO_Direct.yml

-152
This file was deleted.

sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java

+17-18
Original file line numberDiff line numberDiff line change
@@ -373,24 +373,23 @@ public WriteResult expandUntriggered(
373373
}
374374

375375
PCollectionTuple writeRecordsResult =
376-
successfulConvertedRows
377-
.apply(
378-
"StorageApiWriteUnsharded",
379-
new StorageApiWriteUnshardedRecords<>(
380-
dynamicDestinations,
381-
bqServices,
382-
failedRowsTag,
383-
successfulWrittenRowsTag,
384-
successfulRowsPredicate,
385-
BigQueryStorageApiInsertErrorCoder.of(),
386-
TableRowJsonCoder.of(),
387-
autoUpdateSchema,
388-
ignoreUnknownValues,
389-
createDisposition,
390-
kmsKey,
391-
usesCdc,
392-
defaultMissingValueInterpretation,
393-
bigLakeConfiguration));
376+
successfulConvertedRows.apply(
377+
"StorageApiWriteUnsharded",
378+
new StorageApiWriteUnshardedRecords<>(
379+
dynamicDestinations,
380+
bqServices,
381+
failedRowsTag,
382+
successfulWrittenRowsTag,
383+
successfulRowsPredicate,
384+
BigQueryStorageApiInsertErrorCoder.of(),
385+
TableRowJsonCoder.of(),
386+
autoUpdateSchema,
387+
ignoreUnknownValues,
388+
createDisposition,
389+
kmsKey,
390+
usesCdc,
391+
defaultMissingValueInterpretation,
392+
bigLakeConfiguration));
394393

395394
PCollection<BigQueryStorageApiInsertError> insertErrors =
396395
PCollectionList.of(convertMessagesResult.get(failedRowsTag))

sdks/java/testing/load-tests/build.gradle

+7-2
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,10 @@ configurations {
6464
gradleRun
6565
}
6666

67+
def excludeNetty = {
68+
exclude group: "io.netty", module: "*" // exclude more recent Netty version
69+
}
70+
6771
dependencies {
6872
implementation enforcedPlatform(library.java.google_cloud_platform_libraries_bom)
6973

@@ -73,8 +77,9 @@ dependencies {
7377
implementation project(":sdks:java:testing:test-utils")
7478
implementation project(":sdks:java:io:google-cloud-platform")
7579
implementation project(":sdks:java:io:kafka")
76-
implementation project(":sdks:java:io:kinesis")
77-
implementation library.java.aws_java_sdk_core
80+
implementation project(":sdks:java:io:amazon-web-services2")
81+
implementation library.java.aws_java_sdk2_auth, excludeNetty
82+
implementation library.java.aws_java_sdk2_regions, excludeNetty
7883
implementation library.java.google_cloud_core
7984
implementation library.java.joda_time
8085
implementation library.java.vendored_guava_32_1_2_jre

0 commit comments

Comments
 (0)