Skip to content

Commit d318aad

Browse files
lordgamezlordgamez
committed
Fix after rebase
1 parent 74896e8 commit d318aad

File tree

3 files changed

+52
-35
lines changed

3 files changed

+52
-35
lines changed

‎behave_framework/src/minifi_test_framework/steps/flow_building_steps.py‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -192,10 +192,25 @@ def step_impl(context: MinifiTestContext):
192192

193193

194194
@step("the http proxy server is set up")
195-
def step_impl(context):
195+
def step_impl(context: MinifiTestContext):
196196
context.containers["http-proxy"] = HttpProxy(context)
197197

198198

199+
@given("a ProxyConfigurationService controller service is set up with HTTP proxy configuration in the \"{container_name}\" flow")
200+
def step_impl(context: MinifiTestContext, container_name: str):
201+
controller_service = ControllerService(class_name="ProxyConfigurationService", service_name="ProxyConfigurationService")
202+
controller_service.add_property("Proxy Server Host", f"http-proxy-{context.scenario_id}")
203+
controller_service.add_property("Proxy Server Port", "3128")
204+
controller_service.add_property("Proxy User Name", "admin")
205+
controller_service.add_property("Proxy User Password", "test101")
206+
context.get_or_create_minifi_container(container_name).flow_definition.controller_services.append(controller_service)
207+
208+
209+
@given("a ProxyConfigurationService controller service is set up with HTTP proxy configuration")
210+
def step_impl(context: MinifiTestContext):
211+
context.execute_steps(f"given a ProxyConfigurationService controller service is set up with HTTP proxy configuration in the \"{DEFAULT_MINIFI_CONTAINER_NAME}\" flow")
212+
213+
199214
@step("the processors are connected up as described here")
200215
def step_impl(context: MinifiTestContext):
201216
for row in context.table:

‎extensions/aws/tests/features/s3.feature‎

Lines changed: 30 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ Feature: Sending data from MiNiFi-C++ to an AWS server
4444
And a PutS3Object processor set up to communicate with an s3 server
4545
And these processor properties are set
4646
| processor name | property name | property value |
47-
| PutS3Object | Proxy Host | http-proxy-${scenario_id} |
47+
| PutS3Object | Proxy Host | http-proxy-${scenario_id} |
4848
| PutS3Object | Proxy Port | 3128 |
4949
| PutS3Object | Proxy Username | admin |
5050
| PutS3Object | Proxy Password | test101 |
@@ -61,7 +61,7 @@ Feature: Sending data from MiNiFi-C++ to an AWS server
6161
Then a single file with the content "LH_O#L|FD<FASD{FO#@$#$%^ \"#\"$L%:\"@#$L\":test_data#$#%#$%?{\"F{" is placed in the "/tmp/output" directory in less than 20 seconds
6262
And the object on the s3 server is "LH_O#L|FD<FASD{FO#@$#$%^ \"#\"$L%:\"@#$L\":test_data#$#%#$%?{\"F{"
6363
And the object content type on the s3 server is "application/octet-stream" and the object metadata matches use metadata
64-
And no errors were generated on the http-proxy regarding "http://s3-server-s3-1:9090/test_bucket/test_object_key"
64+
And no errors were generated on the http-proxy regarding "http://s3-server-${scenario_id}:9090/test_bucket/test_object_key"
6565

6666
Scenario: A MiNiFi instance transfers encoded data through a http proxy to s3 using proxy configuration service
6767
Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
@@ -77,10 +77,10 @@ Feature: Sending data from MiNiFi-C++ to an AWS server
7777
And the http proxy server is set up
7878
When all instances start up
7979

80-
Then a flowfile with the content "test" is placed in the monitored directory in less than 60 seconds
80+
Then a single file with the content "LH_O#L|FD<FASD{FO#@$#$%^ \"#\"$L%:\"@#$L\":test_data#$#%#$%?{\"F{" is placed in the "/tmp/output" directory in less than 60 seconds
8181
And the object on the s3 server is "LH_O#L|FD<FASD{FO#@$#$%^ \"#\"$L%:\"@#$L\":test_data#$#%#$%?{\"F{"
8282
And the object content type on the s3 server is "application/octet-stream" and the object metadata matches use metadata
83-
And no errors were generated on the http-proxy regarding "http://s3-server-${feature_id}:9090/test_bucket/test_object_key"
83+
And no errors were generated on the http-proxy regarding "http://s3-server-${scenario_id}:9090/test_bucket/test_object_key"
8484

8585
Scenario: A MiNiFi instance can remove s3 bucket objects
8686
Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
@@ -120,13 +120,14 @@ Feature: Sending data from MiNiFi-C++ to an AWS server
120120

121121
Scenario: Deletion of a s3 object through a proxy-server succeeds
122122
Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
123+
And the scheduling period of the GetFile processor is set to "60 sec"
123124
And a directory at "/tmp/input" has a file with the content "LH_O#L|FD<FASD{FO#@$#$%^ \"#\"$L%:\"@#$L\":test_data#$#%#$%?{\"F{"
124125
And a PutS3Object processor set up to communicate with an s3 server
125126
And a DeleteS3Object processor set up to communicate with the same s3 server
126127
And a PutFile processor with the "Directory" property set to "/tmp/output"
127128
And these processor properties are set
128129
| processor name | property name | property value |
129-
| DeleteS3Object | Proxy Host | http-proxy-${scenario_id} |
130+
| DeleteS3Object | Proxy Host | http-proxy-${scenario_id} |
130131
| DeleteS3Object | Proxy Port | 3128 |
131132
| DeleteS3Object | Proxy Username | admin |
132133
| DeleteS3Object | Proxy Password | test101 |
@@ -145,16 +146,17 @@ Feature: Sending data from MiNiFi-C++ to an AWS server
145146

146147
Then a single file with the content "LH_O#L|FD<FASD{FO#@$#$%^ \"#\"$L%:\"@#$L\":test_data#$#%#$%?{\"F{" is placed in the "/tmp/output" directory in less than 20 seconds
147148
And the object bucket on the s3 server is empty in less than 10 seconds
148-
And no errors were generated on the http-proxy regarding "http://s3-server-s3-4:9090/test_bucket/test_object_key"
149+
And no errors were generated on the http-proxy regarding "http://s3-server-${scenario_id}:9090/test_bucket/test_object_key"
149150

150151
Scenario: Deletion of a s3 object through a proxy-server succeeds using proxy configuration service
151152
Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
153+
And the scheduling period of the GetFile processor is set to "60 sec"
152154
And a file with the content "LH_O#L|FD<FASD{FO#@$#$%^ \"#\"$L%:\"@#$L\":test_data#$#%#$%?{\"F{" is present in "/tmp/input"
153155
And a PutS3Object processor set up to communicate with an s3 server
154156
And a DeleteS3Object processor set up to communicate with the same s3 server
155157
And the "Proxy Configuration Service" property of the DeleteS3Object processor is set to "ProxyConfigurationService"
156158
And a PutFile processor with the "Directory" property set to "/tmp/output"
157-
And the processors are connected up as described here:
159+
And the processors are connected up as described here
158160
| source name | relationship name | destination name |
159161
| GetFile | success | PutS3Object |
160162
| PutS3Object | success | DeleteS3Object |
@@ -166,9 +168,9 @@ Feature: Sending data from MiNiFi-C++ to an AWS server
166168

167169
When all instances start up
168170

169-
Then a flowfile with the content "test" is placed in the monitored directory in less than 60 seconds
170-
And the object bucket on the s3 server is empty
171-
And no errors were generated on the http-proxy regarding "http://s3-server-${feature_id}:9090/test_bucket/test_object_key"
171+
Then a single file with the content "LH_O#L|FD<FASD{FO#@$#$%^ \"#\"$L%:\"@#$L\":test_data#$#%#$%?{\"F{" is placed in the "/tmp/output" directory in less than 60 seconds
172+
And the object bucket on the s3 server is empty in less than 10 seconds
173+
And no errors were generated on the http-proxy regarding "http://s3-server-${scenario_id}:9090/test_bucket/test_object_key"
172174

173175
Scenario: A MiNiFi instance can download s3 bucket objects directly
174176
Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
@@ -219,31 +221,31 @@ Feature: Sending data from MiNiFi-C++ to an AWS server
219221
When all instances start up
220222

221223
Then a single file with the content "test" is placed in the "/tmp/output" directory in less than 20 seconds
222-
And no errors were generated on the http-proxy regarding "http://s3-server-s3-6:9090/test_bucket/test_object_key"
224+
And no errors were generated on the http-proxy regarding "http://s3-server-${scenario_id}:9090/test_bucket/test_object_key"
223225

224226
Scenario: A MiNiFi instance can download s3 bucket objects via a http-proxy using proxy configuration service
225227
Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
226228
And a file with the content "test" is present in "/tmp/input"
227229
And a PutS3Object processor set up to communicate with an s3 server
228230
And the "success" relationship of the GetFile processor is connected to the PutS3Object
229231

230-
Given a GenerateFlowFile processor with the "File Size" property set to "1 kB" in a "secondary" flow
232+
Given a GenerateFlowFile processor with the "File Size" property set to "1 kB"
231233
And a FetchS3Object processor set up to communicate with the same s3 server
232234
And the "Proxy Configuration Service" property of the FetchS3Object processor is set to "ProxyConfigurationService"
233235
And a PutFile processor with the "Directory" property set to "/tmp/output"
234-
And the processors are connected up as described here:
236+
And the processors are connected up as described here
235237
| source name | relationship name | destination name |
236238
| GenerateFlowFile | success | FetchS3Object |
237239
| FetchS3Object | success | PutFile |
238-
And a ProxyConfigurationService controller service is set up with HTTP proxy configuration in the "secondary" flow
240+
And a ProxyConfigurationService controller service is set up with HTTP proxy configuration
239241

240242
And a s3 server is set up in correspondence with the PutS3Object
241-
And a http proxy server is set up accordingly
243+
And the http proxy server is set up
242244

243245
When all instances start up
244246

245-
Then a flowfile with the content "test" is placed in the monitored directory in less than 60 seconds
246-
And no errors were generated on the http-proxy regarding "http://s3-server-${feature_id}:9090/test_bucket/test_object_key"
247+
Then a single file with the content "test" is placed in the "/tmp/output" directory in less than 60 seconds
248+
And no errors were generated on the http-proxy regarding "http://s3-server-${scenario_id}:9090/test_bucket/test_object_key"
247249

248250
Scenario: A MiNiFi instance can list an S3 bucket directly
249251
Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
@@ -287,27 +289,27 @@ Feature: Sending data from MiNiFi-C++ to an AWS server
287289
When all instances start up
288290

289291
Then 1 file is placed in the "/tmp/output" directory in less than 20 seconds
290-
And no errors were generated on the http-proxy regarding "http://s3-server-s3-8:9090/test_bucket"
292+
And no errors were generated on the http-proxy regarding "http://s3-server-${scenario_id}:9090/test_bucket"
291293

292294
Scenario: A MiNiFi instance can list an S3 bucket objects via a http-proxy using proxy configuration service
293295
Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
294296
And a file with the content "test" is present in "/tmp/input"
295297
And a PutS3Object processor set up to communicate with an s3 server
296298
And the "success" relationship of the GetFile processor is connected to the PutS3Object
297299

298-
Given a ListS3 processor in the "secondary" flow
300+
Given a ListS3 processor set up to communicate with the same s3 server
299301
And the "Proxy Configuration Service" property of the ListS3 processor is set to "ProxyConfigurationService"
300302
And a PutFile processor with the "Directory" property set to "/tmp/output"
301303
And the "success" relationship of the ListS3 processor is connected to the PutFile
302-
And a ProxyConfigurationService controller service is set up with HTTP proxy configuration in the "secondary" flow
304+
And a ProxyConfigurationService controller service is set up with HTTP proxy configuration
303305

304306
And a s3 server is set up in correspondence with the PutS3Object
305-
And a http proxy server is set up accordingly
307+
And the http proxy server is set up
306308

307309
When all instances start up
308310

309-
Then 1 flowfile is placed in the monitored directory in 120 seconds
310-
And no errors were generated on the http-proxy regarding "http://s3-server-${feature_id}:9090/test_bucket"
311+
Then 1 file is placed in the "/tmp/output" directory in less than 120 seconds
312+
And no errors were generated on the http-proxy regarding "http://s3-server-${scenario_id}:9090/test_bucket"
311313

312314
Scenario: A MiNiFi instance transfers data in multiple parts to s3
313315
Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
@@ -333,11 +335,11 @@ Feature: Sending data from MiNiFi-C++ to an AWS server
333335
And the "Multipart Threshold" property of the PutS3Object processor is set to "5 MB"
334336
And the "Multipart Part Size" property of the PutS3Object processor is set to "5 MB"
335337
And these processor properties are set
336-
| processor name | property name | property value |
338+
| processor name | property name | property value |
337339
| PutS3Object | Proxy Host | http-proxy-${scenario_id} |
338-
| PutS3Object | Proxy Port | 3128 |
339-
| PutS3Object | Proxy Username | admin |
340-
| PutS3Object | Proxy Password | test101 |
340+
| PutS3Object | Proxy Port | 3128 |
341+
| PutS3Object | Proxy Username | admin |
342+
| PutS3Object | Proxy Password | test101 |
341343
And the "success" relationship of the GetFile processor is connected to the PutS3Object
342344
And a PutFile processor with the "Directory" property set to "/tmp/output"
343345
And the "success" relationship of the PutS3Object processor is connected to the PutFile
@@ -349,4 +351,4 @@ Feature: Sending data from MiNiFi-C++ to an AWS server
349351
Then 1 file is placed in the "/tmp/output" directory in less than 20 seconds
350352
And the object on the s3 server is present and matches the original hash
351353
And the Minifi logs contain the following message: "passes the multipart threshold, uploading it in multiple parts" in less than 10 seconds
352-
And no errors were generated on the http-proxy regarding "http://s3-server-s3-10:9090/test_bucket/test_object_key"
354+
And no errors were generated on the http-proxy regarding "http://s3-server-${scenario_id}:9090/test_bucket/test_object_key"

‎extensions/azure/tests/features/azure_storage.feature‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -125,9 +125,9 @@ Feature: Sending data from MiNiFi-C++ to an Azure storage server
125125

126126
When all instances start up
127127

128-
Then a flowfile with the content "test" is placed in the monitored directory in less than 60 seconds
128+
Then a single file with the content "#test_data$123$#" is placed in the "/tmp/output" directory in less than 60 seconds
129129
And the object on the Azure storage server is "#test_data$123$#"
130-
And no errors were generated on the http-proxy regarding "http://azure-storage-server-${feature_id}:10000/devstoreaccount1/test-container/test-blob"
130+
And no errors were generated on the http-proxy regarding "http://azure-storage-server-${scenario_id}:10000/devstoreaccount1/test-container/test-blob"
131131

132132
Scenario: A MiNiFi instance can delete blob from Azure blob storage through a proxy
133133
Given a GenerateFlowFile processor with the "File Size" property set to "0B"
@@ -144,7 +144,7 @@ Feature: Sending data from MiNiFi-C++ to an Azure storage server
144144
And test blob "test" is created on Azure blob storage
145145

146146
Then the Azure blob storage becomes empty in 30 seconds
147-
And no errors were generated on the http-proxy regarding "http://azure-storage-server-${feature_id}:10000/devstoreaccount1/test-container/test"
147+
And no errors were generated on the http-proxy regarding "http://azure-storage-server-${scenario_id}:10000/devstoreaccount1/test-container/test"
148148

149149
Scenario: A MiNiFi instance can fetch a blob from Azure blob storage through a proxy
150150
Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
@@ -166,8 +166,8 @@ Feature: Sending data from MiNiFi-C++ to an Azure storage server
166166
When all instances start up
167167
And test blob "test" with the content "#test_data$123$#" is created on Azure blob storage
168168

169-
Then a flowfile with the content "data$" is placed in the monitored directory in less than 60 seconds
170-
And no errors were generated on the http-proxy regarding "http://azure-storage-server-${feature_id}:10000/devstoreaccount1/test-container/test"
169+
Then a single file with the content "data$" is placed in the "/tmp/output" directory in less than 60 seconds
170+
And no errors were generated on the http-proxy regarding "http://azure-storage-server-${scenario_id}:10000/devstoreaccount1/test-container/test"
171171

172172
Scenario: A MiNiFi instance can list a container on Azure blob storage through a proxy
173173
Given a ListAzureBlobStorage processor set up to communicate with an Azure blob storage
@@ -188,4 +188,4 @@ Feature: Sending data from MiNiFi-C++ to an Azure storage server
188188
Then the Minifi logs contain the following message: "key:azure.blobname value:test_1" in less than 60 seconds
189189
And the Minifi logs contain the following message: "key:azure.blobname value:test_2" in less than 60 seconds
190190
And the Minifi logs do not contain the following message: "key:azure.blobname value:other_test" after 0 seconds
191-
And no errors were generated on the http-proxy regarding "http://azure-storage-server-${feature_id}:10000/devstoreaccount1/test-container"
191+
And no errors were generated on the http-proxy regarding "http://azure-storage-server-${scenario_id}:10000/devstoreaccount1/test-container"

0 commit comments

Comments
 (0)