Skip to content

Commit a8138a9

Browse files
committed
Merge branch 'bugfix/S3UTILS-228/port' into q/1.17
2 parents d80f99c + 2169cb2 commit a8138a9

3 files changed

Lines changed: 59 additions & 18 deletions

File tree

replicationAudit/README.md

Lines changed: 16 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -39,16 +39,17 @@ ansible -i env/$ENV_DIR/inventory runners_s3[0] -m shell \
3939
# Step 3: Find the vault-metadata repd leader IP (port 5300)
4040
ansible -i env/$ENV_DIR/inventory md1-cluster1 -m shell \
4141
-a 'curl -s http://localhost:5300/_/raft/leader'
42-
# Note the "ip" value from the output, e.g., {"ip":"10.160.116.162","port":4300}
42+
# Note both "ip" and "port" values, e.g., {"ip":"10.160.116.162","port":4300}
43+
# On 3-server rings the port may differ per node (4300, 4301, etc.)
4344

44-
# Step 4: Set the LEADER_IP variable and run the permission check script
45+
# Step 4: Set the LEADER variable (ip:port) and run the permission check script
4546
# Note: replace ctrctl with docker on RHEL/CentOS 7
46-
LEADER_IP=<leader-ip-from-step-3>
47+
LEADER=<leader-ip>:<leader-port-from-step-3>
4748

4849
ansible -i env/$ENV_DIR/inventory runners_s3[0] -m shell \
4950
-a "cp /root/buckets-with-replication.json {{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs && \
5051
ctrctl exec scality-vault{{ container_name_suffix | default("")}} node /logs/check-replication-permissions.js \
51-
/logs/buckets-with-replication.json $LEADER_IP /logs/missing.json"
52+
/logs/buckets-with-replication.json $LEADER /logs/missing.json"
5253

5354
# Step 5: Retrieve results
5455
ansible -i env/$ENV_DIR/inventory runners_s3[0] -m shell \
@@ -276,19 +277,20 @@ This ensures portability while using the exact same protocol and key formats as
276277
-a 'curl -s http://localhost:5300/_/raft/leader'
277278
```
278279

279-
This returns JSON like `{"ip":"10.160.116.162","port":4300}` - use the `ip` value.
280+
This returns JSON like `{"ip":"10.160.116.162","port":4300}` - use both `ip`
281+
and `port` values. On 3-server rings the port may differ per node (4300, 4301, etc.).
280282

281283
**Note:** Vault metadata uses port 5300 for admin.
282284

283285
6. Copy files to `/var/tmp` (mounted in vault container) and run the script:
284286

285287
```bash
286-
LEADER_IP=<leader-ip-from-step-5>
288+
LEADER=<leader-ip>:<leader-port-from-step-5>
287289

288290
ansible -i env/$ENV_DIR/inventory runners_s3[0] -m shell \
289291
-a "cp /root/buckets-with-replication.json {{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs && \
290292
ctrctl exec scality-vault{{ container_name_suffix | default("")}} node /logs/check-replication-permissions.js \
291-
/logs/buckets-with-replication.json $LEADER_IP /logs/missing.json"
293+
/logs/buckets-with-replication.json $LEADER /logs/missing.json"
292294
```
293295

294296
8. Retrieve the output:
@@ -301,13 +303,13 @@ This ensures portability while using the exact same protocol and key formats as
301303
### Command Line Arguments
302304

303305
```
304-
node check-replication-permissions.js [input-file] [leader-ip] [output-file] [--include-policies]
306+
node check-replication-permissions.js [input-file] [leader-ip[:port]] [output-file] [--include-policies]
305307
```
306308

307309
| Argument | Default | Description |
308310
|----------|---------|-------------|
309311
| `input-file` | /root/buckets-with-replication.json | Input JSON from list script |
310-
| `leader-ip` | 127.0.0.1 | Vault-metadata repd leader IP |
312+
| `leader-ip[:port]` | 127.0.0.1:4300 | Vault-metadata repd leader IP and optional port (default 4300) |
311313
| `output-file` | /root/missing-replication-permissions.json | Output file path |
312314
| `--include-policies` | (not set) | Include full policy documents in output |
313315

@@ -426,7 +428,7 @@ In this example, the policy is missing `s3:ReplicateObject` - it only has
426428

427429
### Script Logic
428430

429-
1. **Connects to repd**: TCP connection to vault-metadata repd on port 4300
431+
1. **Connects to repd**: TCP connection to vault-metadata repd (default port 4300, configurable via `ip:port` argument)
430432
2. **For each bucket's replication role**:
431433
- Get role ID: `linkRoleArn(arn)` → role ID
432434
- List attached policies: `policyByRoleId(accountId, roleId, '', '')`
@@ -490,9 +492,10 @@ Output saved to: /tmp/missing.json
490492

491493
**Connection timeout or refused**
492494

493-
- Ensure you're connecting to the correct repd leader IP
494-
- The script must run inside a container that can reach repd on port 4300
495-
- Find the leader: `curl -s http://localhost:5300/_/raft/leader`
495+
- Ensure you're connecting to the correct repd leader IP **and port**
496+
- On 3-server rings, the leader port may not be 4300 (e.g. 4301, 4302)
497+
- Find the leader: `curl -s http://localhost:5300/_/raft/leader` — use both `ip` and `port` from the response
498+
- The script must run inside a container that can reach repd on the leader port
496499

497500
**Script timeout**
498501

replicationAudit/check-replication-permissions.js

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,15 @@
66
* Checks if replication roles have s3:ReplicateObject permission
77
* by querying Vault metadata directly via repd protocol.
88
*
9-
* Usage: node check-replication-permissions.js [input-file] [leader-ip] [output-file]
9+
* Usage: node check-replication-permissions.js [input-file] [leader-ip[:port]] [output-file]
1010
*
1111
* How it connects to vault metadata:
1212
*
1313
* Vault metadata has no HTTP frontend (no bucketd). This script connects
14-
* directly to repd (the raft-based metadata store) on TCP port 4300
15-
* using a simple protocol: 4-byte length prefix + JSON payload.
14+
* directly to repd (the raft-based metadata store) using a simple
15+
* protocol: 4-byte length prefix + JSON payload. The default repd port
16+
* is 4300 but can be overridden via the leader-ip argument (e.g.
17+
* 10.0.0.1:4301).
1618
*/
1719

1820
const net = require('net');
@@ -21,11 +23,23 @@ const fs = require('fs');
2123
// ===========================================================================
2224
// Configuration
2325
// ===========================================================================
26+
27+
/** Parse a leader address string into { ip, port } */
28+
function parseLeaderAddress(address) {
29+
if (!address) {
30+
return { ip: '127.0.0.1', port: 4300 };
31+
}
32+
const [ip, portStr] = address.split(':');
33+
return { ip, port: portStr ? parseInt(portStr, 10) : 4300 };
34+
}
35+
36+
const parsedLeader = parseLeaderAddress(process.argv[3]);
37+
2438
const CONFIG = {
2539
inputFile: process.argv[2] || '/root/buckets-with-replication.json',
26-
leaderIp: process.argv[3] || '127.0.0.1',
40+
leaderIp: parsedLeader.ip,
2741
outputFile: process.argv[4] || '/root/missing-replication-permissions.json',
28-
repdPort: 4300,
42+
repdPort: parsedLeader.port,
2943
dbName: 'vaultdb',
3044
includePolicies: process.argv.includes('--include-policies'),
3145
requestTimeoutMs: 10000,
@@ -476,5 +490,6 @@ if (require.main === module) {
476490

477491
// Export for testing
478492
module.exports = {
493+
parseLeaderAddress,
479494
policyAllowsReplication,
480495
};

tests/unit/replicationAudit/policyChecker.js

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,30 @@
11
const {
2+
parseLeaderAddress,
23
policyAllowsReplication,
34
} = require('../../../replicationAudit/check-replication-permissions');
45

6+
describe('parseLeaderAddress', () => {
7+
test('ip only defaults port to 4300', () => {
8+
expect(parseLeaderAddress('10.0.0.1')).toEqual({ ip: '10.0.0.1', port: 4300 });
9+
});
10+
11+
test('ip:port extracts both', () => {
12+
expect(parseLeaderAddress('10.0.0.1:4301')).toEqual({ ip: '10.0.0.1', port: 4301 });
13+
});
14+
15+
test('undefined defaults to 127.0.0.1:4300', () => {
16+
expect(parseLeaderAddress(undefined)).toEqual({ ip: '127.0.0.1', port: 4300 });
17+
});
18+
19+
test('empty string defaults to 127.0.0.1:4300', () => {
20+
expect(parseLeaderAddress('')).toEqual({ ip: '127.0.0.1', port: 4300 });
21+
});
22+
23+
test('custom high port', () => {
24+
expect(parseLeaderAddress('172.16.0.5:4304')).toEqual({ ip: '172.16.0.5', port: 4304 });
25+
});
26+
});
27+
528
describe('policyAllowsReplication', () => {
629
const bucketName = 'source-bucket';
730

0 commit comments

Comments
 (0)