|
| 1 | +import { Callout } from 'nextra/components' |
| 2 | + |
| 3 | +# AWS S3 Access Logs |
| 4 | + |
| 5 | +AWS S3 access logs provide detailed records of requests made to your S3 bucket, including information about who accessed your data, |
| 6 | +when they accessed it, and what operations they performed. These logs capture details such as requester information, |
| 7 | +request details, response status, and error codes, which are essential for security monitoring, compliance auditing, |
| 8 | +and troubleshooting access issues. |
| 9 | + |
| 10 | +## Ingest Methods |
| 11 | + |
| 12 | +Setup the ingestion of this source using one of the following guides. |
| 13 | + |
| 14 | +- [AWS S3 Bucket](/sources/object-storage/s3) |
| 15 | +- [AWS S3 Bucket with Custom SQS](/sources/object-storage/external-s3) |
| 16 | + |
| 17 | +<Callout type='info'> |
| 18 | +If using an AWS S3 bucket use the following SNS topic ARN to send your bucket notifications. |
| 19 | +``` |
| 20 | +arn:aws:sns:<REGION>:253602268883:runreveal_s3access |
| 21 | +``` |
| 22 | +</Callout> |
| 23 | + |
| 24 | +## Setup |
| 25 | + |
| 26 | +### Step 1: Create a Target S3 Bucket for Access Logs |
| 27 | + |
| 28 | +1. Sign in to the AWS Management Console and open the Amazon S3 console. |
| 29 | +2. Click on "Create bucket". |
| 30 | +3. Enter a unique name for your bucket (e.g., `my-s3-access-logs`) and select the region. |
| 31 | +4. Configure the bucket settings as needed (e.g., versioning, encryption). |
| 32 | +5. Click "Create bucket" to finish. |
| 33 | + |
| 34 | +### Step 2: Configure Bucket Policy for Log Delivery |
| 35 | + |
| 36 | +1. In the S3 console, select the bucket you just created for access logs. |
| 37 | +2. Go to the "Permissions" tab. |
| 38 | +3. Under "Bucket policy", click "Edit". |
| 39 | +4. Paste the following policy, replacing `{target-bucket-name}` with your actual bucket name: |
| 40 | + |
| 41 | +```json |
| 42 | +{ |
| 43 | + "Version": "2012-10-17", |
| 44 | + "Statement": [ |
| 45 | + { |
| 46 | + "Effect": "Allow", |
| 47 | + "Principal": { |
| 48 | + "Service": "logging.s3.amazonaws.com" |
| 49 | + }, |
| 50 | + "Action": "s3:PutObject", |
| 51 | + "Resource": "arn:aws:s3:::{target-bucket-name}/*", |
| 52 | + "Condition": { |
| 53 | + "StringEquals": { |
| 54 | + "aws:SourceAccount": "{source-bucket-account-id}" |
| 55 | + } |
| 56 | + } |
| 57 | + } |
| 58 | + ] |
| 59 | +} |
| 60 | +``` |
| 61 | + |
| 62 | +5. Replace `{source-bucket-account-id}` with your AWS account ID. |
| 63 | +6. Click "Save changes". |
| 64 | + |
| 65 | +### Step 3: Enable Access Logging for Your Source S3 Bucket |
| 66 | + |
| 67 | +1. In the S3 console, select the source bucket for which you want to enable access logging. |
| 68 | +2. Go to the "Properties" tab. |
| 69 | +3. Scroll down to the "Server access logging" section and click "Edit". |
| 70 | +4. Check the box next to "Enable server access logging". |
| 71 | +5. Select the target bucket you created earlier from the "Target bucket" dropdown. |
| 72 | +6. Optionally, specify a prefix for the log files (e.g., `logs/`). |
| 73 | +7. Click "Save changes". |
| 74 | + |
| 75 | +### Step 4: Verify Log Delivery |
| 76 | + |
| 77 | +1. Wait for a few minutes to allow some access requests to your source bucket. |
| 78 | +2. Go back to the S3 console and open your target bucket. |
| 79 | +3. Navigate to the folder where you specified the prefix (or the root if no prefix was specified). |
| 80 | +4. You should see log files appearing in this location with names like `YYYY-MM-DD-HH-MM-SS-XXXXXXXXXX`. |
| 81 | + |
| 82 | +### Step 5: Understanding S3 Access Log Format |
| 83 | + |
| 84 | +S3 access logs contain the following fields in space-delimited format: |
| 85 | + |
| 86 | +- **Bucket Owner**: The canonical user ID of the bucket owner |
| 87 | +- **Bucket**: The name of the bucket |
| 88 | +- **Time**: The time when the request was received |
| 89 | +- **Remote IP**: The IP address of the requester |
| 90 | +- **Requester**: The canonical user ID of the requester |
| 91 | +- **Request ID**: A unique identifier for the request |
| 92 | +- **Operation**: The operation being performed (e.g., GET, PUT, DELETE) |
| 93 | +- **Key**: The key (path) of the object being accessed |
| 94 | +- **Request-URI**: The HTTP request URI |
| 95 | +- **HTTP Status**: The HTTP status code returned |
| 96 | +- **Error Code**: The S3 error code (if applicable) |
| 97 | +- **Bytes Sent**: The number of bytes sent |
| 98 | +- **Object Size**: The size of the object |
| 99 | +- **Total Time**: The total time of the request |
| 100 | +- **Turn-Around Time**: The time between when the request was received and the response was sent |
| 101 | +- **Referer**: The HTTP referer header |
| 102 | +- **User-Agent**: The HTTP user-agent header |
| 103 | +- **Version ID**: The version ID of the object (if versioning is enabled) |
| 104 | +- **Host ID**: The host ID of the S3 endpoint |
| 105 | +- **Signature Version**: The signature version used for authentication |
| 106 | +- **Cipher Suite**: The cipher suite used for HTTPS requests |
| 107 | +- **Authentication Type**: The type of authentication used |
| 108 | +- **Host Header**: The host header of the request |
| 109 | +- **TLS Version**: The TLS version used for HTTPS requests |
| 110 | + |
| 111 | +--- |
| 112 | + |
| 113 | +For more information, refer to the [official AWS documentation on S3 server access logging](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ServerLogs.html). |
0 commit comments