|
| 1 | +# Logging standards |
| 2 | + |
| 3 | +Logging is a critical part of any application. It provides a way to monitor the health of the application, debug issues, and provide an audit trail of what has happened. |
| 4 | + |
| 5 | +## Only log useful information |
| 6 | + |
| 7 | +This will depend on the application, but some common things to log include: |
| 8 | +- **Errors**: Capture unexpected issues or failures. |
| 9 | +- **Warnings**: Highlight potential problems that may not immediately impact functionality. |
| 10 | +- **Informational messages**: Provide insights into the application's normal operations. |
| 11 | +- **Debug messages**: Help developers understand the application's behavior during development. |
| 12 | +- **Audit logs**: Record user actions or system events for compliance and security purposes. |
| 13 | + |
| 14 | +Logging excessive or redundant information can make logs harder to analyze and increase storage costs. |
| 15 | + |
| 16 | +Consider the logging any third-party libraries do and configure them appropriately. |
| 17 | + |
| 18 | +## Log different levels across environments |
| 19 | + |
| 20 | +Different log levels are useful in different environments. Use the following as a baseline approach and ensure that log levels are easily configurable: |
| 21 | + |
| 22 | +- **Development**: Log debug messages to assist with troubleshooting. |
| 23 | +- **Testing/Staging**: Focus on warnings and errors to validate application behavior. |
| 24 | +- **Production**: Restrict logs to errors and critical warnings to minimize noise and protect performance. |
| 25 | + |
| 26 | +## Use structured logs |
| 27 | + |
| 28 | +Structured logs are formatted in a way that makes them easy to parse and search. They are typically in a key-value format, where each key represents a piece of information about the log message. |
| 29 | + |
| 30 | +Industry standard structured logging formats should be followed rather than inventing your own. For example, [Elastic Common Schema (ECS)](https://www.elastic.co/docs/reference/ecs) |
| 31 | + |
| 32 | +```json |
| 33 | +{ |
| 34 | + "@timestamp": "2025-04-16T12:00:00Z", |
| 35 | + "log.level": "error", |
| 36 | + "message": "Database connection failed", |
| 37 | + "service.name": "user-service", |
| 38 | + "service.environment": "production", |
| 39 | + "event.category": "database", |
| 40 | + "event.action": "connection_failure", |
| 41 | + "host.name": "example-host", |
| 42 | + "process.name": "user-service-process" |
| 43 | +} |
| 44 | +``` |
| 45 | + |
| 46 | +## Do not log any personally identifiable information (PII) or sensitive data |
| 47 | + |
| 48 | +Logging sensitive data can lead to security breaches and compliance violations. Always sanitize logs to remove any sensitive information before storing or transmitting them. |
| 49 | + |
| 50 | +This includes, but is not limited to: |
| 51 | +- Names |
| 52 | +- Addresses |
| 53 | +- Email addresses |
| 54 | +- Phone numbers |
| 55 | +- National Insurance numbers |
| 56 | +- Bank details |
| 57 | +- Usernames |
| 58 | +- Passwords |
| 59 | +- API keys |
| 60 | +- Tokens |
| 61 | + |
| 62 | +Be aware of any logging third party libraries may do with this data to avoid any unexpected leaks. |
| 63 | + |
| 64 | +## Logs should be written to a centralised logging system |
| 65 | + |
| 66 | +This allows logs to be aggregated and searched in one place. Examples of centralized logging systems include: |
| 67 | + |
| 68 | +- Elasticsearch, Logstash, and Kibana (ELK stack) |
| 69 | +- AWS CloudWatch |
| 70 | +- Azure Monitor |
| 71 | + |
| 72 | +## Protective monitoring events should be written to Security Operations Centre (SOC) |
| 73 | + |
| 74 | +Protective monitoring events should be sent to the SOC for analysis and response. |
| 75 | + |
| 76 | +Teams are expected to engage with the SOC team for guidance on what events should be logged and how to send them. |
| 77 | + |
| 78 | +Examples of events that may need sending to the SOC: |
| 79 | +- Authentication attempts |
| 80 | +- Access control changes |
| 81 | +- Configuration changes |
| 82 | +- Network traffic |
| 83 | +- Decision making |
0 commit comments