Skip to content

Document proper log analysis  #834

@kltm

Description

@kltm

Now that there are rotating apache logs in an S3 bucket, how does the average operator get at them and use them?

  • Have proper AWS permissions, including API keys
  • Get AWS client tools
  • Get logs in obo-service-logs; currently obo/purl/355fea0db941/2022/03/ would be most useful for what we want
    For example: aws s3 sync s3://DESIRED_LOGS . --include "*" --exclude "*error*"
  • Decompress pigz -d purl.obolibrary.org.access.log.202203*.gz && cat purl.obolibrary.org.access.log.202203* > all-log.txt
  • Use your favorite apache2 log tool; for example goaccess all-log.txt --log-format=COMBINED

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions