AAP-54345: library.storage: implement Directory, S3, Segment, CRC, CRCMutual

[himdel]

Issue: AAP-54345

Pulled library, docs & worker example changes from #243 (without cli updates to use it),
pulled segment from #234 and adjusted to match,
pulled crc implementations from #216.

This implements metrics_utility.library.storage:

read-write:

• StorageDirectory
• StorageS3

write-only:

• StorageCRC
• StorageCRCMutual
• StorageSegment

---

# StorageDirectory - local directory structure under base_path
#
# base_path = METRICS_UTILITY_SHIP_PATH
StorageDirectory(
    base_path='./',
)

# StorageS3 - S3 or minio
#
# bucket = METRICS_UTILITY_BUCKET_NAME
# endpoint = METRICS_UTILITY_BUCKET_ENDPOINT
# region = METRICS_UTILITY_BUCKET_REGION
# access_key = METRICS_UTILITY_BUCKET_ACCESS_KEY
# secret_key = METRICS_UTILITY_BUCKET_SECRET_KEY
StorageS3(
    bucket='name',
    endpoint='http://localhost:9000', # or 'https://s3.us-east.example.com'
    region='us-east-1', # optional
    access_key='...',
    secret_ley='...',
)

# StorageSegment - segment analytics (put-only)
#
# debug = bool
# user_id = string, passed to analytics.track
# write_key = https://segment.com/docs/connections/sources/catalog/libraries/server/python/#getting-started
StorageSegment(
    debug=False,
    user_id='unknown',
    write_key='...',
)

# StorageCRC - console.redhat.com, using service accounts (put-only)
#
# client_id = METRICS_UTILITY_SERVICE_ACCOUNT_ID
# client_secret = METRICS_UTILITY_SERVICE_ACCOUNT_SECRET
# ingress_url = METRICS_UTILITY_CRC_INGRESS_URL
# proxy_url = METRICS_UTILITY_PROXY_URL
# sso_url = METRICS_UTILITY_CRC_SSO_URL
# verify_cert_path = '/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'
StorageCRC(
    client_id='00000000-0000-0000-0000-000000000000',
    client_secret='...',
    ingress_url='https://console.redhat.com/api/ingress/v1/upload',
    proxy_url=None,
    sso_url='https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token',
    verify_cert_path='/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'
)

# StorageCRCMutual - console.redhat.com, using mutual tls (put-only)
#
# ingress_url = METRICS_UTILITY_CRC_INGRESS_URL
# proxy_url = METRICS_UTILITY_PROXY_URL
# session_cert = ('/etc/pki/consumer/cert.pem', '/etc/pki/consumer/key.pem')
# verify_cert_path = '/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'
StorageCRCMutual(
    ingress_url='https://console.redhat.com/api/ingress/v1/upload',
    proxy_url=None,
    session_cert=('/etc/pki/consumer/cert.pem', '/etc/pki/consumer/key.pem'),
    verify_cert_path='/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'
)

Supported methods:

• storage.put(name, ...) - should upload to storage, and raise on failure
  • storage.put(name, dict=data) - uploads a dict, likely as json data, or a .json file
  • storage.put(name, filename=path) - uploads a local file (by name)
  • storage.put(name, fileobj=handle) - uploads an opened local file or a compatible object (by a file-like handle)
• storage.get(name) - (context manager) should download from storage into a temporary file, yield the temporary filename, and remove the file again.

Also supported - exists(name) -> Bool, remove(name), glob(pattern) -> [filenames].

(CRC, CRCMutual & Segment only support put.)

[sonarqubecloud]

Quality Gate failed

Failed conditions
67.3% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

[cshiels-ie]

Looks all good !