Recommendations for multi-az deployment in AWS

[ajax-kovtunov-o]

Is your question request related to a specific component?

VictoriaLogs deploy in AWS

Describe the question in detail

Hi, I'm configuring VictoriaLogs in our infrastructure, and I'm looking for the right and recommended setup for VictoriaLogs in AWS (or any other cloud).

I want to deploy VictoriaLogs multi-az in several concurrent instances.

Also, I saw at helm-charts that VL can be deployed in 2 ways: single-binary and cloud.

1. Is it recommended to deploy several instances of VL-single-binary?
2. What kind of AWS Storage do you recommend using in a multi-az deploy: EFS, S3?

Please share your ideal setup for your tool once you are confident in its reliability.

Key concepts docs

• VictoriaLogs key conepts - https://docs.victoriametrics.com/victorialogs/keyconcepts/
• LogsQL - https://docs.victoriametrics.com/victorialogs/logsql/

[dmitryk-dk]

Hi @ajax-kovtunov-o ,

Thanks for the detailed question!
Let me provide some recommendations for multi-AZ deployment of VictoriaLogs in AWS.

Architecture Overview

For multi-AZ deployments requiring high SLA and the ability to survive complete AZ failures,
we recommend the Multi-Cluster / Multi-AZ topology. This approach deploys two or more independent, self-contained
VictoriaLogs instances in separate failure domains.

The key principle is the "bulkhead pattern": each AZ runs a complete, standalone instance,
and a stateless routing layer directs both write and read traffic across these instances.

Deployment Options

Option 1: VictoriaMetrics Operator (Recommended)

We recommend using the https://github.com/VictoriaMetrics/operator which provides native Kubernetes CRDs for VictoriaLogs:

• VLSingle - single-node VictoriaLogs deployment
• VLCluster - clustered VictoriaLogs deployment (vlinsert, vlselect, vlstorage)
• VLAgent - log collection and replication component

The operator simplifies deployment, upgrades, and provides GitOps-friendly declarative configuration. It also supports topology spread constraints, affinity rules, and pod disruption budgets out of the box.

Option 2: Helm Charts

If you prefer Helm, use:

• https://github.com/VictoriaMetrics/helm-charts/tree/master/charts/victoria-logs-single - for single-node deployments
• https://github.com/VictoriaMetrics/helm-charts/tree/master/charts/victoria-logs-cluster - for cluster mode (vlinsert, vlselect, vlstorage)

Multi-AZ Architecture

Recommended Setup: Vector → vlagent → VictoriaLogs

Advantages of This Architecture

• Simplified collector config - Vector has only one output destination (vlagent)
• Centralized replication logic - vlagent handles fan-out to multiple VictoriaLogs instances
• HA for ingestion path - vlagent replicas spread across AZs ensure ingestion survives AZ failure
• Buffering - vlagent provides additional buffering layer if VictoriaLogs instances are temporarily unavailable

Vector Configuration (Simplified)

  sinks:
    vlagent:
      type: http
      inputs: [parser]
      uri: "http://vlagent.namespace.svc:9427/insert/jsonline"
      encoding:
        codec: json
      compression: gzip

vlagent Configuration (simplified)

  remoteWrite:
    - url: "http://vlogs-az-a:9428/insert/jsonline"
    - url: "http://vlogs-az-b:9428/insert/jsonline"
    - url: "http://vlogs-az-c:9428/insert/jsonline"

Query Routing with vmauth

Use vmauth as a global routing layer for queries with automatic failover:

  unauthorizedUser:
    url_map:
      - src_paths:
          - "/select/.*"
        url_prefix:
          - "http://vlogs-az-a:9428/"
          - "http://vlogs-az-b:9428/"
          - "http://vlogs-az-c:9428/"

vmauth automatically re-routes requests to healthy backends if some instances are temporarily unavailable.

Single-Node vs Cluster Mode

Single-Node Instances for Multi-AZ

Single-node VictoriaLogs instances can be effectively used for multi-AZ HA deployments:

Advantages of Single-Node:

• Scales linearly with available CPU, RAM, and disk resources
• Simpler operational model
• Can handle terabytes of uncompressed logs with modest disk allocations
• No coordination overhead between nodes

Storage Recommendations for AWS

EBS gp3/io2 (Recommended)

• Use EBS gp3 volumes for cost-effective performance
• EBS volumes are AZ-bound - this aligns well with the multi-instance approach where each instance lives in its own AZ
• Configure appropriate IOPS and throughput based on your ingestion rate

Why NOT EFS

• EFS has higher latency and lower IOPS compared to EBS
• Not optimal for database workloads requiring fast random I/O
• Higher cost for equivalent performance

Why NOT S3

• VictoriaLogs requires block storage for its data directory
• S3 is not supported as a primary storage backend

Example: Multi-AZ with VictoriaMetrics Operator

  # vlagent for log replication (2 replicas across AZs)
  apiVersion: operator.victoriametrics.com/v1beta1
  kind: VLAgent
  metadata:
    name: vlagent
  spec:
    replicaCount: 2
    topologySpreadConstraints:
      - maxSkew: 1
        topologyKey: topology.kubernetes.io/zone
        whenUnsatisfiable: DoNotSchedule
        labelSelector:
          matchLabels:
            app.kubernetes.io/name: vlagent
    remoteWrite:
      - url: "http://vlogs-az-a-vlsingle:9428/insert/jsonline"
      - url: "http://vlogs-az-b-vlsingle:9428/insert/jsonline"
  ---
  # VLSingle in AZ-A
  apiVersion: operator.victoriametrics.com/v1beta1
  kind: VLSingle
  metadata:
    name: vlogs-az-a
  spec:
    retentionPeriod: "7d"
    storage:
      volumeClaimTemplate:
        spec:
          storageClassName: gp3
          resources:
            requests:
              storage: 100Gi
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
                - key: topology.kubernetes.io/zone
                  operator: In
                  values: ["us-east-1a"]
  ---
  # VLSingle in AZ-B
  apiVersion: operator.victoriametrics.com/v1beta1
  kind: VLSingle
  metadata:
    name: vlogs-az-b
  spec:
    retentionPeriod: "7d"
    storage:
      volumeClaimTemplate:
        spec:
          storageClassName: gp3
          resources:
            requests:
              storage: 100Gi
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
                - key: topology.kubernetes.io/zone
                  operator: In
                  values: ["us-east-1b"]
  ---
  # VMAuth for query load balancing and failover
  apiVersion: operator.victoriametrics.com/v1beta1
  kind: VMAuth
  metadata:
    name: vmauth-vlogs
  spec:
    unauthorizedAccessConfig:
      - src_paths:
          - "/.*"
        url_prefix:
          - "http://vlogs-az-a-vlsingle:9428/"
          - "http://vlogs-az-b-vlsingle:9428/"

Useful Links

• https://docs.victoriametrics.com/victorialogs/

  Please check this architecture. It was built for the VictoriaMetrics., but it also works for VictoriaLogs

• https://docs.victoriametrics.com/guides/vm-architectures/#multi-cluster-and-multi-az

• https://docs.victoriametrics.com/guides/multi-regional-setup-dedicated-regions/

• https://github.com/VictoriaMetrics/operator

• https://github.com/VictoriaMetrics/helm-charts/tree/master/charts/victoria-logs-single

• https://github.com/VictoriaMetrics/helm-charts/tree/master/charts/victoria-logs-cluster