floci-gcp/README.md at main · floci-io/floci-gcp

Light, fluffy, and always free — GCP Local Emulator
No account. No auth token. No feature gates. Just docker compose up.

Quick Start · Features · Services · SDKs · Testcontainers · Compatibility · Docs

What is floci-gcp?

floci-gcp is a free, open-source local GCP emulator for development, testing, and CI.

It gives you GCP-shaped services on your machine without requiring a cloud account, auth token, or paid feature gates. Point your GCP SDK, gcloud CLI, Terraform, or test suite at http://localhost:4588 and keep your existing workflows.

floci-gcp is named after floccus, the cloud formation that looks like popcorn.

Quick Start

# docker-compose.yml
services:
  floci-gcp:
    image: floci/floci-gcp:latest
    ports:
      - "4588:4588"
    volumes:
      - ./data:/app/data
    environment:
      FLOCI_GCP_HOSTNAME: floci-gcp
      FLOCI_GCP_BASE_URL: http://floci-gcp:4588

docker compose up -d

Export the GCP emulator environment variables:

export PUBSUB_EMULATOR_HOST=localhost:4588
export FIRESTORE_EMULATOR_HOST=localhost:4588
export DATASTORE_EMULATOR_HOST=localhost:4588
export STORAGE_EMULATOR_HOST=http://localhost:4588
export SECRET_MANAGER_EMULATOR_HOST=localhost:4588
export GOOGLE_CLOUD_PROJECT=floci-local

All GCP services are available at http://localhost:4588. Credentials are not validated.

Using Docker directly?

docker run -d --name floci-gcp \
  -p 4588:4588 \
  floci/floci-gcp:latest

Features

Local GCP without the cloud account

Run GCP-compatible services locally without a GCP account, service account key, or paid feature gates.

Single port for everything

All GCP services — gRPC and REST — share a single port (4588) via HTTP/2 ALPN negotiation. No per-service daemon setup, no port management.

Real GCP wire protocols

floci-gcp speaks the same protocols as real GCP: protobuf-over-gRPC for Pub/Sub, Firestore, and Secret Manager; binary HTTP/protobuf for Datastore; REST XML and JSON for Cloud Storage; and REST JSON for management APIs such as Cloud Run and Cloud Functions. Existing SDK calls work without modification.

Fast enough for CI

The native image starts in milliseconds and keeps idle memory low, making it practical for local development and test pipelines.

Configurable persistence

Choose from in-memory, persistent, hybrid, and write-ahead log storage depending on the durability profile you need.

Why floci-gcp?

GCP's official emulators are fragmented — each service ships its own binary, runs on a different port, and requires separate setup. floci-gcp unifies them under a single port.

Capability	floci-gcp	GCP official emulators
Single port for all services	✅	❌
gRPC + REST on the same port	✅	❌
No GCP account required	✅	✅
Pub/Sub	✅	✅
Firestore	✅	✅
Datastore	✅	✅
Cloud Storage (GCS)	✅	⚠️ Limited
Secret Manager	✅	❌
Cloud Logging	✅	❌
Cloud KMS	✅	❌
IAM	✅	❌
Managed Kafka	✅	❌
GKE (Kubernetes Engine)	✅	❌
Cloud Run	✅	❌
Cloud Functions	✅	❌
Cloud SQL for PostgreSQL	✅	❌
Cloud Tasks	✅	❌
Cloud Scheduler	✅	❌
Cloud Monitoring	✅	❌
Native binary	✅	❌

Architecture Overview

flowchart LR
    Client["GCP SDK / gcloud CLI"]

    subgraph FlociGCP ["floci-gcp, port 4588"]
        Router["HTTP/2 Router\nALPN negotiation"]

        subgraph GRPC ["gRPC services"]
            A["Pub/Sub\nFirestore\nSecret Manager\nCloud Logging\nCloud KMS\nCloud Tasks\nCloud Scheduler\nCloud Monitoring"]
        end

        subgraph REST ["REST services"]
            B["Cloud Storage\nIAM\nDatastore\nCloud Run\nCloud Functions\nCloud SQL\nGKE"]
        end

        subgraph Docker ["Docker-backed"]
            C["Managed Kafka\n(Redpanda)"]
        end

        Router --> GRPC
        Router --> REST
        Router --> Docker
        GRPC & REST --> Store[("StorageBackend\nmemory · hybrid · persistent · wal")]
    end

    DockerEngine["Docker Engine"]
    Client -->|"HTTP/2 :4588\nGCP wire protocols"| Router
    Docker -->|"Docker API"| DockerEngine

Supported Services

floci-gcp emulates GCP services across storage, messaging, identity, and managed infrastructure.

Category	Services
Object and document storage	Cloud Storage (GCS), Firestore, Datastore
Messaging	Pub/Sub, Managed Kafka
Security and identity	Secret Manager, Cloud KMS, IAM
Container orchestration	GKE (Kubernetes Engine)
Serverless control planes	Cloud Run, Cloud Functions
Task scheduling	Cloud Tasks, Cloud Scheduler
Databases	Cloud SQL for PostgreSQL
Observability	Cloud Logging, Cloud Monitoring

Detailed service notes

Service	Protocol	Notable features
Cloud Storage (GCS)	REST XML + REST JSON	Buckets, objects, multipart upload, object compose, ACLs, bucket IAM, conditional requests (preconditions), versioning, lifecycle, CORS, pre-signed URLs (V4)
Pub/Sub	gRPC	Topics, subscriptions, publish, pull, streaming pull, push delivery, snapshots, seek, field masks on update
Firestore	gRPC	Documents, collections, queries (all operators), field transforms, aggregation (COUNT), transactions, batch writes, real-time listeners (`listen` stream)
Datastore	HTTP/protobuf	Entities, structured queries, GQL queries, aggregation (COUNT), transactions, GQL named/positional bindings
Secret Manager	gRPC	Secrets, versioning, access, `versions/latest` alias, disable/enable/destroy, IAM bindings
Cloud Logging	gRPC + REST JSON	Structured log ingestion (`WriteLogEntries`), read-back (`ListLogEntries`) with a practical filter subset (logName, severity, resource.type, timestamp, labels), `ListLogs`, `DeleteLog`; text/JSON payloads
Cloud KMS	gRPC + REST JSON	Key rings, crypto keys, key versions, symmetric encrypt/decrypt (AES-256-GCM), asymmetric sign (EC P-256, RSA PKCS1) and decrypt (RSA-OAEP), `GetPublicKey`, `GenerateRandomBytes`, CRC32C integrity fields
IAM	REST JSON	Service accounts, RSA-2048 key pairs (JSON key file format), policy bindings, `SignBlob` (V4 signed URLs)
Managed Kafka	REST JSON	Clusters, topics, consumer groups; Redpanda-backed or mock mode
GKE (Kubernetes Engine)	REST JSON	Clusters and operations (`container.googleapis.com` v1); real k3s clusters via Docker (`rancher/k3s`) or mock mode. Reached by SDKs/gcloud through host-based routing (`container.*`) or the `/container/v1` path prefix
Cloud Run	REST JSON	Services, IAM policies, revisions, long-running operations; control plane by default, experimental Docker-backed invocation when enabled
Cloud Functions	REST JSON	Functions, source upload URL generation, long-running operations; control plane only, no runtime invocation
Cloud SQL for PostgreSQL	REST JSON	Instances (Postgres), control-plane lifecycle, long-running operations
Cloud Tasks	gRPC	Queues (rate limits, retry config, pause/resume/purge), tasks (HTTP and App Engine targets, schedule time), `RunTask`; control plane only, tasks are tracked but not dispatched
Cloud Scheduler	gRPC + REST JSON	Cron jobs with Pub/Sub, HTTP, and App Engine targets; `Pause`/`Resume`/`RunJob`; unix-cron + time zones; background tick fires due jobs (Pub/Sub publishes into the local backend)
Cloud Monitoring	gRPC + REST JSON	Metric descriptors (create/get/list/delete), monitored resource descriptors, time series write (`CreateTimeSeries`) and read (`ListTimeSeries`)

Real Docker Integration

floci-gcp uses real Docker containers when in-process emulation would reduce fidelity — stateful databases, connection-heavy protocols, and image-based runtimes. These services spawn sidecar containers via the host Docker daemon. Each is gated by a per-service mock flag: set it to true to keep the service metadata-only without Docker.

Service	Default image	What is real	Mock flag
Managed Kafka	`redpandadata/redpanda:latest`	Kafka-compatible broker via Redpanda	`FLOCI_GCP_SERVICES_KAFKA_MOCK`
Cloud SQL for PostgreSQL	`postgres:15.18-alpine` (15–18)	PostgreSQL engine, JDBC-compatible access	`FLOCI_GCP_SERVICES_CLOUDSQL_MOCK`
Cloud Run	User-specified container image	Image-based service execution and request serving	`FLOCI_GCP_SERVICES_CLOUDRUN_MOCK`

Docker-backed services require the Docker socket:

docker run -d --name floci-gcp \
  -p 4588:4588 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  floci/floci-gcp:latest

Overriding default images

Variable	Default
`FLOCI_GCP_SERVICES_KAFKA_DEFAULT_IMAGE`	`redpandadata/redpanda:latest`
`FLOCI_GCP_SERVICES_CLOUDSQL_POSTGRES15_IMAGE`	`postgres:15.18-alpine`
`FLOCI_GCP_SERVICES_CLOUDSQL_POSTGRES16_IMAGE`	`postgres:16.14-alpine`
`FLOCI_GCP_SERVICES_CLOUDSQL_POSTGRES17_IMAGE`	`postgres:17.10-alpine`
`FLOCI_GCP_SERVICES_CLOUDSQL_POSTGRES18_IMAGE`	`postgres:18.4-alpine`

Persistence and Storage Modes

floci-gcp supports flexible storage modes. Configure globally via FLOCI_GCP_STORAGE_MODE.

Mode	Behavior	Best for	Durability
`memory`	(Default) Entirely in-RAM. Lost on container stop.	Speed, CI pipelines	❌ None
`persistent`	Every write goes directly to disk synchronously.	Durable local dev	✅ Good
`hybrid`	In-memory with async flush every 5 seconds.	Balance of speed and safety	✅ Good
`wal`	Write-Ahead Log. Every mutation written to disk immediately.	Maximum durability	💎 Highest

Use memory for fast CI runs. Use hybrid when you want state preserved across container restarts.

Multi-Project Isolation

GCP resource names follow projects/{project}/.... floci-gcp uses the project ID as the multi-tenancy boundary — resources in project-a are invisible to project-b.

The project ID is resolved in this order:

URL path segment projects/{project}/...
x-goog-request-params header (project=...)
FLOCI_GCP_DEFAULT_PROJECT_ID fallback (default: floci-local)

# Two projects, full isolation
export PUBSUB_EMULATOR_HOST=localhost:4588

gcloud pubsub topics create my-topic --project=project-a
gcloud pubsub topics create my-topic --project=project-b

# Each project has its own independent topic

SDK Integration

Point your existing GCP SDK at http://localhost:4588.

Java (GCP SDK)

// Pub/Sub
ManagedChannel channel = ManagedChannelBuilder
    .forTarget("localhost:4588")
    .usePlaintext()
    .build();

TransportChannelProvider channelProvider =
    FixedTransportChannelProvider.create(GrpcTransportChannel.create(channel));
CredentialsProvider credentialsProvider = NoCredentialsProvider.create();

TopicAdminClient topicClient = TopicAdminClient.create(
    TopicAdminSettings.newBuilder()
        .setTransportChannelProvider(channelProvider)
        .setCredentialsProvider(credentialsProvider)
        .build());

topicClient.createTopic(TopicName.of("floci-local", "my-topic"));

// Cloud Storage
Storage storage = StorageOptions.newBuilder()
    .setHost("http://localhost:4588")
    .setProjectId("floci-local")
    .setCredentials(NoCredentials.getInstance())
    .build()
    .getService();

storage.create(BucketInfo.of("my-bucket"));
storage.create(BlobInfo.newBuilder("my-bucket", "hello.txt").build(),
    "hello from floci-gcp".getBytes());

// Firestore
FirestoreOptions options = FirestoreOptions.newBuilder()
    .setHost("localhost:4588")
    .setProjectId("floci-local")
    .setCredentials(NoCredentials.getInstance())
    .build();

Firestore db = options.getService();
db.collection("users").add(Map.of("name", "Alice", "age", 30)).get();

Python (google-cloud)

import os
os.environ["PUBSUB_EMULATOR_HOST"] = "localhost:4588"

from google.cloud import pubsub_v1

publisher = pubsub_v1.PublisherClient()
topic_path = publisher.topic_path("floci-local", "my-topic")
publisher.create_topic(request={"name": topic_path})
future = publisher.publish(topic_path, b"hello from floci-gcp")
future.result()

import os
os.environ["STORAGE_EMULATOR_HOST"] = "http://localhost:4588"

from google.cloud import storage

client = storage.Client(project="floci-local")
bucket = client.bucket("my-bucket")
client.create_bucket(bucket)

blob = bucket.blob("hello.txt")
blob.upload_from_string("hello from floci-gcp")
print(blob.download_as_text())

import os
os.environ["FIRESTORE_EMULATOR_HOST"] = "localhost:4588"

from google.cloud import firestore

db = firestore.Client(project="floci-local")
db.collection("users").add({"name": "Alice", "age": 30})
docs = db.collection("users").where("name", "==", "Alice").stream()
for doc in docs:
    print(doc.to_dict())

Node.js

import { PubSub } from "@google-cloud/pubsub";

process.env.PUBSUB_EMULATOR_HOST = "localhost:4588";

const pubsub = new PubSub({ projectId: "floci-local" });
await pubsub.createTopic("my-topic");
const [subscription] = await pubsub.topic("my-topic").createSubscription("my-sub");

import { Storage } from "@google-cloud/storage";

const storage = new Storage({
  apiEndpoint: "http://localhost:4588",
  projectId: "floci-local",
});

await storage.createBucket("my-bucket");
await storage.bucket("my-bucket").file("hello.txt").save("hello from floci-gcp");

package main

import (
    "context"
    "fmt"
    "log"

    "cloud.google.com/go/pubsub"
    "google.golang.org/api/option"
    "google.golang.org/grpc"
    "google.golang.org/grpc/credentials/insecure"
)

func main() {
    ctx := context.Background()

    conn, err := grpc.Dial("localhost:4588", grpc.WithTransportCredentials(insecure.NewCredentials()))
    if err != nil {
        log.Fatal(err)
    }

    client, err := pubsub.NewClient(ctx, "floci-local",
        option.WithGRPCConn(conn))
    if err != nil {
        log.Fatal(err)
    }
    defer client.Close()

    topic, err := client.CreateTopic(ctx, "my-topic")
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println("Created topic:", topic.ID())
}

Bash (gcloud CLI)

export PUBSUB_EMULATOR_HOST=localhost:4588
gcloud config set project floci-local

# Pub/Sub
gcloud pubsub topics create my-topic
gcloud pubsub subscriptions create my-sub --topic=my-topic
gcloud pubsub topics publish my-topic --message="hello from floci-gcp"
gcloud pubsub subscriptions pull my-sub --auto-ack

# Cloud Storage
export STORAGE_EMULATOR_HOST=http://localhost:4588
gcloud storage buckets create gs://my-bucket
echo "hello" | gcloud storage cp - gs://my-bucket/hello.txt
gcloud storage ls gs://my-bucket

# Secret Manager
export SECRET_MANAGER_EMULATOR_HOST=localhost:4588
gcloud secrets create my-secret --replication-policy=automatic
echo -n "my-value" | gcloud secrets versions add my-secret --data-file=-
gcloud secrets versions access latest --secret=my-secret

Testcontainers

Use GenericContainer to start an isolated floci-gcp instance directly from your tests. This avoids shared state, manual daemon setup, and port conflicts.

Java

@Testcontainers
class PubSubIntegrationTest {

    @Container
    static GenericContainer<?> flociGcp = new GenericContainer<>("floci/floci-gcp:latest")
        .withExposedPorts(4588)
        .waitingFor(Wait.forHttp("/_floci/health").forPort(4588));

    static TopicAdminClient topicClient;

    @BeforeAll
    static void setup() throws Exception {
        String host = flociGcp.getHost();
        int port = flociGcp.getMappedPort(4588);

        ManagedChannel channel = ManagedChannelBuilder
            .forAddress(host, port)
            .usePlaintext()
            .build();

        topicClient = TopicAdminClient.create(
            TopicAdminSettings.newBuilder()
                .setTransportChannelProvider(
                    FixedTransportChannelProvider.create(GrpcTransportChannel.create(channel)))
                .setCredentialsProvider(NoCredentialsProvider.create())
                .build());
    }

    @Test
    void shouldCreateTopic() {
        topicClient.createTopic(TopicName.of("floci-local", "test-topic"));
    }
}

Python

import pytest
from testcontainers.core.container import DockerContainer
from google.cloud import pubsub_v1


@pytest.fixture(scope="session")
def floci_gcp():
    with DockerContainer("floci/floci-gcp:latest").with_exposed_ports(4588) as container:
        container.get_exposed_port(4588)  # wait for startup
        yield container


def test_pubsub(floci_gcp):
    port = floci_gcp.get_exposed_port(4588)
    host = floci_gcp.get_container_host_ip()

    import os
    os.environ["PUBSUB_EMULATOR_HOST"] = f"{host}:{port}"

    publisher = pubsub_v1.PublisherClient()
    topic_path = publisher.topic_path("floci-local", "test-topic")
    publisher.create_topic(request={"name": topic_path})

Compatibility Testing

The compatibility-tests directory validates floci-gcp across SDKs and IaC tools.

Module	Language / Tool	SDK / Client
`sdk-test-java`	Java	GCP SDK for Java
`sdk-test-node`	Node.js	`@google-cloud/*`
`sdk-test-python`	Python	`google-cloud-*`
`sdk-test-go`	Go	`cloud.google.com/go/*`
`sdk-test-gcloud`	gcloud CLI	`gcloud` (bats)
`compat-terraform`	Terraform	Google provider
`compat-opentofu`	OpenTofu	Google provider

Run the full suite:

cd compatibility-tests && just test-java
cd compatibility-tests && just test-go
cd compatibility-tests && just test-terraform

Migrating from gcloud emulators

Google ships a separate emulator per service (gcloud beta emulators pubsub | firestore | datastore | bigtable | spanner), each its own process on its own port. floci-gcp replaces all of them with one binary on a single port (4588), reusing the same *_EMULATOR_HOST environment variables the GCP SDKs already honor — just point them at floci-gcp.

gcloud emulator	floci-gcp
`gcloud beta emulators pubsub start` → `PUBSUB_EMULATOR_HOST=localhost:8085`	`PUBSUB_EMULATOR_HOST=localhost:4588`
`gcloud beta emulators firestore start` → `FIRESTORE_EMULATOR_HOST=localhost:8080`	`FIRESTORE_EMULATOR_HOST=localhost:4588`
`gcloud beta emulators datastore start` → `DATASTORE_EMULATOR_HOST=localhost:8081`	`DATASTORE_EMULATOR_HOST=localhost:4588`
(no official emulator)	`STORAGE_EMULATOR_HOST=http://localhost:4588`
(no official emulator)	`SECRET_MANAGER_EMULATOR_HOST=localhost:4588`

The GCP SDKs skip credential checks automatically when these variables are set, so no code changes are needed — one container replaces the fragmented per-service emulator processes.

Image Tags

Every tag combines a variant and a channel.

Channel	Tag
Release, floating	`latest`
Release, pinned	`x.y.z`
Nightly, floating	`nightly`
Nightly, dated	`nightly-mmddyyyy`

Use latest for stable releases, a pinned version for reproducible builds, and nightly to track main.

# Recommended
image: floci/floci-gcp:latest

# Pinned release
image: floci/floci-gcp:1.0.0

# Track main
image: floci/floci-gcp:nightly

Configuration

All settings are overridable via environment variables (FLOCI_GCP_ prefix).

Variable	Default	Description
`FLOCI_GCP_PORT`	`4588`	Port for all services (gRPC + REST)
`FLOCI_GCP_DEFAULT_PROJECT_ID`	`floci-local`	Default GCP project ID
`FLOCI_GCP_BASE_URL`	`http://localhost:4588`	Base URL returned in service responses
`FLOCI_GCP_HOSTNAME`	(unset)	Hostname to use in returned URLs when running inside Docker Compose
`FLOCI_GCP_STORAGE_MODE`	`memory`	Storage mode: `memory` · `persistent` · `hybrid` · `wal`
`FLOCI_GCP_STORAGE_PERSISTENT_PATH`	`./data`	Directory for persisted state

Per-service enable flags (FLOCI_GCP_SERVICES_<SERVICE>_ENABLED), mock flags (*_MOCK), sidecar images, Docker networking, and DNS suffixes are documented in the full reference.

Full reference: configuration docs

Multi-container Docker Compose

When your application runs in a different container, set FLOCI_GCP_HOSTNAME to the floci-gcp service name so returned URLs resolve correctly from other containers.

services:
  floci-gcp:
    image: floci/floci-gcp:latest
    ports:
      - "4588:4588"
    environment:
      FLOCI_GCP_HOSTNAME: floci-gcp
      FLOCI_GCP_BASE_URL: http://floci-gcp:4588

  my-app:
    environment:
      PUBSUB_EMULATOR_HOST: floci-gcp:4588
      FIRESTORE_EMULATOR_HOST: floci-gcp:4588
      STORAGE_EMULATOR_HOST: http://floci-gcp:4588
    depends_on:
      - floci-gcp

Community

Join the Floci community on Slack or GitHub Discussions. Feature ideas, compatibility questions, design tradeoffs, and rough proposals are welcome.

Star History

Contributors

License

MIT — use it however you want.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

What is floci-gcp?

Quick Start

Features

Why floci-gcp?

Architecture Overview

Supported Services

Real Docker Integration

Overriding default images

Persistence and Storage Modes

Multi-Project Isolation

SDK Integration

Testcontainers

Compatibility Testing

Migrating from gcloud emulators

Image Tags

Configuration

Multi-container Docker Compose

Community

Star History

Contributors

License

Uh oh!

Uh oh!

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

What is floci-gcp?

Quick Start

Features

Why floci-gcp?

Architecture Overview

Supported Services

Real Docker Integration

Overriding default images

Persistence and Storage Modes

Multi-Project Isolation

SDK Integration

Testcontainers

Compatibility Testing

Migrating from gcloud emulators

Image Tags

Configuration

Multi-container Docker Compose

Community

Star History

Contributors

License