Skip to content

STORAGE_EMULATOR_HOST omits /storage/v1, causes reading to go 404 #2198

Open
Open
STORAGE_EMULATOR_HOST omits /storage/v1, causes reading to go 404#2198

Description

@alexandercerutti
alexandercerutti
opened on Apr 4, 2026

Note

This is not exactly a problem with this emulator but more a description of my findings, since I didn't find > anything else here about this. I really hope this could help other people.

I added a question and proposal at the end, that might overcome this issue.

I'm building by integration of a system that uses GCS, so of course for development I needed to use an emulator.

I have a Docker Compose file that runs my app and the fake emulator with something like the following settings:

  dev:
    profiles: [dev]
    env_file: .env.dev
    image: name-x
    build: .
    ports:
      - '${APP_SERVICE_PORT}:${APP_SERVICE_PORT}'
    environment:
      APP_SERVICE_PORT: ${APP_SERVICE_PORT}
      GCS_BUCKET: ${GCS_BUCKET}
      STORAGE_EMULATOR_HOST: http://${GCS_EMULATOR_HOST}:${GCS_EMULATOR_PORT}
    depends_on:
      - gcs-emulator
    command: node dist/index.js

  gcs-emulator:
    profiles: [dev]
    image: fsouza/fake-gcs-server
    command: -scheme http -public-host ${GCS_EMULATOR_HOST}:${GCS_EMULATOR_PORT} -filesystem-root /data
    volumes:
      - ./gcs-emulator-data:/data
    ports:
      - '${GCS_EMULATOR_PORT}:${GCS_EMULATOR_PORT}' # just to allow myself to make requests through Insomia  or CURL
    expose:
      - '${GCS_EMULATOR_PORT}'

While trying to query the emulator through JS's @google-cloud/storage, I used this code, by relying on the fact that new Storage() assumes (or tries to use) STORAGE_EMULATOR_HOST environment variable (as described by GCS Go package Documentation and by another page I cannot find anymore).

	const storage = new Storage();

	const bucket = storage.bucket(process.env.GCS_BUCKET);
	const file = bucket.file(filePath);

	await file.save(data, options);

	console.log(
		await storage.bucket(process.env.GCS_BUCKET).getFiles({
			prefix: `my-prefix/${uuid}/`,
		}),
	);

If STORAGE_EMULATOR_HOST is set, upload works fine but getFiles fails with logs like:

gcs-emulator-1  | time=2026-04-04T13:44:20.159Z level=INFO msg="172.18.0.4 - - [04/Apr/2026:13:44:19 +0000] \"POST /upload/storage/v1/b/my-bucket/o?uploadType=multipart&name=my-prefix%2F377a1644-fd59-41bf-9f61-cd3a4359d6fa%2Ffile.txt HTTP/1.1\" 200 1055\n"
gcs-emulator-1  | time=2026-04-04T13:44:20.161Z level=INFO msg="172.18.0.4 - - [04/Apr/2026:13:44:20 +0000] \"GET /b/my-bucket/o?prefix=my-prefix%2F377a1644-fd59-41bf-9f61-cd3a4359d6fa%2F HTTP/1.1\" 404 10\n"

Any request through CURL to http://0.0.0.0:4443/storage/v1/b/my-bucket/o works fine.
So I started digging in the package @google-cloud/storage and then in the issues:

Apparently, STORAGE_EMULATOR_HOST is used in a weird way:

https://github.com/googleapis/nodejs-storage/blob/189663a279d85451a65614b47a748d667d7eb3db/src/storage.ts#L726-L746

// Note: EMULATOR_HOST is an experimental configuration variable. Use apiEndpoint instead.
const EMULATOR_HOST = process.env.STORAGE_EMULATOR_HOST;
if (typeof EMULATOR_HOST === 'string') {
    apiEndpoint = Storage.sanitizeEndpoint(EMULATOR_HOST);
    customEndpoint = true;
}
if (options.apiEndpoint && options.apiEndpoint !== apiEndpoint) {
    apiEndpoint = Storage.sanitizeEndpoint(options.apiEndpoint);
    customEndpoint = true;
}
options = Object.assign({}, options, { apiEndpoint });
// Note: EMULATOR_HOST is an experimental configuration variable. Use apiEndpoint instead.
const baseUrl = EMULATOR_HOST || `${options.apiEndpoint}/storage/v1`;

As you can see, when apiEndpoint is declared, it gets appended /storage/v1 while STORAGE_EMULATOR_HOST doesn't... and you can't even add it yourself, because otherwise we get upload request errors like:

gcs-emulator-1  | time=2026-04-04T13:44:59.993Z level=INFO msg="172.18.0.4 - - [04/Apr/2026:13:44:59 +0000] \"POST /storage/v1/upload/storage/v1/b/my-bucket/o?uploadType=multipart&name=my-prefix%2F377a1644-fd59-41bf-9f61-cd3a4359d6fa%2Ffile.txt HTTP/1.1\" 404 119\n"

Digging deeper, I found out some discussions about this: googleapis/nodejs-storage#2069 and linked threads.

Apparently, there is not an exact consensus about STORAGE_EMULATOR_HOST, which should have been only an experimental thing... which got then documented officially in the Go Client (as linked above).

Then, I looked, in the examples of this repository. Both STORAGE_EMULATOR_HOST AND apiEndpoint are deliberately used, probably to overcome this issue (the reason is not explained).

However, using STORAGE_EMULATOR_HOST alone would have been really convenient, in order to not mix in the code things for development.

Instead, we can simply use apiEndpoint instead, even if that mixes code with development details.

const storage = new Storage({
	apiEndpoint: process.env.GCS_EMULATOR_ENDPOINT,
});

I really hope this may help someone else, hitting the wall like me.
I'm not sure is there anything actionable in this project could be done to fix this.

I wonder if adding another route listener to serve /b/my-bucket could actually help for reading files. Like, in this file:

fake-gcs-server/fakestorage/server.go

Lines 253 to 340 in 6f935ce

func (s *Server) buildMuxer() {
const apiPrefix = "/storage/v1"
handler := mux.NewRouter().SkipClean(true).UseEncodedPath()
// healthcheck
handler.Path("/_internal/healthcheck").Methods(http.MethodGet).HandlerFunc(s.healthcheck)
routers := []*mux.Router{
handler.PathPrefix(apiPrefix).Subrouter(),
handler.MatcherFunc(s.publicHostMatcher).PathPrefix(apiPrefix).Subrouter(),
}
for _, r := range routers {
r.Path("/b").Methods(http.MethodGet).HandlerFunc(jsonToHTTPHandler(s.listBuckets))
r.Path("/b/").Methods(http.MethodGet).HandlerFunc(jsonToHTTPHandler(s.listBuckets))
r.Path("/b").Methods(http.MethodPost).HandlerFunc(jsonToHTTPHandler(s.createBucketByPost))
r.Path("/b/").Methods(http.MethodPost).HandlerFunc(jsonToHTTPHandler(s.createBucketByPost))
r.Path("/b/{bucketName}").Methods(http.MethodGet).HandlerFunc(jsonToHTTPHandler(s.getBucket))
r.Path("/b/{bucketName}").Methods(http.MethodPatch).HandlerFunc(jsonToHTTPHandler(s.updateBucket))
r.Path("/b/{bucketName}").Methods(http.MethodPost).Headers("X-HTTP-Method-Override", "PATCH").HandlerFunc(jsonToHTTPHandler(s.updateBucket))
r.Path("/b/{bucketName}").Methods(http.MethodDelete).HandlerFunc(jsonToHTTPHandler(s.deleteBucket))
r.Path("/b/{bucketName}/o").Methods(http.MethodGet).HandlerFunc(jsonToHTTPHandler(s.listObjects))
r.Path("/b/{bucketName}/o/").Methods(http.MethodGet).HandlerFunc(jsonToHTTPHandler(s.listObjects))
r.Path("/b/{bucketName}/o/{objectName:.+}").Methods(http.MethodPatch).HandlerFunc(jsonToHTTPHandler(s.patchObject))
r.Path("/b/{bucketName}/o/{objectName:.+}/acl").Methods(http.MethodGet).HandlerFunc(jsonToHTTPHandler(s.listObjectACL))
r.Path("/b/{bucketName}/o/{objectName:.+}/acl").Methods(http.MethodPost).HandlerFunc(jsonToHTTPHandler(s.setObjectACL))
r.Path("/b/{bucketName}/o/{objectName:.+}/acl/{entity}").Methods(http.MethodDelete).HandlerFunc(jsonToHTTPHandler(s.deleteObjectACL))
r.Path("/b/{bucketName}/o/{objectName:.+}/acl/{entity}").Methods(http.MethodGet).HandlerFunc(jsonToHTTPHandler(s.getObjectACL))
r.Path("/b/{bucketName}/o/{objectName:.+}/acl/{entity}").Methods(http.MethodPut).HandlerFunc(jsonToHTTPHandler(s.setObjectACL))
r.Path("/b/{bucketName}/o/{objectName:.+}").Methods(http.MethodGet, http.MethodHead).HandlerFunc(s.getObject)
r.Path("/b/{bucketName}/o/{objectName:.+}").Methods(http.MethodDelete).HandlerFunc(jsonToHTTPHandler(s.deleteObject))
r.Path("/b/{sourceBucket}/o/{sourceObject:.+}/{copyType:rewriteTo|copyTo}/b/{destinationBucket}/o/{destinationObject:.+}").Methods(http.MethodPost).HandlerFunc(jsonToHTTPHandler(s.rewriteObject))
r.Path("/b/{bucketName}/o/{destinationObject:.+}/compose").Methods(http.MethodPost).HandlerFunc(jsonToHTTPHandler(s.composeObject))
r.Path("/b/{bucketName}/o/{objectName:.+}").Methods(http.MethodPut, http.MethodPost).HandlerFunc(jsonToHTTPHandler(s.updateObject))
r.Path("/b/{bucketName}/notificationConfigs").Methods(http.MethodPost).HandlerFunc(jsonToHTTPHandler(s.insertNotification))
r.Path("/b/{bucketName}/notificationConfigs").Methods(http.MethodGet).HandlerFunc(jsonToHTTPHandler(s.listNotifications))
r.Path("/b/{bucketName}/notificationConfigs/{notificationId}").Methods(http.MethodGet).HandlerFunc(jsonToHTTPHandler(s.getNotification))
r.Path("/b/{bucketName}/notificationConfigs/{notificationId}").Methods(http.MethodDelete).HandlerFunc(jsonToHTTPHandler(s.deleteNotification))
}
// Internal / update server configuration
handler.Path("/_internal/config").Methods(http.MethodPut).HandlerFunc(jsonToHTTPHandler(s.updateServerConfig))
handler.MatcherFunc(s.publicHostMatcher).Path("/_internal/config").Methods(http.MethodPut).HandlerFunc(jsonToHTTPHandler(s.updateServerConfig))
handler.Path("/_internal/reseed").Methods(http.MethodPut, http.MethodPost).HandlerFunc(jsonToHTTPHandler(s.reseedServer))
handler.Path("/_internal/delete_all").Methods(http.MethodPost).HandlerFunc(jsonToHTTPHandler(s.deleteAllFiles))
// Internal - end
// XML API
xmlApiRouters := []*mux.Router{
handler.Host(fmt.Sprintf("{bucketName}.%s", s.publicHost)).Subrouter(),
handler.MatcherFunc(s.publicHostMatcher).PathPrefix(`/{bucketName}`).Subrouter(),
}
for _, r := range xmlApiRouters {
r.Path("/").Methods(http.MethodGet).HandlerFunc(xmlToHTTPHandler(s.xmlListObjects))
r.Path("").Methods(http.MethodGet).HandlerFunc(xmlToHTTPHandler(s.xmlListObjects))
}
bucketHost := fmt.Sprintf("{bucketName}.%s", s.publicHost)
handler.Host(bucketHost).Path("/{objectName:.+}").Methods(http.MethodGet, http.MethodHead).HandlerFunc(s.downloadObject)
handler.Path("/download/storage/v1/b/{bucketName}/o/{objectName:.+}").Methods(http.MethodGet, http.MethodHead).HandlerFunc(s.downloadObject)
handler.Path("/upload/storage/v1/b/{bucketName}/o").Methods(http.MethodPost).HandlerFunc(jsonToHTTPHandler(s.insertObject))
handler.Path("/upload/storage/v1/b/{bucketName}/o/").Methods(http.MethodPost).HandlerFunc(jsonToHTTPHandler(s.insertObject))
handler.Path("/upload/storage/v1/b/{bucketName}/o").Methods(http.MethodPut).HandlerFunc(jsonToHTTPHandler(s.uploadFileContent))
handler.Path("/upload/storage/v1/b/{bucketName}/o/").Methods(http.MethodPut).HandlerFunc(jsonToHTTPHandler(s.uploadFileContent))
handler.Path("/upload/storage/v1/b/{bucketName}/o").Methods(http.MethodDelete).HandlerFunc(jsonToHTTPHandler(s.deleteResumableUpload))
handler.Path("/upload/storage/v1/b/{bucketName}/o/").Methods(http.MethodDelete).HandlerFunc(jsonToHTTPHandler(s.deleteResumableUpload))
handler.Path("/upload/resumable/{uploadId}").Methods(http.MethodPut, http.MethodPost).HandlerFunc(jsonToHTTPHandler(s.uploadFileContent))
// Batch endpoint
handler.MatcherFunc(s.publicHostMatcher).Path("/batch/storage/v1").Methods(http.MethodPost).HandlerFunc(s.handleBatchCall)
handler.Path("/batch/storage/v1").Methods(http.MethodPost).HandlerFunc(s.handleBatchCall)
handler.MatcherFunc(s.publicHostMatcher).Path("/{bucketName}/{objectName:.+}").Methods(http.MethodGet, http.MethodHead).HandlerFunc(s.downloadObject)
handler.Host("{bucketName:.+}").Path("/{objectName:.+}").Methods(http.MethodGet, http.MethodHead).HandlerFunc(s.downloadObject)
// Form Uploads
handler.Host(s.publicHost).Path("/{bucketName}").MatcherFunc(matchFormData).Methods(http.MethodPost, http.MethodPut).HandlerFunc(xmlToHTTPHandler(s.insertFormObject))
handler.Host(bucketHost).MatcherFunc(matchFormData).Methods(http.MethodPost, http.MethodPut).HandlerFunc(xmlToHTTPHandler(s.insertFormObject))
// Signed URLs (upload and download)
handler.MatcherFunc(s.publicHostMatcher).Path("/{bucketName}/{objectName:.+}").Methods(http.MethodPost, http.MethodPut).HandlerFunc(jsonToHTTPHandler(s.insertObject))
handler.MatcherFunc(s.publicHostMatcher).Path("/{bucketName}/{objectName:.+}").Methods(http.MethodGet, http.MethodHead).HandlerFunc(s.getObject)
handler.MatcherFunc(s.publicHostMatcher).Path("/{bucketName}/{objectName:.+}").Methods(http.MethodDelete).HandlerFunc(jsonToHTTPHandler(s.deleteObject))
handler.Host(bucketHost).Path("/{objectName:.+}").Methods(http.MethodPost, http.MethodPut).HandlerFunc(jsonToHTTPHandler(s.insertObject))
handler.Host("{bucketName:.+}").Path("/{objectName:.+}").Methods(http.MethodPost, http.MethodPut).HandlerFunc(jsonToHTTPHandler(s.insertObject))
s.handler = handler
} 

  jsonAPIRouter := r.PathPrefix("/storage/v1").Subrouter()
+ jsonAPIAliasRouter := r.PathPrefix("").Subrouter()

and this one (which may apply to all the routes).

  jsonAPIRouter.Path("/b/{bucketName}/o/{objectName:.+}").Methods(http.MethodGet).HandlerFunc(s.downloadObject)
+ jsonAPIAliasRouter.Path("/b/{bucketName}/o/{objectName:.+}").Methods(http.MethodGet).HandlerFunc(s.downloadObject)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions