Skip to content

Commit eec204f

Browse files
guenth39guenth39
committed
Add option to map fieldnames to different typesearch fieldnames
1 parent 44e01a7 commit eec204f

File tree

5 files changed

+39
-20
lines changed

5 files changed

+39
-20
lines changed

‎README.md

+9-9
Original file line numberDiff line numberDiff line change
@@ -48,15 +48,15 @@ firebase ext:install typesense/firestore-typesense-search --project=[your-projec
4848

4949
When you install this extension, you'll be able to configure the following parameters:
5050

51-
| Parameter | Description |
52-
|-----------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
53-
| Firestore Collection Path | The Firestore collection that needs to be indexed into Typesense. |
54-
| Firestore Collection Fields | A comma separated list of fields that need to be indexed from each Firestore document. Leave blank to index all fields. |
55-
| Flatten Nested Documents | Should nested documents in Firestore be flattened before they are indexed in Typesense? Set to "Yes" for Typesense Server versions v0.23.1 and below, since indexing Nested objects is natively supported only in Typesense Server v0.24 and above. |
56-
| Typesense Hosts | A comma-separated list of Typesense Hosts (only domain without https or port number). For single node clusters, a single hostname is sufficient. For multi-node Highly Available or (Search Delivery Network) SDN Clusters, please be sure to mention all hostnames in a comma-separated list. |
57-
| Typesense API Key | A Typesense API key with admin permissions. Click on "Generate API Key" in cluster dashboard in Typesense Cloud. |
58-
| Typesense Collection Name | Typesense collection name to index data into (you need to create this collection in Typesense yourself. This extension does not create the Typesense Collection for you). |
59-
| Cloud Functions location | Where do you want to deploy the functions created for this extension? You usually want a location close to your database. For help selecting a location, refer to the [location selection guide](https://firebase.google.com/docs/functions/locations). |
51+
| Parameter | Description |
52+
|-----------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
53+
| Firestore Collection Path | The Firestore collection that needs to be indexed into Typesense. |
54+
| Firestore Collection Fields | A comma separated list of fields that need to be indexed from each Firestore document. Leave blank to index all fields. With a "=" separated another field name can be specified for a field, e.g. "field1=otherFieldName1". Thus the firestore field "field1" in typesense is called "otherFieldName1". |
55+
| Flatten Nested Documents | Should nested documents in Firestore be flattened before they are indexed in Typesense? Set to "Yes" for Typesense Server versions v0.23.1 and below, since indexing Nested objects is natively supported only in Typesense Server v0.24 and above. |
56+
| Typesense Hosts | A comma-separated list of Typesense Hosts (only domain without https or port number). For single node clusters, a single hostname is sufficient. For multi-node Highly Available or (Search Delivery Network) SDN Clusters, please be sure to mention all hostnames in a comma-separated list. |
57+
| Typesense API Key | A Typesense API key with admin permissions. Click on "Generate API Key" in cluster dashboard in Typesense Cloud. |
58+
| Typesense Collection Name | Typesense collection name to index data into (you need to create this collection in Typesense yourself. This extension does not create the Typesense Collection for you). |
59+
| Cloud Functions location | Where do you want to deploy the functions created for this extension? You usually want a location close to your database. For help selecting a location, refer to the [location selection guide](https://firebase.google.com/docs/functions/locations). |
6060

6161
> ⚠️ You'll notice that there is no way to configure the port number or protocol.
6262
This is because this extension only supports connecting to Typesense running HTTPS on Port 443, since your data goes from Firebase to Typesense over the public internet and we want your data to be encrypted in transit.

‎extension.yaml

+2-2
Original file line numberDiff line numberDiff line change
@@ -69,8 +69,8 @@ params:
6969
- param: FIRESTORE_COLLECTION_FIELDS
7070
label: Firestore Collection Fields
7171
description: >-
72-
A comma separated list of fields that need to be indexed from each Firestore document. Leave blank to index all fields.
73-
example: field1,field2,field3
72+
A comma separated list of fields that need to be indexed from each Firestore document. Leave blank to index all fields. With a "=" separated another field name can be specified for a field, e.g. "field1=otherFieldName1". Thus the firestore field "field1" in typesense is called "otherFieldName1".
73+
example: field1,field2=otherFieldName2,field3
7474
default: ""
7575
required: false
7676
- param: FLATTEN_NESTED_DOCUMENTS

‎functions/src/backfillToTypesenseFromFirestore.js

+1-1
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ const validateBackfillRun = (snapshot) => {
3131
module.exports = functions.handler.firestore.document
3232
.onWrite(async (snapshot, context) => {
3333
functions.logger.info("Backfilling " +
34-
`${config.firestoreCollectionFields.join(",")} fields in Firestore documents ` +
34+
`${Array.from(config.firestoreCollectionFields.keys()).join(",")} fields in Firestore documents ` +
3535
`from ${config.firestoreCollectionPath} ` +
3636
`into Typesense Collection ${config.typesenseCollectionName} ` +
3737
`on ${config.typesenseHosts.join(",")}`);

‎functions/src/config.js

+3-4
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,9 @@
1+
const {parseFieldNames} = require("./utils");
2+
13
module.exports = {
24
firestoreCollectionPath: process.env.FIRESTORE_COLLECTION_PATH,
35
firestoreCollectionFields:
4-
(process.env.FIRESTORE_COLLECTION_FIELDS || "")
5-
.split(",")
6-
.map((f) => f.trim())
7-
.filter((f) => f),
6+
parseFieldNames(process.env.FIRESTORE_COLLECTION_FIELDS || ""),
87
shouldFlattenNestedDocuments: process.env.FLATTEN_NESTED_DOCUMENTS === "true",
98
typesenseHosts:
109
process.env.TYPESENSE_HOSTS.split(",").map((e) => e.trim()),

‎functions/src/utils.js

+24-4
Original file line numberDiff line numberDiff line change
@@ -20,9 +20,29 @@ const mapValue = (value) => {
2020
}
2121
};
2222

23+
/**
24+
* @param {string} fieldNames
25+
* @return {Map} map of field names where key is the field name in Firestore and value is the field name in Typesense
26+
* @example
27+
* parseFieldNames("foo=bar, baz") => Map { "foo" => "bar", "baz" => "baz" }
28+
* parseFieldNames("foo=bar, baz=qux") => Map { "foo" => "bar", "baz" => "qux" }
29+
* parseFieldNames("foo=bar, baz=qux,") => Map { "foo" => "bar", "baz" => "qux" }
30+
* parseFieldNames("foo, baz = qux, bar , ") => Map { "foo" => "foo", "baz" => "qux", "bar" => "bar" }
31+
*/
32+
exports.parseFieldNames = (fieldNames) => new Map(
33+
fieldNames.split(",")
34+
.filter((v) => v)
35+
.map(
36+
(f) => {
37+
const [key, value = key] = f.split("=").map((p) => p.trim());
38+
return [key, value];
39+
},
40+
),
41+
);
42+
2343
/**
2444
* @param {DocumentSnapshot} firestoreDocumentSnapshot
25-
* @param {Array} fieldsToExtract
45+
* @param {Map<string,string>} fieldsToExtract
2646
* @return {Object} typesenseDocument
2747
*/
2848
exports.typesenseDocumentFromSnapshot = (
@@ -33,12 +53,12 @@ exports.typesenseDocumentFromSnapshot = (
3353

3454
let entries = Object.entries(data);
3555

36-
if (fieldsToExtract.length) {
37-
entries = entries.filter(([key]) => fieldsToExtract.includes(key));
56+
if (fieldsToExtract.size) {
57+
entries = entries.filter(([key]) => fieldsToExtract.has(key));
3858
}
3959

4060
// Build a document with just the fields requested by the user, and mapped from Firestore types to Typesense types
41-
const mappedDocument = Object.fromEntries(entries.map(([key, value]) => [key, mapValue(value)]));
61+
const mappedDocument = Object.fromEntries(entries.map(([key, value]) => [fieldsToExtract.get(key), mapValue(value)]));
4262

4363
// using flat to flatten nested objects for older versions of Typesense that did not support nested fields
4464
// https://typesense.org/docs/0.22.2/api/collections.html#indexing-nested-fields

0 commit comments

Comments
 (0)