Skip to content

Commit e218e11

Browse files
guenth39guenth39
committed
Add option to map fieldnames to different typesearch fieldnames
1 parent 44e01a7 commit e218e11

File tree

5 files changed

+38
-20
lines changed

5 files changed

+38
-20
lines changed

‎README.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -48,15 +48,15 @@ firebase ext:install typesense/firestore-typesense-search --project=[your-projec
4848

4949
When you install this extension, you'll be able to configure the following parameters:
5050

51-
| Parameter | Description |
52-
|-----------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
53-
| Firestore Collection Path | The Firestore collection that needs to be indexed into Typesense. |
54-
| Firestore Collection Fields | A comma separated list of fields that need to be indexed from each Firestore document. Leave blank to index all fields. |
55-
| Flatten Nested Documents | Should nested documents in Firestore be flattened before they are indexed in Typesense? Set to "Yes" for Typesense Server versions v0.23.1 and below, since indexing Nested objects is natively supported only in Typesense Server v0.24 and above. |
56-
| Typesense Hosts | A comma-separated list of Typesense Hosts (only domain without https or port number). For single node clusters, a single hostname is sufficient. For multi-node Highly Available or (Search Delivery Network) SDN Clusters, please be sure to mention all hostnames in a comma-separated list. |
57-
| Typesense API Key | A Typesense API key with admin permissions. Click on "Generate API Key" in cluster dashboard in Typesense Cloud. |
58-
| Typesense Collection Name | Typesense collection name to index data into (you need to create this collection in Typesense yourself. This extension does not create the Typesense Collection for you). |
59-
| Cloud Functions location | Where do you want to deploy the functions created for this extension? You usually want a location close to your database. For help selecting a location, refer to the [location selection guide](https://firebase.google.com/docs/functions/locations). |
51+
| Parameter | Description |
52+
|-----------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
53+
| Firestore Collection Path | The Firestore collection that needs to be indexed into Typesense. |
54+
| Firestore Collection Fields | A comma separated list of fields that need to be indexed from each Firestore document. Leave blank to index all fields. With a "=" separated another field name can be specified for a field, e.g. "field1=otherFieldName1". Thus the firestore field "field1" in typesense is called "otherFieldName1". |
55+
| Flatten Nested Documents | Should nested documents in Firestore be flattened before they are indexed in Typesense? Set to "Yes" for Typesense Server versions v0.23.1 and below, since indexing Nested objects is natively supported only in Typesense Server v0.24 and above. |
56+
| Typesense Hosts | A comma-separated list of Typesense Hosts (only domain without https or port number). For single node clusters, a single hostname is sufficient. For multi-node Highly Available or (Search Delivery Network) SDN Clusters, please be sure to mention all hostnames in a comma-separated list. |
57+
| Typesense API Key | A Typesense API key with admin permissions. Click on "Generate API Key" in cluster dashboard in Typesense Cloud. |
58+
| Typesense Collection Name | Typesense collection name to index data into (you need to create this collection in Typesense yourself. This extension does not create the Typesense Collection for you). |
59+
| Cloud Functions location | Where do you want to deploy the functions created for this extension? You usually want a location close to your database. For help selecting a location, refer to the [location selection guide](https://firebase.google.com/docs/functions/locations). |
6060

6161
> ⚠️ You'll notice that there is no way to configure the port number or protocol.
6262
This is because this extension only supports connecting to Typesense running HTTPS on Port 443, since your data goes from Firebase to Typesense over the public internet and we want your data to be encrypted in transit.

‎extension.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -69,8 +69,8 @@ params:
6969
- param: FIRESTORE_COLLECTION_FIELDS
7070
label: Firestore Collection Fields
7171
description: >-
72-
A comma separated list of fields that need to be indexed from each Firestore document. Leave blank to index all fields.
73-
example: field1,field2,field3
72+
A comma separated list of fields that need to be indexed from each Firestore document. Leave blank to index all fields. With a "=" separated another field name can be specified for a field, e.g. "field1=otherFieldName1". Thus the firestore field "field1" in typesense is called "otherFieldName1".
73+
example: field1,field2=otherFieldName2,field3
7474
default: ""
7575
required: false
7676
- param: FLATTEN_NESTED_DOCUMENTS

‎functions/src/backfillToTypesenseFromFirestore.js

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ const validateBackfillRun = (snapshot) => {
3131
module.exports = functions.handler.firestore.document
3232
.onWrite(async (snapshot, context) => {
3333
functions.logger.info("Backfilling " +
34-
`${config.firestoreCollectionFields.join(",")} fields in Firestore documents ` +
34+
`${Array.from(config.firestoreCollectionFields.keys()).join(",")} fields in Firestore documents ` +
3535
`from ${config.firestoreCollectionPath} ` +
3636
`into Typesense Collection ${config.typesenseCollectionName} ` +
3737
`on ${config.typesenseHosts.join(",")}`);

‎functions/src/config.js

Lines changed: 22 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,27 @@
1+
/**
2+
* @param {string} fieldNames
3+
* @return {Map} map of field names where key is the field name in Firestore and value is the field name in Typesense
4+
* @example
5+
* parseFieldNames("foo=bar, baz") => Map { "foo" => "bar", "baz" => "baz" }
6+
* parseFieldNames("foo=bar, baz=qux") => Map { "foo" => "bar", "baz" => "qux" }
7+
* parseFieldNames("foo=bar, baz=qux,") => Map { "foo" => "bar", "baz" => "qux" }
8+
* parseFieldNames("foo, baz = qux, bar , ") => Map { "foo" => "foo", "baz" => "qux", "bar" => "bar" }
9+
*/
10+
const parseFieldNames = (fieldNames) => new Map(
11+
fieldNames.split(",")
12+
.filter((v) => v)
13+
.map(
14+
(f) => {
15+
const [key, value = key] = f.split("=").map((p) => p.trim());
16+
return [key, value];
17+
},
18+
),
19+
);
20+
121
module.exports = {
222
firestoreCollectionPath: process.env.FIRESTORE_COLLECTION_PATH,
323
firestoreCollectionFields:
4-
(process.env.FIRESTORE_COLLECTION_FIELDS || "")
5-
.split(",")
6-
.map((f) => f.trim())
7-
.filter((f) => f),
24+
parseFieldNames(process.env.FIRESTORE_COLLECTION_FIELDS || ""),
825
shouldFlattenNestedDocuments: process.env.FLATTEN_NESTED_DOCUMENTS === "true",
926
typesenseHosts:
1027
process.env.TYPESENSE_HOSTS.split(",").map((e) => e.trim()),
@@ -15,3 +32,4 @@ module.exports = {
1532
typesenseBackfillTriggerDocumentInFirestore: "typesense_sync/backfill",
1633
typesenseBackfillBatchSize: 1000,
1734
};
35+

‎functions/src/utils.js

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ const mapValue = (value) => {
2222

2323
/**
2424
* @param {DocumentSnapshot} firestoreDocumentSnapshot
25-
* @param {Array} fieldsToExtract
25+
* @param {Map<string,string>} fieldsToExtract
2626
* @return {Object} typesenseDocument
2727
*/
2828
exports.typesenseDocumentFromSnapshot = (
@@ -33,12 +33,12 @@ exports.typesenseDocumentFromSnapshot = (
3333

3434
let entries = Object.entries(data);
3535

36-
if (fieldsToExtract.length) {
37-
entries = entries.filter(([key]) => fieldsToExtract.includes(key));
36+
if (fieldsToExtract.size) {
37+
entries = entries.filter(([key]) => fieldsToExtract.has(key));
3838
}
3939

4040
// Build a document with just the fields requested by the user, and mapped from Firestore types to Typesense types
41-
const mappedDocument = Object.fromEntries(entries.map(([key, value]) => [key, mapValue(value)]));
41+
const mappedDocument = Object.fromEntries(entries.map(([key, value]) => [fieldsToExtract.get(key), mapValue(value)]));
4242

4343
// using flat to flatten nested objects for older versions of Typesense that did not support nested fields
4444
// https://typesense.org/docs/0.22.2/api/collections.html#indexing-nested-fields

0 commit comments

Comments
 (0)