-
-
Notifications
You must be signed in to change notification settings - Fork 51
Open
Labels
Description
Hi,
I'm trying to create a custom streamer to parse the following type of file
{
"attr1": [
{"a": "data", "id": 1 },
{"a": "data", "id": 2 }
],
"attr2": [
{"b": "data", "id": 1 },
{"b": "data", "id": 2 }
]
}The way I would like it to be chucked is by each element of each array. I understand that streamArray() can do this. However, I also would like to keep the key updated and not as a counter.
Desire output
{key: "attr1", value: {a: "data", id: 1}}
{key: "attr1", value: {a: "data", id: 2}}
{key: "attr2", value: {b: "data", id: 1}}
{key: "attr2", value: {b: "data", id: 2}}
I came up with this solution
class StreamObjectArrays extends StreamBase {
private _level: number;
private _lastKey: string | null;
static make(options?: StreamBase.StreamOptions) {
return new StreamObjectArrays(options);
}
static withParser(options: FilterOptions) {
return withParser(StreamObjectArrays.make, options);
}
constructor(options?: StreamBase.StreamOptions) {
super(options);
this._level = 2;
this._lastKey = null;
}
_wait(chunk: any, _: BufferEncoding, callback: TransformCallback) {
if (chunk.name !== 'startObject') {
return callback(new Error('Top-level object should be an object.'));
}
this._transform = this._filter;
return this._transform(chunk, _, callback);
}
_push(discard: boolean) {
if (
this._assembler.stack.length === 2 &&
this._assembler.stack[1] !== this._lastKey
) {
this._lastKey = String(this._assembler.stack[1]);
}
_push(discard: boolean) {
if (
this._assembler.stack.length === 2 &&
this._assembler.stack[1] !== this._lastKey
) {
this._lastKey = String(this._assembler.stack[1]);
}
if (this._lastKey !== null) {
if (this._assembler.current.length) {
this.push({
key: this._lastKey,
value: this._assembler.current.pop(),
});
}
}
}
}I had a previous version that was getting the key from _assembler.key in the same way as done in streamObject and updating the level. But I was unable to find the right way to reset the level and the key as once set to level 2 I was never able to get back the top level key from assembler anymore.
if (this._lastKey === null) {
this._lastKey = this._assembler.key;
if (this._assembler.key !== null) {
this._level = 2;
}
} else {
...
}I feel unsettle of using the assembler stack to retrieve the parent key. I wonder if their is not a better & cleaner solution instead
jamesaspence