-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathindex.html
1188 lines (1140 loc) · 67.5 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="x-ua-compatible" content="ie-edge">
<title>DataWorker</title>
<meta name="description" content="DataWorker documentation">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" href="https://cdn.rawgit.com/Daemonite/material/maintenance/css/base.min.css">
<link rel="stylesheet" href="dw.css">
</head>
<body class="page-brand" data-spy="scroll" data-target="#side-nav">
<header class="header header-standard header-brand">
<ul class="nav nav-list pull-left hidden-md hidden-lg">
<li>
<a data-toggle="menu" href="#side-nav">
<i class="icon icon-lg">menu</i>
</a>
</li>
</ul>
<span class="header-logo margin-left-no">DataWorker</span>
<ul class="nav nav-list pull-right">
<li>
<a class="waves-attach" href="https://github.com/dataworker/dataworker"
tooltip bottom left alt="View code on GitHub">
<i class="icon icon-lg">code</i>
</a>
</li>
</ul>
</header>
<nav class="menu menu-left nav-drawer nav-drawer-md" id="side-nav">
<div class="menu-scroll">
<div class="menu-content">
<ul class="nav">
<li><a class="waves-attach menu-logo" href="#top">DataWorker</a></li>
<li>
<a class="waves-attach" data-toggle="collapse" href="#get-started">Get started!</a>
<ul class="menu-collapse collapse" id="get-started" aria-expanded="false">
<li><a class="waves-attach" href="#local-data">Local Data</a></li>
<li><a class="waves-attach" href="#stream">Streaming Data</a></li>
<li><a class="waves-attach" href="#webworkers">Web Workers</a></li>
<li><a class="waves-attach" href="#clone">Clone</a></li>
<li><a class="waves-attach" href="#child-rows">Child Rows</a></li>
<li><a class="waves-attach" href="#display-values">Display Values</a></li>
</ul>
</li>
<li>
<a class="waves-attach" data-toggle="collapse" href="#manipulate-dataset">Manipulate the dataset!</a>
<ul class="menu-collapse collapse" id="manipulate-dataset" aria-expanded="false">
<li><a class="waves-attach" href="#alter-columns">Alter columns</a></li>
<li><a class="waves-attach" href="#append">Append</a></li>
<li><a class="waves-attach" href="#filter">Filter</a></li>
<li><a class="waves-attach" href="#search">Search</a></li>
<li><a class="waves-attach" href="#group">Group</a></li>
<li><a class="waves-attach" href="#join">Join</a></li>
<li><a class="waves-attach" href="#limit">Limit</a></li>
<li><a class="waves-attach" href="#remove-columns">Remove columns</a></li>
<li><a class="waves-attach" href="#hide-columns">Hide columns</a></li>
<li><a class="waves-attach" href="#clear-dataset">Clear Dataset</a></li>
<li><a class="waves-attach" href="#sort">Sort</a></li>
<li><a class="waves-attach" href="#add-child-rows">Add Child Rows</a></li>
</ul>
</li>
<li>
<a class="waves-attach" data-toggle="collapse" href="#access-dataset">Access the dataset!</a>
<ul class="menu-collapse collapse" id="access-dataset" aria-expanded="false">
<li><a class="waves-attach" href="#get-rows">Get rows</a></li>
<li><a class="waves-attach" href="#get-hashed-rows">Get hashed rows</a></li>
<li><a class="waves-attach" href="#get-columns">Get columns</a></li>
<li><a class="waves-attach" href="#get-columns-and-records">Get columns and records</a></li>
<li><a class="waves-attach" href="#get-number-of-records">Get number of records</a></li>
<li><a class="waves-attach" href="#get-distinct">Get distinct</a></li>
<li><a class="waves-attach" href="#pagination">Pagination</a></li>
<li><a class="waves-attach" href="#partition">Partition</a></li>
<li><a class="waves-attach" href="#render">Render</a></li>
</ul>
</li>
<li>
<a class="waves-attach" data-toggle="collapse" href="#contribute">Contribute!</a>
<ul class="menu-collapse collapse" id="contribute" aria-expanded="false">
<li><a class="waves-attach" href="#compile">Compile Code</a></li>
</ul>
</li>
</ul>
</div>
</div>
</nav>
<main class="content">
<div class="container">
<div class="col-lg-8 col-lg-offset-2 col-sm-10 col-sm-offset-1">
<section id="top">
<div class="card">
<div class="card-main">
<div class="card-inner">
<p>DataWorker is designed to manage large datasets client-side, using modern web technologies such as Web Workers and WebSockets. DataWorker allows users to perform expensive operations (such as sorting, joining, searching, filtering, grouping, and much more) in a multi-threaded environment to maintain responsiveness. Data can be provided locally as an array, or remotely via AJAX or WebSockets. DataWorker has no dependencies on external libraries, but works well with others.</p>
<p>Not all browsers fully support all of the provided features (such as using a WebSocket from within a Web Worker) because some of the technology is quite new, so fallbacks are in place to make sure data can still be accessed across the most amount of devices possible.</p>
<p>To get started, you will need to include one of the distribution files in the <code>dist/</code> folder: <code>dataworker.js</code> for development/testing, and <code>dataworker.min.js</code> for production. For example: </p>
<pre>
<code><script src="dist/dataworker.min.js"></script></code></pre>
</div>
</div>
</div>
</section>
<section id="local-data">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Local Data</h3>
<p>DataWorker takes an array of records to construct the initial dataset. The first record defines column properties.</p>
<pre>
<code>var dataset = [
[
{
name: "column_a",
title: "Column A",
aggType: "max",
sortType: "alpha"
},
{
name: "column_b",
title: "Column B",
aggType: "max",
sortType: "alpha"
},
{
name: "column_c",
title: "Column C",
aggType: "min",
sortType: "alpha"
}
],
[ "apple", "violin", "music" ],
[ "cat", "tissue", "dog" ],
[ "banana", "piano", "gum" ],
[ "gummy", "power", "star" ]
];
var dw = new DataWorker(dataset);</code></pre>
<p>It is also possible to just supply column names:</P>
<pre>
<code>var dataset = [
[ "column_a", "column_b", "column_c" ],
[ "apple", "violin", "music" ],
[ "cat", "tissue", "dog" ],
[ "banana", "piano", "gum" ],
[ "gummy", "power", "star" ]
];
var dw = new DataWorker(dataset);</code></pre>
<p>If only column names are supplied, the following default column properties are used:</p>
<ul>
<li><code>title</code>: <code>columnName</code></li>
<li><code>aggType</code>: "max"</li>
<li><code>sortType</code>: "alpha"</li>
</ul>
<p>Additionally, you should give DataWorker an error handler (by default DataWorker uses <code>console.error</code>) using the <code>onError</code> method or by adding it as a property on the dataset object:</p>
<pre>
<code>dataset.onError = function (msg) { alert(msg) };
new DataWorker(dataset);</code></pre>
<p>or</p>
<pre>
<code>var dw = new DataWorker(dataset).onError(function (errorMsg) {
alert(errorMsg);
});</code></pre>
<h5 class="content-sub-heading card-sub-heading">Error handling</h5>
<p>DataWorker can be provided with an error handler:</p>
<pre>
<code>dw.onError(function (msg) {
alert(msg);
});</code></pre>
<p>By default, errors are printed to the console.</p>
</div>
</div>
</div>
</section>
<section id="stream">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Streaming Data</h3>
<h5 class="content-sub-heading card-sub-heading">Via Websockets</h5>
<p>Alternatively, DataWorker can stream data from a WebSocket server:</p>
<pre>
<code>var dw = new DataWorker({
datasource : "ws://127.0.0.1:8888",
authenticate : "{}",
request : "{\"cmd\":\"requestDataset\"}"
});</code></pre>
<p><code>datasource</code> should be the URL of the websocket server.</p>
<p><code>authenticate</code> is optional. If present, will be the first message sent to the server upon connecting.</p>
<p><code>request</code> is sent after <code>authenticate</code>.<p>
<p>After the request is sent, DataWorker will expect a reply in the form of a JSON object with <code>expectedNumRows</code> and <code>columns</code> properties such as:</p>
<pre>
<code>{
expectedNumRows : 10,
columns : [ "column_a", "column_b", "column_c" ]
}</code></pre>
<p><code>expectedNumRows</code> should be the number of rows to expect in that dataset.</p>
<p><code>columns</code> should be the columns of the expected dataset. They can be provided as simple column names or as hashes with custom column properties.</p>
<p>Note DataWorker cannot proceed without first knowing the columns and the total number of expected rows. That being said, these two properties may be transmitted separately. For example:</p>
<pre>
<code>// First message:
{ "expectedNumRows": 10 }
// Second message:
{ columns: [ "column_a", "column_b", "column_c" ] }</code></pre>
<p>Afterwards, DataWorker will expect arrays of dataset rows from the server. These rows will be appended to the dataset. The reply will be similar to the following:</p>
<pre>
<code>[
[ "apple", "violin", "music" ],
[ "cat", "tissue", "dog" ],
[ "banana", "piano", "gum" ],
[ "gummy", "power", "star" ]
]</code></pre>
<p>The <code>onReceiveColumns</code> callback will be called when columns and the expected number of rows are received from the Websocket server. This callback can be set with the <code>onReceiveColumns</code> method:</p>
<pre>
<code>dw.onReceiveColumns(function () {
alert("Columns have been received!");
});</code></pre>
<p>Alternatively, this callback can be passed into the constructor:</p>
<pre>
<code>var dw = new DataWorker({
datasource : "ws://127.0.0.1:8888",
authenticate : "{}",
request : "{\"cmd\":\"requestDataset\"}",
onReceiveColumns : function (numRows) {
alert("Columns have been received!");
}
});</code></pre>
<p>The <code>onReceiveRows</code> callback will be called whenever rows are received from the Websocket server. This callback can be set with the <code>onReceiveRows</code> method:</p>
<pre>
<code>dw.onReceiveRows(function (numRows) {
alert("Received " + numRows + " rows.");
});</code></pre>
<p>Alternatively, this callback can be passed into the constructor:</p>
<pre>
<code>var dw = new DataWorker({
datasource : "ws://127.0.0.1:8888",
authenticate : "{}",
request : "{\"cmd\":\"requestDataset\"}",
onReceiveRows : function (numRows) {
alert(numRows + " rows received.");
}
});</code></pre>
<p>If not set, this callback defaults to doing nothing.</p>
<p>The <code>onAllRowsReceived</code> callback is called when all expected rows have been received from the Websocket server. This callback can be set with the <code>onAllRowsReceived</code> method:</p>
<pre>
<code>dw.onAllRowsReceived(function () {
alert("All rows have been received!");
});</code></pre>
<p>Alternatively, this callback can be passed into the constructor:</p>
<pre>
<code>var dw = new DataWorker({
datasource : "ws://127.0.0.1:8888",
authenticate : "{}",
request : "{\"cmd\":\"requestDataset\"}",
onReceiveRows : function (numRows) {
alert(numRows + " rows received.");
}
onAllRowsReceived : function () {
alert("All rows have been received!");
}
});</code></pre>
<p>If not set, this callback does nothing.</p>
<h5 class="content-sub-heading card-sub-heading">Via AJAX</h5>
<p>If WebSockets are not available, data can be received via AJAX as well:</p>
<pre>
<code>var dw = new DataWorker({
datasource : "http://127.0.0.1:8888",
request : "?this=is;a=query;string",
onAllRowsReceived : function () {
alert("all rows have been received!");
}
});</code></pre>
<p><code>datasource</code> should be the base url to send GET requests to.</p>
<p><code>request</code> should be the query string to append to the base url. The prepended question mark is optional; if it is missing, one will automatically be inserted. Optionally, <code>request</code> could be a javascript object or a JSON string; DataWorker will convert this into a query string.</p>
<h5 class="content-sub-heading card-sub-heading">Using fallbacks</h5>
<p>If multiple data sources exist, it is possible to provide fallbacks when instantiating DataWorker. This can be useful if you have a server down for maintenance or your WebSocket service is not running. Simply provide the possible sources as an array to <code>datasource</code> in priority order:</p>
<pre>
<code>var dw = new DataWorker({
request : { cmd: "requestDataset" },
datasource : [
"https://example.com",
"ws://127.0.0.1:8888",
"http://127.0.0.1:9999"
]
});</code></pre>
<p>The sources will be tried in order. If they all fail, DataWorker will throw an error.</p>
<p>For finer-grain control, authentications can be supplied on a per-datasource basis:</p>
<pre>
<code>var dw = new DataWorker({
request : "\"cmd\":\"requestDataset\"}",
datasource : [
{
source : "ws://127.0.0.1:8888",
authenticate : "zxcv"
},
{
source : "http://127.0.0.1/data-origin",
authenticate : "asdf"
}
]
});</code></pre>
<h5 class="content-sub-heading card-sub-heading">Requesting further data</h5>
<p>A new dataset can be requested using the <code>requestDataset</code> method:</p>
<pre>
<code>// For WebSockets or AJAX:
dw.requestDataset("{\"cmd\":\"requestDataset\"}");
// For AJAX only:
dw.requestDataset("query=string");</code></pre>
<p>The newly-requested dataset will completely replace the previously-requested dataset.</p>
<p>A new dataset can be requested and appended to the current dataset using the <code>requestDatasetForAppend</code> method:</p>
<pre>
<code>// For WebSockets or AJAX:
dw.requestDatasetForAppend("{\"cmd\":\"requestDataset\"}");
// For AJAX only:
dw.requestDatasetForAppend("query=string");</code></pre>
<p>Note that the newly-requested dataset should have the same columns as the previously-requested dataset.</p>
<h5 class="content-sub-heading card-sub-heading">Cancelling ongoing requests</h5>
<p>Ongoing requests can be cancelled with the <code>cancelOnoingRequests</code> method:</p>
<pre>
<code>dw.cancelOngoingRequests();</code></pre>
<p>When <code>requestDataset</code> is called, ongoing requests are automatically cancelled (<code>requestDatasetForAppend</code> does not cancel ongoing requests).</p>
<p>When connected to a Websocket server, DataWorker will send the <code>cancelRequestsCmd</code> message signalling the server to cancel the previous requests if such a message has been provided:</p>
<pre>
<code>var dw = new DataWorker({
datasource: {
source : "ws://127.0.0.1:8888",
cancelRequestsCmd : "CANCEL",
cancelRequestsAck : "ACK_CANCEL"
}
});</code></pre>
<p>If <code>cancelRequestsAck</code> has been provided, DataWorker will wait for the server to acknowledge with a reply that matches <code>cancelRequestsAck</code> exactly before proceeding.
<h5 class="content-sub-heading card-sub-heading">Triggers</h5>
<p>DataWorker can pass messages to and from the remote server via the <code>postMessage</code> method and triggers:</p>
<pre>
<code>dw.onTrigger(function (reply) {
alert(reply);
}).postMessage("MESSAGE TO SERVER");</code></pre>
<p>In the above example, the string "MESSAGE TO SERVER" is sent to the server, and any reply will by shown as an alert. Server replies that are triggers should be stringified JSON objects that look like this:</p>
<pre>
<code>{
trigger: true,
msg: "REPLY FROM SERVER"
}</code></pre>
<p>The <code>onTrigger</code> callback can be passed into the constructor as well:</p>
<pre>
<code>var dw = new DataWorker({
datasource : "ws://127.0.0.1:8888",
authenticate : "{}",
onTrigger: function (reply) { alert(reply) }
});</code></pre>
</div>
</div>
</div>
</section>
<section id="webworkers">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Web Workers</h3>
<p>Web Workers are enabled by default, to get the most performance out of DataWorker. Without any extra configuration, a Blob will be used to create the Web Worker. However, some browsers do not support remote data from within a Web Worker created in that fashion, though they work fine when created using a standard URL. DataWorker will detect these errors and look for a file named <code>dw-helper.js</code> in the same folder as <code>dw.js</code>. THe method used to determine the source path does not work well in some circumstances, for example when a module loader like RequireJS. To provide alternative sources, extra options can be provided when instantiating DataWorker.</p>
<p>If <code>workerSource</code> is provided, it will be used first. If that source fails to create a Web Worker, or an error occurs on connection to the datasource, a Blob will be created next to try again. If the Blob fails, then <code>backupWorkerSource</code> will be used next. If that fails, or wasn't provided, DataWorker will make it's best guess as to the location of <code>dw-helper.js</code>. When all else fails, DataWorker will revert to a single-threaded environment. Finally, an error will be thrown if a connection still cannot be made and no local dataset is provided.</p>
<pre>
<code>new DataWorker({
datasource: "https://example.com/dataset.json",
workerSource: "/path/to/dw-helper.js",
backupWorkerSource: "/other/path/to/dw-helper.js"
});</code></pre>
<p>There is an initial cost to creating a Web Worker, and they are meant to stay alive as long as possible. This can be important for avoiding memory leaks in single page applications. Rather than terminating a Web Worker, DataWorkerHelper is designed to idle when not in use and be reused when needed. To clean up an instance of DataWorker, simply call <code>finish(optionalCallback)</code> on the instance. If a callback is included it will be called once the Web Worker has responded that it is ready to be reused. Simply creating a new DataWorker will reuse that Web Worker.</p>
<p>Web Workers are still an experimental technology, and may have some issues. If you prefer to avoid using Web Workers at all, you may pass in a flag to force it to be single-threaded.</p>
<pre>
<code>new DataWorker({
datasource: "https://example.com/dataset.json",
forceSingleThread: true
});</code></pre>
</div>
</div>
</div>
</section>
<section id="clone">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Clone</h3>
<p>A DataWorker dataset can be deep-copied by calling <code>clone</code>:</p>
<pre>
<code>dw.clone(function (newD) {
newDataset = newD;
});</code></pre>
<p>Note that any streaming datasources are not copied as part of the cloning process.</p>
</div>
</div>
</div>
</section>
<section id="child-rows">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Child Rows</h3>
<p>DataWorker has a concept of child rows. Child rows are rows that are a subset of another row, and may be used, for example, to provide more detail. The functionality is currently limited to a small subset of functions. Where use of child rows is not explicitly defined, they are treated as if they are distinct rows that have been added to the dataset. These are the currently supported functions:</p>
<ul>
<li><a href="#sort">Sort</a></li>
<li><a href="#add-child-rows">Add Child Rows</a></li>
</ul>
</div>
</div>
</div>
</section>
<section id="display-values">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Display Values</h3>
<p>DataWorker can use differing values for display and dataset operations. This enables you to include HTML formatting for cell values but not have to consider them when actually working with the dataset. To take advantage of this feature, simply pass the cell value an object with the <code>display</code> and <code>raw</code> properties defined:</p>
<pre>
<code>var dataset = [
[ "column_a", "column_b", "column_c" ],
[ { display: "<i>apple</i>", raw: "apple" }, "violin", "music" ],
[ "cat", "tissue", "dog" ],
[ "banana", "piano", "gum" ]
];
var d = new DataWorker(dataset);
d.getRows(function (rows) {
// rows is [
// [ "<i>apple</i>", "violin", "music" ],
// [ "cat", "tissue", "dog" ],
// [ "banana", "piano", "gum" ]
// ]
});
d.applyFilter(/^apple^/, "column_a"); // Operates on the raw value
d.getRows(function (rows) {
// rows is [
// [ "<i>apple</i>", "violin", "music" ],
// ]
});
</code></pre>
<p>The <code>display</code> value will be returned by the methods that allow access to the dataset, while the methods that manipulate the dataset will operate on the <code>raw</code> value. Also note that the two methods of specifying cell values can be mixed.</p>
</div>
</div>
</div>
</section>
<section id="alter-columns">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Alter columns</h3>
<p>Column properties can be altered after DataWorker is instantiated.</p>
<h5 class="content-sub-heading card-sub-heading">Alter column name</h5>
<p>To change the name of <code>column_a</code> to <code>column_a1</code>:</p>
<pre>
<code>dw.alterColumnName("column_a", "column_a1");</code></pre>
<h5 class="content-sub-heading card-sub-heading">Alter column title</h5>
<p>To change the title of <code>column_a</code> to <code>Things I Love</code>:</p>
<pre>
<code>dw.alterColumnTitle("column_a", "Things I Love");</code></pre>
<h5 class="content-sub-heading card-sub-heading">Alter column aggregate type</h5>
<p>To change the aggregate type of <code>column_a</code> to <code>min</code>:</p>
<pre>
<code>dw.alterColumnAggregateType("column_a", "min");</code></pre>
<p>Valid aggregate types are:</p>
<ul>
<li><code>max</code></li>
<li><code>min</code></li>
<li><code>sum</code></li>
</ul>
<h5 class="content-sub-heading card-sub-heading">Alter column sort type</h5>
<p>To change the sort type of <code>column_a</code> to <code>num</code>:</p>
<pre>
<code>dw.alterColumnSortType("column_a", "num");</code></pre>
<p>Valid sort types are:</p>
<ul>
<li><code>alpha</code></li>
<li><code>num</code></li>
</ul>
<p>Alternatively, you may pass in a sort function that takes two arguments <code>(a, b)</code> and returns <code>-1</code> for <code>a < b</code>, <code>1</code> for <code> a > b</code> or <code>0</code> for <code>a == b</code>.</p>
<h5 class="content-sub-heading card-sub-heading">Prepend column names</h5>
<p>To prepend <code>a_</code> to all column names:</p>
<pre>
<code>dw.prependColumnNames("a_");</code></pre>
</div>
</div>
</div>
</section>
<section id="append">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Append</h3>
<p>The <code>append</code> method is used to concatenate two datasets together. The following appends <code>dataset2</code> to <code>dataset1</code>:</p>
<pre>
<code>var dataset1 = [
[ "column_a", "column_b", "column_c" ],
[ "apple", "violin", "music" ],
[ "cat", "tissue", "dog" ],
[ "banana", "piano", "gum" ],
];
var dataset2 = [
[ "column_a", "column_b", "column_c" ],
[ "gummy", "power", "apple" ],
[ "car", "screen", "phone" ],
[ "sign", "bagel", "chips" ]
];
var dw = new DataWorker(dataset1);
dw.append(dataset2);</code></pre>
<p>Alternatively, you may also append a DataWorker dataset:</p>
<pre>
<code>var dataset1 = [
[ "column_a", "column_b", "column_c" ],
[ "apple", "violin", "music" ],
[ "cat", "tissue", "dog" ],
[ "banana", "piano", "gum" ]
];
var dataset2 = [
[ "column_a", "column_b", "column_c" ],
[ "gummy", "power", "apple" ],
[ "car", "screen", "phone" ],
[ "sign", "bagel", "chips" ]
];
var d1 = new DataWorker(dataset1);
var d2 = new DataWorker(dataset2);
d1.append(d2);</code></pre>
<p>Note that column names must match up; an error will be thrown otherwise.</p>
</div>
</div>
</div>
</section>
<section id="filter">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Filter</h3>
<p>The <code>applyFilter</code> method is used to filter out rows that do not contain the specified regex. The following filters out any row that does not contain the word "apple":</p>
<pre>
<code>var dataset = [
[ "column_a", "column_b", "column_c" ],
[ "apple", "red", "fuji" ],
[ "apple", "green", "granny smith" ],
[ "apple", "yellow", "golden delicious" ],
[ "banana", "green", "unripe" ],
[ "banana", "yellow", "ripe" ],
[ "banana", "brown", "beyond ripe" ],
[ "banana", "black", "rotten/frozen" ]
], dw = new DataWorker(dataset);
dw.applyFilter(/\bapple\b/)
/* This results in the following rows:
[ "apple", "red", "fuji" ],
[ "apple", "green", "granny smith" ],
[ "apple", "yellow", "golden delicious" ]
*/</code></pre>
<p>You may also filter only on certain columns:</p>
<pre>
<code>dw.applyFilter(/\bapple\b/, "column_a", "column_b");</code></pre>
<p>Note that the following also works (and results in the exact same dataset):</p>
<pre>
<code>dw.applyFilter(/\bapple\b/, [ "column_a", "column_b" ]);</code></pre>
<h5 class="content-sub-heading card-sub-heading" id="complex-filters">Complex Filters</h5>
<p>You may also use the complex syntax, in which all filters provided must find a match for the row to be visible. The complex filters can take any of the following arguments:</p>
<ul>
<li><code>columns</code> <i>(array of strings or single string)</i>: Columns on which to filter (defaults to all columns)</li>
<li><code>column</code> <i>(single string)</i>: Single column on which to filter (defaults to all columns) (note: if both <code>columns</code> and <code>column</code> are defined, the latter will be used)</li>
<li><code>matchAll</code> <i>(boolean)</i>: If this flag is set to <code>true</code> then all columns provided must match the filter for the row to stay visible. By default, if any of the columns match the row will stay visible</li>
<li><code>accentInsensitive</code> <i>(boolean)</i>: If this flag is set to <code>true</code> then characters with accent marks will be treated as normal ascii character (e.g., applé matches both applé and apple).</li>
<li><code>regex</code> <i>(string or RegExp)</i>: Columns must match this regular expression (if a string is provided it will be converted to a RegExp)</li>
<li><code>!regex</code> <i>(string or RegExp)</i>: Columns must <b>not</b> match this regular expression (if a string is provided it will be converted to a RegExp)</li>
<li><code>eq</code> <i>(value)</i>: Columns must equal (==) this value</li>
<li><code>ne</code> <i>(value)</i>: Columns must not equal (!=) this value</li>
<li><code>gte</code> <i>(value)</i>: Columns must be greater than or equal to (>=) this value</li>
<li><code>gt</code> <i>(value)</i>: Columns must be greater than (>) this value</li>
<li><code>lte</code> <i>(value)</i>: Columns must be lesser than or equal to (<=) this value</li>
<li><code>lt</code> <i>(value)</i>: Columns must be lesser than (<) this value</li>
</ul>
<p>For example:</p>
<pre>
<code>dw.applyFilter(
{
column : "column_a",
eq : "apple"
},
{
column : "column_b",
regex : /yellow/,
"!regex" : /blue/
},
{
columns : [ "column_a", "column_b" ],
gte: "apple",
lt: "zebra"
}
);
/* This results in the single row:
[ "apple", "yellow", "golden delicious" ]
*/</code></pre>
<p>The filter can be cleared by calling the <code>clearFilters</code> method:</p>
<pre>
<code>dw.clearFilters();</code></pre>
<p>Filters stack on top of each other:</p>
<pre>
<code>dw.applyFilter(/banana/);
/* This results in the following rows:
[ "banana", "green", "unripe" ],
[ "banana", "yellow", "ripe" ],
[ "banana", "brown", "beyond ripe" ],
[ "banana", "black", "rotten/frozen" ]
*/
dw.applyFilter(/yellow/);
/* The results are further filtered down to one row:
[ "banana", "yellow", "ripe" ],
*/
dw.clearFilters().applyFilter(/yellow/);
/* Old filters are cleared and a new one is applied:
[ "apple", "yellow", "golden delicious" ],
[ "banana", "yellow", "ripe" ],
*/</code></pre>
<p>To permanently remove rows from a dataset with a filter, use the <code>filter</code> method:</p>
<pre>
<code>dw.filter(/\bapple\b/)</code></pre>
</div>
</div>
</div>
</section>
<section id="search">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Search</h3>
<p>Similar to filters, the <code>search</code> method filters the currently visible dataset. The difference is that this will not modify the dataset. The simple form takes a callback and a search term, which can be either a regular expression, a string, or an array of <a href="#complex-filters">complex filters</a>.</p>
<pre>
<code>dw.search(function (results) { console.log(results); }, /apple/);</code></pre>
<p>It's also possible to pass in extra, optional arguments as an object. The valid options are:</p>
<ul>
<li><code>columns</code> <i>(array of strings or single string)</i>: Columns to be searched and returned (defaults to all columns)</li>
<li><code>searchOn</code> <i>(array of strings or single string)</i>: Columns to be searched (defaults to <code>columns</code>, which defaults to all columns)</li>
<li><code>returnColumns</code> <i>(array of strings or single string)</i>: Columns to be returned (defaults to <code>columns</code>, which defaults to all columns)</li>
<li><code>sortOn</code> <i>(array of strings or single string)</i>: Rows will be sorted on this/these columns, which can be invisible columns</li>
<li><code>limit</code> <i>(integer)</i>: Maximum number of rows to return. Data will be sorted before limiting results</li>
<li><code>fromRow</code> <i>(integer)</i>: Indicates the zero-based row number of the result set to start returning. This can be used, for example, in conjunction with <code>limit</code> to get paged results.</li>
<li><code>allRows</code> <i>(boolean)</i>: Allows the search to include hidden rows. Defaults to <code>false</code>.</li>
<li><code>getDistinct</code> <i>(boolean)</i>: Only returns rows that are unique from each other. Defaults to <code>false</code>.</li>
</ul>
<p>For example:</p>
<pre>
<code>dw.search(function (results) { console.log(results); }, /a.*le/i, {
columns: [ "column_a", "column_c" ],
sortOn: "-column_b",
fromRow: 6,
limit: 3
});</code></pre>
</div>
</div>
</div>
</section>
<section id="group">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Group</h3>
<p>Similar to grouping in SQL, the <code>group</code> method allows you to group rows together.</p>
<pre>
<code>dw.group("column_a");</code></pre>
<p>You may also group by multiple rows:</p>
<pre>
<code>dw.group("column_a", "column_b");</code></pre>
<p>Note that the following is also valid:</p>
<pre>
<code>dw.group([ "column_a", "column_b" ]);</code></pre>
<p>Rows with the same value for the specified column(s) will be combined; the column property <code>aggType</code> determines how values for non-specified columns are combined.</p>
</div>
</div>
</div>
</section>
<section id="join">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Join</h3>
<p>DataWorker also supports joining via the <code>join</code> method. It can inner join, left outer join, or right outer join.</p>
<p>The following inner joins <code>d1</code> with <code>d2</code> on <code>column_a</code> from <code>d1</code> and <code>column_d</code> from <code>d2</code>:</p>
<pre>
<code>d1.join(d2, "column_a", "column_d");</code></pre>
<p>The following left outer joins <code>d1</code> with <code>d2</code> on <code>column_a</code> from <code>d1</code> and <code>column_d</code> from <code>d2</code>:</p>
<pre>
<code>d1.join(d2, "column_a", "column_d", "left");</code></pre>
<p>The following right outer joins <code>d1</code> with <code>d2</code> on <code>column_a</code> from <code>d1</code> and <code>column_d</code> from <code>d2</code>:</p>
<pre>
<code>d1.join(d2, "column_a", "column_d", "right");</code></pre>
<p>Joins can also be performed on multiple columns:</p>
<pre>
<code>d1.join(d2, [ "column_a", "column_b" ], [ "column_d", "column_e" ]);</code></pre>
</div>
</div>
</div>
</section>
<section id="limit">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Limit</h3>
<p>The <code>applyLimit</code> method limits the amount of visible rows in the dataset. The following allows only the first 10 rows in the dataset to be visible:</p>
<pre> <code>dw.applyLimit(10);</code></pre>
<p>The limit can be cleared by calling the <code>clearFilters</code> method:</p>
<pre> <code>dw.clearFilters();</code></pre>
<p>To permanently remove rows from a dataset with a limit, use the <code>limit</code> method:</p>
<pre> <code>dw.limit(10)</code></pre>
</div>
</div>
</div>
</section>
<section id="remove-columns">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Remove columns</h3>
<p>You may completely delete columns from a dataset with the <code>removeColumns</code> method.</p>
<pre> <code>dw.removeColumns("column_a", "column_b");</code></pre>
<p>Note that the following is also valid:</p>
<pre> <code>dw.removeColumns([ "column_a", "column_b" ]);</code></pre>
</div>
</div>
</div>
</section>
<section id="hide-columns">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Hide columns</h3>
<p>Instead of permanently deleting the columns, you may also temporarily hide them from view using the <code>hideColumns</code> method:</p>
<pre> <code>dw.hideColumns("column_a", "column_b");</code></pre>
<p>Note that the following is also valid:</p>
<pre> <code>dw.hideColumns([ "column_a", "column_b" ]);</code></pre>
<p>Hidden columns can be shown with the <code>showColumns</code> method:</p>
<pre> <code>dw.showColumns("column_a", "column_b");</code></pre>
<p>Note that the following is also valid:</p>
<pre> <code>dw.showColumns([ "column_a", "column_b" ]);</code></pre>
<p>The <code>hideColumns</code> and <code>showColumns</code> methods may also take a regex as an argument. Any column name matching the regex will be hidden/shown, respectively.</p>
<pre> <code>dw.hideColumns(/^column_[ab]$/i);</code></pre>
<p>All columns can be hidden with the <code>hideAllColumns</code> method:</p>
<pre> <code>dw.hideAllColumns();</code></pre>
<p>All hidden columns can be revealed with the <code>showAllColumns</code> method:</p>
<pre> <code>dw.showAllColumns();</code></pre>
<p>Alternatively, you may retrieve all columns (visible AND non-visible) by using the <code>getAllColumns</code> method:</p>
<pre>
<code>dw.getAllColumns(function (columns) {
allColumns = columns;
});</code></pre>
</div>
</div>
</div>
</section>
<section id="clear-dataset">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Clear Dataset</h3>
<p>If you want to completely clear the dataset so that you can add new data while leaving any custom handlers intact, you may call <code>clearDataset</code>.</p>
<pre> <code>dw.clearDataset();</code></pre>
<p>This will remove references to all columns and rows. The function takes no parameters.</p>
</div>
</div>
</div>
</section>
<section id="sort">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Sort</h3>
<p>The following sorts the dataset on <code>column_a</code>:</p>
<pre>
<code>dw.sort("column_a")</code></pre>
<p>To reverse sort, prepend the column name with a <code>-</code>:</p>
<pre>
<code>dw.sort("-column_a")</code></pre>
<p>You may also sort on multiple columns:</p>
<pre>
<code>dw.sort("column_a", "-column_b");</code></pre>
<p>In this case, the sort will fallback to <code>column_b</code> if the contents of <code>column_a</code> are equal.</p>
<p>Note that the following does the same thing:</p>
<pre>
<code>dw.sort([ "column_a", "-column_b" ]);</code></pre>
<p>When child rows exist in the dataset, parents and children are kept together. The dataset will be sorted first by the parent, then by the children, using the same column. Using the example from <a href="#add-child-rows">Add Child Rows</a>, <code>dw.sort("-numbers");</code> will produce the following dataset:</p>
<pre>
<code>[
[ "xyz", 789 ],
[ "xyz", 789 ],
[ "abc", 123 ],
[ "abc", 579 ],
[ "abc", 456 ],
[ "def", 0 ]
]</code></pre>
</div>
</div>
</div>
</section>
<section id="add-child-rows">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Add Child Rows</h3>
Child rows can be added to the dataset by calling <code>addChildRows</code>. The columns must be the same as the original dataset as in the following example:</p>
<pre>
<code>var dataset = [
[ "letters", "numbers" ],
[ "abc", 579 ],
[ "def", 0 ],
[ "xyz", 789 ]
], childRows = [
[ "abc", 123 ],
[ "abc", 456 ],
[ "xyz", 789 ]
], dw = DataWorker(dataset);</code></pre>
<p>The call to <code>addChildRows</code> expects a column that will be used to determine to which row the children belong. It also expects a dataset of child rows.</p>
<p>You may either pass in an array of rows:</p>
<pre>
<code>dw.addChildRows(childRows, "letters");</code></pre>
<p><b>or</b> you may pass in another DataWorker object:</p>
<pre>
<code>var d2 = DataWorker([[ "letters", "numbers" ]].concat(childRows), "letters");
dw.addChildRows(d2);</code></pre>
<p>In either case, the result passed into the callback for <code>getRows</code> will be the same:</p>
<pre>
<code>[
[ "abc", 123 ],
[ "abc", 456 ],
[ "abc", 579 ],
[ "def", 0 ],
[ "xyz", 789 ],
[ "xyz", 789 ]
]</code></pre>
<p>The default visibility for a child row depends on its parent. If the parent row was set to hidden then the child row will still be added to the parent, but will be hidden as well.</p>
<p>If a parent row cannot be found for the children, those child rows will be ignored. If multiple parent rows exist for a given child row, the result is undefined.</p>
</div>
</div>
</div>
</section>
<section id="get-rows">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Get rows</h3>
<p>Visible dataset rows can be retrieved for use via the <code>getRows</code> method.</p>
<p>If called with just the callback function, <code>getRows</code> will get all rows. The next two arguments are the start and end of the range. If unspecified, they are the start and end of the dataset. These arguments are 0-based, so a dataset with 15 rows will have rows 0 - 14. If a number larger than the last row is used DataWorker will simply return anything in the range <i>up to</i> (and including) the last row.</p>
<pre>
<code>var dataset = [
[ "column_a", "column_b", "column_c" ],
[ "apple", "violin", "music" ],
[ "cat", "tissue", "dog" ],
[ "banana", "piano", "gum" ],
[ "gummy", "power", "star" ]
];
var dw = new DataWorker(dataset);
var records;
dw.getRows(function (result) { records = result; });</code></pre>
<p>The following is the contents of <code>records</code>:</p>
<pre>
<code>[
[ "apple", "violin", "music" ],
[ "cat", "tissue", "dog" ],
[ "banana", "piano", "gum" ],
[ "gummy", "power", "star" ]
]</code></pre>
<p>If you would only like certain columns, you may provide those columns after the range of rows, either as an array or as extra arguments. All of the following calls to getRows are valid examples:</p>
<pre>
<code>var callback = function (rows) { /* Do something */ };
dw.getRows(callback);
dw.getRows(callback, 5);
dw.getRows(callback, undefined, 10);
dw.getRows(callback, 5, 10);
dw.getRows(callback, 5, 10, "column_a", "column_b");
dw.getRows(callback, 5, 10, [ "column_a", "column_b" ]);
dw.getRows(callback, undefined, undefined, [ "column_a", "column_b" ]);</code></pre>
</div>
</div>
</div>
</section>
<section id="get-hashed-rows">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Get hashed rows</h3>
<p>The <code>getHashedRows</code> function makes it possible to get records as a hash with the column names instead of as a simple array. It works exactly the same as <code>getRows</code> but returns the data in a different format. In the following example</p>
<pre>
<code>var dataset = [
[ "column_a", "column_b", "column_c" ],
[ "apple", "violin", "music" ],
[ "cat", "tissue", "dog" ],
[ "banana", "piano", "gum" ],
[ "gummy", "power", "star" ]
];
var dw = new DataWorker(dataset);
var records;
dw.getHashedRows(function (result) { records = result; });</code></pre>
<p>the contents of <code>records</code> will be:</p>
<pre>
<code>[
{
"column_a": "apple",
"column_b": "violin",
"column_c": "music"
},
{
"column_a": "cat",
"column_b": "tissue",
"column_c": "dog"
},
{
"column_a": "banana",
"column_b": "piano",
"column_c": "gum"
},
{
"column_a": "gummy",
"column_b": "power",
"column_c": "star"
}
]</code></pre>
</div>
</div>
</div>
</section>
<section id="get-columns">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Get columns</h3>
<p>The <code>getColumns</code> method is used to get the visible columns of the dataset:</p>
<pre>
<code>dw.getColumns(function (columns) {
visibleColumns = columns;
});</code></pre>
<p>Use <code>getAllColumns</code> to retrieve both visible and non-visible columns.</p>
</div>
</div>
</div>
</section>
<section id="get-columns-and-records">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Get columns and records</h3>
<p>Visible columns may be retrieved simultaneously with visible records with <code>getColumnsAndRecords</code>.</p>
<pre>
<code>dw.getColumnsAndRecords(function (columns, records) {
// Do something.
});</code></pre>
<p>Columns will be given as a dictionary with the <code>columnName</code> as the key and its properties (also in a dictionary) as the value.</p>
<p>Records will be returned the same as in <code>getRows</code>.</p>
</div>
</div>
</div>
</section>
<section id="get-number-of-records">
<div class="card">
<div class="card-main">
<div class="card-inner">
<h3 class="content-sub-heading card-heading">Get number of records</h3>
<h5 class="content-sub-heading card-sub-heading">Get number of records</h5>
<p>The <code>getNumberOfRecords</code> returns the number of visible rows currently in the dataset:</p>
<pre>
<code>dw.getNumberOfRecords(function (num) {
numberOfRows = num;
});</code></pre>
<p>Note that for streaming datasets, this value will be the current number of rows it has (and not the total number of rows expected). Use <code>getExpectedNumberOfRecords</code> to determine the total number of rows in a streaming dataset.</p>
<h5 class="content-sub-heading card-sub-heading">Get expected number of records</h5>
<p>The <code>getExpectedNumberOfRecords</code> method returns the expected number of records in a streaming dataset.</p>
<pre>
<code>dw.getExpectedNumberOfRecords(function (num) {
expectedNumberOfRows = num;
});</code></pre>