-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathatom.xml
More file actions
1179 lines (1003 loc) · 216 KB
/
atom.xml
File metadata and controls
1179 lines (1003 loc) · 216 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>HEXH's Blog</title>
<subtitle>面朝大海,春暖花开</subtitle>
<link href="/atom.xml" rel="self"/>
<link href="http://dangxia.github.io/"/>
<updated>2016-04-15T16:55:36.150Z</updated>
<id>http://dangxia.github.io/</id>
<author>
<name>Xuehui He</name>
</author>
<generator uri="http://hexo.io/">Hexo</generator>
<entry>
<title>chrome-redirector</title>
<link href="http://dangxia.github.io/2016/04/15/chrome/2016-04-15-chrome-redirector/"/>
<id>http://dangxia.github.io/2016/04/15/chrome/2016-04-15-chrome-redirector/</id>
<published>2016-04-15T15:34:00.000Z</published>
<updated>2016-04-15T16:55:36.150Z</updated>
<content type="html"><h3>起因</h3>
<p><code>https://github-atom-io-herokuapp-com.global.ssl.fastly.net/assets/application-ba07c5c2889a34307a4b7d49410451d9.css</code><br>
不能使用<code>https</code>访问,使用<a href="https://github.com/chrome-redirector/chrome-redirector" target="_blank" rel="external">chrome-redirector</a>(商店已经下架,以防万一,已经备份了一个到百度云盘)重定向到<code>http</code></p>
<figure class="highlight vim"><table><tr><td class="code"><pre><span class="line">Refused <span class="keyword">to</span> load the script <span class="string">'http://github-atom-io-herokuapp-com.global.ssl.fastly.net/assets/application-3db62b578ebfc39ee871abc91b175302.js'</span></span><br><span class="line">because it violates the following Content Security Policy directive: <span class="comment">"script-src 'self' 'unsafe-inline'</span></span><br><span class="line">http<span class="variable">s:</span>//ssl.google-analytics.<span class="keyword">com</span></span><br><span class="line">http<span class="variable">s:</span>//www.google-analytics.<span class="keyword">com</span></span><br><span class="line">http<span class="variable">s:</span>//platform.twitter.<span class="keyword">com</span> http<span class="variable">s:</span>//github-atom-io-herokuapp-<span class="keyword">com</span>.<span class="keyword">global</span>.ssl.fastly.net<span class="comment">".</span></span><br></pre></td></tr></table></figure>
<a id="more"></a>
<p>由于<a href="http://content-security-policy.com/" target="_blank" rel="external">CSP</a>,chrome报错,进而又安装了plugin:<a href="https://chrome.google.com/webstore/detail/disable-content-security/ieelmcmcagommplceebfedjlakkhpden/related" target="_blank" rel="external">Disable Content-Security-Policy</a>,<a href="http://magento.stackexchange.com/questions/74121/how-to-stop-redirect-loading-of-insecure-scripts" target="_blank" rel="external">正确的web解决方法</a></p>
<p>可是仍然报错:</p>
<figure class="highlight mipsasm"><table><tr><td class="code"><pre><span class="line">Mixed Content: The page <span class="built_in">at</span> <span class="string">'https://atom.io/'</span> was loaded over HTTPS,</span><br><span class="line"><span class="keyword">but </span>requested an <span class="keyword">insecure </span>stylesheet</span><br><span class="line"><span class="string">'http://github-atom-io-herokuapp-com.global.ssl.fastly.net/assets/application-ba07c5c2889a34307a4b7d49410451d9.css'</span>.</span><br><span class="line">This request has <span class="keyword">been </span><span class="keyword">blocked; </span>the content must <span class="keyword">be </span>served over HTTPS.</span><br></pre></td></tr></table></figure>
<p>chrome URL的最右边,容许加载不安全的脚本。以下参照<a href="http://wiki.sln.suny.edu/display/SLNKB/Enabling+mixed+content+in+Google+Chrome" target="_blank" rel="external">Enabling mixed content in Google Chrome</a>,<a href="http://superuser.com/questions/487748/how-to-allow-chrome-browser-to-load-insecure-content" target="_blank" rel="external">How to allow Chrome (browser) to load insecure content?</a></p>
<ol>
<li>Click the alert shield icon in the address bar.</li>
<li>In the icon dialog box, click Load anyway.</li>
<li>The site will reload; if within ANGEL, users will be returned to the course homepage.</li>
<li>Upon returning to the page or link in question, the content will now be visible.</li>
<li>补充一下,浏览器重启后失效</li>
</ol>
<h3>rules</h3>
</content>
<summary type="html">
<h3>起因</h3>
<p><code>https://github-atom-io-herokuapp-com.global.ssl.fastly.net/assets/application-ba07c5c2889a34307a4b7d49410451d9.css</code><br>
不能使用<code>https</code>访问,使用<a href="https://github.com/chrome-redirector/chrome-redirector">chrome-redirector</a>(商店已经下架,以防万一,已经备份了一个到百度云盘)重定向到<code>http</code></p>
<figure class="highlight vim"><table><tr><td class="code"><pre><span class="line">Refused <span class="keyword">to</span> load the script <span class="string">'http://github-atom-io-herokuapp-com.global.ssl.fastly.net/assets/application-3db62b578ebfc39ee871abc91b175302.js'</span></span><br><span class="line">because it violates the following Content Security Policy directive: <span class="comment">"script-src 'self' 'unsafe-inline'</span></span><br><span class="line">http<span class="variable">s:</span>//ssl.google-analytics.<span class="keyword">com</span></span><br><span class="line">http<span class="variable">s:</span>//www.google-analytics.<span class="keyword">com</span></span><br><span class="line">http<span class="variable">s:</span>//platform.twitter.<span class="keyword">com</span> http<span class="variable">s:</span>//github-atom-io-herokuapp-<span class="keyword">com</span>.<span class="keyword">global</span>.ssl.fastly.net<span class="comment">".</span></span><br></pre></td></tr></table></figure>
</summary>
<category term="chrome" scheme="http://dangxia.github.io/categories/chrome/"/>
</entry>
<entry>
<title>atom-basic</title>
<link href="http://dangxia.github.io/2016/04/15/atom/2016-04-15-atom-basic/"/>
<id>http://dangxia.github.io/2016/04/15/atom/2016-04-15-atom-basic/</id>
<published>2016-04-15T08:48:00.000Z</published>
<updated>2016-04-15T14:31:23.983Z</updated>
<content type="html"><p>使用atom编辑markdown,大概了解一下怎么使用,参考<a href="http://wiki.jikexueyuan.com/project/atom/split-screen-operation.html" target="_blank" rel="external">Atom 使用教程</a></p>
<h3>常用快捷键</h3>
<table>
<thead>
<tr>
<th>快捷键</th>
<th>效果</th>
</tr>
</thead>
<tbody>
<tr>
<td>cmd + comma</td>
<td>open preferences</td>
</tr>
<tr>
<td>cmd + shift + p</td>
<td>toggle command palette</td>
</tr>
<tr>
<td>cmd + \</td>
<td>toggle tree view</td>
</tr>
<tr>
<td>cmd + shift + k</td>
<td>delete whole line(eclipse:cmd + d,<strong>和markdown-writer热键冲突</strong>)</td>
</tr>
<tr>
<td>cmd + shift + d</td>
<td>duplicate whole line</td>
</tr>
</tbody>
</table>
<a id="more"></a>
<h3>markdown-writer keybindings</h3>
<figure class="highlight scss"><table><tr><td class="code"><pre><span class="line">"<span class="selector-class">.platform-linux</span> atom-text-editor:not(<span class="selector-attr">[mini]</span>)":</span><br><span class="line"> <span class="string">"shift-ctrl-K"</span>: <span class="string">"markdown-writer:insert-link"</span></span><br><span class="line"> <span class="string">"shift-ctrl-I"</span>: <span class="string">"markdown-writer:insert-image"</span></span><br><span class="line"> <span class="string">"ctrl-i"</span>: <span class="string">"markdown-writer:toggle-italic-text"</span></span><br><span class="line"> <span class="string">"ctrl-b"</span>: <span class="string">"markdown-writer:toggle-bold-text"</span></span><br><span class="line"> <span class="string">"ctrl-'"</span>: <span class="string">"markdown-writer:toggle-code-text"</span></span><br><span class="line"> <span class="string">"ctrl-h"</span>: <span class="string">"markdown-writer:toggle-strikethrough-text"</span></span><br><span class="line"> <span class="string">"ctrl-1"</span>: <span class="string">"markdown-writer:toggle-h1"</span></span><br><span class="line"> <span class="string">"ctrl-2"</span>: <span class="string">"markdown-writer:toggle-h2"</span></span><br><span class="line"> <span class="string">"ctrl-3"</span>: <span class="string">"markdown-writer:toggle-h3"</span></span><br><span class="line"> <span class="string">"ctrl-4"</span>: <span class="string">"markdown-writer:toggle-h4"</span></span><br><span class="line"> <span class="string">"ctrl-5"</span>: <span class="string">"markdown-writer:toggle-h5"</span></span><br></pre></td></tr></table></figure>
<h3>notice</h3>
<p>atom 不支持<code>零宽度正回顾后发断言</code>,同样<a href="https://www.kate-editor.org/doc/kate-part-find-replace.html" target="_blank" rel="external">kate也不支持</a>.<br>
<code>esc</code> 可以关闭,finder,markdown-link-insert等窗口</p>
</content>
<summary type="html">
<p>使用atom编辑markdown,大概了解一下怎么使用,参考<a href="http://wiki.jikexueyuan.com/project/atom/split-screen-operation.html">Atom 使用教程</a></p>
<h3>常用快捷键</h3>
<table>
<thead>
<tr>
<th>快捷键</th>
<th>效果</th>
</tr>
</thead>
<tbody>
<tr>
<td>cmd + comma</td>
<td>open preferences</td>
</tr>
<tr>
<td>cmd + shift + p</td>
<td>toggle command palette</td>
</tr>
<tr>
<td>cmd + \</td>
<td>toggle tree view</td>
</tr>
<tr>
<td>cmd + shift + k</td>
<td>delete whole line(eclipse:cmd + d,<strong>和markdown-writer热键冲突</strong>)</td>
</tr>
<tr>
<td>cmd + shift + d</td>
<td>duplicate whole line</td>
</tr>
</tbody>
</table>
</summary>
<category term="atom" scheme="http://dangxia.github.io/categories/atom/"/>
</entry>
<entry>
<title>use-hexo-renderer-markdown-it</title>
<link href="http://dangxia.github.io/2016/04/15/hexo/2016-04-15-use-hexo-renderer-markdown-it/"/>
<id>http://dangxia.github.io/2016/04/15/hexo/2016-04-15-use-hexo-renderer-markdown-it/</id>
<published>2016-04-15T08:27:00.000Z</published>
<updated>2016-04-15T14:31:23.988Z</updated>
<content type="html"><p>use <a href="https://github.com/celsomiranda/hexo-renderer-markdown-it" title="hexo-renderer-markdown-it" target="_blank" rel="external">hexo-renderer-markdown-it</a> replace hexo-renderer-marked</p>
<h3>install</h3>
<figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">npm un hexo-renderer-marked --save</span><br><span class="line">npm i hexo-renderer-markdown-it --save</span><br></pre></td></tr></table></figure>
<a id="more"></a>
<h3><a href="https://github.com/celsomiranda/hexo-renderer-markdown-it/wiki/Advanced-Configuration" title="config" target="_blank" rel="external">config</a></h3>
<figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="attr">markdown:</span></span><br><span class="line"><span class="attr"> render:</span></span><br><span class="line"><span class="attr"> html:</span> <span class="literal">true</span></span><br><span class="line"><span class="attr"> xhtmlOut:</span> <span class="literal">false</span></span><br><span class="line"><span class="attr"> breaks:</span> <span class="literal">true</span></span><br><span class="line"><span class="attr"> linkify:</span> <span class="literal">true</span></span><br><span class="line"><span class="attr"> typographer:</span> <span class="literal">true</span></span><br><span class="line"><span class="attr"> quotes:</span> <span class="string">'“”‘’'</span></span><br><span class="line"><span class="attr"> plugins:</span></span><br><span class="line"><span class="bullet"> -</span> markdown-it-abbr</span><br><span class="line"><span class="bullet"> -</span> markdown-it-footnote</span><br><span class="line"><span class="bullet"> -</span> markdown-it-ins</span><br><span class="line"><span class="bullet"> -</span> markdown-it-sub</span><br><span class="line"><span class="bullet"> -</span> markdown-it-sup</span><br><span class="line"><span class="attr"> anchors:</span></span><br><span class="line"><span class="attr"> level:</span> <span class="number">2</span></span><br><span class="line"><span class="attr"> collisionSuffix:</span> <span class="string">'v'</span></span><br><span class="line"><span class="attr"> permalink:</span> <span class="literal">true</span></span><br><span class="line"><span class="attr"> permalinkClass:</span> header-anchor</span><br><span class="line"><span class="attr"> permalinkSymbol:</span> ¶</span><br></pre></td></tr></table></figure>
<h3>notice</h3>
<p>default <code>breaks</code> is <code>false</code> need set to <code>true</code><br>
配合atom的markdown-preview <code>break on single line</code></p>
<ul>
<li>issue:<a href="http://lnxpgn.github.io/2015/08/01/hexo-markdown-issues/" target="_blank" rel="external">read more</a><br>
<code>Normally, we insert &quot;&lt;!-- more --&gt;&quot; into a post to display its summary, but don't work if using hexo-renderer-markdown-it with default configuration, hexo-renderer-markdown-it escapes HTML. To fix this issue by appending</code></li>
</ul>
</content>
<summary type="html">
<p>use <a href="https://github.com/celsomiranda/hexo-renderer-markdown-it" title="hexo-renderer-markdown-it">hexo-renderer-markdown-it</a> replace hexo-renderer-marked</p>
<h3>install</h3>
<figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">npm un hexo-renderer-marked --save</span><br><span class="line">npm i hexo-renderer-markdown-it --save</span><br></pre></td></tr></table></figure>
</summary>
<category term="hexo" scheme="http://dangxia.github.io/categories/hexo/"/>
</entry>
<entry>
<title>use atom edit markdown files</title>
<link href="http://dangxia.github.io/2016/04/15/atom/use-atom-write-markdown/"/>
<id>http://dangxia.github.io/2016/04/15/atom/use-atom-write-markdown/</id>
<published>2016-04-15T07:43:37.000Z</published>
<updated>2016-04-15T14:31:23.983Z</updated>
<content type="html"><h3>issue</h3>
<ul>
<li>markdown-preview (atom default plugin)<br>
issue:<a href="https://github.com/atom/markdown-preview/issues/348" target="_blank" rel="external">blank line break code</a></li>
<li>markdonw-writer<br>
tab only move whole line</li>
<li>markdown-preview-plus<br>
issue:<a href="https://github.com/mark-hahn/markdown-scroll-sync/issues/4" target="_blank" rel="external">markdown-scroll-sync don't work on it</a></li>
</ul>
<h3>plugins</h3>
<ul>
<li>markdown-preview-plus</li>
<li>markdown-writer</li>
<li>markdown-scroll-sync</li>
</ul>
</content>
<summary type="html">
<h3>issue</h3>
<ul>
<li>markdown-preview (atom default plugin)<br>
issue:<a href="https://github.com/atom/markdown-preview/issues/348" targe
</summary>
<category term="atom" scheme="http://dangxia.github.io/categories/atom/"/>
</entry>
<entry>
<title>hexo-local-fonts</title>
<link href="http://dangxia.github.io/2016/04/14/hexo/hexo-local-fonts/"/>
<id>http://dangxia.github.io/2016/04/14/hexo/hexo-local-fonts/</id>
<published>2016-04-14T07:43:40.000Z</published>
<updated>2016-04-15T14:31:23.988Z</updated>
<content type="html"><h3>下载css</h3>
<p><a href="http://fonts.useso.com/css?family=Lato:300,400,700,400italic&amp;subset=latin,latin-ext" target="_blank" rel="external">google fonts css</a></p>
<h3>修改css 安置在source/fonts/fonts.css</h3>
<a id="more"></a>
<figure class="highlight css"><table><tr><td class="code"><pre><span class="line"><span class="comment">/* latin-ext */</span></span><br><span class="line">@<span class="keyword">font-face</span> &#123;</span><br><span class="line"> <span class="attribute">font-family</span>: <span class="string">'Lato'</span>;</span><br><span class="line"> <span class="attribute">font-style</span>: normal;</span><br><span class="line"> <span class="attribute">font-weight</span>: <span class="number">300</span>;</span><br><span class="line"> <span class="attribute">src</span>: <span class="built_in">local</span>(<span class="string">'Lato Light'</span>), <span class="built_in">local</span>(<span class="string">'Lato-Light'</span>), <span class="built_in">url</span>(/fonts/Lato/dPJ5r9gl3kK6ijoeP1IRsvY6323mHUZFJMgTvxaG2iE.woff2) <span class="built_in">format</span>(<span class="string">'woff2'</span>);</span><br><span class="line"> <span class="attribute">unicode-range</span>: U+<span class="number">0100</span>-<span class="number">024</span>F, U+<span class="number">1</span>E00-<span class="number">1</span>EFF, U+<span class="number">20</span>A0-<span class="number">20</span>AB, U+<span class="number">20</span>AD-<span class="number">20</span>CF, U+<span class="number">2</span>C60-<span class="number">2</span>C7F, U+A720-A7FF;</span><br><span class="line">&#125;</span><br><span class="line"><span class="comment">/* latin */</span></span><br><span class="line">@<span class="keyword">font-face</span> &#123;</span><br><span class="line"> <span class="attribute">font-family</span>: <span class="string">'Lato'</span>;</span><br><span class="line"> <span class="attribute">font-style</span>: normal;</span><br><span class="line"> <span class="attribute">font-weight</span>: <span class="number">300</span>;</span><br><span class="line"> <span class="attribute">src</span>: <span class="built_in">local</span>(<span class="string">'Lato Light'</span>), <span class="built_in">local</span>(<span class="string">'Lato-Light'</span>), <span class="built_in">url</span>(/fonts/Lato/EsvMC5un3kjyUhB9ZEPPwg.woff2) <span class="built_in">format</span>(<span class="string">'woff2'</span>);</span><br><span class="line"> <span class="attribute">unicode-range</span>: U+<span class="number">0000</span>-<span class="number">00</span>FF, U+<span class="number">0131</span>, U+<span class="number">0152</span>-<span class="number">0153</span>, U+<span class="number">02</span>C6, U+<span class="number">02</span>DA, U+<span class="number">02</span>DC, U+<span class="number">2000</span>-<span class="number">206</span>F, U+<span class="number">2074</span>, U+<span class="number">20</span>AC, U+<span class="number">2212</span>, U+<span class="number">2215</span>, U+E0FF, U+EFFD, U+F000;</span><br><span class="line">&#125;</span><br><span class="line"><span class="comment">/* latin-ext */</span></span><br><span class="line">@<span class="keyword">font-face</span> &#123;</span><br><span class="line"> <span class="attribute">font-family</span>: <span class="string">'Lato'</span>;</span><br><span class="line"> <span class="attribute">font-style</span>: normal;</span><br><span class="line"> <span class="attribute">font-weight</span>: <span class="number">400</span>;</span><br><span class="line"> <span class="attribute">src</span>: <span class="built_in">local</span>(<span class="string">'Lato Regular'</span>), <span class="built_in">local</span>(<span class="string">'Lato-Regular'</span>), <span class="built_in">url</span>(/fonts/Lato/UyBMtLsHKBKXelqf4x7VRQ.woff2) <span class="built_in">format</span>(<span class="string">'woff2'</span>);</span><br><span class="line"> <span class="attribute">unicode-range</span>: U+<span class="number">0100</span>-<span class="number">024</span>F, U+<span class="number">1</span>E00-<span class="number">1</span>EFF, U+<span class="number">20</span>A0-<span class="number">20</span>AB, U+<span class="number">20</span>AD-<span class="number">20</span>CF, U+<span class="number">2</span>C60-<span class="number">2</span>C7F, U+A720-A7FF;</span><br><span class="line">&#125;</span><br><span class="line"><span class="comment">/* latin */</span></span><br><span class="line">@<span class="keyword">font-face</span> &#123;</span><br><span class="line"> <span class="attribute">font-family</span>: <span class="string">'Lato'</span>;</span><br><span class="line"> <span class="attribute">font-style</span>: normal;</span><br><span class="line"> <span class="attribute">font-weight</span>: <span class="number">400</span>;</span><br><span class="line"> <span class="attribute">src</span>: <span class="built_in">local</span>(<span class="string">'Lato Regular'</span>), <span class="built_in">local</span>(<span class="string">'Lato-Regular'</span>), <span class="built_in">url</span>(/fonts/Lato/1YwB1sO8YE1Lyjf12WNiUA.woff2) <span class="built_in">format</span>(<span class="string">'woff2'</span>);</span><br><span class="line"> <span class="attribute">unicode-range</span>: U+<span class="number">0000</span>-<span class="number">00</span>FF, U+<span class="number">0131</span>, U+<span class="number">0152</span>-<span class="number">0153</span>, U+<span class="number">02</span>C6, U+<span class="number">02</span>DA, U+<span class="number">02</span>DC, U+<span class="number">2000</span>-<span class="number">206</span>F, U+<span class="number">2074</span>, U+<span class="number">20</span>AC, U+<span class="number">2212</span>, U+<span class="number">2215</span>, U+E0FF, U+EFFD, U+F000;</span><br><span class="line">&#125;</span><br><span class="line"><span class="comment">/* latin-ext */</span></span><br><span class="line">@<span class="keyword">font-face</span> &#123;</span><br><span class="line"> <span class="attribute">font-family</span>: <span class="string">'Lato'</span>;</span><br><span class="line"> <span class="attribute">font-style</span>: normal;</span><br><span class="line"> <span class="attribute">font-weight</span>: <span class="number">700</span>;</span><br><span class="line"> <span class="attribute">src</span>: <span class="built_in">local</span>(<span class="string">'Lato Bold'</span>), <span class="built_in">local</span>(<span class="string">'Lato-Bold'</span>), <span class="built_in">url</span>(/fonts/Lato/ObQr5XYcoH0WBoUxiaYK3_Y6323mHUZFJMgTvxaG2iE.woff2) <span class="built_in">format</span>(<span class="string">'woff2'</span>);</span><br><span class="line"> <span class="attribute">unicode-range</span>: U+<span class="number">0100</span>-<span class="number">024</span>F, U+<span class="number">1</span>E00-<span class="number">1</span>EFF, U+<span class="number">20</span>A0-<span class="number">20</span>AB, U+<span class="number">20</span>AD-<span class="number">20</span>CF, U+<span class="number">2</span>C60-<span class="number">2</span>C7F, U+A720-A7FF;</span><br><span class="line">&#125;</span><br><span class="line"><span class="comment">/* latin */</span></span><br><span class="line">@<span class="keyword">font-face</span> &#123;</span><br><span class="line"> <span class="attribute">font-family</span>: <span class="string">'Lato'</span>;</span><br><span class="line"> <span class="attribute">font-style</span>: normal;</span><br><span class="line"> <span class="attribute">font-weight</span>: <span class="number">700</span>;</span><br><span class="line"> <span class="attribute">src</span>: <span class="built_in">local</span>(<span class="string">'Lato Bold'</span>), <span class="built_in">local</span>(<span class="string">'Lato-Bold'</span>), <span class="built_in">url</span>(/fonts/Lato/H2DMvhDLycM56KNuAtbJYA.woff2) <span class="built_in">format</span>(<span class="string">'woff2'</span>);</span><br><span class="line"> <span class="attribute">unicode-range</span>: U+<span class="number">0000</span>-<span class="number">00</span>FF, U+<span class="number">0131</span>, U+<span class="number">0152</span>-<span class="number">0153</span>, U+<span class="number">02</span>C6, U+<span class="number">02</span>DA, U+<span class="number">02</span>DC, U+<span class="number">2000</span>-<span class="number">206</span>F, U+<span class="number">2074</span>, U+<span class="number">20</span>AC, U+<span class="number">2212</span>, U+<span class="number">2215</span>, U+E0FF, U+EFFD, U+F000;</span><br><span class="line">&#125;</span><br><span class="line"><span class="comment">/* latin-ext */</span></span><br><span class="line">@<span class="keyword">font-face</span> &#123;</span><br><span class="line"> <span class="attribute">font-family</span>: <span class="string">'Lato'</span>;</span><br><span class="line"> <span class="attribute">font-style</span>: italic;</span><br><span class="line"> <span class="attribute">font-weight</span>: <span class="number">300</span>;</span><br><span class="line"> <span class="attribute">src</span>: <span class="built_in">local</span>(<span class="string">'Lato Light Italic'</span>), <span class="built_in">local</span>(<span class="string">'Lato-LightItalic'</span>), <span class="built_in">url</span>(/fonts/Lato/XNVd6tsqi9wmKNvnh5HNEBJtnKITppOI_IvcXXDNrsc.woff2) <span class="built_in">format</span>(<span class="string">'woff2'</span>);</span><br><span class="line"> <span class="attribute">unicode-range</span>: U+<span class="number">0100</span>-<span class="number">024</span>F, U+<span class="number">1</span>E00-<span class="number">1</span>EFF, U+<span class="number">20</span>A0-<span class="number">20</span>AB, U+<span class="number">20</span>AD-<span class="number">20</span>CF, U+<span class="number">2</span>C60-<span class="number">2</span>C7F, U+A720-A7FF;</span><br><span class="line">&#125;</span><br><span class="line"><span class="comment">/* latin */</span></span><br><span class="line">@<span class="keyword">font-face</span> &#123;</span><br><span class="line"> <span class="attribute">font-family</span>: <span class="string">'Lato'</span>;</span><br><span class="line"> <span class="attribute">font-style</span>: italic;</span><br><span class="line"> <span class="attribute">font-weight</span>: <span class="number">300</span>;</span><br><span class="line"> <span class="attribute">src</span>: <span class="built_in">local</span>(<span class="string">'Lato Light Italic'</span>), <span class="built_in">local</span>(<span class="string">'Lato-LightItalic'</span>), <span class="built_in">url</span>(/fonts/Lato/2HG_tEPiQ4Z6795cGfdivFtXRa8TVwTICgirnJhmVJw.woff2) <span class="built_in">format</span>(<span class="string">'woff2'</span>);</span><br><span class="line"> <span class="attribute">unicode-range</span>: U+<span class="number">0000</span>-<span class="number">00</span>FF, U+<span class="number">0131</span>, U+<span class="number">0152</span>-<span class="number">0153</span>, U+<span class="number">02</span>C6, U+<span class="number">02</span>DA, U+<span class="number">02</span>DC, U+<span class="number">2000</span>-<span class="number">206</span>F, U+<span class="number">2074</span>, U+<span class="number">20</span>AC, U+<span class="number">2212</span>, U+<span class="number">2215</span>, U+E0FF, U+EFFD, U+F000;</span><br><span class="line">&#125;</span><br><span class="line"><span class="comment">/* latin-ext */</span></span><br><span class="line">@<span class="keyword">font-face</span> &#123;</span><br><span class="line"> <span class="attribute">font-family</span>: <span class="string">'Lato'</span>;</span><br><span class="line"> <span class="attribute">font-style</span>: italic;</span><br><span class="line"> <span class="attribute">font-weight</span>: <span class="number">400</span>;</span><br><span class="line"> <span class="attribute">src</span>: <span class="built_in">local</span>(<span class="string">'Lato Italic'</span>), <span class="built_in">local</span>(<span class="string">'Lato-Italic'</span>), <span class="built_in">url</span>(/fonts/Lato/YMOYVM-eg6Qs9YzV9OSqZfesZW2xOQ-xsNqO47m55DA.woff2) <span class="built_in">format</span>(<span class="string">'woff2'</span>);</span><br><span class="line"> <span class="attribute">unicode-range</span>: U+<span class="number">0100</span>-<span class="number">024</span>F, U+<span class="number">1</span>E00-<span class="number">1</span>EFF, U+<span class="number">20</span>A0-<span class="number">20</span>AB, U+<span class="number">20</span>AD-<span class="number">20</span>CF, U+<span class="number">2</span>C60-<span class="number">2</span>C7F, U+A720-A7FF;</span><br><span class="line">&#125;</span><br><span class="line"><span class="comment">/* latin */</span></span><br><span class="line">@<span class="keyword">font-face</span> &#123;</span><br><span class="line"> <span class="attribute">font-family</span>: <span class="string">'Lato'</span>;</span><br><span class="line"> <span class="attribute">font-style</span>: italic;</span><br><span class="line"> <span class="attribute">font-weight</span>: <span class="number">400</span>;</span><br><span class="line"> <span class="attribute">src</span>: <span class="built_in">local</span>(<span class="string">'Lato Italic'</span>), <span class="built_in">local</span>(<span class="string">'Lato-Italic'</span>), <span class="built_in">url</span>(/fonts/Lato/PLygLKRVCQnA5fhu3qk5fQ.woff2) <span class="built_in">format</span>(<span class="string">'woff2'</span>);</span><br><span class="line"> <span class="attribute">unicode-range</span>: U+<span class="number">0000</span>-<span class="number">00</span>FF, U+<span class="number">0131</span>, U+<span class="number">0152</span>-<span class="number">0153</span>, U+<span class="number">02</span>C6, U+<span class="number">02</span>DA, U+<span class="number">02</span>DC, U+<span class="number">2000</span>-<span class="number">206</span>F, U+<span class="number">2074</span>, U+<span class="number">20</span>AC, U+<span class="number">2212</span>, U+<span class="number">2215</span>, U+E0FF, U+EFFD, U+F000;</span><br><span class="line">&#125;</span><br><span class="line"><span class="comment">/* latin-ext */</span></span><br><span class="line">@<span class="keyword">font-face</span> &#123;</span><br><span class="line"> <span class="attribute">font-family</span>: <span class="string">'Lato'</span>;</span><br><span class="line"> <span class="attribute">font-style</span>: italic;</span><br><span class="line"> <span class="attribute">font-weight</span>: <span class="number">700</span>;</span><br><span class="line"> <span class="attribute">src</span>: <span class="built_in">local</span>(<span class="string">'Lato Bold Italic'</span>), <span class="built_in">local</span>(<span class="string">'Lato-BoldItalic'</span>), <span class="built_in">url</span>(/fonts/Lato/AcvTq8Q0lyKKNxRlL28RnxJtnKITppOI_IvcXXDNrsc.woff2) <span class="built_in">format</span>(<span class="string">'woff2'</span>);</span><br><span class="line"> <span class="attribute">unicode-range</span>: U+<span class="number">0100</span>-<span class="number">024</span>F, U+<span class="number">1</span>E00-<span class="number">1</span>EFF, U+<span class="number">20</span>A0-<span class="number">20</span>AB, U+<span class="number">20</span>AD-<span class="number">20</span>CF, U+<span class="number">2</span>C60-<span class="number">2</span>C7F, U+A720-A7FF;</span><br><span class="line">&#125;</span><br><span class="line"><span class="comment">/* latin */</span></span><br><span class="line">@<span class="keyword">font-face</span> &#123;</span><br><span class="line"> <span class="attribute">font-family</span>: <span class="string">'Lato'</span>;</span><br><span class="line"> <span class="attribute">font-style</span>: italic;</span><br><span class="line"> <span class="attribute">font-weight</span>: <span class="number">700</span>;</span><br><span class="line"> <span class="attribute">src</span>: <span class="built_in">local</span>(<span class="string">'Lato Bold Italic'</span>), <span class="built_in">local</span>(<span class="string">'Lato-BoldItalic'</span>), <span class="built_in">url</span>(/fonts/Lato/HkF_qI1x_noxlxhrhMQYEFtXRa8TVwTICgirnJhmVJw.woff2) <span class="built_in">format</span>(<span class="string">'woff2'</span>);</span><br><span class="line"> <span class="attribute">unicode-range</span>: U+<span class="number">0000</span>-<span class="number">00</span>FF, U+<span class="number">0131</span>, U+<span class="number">0152</span>-<span class="number">0153</span>, U+<span class="number">02</span>C6, U+<span class="number">02</span>DA, U+<span class="number">02</span>DC, U+<span class="number">2000</span>-<span class="number">206</span>F, U+<span class="number">2074</span>, U+<span class="number">20</span>AC, U+<span class="number">2212</span>, U+<span class="number">2215</span>, U+E0FF, U+EFFD, U+F000;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<h3>下载相应的文件到</h3>
<p><code>source/fonts/Lato</code></p>
<h3>修改next的external-fonts.swig</h3>
<figure class="highlight html"><table><tr><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">link</span> <span class="attr">href</span>=<span class="string">"&#123;&#123; font_host &#125;&#125;/fonts.css?family=&#123;&#123; font_families &#125;&#125;"</span> <span class="attr">rel</span>=<span class="string">"stylesheet"</span> <span class="attr">type</span>=<span class="string">"text/css"</span>&gt;</span></span><br></pre></td></tr></table></figure>
<h3>refer</h3>
<p><a href="https://www.zhihu.com/question/20587762" target="_blank" rel="external">https://www.zhihu.com/question/20587762</a></p>
</content>
<summary type="html">
<h3>下载css</h3>
<p><a href="http://fonts.useso.com/css?family=Lato:300,400,700,400italic&amp;subset=latin,latin-ext">google fonts css</a></p>
<h3>修改css 安置在source/fonts/fonts.css</h3>
</summary>
<category term="hexo" scheme="http://dangxia.github.io/categories/hexo/"/>
</entry>
<entry>
<title>AbstractQueuedSynchronizer Source</title>
<link href="http://dangxia.github.io/2016/03/22/java/AbstractQueuedSynchronizer-Source/"/>
<id>http://dangxia.github.io/2016/03/22/java/AbstractQueuedSynchronizer-Source/</id>
<published>2016-03-22T03:18:02.000Z</published>
<updated>2016-04-15T14:31:23.989Z</updated>
<content type="html"><h3>不添加Node到queue</h3>
<ul>
<li>tryAcquire 可以获得锁</li>
<li>tryAcquireShared 可以获得锁</li>
</ul>
<h3>进入queue</h3>
<p>pred.waitStatus == SIGNAL ==&gt; park<br>
pred.waitStatus == CANCELED ==&gt; 更新祖先节点,直到祖先节点不为CANCELED的节点<br>
其他 =&gt; 更新祖先节点为SIGNAL , park</p>
<h3>acquireQueued</h3>
<p>setHead(node),原先的head被移除</p>
<h3>waitStatus</h3>
<p>enq : 0<br>
被下一个enq:更新为SINGAL<br>
被unparkSuccessor:更新为:0<br>
cancelAcquire:当超时或interrept,更新为CANCELLED</p>
</content>
<summary type="html">
<h3>不添加Node到queue</h3>
<ul>
<li>tryAcquire 可以获得锁</li>
<li>tryAcquireShared 可以获得锁</li>
</ul>
<h3>进入queue</h3>
<p>pred.waitStatus == SIGNAL =
</summary>
<category term="java" scheme="http://dangxia.github.io/categories/java/"/>
<category term="java" scheme="http://dangxia.github.io/tags/java/"/>
</entry>
<entry>
<title>Chapter4:Classes and Objects</title>
<link href="http://dangxia.github.io/2016/01/24/scala/Classes-and-Objects/"/>
<id>http://dangxia.github.io/2016/01/24/scala/Classes-and-Objects/</id>
<published>2016-01-24T07:44:29.000Z</published>
<updated>2016-04-15T14:31:23.994Z</updated>
<content type="html"><h3>Classes, fields, and methods</h3>
<blockquote>
<p><strong>The way you make members public in Scala is by not explicitly specifyingany access modifier. Put another way, where you’d say “public” in Java,you simply say nothing in Scala. Public is Scala’s default access level.</strong></p>
</blockquote>
<h3>Semicolon inference</h3>
<ul>
<li>single line<br>
one statement on a single line,semicolon is optional<br>
multiple statement on a single line,semicolone is require</li>
<li>multiple lines<br>
one statement on multiple lines,most of the time,is treated as one multiple lines statement</li>
</ul>
<blockquote>
<p><strong>The rules of semicolon inference</strong><br>
The precise rules for statement separation are surprisingly simple for how well they work.<br>
In short, a line ending is treated as a semicolon unless one of the following conditions is true:</p>
</blockquote>
<ol>
<li>The line in question ends in a word that would not be legal as theend of a statement, such as a period or an infix operator.</li>
<li>The next line begins with a word that cannot start a statement.</li>
<li>The line ends while inside parentheses (...) or brackets [...],because these cannot contain multiple statements anyway.</li>
</ol>
<h3>Singleton objects</h3>
<p>Scala cannot have static members. Instead, Scala has singleton objects.<br>
A singleton object definition looks like a class definition, except instead of the keyword class you use the keyword object.<br>
When a singleton object shares the same name with a class, it is called that class’s companion object. You must define both the class and its companion object in the same source file. The class is called the companion class of the singleton object. A class and its companion object can access each other’s private members.<br>
However, singleton objects extend a superclass and can mix in traits. Given each singleton object is an instance of its superclasses and mixed-in traits, you can invoke its methods via these types, refer to it from variables of these types, and pass it to methods expecting these types.<br>
A singleton object that does not share the same name with a companion class is called a standalone object. You can use standalone objects for many purposes, including collecting related utility methods together, or defining an entry point to a Scala application</p>
<h3>A Scala application</h3>
<p>To run a Scala program, you must supply the name of a standalone singleton object with a main method that takes one parameter, an Array[String], and has a result type of Unit.</p>
<blockquote>
<p><strong>Scala implicitly imports members of packages java.lang and scala, as well as the members of a singleton object named Predef, into every Scala source file. Predef, which resides in package scala, contains many useful methods. For example, when you say println in a Scala source file, you’re actually invoking println on Predef. (Predef.println turns around and invokes Console.println, which does the real work.) When you say assert, you’re invoking Predef.assert.</strong></p>
</blockquote>
<p>Neither ChecksumAccumulator.scala nor Summer.scala are scripts, because they end in a definition. A script, by contrast, must end in a result expression.<br>
**fsc:**This compiles your source files, but there may be a perceptible delay before the compilation finishes. The reason is that every time the compiler starts up, it spends time scanning the contents of jar files and doing other initial work before it even looks at the fresh source files you submit to it. For this reason, the Scala distribution also includes a Scala compiler daemon called fsc (for fast Scala compiler). You use it like this:</p>
<figure class="highlight stata"><table><tr><td class="code"><pre><span class="line">$ fsc ChecksumAccumulator.<span class="keyword">scala</span> Summer.<span class="keyword">scala</span></span><br></pre></td></tr></table></figure>
<h3>The Application trait</h3>
<p>Scala provides a trait, scala.Application, that can save you some finger typing.<br>
To use the trait, you first write “extends Application” after the name of your singleton object. Then instead of writing a main method, you place the code you would have put in the main method directly between the curly braces of the singleton object. That’s it. You can compile and run this application just like any other.<br>
Inheriting from Application is shorter than writing an explicit main method, but it also has some shortcomings. First, you can’t use this trait if you need to access command-line arguments, because the args array isn’t available. For example, because the Summer application uses command-line arguments, it must be written with an explicit main method, as shown in Listing 4.3. Second, because of some restrictions in the JVM threading model, you need an explicit main method if your program is multi-threaded. Finally, some implementations of the JVM do not optimize the initialization code of an object which is executed by the Application trait. So you should inherit from Application only when your program is relatively simple and single-threaded</p>
</content>
<summary type="html">
<h3>Classes, fields, and methods</h3>
<blockquote>
<p><strong>The way you make members public in Scala is by not explicitly specifyingany ac
</summary>
<category term="Scala" scheme="http://dangxia.github.io/categories/Scala/"/>
<category term="Programming in Scala" scheme="http://dangxia.github.io/tags/Programming-in-Scala/"/>
</entry>
<entry>
<title>English-Names</title>
<link href="http://dangxia.github.io/2016/01/16/english/2015/English-Names/"/>
<id>http://dangxia.github.io/2016/01/16/english/2015/English-Names/</id>
<published>2016-01-16T01:35:21.000Z</published>
<updated>2016-04-15T14:31:23.984Z</updated>
<content type="html"><ul>
<li><a href="#v">Eric /'erɪk/</a> 埃里克,男子名</li>
<li><a href="#v">Raymond /'reɪmənd/</a> n. 雷蒙德(男子名)</li>
<li><a href="#v">Guy /gaɪ/</a> n. 男人,家伙; vt. 嘲弄,取笑; vi. 逃跑; n. (Guy)人名;(西)吉;(法)居伊;(英)盖伊</li>
<li><a href="#v">Steele /sti:l/</a> n. 斯蒂尔(姓氏)</li>
</ul>
</content>
<summary type="html">
<ul>
<li><a href="#v">Eric /'erɪk/</a> 埃里克,男子名</li>
<li><a href="#v">Raymond /'reɪmənd/</a> n. 雷蒙德(男子名)</li>
<li><a href="#v">Guy /gaɪ/</a>
</summary>
<category term="english" scheme="http://dangxia.github.io/categories/english/"/>
</entry>
<entry>
<title>apache-http-core</title>
<link href="http://dangxia.github.io/2016/01/02/trivial/apache-http-core/"/>
<id>http://dangxia.github.io/2016/01/02/trivial/apache-http-core/</id>
<published>2016-01-02T03:00:06.000Z</published>
<updated>2016-04-15T14:31:23.997Z</updated>
<content type="html"><ul>
<li>https://hc.apache.org/httpcomponents-core-ga/tutorial/html/</li>
<li>https://hc.apache.org/httpcomponents-client-ga/tutorial/html/</li>
</ul>
</content>
<summary type="html">
<ul>
<li>https://hc.apache.org/httpcomponents-core-ga/tutorial/html/</li>
<li>https://hc.apache.org/httpcomponents-client-ga/tutorial/html/<
</summary>
<category term="trivial" scheme="http://dangxia.github.io/categories/trivial/"/>
</entry>
<entry>
<title>Chapter6:Functional Objects</title>
<link href="http://dangxia.github.io/2015/12/06/scala/Functional-Objects/"/>
<id>http://dangxia.github.io/2015/12/06/scala/Functional-Objects/</id>
<published>2015-12-06T07:44:29.000Z</published>
<updated>2016-04-15T14:31:23.995Z</updated>
<content type="html"><h3>Constructing a Rational</h3>
<figure class="highlight scala"><table><tr><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Rational</span>(<span class="params">n: <span class="type">Int</span>, d: <span class="type">Int</span></span>)</span></span><br></pre></td></tr></table></figure>
<p>this line of code is that if a class doesn’t have a body, you don’t need to specify empty curly braces (though you could, of course, if you wanted to).</p>
<p><strong>primary constructor</strong></p>
<blockquote>
<p><strong>Immutable object trade-offs</strong><br>
Immutable objects offer several advantages over mutable objects, and one potential disadvantage. First, immutable objects are often easier to reason about than mutable ones, because they do not have complex state spaces that change over time. Second, you can pass immutable objects around quite freely, whereas you may need to make defensive copies of mutable objects before passing them to other code. Third, there is no way for two threads concurrently accessing an immutable to corrupt its state once it has been properly constructed, because no thread can change the state of an immutable. Fourth, immutable objects make safe hash table keys. If a mutable object is mutated after it is placed into a HashSet, for example, that object may not be found the next time you look into the HashSet.<br>
The main disadvantage of immutable objects is that they sometimes require that a large object graph be copied where otherwise an update could be done in place. In some cases this can be awkward to express and might also cause a performance bottleneck. As a result, it is not uncommon for libraries to provide mutable alternatives to immutable classes. For example, class StringBuilder is a mutable alternative to the immutable String. We’ll give you more information on designing mutable objects in Scala in Chapter 18.</p>
</blockquote>
<blockquote>
<p><strong>NOTE:</strong><br>
This initial Rational example highlights a difference between Java and Scala. In Java, classes have constructors, which can take parameters, whereas in Scala, classes can take parameters directly. The Scala notation is more concise—class parameters can be used directly in the body of the class; there’s no need to define fields and write assignments that copy constructor parameters into fields. This can yield substantial savings in boilerplate code, especially for small classes.</p>
</blockquote>
<figure class="highlight scala"><table><tr><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Rational</span>(<span class="params">n: <span class="type">Int</span>, d: <span class="type">Int</span></span>) </span>&#123;</span><br><span class="line"> println(<span class="string">"Created "</span>+ n +<span class="string">"/"</span>+ d)</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<p>The Scala compiler will compile any code you place in the class body, which isn’t part of a field or a method definition, into the primary constructor</p>
<h3>Reimplementing the toString method</h3>
<figure class="highlight scala"><table><tr><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Rational</span>(<span class="params">n: <span class="type">Int</span>, d: <span class="type">Int</span></span>) </span>&#123;</span><br><span class="line"> <span class="keyword">override</span> <span class="function"><span class="keyword">def</span> <span class="title">toString</span> </span>= n +<span class="string">"/"</span>+ d</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<p>The override modifier in front of a method definition signals that a previous method definition is overridden</p>
<h3>Checking preconditions</h3>
<p>A precondition is a constraint on values passed into a method or constructor, a requirement which callers must fulfill</p>
<figure class="highlight scala"><table><tr><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Rational</span>(<span class="params">n: <span class="type">Int</span>, d: <span class="type">Int</span></span>) </span>&#123;</span><br><span class="line"> require(d != <span class="number">0</span>)</span><br><span class="line"> <span class="keyword">override</span> <span class="function"><span class="keyword">def</span> <span class="title">toString</span> </span>= n +<span class="string">"/"</span>+ d</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<h3>Adding fields</h3>
<figure class="highlight scala"><table><tr><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Rational</span>(<span class="params">n: <span class="type">Int</span>, d: <span class="type">Int</span></span>) </span>&#123; <span class="comment">// This won’t compile</span></span><br><span class="line"> require(d != <span class="number">0</span>)</span><br><span class="line"> <span class="keyword">override</span> <span class="function"><span class="keyword">def</span> <span class="title">toString</span> </span>= n +<span class="string">"/"</span>+ d</span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">add</span></span>(that: <span class="type">Rational</span>): <span class="type">Rational</span> = <span class="keyword">new</span> <span class="type">Rational</span>(n * that.d + that.n * d, d * that.d)</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<p>Although class parameters n and d are in scope in the code of your add method, you can only access their value on the object on which add was invoked. Thus, when you say n or d in add’s implementation, the compiler is happy to provide you with the values for these class parameters. But it won’t let you say that.n or that.d, because that does not refer to the Rational object on which add was invoked</p>
<figure class="highlight scala"><table><tr><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Rational</span>(<span class="params">n: <span class="type">Int</span>, d: <span class="type">Int</span></span>) </span>&#123;</span><br><span class="line"> require(d != <span class="number">0</span>)</span><br><span class="line"> <span class="keyword">val</span> numer: <span class="type">Int</span> = n</span><br><span class="line"> <span class="keyword">val</span> denom: <span class="type">Int</span> = d</span><br><span class="line"> <span class="keyword">override</span> <span class="function"><span class="keyword">def</span> <span class="title">toString</span> </span>= numer +<span class="string">"/"</span>+ denom</span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">add</span></span>(that: <span class="type">Rational</span>): <span class="type">Rational</span> =</span><br><span class="line"> <span class="keyword">new</span> <span class="type">Rational</span>(</span><br><span class="line"> numer * that.denom + that.numer * denom,</span><br><span class="line"> denom * that.denom</span><br><span class="line"> )</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<h3>Self references</h3>
<p>The keyword <code>this</code> refers to the object instance on which the currently executing method was invoked, or if used in a constructor, the object instance being constructed</p>
<h3>Auxiliary constructors</h3>
<p>Auxiliary constructors in Scala start with def this(...).</p>
<figure class="highlight scala"><table><tr><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Rational</span>(<span class="params">n: <span class="type">Int</span>, d: <span class="type">Int</span></span>) </span>&#123;</span><br><span class="line"> require(d != <span class="number">0</span>)</span><br><span class="line"> <span class="keyword">val</span> numer: <span class="type">Int</span> = n</span><br><span class="line"> <span class="keyword">val</span> denom: <span class="type">Int</span> = d</span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">this</span></span>(n: <span class="type">Int</span>) = <span class="keyword">this</span>(n, <span class="number">1</span>) <span class="comment">// auxiliary constructor</span></span><br><span class="line">...</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<blockquote>
<p><strong>The primary constructor is thus the single point of entry of a class.</strong><br>
In Scala, every auxiliary constructor must invoke another constructor of the same class as its first action. In other words, the first statement in every auxiliary constructor in every Scala class will have the form “this(. . . )”. The invoked constructor is either the primary constructor (as in the Rational example), or another auxiliary constructor that comes textually before the calling constructor. The net effect of this rule is that every constructor invocation in Scala will end up eventually calling the primary constructor of the class.</p>
</blockquote>
<blockquote>
<p><strong>NOTE</strong><br>
If you’re familiar with Java, you may wonder why Scala’s rules for constructors are a bit more restrictive than Java’s. In Java, a constructor must either invoke another constructor of the same class, or directly invoke a constructor of the superclass, as its first action. In a Scala class, only the primary constructor can invoke a superclass constructor. The increased restriction in Scala is really a design trade-off that needed to be paid in exchange for the greater conciseness and simplicity of Scala’s constructors compared to Java’s.</p>
</blockquote>
<h3>Private fields and methods</h3>
<p>make a field or method private you simply place the private keyword in front of its definition.</p>
<h3>Defining operators</h3>
<p>The first step is to replace add by the usual mathematical symbol</p>
<figure class="highlight scala"><table><tr><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Rational</span>(<span class="params">n: <span class="type">Int</span>, d: <span class="type">Int</span></span>) </span>&#123;</span><br><span class="line"> require(d != <span class="number">0</span>)</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">val</span> g = gcd(n.abs, d.abs)</span><br><span class="line"> <span class="keyword">val</span> numer = n / g</span><br><span class="line"> <span class="keyword">val</span> denom = d / g</span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">this</span></span>(n: <span class="type">Int</span>) = <span class="keyword">this</span>(n, <span class="number">1</span>)</span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">+</span> </span>(that: <span class="type">Rational</span>): <span class="type">Rational</span> =</span><br><span class="line"> <span class="keyword">new</span> <span class="type">Rational</span>(</span><br><span class="line"> numer * that.denom + that.numer * denom,</span><br><span class="line"> denom * that.denom</span><br><span class="line"> )</span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">*</span> </span>(that: <span class="type">Rational</span>): <span class="type">Rational</span> =</span><br><span class="line"> <span class="keyword">new</span> <span class="type">Rational</span>(numer * that.numer, denom * that.denom)</span><br><span class="line"> <span class="keyword">override</span> <span class="function"><span class="keyword">def</span> <span class="title">toString</span> </span>= numer +<span class="string">"/"</span>+ denom</span><br><span class="line"> <span class="keyword">private</span> <span class="function"><span class="keyword">def</span> <span class="title">gcd</span></span>(a: <span class="type">Int</span>, b: <span class="type">Int</span>): <span class="type">Int</span> =</span><br><span class="line"> <span class="keyword">if</span> (b == <span class="number">0</span>) a <span class="keyword">else</span> gcd(b, a % b)</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<h3>Identifiers in Scala</h3>
<h4>alphanumeric identifier</h4>
<p><strong>An alphanumeric identifier starts with a letter or underscore, which can be followed by further letters, digits, or underscores.</strong> The ‘$’ character also counts as a letter, however it is reserved for identifiers generated by the Scala compiler. Identifiers in user programs should not contain ‘$’ characters, even though it will compile; if they do this might lead to name clashes with identifiers generated by the Scala compiler<br>
Scala follows Java’s convention of using camel-case identifiers.Although underscores are legal in identifiers, they are not used that often in Scala programs, in part to be consistent with Java,but also because underscores have many other non-identifier uses in Scala code. As a result, <strong>it is best to avoid identifiers like to_string, __init__, or name_.</strong></p>
<blockquote>
<p><strong>Note</strong><br>
One consequence of using a trailing underscore in an identifier is that if you attempt, for example, to write a declaration like this, “val name_: Int = 1”, you’ll get a compiler error. The compiler will think you are trying to declare a val named “name_:”. To get this to compile, you would need to insert an extra space before the colon, as in: “val name_ : Int = 1”.</p>
</blockquote>
<p><strong>One way in which Scala’s conventions depart from Java’s involves constant names.</strong><br>
In Java, the convention is to give constants names that are all upper case, with underscores separating the words, such as MAX_VALUE or PI.<br>
In Scala, the convention is merely that the first character should be upper case.<br>
Thus, constants named in the Java style, such as X_OFFSET, will work as Scala constants, but the Scala convention is to use camel case for constants, such as XOffset.</p>
<h4>operator identifier</h4>
<p>An operator identifier consists of one or more operator characters. Operator characters are printable ASCII characters such as <code>+, :, ?, ~ or #</code>. Here are some examples of operator identifiers:<code>+ ++ ::: &lt;?&gt; :-&gt;</code><br>
The Scala compiler will internally “mangle” operator identifiers to turn them into legal Java identifiers with embedded $ characters. For instance, the identifier :-&gt; would be represented internally as $colon$minus$greater.<br>
<strong>Because operator identifiers in Scala can become arbitrarily long, there is a small difference between Java and Scala</strong><br>
In Java, the input x&lt;-y would be parsed as four lexical symbols, so it would be equivalent to x &lt; - y. In Scala, &lt;- would be parsed as a single identifier, giving x &lt;- y. If you want the first interpretation, you need to separate the &lt; and the - characters by a space.</p>
<h4>mixed identifier</h4>
<p>A mixed identifier consists of an alphanumeric identifier, which is followed by an underscore and an operator identifier. For example, unary_+ used as a method name defines a unary + operator. Or, myvar_= used as method name defines an assignment operator.</p>
<h4>literal identifier</h4>
<p>A literal identifier is an arbitrary string enclosed in back ticks (<code>. . .</code>). Some examples of literal identifiers are:<br>
`x` `<clinit>` `yield`<br>
The idea is that you can put any string that’s accepted by the runtime as an identifier between back ticks. The result is always a Scala identifier. This works even if the name contained in the back ticks would be a Scala reserved word.</clinit></p>
<h3>Method overloading</h3>
<blockquote>
<p><strong>Note:</strong><br>
Scala’s process of overloaded method resolution is very similar to Java’s. In every case, the chosen overloaded version is the one that best matches the static types of the arguments. Sometimes there is no unique best matching version; in that case the compiler will give you an “ambiguous reference” error.</p>
</blockquote>
<h3>Implicit conversions</h3>
<p>You can create an implicit conversion that automatically converts integers to rational numbers when needed. Try adding this line in the interpreter:</p>
<figure class="highlight scala"><table><tr><td class="code"><pre><span class="line"><span class="keyword">implicit</span> <span class="function"><span class="keyword">def</span> <span class="title">intToRational</span></span>(x: <span class="type">Int</span>) = <span class="keyword">new</span> <span class="type">Rational</span>(x)</span><br></pre></td></tr></table></figure>
<p>Note that for an implicit conversion to work, it needs to be in scope. If you place the implicit method definition inside class Rational, it won’t be in scope in the interpreter. For now, you’ll need to define it directly in the interpreter.</p>
<h3>A word of caution</h3>
<p>As this chapter has demonstrated, creating methods with operator names and defining implicit conversions can help you design libraries for which client code is concise and easy to understand. Scala gives you a great deal of power to design such easy-to-use libraries, but please bear in mind that with power comes responsibility<br>
If used unartfully, both operator methods and implicit conversions can give rise to client code that is hard to read and understand. Because implicit conversions are applied implicitly by the compiler, not explicitly written down in the source code, it can be non-obvious to client programmers what implicit conversions are being applied. And although operator methods will usually make client code more concise, they will only make it more readable to the extent client programmers will be able to recognize and remember the meaning of each operator.<br>
The goal you should keep in mind as you design libraries is not merely enabling concise client code, but readable, understandable client code. Conciseness will often be a big part of that readability, but you can take conciseness too far. By designing libraries that enable tastefully concise and at the same time understandable client code, you can help those client programmers work productively.</p>
</content>
<summary type="html">
<h3>Constructing a Rational</h3>
<figure class="highlight scala"><table><tr><td class="code"><pre><span class="line"><span class="class"><sp
</summary>
<category term="Scala" scheme="http://dangxia.github.io/categories/Scala/"/>
<category term="Programming in Scala" scheme="http://dangxia.github.io/tags/Programming-in-Scala/"/>
</entry>
<entry>
<title>Chapter5:Basic Types and Operations</title>
<link href="http://dangxia.github.io/2015/12/06/scala/Basic-Types-and-Operations/"/>
<id>http://dangxia.github.io/2015/12/06/scala/Basic-Types-and-Operations/</id>
<published>2015-12-06T03:21:24.000Z</published>
<updated>2016-04-15T14:31:23.994Z</updated>
<content type="html"><h3>Literals</h3>
<ul>
<li>String literals</li>
</ul>
<p>Scala includes a special syntaxfor raw strings. You start and end a <code>raw string</code> with three double quotationmarks in a row (&quot;&quot;&quot;). The interior of a raw string may contain any characterswhatsoever, including newlines, quotation marks, and special characters, except of course three quotes in a row.</p>
<figure class="highlight"><figcaption><span>example</span></figcaption><table><tr><td class="code"><pre><span class="line">println("""Welcome to Ultamix 3000.</span><br><span class="line"> Type "HELP" for help.""")</span><br><span class="line">//Welcome to Ultamix 3000.</span><br><span class="line">// Type "HELP" for help.</span><br></pre></td></tr></table></figure>
<figure class="highlight"><figcaption><span>put a pipe character (|) at the front of each line,then call stripMargin on strings,to escape the leading spaces</span></figcaption><table><tr><td class="code"><pre><span class="line">println("""|Welcome to Ultamix 3000.</span><br><span class="line">|Type "HELP" for help.""".stripMargin)</span><br><span class="line">//Welcome to Ultamix 3000.</span><br><span class="line">//Type "HELP" for help.""".stripMargin</span><br></pre></td></tr></table></figure>
<ul>
<li>Symbol literals</li>
</ul>
<p>A symbol literal is written 'ident, where ident can be any alphanumericidentifier. Such literals are mapped to instances of the predefined classscala.Symbol. Specifically, the literal 'cymbal will be expanded by thecompiler to a factory method invocation: Symbol(&quot;cymbal&quot;).</p>
<blockquote>
<p><strong>NOTE:</strong><br>
Another thing that’s noteworthy is that symbols are interned(限定的). If you write the same symbol literal twice, both expressions will refer to the exact same Symbol object.</p>
</blockquote>
<h3>Operators and methods</h3>
<p>Scala provides a rich set of operators for its basic types. As mentioned inprevious chapters, these operators are actually just a nice syntax for ordinary method calls.</p>
<blockquote>
<p><strong>Any method can be an operator</strong><br>
In Scala operators are not special language syntax: any method can be an operator. What makes a method an operator is how you use it. When you write “s.indexOf('o')”, indexOf is not an operator. But when you write “s indexOf 'o'”, indexOf is an operator, because you’re using it in operator notation.</p>
</blockquote>
<ul>
<li>infix operator</li>
</ul>
<figure class="highlight scala"><table><tr><td class="code"><pre><span class="line"><span class="comment">//single argument</span></span><br><span class="line">s indexOf 'o'</span><br><span class="line"><span class="comment">//multiple arguments</span></span><br><span class="line">s indexOf ('o', <span class="number">5</span>)</span><br></pre></td></tr></table></figure>
<ul>
<li>prefix operator</li>
</ul>
<p>Scala will transform the expression -2.0 into the method invocation “(2.0).unary_-”</p>
<blockquote>
<p><strong>NOTE</strong>:The only identifiers that can be used as prefix operators are<code>+, -, !, and ~</code>.Thus, if you define a method named unary_!, you could invoke that methodon a value or variable of the appropriate type using prefix operator notation,such as !p. But if you define a method named unary_*, you wouldn’t be ableto use prefix operator notation, because * isn’t one of the four identifiers thatcan be used as prefix operators.</p>
</blockquote>
<ul>
<li>postfix operator</li>
</ul>
<p>Postfix operators are methods that take no arguments, when they are invoked without a dot or parentheses.</p>
<h3>Relational and logical operations</h3>
<blockquote>
<p><strong>Note:</strong> You may be wondering how short-circuiting can work given operators are just methods. Normally, all arguments are evaluated before entering a method, so how can a method avoid evaluating its second argument? The answer is that all Scala methods have a facility for delaying the evaluation of their arguments, or even declining to evaluate them at all. The facility is called by-name parameters and is discussed in Section 9.5.</p>
</blockquote>
<h3>Object equality</h3>
<p>If you want to compare two objects for equality, you can use either ==, or its inverse !=.</p>
<blockquote>
<p>**How Scala’s == differs from Java’s **<br>
In Java,you can use == to compare both primitive and reference types. On primitive types, Java’s == compares value equality, as in Scala. On reference types, however, Java’s == compares reference equality, which means the two variables point to the same object on the JVM’s heap. Scala provides a facility for comparing reference equality, as well, under the name eq. However, eq and its opposite, ne, only apply to objects that directly map to Java objects. The full details about eq and ne are given in Sections 11.1 and 11.2. Also, see Chapter 30 on how to write a good equals method.</p>
</blockquote>
<h3>Operator precedence and associativity</h3>
<table>
<thead>
<tr>
<th>Operator precedence</th>
</tr>
</thead>
<tbody>
<tr>
<td>(all other special characters)</td>
</tr>
<tr>
<td>* / %</td>
</tr>
<tr>
<td>+ -</td>
</tr>
<tr>
<td>:</td>
</tr>
<tr>
<td>= !</td>
</tr>
<tr>
<td>&lt; &gt;</td>
</tr>
<tr>
<td>&amp; ˆ |</td>
</tr>
<tr>
<td>(all letters)</td>
</tr>
<tr>
<td>(all assignment operators)</td>
</tr>
</tbody>
</table>
<ul>
<li>Scala decides precedence based on the first character of the methods used in operator notation</li>
</ul>
<figure class="highlight scala"><table><tr><td class="code"><pre><span class="line">a +++ b *** c <span class="comment">//a +++ (b *** c)</span></span><br></pre></td></tr></table></figure>
<ul>
<li>The one exception to the precedence rule</li>
</ul>
<p>The one exception to the precedence rule, alluded to above, concerns <strong>assignment operators</strong>, which end in an equals character. If an operator ends in an equals character (=), and the operator is not one of the comparison operators &lt;=, &gt;=, ==, or !=, then the precedence of the operator is the same as that of <strong>simple assignment (=)</strong>.</p>
<figure class="highlight scala"><figcaption><span>because *= is classified as an assignment operator whose precedence is lower than +, even though the operator’s first character is *, which would suggest a precedence higher than +.</span></figcaption><table><tr><td class="code"><pre><span class="line">x *= y + <span class="number">1</span> <span class="comment">//x *= (y + 1)</span></span><br></pre></td></tr></table></figure>
<ul>
<li>associativity</li>
</ul>
<p>No matter what associativity an operator has, however, its operands are always evaluated left to right<br>
If the methods end in ‘:’, they are grouped right to left; otherwise, they are grouped left to right. For example,<code>a ::: b ::: c</code> is treated as <code>a ::: (b ::: c)</code>. But <code>a * b * c</code>, by contrast, is treated as <code>(a * b) * c</code>.</p>
<h3>Rich wrappers</h3>
<p><strong>Some rich operations</strong></p>
<table>
<thead>
<tr>
<th>Code</th>
<th>Result</th>
</tr>
</thead>
<tbody>
<tr>
<td>0 max 5</td>
<td>5</td>
</tr>
<tr>
<td>0 min 5</td>
<td>0</td>
</tr>
<tr>
<td>-2.7 abs</td>
<td>2.7</td>
</tr>
<tr>
<td>-2.7 round</td>
<td>-3L</td>
</tr>
<tr>
<td>1.5 isInfinity</td>
<td>false</td>
</tr>
<tr>
<td>(1.0 / 0) isInfinity</td>
<td>true</td>
</tr>
<tr>
<td>4 to 6</td>
<td>Range(4, 5, 6)</td>
</tr>
<tr>
<td>&quot;bob&quot; capitalize</td>
<td>&quot;Bob&quot;</td>
</tr>
<tr>
<td>&quot;robert&quot; drop 2</td>
<td>&quot;bert&quot;</td>
</tr>
</tbody>
</table>
<p><strong>Rich wrapper classes</strong></p>
<table>
<thead>
<tr>
<th>Basic</th>
<th>type Rich wrapper</th>
</tr>
</thead>
<tbody>
<tr>
<td>Byte</td>
<td>scala.runtime.RichByte</td>
</tr>
<tr>
<td>Short</td>
<td>scala.runtime.RichShort</td>
</tr>
<tr>
<td>Int</td>
<td>scala.runtime.RichInt</td>
</tr>
<tr>
<td>Char</td>
<td>scala.runtime.RichChar</td>
</tr>
<tr>
<td>Float</td>
<td>scala.runtime.RichFloat</td>
</tr>
<tr>
<td>Double</td>
<td>scala.runtime.RichDouble</td>
</tr>
<tr>
<td>Boolean</td>
<td>scala.runtime.RichBoolean</td>
</tr>
<tr>
<td>String</td>
<td>scala.collection.immutable.StringOps</td>
</tr>
</tbody>
</table>
</content>
<summary type="html">
<h3>Literals</h3>
<ul>
<li>String literals</li>
</ul>
<p>Scala includes a special syntaxfor raw strings. You start and end a <code>raw strin
</summary>
<category term="Scala" scheme="http://dangxia.github.io/categories/Scala/"/>
<category term="Programming in Scala" scheme="http://dangxia.github.io/tags/Programming-in-Scala/"/>
</entry>
<entry>
<title>MasterBatchCoordinator源码</title>
<link href="http://dangxia.github.io/2015/12/02/jstorm/JStormMasterBatchCoordinator-Code/"/>
<id>http://dangxia.github.io/2015/12/02/jstorm/JStormMasterBatchCoordinator-Code/</id>
<published>2015-12-02T06:46:49.000Z</published>
<updated>2016-04-15T14:31:23.990Z</updated>
<content type="html"><h4>CoordinatorState Zk</h4>
<figure class="highlight haml"><table><tr><td class="code"><pre><span class="line">$&#123;spout_id&#125;</span><br><span class="line"> -<span class="ruby">coordinator</span><br><span class="line"></span> -<span class="ruby">currtx 当前txid</span><br><span class="line"></span> -<span class="ruby">currattempts txid =&gt; attempt</span><br><span class="line"></span> -<span class="ruby">meta</span><br><span class="line"></span> -<span class="ruby">txid1 存储的metaData</span><br><span class="line"></span> -<span class="ruby">txid2</span><br><span class="line"></span> -<span class="ruby">user</span></span><br></pre></td></tr></table></figure>
<figure class="highlight java"><figcaption><span>AttemptStatus</span></figcaption><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">enum</span> AttemptStatus &#123;</span><br><span class="line"> PROCESSING, PROCESSED, COMMITTING</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">//multi spout state relate to zk</span></span><br><span class="line"><span class="keyword">private</span> List&lt;TransactionalState&gt; _states = <span class="keyword">new</span> ArrayList();</span><br><span class="line"><span class="comment">//active Tx txid =&gt; &#123;status,attempt&#125;</span></span><br><span class="line"><span class="comment">//processing--[first ack]--&gt;processed--[second ack]--&gt;committing--[third ack]--&gt;&#123;removed,_currTransaction++&#125;</span></span><br><span class="line"><span class="comment">//any status --[fail ack]--&gt;&#123;remove all larger or equals tx&#125;</span></span><br><span class="line">TreeMap&lt;Long, TransactionStatus&gt; _activeTx = <span class="keyword">new</span> TreeMap&lt;Long, TransactionStatus&gt;();</span><br><span class="line">TreeMap&lt;Long, Integer&gt; _attemptIds;</span><br><span class="line"></span><br><span class="line"><span class="keyword">private</span> SpoutOutputCollector _collector;</span><br><span class="line">Long _currTransaction;</span><br><span class="line"><span class="keyword">int</span> _maxTransactionActive;</span><br><span class="line"></span><br><span class="line">List&lt;ITridentSpout.BatchCoordinator&gt; _coordinators = <span class="keyword">new</span> ArrayList();</span><br><span class="line"></span><br><span class="line">List&lt;String&gt; _managedSpoutIds;</span><br><span class="line">List&lt;ITridentSpout&gt; _spouts;</span><br><span class="line">WindowedTimeThrottler _throttler;</span><br></pre></td></tr></table></figure>
</content>
<summary type="html">
<h4>CoordinatorState Zk</h4>
<figure class="highlight haml"><table><tr><td class="code"><pre><span class="line">$&#123;spout_id&#125;</span>
</summary>
<category term="jstorm" scheme="http://dangxia.github.io/categories/jstorm/"/>
<category term="jstorm" scheme="http://dangxia.github.io/tags/jstorm/"/>
</entry>
<entry>
<title>TridentTopology源码</title>
<link href="http://dangxia.github.io/2015/11/30/jstorm/JStorm-TridentTopology-Code/"/>
<id>http://dangxia.github.io/2015/11/30/jstorm/JStorm-TridentTopology-Code/</id>
<published>2015-11-30T06:46:49.000Z</published>
<updated>2016-04-15T14:31:23.990Z</updated>
<content type="html"><h4>参考:</h4>
<ul>
<li>http://www.cnblogs.com/hseagle/p/3756862.html</li>
<li>http://www.flyne.org/article/216</li>
</ul>
<p>从TridentTopology到基本的Topology有三层,下图给出一个全局的视图。下图引自<a href="http://www.cnblogs.com/hseagle/p/3490635.html" title="徽沪一郎" target="_blank" rel="external">徽沪一郎</a>的blog<br>
<img src="/img/jstorm/trident-view.png" alt="" title="trident-view"></p>
<h4>TridentTopology属性</h4>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">//一个simple directed graph</span></span><br><span class="line">DefaultDirectedGraph&lt;Node, IndexedEdge&gt; _graph;</span><br><span class="line"><span class="comment">//state.node.id=&gt;[self,stateQueryNode]</span></span><br><span class="line">Map&lt;String, List&lt;Node&gt;&gt; _colocate = <span class="keyword">new</span> HashMap();</span><br><span class="line"><span class="comment">//生成唯一的stream,state的ID</span></span><br><span class="line">UniqueIdGen _gen;</span><br></pre></td></tr></table></figure>
<h4>Node Type</h4>
<ul>
<li>Node</li>
<li>SpoutNode</li>
<li>ProcessorNode</li>
<li>PartitionNode</li>
</ul>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">enum</span> SpoutType &#123;</span><br><span class="line"> DRPC, BATCH</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<p><code>TridentTopology#newStream</code> create a BATCH <code>SpoutNode</code> with <code>ITridentSpout</code>,若不是<code>ITridentSpout</code>最终也会被包装成<code>ITridentSpout</code></p>
<ul>
<li>IRichSpout : RichSpoutBatchExecutor</li>
<li>IBatchSpout : BatchSpoutExecutor</li>
<li>IPartitionedTridentSpout : PartitionedTridentSpoutExecutor</li>
<li>IOpaquePartitionedTridentSpout : OpaquePartitionedTridentSpoutExecutor</li>
</ul>
<p><code>TridentTopology#newDRPCStream</code> create a DRPC <code>SpoutNode</code> with <code>DRPCSpout</code>,没有txid</p>
<p><code>TridentTopology#newStaticState</code> create a <code>Node</code> with <code>NodeStateInfo</code><br>
奇怪,一直以为只会创建<code>Node</code>的子类?</p>
<h3>Node name,Stream name,StreamId</h3>
<p>TridentTopology 直接创建的Node name都为空,相应的Stream的name也为空。<br>
Stream的name可以通过Stream#name(String name)来修改,但与之相依的Node的name不能相应改变。<br>
当在Stream通过operation创建新的Node时,Node的name为创建它的Stream的name。</p>
<p>Stream operation create new Node,generally,create a new stream id,except operation <code>partition</code>,new node using it's parent streamId.</p>
<h4>build</h4>
<figure class="highlight java"><figcaption><span>TridentTopology#build</span></figcaption><table><tr><td class="code"><pre><span class="line"><span class="comment">//其中mergedGroups为spoutGroup,boltGroup</span></span><br><span class="line"><span class="comment">//spoutNodes 为SpoutNode集合</span></span><br><span class="line"></span><br><span class="line"><span class="comment">//已经省略了,完成drpc的环,以及node合并成group等。</span></span><br><span class="line"></span><br><span class="line"><span class="comment">//node =&gt; batchGroupName,ex:bg0,bg1</span></span><br><span class="line"><span class="comment">//可能有多个batchGroup,亲测可以&#123;没有连接的多个拓扑图&#125;</span></span><br><span class="line">Map&lt;Node, String&gt; batchGroupMap = <span class="keyword">new</span> HashMap();</span><br><span class="line">List&lt;Set&lt;Node&gt;&gt; connectedComponents = <span class="keyword">new</span> ConnectivityInspector&lt;Node, IndexedEdge&gt;(graph).connectedSets();</span><br><span class="line"><span class="keyword">for</span> (<span class="keyword">int</span> i = <span class="number">0</span>; i &lt; connectedComponents.size(); i++) &#123;</span><br><span class="line"> String groupId = <span class="string">"bg"</span> + i;</span><br><span class="line"> <span class="keyword">for</span> (Node n : connectedComponents.get(i)) &#123;</span><br><span class="line"> batchGroupMap.put(n, groupId);</span><br><span class="line"> &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">//计算每个Group的parallelism</span></span><br><span class="line">Map&lt;Group, Integer&gt; parallelisms = getGroupParallelisms(graph, grouper, mergedGroups);</span><br><span class="line"></span><br><span class="line">TridentTopologyBuilder builder = <span class="keyword">new</span> TridentTopologyBuilder();</span><br><span class="line"></span><br><span class="line"><span class="comment">//spoutNode =&gt; spoutId ,ex:spout0,spout1 String为componentId</span></span><br><span class="line">Map&lt;Node, String&gt; spoutIds = genSpoutIds(spoutNodes);</span><br><span class="line"><span class="comment">//opretionNode =&gt; boltId,ex:b-0,b-1,b-2-kkk String为componentId</span></span><br><span class="line">Map&lt;Group, String&gt; boltIds = genBoltIds(mergedGroups);</span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> (SpoutNode sn : spoutNodes) &#123;</span><br><span class="line"> Integer parallelism = parallelisms.get(grouper.nodeGroup(sn));</span><br><span class="line"> <span class="keyword">if</span> (sn.type == SpoutNode.SpoutType.DRPC) &#123;</span><br><span class="line"> <span class="comment">//spout0,s0,IRichSpout,parallelism,bg0</span></span><br><span class="line"> builder.setBatchPerTupleSpout(spoutIds.get(sn), sn.streamId, (IRichSpout) sn.spout, parallelism,</span><br><span class="line"> batchGroupMap.get(sn));</span><br><span class="line"> &#125; <span class="keyword">else</span> &#123;</span><br><span class="line"> ITridentSpout s;</span><br><span class="line"> <span class="keyword">if</span> (sn.spout <span class="keyword">instanceof</span> IBatchSpout) &#123;</span><br><span class="line"> s = <span class="keyword">new</span> BatchSpoutExecutor((IBatchSpout) sn.spout);</span><br><span class="line"> &#125; <span class="keyword">else</span> <span class="keyword">if</span> (sn.spout <span class="keyword">instanceof</span> ITridentSpout) &#123;</span><br><span class="line"> s = (ITridentSpout) sn.spout;</span><br><span class="line"> &#125; <span class="keyword">else</span> &#123;</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> RuntimeException(</span><br><span class="line"> <span class="string">"Regular rich spouts not supported yet... try wrapping in a RichSpoutBatchExecutor"</span>);</span><br><span class="line"> <span class="comment">// <span class="doctag">TODO:</span> handle regular rich spout without batches (need</span></span><br><span class="line"> <span class="comment">// lots of updates to support this throughout)</span></span><br><span class="line"> &#125;</span><br><span class="line"> <span class="comment">//spout0,s0,txStateId,ITridentSpout,parallelism,bg0</span></span><br><span class="line"> builder.setSpout(spoutIds.get(sn), sn.streamId, sn.txId, s, parallelism, batchGroupMap.get(sn));</span><br><span class="line"> &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> (Group g : mergedGroups) &#123;</span><br><span class="line"> <span class="keyword">if</span> (!isSpoutGroup(g)) &#123;</span><br><span class="line"> Integer p = parallelisms.get(g);</span><br><span class="line"> <span class="comment">//stream to batchGroup &#123;s0=&gt;bg0&#125;</span></span><br><span class="line"> Map&lt;String, String&gt; streamToGroup = getOutputStreamBatchGroups(g, batchGroupMap);</span><br><span class="line"> <span class="comment">//b-0,SubtopologyBolt(graph, g.nodes, batchGroupMap),parallelism,&#123;bg0&#125;,&#123;s0=&gt;bg0&#125;</span></span><br><span class="line"> BoltDeclarer d = builder.setBolt(boltIds.get(g), <span class="keyword">new</span> SubtopologyBolt(graph, g.nodes, batchGroupMap), p,</span><br><span class="line"> committerBatches(g, batchGroupMap), streamToGroup);</span><br><span class="line"> Collection&lt;PartitionNode&gt; inputs = uniquedSubscriptions(externalGroupInputs(g));</span><br><span class="line"> <span class="keyword">for</span> (PartitionNode n : inputs) &#123;</span><br><span class="line"> Node parent = TridentUtils.getParent(graph, n);</span><br><span class="line"> String componentId;</span><br><span class="line"> <span class="keyword">if</span> (parent <span class="keyword">instanceof</span> SpoutNode) &#123;</span><br><span class="line"> componentId = spoutIds.get(parent);</span><br><span class="line"> &#125; <span class="keyword">else</span> &#123;</span><br><span class="line"> componentId = boltIds.get(grouper.nodeGroup(parent));</span><br><span class="line"> &#125;</span><br><span class="line"> d.grouping(<span class="keyword">new</span> GlobalStreamId(componentId, n.streamId), n.thriftGrouping);</span><br><span class="line"> &#125;</span><br><span class="line"> &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">return</span> builder.buildTopology();</span><br></pre></td></tr></table></figure>
<figure class="highlight java"><figcaption><span>TridentTopologyBuilder#setBatchPerTupleSpout</span></figcaption><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> SpoutDeclarer <span class="title">setBatchPerTupleSpout</span><span class="params">(String id, String streamName, IRichSpout spout, Integer parallelism,</span><br><span class="line"> String batchGroup)</span> </span>&#123;</span><br><span class="line"> Map&lt;String, String&gt; batchGroups = <span class="keyword">new</span> HashMap();</span><br><span class="line"> batchGroups.put(streamName, batchGroup);</span><br><span class="line"> <span class="comment">//_batchIds.put(new GlobalStreamId(id, streamName), batchGroup);GlobalStreamId=&gt;batchGroup</span></span><br><span class="line"> markBatchGroups(id, batchGroups);</span><br><span class="line"> SpoutComponent c = <span class="keyword">new</span> SpoutComponent(spout, streamName, parallelism, batchGroup);</span><br><span class="line"> <span class="comment">//componentId =&gt; SpoutComponent</span></span><br><span class="line"> _batchPerTupleSpouts.put(id, c);</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> SpoutDeclarerImpl(c);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<figure class="highlight java"><figcaption><span>TridentTopologyBuilder#setSpout</span></figcaption><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> SpoutDeclarer <span class="title">setSpout</span><span class="params">(String id, String streamName, String txStateId, ITridentSpout spout,</span><br><span class="line"> Integer parallelism, String batchGroup)</span> </span>&#123;</span><br><span class="line"> Map&lt;String, String&gt; batchGroups = <span class="keyword">new</span> HashMap();</span><br><span class="line"> batchGroups.put(streamName, batchGroup);</span><br><span class="line"> markBatchGroups(id, batchGroups);</span><br><span class="line"></span><br><span class="line"> TransactionalSpoutComponent c = <span class="keyword">new</span> TransactionalSpoutComponent(spout, streamName, parallelism, txStateId,</span><br><span class="line"> batchGroup);</span><br><span class="line"> _spouts.put(id, c);</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> SpoutDeclarerImpl(c);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
</content>
<summary type="html">
<h4>参考:</h4>
<ul>
<li>http://www.cnblogs.com/hseagle/p/3756862.html</li>
<li>http://www.flyne.org/article/216</li>
</ul>
<p>从TridentTopology
</summary>
<category term="jstorm" scheme="http://dangxia.github.io/categories/jstorm/"/>
<category term="jstorm" scheme="http://dangxia.github.io/tags/jstorm/"/>
</entry>
<entry>
<title>TopologyBuilder源码</title>
<link href="http://dangxia.github.io/2015/11/30/jstorm/JStorm-TopologyBuilder-Code/"/>
<id>http://dangxia.github.io/2015/11/30/jstorm/JStorm-TopologyBuilder-Code/</id>
<published>2015-11-30T06:46:49.000Z</published>
<updated>2016-04-15T14:31:23.990Z</updated>
<content type="html"><h4>TopologyBuilder功能:</h4>
<p>编织<code>ISpout,IBolt</code>的拓扑图,并生成StormTopology实例,以便thrift使用.</p>
<h4>用户编程的接口:IBolt,ISpout,IStateSpout</h4>
<p><code>IRichBolt,IBasicBolt,IRichSpout,IRichStateSpout</code></p>
<ul>
<li><code>IBasicBolt</code>没有继承<code>IBolt</code>,在编织拓扑结构时,使用<code>BasicBoltExecutor</code>包装成<code>IRichBolt</code></li>
</ul>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> BoltDeclarer <span class="title">setBolt</span><span class="params">(String id, IBasicBolt bolt, Number parallelism_hint)</span> </span>&#123;</span><br><span class="line"> <span class="keyword">return</span> setBolt(id, <span class="keyword">new</span> BasicBoltExecutor(bolt), parallelism_hint);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<ul>
<li><code>IRichStateSpout</code>暂时没有实现</li>
</ul>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">setStateSpout</span><span class="params">(String id, IRichStateSpout stateSpout, Number parallelism_hint)</span> </span>&#123;</span><br><span class="line"> validateUnusedId(id);</span><br><span class="line"> <span class="comment">// <span class="doctag">TODO:</span> finish</span></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<a id="more"></a>
<h4>编织拓扑图<code>BoltDeclarer,SpoutDeclarer</code></h4>
<p><code>SpoutDeclarer</code>配置<code>IRichSpout</code><br>
<code>BoltDeclarer</code>配置<code>IRichBolt</code>并,通过grouping,配置<code>ComponentCommon</code>的<code>inputs</code>等。</p>
<h4><code>TopologyBuilder</code>与<code>IComponent</code></h4>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span><br><span class="line"> * Common methods for all possible components in a topology. This interface is used when defining topologies using the Java API.</span><br><span class="line"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">interface</span> <span class="title">IComponent</span> <span class="keyword">extends</span> <span class="title">Serializable</span> </span>&#123;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span><br><span class="line"> * Declare the output schema for all the streams of this topology.</span><br><span class="line"> *</span><br><span class="line"> * <span class="doctag">@param</span> declarer this is used to declare output stream ids, output fields, and whether or not each output stream is a direct stream</span><br><span class="line"> */</span></span><br><span class="line"> <span class="function"><span class="keyword">void</span> <span class="title">declareOutputFields</span><span class="params">(OutputFieldsDeclarer declarer)</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span><br><span class="line"> * Declare configuration specific to this component. Only a subset of the "topology.*" configs can be overridden. The component configuration can be further</span><br><span class="line"> * overridden when constructing the topology using &#123;<span class="doctag">@link</span> TopologyBuilder&#125;</span><br><span class="line"> *</span><br><span class="line"> */</span></span><br><span class="line"> <span class="function">Map&lt;String, Object&gt; <span class="title">getComponentConfiguration</span><span class="params">()</span></span>;</span><br><span class="line"></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<p>TopologyBuilder#initCommon,调用ComponentConfiguration到ComponentCommon的json_conf中</p>
<figure class="highlight java"><figcaption><span>TopologyBuilder#initCommon</span></figcaption><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">initCommon</span><span class="params">(String id, IComponent component, Number parallelism)</span> </span>&#123;</span><br><span class="line"> ComponentCommon common = <span class="keyword">new</span> ComponentCommon();</span><br><span class="line"> common.set_inputs(<span class="keyword">new</span> HashMap&lt;GlobalStreamId, Grouping&gt;());</span><br><span class="line"> <span class="keyword">if</span> (parallelism != <span class="keyword">null</span>) &#123;</span><br><span class="line"> common.set_parallelism_hint(parallelism.intValue());</span><br><span class="line"> &#125; <span class="keyword">else</span> &#123;</span><br><span class="line"> common.set_parallelism_hint(<span class="number">1</span>);</span><br><span class="line"> &#125;</span><br><span class="line"> Map conf = component.getComponentConfiguration();</span><br><span class="line"> <span class="keyword">if</span> (conf != <span class="keyword">null</span>)</span><br><span class="line"> common.set_json_conf(JSONValue.toJSONString(conf));</span><br><span class="line"> _commons.put(id, common);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<p>TopologyBuilder#getComponentCommon,调用getFieldsDeclaration到ComponentCommon的streams中</p>
<figure class="highlight java"><figcaption><span>TopologyBuilder#getComponentCommon</span></figcaption><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">private</span> ComponentCommon <span class="title">getComponentCommon</span><span class="params">(String id, IComponent component)</span> </span>&#123;</span><br><span class="line"> ComponentCommon ret = <span class="keyword">new</span> ComponentCommon(_commons.get(id));</span><br><span class="line"></span><br><span class="line"> OutputFieldsGetter getter = <span class="keyword">new</span> OutputFieldsGetter();</span><br><span class="line"> component.declareOutputFields(getter);</span><br><span class="line"> ret.set_streams(getter.getFieldsDeclaration());</span><br><span class="line"> <span class="keyword">return</span> ret;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<h4>重点关注<code>ComponentCommon</code></h4>
<figure class="highlight thrift"><table><tr><td class="code"><pre><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">ComponentCommon</span> </span>&#123;</span><br><span class="line"> <span class="number">1</span>: <span class="keyword">required</span> map&lt;GlobalStreamId, Grouping&gt; inputs;</span><br><span class="line"> <span class="number">2</span>: <span class="keyword">required</span> map&lt;<span class="keyword">string</span>, StreamInfo&gt; streams; <span class="comment">//key is stream id</span></span><br><span class="line"> <span class="number">3</span>: <span class="keyword">optional</span> <span class="built_in">i32</span> parallelism_hint; <span class="comment">//how many threads across the cluster should be dedicated to this component</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">// component specific configuration respects:</span></span><br><span class="line"> <span class="comment">// topology.de<span class="doctag">bug:</span> false</span></span><br><span class="line"> <span class="comment">// topology.max.task.parallelism: null // can replace isDistributed with this</span></span><br><span class="line"> <span class="comment">// topology.max.spout.pending: null</span></span><br><span class="line"> <span class="comment">// topology.kryo.register // this is the only additive one</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">// component specific configuration</span></span><br><span class="line"> <span class="number">4</span>: <span class="keyword">optional</span> <span class="built_in">string</span> json_conf;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<p><code>IComponent</code>是通过主动push的方式,这就涉及到问题push给谁?<br>
从<code>ComponentCommon</code>定义来看,它只声明了生成的stream的相关信息。</p>
<figure class="highlight java"><figcaption><span>Task#makeSendTargets</span></figcaption><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">private</span> TaskSendTargets <span class="title">makeSendTargets</span><span class="params">()</span> </span>&#123;</span><br><span class="line"> String component = topologyContext.getThisComponentId();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// get current task's output</span></span><br><span class="line"> <span class="comment">// &lt;Stream_id,&lt;component, Grouping&gt;&gt;</span></span><br><span class="line"> Map&lt;String, Map&lt;String, MkGrouper&gt;&gt; streamComponentGrouper = Common.outbound_components(topologyContext, workerData);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> TaskSendTargets(stormConf, component, streamComponentGrouper, topologyContext, taskStats);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<p><code>Common#outbound_components</code>从<code>ComponentCommon</code>声明中的谁消费什么,转变为我push给谁。</p>
<figure class="highlight java"><figcaption><span>Common#outbound_components</span></figcaption><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> Map&lt;String, Map&lt;String, MkGrouper&gt;&gt; outbound_components(TopologyContext topology_context, WorkerData workerData) &#123;</span><br><span class="line"> Map&lt;String, Map&lt;String, MkGrouper&gt;&gt; rr = <span class="keyword">new</span> HashMap&lt;String, Map&lt;String, MkGrouper&gt;&gt;();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// &lt;Stream_id,&lt;component,Grouping&gt;&gt;</span></span><br><span class="line"> Map&lt;String, Map&lt;String, Grouping&gt;&gt; output_groupings = topology_context.getThisTargets();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (Entry&lt;String, Map&lt;String, Grouping&gt;&gt; entry : output_groupings.entrySet()) &#123;</span><br><span class="line"></span><br><span class="line"> String stream_id = entry.getKey();</span><br><span class="line"> Map&lt;String, Grouping&gt; component_grouping = entry.getValue();</span><br><span class="line"></span><br><span class="line"> Fields out_fields = topology_context.getThisOutputFields(stream_id);</span><br><span class="line"></span><br><span class="line"> Map&lt;String, MkGrouper&gt; componentGrouper = <span class="keyword">new</span> HashMap&lt;String, MkGrouper&gt;();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (Entry&lt;String, Grouping&gt; cg : component_grouping.entrySet()) &#123;</span><br><span class="line"></span><br><span class="line"> String component = cg.getKey();</span><br><span class="line"> Grouping tgrouping = cg.getValue();</span><br><span class="line"></span><br><span class="line"> List&lt;Integer&gt; outTasks = topology_context.getComponentTasks(component);</span><br><span class="line"> <span class="comment">// ATTENTION: If topology set one component parallelism as 0</span></span><br><span class="line"> <span class="comment">// so we don't need send tuple to it</span></span><br><span class="line"> <span class="keyword">if</span> (outTasks.size() &gt; <span class="number">0</span>) &#123;</span><br><span class="line"> MkGrouper grouper = <span class="keyword">new</span> MkGrouper(topology_context, out_fields, tgrouping, outTasks, stream_id, workerData);</span><br><span class="line"> componentGrouper.put(component, grouper);</span><br><span class="line"> &#125;</span><br><span class="line"> LOG.info(<span class="string">"outbound_components, outTasks="</span> + outTasks + <span class="string">" for task-"</span> + topology_context.getThisTaskId());</span><br><span class="line"> &#125;</span><br><span class="line"> <span class="keyword">if</span> (componentGrouper.size() &gt; <span class="number">0</span>) &#123;</span><br><span class="line"> rr.put(stream_id, componentGrouper);</span><br><span class="line"> &#125;</span><br><span class="line"> &#125;</span><br><span class="line"> <span class="keyword">return</span> rr;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<h4>thrift</h4>
<figure class="highlight thrift"><table><tr><td class="code"><pre><span class="line">union JavaObjectArg &#123;</span><br><span class="line"> <span class="number">1</span>: <span class="built_in">i32</span> int_arg;</span><br><span class="line"> <span class="number">2</span>: <span class="built_in">i64</span> long_arg;</span><br><span class="line"> <span class="number">3</span>: <span class="built_in">string</span> string_arg;</span><br><span class="line"> <span class="number">4</span>: <span class="built_in">bool</span> bool_arg;</span><br><span class="line"> <span class="number">5</span>: <span class="built_in">binary</span> binary_arg;</span><br><span class="line"> <span class="number">6</span>: <span class="built_in">double</span> double_arg;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">JavaObject</span> </span>&#123;</span><br><span class="line"> <span class="number">1</span>: <span class="keyword">required</span> <span class="built_in">string</span> full_class_name;</span><br><span class="line"> <span class="number">2</span>: <span class="keyword">required</span> list&lt;JavaObjectArg&gt; args_list;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">NullStruct</span> </span>&#123;</span><br><span class="line"></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">GlobalStreamId</span> </span>&#123;</span><br><span class="line"> <span class="number">1</span>: <span class="keyword">required</span> <span class="built_in">string</span> componentId;</span><br><span class="line"> <span class="number">2</span>: <span class="keyword">required</span> <span class="built_in">string</span> streamId;</span><br><span class="line"> #Going to need to add an <span class="class"><span class="keyword">enum</span> <span class="title">for</span> the stream type (NORMAL or FAILURE)</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">union Grouping </span>&#123;</span><br><span class="line"> <span class="number">1</span>: list&lt;<span class="keyword">string</span>&gt; fields; <span class="comment">//empty list means global grouping</span></span><br><span class="line"> <span class="number">2</span>: NullStruct shuffle; <span class="comment">// tuple is sent to random task</span></span><br><span class="line"> <span class="number">3</span>: NullStruct all; <span class="comment">// tuple is sent to every task</span></span><br><span class="line"> <span class="number">4</span>: NullStruct none; <span class="comment">// tuple is sent to a single task (storm's choice) -&gt; allows storm to optimize the topology by bundling tasks into a single process</span></span><br><span class="line"> <span class="number">5</span>: NullStruct direct; <span class="comment">// this bolt expects the source bolt to send tuples directly to it</span></span><br><span class="line"> <span class="number">6</span>: JavaObject custom_object;</span><br><span class="line"> <span class="number">7</span>: <span class="built_in">binary</span> custom_serialized;</span><br><span class="line"> <span class="number">8</span>: NullStruct local_or_shuffle; <span class="comment">// prefer sending to tasks in the same worker process, otherwise shuffle</span></span><br><span class="line"> <span class="number">9</span>: NullStruct localFirst; <span class="comment">// local worker shuffle &gt; local node shuffle &gt; other node shuffle</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">StreamInfo</span> </span>&#123;</span><br><span class="line"> <span class="number">1</span>: <span class="keyword">required</span> list&lt;<span class="keyword">string</span>&gt; output_fields;</span><br><span class="line"> <span class="number">2</span>: <span class="keyword">required</span> <span class="built_in">bool</span> direct;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">ShellComponent</span> </span>&#123;</span><br><span class="line"> <span class="comment">// should change this to 1: required list&lt;string&gt; execution_command;</span></span><br><span class="line"> <span class="number">1</span>: <span class="built_in">string</span> execution_command;</span><br><span class="line"> <span class="number">2</span>: <span class="built_in">string</span> script;</span><br><span class="line">&#125;</span><br><span class="line">union ComponentObject &#123;</span><br><span class="line"> <span class="number">1</span>: <span class="built_in">binary</span> serialized_java;</span><br><span class="line"> <span class="number">2</span>: ShellComponent shell;</span><br><span class="line"> <span class="number">3</span>: JavaObject java_object;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">ComponentCommon</span> </span>&#123;</span><br><span class="line"> <span class="number">1</span>: <span class="keyword">required</span> map&lt;GlobalStreamId, Grouping&gt; inputs;</span><br><span class="line"> <span class="number">2</span>: <span class="keyword">required</span> map&lt;<span class="keyword">string</span>, StreamInfo&gt; streams; <span class="comment">//key is stream id</span></span><br><span class="line"> <span class="number">3</span>: <span class="keyword">optional</span> <span class="built_in">i32</span> parallelism_hint; <span class="comment">//how many threads across the cluster should be dedicated to this component</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">// component specific configuration respects:</span></span><br><span class="line"> <span class="comment">// topology.de<span class="doctag">bug:</span> false</span></span><br><span class="line"> <span class="comment">// topology.max.task.parallelism: null // can replace isDistributed with this</span></span><br><span class="line"> <span class="comment">// topology.max.spout.pending: null</span></span><br><span class="line"> <span class="comment">// topology.kryo.register // this is the only additive one</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">// component specific configuration</span></span><br><span class="line"> <span class="number">4</span>: <span class="keyword">optional</span> <span class="built_in">string</span> json_conf;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">SpoutSpec</span> </span>&#123;</span><br><span class="line"> <span class="number">1</span>: <span class="keyword">required</span> ComponentObject spout_object;</span><br><span class="line"> <span class="number">2</span>: <span class="keyword">required</span> ComponentCommon common;</span><br><span class="line"> <span class="comment">// can force a spout to be non-distributed by overriding the component configuration</span></span><br><span class="line"> <span class="comment">// and setting TOPOLOGY_MAX_TASK_PARALLELISM to 1</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">Bolt</span> </span>&#123;</span><br><span class="line"> <span class="number">1</span>: <span class="keyword">required</span> ComponentObject bolt_object;</span><br><span class="line"> <span class="number">2</span>: <span class="keyword">required</span> ComponentCommon common;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// not implemented yet</span></span><br><span class="line"><span class="comment">// this will eventually be the basis for subscription implementation in storm</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">StateSpoutSpec</span> </span>&#123;</span><br><span class="line"> <span class="number">1</span>: <span class="keyword">required</span> ComponentObject state_spout_object;</span><br><span class="line"> <span class="number">2</span>: <span class="keyword">required</span> ComponentCommon common;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">StormTopology</span> </span>&#123;</span><br><span class="line"> <span class="comment">//ids must be unique across maps</span></span><br><span class="line"> <span class="comment">// #workers to use is in conf</span></span><br><span class="line"> <span class="number">1</span>: <span class="keyword">required</span> map&lt;<span class="keyword">string</span>, SpoutSpec&gt; spouts;</span><br><span class="line"> <span class="number">2</span>: <span class="keyword">required</span> map&lt;<span class="keyword">string</span>, Bolt&gt; bolts;</span><br><span class="line"> <span class="number">3</span>: <span class="keyword">required</span> map&lt;<span class="keyword">string</span>, StateSpoutSpec&gt; state_spouts;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
</content>
<summary type="html">
<h4>TopologyBuilder功能:</h4>
<p>编织<code>ISpout,IBolt</code>的拓扑图,并生成StormTopology实例,以便thrift使用.</p>
<h4>用户编程的接口:IBolt,ISpout,IStateSpout</h4>
<p><code>IRichBolt,IBasicBolt,IRichSpout,IRichStateSpout</code></p>
<ul>
<li><code>IBasicBolt</code>没有继承<code>IBolt</code>,在编织拓扑结构时,使用<code>BasicBoltExecutor</code>包装成<code>IRichBolt</code></li>
</ul>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> BoltDeclarer <span class="title">setBolt</span><span class="params">(String id, IBasicBolt bolt, Number parallelism_hint)</span> </span>&#123;</span><br><span class="line"> <span class="keyword">return</span> setBolt(id, <span class="keyword">new</span> BasicBoltExecutor(bolt), parallelism_hint);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<ul>
<li><code>IRichStateSpout</code>暂时没有实现</li>
</ul>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">setStateSpout</span><span class="params">(String id, IRichStateSpout stateSpout, Number parallelism_hint)</span> </span>&#123;</span><br><span class="line"> validateUnusedId(id);</span><br><span class="line"> <span class="comment">// <span class="doctag">TODO:</span> finish</span></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
</summary>
<category term="jstorm" scheme="http://dangxia.github.io/categories/jstorm/"/>
<category term="jstorm" scheme="http://dangxia.github.io/tags/jstorm/"/>
</entry>
<entry>
<title>NimbusData源码</title>
<link href="http://dangxia.github.io/2015/11/25/jstorm/NimbusData-Code/"/>
<id>http://dangxia.github.io/2015/11/25/jstorm/NimbusData-Code/</id>
<published>2015-11-25T06:46:49.000Z</published>
<updated>2016-04-15T14:31:23.990Z</updated>
<content type="html"><h3>属性</h3>
<ul>
<li>downloaders,uploaders</li>
</ul>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// TODO two kind of value:Channel/BufferFileInputStream</span></span><br><span class="line"><span class="keyword">private</span> TimeCacheMap&lt;Object, Object&gt; downloaders;</span><br><span class="line"><span class="keyword">private</span> TimeCacheMap&lt;Object, Object&gt; uploaders;</span><br><span class="line"><span class="comment">/**</span><br><span class="line">* During upload/download with the master, how long an upload or download connection is idle before nimbus considers it dead and drops the connection.</span><br><span class="line">*/</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> String NIMBUS_FILE_COPY_EXPIRATION_SECS = <span class="string">"nimbus.file.copy.expiration.secs"</span>;</span><br></pre></td></tr></table></figure>
<ul>
<li>submittedCount</li>
<li>stormClusterState ==&gt; com.alibaba.jstorm.cluster.StormClusterState</li>
<li>statusTransition</li>
</ul>
<h3>StormClusterState</h3>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">disconnect</span><span class="params">()</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">remove_storm</span><span class="params">(String topology_id)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">try_remove_storm</span><span class="params">(String topology_id)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> List&lt;String&gt; <span class="title">assignments</span><span class="params">(RunnableCallback callback)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> Assignment <span class="title">assignment_info</span><span class="params">(String topology_id, RunnableCallback callback)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">set_assignment</span><span class="params">(String topology_id, Assignment info)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> AssignmentBak <span class="title">assignment_bak</span><span class="params">(String topologyName)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">backup_assignment</span><span class="params">(String topology_id, AssignmentBak info)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> List&lt;String&gt; <span class="title">active_storms</span><span class="params">()</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> StormBase <span class="title">storm_base</span><span class="params">(String topology_id, RunnableCallback callback)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">activate_storm</span><span class="params">(String topology_id, StormBase storm_base)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">update_storm</span><span class="params">(String topology_id, StormStatus new_elems)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">set_storm_monitor</span><span class="params">(String topologyId, <span class="keyword">boolean</span> isEnable)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">remove_storm_base</span><span class="params">(String topology_id)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> List&lt;String&gt; <span class="title">task_storms</span><span class="params">()</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> Set&lt;Integer&gt; <span class="title">task_ids</span><span class="params">(String topology_id)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> Set&lt;Integer&gt; <span class="title">task_ids_by_componentId</span><span class="params">(String topologyId, String componentId)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">set_task</span><span class="params">(String topologyId, Map&lt;Integer, TaskInfo&gt; taskInfoMap)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">add_task</span><span class="params">(String topology_id, Map&lt;Integer, TaskInfo&gt; taskInfoMap)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">remove_task</span><span class="params">(String topologyId, Set&lt;Integer&gt; taskIds)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> Map&lt;Integer, TaskInfo&gt; <span class="title">task_all_info</span><span class="params">(String topology_id)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> List&lt;String&gt; <span class="title">heartbeat_storms</span><span class="params">()</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">topology_heartbeat</span><span class="params">(String topology_id, TopologyTaskHbInfo info)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> TopologyTaskHbInfo <span class="title">topology_heartbeat</span><span class="params">(String topologyId)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">teardown_heartbeats</span><span class="params">(String topology_id)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> List&lt;String&gt; <span class="title">task_error_storms</span><span class="params">()</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> List&lt;String&gt; <span class="title">task_error_ids</span><span class="params">(String topologyId)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">report_task_error</span><span class="params">(String topology_id, <span class="keyword">int</span> task_id, Throwable error)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">report_task_error</span><span class="params">(String topology_id, <span class="keyword">int</span> task_id, String error, String tag)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> Map&lt;Integer, String&gt; <span class="title">topo_lastErr_time</span><span class="params">(String topologyId)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">remove_lastErr_time</span><span class="params">(String topologyId)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> List&lt;TaskError&gt; <span class="title">task_errors</span><span class="params">(String topology_id, <span class="keyword">int</span> task_id)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">remove_task_error</span><span class="params">(String topologyId, <span class="keyword">int</span> taskId)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> List&lt;String&gt; <span class="title">task_error_time</span><span class="params">(String topologyId, <span class="keyword">int</span> taskId)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> String <span class="title">task_error_info</span><span class="params">(String topologyId, <span class="keyword">int</span> taskId, <span class="keyword">long</span> timeStamp)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">teardown_task_errors</span><span class="params">(String topology_id)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> List&lt;String&gt; <span class="title">supervisors</span><span class="params">(RunnableCallback callback)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> SupervisorInfo <span class="title">supervisor_info</span><span class="params">(String supervisor_id)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">supervisor_heartbeat</span><span class="params">(String supervisor_id, SupervisorInfo info)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">boolean</span> <span class="title">try_to_be_leader</span><span class="params">(String path, String host, RunnableCallback callback)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> String <span class="title">get_leader_host</span><span class="params">()</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">boolean</span> <span class="title">leader_existed</span><span class="params">()</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> List&lt;String&gt; <span class="title">get_nimbus_slaves</span><span class="params">()</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">update_nimbus_slave</span><span class="params">(String host, <span class="keyword">int</span> time)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> String <span class="title">get_nimbus_slave_time</span><span class="params">(String host)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">unregister_nimbus_host</span><span class="params">(String host)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">update_nimbus_detail</span><span class="params">(String hostPort, Map map)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> Map <span class="title">get_nimbus_detail</span><span class="params">(String hostPort, <span class="keyword">boolean</span> watch)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">unregister_nimbus_detail</span><span class="params">(String hostPort)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">set_topology_metric</span><span class="params">(String topologyId, Object metric)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> Object <span class="title">get_topology_metric</span><span class="params">(String topologyId)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> List&lt;String&gt; <span class="title">get_metrics</span><span class="params">()</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> List&lt;String&gt; <span class="title">list_dirs</span><span class="params">(String path, <span class="keyword">boolean</span> watch)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> List&lt;String&gt; <span class="title">backpressureInfos</span><span class="params">()</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">set_backpressure_info</span><span class="params">(String topologyId, Map&lt;String, SourceBackpressureInfo&gt; sourceToBackpressureInfo)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> Map&lt;String, SourceBackpressureInfo&gt; <span class="title">get_backpressure_info</span><span class="params">(String topologyId)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">teardown_backpressure</span><span class="params">(String topologyId)</span> <span class="keyword">throws</span> Exception</span>;</span><br></pre></td></tr></table></figure>
</content>
<summary type="html">
<h3>属性</h3>
<ul>
<li>downloaders,uploaders</li>
</ul>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><sp
</summary>
<category term="jstorm" scheme="http://dangxia.github.io/categories/jstorm/"/>
<category term="jstorm" scheme="http://dangxia.github.io/tags/jstorm/"/>
</entry>
<entry>
<title>JStormCache源码</title>
<link href="http://dangxia.github.io/2015/11/25/jstorm/JStormCache-Code/"/>
<id>http://dangxia.github.io/2015/11/25/jstorm/JStormCache-Code/</id>
<published>2015-11-25T06:46:49.000Z</published>
<updated>2016-04-15T14:31:23.990Z</updated>
<content type="html"><h3>Interface</h3>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">init</span><span class="params">(Map&lt;Object, Object&gt; conf)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">cleanup</span><span class="params">()</span></span>;</span><br><span class="line"><span class="function">Object <span class="title">get</span><span class="params">(String key)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">getBatch</span><span class="params">(Map&lt;String, Object&gt; map)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">remove</span><span class="params">(String key)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">removeBatch</span><span class="params">(Collection&lt;String&gt; keys)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">put</span><span class="params">(String key, Object value, <span class="keyword">int</span> timeoutSecond)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">put</span><span class="params">(String key, Object value)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">putBatch</span><span class="params">(Map&lt;String, Object&gt; map)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">putBatch</span><span class="params">(Map&lt;String, Object&gt; map, <span class="keyword">int</span> timeoutSeconds)</span></span>;</span><br></pre></td></tr></table></figure>
<h3>JStormCache的实现</h3>
<ul>
<li>com.alibaba.jstorm.cache.TimeoutMemCache</li>
<li>com.alibaba.jstorm.cache.RocksDBCache</li>
<li>com.alibaba.jstorm.cache.RocksTTLDBCache</li>
</ul>
<a id="more"></a>
<h3>TimeoutMemCache</h3>
<p><code>TreeMap&lt;Integer, TimeCacheMap&lt;String, Object&gt;&gt; cacheWindows</code>维持了多个timeout的TimeCacheMap:</p>
<ul>
<li>default.cache.timeout default:60</li>
<li>cache.timeout.list default:null</li>
</ul>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">put</span><span class="params">(String key, Object value, <span class="keyword">int</span> timeoutSecond)</span> </span>&#123;</span><br><span class="line"> Entry&lt;Integer, TimeCacheMap&lt;String, Object&gt;&gt; ceilingEntry = cacheWindows.ceilingEntry(timeoutSecond);</span><br><span class="line"> <span class="keyword">if</span> (ceilingEntry == <span class="keyword">null</span>) &#123;</span><br><span class="line"> put(key, value);</span><br><span class="line"> &#125; <span class="keyword">else</span> &#123;</span><br><span class="line"> remove(key);</span><br><span class="line"> ceilingEntry.getValue().put(key, value);</span><br><span class="line"> &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">put</span><span class="params">(String key, Object value)</span> </span>&#123;</span><br><span class="line"> remove(key);</span><br><span class="line"> TimeCacheMap&lt;String, Object&gt; bestWindow = cacheWindows.get(defaultTimeout);</span><br><span class="line"> bestWindow.put(key, value);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<h3>RocksDBCache</h3>
<p>使用RocksDB实现的一个key=&gt;value持久化的DB.data_dir = ${storm.local.dir}/nimbus/rocksdb/</p>
<h3>RocksTTLDBCache</h3>
<p>RocksDBCache的TTL实现,TreeMap&lt;Integer, ColumnFamilyHandle&gt; windowHandlers维持多个timeout的ColumnFamilyHandle</p>
<ul>
<li>infinite lifetime <code>RocksDB.DEFAULT_COLUMN_FAMILY</code></li>
<li>cache.timeout.list default:null</li>
</ul>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">protected</span> Entry&lt;Integer, ColumnFamilyHandle&gt; <span class="title">getHandler</span><span class="params">(<span class="keyword">int</span> timeoutSecond)</span> </span>&#123;</span><br><span class="line"> ColumnFamilyHandle cfHandler = <span class="keyword">null</span>;</span><br><span class="line"> Entry&lt;Integer, ColumnFamilyHandle&gt; ceilingEntry = windowHandlers.ceilingEntry(timeoutSecond);</span><br><span class="line"> <span class="keyword">if</span> (ceilingEntry != <span class="keyword">null</span>) &#123;</span><br><span class="line"> <span class="keyword">return</span> ceilingEntry;</span><br><span class="line"> &#125; <span class="keyword">else</span> &#123;</span><br><span class="line"> <span class="keyword">return</span> windowHandlers.firstEntry();</span><br><span class="line"> &#125;</span><br><span class="line">&#125;</span><br><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">put</span><span class="params">(String key, Object value, <span class="keyword">int</span> timeoutSecond)</span> </span>&#123;</span><br><span class="line"> <span class="comment">// TODO Auto-generated method stub</span></span><br><span class="line"></span><br><span class="line"> put(key, value, getHandler(timeoutSecond));</span><br><span class="line"></span><br><span class="line">&#125;</span><br><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">put</span><span class="params">(String key, Object value)</span> </span>&#123;</span><br><span class="line"> put(key, value, windowHandlers.firstEntry());</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<h3>NimbusCache</h3>
<ul>
<li>memCache 内存cache</li>
<li>dbCache DB cache<br>
若dbCache为内存cache,memCache=dbCache;</li>
</ul>
<figure class="highlight"><table><tr><td class="code"><pre><span class="line">public String getNimbusCacheClass(Map conf) &#123;</span><br><span class="line"> boolean isLinux = OSInfo.isLinux();</span><br><span class="line"> boolean isMac = OSInfo.isMac();</span><br><span class="line"> boolean isLocal = StormConfig.local_mode(conf);</span><br><span class="line"></span><br><span class="line"> if (isLocal == true) &#123;</span><br><span class="line"> return TIMEOUT_MEM_CACHE_CLASS;</span><br><span class="line"> &#125;</span><br><span class="line"></span><br><span class="line"> if (isLinux == false &amp;&amp; isMac == false) &#123;</span><br><span class="line"> return TIMEOUT_MEM_CACHE_CLASS;</span><br><span class="line"> &#125;</span><br><span class="line"></span><br><span class="line"> String nimbusCacheClass = ConfigExtension.getNimbusCacheClass(conf);</span><br><span class="line"> if (StringUtils.isBlank(nimbusCacheClass) == false) &#123;</span><br><span class="line"> return nimbusCacheClass;</span><br><span class="line"> &#125;</span><br><span class="line"></span><br><span class="line"> return ROCKS_DB_CACHE_CLASS;</span><br><span class="line"></span><br><span class="line">&#125;</span><br><span class="line">### JStormMetricCache</span><br><span class="line">只有一个`JStormCache cache`与NimbusCache结构类似。</span><br></pre></td></tr></table></figure>
<p>配置:<br>
## Two type cache<br>
## &quot;com.alibaba.jstorm.cache.TimeoutMemCache&quot; is suitable for small cluster<br>
## &quot;com.alibaba.jstorm.cache.<s>TimeoutMemCache</s>RocksDBCache&quot; can only run under linux/mac, it is suitable for huge cluster<br>
## if it is null, it will detected by environment<br>
nimbus.cache.class: null<br>
## if this is true, nimbus db cache will be reset when start nimbus<br>
nimbus.cache.reset: true<br>
cache.timeout.list: null</p>
</content>
<summary type="html">
<h3>Interface</h3>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">init</span><span class="params">(Map&lt;Object, Object&gt; conf)</span> <span class="keyword">throws</span> Exception</span>;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">cleanup</span><span class="params">()</span></span>;</span><br><span class="line"><span class="function">Object <span class="title">get</span><span class="params">(String key)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">getBatch</span><span class="params">(Map&lt;String, Object&gt; map)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">remove</span><span class="params">(String key)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">removeBatch</span><span class="params">(Collection&lt;String&gt; keys)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">put</span><span class="params">(String key, Object value, <span class="keyword">int</span> timeoutSecond)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">put</span><span class="params">(String key, Object value)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">putBatch</span><span class="params">(Map&lt;String, Object&gt; map)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">putBatch</span><span class="params">(Map&lt;String, Object&gt; map, <span class="keyword">int</span> timeoutSeconds)</span></span>;</span><br></pre></td></tr></table></figure>
<h3>JStormCache的实现</h3>
<ul>
<li>com.alibaba.jstorm.cache.TimeoutMemCache</li>
<li>com.alibaba.jstorm.cache.RocksDBCache</li>
<li>com.alibaba.jstorm.cache.RocksTTLDBCache</li>
</ul>
</summary>
<category term="jstorm" scheme="http://dangxia.github.io/categories/jstorm/"/>
<category term="jstorm" scheme="http://dangxia.github.io/tags/jstorm/"/>
</entry>
<entry>
<title>links</title>
<link href="http://dangxia.github.io/2015/11/25/trivial/links/"/>
<id>http://dangxia.github.io/2015/11/25/trivial/links/</id>
<published>2015-11-25T03:44:02.000Z</published>
<updated>2016-04-15T14:31:23.997Z</updated>