-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathatom.xml
615 lines (386 loc) · 411 KB
/
atom.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>TungSing's Blog</title>
<subtitle>用心记录一切！</subtitle>
<link href="http://tungsing.cc/atom.xml" rel="self"/>
<link href="http://tungsing.cc/"/>
<updated>2025-01-16T07:02:36.241Z</updated>
<id>http://tungsing.cc/</id>
<author>
<name>TungSing</name>
</author>
<generator uri="https://hexo.io/">Hexo</generator>
<entry>
<title>回顾2024</title>
<link href="http://tungsing.cc/2025/01/14/my/2024/"/>
<id>http://tungsing.cc/2025/01/14/my/2024/</id>
<published>2025-01-14T02:47:42.024Z</published>
<updated>2025-01-16T07:02:36.241Z</updated>
<content type="html"><![CDATA[<p><img src="/images/my/2024.jpg"></p><p>2024年已悄然落幕，2025年崭新开启。是时候回顾过往，总结经验，为未来的发展积累宝贵的参考与启示。</p><span id="more"></span><p><strong>生活</strong></p><p>回顾过去的一年，似乎没有什么特别值得铭记的事情，但有一些片段依然深刻地留在脑海中。</p><p>最深刻的经历莫过于频繁的医院就诊，不仅花费了不少时间和精力，医保卡里这么多年的钱也被清空了。这让我更加意识到健康的重要性，希望新的一年能更加注重身体管理，减少此类经历。</p><p>在阅读方面，去年完成了一本颇具深度的书籍——《原则：应对变化中的世界秩序》。此外，还在阅读《逻辑新引：怎样辨别是非》，但仅阅读了100页，书中大量的逻辑推理需要深入思考，阅读进度有所放缓，不过我计划在新的一年里继续推进，争取彻底理解其中的核心思想。另外，《Dinosaurs Before Dark》也已经接近尾声，争取在农历新年前完成。另外视频课程今年只学习了《透视HTTP协议》，对HTTP协议的新版本也有全面的认识。</p><p>财务方面，去年在银行办理的数十万元的贷款，目前即将到期，需要认真筹划还款方案，以免对生活产生不必要的压力。同时，投资方面也有些不尽人意，手中的旅游股始终表现平平，即使在大盘大涨的日子里也未见起色，因此打算离场不玩了。</p><p><strong>工作</strong></p><p>工作方面，年初的时候完成了中文天气在线产品的研发，之后把产品发布上线，并持续维护中，目前该产品用户数在10W+，尚未达到收支平衡，新的一年要加大推广和新功能的迭代；同时年初的时候还完成了专家气象图的一个项目，核心参与基本就我们的算法工程师和前端工程师，我只是让大家舒服的配合起来，项目比较小，历时1个月完成；还有就是法院项目了，完成了验收和新项目继续签订，可能是由于大环境的影响，新旧项目中间耽搁大约半年，但平时活还一样的干。去年收获最大的项目应该是9月份开始的气象交通项目，在这个项目中接触了更多的GIS知识、气象数据和行业数据结合的难点，收获最大的应该是如何保证气象数据和行业数据能以最快的方式产生出最新的产品，已提供给用户使用。</p><p>在工作方面，年初成功完成了中文天气在线产品的研发，并顺利发布上线。目前产品用户数已突破10W+，尽管尚未达到收支平衡，但新的一年将加大推广力度，并持续进行新功能的开发与迭代。</p><p>同时，在年初还顺利完成了专家气象图项目的研发，团队核心成员为算法工程师和前端工程师。我主要负责协调团队合作，确保项目顺利推进。项目规模较小，仅用一个月便圆满完成。</p><p>此外，法院项目也取得了阶段性成果，顺利完成验收并签订了新项目合同。然而，由于大环境影响，新旧项目间隔近半年，但日常工作仍有序推进。</p><p>去年收获最大的项目当属9月份启动的气象交通项目。该项目基于气象数据，结合高速与铁路等交通数据，实现了风险预报与预测，能够为交通出行与活动提供科学参考。在该项目中，我进一步深入了解了GIS知识，探索了气象数据与行业数据融合的难点，并重点掌握了如何以最快的方式将数据转化为用户可用的最新产品。</p><p>回望这一年，有收获也有挑战。新的一年，希望自己能够更加从容地面对生活中的各种变化，保持健康、持续学习，同时更加理性地规划财务，协调好工作，让生活更加稳定与充实。</p><p><strong>思考</strong></p><p>如何苟住不失业，并能有所突破？</p><p>我的想法是：保持低调且负责任的态度，持续自我提升，以增强核心竞争力和适应能力。以不变应万变。</p><p><strong>2025</strong></p><ul><li>多锻炼</li><li>多读书</li><li>搭建AI大模型，学习其使用及原理</li><li>努力搞钱</li></ul>]]></content>
<summary type="html"><p><img src="/images/my/2024.jpg"></p>
<p>2024年已悄然落幕，2025年崭新开启。是时候回顾过往，总结经验，为未来的发展积累宝贵的参考与启示。</p></summary>
<category term="My" scheme="http://tungsing.cc/tags/My/"/>
</entry>
<entry>
<title>利用Mailx发送邮件</title>
<link href="http://tungsing.cc/2024/07/14/linux/linux-mailx/"/>
<id>http://tungsing.cc/2024/07/14/linux/linux-mailx/</id>
<published>2024-07-14T03:07:11.595Z</published>
<updated>2024-07-14T04:09:07.755Z</updated>
<content type="html"><![CDATA[<p><img src="/images/linux/linux-mailx/mail.jpg"><br>为了监控服务器运行情况，防止出故障后，长时间得不到解决，客户不满意。所以打算采用被动通知的方式，提醒运维人员及时处理系统故障。解决方案有上监控系统，比如Prometheus、Zabbix等，但项目本身不大，故采用免费的Mailx发送邮件信息来解决。</p><span id="more"></span><h3 id="环境准备"><a href="#环境准备" class="headerlink" title="环境准备"></a>环境准备</h3><table><thead><tr><th>序号</th><th>系统</th><th>备注</th></tr></thead><tbody><tr><td>A</td><td>Centos 7</td><td></td></tr><tr><td>B</td><td>QQ邮箱</td><td></td></tr></tbody></table><h3 id="安装"><a href="#安装" class="headerlink" title="安装"></a>安装</h3><p>在A服务器上安装Mailx，一般发行自带，如何没有利用yum安装</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># yum install samba</span></span><br></pre></td></tr></table></figure><h3 id="配置Mailx"><a href="#配置Mailx" class="headerlink" title="配置Mailx"></a>配置Mailx</h3><h4 id="配置文件-etc-mail-rc"><a href="#配置文件-etc-mail-rc" class="headerlink" title="配置文件 /etc/mail.rc"></a>配置文件 <code>/etc/mail.rc</code></h4><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 邮箱账户，发件人的电子邮件地址</span></span><br><span class="line"><span class="built_in">set</span> from=<span class="built_in">test</span>@qq.com</span><br><span class="line"><span class="comment"># smtp 服务器地址，smtps是ssl，普通用smtp即可</span></span><br><span class="line"><span class="built_in">set</span> smtp=smtps://smtp.qq.com:465</span><br><span class="line"><span class="comment"># SMTP认证的用户名</span></span><br><span class="line"><span class="built_in">set</span> smtp-auth-user=<span class="built_in">test</span>@qq.com</span><br><span class="line"><span class="comment">#MTP认证的密码，QQ邮箱为授权密码而非邮箱密码</span></span><br><span class="line"><span class="built_in">set</span> smtp-auth-password=xxxxx</span><br><span class="line"><span class="comment"># 是否验证SSL证书，配置 SSL 证书就可以注释掉</span></span><br><span class="line"><span class="built_in">set</span> ssl-verify=ignore</span><br><span class="line"><span class="comment"># 设置 nss 配置目录， SSL 证书目录</span></span><br><span class="line"><span class="built_in">set</span> nss-config-dir=/root/.certs</span><br></pre></td></tr></table></figure><h4 id="配置SSL证书"><a href="#配置SSL证书" class="headerlink" title="配置SSL证书"></a>配置SSL证书</h4><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">mkdir -p /root/.certs/</span><br><span class="line"><span class="built_in">echo</span> -n | openssl s_client -connect smtp.qq.com:465 | sed -ne <span class="string">'/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p'</span> > ~/.certs/qq.crt</span><br><span class="line">certutil -A -n <span class="string">"GeoTrust SSL CA"</span> -t <span class="string">"C,,"</span> -d ~/.certs -i ~/.certs/qq.crt</span><br><span class="line">certutil -A -n <span class="string">"GeoTrust Global CA"</span> -t <span class="string">"C,,"</span> -d ~/.certs -i ~/.certs/qq.crt</span><br><span class="line">certutil -L -d /root/.certs</span><br></pre></td></tr></table></figure><p>为了防止出现发送邮件警告提示，还需要进入邮箱SSL证书存放目录<code>/root/.certs</code>里执行如下命令</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">certutil -A -n <span class="string">"GeoTrust SSL CA - G3"</span> -t <span class="string">"Pu,Pu,Pu"</span> -d ./ -i qq.crt</span><br></pre></td></tr></table></figure><p>返回如下提示即可</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">**Notice: Trust flag u is <span class="built_in">set</span> automatically <span class="keyword">if</span> the private key is present.**</span><br></pre></td></tr></table></figure><h3 id="测试"><a href="#测试" class="headerlink" title="测试"></a>测试</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">echo</span> <span class="string">"test body"</span> | mail -s <span class="string">"test"</span> [email protected]</span><br></pre></td></tr></table></figure><p>如果控制台没有错误和邮箱里面已经收到了邮件，就证明配置成功了，接下来就可以是写业务的监控脚本。</p><h3 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h3><p>SMTP邮件服务器默认端口是25，现在25端口管控比较严格，如果25端口不可以用，就尝试采用SSL方式对应的其他端口</p><h4 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h4><p><a href="https://www.woniusnail.com/?p=1411">Centos配置Mailx发送邮件</a><br><a href="https://www.imydl.tech/lnmp/198.html">Linux下利用mailx结合QQ邮箱发送系统邮件提醒等</a></p>]]></content>
<summary type="html"><p><img src="/images/linux/linux-mailx/mail.jpg"><br>为了监控服务器运行情况，防止出故障后，长时间得不到解决，客户不满意。所以打算采用被动通知的方式，提醒运维人员及时处理系统故障。解决方案有上监控系统，比如Prometheus、Zabbix等，但项目本身不大，故采用免费的Mailx发送邮件信息来解决。</p></summary>
<category term="Linux" scheme="http://tungsing.cc/tags/Linux/"/>
<category term="Mailx" scheme="http://tungsing.cc/tags/Mailx/"/>
</entry>
<entry>
<title>回顾2023</title>
<link href="http://tungsing.cc/2024/01/08/my/2023/"/>
<id>http://tungsing.cc/2024/01/08/my/2023/</id>
<published>2024-01-08T08:02:37.000Z</published>
<updated>2024-02-18T06:05:22.186Z</updated>
<content type="html"><![CDATA[<p><img src="/images/my/2023.jpg"></p><p>2023已经结束，是时候总结一下这一年的情况，为未来作借鉴，为之前做回忆。</p><span id="more"></span><p><strong>生活</strong>：</p><p>被疫情耽误的婚礼终于在2023-05-09这天顺利完成了，再次感谢亲朋好友们的祝福和帮忙。未来就是新的旅程了。</p><p>搬离住了很多年的回龙观，从18年就开始在这片住，满满的回忆啊，因为18年之前的3年是住在西三旗桥附近。</p><p>年初的时候，想着疫情已结束，未来形式肯定会一片大好，然后就买了两只旅游股，没想到年底亏了40%，辛亏买的少。</p><p>学习上今年就差很多了，只把去年没读完的《原则》给读完了；庆幸的是背单词坚持下来了。明年加油！</p><p><strong>工作</strong></p><p>今年工作上的事情也比较多，下面就重要的事情一一回顾下。</p><p>首先是年初带两个小伙伴做了7个月的一个运维项目，这个项目本身的技术难度到是不大。比较难的是两个小伙伴对这些项目的熟悉程度不搞，需要指导、培训的比较多；然后就是客户关系了，由于工作过程中甲方换了新的项目负责人，导致项目不在有后续。</p><p>同时和孔博士开发《东北冷涡预报》项目，我主要负责数据分析、下载、清洗、转换等工作，博士负责算法的开发；</p><p>4月初，和一个前端的小伙伴开发了《审判大辞典》APP。</p><p>5月中旬，按照标准的项目流程开发《数字图书馆》APP，从需求、设计、开发、测试、部署、运行维护全生命流程进行。</p><p>10月初，继续开发《天气在线》APP。主体功能全部开发完成，待上架。上架流程太繁琐。</p><p>还有就是全年都在兼顾另一个运维项目，配合项目现场的同事处理他解决不了的所有问题。</p><p>剩下的就是其他杂七杂八的事情了，有人员招聘、管理工作、基础开发平台的升级和完善，其他项目的技术支持等等。</p><p>总体来说这一年干的工作不少，但没什么成就感。</p><p><strong>思考</strong></p><p>这么差的环境下该如何生存下去？</p><p>我的想法是：保持开放的头脑，不断提升自己，多认识人，然后顺势而为。</p><p><strong>2024</strong></p><ul><li>多读书</li><li>搭建AI大模型，学习其使用及原理</li><li>努力搞钱</li></ul>]]></content>
<summary type="html"><p><img src="/images/my/2023.jpg"></p>
<p>2023已经结束，是时候总结一下这一年的情况，为未来作借鉴，为之前做回忆。</p></summary>
<category term="My" scheme="http://tungsing.cc/tags/My/"/>
</entry>
<entry>
<title>记录解决MySQL LEFT JOIN 查询慢</title>
<link href="http://tungsing.cc/2023/08/22/mysql/mysql-query/"/>
<id>http://tungsing.cc/2023/08/22/mysql/mysql-query/</id>
<published>2023-08-22T06:00:53.418Z</published>
<updated>2023-08-25T09:38:52.491Z</updated>
<content type="html"><![CDATA[<p><img src="/images/mysql/mysql.jpg"></p><span id="more"></span><h3 id="环境"><a href="#环境" class="headerlink" title="环境"></a>环境</h3><ul><li> Linux CentOS7 操作系统</li><li> Mysql5.7</li></ul><h3 id="背景"><a href="#背景" class="headerlink" title="背景"></a>背景</h3><p>在和前端开发人员过页面的时候，发现其中一个页面响应很慢，定位原因是API接口响应要24s左右，这响应速度太慢了。排查后端代码发现涉及到的查询SQL语句是一个简单的左连接查询，按之前的经验不能这么慢呢，所以在SQL IDE中排查。</p><p>先说一下涉及的表，如下：</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">-- 书架表,数量量：4条</span></span><br><span class="line"><span class="keyword">create</span> <span class="keyword">table</span> cdlm_bookshelf</span><br><span class="line">(</span><br><span class="line"> bookshelf_id <span class="type">varchar</span>(<span class="number">32</span>) <span class="keyword">not</span> <span class="keyword">null</span> comment <span class="string">'书架唯一标识'</span>,</span><br><span class="line"> user_id <span class="type">varchar</span>(<span class="number">32</span>) <span class="keyword">not</span> <span class="keyword">null</span> comment <span class="string">'用户唯一标识'</span>,</span><br><span class="line"> resource_id <span class="type">varchar</span>(<span class="number">32</span>) <span class="keyword">not</span> <span class="keyword">null</span> comment <span class="string">'资源ID'</span>,</span><br><span class="line"> resource_type <span class="type">varchar</span>(<span class="number">16</span>) <span class="keyword">not</span> <span class="keyword">null</span> comment <span class="string">'资源类型'</span>,</span><br><span class="line"> resource_name <span class="type">varchar</span>(<span class="number">512</span>) <span class="keyword">not</span> <span class="keyword">null</span> comment <span class="string">'资源名称'</span>,</span><br><span class="line"> cover_id <span class="type">varchar</span>(<span class="number">32</span>) comment <span class="string">'资源封面,附件表id'</span>,</span><br><span class="line"> read_position <span class="type">varchar</span>(<span class="number">128</span>) comment <span class="string">'阅读位置'</span>,</span><br><span class="line"> read_percent <span class="type">int</span> comment <span class="string">'阅读百分比(0-100)'</span>,</span><br><span class="line"> <span class="keyword">primary</span> key (bookshelf_id)</span><br><span class="line">);</span><br><span class="line"><span class="comment">-- 附件表，数据量：5136453条</span></span><br><span class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> t_attachment_info (</span><br><span class="line"> attachment_id <span class="type">varchar</span>(<span class="number">40</span>) <span class="keyword">NOT</span> <span class="keyword">NULL</span> COMMENT <span class="string">'附件id'</span>,</span><br><span class="line"> attachment_name <span class="type">varchar</span>(<span class="number">2000</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span> COMMENT <span class="string">'附件名称'</span>,</span><br><span class="line"> attachment_type <span class="type">varchar</span>(<span class="number">200</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span> COMMENT <span class="string">'附件类型'</span>,</span><br><span class="line"> attachment_size <span class="keyword">double</span> <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span> COMMENT <span class="string">'附件大小'</span>,</span><br><span class="line"> attachment_path <span class="type">varchar</span>(<span class="number">200</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span> COMMENT <span class="string">'附件路径(相对)'</span></span><br><span class="line"> <span class="keyword">PRIMARY</span> KEY (attachment_id)</span><br><span class="line">);</span><br></pre></td></tr></table></figure><p>查询的语句：</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> b.<span class="operator">*</span>, a.attachment_path <span class="keyword">AS</span> cover_path</span><br><span class="line"><span class="keyword">FROM</span> cdlm_bookshelf_test b</span><br><span class="line"><span class="keyword">LEFT</span> <span class="keyword">JOIN</span> t_attachment_info a</span><br><span class="line"><span class="keyword">ON</span> b.cover_id <span class="operator">=</span> a.attachment_id</span><br><span class="line"><span class="keyword">WHERE</span> b.user_id<span class="operator">=</span><span class="string">'7f52280cbdd04e29ae899884a4a2193a'</span>;</span><br></pre></td></tr></table></figure><p>查询耗时，10.3秒左右：</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">1 queries executed, 1 success, 0 errors, 0 warnings</span><br><span class="line"></span><br><span class="line">查询：SELECT b.*, a.attachment_path AS cover_path FROM cdlm_bookshelf_test b left JOIN t_attachment_info a ON b.cover_id = a.attachmen...</span><br><span class="line"></span><br><span class="line">共 5 行受到影响</span><br><span class="line"></span><br><span class="line">执行耗时 : 10.318 sec</span><br><span class="line">传送时间 : 0 sec</span><br><span class="line">总耗时 : 10.318 sec</span><br><span class="line"></span><br></pre></td></tr></table></figure><h3 id="分析"><a href="#分析" class="headerlink" title="分析"></a>分析</h3><p>首先想到的肯定是用<code>EXPLAIN</code> 关键词分析执行情况，结果如下：</p><p><img src="/images/mysql/query_explain_1.png"></p><p>通过对执行计划结果的分析<code>t_attachment_info</code>表是全表扫描呀，这肯定慢。对照我们的查询语句，为什么是全表扫描没用到索引呢? <code>t_attachment_info</code>表中的<code>attachment_id</code> 是主键，有主键索引。那就是<code>cdlm_bookshelf_test</code> 表中的<code>cover_id</code> 没用索引导致的了，那我们就给创建个索引试试。</p><h4 id="创建索引"><a href="#创建索引" class="headerlink" title="创建索引"></a>创建索引</h4><p>给<code>cdlm_bookshelf_test</code> 表中的<code>cover_id</code> 创建索引</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">ALTER</span> <span class="keyword">TABLE</span> cdlm_bookshelf_test</span><br><span class="line"> <span class="keyword">ADD</span> INDEX idx_cover_Id (cover_id);</span><br></pre></td></tr></table></figure><p>验证一下效果，结果花了11.3秒左右，更慢了，这是啥情况呢？继续用<code>EXPLAIN</code> 分析执行情况</p><p><img src="/images/mysql/query_explain_1.png"></p><p>从执行计划结果看，没啥变化呢？但是我看到<code>join buffer</code> 。难道是配置的问题？那就查看并配置一下<code>join_buffer_size</code>。</p><h4 id="join-buffer-size配置"><a href="#join-buffer-size配置" class="headerlink" title="join_buffer_size配置"></a>join_buffer_size配置</h4><p>查看MySQL的配置，之前并没有配置<code>join_buffer_size</code>的值，那就是使用的默认值。那我们给她配置一个大一点值，再观察下效果。</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"># myslq 配置文件</span><br><span class="line">join_buffer_size=2M</span><br></pre></td></tr></table></figure><p>结果，查询数据时间确实变短了，现在5-6秒了，但这也很慢，同样查看执行计划，一样没变化。</p><p>我又找个一个之前的左连接查询语句，速度很快。所以这个参数确实能提升查询速度，但不是该语句慢的核心原因。到这我只能利用搜索引擎了。</p><h4 id="字段类型不一致"><a href="#字段类型不一致" class="headerlink" title="字段类型不一致"></a>字段类型不一致</h4><p>查询到一篇文章的情况是，左连接关联的字段的类型不一致导致查询很慢。字段类型不一致数据库会先做类型转换再去关联查询，所以就慢。而我的查询语句关联字段类型是一致，都是<code>varchar</code>,只是长度是不一样的。难道是这样原因？修改<code>cdlm_bookshelf_test</code> 表中的<code>cover_id</code> 字段的长度也是<code>40</code></p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">ALTER</span> <span class="keyword">TABLE</span> cdlm_bookshelf_test</span><br><span class="line"> CHANGE cover_id cover_id <span class="type">VARCHAR</span> (<span class="number">40</span>) <span class="keyword">NULL</span> COMMENT <span class="string">'资源封面,附件表id'</span>;</span><br></pre></td></tr></table></figure><p>结果没啥变化。说明字段的长度不是影响因素。继续</p><h4 id="字段字符集不一致"><a href="#字段字符集不一致" class="headerlink" title="字段字符集不一致"></a>字段字符集不一致</h4><p>另一篇文章提到了，字段字符集不一致，也会导致查询慢。赶紧查看了所涉及的表的字符集是否不一致。</p><p>果然表的字符集是不一样的。</p><p><code>t_attachment_info</code>是<code>utf-8</code>，而<code>cdlm_bookshelf_test</code>是<code>utf8mb4</code>。</p><p>导致表的字符集不一致的原因是，<code>cdlm_bookshelf_test</code>是新创建的，而<code>t_attachment_info</code>表是从旧系统导入进来的，这是历史原因了。那赶紧动手先修改字符集使其保持一致。</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">-- 先修改表</span></span><br><span class="line"><span class="keyword">ALTER</span> <span class="keyword">TABLE</span> cdlm_bookshelf_test</span><br><span class="line"> CHARSET <span class="operator">=</span> utf8;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 修改字段</span></span><br><span class="line"><span class="keyword">alter</span> <span class="keyword">table</span> cdlm_bookshelf_test <span class="keyword">convert</span> <span class="keyword">to</span> <span class="type">character</span> <span class="keyword">set</span> utf8;</span><br><span class="line"></span><br></pre></td></tr></table></figure><p>查询验证，只需要0.008秒。这才是正常的速度!</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">1 queries executed, 1 success, 0 errors, 0 warnings</span><br><span class="line"></span><br><span class="line">查询：SELECT b.*, a.attachment_path AS cover_path FROM cdlm_bookshelf_test b left JOIN t_attachment_info a ON b.cover_id = a.attachmen...</span><br><span class="line"></span><br><span class="line">共 5 行受到影响</span><br><span class="line"></span><br><span class="line">执行耗时 : 0.008 sec</span><br><span class="line">传送时间 : 0 sec</span><br><span class="line">总耗时 : 0.008 sec</span><br></pre></td></tr></table></figure><p>再看一下执行计划：</p><p><img src="/images/mysql/query_explain_2.png"></p><p>利用了eq_ref，只是对主键或者唯一索引的扫描，只读取了一行数据。</p><h3 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h3><p>本文主要是记录影响MySQL left join 查询慢的点，有索引、join_buffer_size、关联字段类型、字段字符集。先记录这些点，慢慢的可以深入其内部的工作原理，了解和掌握更多知识。</p><h3 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h3><p><a href="https://xie.infoq.cn/article/8aa2cd409dff1d9d190bba937">MySQL 性能优化：EXPLAIN 执行计划与 join</a></p><p><a href="https://blog.51cto.com/u_15054039/4267591">mysql left join优化 left join查询慢</a></p>]]></content>
<summary type="html"><p><img src="/images/mysql/mysql.jpg"></p></summary>
<category term="database" scheme="http://tungsing.cc/categories/database/"/>
<category term="MySQL" scheme="http://tungsing.cc/tags/MySQL/"/>
</entry>
<entry>
<title>感染新冠(2023-06-09)</title>
<link href="http://tungsing.cc/2023/07/17/my/covid-19/"/>
<id>http://tungsing.cc/2023/07/17/my/covid-19/</id>
<published>2023-07-17T02:45:52.450Z</published>
<updated>2023-07-17T03:23:26.660Z</updated>
<content type="html"><![CDATA[<p><img src="/images/my/covid-19.jpg"></p><p>最终还是感染了新冠病毒，记录下当时的情况，也算是3年的见证了。</p><span id="more"></span><p><strong>第一天</strong>：</p><p>第一天的症状还是比较轻的，主要是症状是眨眼睛的时候右眼胀痛（每次生病先从眼睛不舒服开始），还有就是鼻子稍微不舒服。</p><p><strong>第二天</strong></p><p>上午起床右眼已经没有第一天痛了，就有稍微一点点疼。但是胳膊腿出现了一点点酸麻，并逐步加重，伴随着体温每两个小时升高0.2-0.3度，头开始蒙疼，也开始流鼻涕、鼻塞（这个不是很严重），用一个词形容就是坐立不安，怎么待着都不舒服，目前体温还在有节奏的上升，下午体温上升到38度左右就不升了，但是头疼和喘气很严重，晚上7点抗不住了，吃颗泰诺。</p><p><strong>第三天</strong></p><p>早晨起来变嗓子疼了，体温37.2左右，没想到处理了工作可能有1个小时吧，然后全身发冷，温度直接干了到了39.5，赶紧吃颗太诺压压惊，吃完药温度慢慢下来了，到晚上温度再次升高到，39.5，继续吃药。</p><p><strong>第四/五天</strong></p><p>这个两天的主要症状是反复发烧，药效一过，温度就上去。前几天都是吃的泰诺，后续换了布洛芬。</p><p><strong>第六天</strong></p><p>体温稳定在了37度，不再反复发烧了，也不需要吃退烧药了，开心。</p><p><strong>第七天</strong></p><p>本来以为今天会更好，没想到早晨起床后，发现嗓子疼（喝水都疼），嘴里面还起泡了，鼻塞还严重，伴随干咳。查询了下说地喹氯铵含片可以缓解嗓子疼，我赶紧下单安排上，含上后确实有一些效果。</p><p><strong>第八天</strong><br>嗓子疼有明显好转，但有一种头闷、没精神的感觉，咳嗽还没好转，并且会咳痰的情况。</p><p><strong>后续</strong></p><p>就是一个症状了持续干咳…</p>]]></content>
<summary type="html"><p><img src="/images/my/covid-19.jpg"></p>
<p>最终还是感染了新冠病毒，记录下当时的情况，也算是3年的见证了。</p></summary>
<category term="My" scheme="http://tungsing.cc/tags/My/"/>
</entry>
<entry>
<title>搭建MariaDB读写分离、高可用</title>
<link href="http://tungsing.cc/2023/03/15/mysql/mariadb-ha/"/>
<id>http://tungsing.cc/2023/03/15/mysql/mariadb-ha/</id>
<published>2023-03-15T01:39:57.236Z</published>
<updated>2023-03-20T09:57:05.508Z</updated>
<content type="html"><![CDATA[<h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote><p>在做基础脚手架开发的过程中涉及到代码要不要支持读写分离，支持的话的需要一定的工作量，虽然代码不多，也比较容易实现，但毕竟需要维护。为了避免重新写轮子，先查询。还有真有不少解决方案，第一种就是基于应用的，自己在应用程序中编码实现；第二种就是基于中间件，有MySQL Proxy、MaxScale和HAProxy等，MySQL Proxy已经不维护了；HAProxy本身不支持读写分离，还需要搭配其他才能实现；MaxScale 文档比较全，并且是MariaDB 开发的，所以先用起来看看效果。</p></blockquote><p><img src="/images/book/design_pattern.jpg"></p><span id="more"></span><h4 id="环境"><a href="#环境" class="headerlink" title="环境"></a>环境</h4><ul><li> Linux rocky 8 </li><li> MariaDB 10.3</li><li> MariaDB MaxScale 6.4</li><li> Keepalived 2.1.5</li></ul><h4 id="环境规划"><a href="#环境规划" class="headerlink" title="环境规划"></a>环境规划</h4><p><img src="/images/mysql/mariadb_ha.png"></p><table><thead><tr><th align="center">序号</th><th>应用</th><th>hostname</th><th>IP</th><th>端口</th></tr></thead><tbody><tr><td align="center">1</td><td>MariaDB Master</td><td>mariadb_master</td><td>192.168.56.116</td><td>3306</td></tr><tr><td align="center">2</td><td>MariaDB Slave</td><td>mariadb_slave</td><td>192.168.56.117</td><td>3306</td></tr><tr><td align="center">3</td><td>MaxScale Master 和 Keepalived Master</td><td>msk_master</td><td>192.168.56.118</td><td>4006/8989</td></tr><tr><td align="center">4</td><td>MaxScale Backup 和 Keepalived Backup</td><td>msk_backup</td><td>192.168.56.119</td><td>4006/8989</td></tr><tr><td align="center">5</td><td>VIP</td><td></td><td>192.168.56.120</td><td></td></tr></tbody></table><h4 id="MariaDB-主从搭建"><a href="#MariaDB-主从搭建" class="headerlink" title="MariaDB 主从搭建"></a>MariaDB 主从搭建</h4><h5 id="Master-安装"><a href="#Master-安装" class="headerlink" title="Master 安装"></a>Master 安装</h5><ol><li><p>检查是否之前是否有安装，有删除</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">yum list installed | grep -i maria</span><br><span class="line">yum list installed | grep -i mysql</span><br></pre></td></tr></table></figure></li><li><p>安装</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum install mariadb-server</span><br></pre></td></tr></table></figure></li><li><p>修改默认数据目录，</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 创建目录</span></span><br><span class="line">mkdir /home/mariadb_data</span><br><span class="line"><span class="comment"># 2. 拷贝目录数据目录</span></span><br><span class="line">cp -a /var/lib/mysql /home/mariadb_data/</span><br><span class="line"><span class="comment"># 3. 修改mariadb server 配置文件</span></span><br><span class="line">vi /etc/my.cnf.d/mariadb-server.cnf</span><br><span class="line"><span class="comment"># 需要修改的内容</span></span><br><span class="line">datadir=/home/mariadb_data/mysql</span><br><span class="line">socket=/home/mariadb_data/mysql/mysql.sock</span><br><span class="line"><span class="comment"># 4. 修改客户端配置文件</span></span><br><span class="line">vi /etc/my.cnf.d/client.cnf</span><br><span class="line"><span class="comment"># 需要修改的内容</span></span><br><span class="line">[client]</span><br><span class="line">socket=/home/mariadb_data/mysql/mysql.sock</span><br></pre></td></tr></table></figure></li><li><p>修改SELinux对MySQL默认数据目录的保护</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 打开/etc/selinux/targeted/contexts/files/file_contexts.local</span></span><br><span class="line">vi /etc/selinux/targeted/contexts/files/file_contexts.local</span><br><span class="line"><span class="comment"># 2. 添加以下内容</span></span><br><span class="line">/home/mariadb_data(/.*)? system_u:object_r:mysqld_db_t:s0</span><br><span class="line"><span class="comment"># 如果已执行过启动执行命令请重新授权</span></span><br><span class="line">chcon -R -t mysqld_db_t /home/mariadb_data/</span><br></pre></td></tr></table></figure></li><li><p>启动MySQL</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 启动</span></span><br><span class="line">systemctl start mariadb</span><br><span class="line"><span class="comment"># 开启自启动</span></span><br><span class="line">systemctl <span class="built_in">enable</span> mariadb</span><br></pre></td></tr></table></figure></li><li><p>数据库初始化</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 临时设置MYSQL_UNIX_PORT变量</span></span><br><span class="line"><span class="built_in">export</span> MYSQL_UNIX_PORT=/home/mariadb_data/mysql/mysql.sock</span><br><span class="line"><span class="comment"># 2. 执行命令</span></span><br><span class="line">mysql_secure_installation</span><br><span class="line"><span class="comment"># 说明</span></span><br><span class="line"><span class="comment"># 首先是设置密码，会提示先输入密码:</span></span><br><span class="line">Enter current password <span class="keyword">for</span> root (enter <span class="keyword">for</span> none):<–初次运行直接回车</span><br><span class="line"><span class="comment"># 设置密码</span></span><br><span class="line">Set root password? [Y/n] <– 是否设置root用户密码，输入y并回车或直接回车</span><br><span class="line">New password: <– 设置root用户的密码</span><br><span class="line">Re-enter new password: <– 再输入一次你设置的密码</span><br><span class="line"><span class="comment"># 其它配置</span></span><br><span class="line">Remove anonymous users? [Y/n] <– 是否删除匿名用户，回车</span><br><span class="line">Disallow root login remotely? [Y/n] <–是否禁止root远程登录,回车,</span><br><span class="line">Remove <span class="built_in">test</span> database and access to it? [Y/n] <– 是否删除<span class="built_in">test</span>数据库，回车</span><br><span class="line">Reload privilege tables now? [Y/n] <– 是否重新加载权限表，回车</span><br></pre></td></tr></table></figure></li><li><p>测试登陆</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 登陆到MySQL Server</span></span><br><span class="line">mysql -uroot -p</span><br></pre></td></tr></table></figure></li></ol><h5 id="Slave-安装"><a href="#Slave-安装" class="headerlink" title="Slave 安装"></a>Slave 安装</h5><p>同master</p><h5 id="搭建配置"><a href="#搭建配置" class="headerlink" title="搭建配置"></a>搭建配置</h5><ol><li><p>修改master的<code>my.cnf</code>配置文件</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># server id 唯一</span></span><br><span class="line">server_id=116</span><br><span class="line"><span class="comment"># GTID 模式</span></span><br><span class="line">gtid_strict_mode=1</span><br><span class="line"><span class="comment"># 是否记录从服务器同步数据动作</span></span><br><span class="line">log-slave-updates=ON</span><br><span class="line"><span class="comment"># 开启及设置二进制日志文件名称</span></span><br><span class="line">log_bin=mysql-bin</span><br><span class="line"><span class="comment"># 设置binlog文件格式</span></span><br><span class="line">binlog_format=MIXED</span><br><span class="line"></span><br><span class="line"><span class="comment"># 记录IO线程读取已经读取到的master binlog位置，用于slave宕机后IO线程根据文件中的POS点重新拉取binlog日志</span></span><br><span class="line">master-info-repository = TABLE</span><br><span class="line"><span class="comment"># 记录SQL线程读取Master binlog的位置，用于Slave 宕机后根据文件中记录的pos点恢复Sql线程</span></span><br><span class="line">relay-log-info-repository = TABLE </span><br><span class="line"><span class="comment"># 启用确保无信息丢失；任何一个事务提交后, 将二进制日志的文件名及事件位置记录到文件中</span></span><br><span class="line">sync-master-info = 1</span><br><span class="line"><span class="comment"># 设定从服务器的复制线程数；0表示关闭多线程复制功能</span></span><br><span class="line">slave-parallel-workers = 2</span><br><span class="line"><span class="comment"># 设置binlog校验算法（循环冗余校验码）</span></span><br><span class="line">binlog-checksum = CRC32</span><br><span class="line"><span class="comment"># 设置主服务器是否校验</span></span><br><span class="line">master-verify-checksum = 1</span><br><span class="line"><span class="comment"># 设置从服务器是否校验</span></span><br><span class="line">slave-sql-verify-checksum = 1</span><br><span class="line"><span class="comment"># 用于在二进制日志记录事件相关的信息，可降低故障排除的复杂度</span></span><br><span class="line">binlog-rows-query-log_events = 1</span><br><span class="line"><span class="comment"># 保证master crash safe，该参数必须设置为1</span></span><br><span class="line">sync_binlog = 1</span><br><span class="line"><span class="comment"># 保证master crash safe，该参数必须设置为1</span></span><br><span class="line">innodb_flush_log_at_trx_commit = 1 </span><br></pre></td></tr></table></figure></li><li><p>重启MariaDB Server</p></li><li><p>添加同步用户</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">GRANT REPLICATION SLAVE,RELOAD,SUPER ON *.* TO replication@<span class="string">'192.168.56.%'</span> IDENTIFIED BY <span class="string">'1234@abCD'</span>;</span><br><span class="line">FLUSH PRIVILEGES;</span><br></pre></td></tr></table></figure></li><li><p>查看master 状态并记录下位置信息</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">SELECT @@GLOBAL.gtid_current_pos;</span><br><span class="line">+---------------------------+</span><br><span class="line">| @@GLOBAL.gtid_current_pos |</span><br><span class="line">+---------------------------+</span><br><span class="line">| 0-116-12 |</span><br><span class="line">+---------------------------+</span><br></pre></td></tr></table></figure></li><li><p>修改slave的<code>my.cnf</code> 配置文件</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># server id 唯一</span></span><br><span class="line">server-id=117</span><br><span class="line"><span class="comment"># GTID 模式</span></span><br><span class="line">gtid_strict_mode=1</span><br><span class="line"><span class="comment"># 是否记录从服务器同步数据动作</span></span><br><span class="line">log-slave-updates=ON</span><br><span class="line"><span class="comment"># 开启及设置二进制日志文件名称</span></span><br><span class="line">log_bin=mysql-bin</span><br><span class="line"><span class="comment"># 设置binlog文件格式</span></span><br><span class="line">binlog_format=MIXED</span><br><span class="line"><span class="comment"># 设置一般用户为只读模式</span></span><br><span class="line">read-only=1</span><br><span class="line"></span><br><span class="line">master-info-repository = TABLE</span><br><span class="line">relay-log-info-repository = TABLE</span><br><span class="line">sync-master-info = 1</span><br><span class="line">slave-parallel-workers = 4</span><br><span class="line">binlog-checksum = CRC32</span><br><span class="line">master-verify-checksum = 1</span><br><span class="line">slave-sql-verify-checksum = 1</span><br><span class="line">binlog-rows-query-log_events = 1</span><br><span class="line"><span class="comment"># crash safe slave</span></span><br><span class="line">relay_log_recovery = 1</span><br></pre></td></tr></table></figure></li><li><p>重启MariaDB Server</p></li><li><p>打开master 防火墙的3306 端口</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">firewall-cmd --zone=public --add-port=3306/tcp --permanent</span><br><span class="line">firewall-cmd --reload</span><br></pre></td></tr></table></figure></li><li><p>配置从主库同步数据</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 登陆MySQL slave Server</span></span><br><span class="line">mysql -uroot -p</span><br><span class="line"><span class="built_in">set</span> global gtid_slave_pos = <span class="string">"0-116-12"</span>;</span><br><span class="line"><span class="comment"># 2. 配置</span></span><br><span class="line"><span class="comment"># MASTER_LOG_FILE 和 MASTER_LOG_POS 的值就是从 主库 show master status 命令中获取的信息</span></span><br><span class="line">mysql> CHANGE MASTER TO MASTER_HOST=<span class="string">'192.168.56.116'</span>,</span><br><span class="line"> MASTER_USER=<span class="string">'replication'</span>,</span><br><span class="line"> MASTER_PASSWORD=<span class="string">'1234@abCD'</span>,</span><br><span class="line"> master_use_gtid=slave_pos;</span><br><span class="line"><span class="comment"># 3. 开启同步</span></span><br><span class="line">mysql> start slave;</span><br></pre></td></tr></table></figure></li><li><p>查看同步情况</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">mysql> show slave status\G</span><br><span class="line"><span class="comment"># 检查以下参数都是 Yes 代表正常</span></span><br><span class="line">Slave_IO_Running: Yes</span><br><span class="line">Slave_SQL_Running: Yes</span><br><span class="line"></span><br></pre></td></tr></table></figure></li><li><p>测试，通过创建数据库、表，添加数据来看同步是否正常</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 创建测试库和测试用户</span></span><br><span class="line">CREATE DATABASE ivy_base CHARACTER SET utf8mb4;</span><br><span class="line">grant all on ivy_base.* to <span class="string">'ivy_base'</span>@<span class="string">'%'</span> identified by <span class="string">'1234@abCD'</span>;</span><br><span class="line">flush privileges;</span><br><span class="line"><span class="comment"># 2. 执行创建表、添加数据、查询数据等操作</span></span><br><span class="line">CREATE TABLE `<span class="built_in">test</span>` (</span><br><span class="line"> `id` int(11) NOT NULL,</span><br><span class="line"> `name` varchar(200) DEFAULT NULL,</span><br><span class="line"> PRIMARY KEY (`id`)</span><br><span class="line">) ;</span><br><span class="line">INSERT INTO <span class="built_in">test</span> (id, NAME) VALUES (<span class="string">'2'</span>, <span class="string">'1'</span>);</span><br><span class="line">select * from <span class="built_in">test</span>;</span><br></pre></td></tr></table></figure></li></ol><h4 id="MaxScale-搭建"><a href="#MaxScale-搭建" class="headerlink" title="MaxScale 搭建"></a>MaxScale 搭建</h4><h5 id="Master-安装配置"><a href="#Master-安装配置" class="headerlink" title="Master 安装配置"></a>Master 安装配置</h5><ol><li><p>通过<a href="https://mariadb.com/downloads/community/maxscale/">官网</a>下载安装包</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">wget https://dlm.mariadb.com/2717592/MaxScale/6.4.5/yum/centos/8/x86_64/maxscale-6.4.5-1.rhel.8.x86_64.rpm</span><br></pre></td></tr></table></figure></li><li><p>安装</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum install maxscale-6.4.5-1.rhel.8.x86_64.rpm</span><br></pre></td></tr></table></figure></li><li><p>在MySQL Master 上添加监控和路由用户</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 连接MySQL Server</span></span><br><span class="line">mysql -uroot -p</span><br><span class="line"><span class="comment"># 2. 添加监控用户</span></span><br><span class="line">grant Replication slave,Replication client,Super,Reload on *.* to <span class="string">'maxmon'</span>@<span class="string">'192.168.56.%'</span> identified by <span class="string">'1234@abCD'</span>;</span><br><span class="line"><span class="comment"># 3. 添加路由用户</span></span><br><span class="line">grant select on mysql.* to <span class="string">'maxrou'</span>@<span class="string">'192.168.56.%'</span> identified by <span class="string">'1234@abCD'</span>;</span><br><span class="line">grant show databases on *.* to <span class="string">'maxrou'</span>@<span class="string">'192.168.56.%'</span>;</span><br><span class="line"><span class="comment"># 4. 刷新权限信息</span></span><br><span class="line">flush privileges;</span><br></pre></td></tr></table></figure></li><li><p>读写分离配置，故障自动转移。具体可阅读<a href="https://github.com/mariadb-corporation/MaxScale/blob/6.4.5/Documentation/Documentation-Contents.md">官方配置</a></p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 打开配置文件</span></span><br><span class="line">vim /etc/maxscale.cnf</span><br><span class="line"><span class="comment"># 2. 配置</span></span><br><span class="line"><span class="comment"># 基础配置</span></span><br><span class="line">[maxscale]</span><br><span class="line">threads=auto</span><br><span class="line">admin_host=0.0.0.0</span><br><span class="line">admin_secure_gui=<span class="literal">false</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># MySQL Server 配置</span></span><br><span class="line">[server1]</span><br><span class="line"><span class="built_in">type</span>=server</span><br><span class="line">address=192.168.56.116</span><br><span class="line">port=3306</span><br><span class="line">protocol=MariaDBBackend</span><br><span class="line"></span><br><span class="line">[server2]</span><br><span class="line"><span class="built_in">type</span>=server</span><br><span class="line">address=192.168.56.117</span><br><span class="line">port=3306</span><br><span class="line">protocol=MariaDBBackend</span><br><span class="line"><span class="comment"># 监控配置</span></span><br><span class="line">[MariaDB-Monitor]</span><br><span class="line"><span class="built_in">type</span>=monitor</span><br><span class="line">module=mariadbmon</span><br><span class="line">servers=server1,server2</span><br><span class="line">user=maxmon</span><br><span class="line">password=password</span><br><span class="line">monitor_interval=2000</span><br><span class="line"><span class="comment"># 用来执行CHANGE MASTER TO 命令的</span></span><br><span class="line">replication_user=replication</span><br><span class="line">replication_password=1234@abCD</span><br><span class="line"><span class="comment"># 打开自动故障转移</span></span><br><span class="line">auto_failover=<span class="literal">true</span></span><br><span class="line"><span class="comment"># 打开自动重新加入</span></span><br><span class="line">auto_rejoin=<span class="literal">true</span></span><br><span class="line"><span class="comment"># slave 全部失效时 master 支撑全部请求</span></span><br><span class="line">detect_stale_master=<span class="literal">true</span></span><br><span class="line">failover_timeout=5 </span><br><span class="line">failcount=5</span><br><span class="line">master_failure_timeout=2</span><br><span class="line">verify_master_failure=<span class="literal">true</span></span><br><span class="line">switchover_timeout=90</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment"># 过滤器配置</span></span><br><span class="line"><span class="comment"># 非必须配置，是我使用的客户端工具默认会执行解释计划，但默认explain plan 会被路由到从库</span></span><br><span class="line">[Named-Server-Filter]</span><br><span class="line"><span class="built_in">type</span>=filter</span><br><span class="line">module=namedserverfilter</span><br><span class="line">match01=^explain.*</span><br><span class="line">target01=server1</span><br><span class="line"></span><br><span class="line"><span class="comment"># 读写分离服务配置</span></span><br><span class="line">[Read-Write-Service]</span><br><span class="line"><span class="built_in">type</span>=service</span><br><span class="line">router=readwritesplit</span><br><span class="line">servers=server1,server2</span><br><span class="line">user=maxrou</span><br><span class="line">password=password</span><br><span class="line">filters=Named-Server-Filter</span><br><span class="line"></span><br><span class="line"><span class="comment"># 读写分离监听配置</span></span><br><span class="line">[Read-Write-Listener]</span><br><span class="line"><span class="built_in">type</span>=listener</span><br><span class="line">service=Read-Write-Service</span><br><span class="line">protocol=MySQLClient</span><br><span class="line">port=4006</span><br></pre></td></tr></table></figure></li><li><p>启动MaxScale </p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 启动</span></span><br><span class="line">systemctl start maxscale</span><br><span class="line"><span class="comment"># 加入开机自启动</span></span><br><span class="line">systemctl <span class="built_in">enable</span> maxscale</span><br></pre></td></tr></table></figure></li><li><p>测试读写分离</p><p>用MySQL客户端或者命令连接上MaxScale，对数据库的表进行添加和查询数据，观察read/write值的变化</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 通过命令监控 read/write 的数量变化</span></span><br><span class="line">maxctrl show services</span><br><span class="line"><span class="comment"># 结果</span></span><br><span class="line">│ Router Diagnostics │ { │</span><br><span class="line">│ │ <span class="string">"avg_sescmd_history_length"</span>: 0, │</span><br><span class="line">│ │ <span class="string">"max_sescmd_history_length"</span>: 32, │</span><br><span class="line">│ │ <span class="string">"queries"</span>: 117, │</span><br><span class="line">│ │ <span class="string">"replayed_transactions"</span>: 0, │</span><br><span class="line">│ │ <span class="string">"ro_transactions"</span>: 0, │</span><br><span class="line">│ │ <span class="string">"route_all"</span>: 33, │</span><br><span class="line">│ │ <span class="string">"route_master"</span>: 9, │</span><br><span class="line">│ │ <span class="string">"route_slave"</span>: 75, │</span><br><span class="line">│ │ <span class="string">"rw_transactions"</span>: 0, │</span><br><span class="line">│ │ <span class="string">"server_query_statistics"</span>: [ │</span><br><span class="line">│ │ { │</span><br><span class="line">│ │ <span class="string">"avg_selects_per_session"</span>: 0, │</span><br><span class="line">│ │ <span class="string">"avg_sess_duration"</span>: <span class="string">"0ns"</span>, │</span><br><span class="line">│ │ <span class="string">"id"</span>: <span class="string">"server1"</span>, │</span><br><span class="line">│ │ <span class="string">"read"</span>: 33, │</span><br><span class="line">│ │ <span class="string">"total"</span>: 42, │</span><br><span class="line">│ │ <span class="string">"write"</span>: 9 │</span><br><span class="line">│ │ }, │</span><br><span class="line">│ │ { │</span><br><span class="line">│ │ <span class="string">"avg_selects_per_session"</span>: 0, │</span><br><span class="line">│ │ <span class="string">"avg_sess_duration"</span>: <span class="string">"0ns"</span>, │</span><br><span class="line">│ │ <span class="string">"id"</span>: <span class="string">"server2"</span>, │</span><br><span class="line">│ │ <span class="string">"read"</span>: 108, │</span><br><span class="line">│ │ <span class="string">"total"</span>: 108, │</span><br><span class="line">│ │ <span class="string">"write"</span>: 0 │</span><br><span class="line">│ │ } │</span><br><span class="line">│ │ ] │</span><br><span class="line">│ │ } </span><br><span class="line"></span><br><span class="line"><span class="comment"># 2. 通过MaxGUI监控 http://127.0.0.1:8989</span></span><br></pre></td></tr></table></figure></li><li><p>测试故障转移</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 查看server 情况</span></span><br><span class="line">maxctrl list servers</span><br><span class="line">┌─────────┬────────────────┬──────┬─────────────┬──────────────────────────────┬──────────┐</span><br><span class="line">│ Server │ Address │ Port │ Connections │ State │ GTID │</span><br><span class="line">├─────────┼────────────────┼──────┼─────────────┼──────────────────────────────┼──────────┤</span><br><span class="line">│ server1 │ 192.168.56.116 │ 3306 │ 0 │ Master, Running │ 0-117-20 │</span><br><span class="line">├─────────┼────────────────┼──────┼─────────────┼──────────────────────────────┼──────────┤</span><br><span class="line">│ server2 │ 192.168.56.117 │ 3306 │ 0 │ Relay Master, Slave, Running │ 0-117-20 │</span><br><span class="line">└─────────┴────────────────┴──────┴─────────────┴──────────────────────────────┴──────────┘</span><br><span class="line"></span><br><span class="line"><span class="comment"># 2. 登陆到master 服务器上停掉MariaDB</span></span><br><span class="line">systemctl stop MariaDB</span><br><span class="line"><span class="comment"># 3. 再次查看server情况</span></span><br><span class="line">maxctrl list servers</span><br><span class="line">┌─────────┬────────────────┬──────┬─────────────┬─────────────────┬──────────┐</span><br><span class="line">│ Server │ Address │ Port │ Connections │ State │ GTID │</span><br><span class="line">├─────────┼────────────────┼──────┼─────────────┼─────────────────┼──────────┤</span><br><span class="line">│ server1 │ 192.168.56.116 │ 3306 │ 0 │ Down │ 0-117-20 │</span><br><span class="line">├─────────┼────────────────┼──────┼─────────────┼─────────────────┼──────────┤</span><br><span class="line">│ server2 │ 192.168.56.117 │ 3306 │ 0 │ Master, Running │ 0-117-20 │</span><br><span class="line">└─────────┴────────────────┴──────┴─────────────┴─────────────────┴──────────┘</span><br><span class="line"><span class="comment"># 4. 再次启动停掉的MariaDB</span></span><br><span class="line">systemctl start mariadb</span><br><span class="line"><span class="comment"># 5. 再次查看server情况，启动的节点已经重新加入</span></span><br><span class="line">maxctrl list servers</span><br><span class="line">┌─────────┬────────────────┬──────┬─────────────┬─────────────────┬──────────┐</span><br><span class="line">│ Server │ Address │ Port │ Connections │ State │ GTID │</span><br><span class="line">├─────────┼────────────────┼──────┼─────────────┼─────────────────┼──────────┤</span><br><span class="line">│ server1 │ 192.168.56.116 │ 3306 │ 0 │ Slave, Running │ 0-117-20 │</span><br><span class="line">├─────────┼────────────────┼──────┼─────────────┼─────────────────┼──────────┤</span><br><span class="line">│ server2 │ 192.168.56.117 │ 3306 │ 0 │ Master, Running │ 0-117-20 │</span><br><span class="line">└─────────┴────────────────┴──────┴─────────────┴─────────────────┴──────────┘</span><br></pre></td></tr></table></figure></li></ol><h5 id="Slave-安装配置"><a href="#Slave-安装配置" class="headerlink" title="Slave 安装配置"></a>Slave 安装配置</h5><p> 同master安装配置</p><h4 id="Keepalived-搭建"><a href="#Keepalived-搭建" class="headerlink" title="Keepalived 搭建"></a>Keepalived 搭建</h4><h5 id="Master-安装配置-1"><a href="#Master-安装配置-1" class="headerlink" title="Master 安装配置"></a>Master 安装配置</h5><ol><li><p>安装</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum install keepalived</span><br></pre></td></tr></table></figure></li><li><p>配置</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1.先备份原来的配置文件，然后开始配置</span></span><br><span class="line">vi /etc/keepalived/keepalived.conf</span><br><span class="line">! Configuration File <span class="keyword">for</span> keepalived</span><br><span class="line"><span class="comment"># 全局配置</span></span><br><span class="line">global_defs {</span><br><span class="line"> router_id LVS_DEVEL_MASTER</span><br><span class="line"> script_user root</span><br><span class="line"> enable_script_security</span><br><span class="line">}</span><br><span class="line"><span class="comment"># 健康检查脚本</span></span><br><span class="line">vrrp_script chk_maxscale {</span><br><span class="line"> script <span class="string">"/usr/bin/pidof maxscale"</span></span><br><span class="line"> interval 2</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment"># 实例</span></span><br><span class="line">vrrp_instance VI_1 {</span><br><span class="line"> state MASTER</span><br><span class="line"> interface enp0s3</span><br><span class="line"> virtual_router_id 120</span><br><span class="line"> priority 150</span><br><span class="line"> advert_int 1</span><br><span class="line"> authentication {</span><br><span class="line"> auth_type PASS</span><br><span class="line"> auth_pass 1111</span><br><span class="line"> }</span><br><span class="line"> virtual_ipaddress {</span><br><span class="line"> 192.168.56.120/24</span><br><span class="line"> }</span><br><span class="line"> track_script {</span><br><span class="line"> chk_maxscale</span><br><span class="line"> }</span><br><span class="line"> <span class="comment"># 调试用,配置好可删除</span></span><br><span class="line"> debug 1</span><br><span class="line"> <span class="comment"># 通知脚本，用来改变MaxScale的passive的状态，防止多个MaxScale进行故障自动转移的操作</span></span><br><span class="line"> notify /etc/keepalived/scripts/notify_script.sh</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure></li><li><p> 通知脚本，该脚本最好放在<code>/etc/keepalived/</code>目录下，其他目录需要额外处理权限</p></li></ol> <figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#!/bin/bash</span></span><br><span class="line">TYPE=<span class="variable">$1</span></span><br><span class="line">NAME=<span class="variable">$2</span></span><br><span class="line">STATE=<span class="variable">$3</span></span><br><span class="line"><span class="comment"># 记录状态的文件</span></span><br><span class="line">OUTFILE=/etc/keepalived/scripts/state.txt</span><br><span class="line"></span><br><span class="line"><span class="keyword">case</span> <span class="variable">$STATE</span> <span class="keyword">in</span></span><br><span class="line"> <span class="string">"MASTER"</span>) <span class="built_in">echo</span> <span class="string">"Setting this MaxScale node to active mode"</span> > <span class="variable">$OUTFILE</span></span><br><span class="line"> <span class="comment">#/usr/bin/maxctrl alter maxscale passive false</span></span><br><span class="line"> <span class="built_in">exit</span> 0</span><br><span class="line"> ;;</span><br><span class="line"> <span class="string">"BACKUP"</span>) <span class="built_in">echo</span> <span class="string">"Setting this MaxScale node to passive mode"</span> > <span class="variable">$OUTFILE</span></span><br><span class="line"> <span class="comment">#/usr/bin/maxctrl alter maxscale passive true</span></span><br><span class="line"> <span class="built_in">exit</span> 0</span><br><span class="line"> ;;</span><br><span class="line"> <span class="string">"FAULT"</span>) <span class="built_in">echo</span> <span class="string">"MaxScale failed the status check."</span> > <span class="variable">$OUTFILE</span></span><br><span class="line"> <span class="comment">#/usr/bin/maxctrl alter maxscale passive true</span></span><br><span class="line"> <span class="built_in">exit</span> 0</span><br><span class="line"> ;;</span><br><span class="line"> *) <span class="built_in">echo</span> <span class="string">"Unknown state"</span> > <span class="variable">$OUTFILE</span></span><br><span class="line"> <span class="built_in">exit</span> 1</span><br><span class="line"> ;;</span><br><span class="line"><span class="keyword">esac</span></span><br></pre></td></tr></table></figure><ol start="4"><li><p><code>/etc/keepalived/scripts/notify_script.sh</code>脚本SELinux 权限配置</p><p> 网上很多方法是关闭SELinux ，实在解决不了可以关闭掉SELinux ，最好是不关闭，多一层防护</p></li></ol> <figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">chcon -t keepalived_unconfined_script_exec_t /etc/keepalived/scripts/notify_script.sh</span><br><span class="line">chcon -t keepalived_unconfined_script_exec_t /etc/keepalived/scripts/state.txt</span><br></pre></td></tr></table></figure><ol start="5"><li> 开通防火墙</li></ol> <figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">firewall-cmd --add-protocol=vrrp --permanent</span><br><span class="line">firewall-cmd --reload</span><br></pre></td></tr></table></figure><ol start="6"><li> 启动</li></ol> <figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 启动</span></span><br><span class="line">systemctl start keepalived</span><br><span class="line"><span class="comment"># 加入开机自启动</span></span><br><span class="line">systemctl <span class="built_in">enable</span> keepalived</span><br></pre></td></tr></table></figure><h5 id="Backup-安装配置"><a href="#Backup-安装配置" class="headerlink" title="Backup 安装配置"></a>Backup 安装配置</h5><p>和Master唯一不同是第5点，直接通过下面的配置代码介绍</p><ol><li> backup的配置</li></ol> <figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br></pre></td><td class="code"><pre><span class="line">global_defs {</span><br><span class="line"> <span class="comment"># router_id 不能和Master一样</span></span><br><span class="line"> router_id LVS_DEVEL_BACKUP</span><br><span class="line"> script_user root</span><br><span class="line"> enable_script_security</span><br><span class="line">}</span><br><span class="line">vrrp_script chk_maxscale {</span><br><span class="line"> script <span class="string">"/usr/sbin/pidof maxscale"</span></span><br><span class="line"> interval 2</span><br><span class="line">}</span><br><span class="line">vrrp_instance VI_1 {</span><br><span class="line"> <span class="comment"># 状态不一样，该机器是备机</span></span><br><span class="line"> state BACKUP</span><br><span class="line"> interface enp0s3</span><br><span class="line"> virtual_router_id 120</span><br><span class="line"> <span class="comment"># 优先级不一样，要低于Master</span></span><br><span class="line"> priority 100</span><br><span class="line"> advert_int 1</span><br><span class="line"> authentication {</span><br><span class="line"> auth_type PASS</span><br><span class="line"> auth_pass 1111</span><br><span class="line"> }</span><br><span class="line"> virtual_ipaddress {</span><br><span class="line"> 192.168.56.120/24</span><br><span class="line"> }</span><br><span class="line"> track_script {</span><br><span class="line"> chk_maxscale</span><br><span class="line"> }</span><br><span class="line"> debug 1</span><br><span class="line"> notify /etc/keepalived/scripts/notify_script.sh</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h5 id="测试"><a href="#测试" class="headerlink" title="测试"></a>测试</h5><ol><li> 首先分别监控keepalived master和backup 的日志</li></ol> <figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">journalctl -fu keepalived.service</span><br></pre></td></tr></table></figure><ol start="2"><li><p>监控MaxScale Master 和 backup的 passive 值</p><p>master：</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 执行命令</span></span><br><span class="line">maxctrl show maxscale</span><br><span class="line"><span class="comment"># 结果中的关键信息</span></span><br><span class="line">┌──────────────┬──────────────────────────────────────────────────────────┐</span><br><span class="line">│ Version │ 6.4.5 │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Commit │ e716c9cfc5f68f2e4ffada46c2d145918e7433bc │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Started At │ Fri, 17 Mar 2023 14:07:19 GMT │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Activated At │ Fri, 17 Mar 2023 14:07:19 GMT │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Uptime │ 1827 │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Config Sync │ null │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Parameters │ { │</span><br><span class="line">│ │ <span class="string">"passive"</span>: <span class="literal">false</span>, │</span><br><span class="line">│ │ <span class="string">"writeq_low_water"</span>: 8192 │</span><br><span class="line">│ │ } │</span><br><span class="line">└──────────────┴──────────────────────────────────────────────────────────┘</span><br></pre></td></tr></table></figure><p>backup:</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 执行命令</span></span><br><span class="line">maxctrl show maxscale</span><br><span class="line"><span class="comment"># 结果中的关键信息</span></span><br><span class="line">┌──────────────┬──────────────────────────────────────────────────────────┐</span><br><span class="line">│ Version │ 6.4.5 │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Commit │ e716c9cfc5f68f2e4ffada46c2d145918e7433bc │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Started At │ Fri, 17 Mar 2023 14:07:19 GMT │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Activated At │ Fri, 17 Mar 2023 14:07:19 GMT │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Uptime │ 1827 │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Config Sync │ null │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Parameters │ { │</span><br><span class="line">│ │ <span class="string">"passive"</span>: <span class="literal">true</span> , │</span><br><span class="line">│ │ <span class="string">"writeq_low_water"</span>: 8192 │</span><br><span class="line">│ │ } │</span><br><span class="line">└──────────────┴──────────────────────────────────────────────────────────┘</span><br></pre></td></tr></table></figure></li><li><p> 停掉Master 机器上的MaxScale 服务，观察keepalived master和backup 的日志情况</p></li></ol> <figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">systemctl stop maxscale</span><br></pre></td></tr></table></figure><p> keepalived master日志：结果显示进入了<code>Entering FAULT STATE</code></p> <figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">Mar 17 22:42:38 msk_master Keepalived_vrrp[2597]: Script `chk_maxscale` now returning 1</span><br><span class="line">Mar 17 22:42:38 msk_master Keepalived_vrrp[2597]: VRRP_Script(chk_maxscale) failed (exited with status 1)</span><br><span class="line">Mar 17 22:42:38 msk_master Keepalived_vrrp[2597]: (VI_1) Entering FAULT STATE</span><br><span class="line">Mar 17 22:42:38 msk_master Keepalived_vrrp[2597]: (VI_1) sent 0 priority</span><br><span class="line">Mar 17 22:42:38 msk_master Keepalived_vrrp[2597]: (VI_1) removing VIPs.</span><br></pre></td></tr></table></figure><p> keepalived backup 日志：结果显示已经进入<code>Entering MASTER STATE</code></p> <figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">Mar 17 22:42:38 msk_slave Keepalived_vrrp[4215]: (VI_1) Backup received priority 0 advertisement</span><br><span class="line">Mar 17 22:42:38 msk_slave Keepalived_vrrp[4215]: (VI_1) Backup received priority 0 advertisement</span><br><span class="line">Mar 17 22:42:39 msk_slave Keepalived_vrrp[4215]: (VI_1) Receive advertisement timeout</span><br><span class="line">Mar 17 22:42:39 msk_slave Keepalived_vrrp[4215]: (VI_1) Entering MASTER STATE</span><br><span class="line">Mar 17 22:42:39 msk_slave Keepalived_vrrp[4215]: (VI_1) setting VIPs.</span><br><span class="line">Mar 17 22:42:39 msk_slave Keepalived_vrrp[4215]: (VI_1) Sending/queueing gratuitous ARPs on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:42:39 msk_slave Keepalived_vrrp[4215]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:42:39 msk_slave Keepalived_vrrp[4215]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:42:39 msk_slave Keepalived_vrrp[4215]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:42:39 msk_slave Keepalived_vrrp[4215]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:42:39 msk_slave Keepalived_vrrp[4215]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:42:44 msk_slave Keepalived_vrrp[4215]: (VI_1) Sending/queueing gratuitous ARPs on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:42:44 msk_slave Keepalived_vrrp[4215]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:42:44 msk_slave Keepalived_vrrp[4215]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:42:44 msk_slave Keepalived_vrrp[4215]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:42:44 msk_slave Keepalived_vrrp[4215]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:42:44 msk_slave Keepalived_vrrp[4215]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br></pre></td></tr></table></figure><ol start="4"><li> 再查看MaxScale backup的状态</li></ol> <figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 执行命令</span></span><br><span class="line">maxctrl show maxscale</span><br><span class="line"><span class="comment"># 结果中的关键信息</span></span><br><span class="line">┌──────────────┬──────────────────────────────────────────────────────────┐</span><br><span class="line">│ Version │ 6.4.5 │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Commit │ e716c9cfc5f68f2e4ffada46c2d145918e7433bc │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Started At │ Fri, 17 Mar 2023 14:07:19 GMT │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Activated At │ Fri, 17 Mar 2023 14:07:19 GMT │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Uptime │ 1827 │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Config Sync │ null │</span><br><span class="line">├──────────────┼──────────────────────────────────────────────────────────┤</span><br><span class="line">│ Parameters │ { │</span><br><span class="line">│ │ <span class="string">"passive"</span>: <span class="literal">false</span>, │</span><br><span class="line">│ │ <span class="string">"writeq_low_water"</span>: 8192 │</span><br><span class="line">│ │ } │</span><br><span class="line">└──────────────┴──────────────────────────────────────────────────────────┘</span><br></pre></td></tr></table></figure><ol start="5"><li><p>再启动MaxScale Master服务，继续观察日志，可以看到之前的keepalived master 又恢复到了master状态，keepalived backup 也恢复到back状态</p><p>keepalived master日志：</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line">Mar 17 22:48:46 msk_master Keepalived_vrrp[2597]: Script `chk_maxscale` now returning 0</span><br><span class="line">Mar 17 22:48:46 msk_master Keepalived_vrrp[2597]: VRRP_Script(chk_maxscale) succeeded</span><br><span class="line">Mar 17 22:48:46 msk_master Keepalived_vrrp[2597]: (VI_1) Entering BACKUP STATE</span><br><span class="line">Mar 17 22:48:47 msk_master Keepalived_vrrp[2597]: (VI_1) received lower priority (100) advert from 192.168.56.119 - discarding</span><br><span class="line">Mar 17 22:48:48 msk_master Keepalived_vrrp[2597]: (VI_1) received lower priority (100) advert from 192.168.56.119 - discarding</span><br><span class="line">Mar 17 22:48:49 msk_master Keepalived_vrrp[2597]: (VI_1) received lower priority (100) advert from 192.168.56.119 - discarding</span><br><span class="line">Mar 17 22:48:50 msk_master Keepalived_vrrp[2597]: (VI_1) received lower priority (100) advert from 192.168.56.119 - discarding</span><br><span class="line">Mar 17 22:48:50 msk_master Keepalived_vrrp[2597]: (VI_1) Receive advertisement timeout</span><br><span class="line">Mar 17 22:48:50 msk_master Keepalived_vrrp[2597]: (VI_1) Entering MASTER STATE</span><br><span class="line">Mar 17 22:48:50 msk_master Keepalived_vrrp[2597]: (VI_1) setting VIPs.</span><br><span class="line">Mar 17 22:48:50 msk_master Keepalived_vrrp[2597]: (VI_1) Sending/queueing gratuitous ARPs on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:48:50 msk_master Keepalived_vrrp[2597]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:48:50 msk_master Keepalived_vrrp[2597]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:48:50 msk_master Keepalived_vrrp[2597]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:48:50 msk_master Keepalived_vrrp[2597]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:48:50 msk_master Keepalived_vrrp[2597]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:48:55 msk_master Keepalived_vrrp[2597]: (VI_1) Sending/queueing gratuitous ARPs on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:48:55 msk_master Keepalived_vrrp[2597]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:48:55 msk_master Keepalived_vrrp[2597]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:48:55 msk_master Keepalived_vrrp[2597]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:48:55 msk_master Keepalived_vrrp[2597]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br><span class="line">Mar 17 22:48:55 msk_master Keepalived_vrrp[2597]: Sending gratuitous ARP on enp0s3 <span class="keyword">for</span> 192.168.56.120</span><br></pre></td></tr></table></figure><p>keepalived backup 日志：</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">Mar 17 22:48:50 msk_slave Keepalived_vrrp[4215]: (VI_1) Master received advert from 192.168.56.118 with higher priority 150, ours 100</span><br><span class="line">Mar 17 22:48:50 msk_slave Keepalived_vrrp[4215]: (VI_1) Entering BACKUP STATE</span><br><span class="line">Mar 17 22:48:50 msk_slave Keepalived_vrrp[4215]: (VI_1) removing VIPs.</span><br></pre></td></tr></table></figure></li></ol><h4 id="结语"><a href="#结语" class="headerlink" title="结语"></a>结语</h4><p>算算刚刚起步，开始用起来！</p><h4 id="参考资料"><a href="#参考资料" class="headerlink" title="参考资料"></a>参考资料</h4><ol><li><a href="https://unix.stackexchange.com/questions/102558/mysql-permission-denied-error-even-after-setting-security-context-for-selinux-on">MySQL permission denied error even after setting security context for SELinux on VM</a></li><li><a href="https://github.com/malongshuai/proxysql/wiki">中文文档</a></li><li><a href="https://stackoverflow.com/questions/70923232/keepalived-notify-not-running-the-script">Keepalived notify not running the script</a></li></ol>]]></content>
<summary type="html"><h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote>
<p>在做基础脚手架开发的过程中涉及到代码要不要支持读写分离，支持的话的需要一定的工作量，虽然代码不多，也比较容易实现，但毕竟需要维护。为了避免重新写轮子，先查询。还有真有不少解决方案，第一种就是基于应用的，自己在应用程序中编码实现；第二种就是基于中间件，有MySQL Proxy、MaxScale和HAProxy等，MySQL Proxy已经不维护了；HAProxy本身不支持读写分离，还需要搭配其他才能实现；MaxScale 文档比较全，并且是MariaDB 开发的，所以先用起来看看效果。</p>
</blockquote>
<p><img src="/images/book/design_pattern.jpg"></p></summary>
<category term="database" scheme="http://tungsing.cc/categories/database/"/>
<category term="HA" scheme="http://tungsing.cc/tags/HA/"/>
<category term="High Availability" scheme="http://tungsing.cc/tags/High-Availability/"/>
<category term="MariaDB" scheme="http://tungsing.cc/tags/MariaDB/"/>
</entry>
<entry>
<title>搭建MySQL读写分离、高可用</title>
<link href="http://tungsing.cc/2023/03/04/mysql/mysql-ha/"/>
<id>http://tungsing.cc/2023/03/04/mysql/mysql-ha/</id>
<published>2023-03-04T13:24:20.407Z</published>
<updated>2023-03-20T09:57:26.910Z</updated>
<content type="html"><![CDATA[<h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote><p>在做基础脚手架开发的过程中涉及到代码要不要支持读写分离，支持的话的需要一定的工作量，虽然代码不多，也比较容易实现，但毕竟需要维护。为了避免重新写轮子，先查询。还有真有不少解决方案，第一种就是基于应用的，自己在应用程序中编码实现；第二种就是基于中间件，有MySQL Proxy、MaxScale和HAProxy等，MySQL Proxy已经不维护了；HAProxy本身不支持读写分离，还需要搭配其他才能实现；MaxScale 文档比较全，并且是MariaDB 开发的，所以先用起来看看效果。</p></blockquote><p><img src="/images/book/design_pattern.jpg"></p><span id="more"></span><h4 id="环境"><a href="#环境" class="headerlink" title="环境"></a>环境</h4><ul><li> Linux rocky 8 </li><li> MySQL 5.7</li><li> MariaDB MaxScale 6.4</li><li> ProxySQL 2.4.8</li></ul><h4 id="环境规划"><a href="#环境规划" class="headerlink" title="环境规划"></a>环境规划</h4><table><thead><tr><th align="center">序号</th><th>应用</th><th>hostname</th><th>IP</th><th>端口</th></tr></thead><tbody><tr><td align="center">1</td><td>MySQL Master</td><td></td><td>192.168.56.100</td><td>3306</td></tr><tr><td align="center">2</td><td>MySQL Slave</td><td></td><td>192.168.56.101</td><td>3306</td></tr><tr><td align="center">3</td><td>MaxScale</td><td></td><td>192.168.56.102</td><td>4006/8989</td></tr><tr><td align="center">4</td><td>ProxySQL</td><td></td><td>192.168.56.103</td><td>6033</td></tr></tbody></table><h4 id="MySQL-主从搭建"><a href="#MySQL-主从搭建" class="headerlink" title="MySQL 主从搭建"></a>MySQL 主从搭建</h4><h5 id="Master-安装"><a href="#Master-安装" class="headerlink" title="Master 安装"></a>Master 安装</h5><ol><li><p>下载安装包</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">wget https://cdn.mysql.com/archives/mysql-5.7/mysql-5.7.40-1.el7.x86_64.rpm-bundle.tar</span><br><span class="line">tar -xvf mysql-5.7.40-1.el7.x86_64.rpm-bundle.tar</span><br></pre></td></tr></table></figure></li><li><p>检查是否之前是否有安装，有删除</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">yum list installed | grep -i maria</span><br><span class="line">yum list installed | grep -i mysql</span><br></pre></td></tr></table></figure></li><li><p>安装</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum install mysql-community-{server,client,common,libs}-* --exclude=<span class="string">'*minimal*'</span></span><br></pre></td></tr></table></figure></li><li><p>修改默认数据目录，</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 创建目录</span></span><br><span class="line">mkdir /home/mysql_data</span><br><span class="line"><span class="comment"># 2. 拷贝目录数据目录</span></span><br><span class="line">cp -a /var/lib/mysql /home/mysql_data/</span><br><span class="line"><span class="comment"># 3. 修改MySQL配置文件</span></span><br><span class="line">vi /etc/my.cnf</span><br><span class="line"><span class="comment"># 4. 需要修改的内容</span></span><br><span class="line">datadir=/home/mysql_data/mysql</span><br><span class="line">socket=/home/mysql_data/mysql/mysql.sock</span><br><span class="line">[client]</span><br><span class="line">socket=/home/mysql_data/mysql/mysql.sock</span><br></pre></td></tr></table></figure></li><li><p>修改SELinux对MySQL默认数据目录的保护</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 打开/etc/selinux/targeted/contexts/files/file_contexts.local</span></span><br><span class="line">vi /etc/selinux/targeted/contexts/files/file_contexts.local</span><br><span class="line"><span class="comment"># 2. 添加以下内容</span></span><br><span class="line">/home/mysql_data(/.*)? system_u:object_r:mysqld_db_t:s0</span><br></pre></td></tr></table></figure></li><li><p>启动MySQL</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 启动</span></span><br><span class="line">systemctl start mysqld</span><br><span class="line"><span class="comment"># 开启自启动</span></span><br><span class="line">systemctl <span class="built_in">enable</span> mysqld</span><br></pre></td></tr></table></figure></li><li><p>找到<code>root</code>的默认密码</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">grep <span class="string">'A temporary password'</span> /var/<span class="built_in">log</span>/mysqld.log |tail -1</span><br></pre></td></tr></table></figure></li><li><p>修改默认密码</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 登陆到MySQL Server</span></span><br><span class="line">mysql -uroot -p</span><br><span class="line"><span class="comment"># 设置密码</span></span><br><span class="line">ALTER USER <span class="string">'root'</span>@<span class="string">'localhost'</span> IDENTIFIED BY <span class="string">'password'</span>;</span><br></pre></td></tr></table></figure></li></ol><h5 id="Slave-安装"><a href="#Slave-安装" class="headerlink" title="Slave 安装"></a>Slave 安装</h5><p>同master</p><h5 id="搭建配置"><a href="#搭建配置" class="headerlink" title="搭建配置"></a>搭建配置</h5><ol><li><p>修改master的<code>my.cnf</code>配置文件</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># server id 唯一</span></span><br><span class="line">server_id=112</span><br><span class="line"><span class="comment"># GTID 模式</span></span><br><span class="line">gtid-mode = ON</span><br><span class="line"><span class="comment"># 强制GTID一致性</span></span><br><span class="line">enforce-gtid-consistency = ON</span><br><span class="line"><span class="comment"># 是否记录从服务器同步数据动作</span></span><br><span class="line">log-slave-updates=ON</span><br><span class="line"><span class="comment"># 开启及设置二进制日志文件名称</span></span><br><span class="line">log_bin=mysql-bin</span><br><span class="line"><span class="comment"># 设置binlog文件格式</span></span><br><span class="line">binlog_format=MIXED</span><br><span class="line"></span><br><span class="line"><span class="comment"># 记录IO线程读取已经读取到的master binlog位置，用于slave宕机后IO线程根据文件中的POS点重新拉取binlog日志</span></span><br><span class="line">master-info-repository = TABLE</span><br><span class="line"><span class="comment"># 记录SQL线程读取Master binlog的位置，用于Slave 宕机后根据文件中记录的pos点恢复Sql线程</span></span><br><span class="line">relay-log-info-repository = TABLE </span><br><span class="line"><span class="comment"># 启用确保无信息丢失；任何一个事务提交后, 将二进制日志的文件名及事件位置记录到文件中</span></span><br><span class="line">sync-master-info = 1</span><br><span class="line"><span class="comment"># 设定从服务器的复制线程数；0表示关闭多线程复制功能</span></span><br><span class="line">slave-parallel-workers = 2</span><br><span class="line"><span class="comment"># 设置binlog校验算法（循环冗余校验码）</span></span><br><span class="line">binlog-checksum = CRC32</span><br><span class="line"><span class="comment"># 设置主服务器是否校验</span></span><br><span class="line">master-verify-checksum = 1</span><br><span class="line"><span class="comment"># 设置从服务器是否校验</span></span><br><span class="line">slave-sql-verify-checksum = 1</span><br><span class="line"><span class="comment"># 用于在二进制日志记录事件相关的信息，可降低故障排除的复杂度</span></span><br><span class="line">binlog-rows-query-log_events = 1</span><br><span class="line"><span class="comment"># 保证master crash safe，该参数必须设置为1</span></span><br><span class="line">sync_binlog = 1</span><br><span class="line"><span class="comment"># 保证master crash safe，该参数必须设置为1</span></span><br><span class="line">innodb_flush_log_at_trx_commit = 1 </span><br></pre></td></tr></table></figure></li><li><p> 重启MySQL Server</p></li><li><p>添加同步用户</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">GRANT REPLICATION SLAVE,RELOAD,SUPER ON *.* TO sync@<span class="string">'192.168.56.%'</span> IDENTIFIED BY <span class="string">'password'</span>;</span><br><span class="line">FLUSH PRIVILEGES;</span><br></pre></td></tr></table></figure></li><li><p>查看master 状态并记录下位置信息</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">mysql> show master status;</span><br><span class="line">+------------------+----------+--------------+-------------------------------------------------+-------------------+</span><br><span class="line">| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |</span><br><span class="line">+------------------+----------+--------------+-------------------------------------------------+-------------------+</span><br><span class="line">| mysql-bin.000008 | 154 | | sys,mysql,information_schema,performance_schema | |</span><br><span class="line">+------------------+----------+--------------+-------------------------------------------------+-------------------+</span><br><span class="line">1 row <span class="keyword">in</span> <span class="built_in">set</span> (0.00 sec)</span><br></pre></td></tr></table></figure></li><li><p>修改slave的<code>my.cnf</code> 配置文件</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># server id 唯一</span></span><br><span class="line">server-id=113</span><br><span class="line"><span class="comment"># GTID 模式</span></span><br><span class="line">gtid-mode = ON</span><br><span class="line"><span class="comment"># 强制GTID一致性</span></span><br><span class="line">enforce-gtid-consistency = ON</span><br><span class="line"><span class="comment"># 是否记录从服务器同步数据动作</span></span><br><span class="line">log-slave-updates=ON</span><br><span class="line"><span class="comment"># 开启及设置二进制日志文件名称</span></span><br><span class="line">log_bin=mysql-bin</span><br><span class="line"><span class="comment"># 设置binlog文件格式</span></span><br><span class="line">binlog_format=MIXED</span><br><span class="line"><span class="comment"># 设置一般用户为只读模式</span></span><br><span class="line">read-only=1</span><br><span class="line"></span><br><span class="line">master-info-repository = TABLE</span><br><span class="line">relay-log-info-repository = TABLE</span><br><span class="line">sync-master-info = 1</span><br><span class="line">slave-parallel-workers = 4</span><br><span class="line">binlog-checksum = CRC32</span><br><span class="line">master-verify-checksum = 1</span><br><span class="line">slave-sql-verify-checksum = 1</span><br><span class="line">binlog-rows-query-log_events = 1</span><br><span class="line"><span class="comment"># crash safe slave</span></span><br><span class="line">relay_log_recovery = 1</span><br></pre></td></tr></table></figure></li><li><p>重启MySQL Server</p></li><li><p>打开master 防火墙的3306 端口</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">firewall-cmd --zone=public --add-port=3306/tcp --permanent</span><br><span class="line">firewall-cmd --reload</span><br></pre></td></tr></table></figure></li><li><p>配置从主库同步数据</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 登陆MySQL slave Server</span></span><br><span class="line">mysql -uroot -p</span><br><span class="line"><span class="comment"># 2. 配置</span></span><br><span class="line"><span class="comment"># MASTER_LOG_FILE 和 MASTER_LOG_POS 的值就是从 主库 show master status 命令中获取的信息</span></span><br><span class="line">mysql> CHANGE MASTER TO MASTER_HOST=<span class="string">'192.168.56.115'</span>,</span><br><span class="line"> MASTER_USER=<span class="string">'sync'</span>,</span><br><span class="line"> MASTER_PASSWORD=<span class="string">'password'</span>,</span><br><span class="line"> MASTER_AUTO_POSITION=1;</span><br><span class="line"><span class="comment"># 3. 开启同步</span></span><br><span class="line">mysql> start slave;</span><br></pre></td></tr></table></figure></li><li><p> 查看同步情况</p></li></ol> <figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">mysql> show slave status\G</span><br><span class="line"><span class="comment"># 检查以下参数都是 Yes 代表正常</span></span><br><span class="line">Slave_IO_Running: Yes</span><br><span class="line">Slave_SQL_Running: Yes</span><br><span class="line"></span><br></pre></td></tr></table></figure><ol start="10"><li> 测试，通过创建数据库、表，添加数据来看同步是否正常</li></ol><h4 id="MaxScale-搭建"><a href="#MaxScale-搭建" class="headerlink" title="MaxScale 搭建"></a>MaxScale 搭建</h4><h5 id="Master-安装配置"><a href="#Master-安装配置" class="headerlink" title="Master 安装配置"></a>Master 安装配置</h5><ol><li><p>通过<a href="https://mariadb.com/downloads/community/maxscale/">官网</a>下载安装包</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">wget https://dlm.mariadb.com/2717592/MaxScale/6.4.5/yum/centos/8/x86_64/maxscale-6.4.5-1.rhel.8.x86_64.rpm</span><br></pre></td></tr></table></figure></li><li><p>安装</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum install maxscale-6.4.5-1.rhel.8.x86_64.rpm</span><br></pre></td></tr></table></figure></li><li><p>在MySQL Master 上添加监控和路由用户</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 连接MySQL Server</span></span><br><span class="line">mysql -uroot -p</span><br><span class="line"><span class="comment"># 2. 添加监控用户</span></span><br><span class="line">grant replication slave,replication client on *.* to <span class="string">'maxmon'</span>@<span class="string">'192.168.56.%'</span> identified by <span class="string">'password'</span>;</span><br><span class="line"><span class="comment"># 3. 添加路由用户</span></span><br><span class="line">grant select on mysql.* to <span class="string">'maxrou'</span>@<span class="string">'192.168.56.%'</span> identified by <span class="string">'password'</span>;</span><br><span class="line"><span class="comment"># 4. 刷新权限信息</span></span><br><span class="line">flush privileges;</span><br></pre></td></tr></table></figure></li><li><p>读写分离配置，具体可阅读<a href="https://github.com/mariadb-corporation/MaxScale/blob/6.4.5/Documentation/Documentation-Contents.md">官方配置</a></p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 打开配置文件</span></span><br><span class="line">vim /etc/maxscale.cnf</span><br><span class="line"><span class="comment"># 2. 配置</span></span><br><span class="line"><span class="comment"># 基础配置</span></span><br><span class="line">[maxscale]</span><br><span class="line">threads=auto</span><br><span class="line">admin_host=0.0.0.0</span><br><span class="line">admin_secure_gui=<span class="literal">false</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># MySQL Server 配置</span></span><br><span class="line">[server1]</span><br><span class="line"><span class="built_in">type</span>=server</span><br><span class="line">address=192.168.56.112</span><br><span class="line">port=3306</span><br><span class="line">protocol=MySQLBackend</span><br><span class="line"></span><br><span class="line">[server2]</span><br><span class="line"><span class="built_in">type</span>=server</span><br><span class="line">address=192.168.56.113</span><br><span class="line">port=3306</span><br><span class="line">protocol=MySQLBackend</span><br><span class="line"><span class="comment"># 监控配置</span></span><br><span class="line">[MariaDB-Monitor]</span><br><span class="line"><span class="built_in">type</span>=monitor</span><br><span class="line">module=mariadbmon</span><br><span class="line">servers=server1,server2</span><br><span class="line">user=maxmon</span><br><span class="line">password=password</span><br><span class="line">monitor_interval=2000</span><br><span class="line"><span class="comment"># 过滤器配置</span></span><br><span class="line"><span class="comment"># 非必须配置，是我使用的客户端工具默认会执行解释计划，但默认explain plan 会被路由到从库</span></span><br><span class="line">[Named-Server-Filter]</span><br><span class="line"><span class="built_in">type</span>=filter</span><br><span class="line">module=namedserverfilter</span><br><span class="line">match01=^explain.*</span><br><span class="line">target01=server1</span><br><span class="line"></span><br><span class="line"><span class="comment"># 读写分离服务配置</span></span><br><span class="line">[Read-Write-Service]</span><br><span class="line"><span class="built_in">type</span>=service</span><br><span class="line">router=readwritesplit</span><br><span class="line">servers=server1,server2</span><br><span class="line">user=maxrou</span><br><span class="line">password=password</span><br><span class="line">filters=Named-Server-Filter</span><br><span class="line"></span><br><span class="line"><span class="comment"># 读写分离监听配置</span></span><br><span class="line">[Read-Write-Listener]</span><br><span class="line"><span class="built_in">type</span>=listener</span><br><span class="line">service=Read-Write-Service</span><br><span class="line">protocol=MySQLClient</span><br><span class="line">port=4006</span><br></pre></td></tr></table></figure></li><li><p>启动MaxScale </p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 启动</span></span><br><span class="line">systemctl start maxscale</span><br><span class="line"><span class="comment"># 加入开机自启动</span></span><br><span class="line">systemctl <span class="built_in">enable</span> maxscale</span><br></pre></td></tr></table></figure></li><li><p>测试</p><p>用MySQL客户端或者命令连接上MaxScale，对数据库的表进行添加和查询数据，观察read/write值的变化</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 通过命令监控 read/write 的数量变化</span></span><br><span class="line">maxctrl show services</span><br><span class="line"><span class="comment"># 结果</span></span><br><span class="line">│ Router Diagnostics │ { │</span><br><span class="line">│ │ <span class="string">"avg_sescmd_history_length"</span>: 0, │</span><br><span class="line">│ │ <span class="string">"max_sescmd_history_length"</span>: 32, │</span><br><span class="line">│ │ <span class="string">"queries"</span>: 117, │</span><br><span class="line">│ │ <span class="string">"replayed_transactions"</span>: 0, │</span><br><span class="line">│ │ <span class="string">"ro_transactions"</span>: 0, │</span><br><span class="line">│ │ <span class="string">"route_all"</span>: 33, │</span><br><span class="line">│ │ <span class="string">"route_master"</span>: 9, │</span><br><span class="line">│ │ <span class="string">"route_slave"</span>: 75, │</span><br><span class="line">│ │ <span class="string">"rw_transactions"</span>: 0, │</span><br><span class="line">│ │ <span class="string">"server_query_statistics"</span>: [ │</span><br><span class="line">│ │ { │</span><br><span class="line">│ │ <span class="string">"avg_selects_per_session"</span>: 0, │</span><br><span class="line">│ │ <span class="string">"avg_sess_duration"</span>: <span class="string">"0ns"</span>, │</span><br><span class="line">│ │ <span class="string">"id"</span>: <span class="string">"server1"</span>, │</span><br><span class="line">│ │ <span class="string">"read"</span>: 33, │</span><br><span class="line">│ │ <span class="string">"total"</span>: 42, │</span><br><span class="line">│ │ <span class="string">"write"</span>: 9 │</span><br><span class="line">│ │ }, │</span><br><span class="line">│ │ { │</span><br><span class="line">│ │ <span class="string">"avg_selects_per_session"</span>: 0, │</span><br><span class="line">│ │ <span class="string">"avg_sess_duration"</span>: <span class="string">"0ns"</span>, │</span><br><span class="line">│ │ <span class="string">"id"</span>: <span class="string">"server2"</span>, │</span><br><span class="line">│ │ <span class="string">"read"</span>: 108, │</span><br><span class="line">│ │ <span class="string">"total"</span>: 108, │</span><br><span class="line">│ │ <span class="string">"write"</span>: 0 │</span><br><span class="line">│ │ } │</span><br><span class="line">│ │ ] │</span><br><span class="line">│ │ } </span><br><span class="line"></span><br><span class="line"><span class="comment"># 2. 通过MaxGUI监控 http://127.0.0.1:8989</span></span><br></pre></td></tr></table></figure></li><li><p>注意</p><p>MaxScale 不支持MySQL的故障自动转移</p></li></ol><h4 id="ProxySQL-搭建"><a href="#ProxySQL-搭建" class="headerlink" title="ProxySQL 搭建"></a>ProxySQL 搭建</h4><h5 id="Node-1-搭建"><a href="#Node-1-搭建" class="headerlink" title="Node 1 搭建"></a>Node 1 搭建</h5><ol><li><p>通过<a href="https://github.com/sysown/proxysql">github</a>下载安装包</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">wget https://github.com/sysown/proxysql/releases/download/v2.4.8/proxysql-2.4.8-1-centos8.x86_64.rpm</span><br></pre></td></tr></table></figure></li><li><p>安装</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum install proxysql-2.4.8-1-centos8.x86_64.rpm</span><br></pre></td></tr></table></figure></li><li><p>在MySQL Master 上添加监控用户和测试用户</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"># <span class="number">1.</span> 连接MySQL Server</span><br><span class="line">mysql <span class="operator">-</span>uroot <span class="operator">-</span>p</span><br><span class="line"># <span class="number">2.</span> 添加监控用户</span><br><span class="line"><span class="keyword">grant</span> replication slave,replication client <span class="keyword">on</span> <span class="operator">*</span>.<span class="operator">*</span> <span class="keyword">to</span> <span class="string">'maxmon'</span>@<span class="string">'192.168.56.%'</span> identified <span class="keyword">by</span> <span class="string">'password'</span>;</span><br><span class="line"># <span class="number">3.</span> 测试用户</span><br><span class="line"><span class="keyword">grant</span> <span class="keyword">all</span> <span class="keyword">on</span> ivy_base.<span class="operator">*</span> <span class="keyword">to</span> <span class="string">'ivy_base'</span>@<span class="string">'%'</span> identified <span class="keyword">by</span> <span class="string">'password'</span>;</span><br><span class="line"># <span class="number">3.</span> 刷新权限信息</span><br><span class="line">flush privileges;</span><br></pre></td></tr></table></figure></li><li><p>读写分离配置</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 打开配置文件</span></span><br><span class="line">vi /etc/proxysql.cnf</span><br><span class="line"><span class="comment"># 2. 配置</span></span><br><span class="line"><span class="comment"># 数据目录和日志文件</span></span><br><span class="line">datadir=<span class="string">"/var/lib/proxysql"</span></span><br><span class="line">errorlog=<span class="string">"/var/lib/proxysql/proxysql.log"</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 设置管理变量，包括监听的端口、管理账号等。</span></span><br><span class="line">admin_variables=</span><br><span class="line">{</span><br><span class="line">admin_credentials=<span class="string">"admin:admin"</span></span><br><span class="line">mysql_ifaces=<span class="string">"0.0.0.0:6032"</span></span><br><span class="line">debug=<span class="literal">true</span></span><br><span class="line"> web_enabled=<span class="literal">true</span></span><br><span class="line">}</span><br><span class="line"><span class="comment"># 设置MySQL 数据库 变量，监听账号、密码等。</span></span><br><span class="line">mysql_variables=</span><br><span class="line">{</span><br><span class="line">threads=4</span><br><span class="line">max_connections=2048</span><br><span class="line">default_query_delay=0</span><br><span class="line">default_query_timeout=36000000</span><br><span class="line">have_compress=<span class="literal">true</span></span><br><span class="line">poll_timeout=2000</span><br><span class="line">interfaces=<span class="string">"0.0.0.0:6033"</span></span><br><span class="line">default_schema=<span class="string">"information_schema"</span></span><br><span class="line">stacksize=1048576</span><br><span class="line">server_version=<span class="string">"5.5.30"</span></span><br><span class="line">connect_timeout_server=3000</span><br><span class="line">monitor_username=<span class="string">"maxmon"</span></span><br><span class="line">monitor_password=<span class="string">"1234@abCD"</span></span><br><span class="line">monitor_history=600000</span><br><span class="line">monitor_connect_interval=60000</span><br><span class="line">monitor_ping_interval=10000</span><br><span class="line">monitor_read_only_interval=1500</span><br><span class="line">monitor_read_only_timeout=500</span><br><span class="line">ping_interval_server_msec=120000</span><br><span class="line">ping_timeout_server=500</span><br><span class="line">commands_stats=<span class="literal">true</span></span><br><span class="line">sessions_sort=<span class="literal">true</span></span><br><span class="line">connect_retries_on_failure=10</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment"># defines all the MySQL servers</span></span><br><span class="line">mysql_servers =</span><br><span class="line">(</span><br><span class="line">{ address=<span class="string">"192.168.56.115"</span> , port=3306 , hostgroup=0 },</span><br><span class="line">{ address=<span class="string">"192.168.56.113"</span> , port=3306 , hostgroup=1 },</span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="comment"># defines all the MySQL users</span></span><br><span class="line">mysql_users:</span><br><span class="line">(</span><br><span class="line"> { username = <span class="string">"ivy_base"</span> , password = <span class="string">"1234@abCD"</span> , default_hostgroup = 0 , active = 1 }</span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="comment">#定义查询路由规则</span></span><br><span class="line">mysql_query_rules:</span><br><span class="line">(</span><br><span class="line">{</span><br><span class="line">rule_id=1</span><br><span class="line">active=1</span><br><span class="line">match_pattern=<span class="string">"^SELECT .* FOR UPDATE$"</span></span><br><span class="line">destination_hostgroup=0</span><br><span class="line">apply=1</span><br><span class="line">},</span><br><span class="line">{</span><br><span class="line">rule_id=2</span><br><span class="line">active=1</span><br><span class="line">match_pattern=<span class="string">"^SELECT"</span></span><br><span class="line">destination_hostgroup=1</span><br><span class="line">apply=1</span><br><span class="line">},</span><br><span class="line"> {</span><br><span class="line"> rule_id=3</span><br><span class="line"> active=1</span><br><span class="line"> match_pattern=<span class="string">"^explain"</span></span><br><span class="line"> destination_hostgroup=0</span><br><span class="line"> apply=1</span><br><span class="line"> }</span><br><span class="line">)</span><br><span class="line"><span class="comment">#调度器是一个类似于cron的实现，具有毫秒的粒度。</span></span><br><span class="line">scheduler=</span><br><span class="line">(</span><br><span class="line">)</span><br><span class="line"><span class="comment"># 定义 hostgroup 的主从关系</span></span><br><span class="line">mysql_replication_hostgroups=</span><br><span class="line">(</span><br><span class="line">)</span><br></pre></td></tr></table></figure></li><li><p> 启动proxysql</p></li></ol> <figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 启动</span></span><br><span class="line">systemctl start proxysql</span><br><span class="line"><span class="comment"># 加入开机自启动</span></span><br><span class="line">systemctl <span class="built_in">enable</span> proxysql</span><br></pre></td></tr></table></figure><ol start="6"><li><p>在ProxySQL 服务器上安装MySQL client</p><p>同MySQL Master 搭建相同，只是不安装server端</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum install mysql-community-{client,common,libs}-* --exclude=<span class="string">'*minimal*'</span></span><br></pre></td></tr></table></figure></li><li><p>打开防火墙端口<code>6033</code></p></li><li><p>测试</p><p>用MySQL客户端或者命令连接上proxysql，对数据库的表进行添加和查询数据，观察read/write值的变化</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 在主或从MySQL数据库服务器上连接proxysql</span></span><br><span class="line">mysql -h 192.168.56.112 -P 6033 -uivy_base -p</span><br><span class="line"><span class="comment"># 2. 执行创建表、添加数据、查询数据等操作</span></span><br><span class="line">CREATE TABLE `<span class="built_in">test</span>` (</span><br><span class="line"> `id` int(11) NOT NULL,</span><br><span class="line"> `name` varchar(200) DEFAULT NULL,</span><br><span class="line"> PRIMARY KEY (`id`)</span><br><span class="line">) ;</span><br><span class="line">INSERT INTO <span class="built_in">test</span> (id, NAME) VALUES (<span class="string">'2'</span>, <span class="string">'1'</span>);</span><br><span class="line">select * from <span class="built_in">test</span>;</span><br><span class="line"><span class="comment"># 3. 在proxysql观察情况</span></span><br><span class="line">mysql -uadmin -p -h 127.0.0.1 -P 6032 --prompt=<span class="string">'admin>'</span></span><br><span class="line"><span class="comment">#执行语句监控</span></span><br><span class="line">select hostgroup,username,digest_text,count_star from stats_mysql_query_digest;</span><br><span class="line"><span class="comment"># 结果，通过缓存hostgroup和语句我们发现我们是配置成功的</span></span><br><span class="line">+-----------+-------------------------------------------------------------------------------------------------+------------+</span><br><span class="line">| hostgroup | digest_text | count_star |</span><br><span class="line">+-----------+-------------------------------------------------------------------------------------------------+------------+</span><br><span class="line">| 0 | INSERT INTO <span class="built_in">test</span> (id,NAME) VALUES (?,?) | 1 |</span><br><span class="line">| 0 | CREATE TABLE `<span class="built_in">test</span>` ( `id` int(?) NOT NULL, `name` varchar(?) DEFAULT NULL, PRIMARY KEY (`id`)) | 1 |</span><br><span class="line">| 1 | select * from <span class="built_in">test</span> | 3 |</span><br><span class="line">| 0 | CREATE TABLE `<span class="built_in">test</span>` ( `id` int(?) NOT NULL, `name` varchar(?) DEFAULT NULL, PRIMARY KEY (`id`)) | 1 |</span><br><span class="line">| 1 | SELECT * FROM `douban_book` WHERE ?=? | 1 |</span><br><span class="line">| 0 | select @@version_comment <span class="built_in">limit</span> ? | 1 |</span><br><span class="line">| 1 | SELECT DATABASE() | 1 |</span><br><span class="line">| 0 | show databases | 1 |</span><br><span class="line">| 0 | show tables | 1 |</span><br><span class="line">+-----------+-------------------------------------------------------------------------------------------------+------------+</span><br><span class="line"></span><br><span class="line"></span><br></pre></td></tr></table></figure></li></ol><h5 id="ProxySQL-配置说明"><a href="#ProxySQL-配置说明" class="headerlink" title="ProxySQL 配置说明"></a>ProxySQL 配置说明</h5><p>ProxySQL 配置有两种方式一种是通过<code>/etc/proxysql.cnf</code>配置文件,另一种是通过admin interface方式（即在proxysql本机使用mysql客户端连接管理端口），但第二种方式依赖于第一种方式，因为一些基础的配置必须通过配置文件来配置。</p><p>ProxySQL有配置文件<code>/etc/proxysql.cnf</code>和配置数据库文件<code>/var/lib/proxysql/proxysql.db</code>。</p><p><strong>特别注意：</strong></p><p>ProxySQL服务只有在第一次启动时才会去读取proxysql.cnf文件并解析；后面启动会就不会读取proxysql.cnf文件了！</p><p>如果想要让<code>proxysql.cnf</code>文件里的配置在重启proxysql服务后生效(即想要让proxysql重启时读取并解析proxysql.cnf配置文件)，则需要先删除<code>/var/lib/proxysql/proxysql.db</code>数据库文件，然后再重启proxysql服务。这样就相当于初始化启动proxysql服务了，会再次生产一个纯净的proxysql.db数据库文件(如果通过admin interface方式配置的数据，则就会被抹掉)。</p><p><strong>官方推荐</strong>用admin interface方式！</p><ol><li><p>ProxySQL 库介绍</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">admin<span class="operator">></span><span class="keyword">show</span> databases;</span><br><span class="line"><span class="operator">+</span><span class="comment">-----+---------------+-------------------------------------+</span></span><br><span class="line"><span class="operator">|</span> seq <span class="operator">|</span> name <span class="operator">|</span> file <span class="operator">|</span></span><br><span class="line"><span class="operator">+</span><span class="comment">-----+---------------+-------------------------------------+</span></span><br><span class="line"><span class="operator">|</span> <span class="number">0</span> <span class="operator">|</span> main <span class="operator">|</span> <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> <span class="number">2</span> <span class="operator">|</span> disk <span class="operator">|</span> <span class="operator">/</span>var<span class="operator">/</span>lib<span class="operator">/</span>proxysql<span class="operator">/</span>proxysql.db <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> <span class="number">3</span> <span class="operator">|</span> stats <span class="operator">|</span> <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> <span class="number">4</span> <span class="operator">|</span> monitor <span class="operator">|</span> <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> <span class="number">5</span> <span class="operator">|</span> stats_history <span class="operator">|</span> <span class="operator">/</span>var<span class="operator">/</span>lib<span class="operator">/</span>proxysql<span class="operator">/</span>proxysql_stats.db <span class="operator">|</span></span><br><span class="line"><span class="operator">+</span><span class="comment">-----+---------------+-------------------------------------+</span></span><br></pre></td></tr></table></figure><p>**main:**配置数据库，表里存放后端DB实例、用户验证、路由规则等信息。表名以 runtime_开头的表示proxysql当前运行的配置内容，不能通过dml语句修改，只能修改对应的不以 runtime_ 开头的（在内存）里的表，然后 LOAD 使其生效， SAVE 使其存到硬盘以供下次重启加载。</p><p>**disk:**是持久化到硬盘的配置，sqlite数据文件。默认位置为 $（DATADIR）/proxysql.db</p><p>**stats:**proxysql运行抓取的统计信息，包括到后端各命令的执行次数、流量、processlist、查询种类汇总/执行时间等等</p><p>**monitor:**存储 monitor 模块收集的信息，主要是对后端db的健康/延迟检查。</p><p>**stats_history:**统计信息历史库</p></li><li><p>main 库</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br></pre></td><td class="code"><pre><span class="line">admin<span class="operator">></span> <span class="keyword">show</span> tables <span class="keyword">from</span> main;</span><br><span class="line"><span class="operator">+</span><span class="comment">----------------------------------------------------+</span></span><br><span class="line"><span class="operator">|</span> tables <span class="operator">|</span></span><br><span class="line"><span class="operator">+</span><span class="comment">----------------------------------------------------+</span></span><br><span class="line"><span class="operator">|</span> global_variables <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_aws_aurora_hostgroups <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_collations <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_firewall_whitelist_rules <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_firewall_whitelist_sqli_fingerprints <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_firewall_whitelist_users <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_galera_hostgroups <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_group_replication_hostgroups <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_query_rules <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_query_rules_fast_routing <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_replication_hostgroups <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_servers <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_users <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> proxysql_servers <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> restapi_routes <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> runtime_checksums_values <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> runtime_global_variables <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> runtime_mysql_aws_aurora_hostgroups <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> runtime_mysql_firewall_whitelist_rules <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> runtime_mysql_firewall_whitelist_sqli_fingerprints <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> runtime_mysql_firewall_whitelist_users <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> runtime_mysql_galera_hostgroups <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> runtime_mysql_group_replication_hostgroups <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> runtime_mysql_query_rules <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> runtime_mysql_query_rules_fast_routing <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> runtime_mysql_replication_hostgroups <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> runtime_mysql_servers <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> runtime_mysql_users <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> runtime_proxysql_servers <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> runtime_restapi_routes <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> runtime_scheduler <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> scheduler <span class="operator">|</span></span><br><span class="line"><span class="operator">+</span><span class="comment">----------------------------------------------------+</span></span><br></pre></td></tr></table></figure><p>常用表介绍：</p><p><strong>global_variables：</strong>用于定义全局变量。这是一个非常简单的表，本质上是一个key/value存储系统。<br><strong>mysql_collations：</strong>相关字符集和校验规则。<br><strong>mysql_query_rules：</strong>定义查询路由规则。<br><strong>mysql_servers：</strong>设置后端MySQL的表<br><strong>mysql_users：</strong>配置后端数据库的程序账号和监控账号。<br><strong>scheduler：</strong>调度器是一个类似于cron的实现，集成在ProxySQL中，具有毫秒的粒度。通过脚本检测来设置ProxySQL。</p><p><strong>mysql_replication_hostgroups：</strong>监视指定主机组中所有服务器的read_only值，并且根据read_only的值将服务器分配给写入器或读取器主机组。ProxySQL monitor模块会监控hostgroups后端所有servers 的read_only 变量，如果发现从库的read_only变为0、主库变为1，则认为角色互换了，自动改写mysql_servers表里面 hostgroup关系，达到自动 Failover 效果。</p><p>*<em>runtime_</em>:**运行时的配置，这些是不能修改的</p></li><li><p>disk 库</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line">admin<span class="operator">></span> <span class="keyword">show</span> tables <span class="keyword">from</span> disk;</span><br><span class="line"><span class="operator">+</span><span class="comment">--------------------------------------------+</span></span><br><span class="line"><span class="operator">|</span> tables <span class="operator">|</span></span><br><span class="line"><span class="operator">+</span><span class="comment">--------------------------------------------+</span></span><br><span class="line"><span class="operator">|</span> global_settings <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> global_variables <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_aws_aurora_hostgroups <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_collations <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_firewall_whitelist_rules <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_firewall_whitelist_sqli_fingerprints <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_firewall_whitelist_users <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_galera_hostgroups <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_group_replication_hostgroups <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_query_rules <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_query_rules_fast_routing <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_replication_hostgroups <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_servers <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_users <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> proxysql_servers <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> restapi_routes <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> scheduler <span class="operator">|</span></span><br><span class="line"><span class="operator">+</span><span class="comment">--------------------------------------------+</span></span><br></pre></td></tr></table></figure><p>用于将配置持久化到磁盘上。配置持久化后，下次重启ProxySQL时就会读取这些已被持久化的配置。这些表完全对应于内存数据库中的表。</p></li><li><p>stats 库</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br></pre></td><td class="code"><pre><span class="line">admin<span class="operator">></span> <span class="keyword">show</span> tables <span class="keyword">from</span> stats;</span><br><span class="line"><span class="operator">+</span><span class="comment">---------------------------------------+</span></span><br><span class="line"><span class="operator">|</span> tables <span class="operator">|</span></span><br><span class="line"><span class="operator">+</span><span class="comment">---------------------------------------+</span></span><br><span class="line"><span class="operator">|</span> global_variables <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_memory_metrics <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_mysql_client_host_cache <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_mysql_client_host_cache_reset <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_mysql_commands_counters <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_mysql_connection_pool <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_mysql_connection_pool_reset <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_mysql_errors <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_mysql_errors_reset <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_mysql_free_connections <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_mysql_global <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_mysql_gtid_executed <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_mysql_prepared_statements_info <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_mysql_processlist <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_mysql_query_digest <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_mysql_query_digest_reset <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_mysql_query_rules <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_mysql_users <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_proxysql_message_metrics <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_proxysql_message_metrics_reset <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_proxysql_servers_checksums <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_proxysql_servers_clients_status <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_proxysql_servers_metrics <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> stats_proxysql_servers_status <span class="operator">|</span></span><br><span class="line"><span class="operator">+</span><span class="comment">---------------------------------------+</span></span><br></pre></td></tr></table></figure><p>常用表：</p><p><strong>stats_mysql_commands_counters：</strong>统计各种SQL类型的执行次数和时间，通过参数mysql-commands_stats控制开关，默认是ture。<br><strong>stats_mysql_connection_pool：</strong>连接后端MySQL的连接信息。<br><strong>stats_mysql_processlist：</strong>类似MySQL的show processlist的命令，查看各线程的状态。<br><strong>stats_mysql_query_digest：</strong>表示SQL的执行次数、时间消耗等。通过变量mysql-query_digests控制开关，默认是开。<br><strong>stats_mysql_query_rules：</strong>路由命中次数统计。</p></li><li><p>monitor 库</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">admin<span class="operator">></span> <span class="keyword">show</span> tables <span class="keyword">from</span> monitor;</span><br><span class="line"><span class="operator">+</span><span class="comment">--------------------------------------+</span></span><br><span class="line"><span class="operator">|</span> tables <span class="operator">|</span></span><br><span class="line"><span class="operator">+</span><span class="comment">--------------------------------------+</span></span><br><span class="line"><span class="operator">|</span> mysql_server_aws_aurora_check_status <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_server_aws_aurora_failovers <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_server_aws_aurora_log <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_server_connect_log <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_server_galera_log <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_server_group_replication_log <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_server_ping_log <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_server_read_only_log <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_server_replication_lag_log <span class="operator">|</span></span><br><span class="line"><span class="operator">+</span><span class="comment">--------------------------------------+</span></span><br></pre></td></tr></table></figure><p>常用表：</p><p><strong>mysql_server_connect_log：</strong>连接到所有MySQL服务器以检查它们是否可用，该表用来存放检测连接的日志。<br><strong>mysql_server_ping_log：</strong>使用mysql_ping API ping后端MySQL服务器，检查它们是否可用，该表用来存放ping的日志。<br><strong>mysql_server_replication_lag_log：</strong>后端MySQL服务主从延迟的检测。</p></li><li><p>stats_history库</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line">admin<span class="operator">></span> <span class="keyword">show</span> tables <span class="keyword">from</span> stats_history;</span><br><span class="line"><span class="operator">+</span><span class="comment">---------------------------------------+</span></span><br><span class="line"><span class="operator">|</span> tables <span class="operator">|</span></span><br><span class="line"><span class="operator">+</span><span class="comment">---------------------------------------+</span></span><br><span class="line"><span class="operator">|</span> history_mysql_query_digest <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> history_mysql_status_variables <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> history_mysql_status_variables_lookup <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> history_stats_mysql_connection_pool <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> myhgm_connections <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> myhgm_connections_day <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> myhgm_connections_hour <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_connections <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_connections_day <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_connections_hour <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_query_cache <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_query_cache_day <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> mysql_query_cache_hour <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> system_cpu <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> system_cpu_day <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> system_cpu_hour <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> system_memory <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> system_memory_day <span class="operator">|</span></span><br><span class="line"><span class="operator">|</span> system_memory_hour <span class="operator">|</span></span><br><span class="line"><span class="operator">+</span><span class="comment">---------------------------------------+</span></span><br></pre></td></tr></table></figure></li><li><p>配置主要表介绍</p><ul><li><strong>mysql_servers</strong></li></ul><table><thead><tr><th align="left">列</th><th align="left">说明</th></tr></thead><tbody><tr><td align="left">hostgroup_id</td><td align="left">ProxySQL通过 hostgroup (<strong>下称HG</strong>) 的形式组织后端db实例。一个 HG 代表同属于一个角色<br/>该表的主键是 (hostgroup_id, hostname, port)，可以看到一个 hostname:port 可以在多个hostgroup里面，这样可以避免 HG 1000 的从库全都不可用时，依然可以把读请求发到主库上。<br/>一个 HG 可以有多个实例，即多个从库，可以通过 weight 分配权重<br/>hostgroup_id 0 是一个特殊的HG，路由查询的时候，没有匹配到规则则默认选择 HG 0</td></tr><tr><td align="left">hostname</td><td align="left">后端MySQL监听的地址</td></tr><tr><td align="left">port</td><td align="left">后端MySQL监听的端口</td></tr><tr><td align="left">gtid_port</td><td align="left">ProxySQL Binlog Reader为追踪GTID而监听的后端服务器端口。</td></tr><tr><td align="left">status</td><td align="left">ONLINE: 当前后端实例状态正常<br/>SHUNNED: 临时被剔除，可能因为后端 too many connections error，或者超过了可容忍延迟阀值 max_replication_lag<br/>OFFLINE_SOFT: “软离线”状态，不再接受新的连接，但已建立的连接会等待活跃事务完成。<br/>OFFLINE_HARD: “硬离线”状态，不再接受新的连接，已建立的连接或被强制中断。当后端实例宕机或网络不可达，会出现。</td></tr><tr><td align="left">weight</td><td align="left">节点在组中的权重值越高，ProxySQL会发送越多请求给它们。</td></tr><tr><td align="left">compression</td><td align="left">如果该字段的值设置为大于0，ProxySQL和该后端新建的连接中，ProxySQL将会先压缩数据再传输。</td></tr><tr><td align="left">max_connections</td><td align="left">允许连接到该后端mysql实例的最大连接数。不要大于MySQL设置的 max_connections，如果后端实例 hostname:port 在多个 hostgroup 里，以较大者为准，而不是各自独立允许的最大连接数。</td></tr><tr><td align="left">max_replication_lag</td><td align="left">允许的最大延迟，主库不受这个影响，默认0。如果 > 0， monitor 模块监控主从延迟大于阀值时，会临时把它变为 SHUNNED</td></tr><tr><td align="left">use_ssl</td><td align="left">如果设置为1，则和该后端MySQL建立SSL连接。</td></tr><tr><td align="left">max_latency_ms</td><td align="left">mysql_ping 响应时间，大于这个阀值会把它从连接池剔除（即使是ONLINE）</td></tr><tr><td align="left">comment</td><td align="left">备注，不建议留空。可以通过它的内容如json格式的数据，配合自己写的check脚本，完成一些自动化的工作。</td></tr></tbody></table><ul><li><p><strong>mysql_query_rules</strong></p><p>ProxySQL非常核心一个表，定义查询路由规则</p></li></ul><table><thead><tr><th align="left">列</th><th>说明</th></tr></thead><tbody><tr><td align="left">rule_id</td><td>表主键，自增。规则处理是以 rule_id 的顺序进行。</td></tr><tr><td align="left">active</td><td>只有 active=1 的规则才会参与匹配。</td></tr><tr><td align="left">username</td><td>如果非 NULL，只有连接用户是 username 的值才会匹配。</td></tr><tr><td align="left">schemaname</td><td>如果非 NULL，只有查询连接使用的db是 schemaname 的值才会匹配。注意如果是 NULL，不代表连接没有使用schema，而是不伦任何schema都进一步匹配。</td></tr><tr><td align="left">flagIN</td><td>flagIN, flagOUT, apply: 用来定义路由链 chains of rules。<br/> <strong>-</strong> 首先会检查 flagIN=0 的规则，以rule_id的顺序；如果都没匹配上，则走这个用户的 default_hostgroup。<br/> <strong>-</strong> 当匹配一条规则后，会检查 flagOUT。<br/> <strong>-</strong> 如果不为NULL，并且 flagIN != flagOUT ，则进入以flagIN为上一个flagOUT值的新规则链。<br/> <strong>-</strong> 如果不为NULL，并且 flagIN = flagOUT，则应用这条规则。<br/> <strong>-</strong> 如果为NULL，或者 apply=1，则结束，应用这条规则。<br/> <strong>-</strong> 如果最终没有匹配到，则找到这个用户的 default_hostgroup。</td></tr><tr><td align="left">client_addr</td><td>匹配客户端来源IP</td></tr><tr><td align="left">proxy_addr</td><td>当流入的查询是在本地某地址上时，将匹配。</td></tr><tr><td align="left">proxy_port</td><td>当流入的查询是在本地某端口上时，将匹配。</td></tr><tr><td align="left">digest</td><td>精确的匹配一类查询。</td></tr><tr><td align="left">match_digest</td><td>正则匹配一类查询。query digest 是指对查询去掉具体值后进行“模糊化”后的查询，类似 pt-fingerprint / pt-query-digest 的效果。</td></tr><tr><td align="left">match_pattern</td><td>正则匹配查询。推荐用 match_digest 。关于每条查询都会计算digest对性能的影响，计算query digest确实会有性能损失，但是这却是proxysql里面非常重要的特性，主要是两点：<br/> <strong>-</strong> proxysql无法知道连接复用(multipexing)是否必须被自动禁用，比如连接里面有variables/tmp tables/lock table等特殊命令，是不能复用的。<br/> <strong>-</strong> 完整的查询去匹配正则的效率，一般没有参数化后的查询匹配效率高，因为有很长的字符串内容需要处理。再者，SELECT * FROM randomtable WHERE comment LIKE ‘%INTO sbtest1 % FROM sbtest2 %’字符串里有类似这样的语句，很难排除误匹配。</td></tr><tr><td align="left">negate_match_pattern</td><td>反向匹配，相当于对 match_digest/match_pattern 的匹配取反。</td></tr><tr><td align="left">re_modifiers</td><td>修改正则匹配的参数，比如默认的：忽略大小写CASELESS、禁用GLOBAL.</td></tr><tr><td align="left">flagOUT</td><td><strong>上面都是匹配规则，下面是匹配后的行为</strong></td></tr><tr><td align="left">replace_pattern</td><td>查询重写，默认为空，不rewrite。</td></tr><tr><td align="left">destination_hostgroup</td><td>路由查询到这个 hostgroup。当然如果用户显式 start transaction 且 transaction_persistent=1，那么即使匹配到了，也依然按照事务里第一条sql的路由规则去走。</td></tr><tr><td align="left">cache_ttl</td><td>查询结果缓存的毫秒数。</td></tr><tr><td align="left">cache_empty_result</td><td>控制结果集为空是否会被缓存。</td></tr><tr><td align="left">cache_timeout</td><td>没有实现</td></tr><tr><td align="left">reconnect</td><td>未使用的功能</td></tr><tr><td align="left">timeout</td><td>这一类查询执行的最大时间（毫秒），超时则自动kill。<br/>这是对后端DB的保护机制，相当于阿里云RDS loose_max_statement_time 变量的功能，但是注意不同的是，阿里云这个变量的时间时不包括DML操作出现InnoDB行锁等待的时间，而ProxySQL的这个 timeout 是计算从发送sql到等待响应的时间。默认mysql-default_query_timeout给的是 10h .</td></tr><tr><td align="left">retries</td><td>语句在执行时失败时，重试次数。默认由 mysql-query_retries_on_failure变量指定，为1 。<br/> 个人建议把它设成0，即不重试。因为执行失败，对select而言很少见，主要是dml，但自己重试对数据不放心。</td></tr><tr><td align="left">delay</td><td>查询延迟执行，这是ProxySQL提供的限流机制，会让其它的查询优先执行。<br/> 默认值 mysql-default_query_delay，为0。我们一般不用，其实还是要配合应用端使用，比如这边延迟执行，但上层等待你返回，那前端不就堵住了，没准出现雪崩效应。</td></tr><tr><td align="left">next_query_flagIN</td><td></td></tr><tr><td align="left">mirror_flagOUT</td><td></td></tr><tr><td align="left">mirror_hostgroup</td><td></td></tr><tr><td align="left">error_msg</td><td>默认为NULL，如果指定了则这个查询直接被 block 掉，马上返回这个错误信息。<br/>这个功能也很实用，比如线上突然冒出一个 “坏查询”，应用端不方便马上发版解决，我们就可以在这配置一个规则，把查询屏蔽掉，想正常的mysql报错那样抛异常。</td></tr><tr><td align="left">OK_msg</td><td>对于使用定义的规则的查询，将返回指定的信息。</td></tr><tr><td align="left">sticky_conn</td><td>没实现</td></tr><tr><td align="left">multiplex</td><td>连接是否复用。</td></tr><tr><td align="left">gtid_from_hostgroup</td><td>定义哪个主机组应该被用作GTID一致读取的领导者（通常是复制主机组对中定义的WRITER主机组）。</td></tr><tr><td align="left">log</td><td>是否记录查询日志。可以看到log是否记录的对象是根据规则。<br/>要开启日志记录，需要设置变量 mysql-eventslog_filename 来指定文件名，然后这个 log 标记为1。但是目前proxysql记录的日志是二进制格式，需要特定的工具才能读取： eventslog_reader_sample 。这个工具在源码目录 tools下面。</td></tr><tr><td align="left">apply</td><td>当设置为1后，当匹配到该规则后，将立即应用该规则，不会再评估其它的规则(注意：应用之后，将不会评估<code>mysql_query_rules_fast_routing</code>中的规则)。</td></tr><tr><td align="left">attributes</td><td>没实现</td></tr><tr><td align="left">comment</td><td>注释说明字段，例如描述规则的意义</td></tr></tbody></table><ul><li><strong>mysql_users</strong></li></ul><table><thead><tr><th>列</th><th>说明</th></tr></thead><tbody><tr><td>username</td><td>连接后端db的用户密码。</td></tr><tr><td>password</td><td>这个密码你可以插入明文，也可以插入hash加密后的密文，proxysql会检查你插入的时候密码是否以 * 开头来判断，而且密文要在其它地方使用 PASSWORD()生成。但到 runtime_mysql_users 里，都统一变成了密文，所以可以明文插入，再 SAVE MYSQL USERS TO MEM，此时看到的也是HASH密文。</td></tr><tr><td>active</td><td>是否生效该用户。默认1</td></tr><tr><td>use_ssl</td><td>是否开启ssl 默认1</td></tr><tr><td>default_hostgroup</td><td>这个用户的请求没有匹配到规则时，默认发到这个 hostgroup，默认0</td></tr><tr><td>default_schema</td><td>这个用户连接时没有指定 database name 时，默认使用的schema<br/>注意表面上看默认为NULL，但实际上受到变量 mysql-default_schema 的影响，默认为 information_schema。</td></tr><tr><td>schema_locked</td><td>还不支持</td></tr><tr><td>transaction_persistent</td><td>如果设置为1，连接上ProxySQL的会话后，如果在一个hostgroup上开启了事务，那么后续的sql都继续维持在这个hostgroup上，不伦是否会匹配上其它路由规则，直到事务结束。默认1</td></tr><tr><td>fast_forward</td><td>忽略查询重写/缓存层，直接把这个用户的请求透传到后端DB。相当于只用它的连接池功能，一般不用，路由规则 .* 就行了。</td></tr><tr><td>backend</td><td>目前版本这两个都需要使用默认的1，将来有可能会把 Client -> ProxySQL (frontend) 与 ProxySQL -> BackendDB (backend)的认证分开。从 runtime_mysql_users 表内容看到，记录数比 mysql_users 多了一倍，就是把前端认证与后端认证独立出来的结果</td></tr><tr><td>frontend</td><td></td></tr><tr><td>max_connections</td><td>定义了一个特定用户的最大允许的前端连接数。</td></tr><tr><td>attributes</td><td>没实现</td></tr><tr><td>comment</td><td>文本字段，可用于用户定义的任何目的。可以是对集群存储内容的描述，也可以是对主机组添加或禁用时间的提醒，或者是一些检查器脚本所处理的JSON。</td></tr></tbody></table><ul><li><strong>mysql_replication_hostgroups</strong></li></ul><table><thead><tr><th>列</th><th>说明</th></tr></thead><tbody><tr><td>writer_hostgroup</td><td>默认的写组。后端<code>read_only=0</code>的节点会分配到这个组中。</td></tr><tr><td>reader_hostgroup</td><td>负责读的组。查询规则或者只具有只读权限的用户的读请求都会路由到该主机组中的节点。后端<code>read_only=1</code>的节点会分配到这个组中。</td></tr><tr><td>check_type</td><td>默认read_only</td></tr><tr><td>comment</td><td>该字段用于说明、注释。</td></tr></tbody></table></li></ol><h4 id="结语"><a href="#结语" class="headerlink" title="结语"></a>结语</h4><p>只支持读写分离，MySQL 主节点出问题，还是会有问题，后续可以把MySQL搭建为双主</p><h4 id="参考资料"><a href="#参考资料" class="headerlink" title="参考资料"></a>参考资料</h4><ol><li><a href="https://unix.stackexchange.com/questions/102558/mysql-permission-denied-error-even-after-setting-security-context-for-selinux-on">MySQL permission denied error even after setting security context for SELinux on VM</a></li><li><a href="https://gist.github.com/linuxkathirvel/96b6f6be705d5e569c0c7425cea70273">MySQL 5.7 installation in CentOS 7</a></li><li><a href="https://www.cnblogs.com/kevingrace/p/10329714.html">ProxySQL 配置详解及读写分离(+GTID)等功能说明 (完整篇)</a></li><li><a href="https://github.com/malongshuai/proxysql/wiki">中文文档</a></li></ol>]]></content>
<summary type="html"><h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote>
<p>在做基础脚手架开发的过程中涉及到代码要不要支持读写分离，支持的话的需要一定的工作量，虽然代码不多，也比较容易实现，但毕竟需要维护。为了避免重新写轮子，先查询。还有真有不少解决方案，第一种就是基于应用的，自己在应用程序中编码实现；第二种就是基于中间件，有MySQL Proxy、MaxScale和HAProxy等，MySQL Proxy已经不维护了；HAProxy本身不支持读写分离，还需要搭配其他才能实现；MaxScale 文档比较全，并且是MariaDB 开发的，所以先用起来看看效果。</p>
</blockquote>
<p><img src="/images/book/design_pattern.jpg"></p></summary>
<category term="database" scheme="http://tungsing.cc/categories/database/"/>
<category term="HA" scheme="http://tungsing.cc/tags/HA/"/>
<category term="High Availability" scheme="http://tungsing.cc/tags/High-Availability/"/>
<category term="MySQL" scheme="http://tungsing.cc/tags/MySQL/"/>
</entry>
<entry>
<title>Elasticsearch learning</title>
<link href="http://tungsing.cc/2023/02/21/es/es_learning/"/>
<id>http://tungsing.cc/2023/02/21/es/es_learning/</id>
<published>2023-02-21T02:49:03.521Z</published>
<updated>2023-03-20T09:55:22.457Z</updated>
<content type="html"><![CDATA[<h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote><p>学习了一遍Elasticsearch，做个demo巩固一下。</p></blockquote><p><img src="/images/es/search.jpg"></p><span id="more"></span><h4 id="环境情况"><a href="#环境情况" class="headerlink" title="环境情况"></a>环境情况</h4><ul><li>Rocky Linux 8 </li><li>Elasticsearch 8.6.1 </li><li>Logstash 8.6.2 </li><li>Enterprise-search 8.6.2 </li><li>MySQL 5.7 </li></ul><h4 id="搭建Elasticsearch"><a href="#搭建Elasticsearch" class="headerlink" title="搭建Elasticsearch"></a>搭建Elasticsearch</h4><ol><li><p><a href="https://www.elastic.co/cn/downloads/past-releases#elasticsearch">官网</a>下载安装包</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 下载</span></span><br><span class="line">wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.6.1-linux-x86_64.tar.gz</span><br><span class="line"><span class="comment"># 解压</span></span><br><span class="line">tar -zxvf elasticsearch-8.6.1-linux-x86_64.tar.gz</span><br></pre></td></tr></table></figure></li><li><p>添加用户和用户组</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 添加用户组</span></span><br><span class="line">groupadd elasticsearch</span><br><span class="line"><span class="comment"># 添加用户</span></span><br><span class="line">adduser elasticsearch</span><br><span class="line"><span class="comment"># 设置密码</span></span><br><span class="line">passwd elasticsearch</span><br></pre></td></tr></table></figure></li><li><p>授权</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 移动到用户目录，并授权</span></span><br><span class="line">mv elasticsearch-8.6.1 /home/elasticsearch/</span><br><span class="line">chown -R elasticsearch:elasticsearch elasticsearch-8.6.1</span><br></pre></td></tr></table></figure></li><li><p>修改配置</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 打开配置文件</span></span><br><span class="line">vi /home/elasticsearch/elasticsearch-8.6.1/config/elasticsearch.yml</span><br><span class="line"><span class="comment"># 修改内容</span></span><br><span class="line">cluster.name: book-search</span><br><span class="line">network.host: 0.0.0.0</span><br><span class="line">http.host: 0.0.0.0</span><br></pre></td></tr></table></figure></li><li><p>启动</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 启动</span></span><br><span class="line">./elasticsearch</span><br><span class="line"><span class="comment">#观察日志输出，找到初始化密码，方便后续修改</span></span><br><span class="line">✅ Elasticsearch security features have been automatically configured!</span><br><span class="line">✅ Authentication is enabled and cluster connections are encrypted.</span><br><span class="line"></span><br><span class="line">ℹ️ Password <span class="keyword">for</span> the elastic user (reset with `bin/elasticsearch-reset-password -u elastic`):</span><br><span class="line"> IUFw46*Ws5cLJ*lMi4y9</span><br><span class="line"></span><br><span class="line">ℹ️ HTTP CA certificate SHA-256 fingerprint:</span><br><span class="line"> 2e4531eebb1aa5d4d1c57948672f2fcb1b08bbdc748b757ee48d722483ca02d5</span><br><span class="line"></span><br><span class="line">ℹ️ Configure Kibana to use this cluster:</span><br><span class="line">• Run Kibana and click the configuration link <span class="keyword">in</span> the terminal when Kibana starts.</span><br><span class="line">• Copy the following enrollment token and paste it into Kibana <span class="keyword">in</span> your browser (valid <span class="keyword">for</span> the next 30 minutes):</span><br><span class="line"> eyJ2ZXIiOiI4LjYuMSIsImFkciI6WyIxOTIuMTY4LjU2LjExMjo5MjAwIl0sImZnciI6IjJlNDUzMWVlYmIxYWE1ZDRkMWM1Nzk0ODY3MmYyZmNiMWIwOGJiZGM3NDhiNzU3ZWU0OGQ3MjI0ODNjYTAyZDUiLCJrZXkiOiI1di13YzRZQmgtYVdzNDhpZGV6XzpERkF2empPVlQ3V0tTbXgwRHk3NE5nIn0=</span><br><span class="line"></span><br><span class="line">ℹ️ Configure other nodes to join this cluster:</span><br><span class="line">• On this node:</span><br><span class="line"> ⁃ Create an enrollment token with `bin/elasticsearch-create-enrollment-token -s node`.</span><br><span class="line"> ⁃ Uncomment the transport.host setting at the end of config/elasticsearch.yml.</span><br><span class="line"> ⁃ Restart Elasticsearch.</span><br><span class="line">• On other nodes:</span><br><span class="line"> ⁃ Start Elasticsearch with `bin/elasticsearch --enrollment-token <token>`, using the enrollment token that you generated.</span><br></pre></td></tr></table></figure></li><li><p>其他</p><p>其他可根据自己环境的实际情况来配置和添加</p></li><li><p>测试</p><p>在浏览器输入Elasticsearch的地址查看信息</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 地址</span></span><br><span class="line">https://192.168.56.112:9200/</span><br><span class="line"><span class="comment"># 结果</span></span><br><span class="line">{</span><br><span class="line"> <span class="string">"name"</span>: <span class="string">"localhost.localdomain"</span>,</span><br><span class="line"> <span class="string">"cluster_name"</span>: <span class="string">"book-search"</span>,</span><br><span class="line"> <span class="string">"cluster_uuid"</span>: <span class="string">"Ms7sU-N1QpSE6y4ZBTQi0g"</span>,</span><br><span class="line"> <span class="string">"version"</span>: {</span><br><span class="line"> <span class="string">"number"</span>: <span class="string">"8.6.1"</span>,</span><br><span class="line"> <span class="string">"build_flavor"</span>: <span class="string">"default"</span>,</span><br><span class="line"> <span class="string">"build_type"</span>: <span class="string">"tar"</span>,</span><br><span class="line"> <span class="string">"build_hash"</span>: <span class="string">"180c9830da956993e59e2cd70eb32b5e383ea42c"</span>,</span><br><span class="line"> <span class="string">"build_date"</span>: <span class="string">"2023-01-24T21:35:11.506992272Z"</span>,</span><br><span class="line"> <span class="string">"build_snapshot"</span>: <span class="literal">false</span>,</span><br><span class="line"> <span class="string">"lucene_version"</span>: <span class="string">"9.4.2"</span>,</span><br><span class="line"> <span class="string">"minimum_wire_compatibility_version"</span>: <span class="string">"7.17.0"</span>,</span><br><span class="line"> <span class="string">"minimum_index_compatibility_version"</span>: <span class="string">"7.0.0"</span></span><br><span class="line"> },</span><br><span class="line"> <span class="string">"tagline"</span>: <span class="string">"You Know, for Search"</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure></li></ol><h4 id="准备图书数据"><a href="#准备图书数据" class="headerlink" title="准备图书数据"></a>准备图书数据</h4><p>从互联网中找到一份图书数据导入MySQL数据中</p><p>数据结构：</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> `douban_book` (</span><br><span class="line"> `id` <span class="type">bigint</span>(<span class="number">20</span>) <span class="keyword">NOT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `book_name` <span class="type">varchar</span>(<span class="number">512</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `rank` <span class="type">decimal</span>(<span class="number">10</span>,<span class="number">0</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `people_num` <span class="type">bigint</span>(<span class="number">20</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `author` <span class="type">varchar</span>(<span class="number">512</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `public_infos` <span class="type">varchar</span>(<span class="number">1024</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `type` <span class="type">varchar</span>(<span class="number">256</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `public` <span class="type">varchar</span>(<span class="number">128</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `public_time` <span class="type">varchar</span>(<span class="number">128</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `price` <span class="type">varchar</span>(<span class="number">128</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `rank_rank` <span class="type">bigint</span>(<span class="number">20</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `people_num_rank` <span class="type">bigint</span>(<span class="number">20</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `scores` <span class="type">decimal</span>(<span class="number">10</span>,<span class="number">0</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `update_time` <span class="type">timestamp</span> <span class="keyword">NULL</span> <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> `valid` <span class="type">varchar</span>(<span class="number">1</span>) <span class="keyword">DEFAULT</span> <span class="keyword">NULL</span>,</span><br><span class="line"> <span class="keyword">PRIMARY</span> KEY (`id`)</span><br><span class="line">) ENGINE<span class="operator">=</span>InnoDB <span class="keyword">DEFAULT</span> CHARSET<span class="operator">=</span>utf8mb4</span><br></pre></td></tr></table></figure><p>示例数据：</p><p><img src="/images/es/mysql_table_book.png"></p><h4 id="MySQL数据同步到Elasticsearch"><a href="#MySQL数据同步到Elasticsearch" class="headerlink" title="MySQL数据同步到Elasticsearch"></a>MySQL数据同步到Elasticsearch</h4><ol><li><p>先创建索引</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 创建索引</span></span><br><span class="line">PUT <span class="string">"/idx_book?pretty"</span></span><br><span class="line"><span class="comment"># 更新索引索引mapping，当然也可以在创建索引的时候设置mapping</span></span><br><span class="line">PUT <span class="string">"idx_book_test/_mapping"</span></span><br><span class="line">{</span><br><span class="line"><span class="string">"properties"</span>: {</span><br><span class="line"><span class="string">"author"</span>: {</span><br><span class="line"><span class="string">"type"</span>: <span class="string">"text"</span></span><br><span class="line">},</span><br><span class="line"><span class="string">"bookName"</span>: {</span><br><span class="line"><span class="string">"type"</span>: <span class="string">"text"</span></span><br><span class="line">},</span><br><span class="line"><span class="string">"id"</span>: {</span><br><span class="line"><span class="string">"type"</span>: <span class="string">"integer"</span></span><br><span class="line">},</span><br><span class="line"><span class="string">"peopleNum"</span>: {</span><br><span class="line"><span class="string">"type"</span>: <span class="string">"integer"</span></span><br><span class="line">},</span><br><span class="line"><span class="string">"peopleNumRank"</span>: {</span><br><span class="line"><span class="string">"type"</span>: <span class="string">"integer"</span></span><br><span class="line">},</span><br><span class="line"><span class="string">"price"</span>: {</span><br><span class="line"><span class="string">"type"</span>: <span class="string">"text"</span></span><br><span class="line">},</span><br><span class="line"><span class="string">"public"</span>: {</span><br><span class="line"><span class="string">"type"</span>: <span class="string">"text"</span></span><br><span class="line">},</span><br><span class="line"><span class="string">"publicInfos"</span>: {</span><br><span class="line"><span class="string">"type"</span>: <span class="string">"text"</span></span><br><span class="line">},</span><br><span class="line"><span class="string">"publicTime"</span>: {</span><br><span class="line"><span class="string">"type"</span>: <span class="string">"text"</span></span><br><span class="line">},</span><br><span class="line"><span class="string">"rank"</span>: {</span><br><span class="line"><span class="string">"type"</span>: <span class="string">"double"</span></span><br><span class="line">},</span><br><span class="line"><span class="string">"rankRank"</span>: {</span><br><span class="line"><span class="string">"type"</span>: <span class="string">"integer"</span></span><br><span class="line">},</span><br><span class="line"><span class="string">"scores"</span>: {</span><br><span class="line"><span class="string">"type"</span>: <span class="string">"double"</span></span><br><span class="line">},</span><br><span class="line"><span class="string">"type"</span>: {</span><br><span class="line"><span class="string">"type"</span>: <span class="string">"text"</span></span><br><span class="line">}</span><br><span class="line">}</span><br><span class="line">}</span><br></pre></td></tr></table></figure></li><li><p> Logstash 安装-略</p></li><li><p>使用Logstash 同步MySQL数据到Elasticsearch</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. 把MySQL JDBC 驱动拷贝到服务器上</span></span><br><span class="line">/home/elasticsearch/logstash-8.6.2/lib/mysql-connector-java-6.0.6.jar</span><br><span class="line"><span class="comment"># 2. 编写一个同步配置文件mysql-2-es.conf</span></span><br><span class="line">input {</span><br><span class="line"> jdbc {</span><br><span class="line"> jdbc_driver_library => <span class="string">"/home/elasticsearch/logstash-8.6.2/lib/mysql-connector-java-6.0.6.jar"</span></span><br><span class="line"> jdbc_driver_class => <span class="string">"com.mysql.cj.jdbc.Driver"</span></span><br><span class="line"> jdbc_connection_string => <span class="string">"jdbc:mysql://192.168.56.1:3306/base?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&useSSL=false"</span></span><br><span class="line"> jdbc_user => root</span><br><span class="line"> jdbc_password => <span class="string">"123456"</span></span><br><span class="line"> use_column_value => <span class="literal">true</span></span><br><span class="line"> tracking_column => <span class="string">"update_time"</span></span><br><span class="line"> tracking_column_type => <span class="string">"numeric"</span></span><br><span class="line"> record_last_run => <span class="literal">true</span></span><br><span class="line"> last_run_metadata_path => <span class="string">"jdbc-position.txt"</span></span><br><span class="line"> statement => <span class="string">"SELECT * FROM douban_book where update_time >:sql_last_value;"</span></span><br><span class="line"> schedule => <span class="string">" * 0/10 * * * *"</span></span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">output {</span><br><span class="line"> elasticsearch {</span><br><span class="line"> document_id => <span class="string">"%{id}"</span></span><br><span class="line"> document_type => <span class="string">"_doc"</span></span><br><span class="line"> index => <span class="string">"idx_book_test"</span></span><br><span class="line"> hosts => [<span class="string">"https://127.0.0.1:9200"</span>]</span><br><span class="line"> cacert => <span class="string">"/home/elasticsearch/elasticsearch-8.6.1/config/certs/http_ca.crt"</span> </span><br><span class="line"> user => <span class="string">"elastic"</span></span><br><span class="line"> password => <span class="string">"IUFw46*Ws5cLJ*lMi4y9"</span></span><br><span class="line"> }</span><br><span class="line"> stdout{</span><br><span class="line"> codec => rubydebug</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure></li><li><p>启动</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">./logstash -f /home/elasticsearch/logstash-8.6.2/config/mysql-2-es.conf</span><br></pre></td></tr></table></figure></li><li><p>可以通过API或者ES工具查询索引数据</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">GET idx_book_test/_search</span><br></pre></td></tr></table></figure></li><li><p>总结</p><p>关于数据同步有几种方案</p><ul><li>如果业务数量小，可以直接用代码写同步逻辑</li><li>中小规模的情况下，可以使用Logstash 等数据同步工具</li><li>再大的数据规模就用读取MySQL binlog 日志文件的方式了</li></ul></li></ol><h4 id="开发图书检索页面"><a href="#开发图书检索页面" class="headerlink" title="开发图书检索页面"></a>开发图书检索页面</h4><p>页面开发基于enterprise-search和search-ui开发的</p><ol><li><p>下载enterprise-search安装配置</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">wget https://artifacts.elastic.co/downloads/enterprise-search/enterprise-search-8.6.2.tar.gz</span><br></pre></td></tr></table></figure></li><li><p>安装配置enterprise-search</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1. java 环境变量配置</span></span><br><span class="line">vi .bashrc</span><br><span class="line"><span class="comment"># 添加</span></span><br><span class="line"><span class="built_in">export</span> JAVA_HOME=/home/elasticsearch/enterprise-search-8.6.2/jdk-11.0.16.1</span><br><span class="line"><span class="built_in">export</span> PATH=<span class="variable">$JAVA_HOME</span>/bin:<span class="variable">$PATH</span></span><br><span class="line"><span class="built_in">export</span> CLASSPATH=.:<span class="variable">$JAVA_HOME</span>/lib/dt.jar:<span class="variable">$JAVA_HOME</span>/lib/tools.jar</span><br><span class="line"></span><br><span class="line"><span class="comment"># 重新加载</span></span><br><span class="line"><span class="built_in">source</span> .bashrc</span><br><span class="line"></span><br><span class="line"><span class="comment">#2. 配置/home/elasticsearch/enterprise-search-8.6.2/config/enterprise-search.yml</span></span><br><span class="line">secret_management.encryption_keys: [<span class="string">'6dd517534da70c63094dcde920585fb1a8a1b2bc9b96e93bbc7a553efaad0b78'</span>]</span><br><span class="line">allow_es_settings_modification: <span class="literal">true</span></span><br><span class="line">elasticsearch.host: https://127.0.0.1:9200</span><br><span class="line">elasticsearch.username: elastic</span><br><span class="line">elasticsearch.password: IUFw46*Ws5cLJ*lMi4y9</span><br><span class="line">elasticsearch.ssl.enabled: <span class="literal">true</span></span><br><span class="line">elasticsearch.ssl.certificate_authority: /home/elasticsearch/elasticsearch-8.6.1/config/certs/http_ca.crt</span><br><span class="line">kibana.host: http://192.168.56.112:5601</span><br><span class="line">ent_search.listen_host: 0.0.0.0</span><br><span class="line">ent_search.external_url: http://192.168.56.112:3002</span><br></pre></td></tr></table></figure></li><li><p>启动</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">/home/elasticsearch/enterprise-search-8.6.2/bin/enterprise-search</span><br><span class="line"><span class="comment"># 第一次启动会生成用户名和密码，请记录</span></span><br><span class="line">*** Default user credentials have been setup. These are only printed once, so please ensure they are recorded. ***</span><br><span class="line"> username: enterprise_search</span><br><span class="line"> password: c99czde429sxvx5m</span><br></pre></td></tr></table></figure></li><li><p>访问<code>http://192.168.56.112:5601</code></p><p>在菜单<code>Enterprise Search</code> - <code>App search</code> 就可以配置对应的索引库为检索页面开发做准备</p><p><img src="/images/es/es_app_search.png"></p></li><li><p>创建<code>bookdb</code>，然后配置search UI，配置好点击<code>Generate search experience</code>可以预览效果和下载前端UI包</p><p><img src="/images/es/es_search_ui.png"></p></li><li><p>官方默认前端UI是用react开发，同时官方也提供了VUE前端UI的例子，比较熟悉VUE所以下载VUE的例子</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 1.下载该代码</span></span><br><span class="line">https://github.com/elastic/search-ui/tree/main/examples/vue</span><br></pre></td></tr></table></figure></li><li><p>VUE Search UI 代码的安装和配置</p><p>代码是运行需要Nodejs环境，所以需要安排Nodejs环境，然后按文档进行构建和配置即可</p><p>关键点：searchConfig.js文件中的以下代码</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// searchKey可以在生成的UI包中获取</span></span><br><span class="line"><span class="keyword">const</span> connector = <span class="keyword">new</span> AppSearchAPIConnector({</span><br><span class="line"> <span class="attr">searchKey</span>: <span class="string">"search-sdnj9fcxy7yp7xpmv5qqc43q"</span>,</span><br><span class="line"> <span class="attr">engineName</span>: <span class="string">"bookdb"</span>,</span><br><span class="line"> <span class="attr">endpointBase</span>: <span class="string">"http://192.168.56.112:3002"</span></span><br><span class="line">});</span><br></pre></td></tr></table></figure></li><li><p>其他代码根据自己的业务进行对应修改</p></li></ol><h4 id="效果图"><a href="#效果图" class="headerlink" title="效果图"></a>效果图</h4><p><img src="/images/es/es_search.gif"></p><h4 id="结语"><a href="#结语" class="headerlink" title="结语"></a>结语</h4><p>流水式的记录的开发一个搜索功能的过程，这只是个demo，还有很多需要完善的地方，比如服务启动，怎么也得是后台启动；索引现在也没有做分词，也没考虑同义词等等。架子已经搭好了，后续慢慢完善。过程中用到kibana，我并没有单写出来，这个是需要注意的。官网推荐用kibana做为开发工具，我觉得有点重就没用，我用的是一个<a href="https://elasticvue.com/">浏览器插件</a>。</p><h4 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h4><ul><li><a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html">官网</a></li></ul>]]></content>
<summary type="html"><h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote>
<p>学习了一遍Elasticsearch，做个demo巩固一下。</p>
</blockquote>
<p><img src="/images/es/search.jpg"></p></summary>
<category term="database" scheme="http://tungsing.cc/categories/database/"/>
<category term="ES" scheme="http://tungsing.cc/tags/ES/"/>
<category term="Elasticsearch" scheme="http://tungsing.cc/tags/Elasticsearch/"/>
</entry>
<entry>
<title>Python 基础学习</title>
<link href="http://tungsing.cc/2023/02/01/python/python-leaning/"/>
<id>http://tungsing.cc/2023/02/01/python/python-leaning/</id>
<published>2023-02-01T02:34:18.177Z</published>
<updated>2023-02-01T03:18:23.547Z</updated>
<content type="html"><![CDATA[<h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote><p>学习python原因有二，一是在开发人工智能平台的时候涉及到很多算法，都是我们的python算法工程师处理的，但我也想了解下这些算法；二是随着ChatGPT的大火，未来AI一定会在程序开发中占比很高，AI中涉及到的算法、框架很大部分都是用python写的，为未来做准备。</p></blockquote><p><img src="/images/python/python-learning.jpg"></p><span id="more"></span><h4 id="学习记录"><a href="#学习记录" class="headerlink" title="学习记录"></a>学习记录</h4><p>代码记录在<a href="https://github.com/tungSing/learning_python">github</a>上面,记录一下代码的情况，为以后查漏补缺。</p><table><thead><tr><th align="left">文件</th><th align="left">内容</th><th align="left">备注</th></tr></thead><tbody><tr><td align="left">Hello.py</td><td align="left">经典Hello World</td><td align="left"></td></tr><tr><td align="left">Commnets.py</td><td align="left">注释</td><td align="left"></td></tr><tr><td align="left">DataType.py</td><td align="left">数据类型</td><td align="left"></td></tr><tr><td align="left">Lambda.py</td><td align="left">Lambda 表达式</td><td align="left"></td></tr><tr><td align="left">Modules.py</td><td align="left">模块</td><td align="left"></td></tr><tr><td align="left">Pkg.py</td><td align="left">包</td><td align="left"></td></tr><tr><td align="left">FlowControl.py</td><td align="left">流程控制</td><td align="left"></td></tr><tr><td align="left">Functions.py</td><td align="left">函数</td><td align="left"></td></tr><tr><td align="left">Clazz.py</td><td align="left">类</td><td align="left"></td></tr><tr><td align="left">Exception.py</td><td align="left">异常</td><td align="left"></td></tr></tbody></table><h4 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h4><ol><li><a href="https://docs.python.org/zh-cn/3.10/tutorial/">官网</a></li></ol>]]></content>
<summary type="html"><h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote>
<p>学习python原因有二，一是在开发人工智能平台的时候涉及到很多算法，都是我们的python算法工程师处理的，但我也想了解下这些算法；二是随着ChatGPT的大火，未来AI一定会在程序开发中占比很高，AI中涉及到的算法、框架很大部分都是用python写的，为未来做准备。</p>
</blockquote>
<p><img src="/images/python/python-learning.jpg"></p></summary>
<category term="python" scheme="http://tungsing.cc/categories/python/"/>
<category term="python" scheme="http://tungsing.cc/tags/python/"/>
</entry>
<entry>
<title>回顾2022</title>
<link href="http://tungsing.cc/2023/01/31/my/2022/"/>
<id>http://tungsing.cc/2023/01/31/my/2022/</id>
<published>2023-01-31T06:11:25.266Z</published>
<updated>2023-02-10T06:17:11.345Z</updated>
<content type="html"><![CDATA[<p><img src="/images/my/2022.jpg"></p><p>2022年是混乱的一年，提心吊胆的一年、收获甚微的一年。</p><span id="more"></span><p><strong>生活</strong>：</p><p>婚礼还是没办成，还是疫情。不过2023年肯定没问题，疫情结束了。</p><p>抗疫情况，这是2022年生活的一大部分：疫情居家隔离5次，核酸上百次，囤物资2次，抢药1次（安全度过2022没感染，希望2023也不会用到这些药）、还阅读学习了各种防疫法律法规《新型冠状病毒肺炎诊疗方案（试行第九版）》《中华人民共和国传染病防治法》等等吧、意外收获是2022年年初买了隔离保险，结果还赔付了。可以说天天在抗疫。</p><p>通过PMP的考试。</p><p>今年书读的有点少，纸质版《原则》只读了一半，电子书把《设计模式之美》学完了。</p><p>还增驾了D证</p><p><strong>工作</strong></p><p>工作上这一年主要是和同事们合作开发了三个产品有《人工智能平台》《基础业务管理平台》《基础互联网平台》。人工智能平台主要给用户提供可以零代码搭建、训练、优化模型。剩下的两个平台是为了快速交付项目所抽象出来的公共组件、功能的集合和框架。其他工作就是支撑具体项目，服务好客户和开发人员。这一年总算有惊无险的坚持过来了，公司没黄。2023年加油。</p><p><strong>2023</strong></p><ul><li>办婚礼</li><li>多读书</li><li>工作多努力，多赚钱</li><li>讨薪</li></ul>]]></content>
<summary type="html"><p><img src="/images/my/2022.jpg"></p>
<p>2022年是混乱的一年，提心吊胆的一年、收获甚微的一年。</p></summary>
<category term="My" scheme="http://tungsing.cc/tags/My/"/>
</entry>
<entry>
<title>设计模式</title>
<link href="http://tungsing.cc/2022/12/12/book/design_pattern/"/>
<id>http://tungsing.cc/2022/12/12/book/design_pattern/</id>
<published>2022-12-12T01:58:51.650Z</published>
<updated>2023-03-22T08:44:56.383Z</updated>
<content type="html"><![CDATA[<h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><p>我学习设计模式就两个目的，一是可以看懂开源代码，咱们熟悉的框架或者类库都会用到设计模型，为了能在阅读优秀代码的时候，不会一头雾水。二是为了不写烂代码，做为一个工程师，要有工匠精神，把自己的代码尽量写的优雅和质量高一些，尽量提高代码的可读性、可扩展性、复用性、可维护性。</p><p><img src="/images/book/design_pattern.jpg"></p><span id="more"></span><h4 id="SOLID原则"><a href="#SOLID原则" class="headerlink" title="SOLID原则"></a>SOLID原则</h4><h5 id="单一职责原则（Single-Responsibility-Principle）"><a href="#单一职责原则（Single-Responsibility-Principle）" class="headerlink" title="单一职责原则（Single Responsibility Principle）"></a>单一职责原则（Single Responsibility Principle）</h5><p>定义：</p><p>单一职责原则的英文是 Single Responsibility Principle，缩写为 SRP。</p><p>这个原则的英文描述是这样的：A class or module should have a single responsibility。</p><p>如果我们把它翻译成中文，那就是：一个类或者模块只负责完成一个职责（或者功能）。</p><p>理解：</p><p>一个类只负责完成一个职责或者功能。不要设计大而全的类，要设计粒度小、功能单一的类。单一职责原则是为了实现代码高内聚、低耦合，提高代码的复用性、可读性、可维护性。</p><p>判定一个类的职责是否够单一职责，小技巧：</p><ul><li>类中的代码行数、函数或属性过多，会影响代码的可读性和可维护性，我们就需要考虑对类进行拆分；</li><li>类依赖的其他类过多，或者依赖类的其他类过多，不符合高内聚、低耦合的设计思想，我们就需要考虑对类进行拆分；</li><li>私有方法过多，我们就要考虑能否将私有方法独立到新的类中，设置为 public 方法，供更多的类使用，从而提高代码的复用性；</li><li>比较难给类起一个合适名字，很难用一个业务名词概括，或者只能用一些笼统的 Manager、Context 之类的词语来命名，这就说明类的职责定义得可能不够清晰；</li><li>类中大量的方法都是集中操作类中的某几个属性，比如，在 UserInfo 类中，如果一半的方法都是在操作 address 信息，那就可以考虑将这几个属性和对应的方法拆分出来。</li></ul><h5 id="开闭原则（OCP）"><a href="#开闭原则（OCP）" class="headerlink" title="开闭原则（OCP）"></a>开闭原则（OCP）</h5><p>定义：</p><p>开闭原则的英文全称是 Open Closed Principle，简写为 OCP。它的英文描述是：software entities (modules, classes, functions, etc.) should be open for extension , but closed for modification。我们把它翻译成中文就是：软件实体（模块、类、方法等）应该“对扩展开放、对修改关闭”。</p><p>理解：</p><p>添加一个新的功能，应该是通过在已有代码基础上扩展代码（新增模块、类、方法、属性等），而非修改已有代码（修改模块、类、方法、属性等）的方式来完成。关于定义，我们有两点要注意。第一点是，开闭原则并不是说完全杜绝修改，而是以最小的修改代码的代价来完成新功能的开发。第二点是，同样的代码改动，在粗代码粒度下，可能被认定为“修改”；在细代码粒度下，可能又被认定为“扩展”。</p><p>如何做:</p><p>我们要时刻具备扩展意识、抽象意识、封装意识。在写代码的时候，我们要多花点时间思考一下，这段代码未来可能有哪些需求变更，如何设计代码结构，事先留好扩展点，以便在未来需求变更的时候，在不改动代码整体结构、做到最小代码改动的情况下，将新的代码灵活地插入到扩展点上。</p><p>注意点：</p><p>尽管开闭原则描述为对扩展开放、对修改关闭，但也并不是说杜绝一切代码修改，正确的理解是以最小化修改代价来完成新功能的添加。实际上，在平时的开发中，我们要时刻思考，目前的设计在以后应对新功能扩展的时候，是否能做到不需要大的代码修改（比如调整代码结构）就能完成。</p><h5 id="依赖倒置原则（DIP）"><a href="#依赖倒置原则（DIP）" class="headerlink" title="依赖倒置原则（DIP）"></a>依赖倒置原则（DIP）</h5><p>定义：依赖反转原则。依赖反转原则的英文翻译是 Dependency Inversion Principle，缩写为 DIP。中文翻译有时候也叫依赖倒置原则。</p><p>这条原则最原汁原味的英文描述：</p><blockquote><p>High-level modules shouldn’t depend on low-level modules. Both modules should depend on abstractions. In addition, abstractions shouldn’t depend on details. Details depend on abstractions.</p></blockquote><p>我们将它翻译成中文，大概意思就是：高层模块（high-level modules）不要依赖低层模块（low-level）。高层模块和低层模块应该通过抽象（abstractions）来互相依赖。除此之外，抽象（abstractions）不要依赖具体实现细节（details），具体实现细节（details）依赖抽象（abstractions）。</p><p>例子：</p><p>Tomcat 是运行 Java Web 应用程序的容器。我们编写的 Web 应用程序代码只需要部署在 Tomcat 容器下，便可以被 Tomcat 容器调用执行。按照之前的划分原则，Tomcat 就是高层模块，我们编写的 Web 应用程序代码就是低层模块。Tomcat 和应用程序代码之间并没有直接的依赖关系，两者都依赖同一个“抽象”，也就是 Servlet 规范。Servlet 规范不依赖具体的 Tomcat 容器和应用程序的实现细节，而 Tomcat 容器和应用程序依赖 Servlet 规范。</p><p>理解：</p><p>依赖反转原则也叫作依赖倒置原则。这条原则跟控制反转有点类似，主要用来指导框架层面的设计。高层模块不依赖低层模块，它们共同依赖同一个抽象。抽象不要依赖具体实现细节，具体实现细节依赖抽象。</p><p>其他:</p><p><strong>控制反转：</strong>控制反转是一个比较笼统的设计思想，并不是一种具体的实现方法，一般用来指导框架层面的设计。</p><p><strong>依赖注入：</strong>依赖注入和控制反转恰恰相反，它是一种具体的编码技巧。我们不通过 new 的方式在类内部创建依赖类的对象，而是将依赖的类对象在外部创建好之后，通过构造函数、函数参数等方式传递（或注入）给类来使用。</p><h5 id="里斯替换原则（LSP）"><a href="#里斯替换原则（LSP）" class="headerlink" title="里斯替换原则（LSP）"></a>里斯替换原则（LSP）</h5><p>定义：</p><p>里式替换原则的英文翻译是：Liskov Substitution Principle，缩写为 LSP。这个原则最早是在 1986 年由 Barbara Liskov 提出，他是这么描述这条原则的：</p><blockquote><p>If S is a subtype of T, then objects of type T may be replaced with objects of type S, without breaking the program。</p></blockquote><p>在 1996 年，Robert Martin 在他的 SOLID 原则中，重新描述了这个原则，英文原话是这样的：</p><blockquote><p>Functions that use pointers of references to base classes must be able to use objects of derived classes without knowing it。</p></blockquote><p>我们综合两者的描述，将这条原则用中文描述出来，是这样的：子类对象（object of subtype/derived class）能够替换程序（program）中父类对象（object of base/parent class）出现的任何地方，并且保证原来程序的逻辑行为（behavior）不变及正确性不被破坏。</p><p>理解：</p><p>里式替换原则是用来指导，继承关系中子类该如何设计的一个原则。理解里式替换原则，最核心的就是理解“design by contract，按照协议来设计”这几个字。父类定义了函数的“约定”（或者叫协议），那子类可以改变函数的内部实现逻辑，但不能改变函数原有的“约定”。这里的约定包括：函数声明要实现的功能；对输入、输出、异常的约定；甚至包括注释中所罗列的任何特殊说明。</p><h5 id="接口隔离原则（ISP）"><a href="#接口隔离原则（ISP）" class="headerlink" title="接口隔离原则（ISP）"></a>接口隔离原则（ISP）</h5><p>定义：</p><p>接口隔离原则的英文翻译是“ Interface Segregation Principle”，缩写为 ISP。Robert Martin 在 SOLID 原则中是这样定义它的：“Clients should not be forced to depend upon interfaces that they do not use。”直译成中文的话就是：客户端不应该被强迫依赖它不需要的接口。其中的“客户端”，可以理解为接口的调用者或者使用者。</p><p>理解：</p><p>理解“接口隔离原则”的重点是理解其中的“接口”二字。这里有三种不同的理解。</p><p>如果把“接口”理解为一组接口集合，可以是某个微服务的接口，也可以是某个类库的接口等。如果部分接口只被部分调用者使用，我们就需要将这部分接口隔离出来，单独给这部分调用者使用，而不强迫其他调用者也依赖这部分不会被用到的接口。</p><p>如果把“接口”理解为单个 API 接口或函数，部分调用者只需要函数中的部分功能，那我们就需要把函数拆分成粒度更细的多个函数，让调用者只依赖它需要的那个细粒度函数。</p><p>如果把“接口”理解为 OOP 中的接口，也可以理解为面向对象编程语言中的接口语法。那接口的设计要尽量单一，不要让接口的实现类和调用者，依赖不需要的接口函数。</p><h4 id="DRY原则"><a href="#DRY原则" class="headerlink" title="DRY原则"></a>DRY原则</h4><p>定义：它的英文描述为：Don’t Repeat Yourself。中文直译为：不要重复自己。将它应用在编程中，可以理解为：不要写重复的代码。</p><p>理解：</p><p>从以下三种三种典型情况：实现逻辑重复、功能语义重复、代码执行重复来理解</p><p>实现逻辑重复，但功能语义不重复的代码，并不违反 DRY 原则。</p><p>实现逻辑不重复，但功能语义重复的代码，也算是违反 DRY 原则。</p><p>除此之外，代码执行重复也算是违反 DRY 原则。</p><h4 id="KISS原则"><a href="#KISS原则" class="headerlink" title="KISS原则"></a>KISS原则</h4><p>定义：</p><p>KISS 原则的英文描述有好几个版本，比如下面这几个。</p><ul><li>Keep It Simple and Stupid.</li><li>Keep It Short and Simple.</li><li>Keep It Simple and Straightforward.</li></ul><p>不过，它们要表达的意思其实差不多，翻译成中文就是：尽量保持简单。</p><p>理解：</p><p>KISS 原则中的“简单”并不是以代码行数来考量的。代码行数越少并不代表代码越简单，</p><p>我们还要考虑逻辑复杂度、实现难度、代码的可读性等。而且，本身就复杂的问题，用复杂的方法解决，并不违背 KISS 原则。</p><p>除此之外，同样的代码，在某个业务场景下满足 KISS 原则，换一个应用场景可能就不满足了。</p><p>怎么做：</p><ul><li>不要使用同事可能不懂的技术来实现代码；</li><li>不要重复造轮子，要善于使用已经有的工具类库；</li><li>不要过度优化。</li></ul><h4 id="YAGNI原则"><a href="#YAGNI原则" class="headerlink" title="YAGNI原则"></a>YAGNI原则</h4><p>定义：YAGNI 原则的英文全称是：You Ain’t Gonna Need It。直译就是：你不会需要它。</p><p>理解：</p><p>当用在软件开发中的时候，它的意思是：不要去设计当前用不到的功能；不要去编写当前用不到的代码。实际上，这条原则的核心思想就是：不要做过度设计。</p><p>当然，这并不是说我们就不需要考虑代码的扩展性。我们还是要预留好扩展点，等到需要的时候再去实现对应的代码。</p><h4 id="LOD原则"><a href="#LOD原则" class="headerlink" title="LOD原则"></a>LOD原则</h4><p>定义：</p><p>Law of Demeter，缩写是 LOD。单从这个名字上来看，我们完全猜不出这个原则讲的是什么。不过，它还有另外一个更加达意的名字，叫作最小知识原则，英文翻译为：The Least Knowledge Principle。</p><p>原汁原味的英文定义：</p><blockquote><p>Each unit should have only limited knowledge about other units: only units “closely” related to the current unit. Or: Each unit should only talk to its friends; Don’t talk to strangers.</p></blockquote><p>中文直译：</p><blockquote><p>每个模块（unit）只应该了解那些与它关系密切的模块（units: only units “closely” related to the current unit）的有限知识（knowledge）。或者说，每个模块只和自己的朋友“说话”（talk），不和陌生人“说话”（talk）。</p></blockquote><p>定义理解：</p><blockquote><p>不该有直接依赖关系的类之间，不要有依赖；有依赖关系的类之间，尽量只依赖必要的接口。迪米特法则是希望减少类之间的耦合，让类越独立越好。每个类都应该少了解系统的其他部分。一旦发生变化，需要了解这一变化的类就会比较少。</p></blockquote><p>其他：</p><p>“高内聚、松耦合”是一个非常重要的设计思想，能够有效提高代码的可读性和可维护性，缩小功能改动导致的代码改动范围。</p><p>“高内聚”用来指导类本身的设计，“松耦合”用来指导类与类之间依赖关系的设计。</p><p>所谓高内聚，就是指相近的功能应该放到同一个类中，不相近的功能不要放到同一类中。相近的功能往往会被同时修改，放到同一个类中，修改会比较集中。</p><p>所谓松耦合指的是，在代码中，类与类之间的依赖关系简单清晰。即使两个类有依赖关系，一个类的代码改动也不会或者很少导致依赖类的代码改动。</p><h4 id="设计模式"><a href="#设计模式" class="headerlink" title="设计模式"></a>设计模式</h4><p>经典的设计模式有 23 种，分三种类型：创建型、结构型和行为型。其中，创建型设计模式主要解决“对象的创建”问题，结构型设计模式主要解决“类或对象的组合”问题，行为型设计模式主要解决“类或对象之间的交互”问题。</p><p>常用的主要有：单例、工厂、建造者、代理、装饰器、适配器、观察者、模板、策略、职责链、迭代器这 11 种，所以只要集中精力，把这 11 种搞明白就可以了，剩下的那 12 种稍微了解，混个眼熟，等到真正用到的时候，再深入地去研究学习就可以了。</p><p><a href="https://github.com/tungSing/java_learning/tree/master/learning_base/src/main/java/cc/tungsing/learning/design/pattern">代码记录</a></p><h4 id="结语"><a href="#结语" class="headerlink" title="结语"></a>结语</h4><p>设计原则和设计模式需要在工作中反复使用和体会才能理解的比较透彻。</p>]]></content>
<summary type="html"><h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><p>我学习设计模式就两个目的，一是可以看懂开源代码，咱们熟悉的框架或者类库都会用到设计模型，为了能在阅读优秀代码的时候，不会一头雾水。二是为了不写烂代码，做为一个工程师，要有工匠精神，把自己的代码尽量写的优雅和质量高一些，尽量提高代码的可读性、可扩展性、复用性、可维护性。</p>
<p><img src="/images/book/design_pattern.jpg"></p></summary>
<category term="book" scheme="http://tungsing.cc/categories/book/"/>
<category term="book" scheme="http://tungsing.cc/tags/book/"/>
<category term="读书笔记" scheme="http://tungsing.cc/tags/%E8%AF%BB%E4%B9%A6%E7%AC%94%E8%AE%B0/"/>
</entry>
<entry>
<title>Docker启用TLS</title>
<link href="http://tungsing.cc/2022/09/26/docker/docker_tls/"/>
<id>http://tungsing.cc/2022/09/26/docker/docker_tls/</id>
<published>2022-09-26T05:22:50.563Z</published>
<updated>2022-09-26T06:34:04.243Z</updated>
<content type="html"><![CDATA[<h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote><p>最近在开发人工智能平台，需要远程操控docker ，正好docker提供了 Remote API。普通的没有任何安全防护，同时也提供TLS的安全认证方式，为了安全我们采用TLS方式，同时把操作过程记录下来</p></blockquote><p><img src="/images/docker/docker.jpg"></p><span id="more"></span><h4 id="环境"><a href="#环境" class="headerlink" title="环境"></a>环境</h4><p>CentOS7、docker ce 20.10.9</p><h4 id="安装docker"><a href="#安装docker" class="headerlink" title="安装docker"></a>安装docker</h4><p>通过yum 命令安装</p><h4 id="开启-Remote-API"><a href="#开启-Remote-API" class="headerlink" title="开启 Remote API"></a>开启 Remote API</h4><p>为了安全考虑docker默认是不开启远程访问，所以需要配置</p><ol><li><p>在<code>/etc/docker/daemon.json</code> 文件中添加以下配置</p><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">{</span><br><span class="line"> <span class="attr">"hosts"</span>: [<span class="string">"tcp://0.0.0.0:2375"</span>, <span class="string">"unix:///var/run/docker.sock"</span>]</span><br><span class="line">}</span><br></pre></td></tr></table></figure></li><li><p>修改docker.service文件，把<code>ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock</code>修改为<code>ExecStart=/usr/bin/dockerd</code></p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock</span></span><br><span class="line">ExecStart=/usr/bin/dockerd</span><br></pre></td></tr></table></figure></li><li><p>重启docker服务</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">systemctl deamon-reload</span><br><span class="line">systemctl restart docker</span><br></pre></td></tr></table></figure></li><li><p>测试<br>在浏览器输入docker服务器的IP加端口就可以访问</p></li></ol><h4 id="开启安全的远程访问"><a href="#开启安全的远程访问" class="headerlink" title="开启安全的远程访问"></a>开启安全的远程访问</h4><ol><li><p>在docker服务器上，生成 CA 私钥和公钥</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line">$ openssl genrsa -aes256 -out ca-key.pem 4096</span><br><span class="line">Generating RSA private key, 4096 bit long modulus</span><br><span class="line">............................................................................................................................................................................................++</span><br><span class="line">........++</span><br><span class="line">e is 65537 (0x10001)</span><br><span class="line">Enter pass phrase <span class="keyword">for</span> ca-key.pem:</span><br><span class="line">Verifying - Enter pass phrase <span class="keyword">for</span> ca-key.pem:</span><br><span class="line"></span><br><span class="line">$ openssl req -new -x509 -days 365 -key ca-key.pem -sha256 -out ca.pem</span><br><span class="line">Enter pass phrase <span class="keyword">for</span> ca-key.pem:</span><br><span class="line">You are about to be asked to enter information that will be incorporated</span><br><span class="line">into your certificate request.</span><br><span class="line">What you are about to enter is what is called a Distinguished Name or a DN.</span><br><span class="line">There are quite a few fields but you can leave some blank</span><br><span class="line">For some fields there will be a default value,</span><br><span class="line">If you enter <span class="string">'.'</span>, the field will be left blank.</span><br><span class="line">-----</span><br><span class="line">Country Name (2 letter code) [AU]:</span><br><span class="line">State or Province Name (full name) [Some-State]:Queensland</span><br><span class="line">Locality Name (eg, city) []:Brisbane</span><br><span class="line">Organization Name (eg, company) [Internet Widgits Pty Ltd]:Docker Inc</span><br><span class="line">Organizational Unit Name (eg, section) []:Sales</span><br><span class="line">Common Name (e.g. server FQDN or YOUR name) []:<span class="variable">$HOST</span></span><br><span class="line">Email Address []:[email protected]</span><br></pre></td></tr></table></figure><p>PS：$HOST就是当前docker服务器的IP</p></li><li><p>有了 CA，可以创建服务器密钥和证书签名请求 (CSR)</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">$ openssl genrsa -out server-key.pem 4096</span><br><span class="line">Generating RSA private key, 4096 bit long modulus</span><br><span class="line">.....................................................................++</span><br><span class="line">.................................................................................................++</span><br><span class="line">e is 65537 (0x10001)</span><br><span class="line"></span><br><span class="line">$ openssl req -subj <span class="string">"/CN=<span class="variable">$HOST</span>"</span> -sha256 -new -key server-key.pem -out server.csr</span><br></pre></td></tr></table></figure></li><li><p>用 CA 签署公钥</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">echo</span> subjectAltName = DNS:<span class="variable">$HOST</span>,IP:10.10.10.20,IP:127.0.0.1 >> extfile.cnf</span><br><span class="line"><span class="built_in">echo</span> extendedKeyUsage = serverAuth >> extfile.cnf</span><br></pre></td></tr></table></figure></li><li><p>生成服务端签名证书</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">$ openssl x509 -req -days 365 -sha256 -<span class="keyword">in</span> server.csr -CA ca.pem -CAkey ca-key.pem \</span><br><span class="line"> -CAcreateserial -out server-cert.pem -extfile extfile.cnf</span><br><span class="line">Signature ok</span><br><span class="line">subject=/CN=your.host.com</span><br><span class="line">Getting CA Private Key</span><br><span class="line">Enter pass phrase <span class="keyword">for</span> ca-key.pem:</span><br><span class="line"></span><br></pre></td></tr></table></figure></li><li><p>创建客户端密钥和证书签名请求</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">$ openssl genrsa -out key.pem 4096</span><br><span class="line">Generating RSA private key, 4096 bit long modulus</span><br><span class="line">.........................................................++</span><br><span class="line">................++</span><br><span class="line">e is 65537 (0x10001)</span><br><span class="line"></span><br><span class="line">$ openssl req -subj <span class="string">'/CN=client'</span> -new -key key.pem -out client.csr</span><br><span class="line"></span><br></pre></td></tr></table></figure></li><li><p>为生成客户端签名证书，创建一个新的扩展配置文件</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">echo</span> extendedKeyUsage = clientAuth > extfile-client.cnf</span><br></pre></td></tr></table></figure></li><li><p>生成客户端签名证书</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">$ openssl x509 -req -days 365 -sha256 -<span class="keyword">in</span> client.csr -CA ca.pem -CAkey ca-key.pem \</span><br><span class="line"> -CAcreateserial -out cert.pem -extfile extfile-client.cnf</span><br><span class="line">Signature ok</span><br><span class="line">subject=/CN=client</span><br><span class="line">Getting CA Private Key</span><br><span class="line">Enter pass phrase <span class="keyword">for</span> ca-key.pem:</span><br></pre></td></tr></table></figure></li><li><p>删除证书签名和扩展配置文件</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">rm -v client.csr server.csr extfile.cnf extfile-client.cnf</span><br></pre></td></tr></table></figure></li><li><p>在在`/etc/docker/daemon.json 文件中配置TLS</p><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">{ </span><br><span class="line"> <span class="attr">"hosts"</span>: [<span class="string">"tcp://0.0.0.0:2375"</span>, <span class="string">"unix:///var/run/docker.sock"</span>],</span><br><span class="line"> <span class="attr">"tls"</span>: <span class="literal">true</span>,</span><br><span class="line"> <span class="attr">"tlsverify"</span>: <span class="literal">true</span>,</span><br><span class="line"> <span class="attr">"tlscacert"</span>: <span class="string">"/etc/docker/ca.pem"</span>,</span><br><span class="line"> <span class="attr">"tlscert"</span>: <span class="string">"/etc/docker/server-cert.pem"</span>,</span><br><span class="line"> <span class="attr">"tlskey"</span>: <span class="string">"/etc/docker/server-key.pem"</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure></li><li><p>重启启动docker服务</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">systemctl restart docker</span><br></pre></td></tr></table></figure></li><li><p>用<code>curl</code> 命令来验证</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">curl https://192.168.56.105:2375/info --cert /etc/docker/cert.pem --key /etc/docker/key.pem --cacert /etc/docker/ca.pem</span><br></pre></td></tr></table></figure></li></ol><h4 id="docker-java-客户端来调用docker"><a href="#docker-java-客户端来调用docker" class="headerlink" title="docker java 客户端来调用docker"></a>docker java 客户端来调用docker</h4><ol><li><p>复制<code>ca.pem</code>,<code>cert.pem</code>,<code>key.pem</code>三个文件到客户端</p><p>我们把这三个文件复制到开发机器的<code>D:\\docker\\</code>目录下，需要注意，不要修改这三个文件的名称</p></li><li><p>创建maven工程并引人docker-java的坐标</p></li><li><p>编写测试代码</p><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">DockerClientDemo</span> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">main</span><span class="params">(String[] args)</span> </span>{</span><br><span class="line"> DockerClient dockerClient = getDC();</span><br><span class="line"> List<Image> images = dockerClient.listImagesCmd().exec();</span><br><span class="line"> images.forEach(item -> System.out.println(<span class="string">"id : "</span> + item.getId()));</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> DockerClient <span class="title">getDC</span><span class="params">()</span> </span>{</span><br><span class="line"> DockerClientConfig dockerClientConfig = DefaultDockerClientConfig.createDefaultConfigBuilder()</span><br><span class="line"> .withApiVersion(<span class="string">"1.41"</span>)</span><br><span class="line"> .withDockerTlsVerify(<span class="keyword">true</span>)</span><br><span class="line"> .withDockerCertPath(<span class="string">"D:\\docker\\"</span>)</span><br><span class="line"> .withDockerHost(<span class="string">"tcp://192.168.56.105:2375"</span>).build();</span><br><span class="line"></span><br><span class="line"> DockerClient dockerClient = DockerClientBuilder.getInstance(dockerClientConfig).build();</span><br><span class="line"> <span class="keyword">return</span> dockerClient;</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure></li></ol><h4 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h4><ol><li><a href="https://docs.docker.com/engine/security/protect-access/">官网：Protect the Docker daemon socket</a></li><li><a href="https://www.cnblogs.com/xiaoqi/p/docker-tls.html">Docker启用TLS进行安全配置</a></li><li><a href="https://help.aliyun.com/document_detail/160093.html">HTTPS双向认证</a></li></ol>]]></content>
<summary type="html"><h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote>
<p>最近在开发人工智能平台，需要远程操控docker ，正好docker提供了 Remote API。普通的没有任何安全防护，同时也提供TLS的安全认证方式，为了安全我们采用TLS方式，同时把操作过程记录下来</p>
</blockquote>
<p><img src="/images/docker/docker.jpg"></p></summary>
<category term="java" scheme="http://tungsing.cc/tags/java/"/>
<category term="docker" scheme="http://tungsing.cc/tags/docker/"/>
</entry>
<entry>
<title>Memcached的安装及认证配置</title>
<link href="http://tungsing.cc/2022/04/12/cache/memcached/memcached/"/>
<id>http://tungsing.cc/2022/04/12/cache/memcached/memcached/</id>
<published>2022-04-12T01:00:16.338Z</published>
<updated>2022-04-12T03:55:43.615Z</updated>
<content type="html"><![CDATA[<h4 id="Memcached-介绍"><a href="#Memcached-介绍" class="headerlink" title="Memcached 介绍"></a>Memcached 介绍</h4><p>Memcached 是一种内存键值存储，用于存储来自数据库调用、API 调用或页面渲染结果的任意数据（字符串、对象）的小块。Memcached 简单但功能强大。其简单的设计促进了快速部署、易于开发，并解决了大数据缓存面临的许多问题。它的 API 适用于大多数流行的语言。</p><span id="more"></span><h4 id="Memcached的安装"><a href="#Memcached的安装" class="headerlink" title="Memcached的安装"></a>Memcached的安装</h4><h5 id="Linux"><a href="#Linux" class="headerlink" title="Linux"></a>Linux</h5><p>操作系统为：CentOS7</p><ol><li><p>使用yum命令安装</p> <figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># yum update</span></span><br><span class="line"><span class="comment"># yum install memcached</span></span><br></pre></td></tr></table></figure><p> <img src="Install-Memcached-in-CentOS-7.png"></p></li></ol><ol start="2"><li><p>安装客户端（可选）</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># yum install libmemcached</span></span><br></pre></td></tr></table></figure><p><img src="Install-Memcached-Library-in-CentOs-7.png"></p></li></ol><ol start="3"><li><p>配置</p><p>在<code>/etc/sysconfig/memcached</code> 配置文件中配置Memcached服务的选项</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># vi /etc/sysconfig/memcached</span></span><br></pre></td></tr></table></figure><p>配置内容</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">PORT="11211"</span><br><span class="line">USER="memcached"</span><br><span class="line">MAXCONN="1024"</span><br><span class="line">CACHESIZE="64"</span><br></pre></td></tr></table></figure><p>参数介绍：</p><ul><li><strong>PORT</strong> : Memcached的运行端口</li><li><strong>USER</strong> : 启动 Memcached 服务的用户</li><li><strong>MAXCONN</strong> : 最大连接数</li><li><strong>CACHESIZE</strong> : 缓存占内存的大小</li><li><strong>OPTIONS</strong> : 可以设置服务器的IP等其他信息</li></ul></li><li><p>启动</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># systemctl restart memcached</span></span><br><span class="line"><span class="comment"># systemctl enable memcached</span></span><br></pre></td></tr></table></figure></li><li><p>用客户端连接测试</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># memcached-tool 127.0.0.1 stats</span></span><br></pre></td></tr></table></figure><p><img src="Check-Memcached-Running-Status.png"></p></li></ol><h5 id="Windows"><a href="#Windows" class="headerlink" title="Windows"></a>Windows</h5><h4 id="Memcached-在Linux的认证配置"><a href="#Memcached-在Linux的认证配置" class="headerlink" title="Memcached 在Linux的认证配置"></a>Memcached 在Linux的认证配置</h4><h5 id="为什么需要认证"><a href="#为什么需要认证" class="headerlink" title="为什么需要认证"></a>为什么需要认证</h5><p>客户环境会做漏洞扫描，如果没有认证的话属于高危漏洞</p><h5 id="memcached认证机制"><a href="#memcached认证机制" class="headerlink" title="memcached认证机制"></a>memcached认证机制</h5><p>memcached目前本身并没有包含认证功能，但支持SASL认证，由于memcached在1.4.3版本才加入对SASL的支持，所以要使用认证的话需要将memcached升级到>=1.4.3的版本。</p><h5 id="SASL是什么"><a href="#SASL是什么" class="headerlink" title="SASL是什么"></a>SASL是什么</h5><p>SASL全称Simple Authentication and Security Layer（简单认证与安全层），是一种用来扩充C/S模式验证能力的机制，SASL只是认证过程，将应用层与系统认证机制整合起来，SASL本身不能进行认证。</p><p>SASL支持的认证方法为：getpwent kerberos5 pam rimap shadow ldap httpform。</p><h5 id="配置支持PLAIN的认证方案"><a href="#配置支持PLAIN的认证方案" class="headerlink" title="配置支持PLAIN的认证方案"></a>配置支持PLAIN的认证方案</h5><ol><li><p>安装<code>cyrus-sasl-devel</code> 和 <code>cyrus-sasl-plain</code> 包</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># yum install cyrus-sasl-devel cyrus-sasl-plain</span></span><br></pre></td></tr></table></figure></li><li><p>接下来，我们将创建目录和配置文件</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># mkdir -p /etc/sasl2</span></span><br><span class="line"><span class="comment"># vi /etc/sasl2/memcached.conf </span></span><br></pre></td></tr></table></figure></li><li><p>配置<code>/etc/sasl2/memcached.conf</code></p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">mech_list: plain</span><br><span class="line">log_level: 5</span><br><span class="line">sasldb_path: /etc/sasl2/memcached-sasldb2</span><br></pre></td></tr></table></figure><p>配置说明：</p><ul><li>mech_list：认证模式</li><li>log_level：日志级别</li><li>sasldb_path：sasl数据库路径</li></ul></li><li><p>创建SASL数据库来保存我们的用户认证信息，使用saslpasswd2命令来创建</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># saslpasswd2 -a memcached -c -f /etc/sasl2/memcached-sasldb2 tungsing</span></span><br></pre></td></tr></table></figure><p>说明：<code>tungsing</code>就是用户，请根据实际情况设置</p></li><li><p> 把SASL数据库授权给<code>memcached</code>用户</p></li></ol> <figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># chown memcached:memcached /etc/sasl2/memcached-sasldb2</span></span><br></pre></td></tr></table></figure><ol start="6"><li><p>配置我们已经安装好的memcached</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># vi /etc/sysconfig/memcached</span></span><br></pre></td></tr></table></figure><p>在<code>OPTIONS</code>中添加<code>-S</code>和<code>-vv</code>两个参数，<code>-S</code>代表开启SASL认证，<code>-vv</code>是信息会输出到<code>/var/log/memcached</code>中</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">OPTIONS="-l 127.0.0.1 -U 0 -S -vv" </span><br></pre></td></tr></table></figure></li><li><p>重启Memcached服务</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># systemctl restart memcached</span></span><br></pre></td></tr></table></figure></li><li><p>验证</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># memstat --servers="127.0.0.1" --username=tungsing --password=your_password</span></span><br></pre></td></tr></table></figure><p>成功会输出</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">Server: 127.0.0.1 (11211)</span><br><span class="line"> pid: 3831</span><br><span class="line"> uptime: 9</span><br><span class="line"> time: 1520028517</span><br><span class="line"> version: 1.4.25</span><br><span class="line"> . . .</span><br></pre></td></tr></table></figure></li></ol><h4 id="参考资料"><a href="#参考资料" class="headerlink" title="参考资料"></a>参考资料</h4><ul><li> <a href="https://www.memcached.org/">Memcached</a></li><li> <a href="https://zh.wikipedia.org/wiki/%E7%AE%80%E5%8D%95%E8%AE%A4%E8%AF%81%E4%B8%8E%E5%AE%89%E5%85%A8%E5%B1%82">简单认证与安全层</a></li><li> <a href="https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-memcached-on-centos-7">How To Install and Secure Memcached on CentOS 7</a></li></ul>]]></content>
<summary type="html"><h4 id="Memcached-介绍"><a href="#Memcached-介绍" class="headerlink" title="Memcached 介绍"></a>Memcached 介绍</h4><p>Memcached 是一种内存键值存储，用于存储来自数据库调用、API 调用或页面渲染结果的任意数据（字符串、对象）的小块。Memcached 简单但功能强大。其简单的设计促进了快速部署、易于开发，并解决了大数据缓存面临的许多问题。它的 API 适用于大多数流行的语言。</p></summary>
<category term="cache" scheme="http://tungsing.cc/categories/cache/"/>
<category term="cache" scheme="http://tungsing.cc/tags/cache/"/>
<category term="memcached" scheme="http://tungsing.cc/tags/memcached/"/>
</entry>
<entry>
<title>回顾2021</title>
<link href="http://tungsing.cc/2022/01/26/my/2021/"/>
<id>http://tungsing.cc/2022/01/26/my/2021/</id>
<published>2022-01-26T07:37:01.082Z</published>
<updated>2022-01-26T08:07:53.571Z</updated>
<content type="html"><![CDATA[<p><img src="/images/my/2021.jpg"></p><p>整体来说2021年是一个收获、成长的一年。过程中收获了感悟、经历、快乐、难过等。感谢这一年遇到的人和事！</p><span id="more"></span><p><strong>生活</strong>：</p><p>重新租了房子，搬了家，搬家真是个累活，希望是最后一次搬家。</p><p>筹备了婚礼，按计划现在应该已婚人士中的一员了，没想到遇到疫情没办成，延期到了2022年。</p><p>很早报名的考试通过了，证明只要努力，任何事情都是可以做到的，只是时间的长和短的区别，未来要把更多时间投入到自己喜欢的事情上。</p><p>纸质书读了《博弈论》、《清醒思考的艺术》、《如何读一本书》(才读了三分之一)。</p><p>电子书读了极客时间的《设计模式之美》(读完60%)、《从0开始学架构》、《MySQL 实战 45 讲》、《趣谈网络协议》。</p><p><strong>工作</strong></p><p>工作上是收获满满的一年，见识了非常努力、认真的甲方、见证了老板创业的不易、旁观了其他人赚钱的方式、方法等。</p><p>2021年是努力工作的一年，完成档案管理管理端产品，个性化定制的有5家大客户，每家反馈都比较好。</p><p>2021年是我工作以来出差最多的一年，在汉庭直接住到了铂金会员。</p><p>2021年是我认识人最多的一年，项目上接触的客户、合作上的供应商、工作上的同事等等。</p><p>2021年是有遗憾的一年，年底并没有收到7月份的工资，哎…</p><p><strong>2022</strong></p><ul><li>希望疫情结束</li><li>办婚礼</li><li>多读书</li><li>工作稳定</li><li>过PMP</li></ul>]]></content>
<summary type="html"><p><img src="/images/my/2021.jpg"></p>
<p>整体来说2021年是一个收获、成长的一年。过程中收获了感悟、经历、快乐、难过等。感谢这一年遇到的人和事！</p></summary>
<category term="My" scheme="http://tungsing.cc/tags/My/"/>
</entry>
<entry>
<title>清醒思考的艺术阅读笔记</title>
<link href="http://tungsing.cc/2022/01/06/book/the_art_of_thinking_clearly/chapter_two/"/>
<id>http://tungsing.cc/2022/01/06/book/the_art_of_thinking_clearly/chapter_two/</id>
<published>2022-01-06T10:01:23.161Z</published>
<updated>2023-03-22T08:44:47.481Z</updated>
<content type="html"><![CDATA[<h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote><p>记录一下阅读该书籍的理解与感想。</p></blockquote><h4 id="026：忽视概率偏误"><a href="#026：忽视概率偏误" class="headerlink" title="026：忽视概率偏误"></a>026：忽视概率偏误</h4><blockquote><p>累计奖金为什么会越来越多</p></blockquote><p>有两种赌博：第一种赌博你有可能赢1000万欧元，第二种赌博你可能赢1万欧元。你会参加哪一种呢？如果你在第一种赌博里赢了，你的生活将会彻底改变：你可以辞掉工作，从此靠利息生活。如果你在第二种赌博里赢了，你可以美美地前往加勒比海度假，然后一切照旧。第一种赌博赢的概率是亿分之一，第二种赌博赢的概率是万分之一。好吧，你会玩哪一 种？我们的感情冲动会将我们拉向第一种，虽然客观地看，第二种赢的概率要大无数倍。因 此累计奖金越来越多——数百万、数亿、数万亿美元——无论赢的机会是多么微弱。 </p><p>结论：我们很难区分各种风险，除非风险为零。由于我们不能直觉地理解风险，我们必须计算。在概率公开的地方——像彩票——这就很容易。而在普通生活中，风险很难估计，但又是躲也躲不过的。</p><p>个人理解：我们对生活中的概率不太敏感，以后遇到有事情要多思考概率的大小，以便做出合理的判断。</p><span id="more"></span><h4 id="027：零风险偏误"><a href="#027：零风险偏误" class="headerlink" title="027：零风险偏误"></a>027：零风险偏误</h4><blockquote><p>你为什么会为零风险支付过多</p></blockquote><p>我们常愿意投资过多的钱，就为了彻底消除微小的剩余风险。几乎在所有情况下，人们本来都能更好地投资这笔钱，更显著地降低另一种风险。人们把这种思维错误称为零风险偏误。</p><p>在道路交通中，只有当速度限制降到每小时零公里，才能达到零风险。在这里，我们理性地容忍可以明确统计的每年的死亡人数。</p><p>假设你是国家首脑，你想排除恐怖袭击的风险。那你必须派给每个市民一名间谍，每名间谍再派一名间谍。那么很快，90%的人口就都成了间谍。我们知道，这样的社会是不存在的。</p><p>那么在股市中呢？股市中存在零风险吗？很可惜，不存在。即使你卖掉股票，将钱存进 你的账户，银行也有可能破产，通货膨胀也会蚕食掉你节约下的钱，或者一次货币改革也可 能会毁掉你的财产。</p><p>结论：请你告别零风险的想象，学会怀着“没有什么是安全的”想法生活——无论是你的积蓄、你的健康、你的婚姻、你的友谊、你的敌人，还是你的土地。请你满足于至少有东 西让你保持相对稳定并体验自身的快乐。研究表明，无论是中了百万彩票还是半身瘫痪都不会长期改变你的满意程度。不管发生什么事，快乐的人照样快乐，不快乐的人依旧不快乐。 </p><p>个人理解：不存在零风险的，我们要做的就是分析风险的概率，选择有利于我们的风险，快乐的生活就好。</p><h4 id="028：稀少性谬误"><a href="#028：稀少性谬误" class="headerlink" title="028：稀少性谬误"></a>028：稀少性谬误</h4><blockquote><p>为什么饼干越少越好吃</p></blockquote><p>斯蒂芬·沃切尔将受试者分成两组，请他们评价饼干的质量。第一组得到整整一盒饼 干，第二组只得到两块。结果：第二组受试者对饼干质量的评价要比第一组高得多。这一试 验经过多次重复——结果都相同。</p><p>结论：我们对稀少性的典型反应是丧失清晰思考的能力。因此请你仅按价格和作用判断 一样东西。你不要在乎它是否稀少，是否有哪位“伦敦来的医生”也想要它。</p><p>个人理解：生活或工作中我们要对“库存清货”、”仅今日有售“等警惕，也要可以利用物以稀为贵。</p><h4 id="029：忽视基本比率"><a href="#029：忽视基本比率" class="headerlink" title="029：忽视基本比率"></a>029：忽视基本比率</h4><blockquote><p>当你在怀俄明州听到马蹄声、见到黑白条纹时……</p></blockquote><p><strong>忽略基本比率</strong>（Neglecting Base Rates）是一种几率谬误，系因不明统计学上的基本比率导致的推论谬误。</p><p>例子：一名少年被刀砍成重伤。根据以上描述，下列哪种情况的可能性更大呢？(1)凶手是个非法持有刀具的波斯尼亚人；（2)凶手是个来自中产阶层的德国少年。现在你知 道答案了：答案是(2)的可能性要大得多，因为比起带刀的波斯尼亚人，德国少年要多得多。</p><p>结论：好好学概率论，凡事多想一下基本比率。</p><h4 id="030：赌徒谬误"><a href="#030：赌徒谬误" class="headerlink" title="030：赌徒谬误"></a>030：赌徒谬误</h4><blockquote><p>为什么没有一种平衡命运的力量</p></blockquote><p><strong>赌徒谬误</strong>（The Gambler’s Fallacy）亦称为<strong>蒙地卡罗谬误</strong>（The Monte Carlo Fallacy），是一种几率谬误，主张由于某事发生了很多次，因此接下来不太可能发生；或者由于某事很久没发生，因此接下来很可能会发生。</p><p>一枚硬币被连抛3次，每次都是人头朝上。假如有人强迫你，让你自己掏出1 000欧元为 下一抛下注。你会押人头还是数字呢？如果你像大多数人那样思考，你会押数字，虽然人头同样是有可能的——这就是著名的赌徒谬误。</p><p>将一枚硬币连抛50次，50次都是人头朝上。又有人强迫你为下一抛押1 000欧元。你会押 人头还是数字呢？你淡定地微微一笑，因为你已经读了这一章，知道之前的结果无关紧要。 但如果你足够理智，你肯定会押人头，因为你必然会想到，这枚硬币应该是镀锌的。</p><p>结论：请你仔细观看，看你面对的是否是独立的事件——不过这主要存在于赌场、彩票 和理论书籍里。现实生活中这些事件大多有着相互联系——已经发生的事情，会影响未来将要发生的事情。因此请你忘记命运的平衡力量(除了回归均值的情形）。</p><p>个人理解：遇事先分析事情间联系的强弱，如果关联很弱，看看是否为一个独立事件，如果是独立事件就要避免落入赌徒谬误的陷阱中。</p><h4 id="031：锚定效应"><a href="#031：锚定效应" class="headerlink" title="031：锚定效应"></a>031：锚定效应</h4><blockquote><p>数字轮盘如何搞得我们晕头转向</p></blockquote><p>人类在进行决策时，会过度偏重先前取得的资讯（这称为锚点），即使这个资讯与这项决定无关。在进行决策时，人类倾向于利用此片断资讯（锚点），快速做出决定。在接下来的决定中，再以第一个决定为基准，逐步修正。但是人类容易过度利用锚点，来对其他资讯与决定做出诠释，当锚点与实际上的事实之间的有很大出入，就会出现当局者迷的情况。</p><p>心理学家阿莫斯·特沃斯基拿出一个数字轮盘，让受试者转动轮盘，然后问他们，联合 国有多少成员国。那些轮盘停在一个较大数字上的人，给出的成员国的数量就会大于轮盘停 在较小数字上的人给出的数量。</p><p>现实生活中，锚定效应随处可见。科学证明，如果老师知道一位学生过去的学习成绩， 就会影响他给学生的新论文打分。在这里，过去的成绩起了锚定的作用。许多产品在包装袋 上刊印的“建议销售价”也是一个锚定。职业销售人员知道，他们必须先设定一个锚——远在他们报价之前。</p><p>个人理解：在产品销售上可以利用锚定效应，买产品的时候要警惕锚定效应。</p><h4 id="032：归纳法"><a href="#032：归纳法" class="headerlink" title="032：归纳法"></a>032：归纳法</h4><blockquote><p>如何把别人的钱弄进自己的口袋</p></blockquote><p>一个农民喂食一只鹅。一开始鹅畏畏缩缩，想：“这个人为什么要喂我？这背后一定有 什么阴谋。”数星期过去了，农民天天都过来，扔给它谷子。它的疑心渐渐减弱。几个月后 这只鹅肯定地想：“这个人很喜欢我！”——这一信念每天都得到证明，于是它越来越坚定。 它对农民的善良坚信不疑。鵝没料到，农民在圣诞节会将它从鹅舍里取出并杀掉。这只圣诞 鹅成了归纳法思考的牺牲品。大卫·休谟早在18世纪就举过同样的例子，警告人们要小心归 纳法。可犯这种错误的不仅是鹅，我们大家都有由观察个体得出普遍适用的结论的倾向。这 是危险的。</p><p>我们也可以利用归纳法进行思考。这里将告诉你如何使用归纳法从别人的口袋里掏钱。 请你寄出100000份股指预测的邮件。你在一半邮件里预测下个月股票行情会上升，在另一半 邮件里预测股票行情将回落。假定一个月后股指回落了，你就再发一遍邮件，这回只发给你 作出了正确预测(股指会跌)的那50000人。你再将这50000人分成两组，写信给第一组，说股 指在接下来的一个月会上升，给另一组写信说股指会跌。如此类推，10个月后还剩下100个 人，你给他们作的预测从没有出错。在这100个人的眼里你就是英雄。你证明了，你拥有真正 的先知式的预测能力。于是这100个人中的几位会将他们的财产托付给你。然后你就可以怀揣 这笔钱逃去巴西了。</p><p>因此归纳式思考有时会产生严重后果，但没有它也不行。我们相信，当我们登上飞机 时，气体动力学的原理明天也管用。我们估计，我们不会在大街上被无缘无故地殴打。我们 指望，我们的心脏明天也照常跳动。我们需要归纳法，但我们不可以忘记，所有确信都只是暂时的。本杰明?富兰克林怎么说来着？ “除了死亡和税收，没有什么是肯定的。”</p><p>归纳法有可能具有诱惑性：“人类一直都是成功的，因此我们也将征服未来的挑战。”听起来不错，但我们没有考虑：只有那些幸存到现在的物种才能这么说。以我们存在的事实 来说明将来我们也会存在，这是一个严重的思维错误——估计是最严重的。 </p><p>个人理解：遇到利用归纳法得出的结论要多加小心，所有的确信都是暂时的，要尝试寻找一下反例。 </p><h4 id="033：规避损失"><a href="#033：规避损失" class="headerlink" title="033：规避损失"></a>033：规避损失</h4><blockquote><p>为什么凶恶的面孔比友善的面孔更容易引起我们注意</p></blockquote><p><strong>损失规避</strong>（英语：<strong>Loss aversion</strong>）是指人们面对同样数量的收益和损失时，认为损失更加令他们难以忍受。损失带来的负效用为收益正效用的2至2.5倍。损失厌恶反映了人们的风险偏好并不是一致的，当涉及的是收益时，人们表现为风险厌恶；当涉及的是损失时，人们则表现为风险寻求。</p><p>人们害怕失去某种东西的想法要比获得某种同等价值的东西的想法强烈。假定你是为房 屋提供隔热层的。假如你告诉顾客，缺少隔热层他们有可能损失多少钱，就要比告诉他们使 用好的隔热层他们能够节约多少钱，更能说服他们在房屋里使用隔热层。虽然其实金额是一 样的。 </p><p>员工(如果他们是独自承担责任，而不是集体作决定的话)都有畏惧风险的倾向。站在他 们的立场，这样做是有意义的：如果做成某件事情最多会带给他们一笔奖金，但一旦失败就 有可能让他们丢掉工作岗位，那他们干吗要冒这种风险呢？在几乎所有公司的所有情况下，风险都大于可能的收益。如果你作为董事长抱怨你的员工缺少冒险精神，那你现在知道是为 什么了——规避损失。</p><p>我们无法改变：恶比善更有影响力。我们对不利东西的反应要比对有利东西的反应敏感。走在大街上，一张凶恶的脸要比一张友善的脸更容易引起我们注意。恶行要比善行更久 地留存在我们的记忆里。当然也有例外：在事关我们自己的时候。 </p><p>个人理解：规避风险是人类与生俱来的，我们要理解和利用好损失规避，比如说服老板干一件事情的时候，收益和风险是一样的情况下，可以选择表达倾向于损失，这样冒险倾向就会增加。</p><h4 id="034：社会性懈怠"><a href="#034：社会性懈怠" class="headerlink" title="034：社会性懈怠"></a>034：社会性懈怠</h4><blockquote><p>团队为什么会使人懒惰</p></blockquote><p>马克西米利安·林格尔曼是一位法国工程师。1913年，他对马拉车的效率进行调查。他 发现：两匹马一起拉一驾马车，效率并非一匹马效率的双倍。这一结果令他意外，遂将他的 调查延伸到人类。他让许多人一起拉一根绳子，测量每人释放出的力量。他发现两个人一起 拉一根绳子，平均每人只投入其力量的93%,如果是3个人一起拉，每人只投入85%, 8个人一 起拉时就只剩下49% 了。除了心理学家，这个结果没让任何人感到意外。科学界称这一效应为社会性懈怠。</p><p>社会性懈怠是一种理性行为：假如使出一半力就行，又不会引起注意，为什么要使出全力呢？一句话，社会性懈怠是一种我们让自己亏欠所有人的欺骗形式。这一欺骗大多不是故 意的，而是不知不觉地发生的——就像马拉车一样。</p><p>社会性懈怠不仅出现在体能效率方面，我们在精神上也会懈怠，比如开会的时候。团队越大，我们个人参与的程度就越小–定大小的团队，当效率达到一个水时，就不会继续下降了。此时一支团队有20人还是100人，再也无关紧要，它已达到了最大的懒惰度。</p><p>结论：人们在团队里的行为不同于单独一人的时候(否则就不存在团队了）。可以通过尽可能彰显个人效率，来缓和团队的弊病。</p><p>减少社会惰化的途径</p><ol><li>合理控制团队成员，团队不宜过大，大团队应该再分成若干个小团队。</li><li>公布团队每个成员的业绩，同时根据团队成员的贡献、业绩进行评价。</li><li>帮助团队成员认识到其他成员的工作业绩，使他们了解不仅自己是努力工作的，他人也是努力工作的。</li><li>团队合理分工，权责分明。</li></ol><p>个人理解：作为团队领导要通过管理手段来来是团队发挥最大效率，避免陷入社会性懈怠中。</p><h4 id="035：指数增长"><a href="#035：指数增长" class="headerlink" title="035：指数增长"></a>035：指数增长</h4><blockquote><p>一张对折的纸为什么会超出我们的想象</p></blockquote><p>如果我们假设，一张纸的厚度为0. 1毫米，那么对折50次之后它的厚度 就是一亿千米。这相当于地球到太阳的距离，使用计算器很容易计算出来。</p><p>有一位政治家说：“交通事故的数量每年递增7%。”老实说，我们无法直观地理解这意 味着什么。因此，请你使用一个窍门：计算倍增时间。请用数字70除以增长的百分比。在上 述情况下，倍增时间为70 除以7=10(年）。于是这位政治家的意思是：“交通事故的数量每10年 翻一倍。”这相当令人警惕。</p><p>结论：当事关增长率时，请不要相信你的感觉。你的感觉是没有用的——请你承认这一 点。真正有助于你的是计算器，或者，在增长率小时，就使用计算倍增时间的诀窍。</p><p>个人理解：现在高速发展的时代，要警惕百分比的情况，多思考几分钟，如果心算得不到直观的答案，请掏出手机用计数器计算，不丢人。 </p><p><strong>70规则（Rule of 70）</strong> 是经济学里面的一个古老规律，是估计的捷径。70规则是指用来评估在当前的通货膨胀率水平下，物价需要花费多长时间才能翻一番的计算方法。假设一个经济体每年的通货膨胀率都相同，那么用70除以每年的通货膨胀率就可以得到物价翻番的年份。70规则还可以用来判断或翻番的年份。所以，可以说“70规则”是指某个变量年增长率为X%，则该变量在70/X年内将会翻一番。</p><h4 id="036：赢家的诅咒"><a href="#036：赢家的诅咒" class="headerlink" title="036：赢家的诅咒"></a>036：赢家的诅咒</h4><blockquote><p>你愿意为100欧元付出多少钱？</p></blockquote><p>赢家的诅咒是指：拍卖的赢家大多是事实上的输家。</p><p>你会为100欧元支付多少钱呢？请你想象一下，你和你的竞争对手被邀请参加这么一场拍 卖。游戏规则是：谁出价最高，谁就能得到这张100欧元的钞票；还有，很重要的一点是，两 位出价者都必须支付他们最后的报价。你会出到多高呢？在你看来，为这张100欧元的钞票支 付20、30或40欧元都是值得的。你的竞争对手当然也是同样的看法。哪怕支付99欧元都是有 意义的。现在你的竞争对手出到了 100欧元。如果这是最高报价，他会以零利润结束，但你必 须支付99欧元(你最后的出价)——没有回报。于是你会继续出价。出到110欧元时你肯定会损 失10欧元，但你的竞争对手会损失109欧元(他最后的报价)。因此他也会继续出价。你会在哪 里停止呢？你的竞争对手会在哪里停止呢？请你跟你的朋友们玩一次吧。</p><p>建议：“千万不要参与拍卖。”做不到？你是在一个无法躲开 拍卖的行业工作？那你就确定一个最高价，从中扣除20%支付赢家的诅咒效应。请你将这个数 字写在一张纸上，然后坚决遵守它。</p><p>个人理解：竞标的时候一定要事先定好最高价，然后严格执行；避免陷入赢家的诅咒，最后交付不了而破产。</p><h4 id="037：基本特征谬误"><a href="#037：基本特征谬误" class="headerlink" title="037：基本特征谬误"></a>037：基本特征谬误</h4><blockquote><p>千万别问一位作家他的小说是不是自传</p></blockquote><p>基本特征谬误是指，系统性地高估人的影响，在解释某些东西时低估情境因素。</p><p>尤其是在面对负面事件时，基本特征谬误会体现得更明显。我们将战争的“罪责”推给个人——第二次世界大战是希特勒的错，第一次世界大战是萨拉热窝的行剌者的错。虽然战 争是不可预见的事件，我们至今无法理解它的成因。因此，我们总是先在企业老板身上寻找业务好坏的原因。即使我们本该知道，领导才能 对经济成功的影响程度要远远小于普遍的经济形势和行业的影响力。有趣的是，在一个存在 危机的行业里，首席执行官们常被撤换，而在朝阳行业里却很少发生这样的事。这种决定一 点儿也不比更换足球俱乐部的教练的决定理智。</p><p>身为作家，我遇到的基本特征谬误是这样的：在每次读者见面会上，我碰到的第一个问 题总是：“你的小说里有什么是自传性质的吗？”我真想对着在座的人大喊：“见鬼，我们 要谈的可不是我，而是这本书，是书的文字、语言，是故事的可信性！”可惜我接受的教育不允许我这样发作。<br> <br>另外，我们也必须理解基本特征谬误：极度关注他人源自我们过去的进化史，隶属于一 个群体是生存所必需的，被排斥意味着死亡。繁殖、自卫和狩猎，大多是个体做不到的。我 们需要别人协助。特立独行的人——其中肯定有一些——早就从基因池里消失了。因此我们 才会这样过度地关注人。我们将90%的精力都用来关注人，只用10%的精力关注情境。</p><p>结论：不管戏剧多么让我们着迷，舞台上的人绝非孤立的，他们的表演离不开一个个情境。你若真想理解正在表演的戏剧，就请你不要只注重表演者，而是多关注他们的表演或舞蹈。</p><p>个人理解：要综合的看待事情，古人都告诉我们了，一件事情的成功离不开天时、地利、人和。</p><h4 id="038：错误的因果关系"><a href="#038：错误的因果关系" class="headerlink" title="038：错误的因果关系"></a>038：错误的因果关系</h4><blockquote><p>你为什么不该相信仙鹤送子</p></blockquote><p>最常见的因果谬误包括：把相关当因果、把必要当充分、把可能当必然</p><ol><li><strong>把相关当因果,也就是过于简单化因果关系的前因，给出的原因只是和结果相关，而不足以推论出结果。</strong><br>错误的因果关系的最好例子是出生率下降和德国的仙鹤配偶数量下降之间的联系。如果 画出1965~1987年两条线的发展，它们几乎可以完美地重叠在一起。那么，真的是仙鹤送子吗？不可能。因为这是一个纯属巧合的相互关系，肯定不是因果关系。</li><li><strong>把必要当充分</strong><br> 比如，如果想要成为钢琴家，坚持练琴是必要条件，但井不代表坚持练琴就一定能成为钢钢琴家。</li><li><strong>把可能当必然</strong><br> 比如你的同事说自己上一次的汇报表现不好，然后就觉得晋升无望了，事业不会再有发展了。</li></ol><p>结论：相互关系不等于因果关系。你要看仔细。有时两者之间因果恰恰相反，有时两者 之间根本就没有因果关系——就像仙鹤和婴儿一样。</p><p>个人理解：当我们在分析问题的时候，其实最重要的就是对问题的原因进行分析，但是导致一个事情结果的原因往往不是单一的，所以这时我们需要提高警惕，避免自己或他人出现因果谬误，对问题进行简单粗暴地归因。</p><h4 id="039：光环效应"><a href="#039：光环效应" class="headerlink" title="039：光环效应"></a>039：光环效应</h4><blockquote><p>长相好的人为什么容易事业有成</p></blockquote><p>光环效应是指：我们让某一个方面照花了眼睛，并由此推及全貌。</p><p>心理学家爱德华·李·桑代克在近100年前发现了光环效应。如果某人的某一个方面（比 如美貌、社会地位、年龄)造成了正面或负面的印象，那么它将“普照”其他的一切，从而过 度影响其总体印象。美貌是得到研究最多的例子。几十项研究都证明，我们会自动认为漂亮 的人更可爱、更诚实、更聪颖。事实也证明，有魅力的人更容易事业有成。光环效应在学校 里就已经得到了证明：教师们会本能地给长相好的学生打出更高的分数。</p><p>结论：光环效应挡住了我们的视线，让我们看不到真实的特征。因此你要看仔细。请你排除醒目的特征。世界级交响乐队就是这么选择队员的，他们让选手在一块幕布后面演奏， 通过这样做来避免性别、种族或外貌影响他们的评价。我衷心地建议经济记者们，不要靠季 度数据来评价一家公司（这已经有股市解决了），而要更深入地挖掘下去。这样挖掘出的东 西，并不总是漂亮的，但有时很有教益。</p><p>个人理解： 对待事物的时候要警惕从多方面，多角度立体、全面的了解，避免陷入光环效应中。 </p><h4 id="040：替代途径"><a href="#040：替代途径" class="headerlink" title="040：替代途径"></a>040：替代途径</h4><blockquote><p>恭喜你赢了俄罗斯轮盘赌</p></blockquote><p>替代途径是指，所有同样可能发生但没有发生的事情。</p><p>替代途径是无形的，因此我们很少会想到它们。任何玩垃圾债券、期权和信用违约互 换，挣到数百万欧元的人，都不该忘记，他同时有一堆危险的替代途径，它们会直接将他拖 进毁灭。</p><p>结论：风险从来不是一眼就能看到的。因此，请你时刻考虑你有什么样的替代途径。比起你通过无惊险的平凡途径（比如从事律师、牙医、滑雪教练、飞行员或企业顾问的辛苦工 作)获得的成功，别拿通过冒险的替代途径获得的成功太当真。蒙田怎么说来着：“我的生命充满不幸——这些不幸大多没有发生。”</p><p>个人理解：做事情前，考虑一下风险，如果风险发生是否是你能承受的。</p><h4 id="041：预测的错觉"><a href="#041：预测的错觉" class="headerlink" title="041：预测的错觉"></a>041：预测的错觉</h4><blockquote><p>水晶球如何歪曲了你的目光</p></blockquote><p>伯克利大学的教授分析了总共284位专家在10年内所作的82361个预言。结果，预测 的准确性几乎不及你询问一台随机数字生成器。事实证明，最糟的预测家恰恰是那些媒体关 注度最高的专家，尤其是世界灭亡预言家，其中又以瓦解论的代表为最——我们还一直在等 待加拿大、尼日利亚、中国、印度、印度尼西亚、南非、比利时和欧盟的瓦解呢(奇怪的是没 有专家想到过黎巴嫩）。</p><p>什么可以预测，什么不可以预测呢？比如我在预测我一年内的体重变化时误差就不会太大。而一件事情越复杂、时间跨度越长，其未来的发展变化就越多。气候变暖、油价或汇率的走势几乎是不可以预测的。发明则是完全不可能预测的。如果我们知道有一天我们会有幸获得什么技术，它们此刻就已经被发明出来了。</p><p>结论：请你对预测持批评态度。我为此训练出了一种条件反射——我会对每个预测报以 一笑，以此去除它的装腔作势，随后我会问自己两个问题。第一，这位专家的预测有何约束机制？假如他是雇员，如果他不断出错，他会失去他的工作吗？或者他只是一个自封的预测大师，靠图书和报告增加他的收入？第二，这位专家的预测准确率有多高？他在过去5年里作 过多少预测？其中有多少应验了，有多少落空了？我希望媒体在发布所有预测时，一同公布被误以为是大师的人们的成绩证明。最后，我要引用英国前首相托尼?布莱尔的一句话：“我不作预言。我从没做过，我永远不会做。”</p><p>个人理解：专注当下，停止预测，也不要太关注专家的预测。</p><h4 id="042：关联谬误"><a href="#042：关联谬误" class="headerlink" title="042：关联谬误"></a>042：关联谬误</h4><blockquote><p>有说服力的故事为什么会误导人</p></blockquote><p><strong>关联谬误</strong>指的是一种轻率概化方面的非形式归纳谬误，此类的谬误借由利用实质上不相关的关联（且常常诉诸情感）的论述，主张说某事物持有的性质也存在于另一种事物之上。</p><p>这里举一个例子，你看看哪种可能性更大：（1)法兰克福机场关闭了，航班被取消了； (2)法兰克福机场因天气恶劣关闭了，航班被取消了。这回你肯定能答对：答案（1)的可能性 更大，因为答案(2)必须满足另外一个条件，也就是恶劣天气。机场也可能是由于炸弹威胁、 意外事故或罢工关闭的。只是面对“可信的故事”我们不会想到这些事，至少，当我们——像你现在这样——对它们不敏感的时候。请你与你的朋友们做做这个测试。你会看到，大多数人都会选答案(2)。</p><p>结论：请你忘记“左半脑和右半脑”的说法吧，更重要的是直觉思维和有意识的思维之 间的区别。直觉思维偏好可信的故事，作重要决定时不顺从它们对你是有好处的。</p><p>个人理解：涉及情感的故事要警惕，摘除情感后看看有没有意外收获。</p><h4 id="043：框架效应"><a href="#043：框架效应" class="headerlink" title="043：框架效应"></a>043：框架效应</h4><blockquote><p>言为心声</p></blockquote><p>框架效应是指：视表达方式的不同，我们会对同样的事情做出不同的反应。</p><p>看一个例子，研究人员拿出两种肉：99%无脂的和1%含脂的。被问者都认为第一种肉更健康，虽然两种肉其实是一样的。即使是在98%无脂的和1%含脂的肉之间进行选择时，大多数被问者还是会选第一种肉——即使事实上它含有的脂肪是第二种肉的两倍。</p><p>作家们会有意识地使用框架效应。因为如果按照时间顺序，一步步描写出谋杀过程，侦 探小说就会很无聊。那将不是侦探小说，而是纪实作品。虽然最后都是讲述了整个故事，但 合理地利用框架效应才让故事更加吸引人。 </p><p>结论：你要意识到，没有框架效应，你什么也不能描述，每个事实——不管你是从一位 朋友那儿听到的，还是在一份严肃的报刊上读到的——都会受到框架效应的影响。本章也不 例外。</p><p>个人理解：我们要懂得合理利用框架效应，比如打广告，写作、汇报工作等，也要清醒的分辨是否有框架效应。</p><h4 id="044：行动偏误"><a href="#044：行动偏误" class="headerlink" title="044：行动偏误"></a>044：行动偏误</h4><blockquote><p>为什么不行动光等待是种痛苦</p></blockquote><p>足球运动员罚点球，有1/3的概率是射向球门中央，1/3的概率射向左边，1/3的概率射向 右边。守门员会怎么办？他们或者扑向左，或者扑向右。反正他们很少待在中间——虽然有 1/3的球会射向那里。这是为什么？因为扑向错误的一侧要比傻瓜似的呆立原地，看着球从左 边或右边飞过去好看得多，看起来也没那么难堪。这就是行动偏误：即使毫无用处，也要采取行动。</p><p>为什么会有行动偏误？在一个狩猎采野果的环境中，行动比思考价值大得多。在过去， 闪电式反应关乎生死存亡，思考则可能会致命。如果我们的祖先看到森林边缘出现一个看上 去像是剑齿虎的影子，他们不会像罗丹的“思想者”那样坐到一块石头上，去进行分类学思考。他们会逃跑，而且是拔腿就跑。我们全都是这些迅速反应者的后代，他们宁可不必要地 多逃跑一次。但我们今天的世界与过去不同——与行动相比，今天的世界奖励深刻的思考。 这一转向让我们很难适应。</p><p>结论：在不明情形下我们会产生要做点什么的冲动，随便什么——不管它有没有帮助。 之后我们会感觉好受些，虽然其实什么也没有好转——事实甚至往往正好相反。因此，如果 情况不明，请你不要采取任何行动，直到你能更好地分析形势。你要克制自己。“人类的全 部不幸就是他们不能安静地待在他们的房间里。”帕斯卡尔就曾经这么写道，在他的书房里。</p><p>个人理解： 在现在文明社会里，还是思考比立即行动更有作用，做事情前请多思考。</p><h4 id="045：不作为偏误"><a href="#045：不作为偏误" class="headerlink" title="045：不作为偏误"></a>045：不作为偏误</h4><blockquote><p>为什么你不是答案就是问题</p></blockquote><p><strong>不作为偏误</strong>是指一个人在面对不作为和作为时倾向选择前者。它的产生有多种原因，包括心理惯性对交易成本的认知，以及认为若不作为和作为会造成同样严重后果，作为比不作为更糟糕或不道德。</p><p>不作为偏误解释了，为什么我们更喜欢让某人自己撞上刀口，而不愿直接伤害他。</p><p>投资者和经济记者们感觉不研发新产品不如研发错误产品严重，虽然两者都会导致公司破产。</p><p>我们觉得死抱着几年前买的一堆差股票不卖不如卖了再买错股票严重；</p><p>不在一座煤电厂安装废气清洗器不如因成本原因拆除废气清洗器严重；</p><p>不给自己的房子安装隔热层不如为了取暖燃烧明火严重；</p><p>不声报收入不如伪造纳税资料严重——虽然结果都是相同的。</p><p>不作为偏误的形势大多是一目了然的：今天的行为可以防止未来的损害，但防 止损害对我们的激励并不强。不作为偏误很难辨认——放弃行动不像采取行动那么容易看出。</p><p>解释：“为什么你不是答案就是问题” 这句话意思是“如果你不解决问题，那你等于就是在延续问题、加重问题”，当然你就成了 “问题的一部分”</p><h4 id="046：自利偏误"><a href="#046：自利偏误" class="headerlink" title="046：自利偏误"></a>046：自利偏误</h4><blockquote><p>你为什么从不自责</p></blockquote><p>成功归于自己，失败归于外因。这就是自利偏误。</p><p>满分总是你自己的功劳，这么优秀的成绩反映了你的真实能力。如果你出现一次掉队，考了个低分呢？那就是考 试不公正。今天你不再在乎学校的分数，但你也许会关心股市行情。赚钱了，你沾沾自喜；亏钱了，你怪罪“股市氛围”或你的投资顾问。</p><p>如果公司这一年经营出色，首席执行官就会将其归因于他英明的决定、他本人不知疲倦的奉献和他所宣扬的活 跃的企业文化。相反，如果公司这一年经营不善，责任就都是欧元走强、政府的政策、中国 人的贸易活动、美国人的隐形关税，还有消费者的压抑情绪等。 </p><p>如何应对自利偏误呢？你有对你直言不讳的朋友吗？如果有，你很幸运。如果没有，那 你至少有个死对头吧？好，那你就挑战一下自己，请他喝咖啡，请他不加掩饰地说出对你个 人的意见。你会永远感激他的。</p><p>个人理解：这个情况还是非常普遍的，我们要做的就是认识到慢慢改善自己。别人有自利偏误一笑了之，心里有数就好。</p><h4 id="047：享乐适应症"><a href="#047：享乐适应症" class="headerlink" title="047：享乐适应症"></a>047：享乐适应症</h4><blockquote><p>你为什么应该缩短上班路程</p></blockquote><p>事业上迈进了一步的人在平均3个月后的幸福感又与先前一样了。就连那些非要驾驶最新款保时捷的人也一样。科学里称这一效应为享乐适应症： 我们工作、升迁，给自己购买更多更漂亮的东西，但我们不会变得更幸福。</p><p>科学的建议：</p><p>（1)请你避免很长时间也不会习惯的负面效应，例如往 返交通、噪音、慢性疲累等；</p><p>（2)请你对物质的东西只期待短期效果，例如汽车、房屋、分 红、中彩票、得金奖等；</p><p>（3)持续的正面效应主要与你如何利用你的时间有关。你要设法让自己得到尽可能多的自由时间和自主权。请你做你最爱做的事情——哪怕你要付出部分收入。 请你为友谊投资。</p><p>个人理解：任何物质或者升职等情况给我们带来的幸福和快乐都是短暂的，我们要把尽可能的时间投资到自己喜欢的事情上，这样就可能持续快乐和幸福了。</p><h4 id="048：自我选择偏误"><a href="#048：自我选择偏误" class="headerlink" title="048：自我选择偏误"></a>048：自我选择偏误</h4><blockquote><p>请不要惊讶有你存在</p></blockquote><p>自我选择偏误：当我们本身是抽样样品的一部分时，我们总是认为自己是特殊的，实际上我们不过是那个大样本中的一个。</p><p>自我选择偏误无所不在。营销负责人经常会掉进它的陷阱。例如：一家时事通讯报社给 他的客户寄去一张调查表，旨在调查他们觉得这份时事通讯报有多重要。可惜只有订阅了这 份报纸、尚未退订的客户收到了调查表——也就是对报纸较为满意的客户（其他人不再做样 品）。结果这调查自然是无用的。</p><p>特别有趣的是最近的一项电话调查：一家公司想查明 每个家庭平均有多少部电话(包括座机和手机)。在分析调查结果时，他们居然对没有一个家 庭声称没有电话而感到吃惊。这真是太可笑了！</p><p>解决方法：从自己样本的身份和角度中跳出来，分清楚总体和自己所处的样本位置。</p><h4 id="049：联想偏误"><a href="#049：联想偏误" class="headerlink" title="049：联想偏误"></a>049：联想偏误</h4><blockquote><p>为什么经验有时让人变蠢</p></blockquote><p>我们的大脑是一部联想机器。原则上这样也很好：我们食用一种陌生果子，食后我们感觉不舒服，于是将来我们就会回避相应的植物，认为它的果子有毒或至少是吃不得的。知识就是由此形成的。只是，错误的知识也是这么形成的。</p><p>我们从中可以学到什么？没有谁讲得比马克·吐温更贴切了： “我们应该注意，一个经历里隐藏着多少智慧，我们就只吸取多少——不要多；好让我们不像坐过热灶台的猫一样。 被烫过的猫永远不会再坐到热灶台上去——这是对的；但它也永远不会再坐到冷灶台上去 了。” </p><p>个人理解：在某种情况下，我们的大脑会把彼此毫无关系的两样东西联结在一起。假如我们收获到错误知识，我们要通过思考和验证把错误的联系给删除掉。</p><h4 id="050：新手的运气"><a href="#050：新手的运气" class="headerlink" title="050：新手的运气"></a>050：新手的运气</h4><blockquote><p>假如开始时一切顺利，请务必多加小心</p></blockquote><p>新手的运气是联想偏误的一种特殊情况：错误地与从前的成功建立联系。</p><p>新手的运气在经济生活里扮演着重要角色：A公司买下了较小的B、C、D公司。由于每次 收购都很成功，A公司领导信心倍增，相信自己擅长收购公司。因为受到鼓舞，A公司又买下了比它大得多的E公司。事实证明，这次收购是个灾难。清醒地看，这本是能够预料到的，但 A公司被新手的运气照花了眼睛。</p><p>在2001~2007年美国房地产繁荣期间可以观察到同样生动的例子。牙医、律师、教师和出 租车司机纷纷放弃他们的工作参与“炒”房——买房，然后再以一个更高的价格出售。最初的丰厚利润证明他们做得对，但这当然也与能力无关：房地产泡沫将每个还十分笨拙的业余 掮客捧到意想不到的高度。许多人举债来“炒”更多更大的别墅。当市场最终崩溃时，他们 债台高筑。</p><p>从哪一刻开始就不再是新手的运气，而是天才呢？没有明确的分界，但有两条线索。第 一，如果你长期比其他人成功，你可以认为，自己的才华可能起到作用，但你绝不能过于自 信、自满。第二，参与的人越多，某人出于纯粹的运气长期成功的概率就越大。也许你就是 这个某人。如果你在一个只有10名竞争者的市场上脱颖而出，这说明你有一定的才华。如果 你在一个拥有千万名竞争者的市场上成功了，你就不应该太骄傲（比如说在金融市场上）。这 种情况下你应该认为，你只是很幸运。不管怎样，请你不要急着作出判断。</p><p>个人理解：这本质是一个概率问题，我们理解事情上还是要多从概率上想想，成功的概率是多少，失败的概率是多少，从理性出发。</p><h4 id="051：认知失调"><a href="#051：认知失调" class="headerlink" title="051：认知失调"></a>051：认知失调</h4><blockquote><p>你如何撒点小谎，让自己感觉好一些</p></blockquote><p>认知失调是指打算与结果不符，事后作出别的解释。</p><p>一只狐狸偷偷地靠近一棵葡萄树，渴望地盯着树上熟得发紫的大葡萄。它拿前爪撑着树 干，伸长脖子，想摘几串葡萄，可葡萄太高了。它恼怒地想再次试试它的运气，它纵身跃 起，但扑了个空。第三回它用尽全身的力气一跳——又扑空了，它背朝下摔在地上，而葡萄 树连一片叶子都没有动一下。狐狸耸耸鼻子，说：“我觉得它们还没熟透，我不喜欢酸葡萄。”它骄傲地昂首走回了森林里。</p><p>你购买了一辆新轿车。你很快就后悔了——发动机太响、座位不舒服。怎么办？你没有将轿车退回去——不，那将是承认你犯了一个错误，你不降价汽车商很可能也不会再要它。于是你对自己说，马达响、座位不舒服，正好可以防止你开车时睡着觉 ——因此你买回了一辆特别安全的车。一点儿也不蠢，你想，你对自己的选择又感到满意了。</p><p>假设你要申请一份工作，但人家选择了另一位候选人而没选你。你不会承认是你资质不 够，而是会劝自己你其实根本不想要这份工作，你只是想再测试一下你的“市场价值”，看看人家到底还会不会请你去面试。 </p><p>减少认知失调的方法</p><ol><li>改变认知。如果两个认知相互矛盾，我们可以改变其中一个认知，使他与另一个相一致。</li><li>增加新的认知。如果两个不一致的认知导致了失调，那么失调程度可由增加更多的协调认知来减少。</li><li>改变认知的相对重要性。因为一致和不一致的认知必须根据其重要性来加权，因此可以通过改变认知的重要性来减少失调。</li><li>改变行为。认知失调也可通过改变行为来减少，但一般情况下，行为比态度更难改变。</li></ol><p>个人理解：有很大成分是因为面子或自尊心问题，才做出其他的解释。我们丢弃面子或自尊心，提升自己的能力，改变认知。</p><h4 id="052：双曲贴现"><a href="#052：双曲贴现" class="headerlink" title="052：双曲贴现"></a>052：双曲贴现</h4><blockquote><p>及时行乐——但请只限于星期天</p></blockquote><p>双曲贴现是指一个决定离现在越近，我们的 “情感利息”就越多。</p><p>米舍尔在20世纪60年代就延迟满足做过一次著名的试验，名为“棉花糖试验”。他将一块棉花糖(甜食)放 在一群4岁的小男孩面前，让他们选择要么立即吃掉，要么，如果他们愿意等上几分钟，不吃 第一块，就会再得到一块。惊人的是，只有极少数孩子愿意等。更惊人的是，米舍尔发现， 是否拥有延迟满足的能力是他们后来事业是否成功的一个可靠的指示器。 </p><p>结论：及时行乐的诱惑力极大——尽管如此，双曲贴现也是一种思维错误。我们越能控制我们的冲动，我们就越能成功地规避这一错误。我们对我们的冲动控制越小——比如在酒精的影响下——我们就越容易犯这个错误。及时行乐是个好主意——如果每星期一次的话。 但天天享受，好像每天都是末日似的，却是不明智的。</p><p>个人理解：我们宁要相对较少的眼前，也不愿等待数额更多的日后报酬。或者说，做决策时，大部分人愿意选择短期收益，而不愿选择长期更多的收益。</p>]]></content>
<summary type="html"><h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote>
<p>记录一下阅读该书籍的理解与感想。</p>
</blockquote>
<h4 id="026：忽视概率偏误"><a href="#026：忽视概率偏误" class="headerlink" title="026：忽视概率偏误"></a>026：忽视概率偏误</h4><blockquote>
<p>累计奖金为什么会越来越多</p>
</blockquote>
<p>有两种赌博：第一种赌博你有可能赢1000万欧元，第二种赌博你可能赢1万欧元。你会参加哪一种呢？如果你在第一种赌博里赢了，你的生活将会彻底改变：你可以辞掉工作，从此靠利息生活。如果你在第二种赌博里赢了，你可以美美地前往加勒比海度假，然后一切照旧。第一种赌博赢的概率是亿分之一，第二种赌博赢的概率是万分之一。好吧，你会玩哪一 种？我们的感情冲动会将我们拉向第一种，虽然客观地看，第二种赢的概率要大无数倍。因 此累计奖金越来越多——数百万、数亿、数万亿美元——无论赢的机会是多么微弱。
</p>
<p>结论：我们很难区分各种风险，除非风险为零。由于我们不能直觉地理解风险，我们必须计算。在概率公开的地方——像彩票——这就很容易。而在普通生活中，风险很难估计，但又是躲也躲不过的。</p>
<p>个人理解：我们对生活中的概率不太敏感，以后遇到有事情要多思考概率的大小，以便做出合理的判断。</p></summary>
<category term="book" scheme="http://tungsing.cc/categories/book/"/>
<category term="book" scheme="http://tungsing.cc/tags/book/"/>
<category term="思考" scheme="http://tungsing.cc/tags/%E6%80%9D%E8%80%83/"/>
<category term="逻辑" scheme="http://tungsing.cc/tags/%E9%80%BB%E8%BE%91/"/>
</entry>
<entry>
<title>清醒思考的艺术阅读笔记</title>
<link href="http://tungsing.cc/2021/12/05/book/the_art_of_thinking_clearly/chapter_one/"/>
<id>http://tungsing.cc/2021/12/05/book/the_art_of_thinking_clearly/chapter_one/</id>
<published>2021-12-05T02:18:26.092Z</published>
<updated>2023-03-22T08:44:52.762Z</updated>
<content type="html"><![CDATA[<h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote><p>记录一下阅读该书籍的理解与感想。</p></blockquote><h4 id="介绍"><a href="#介绍" class="headerlink" title="介绍"></a>介绍</h4><p>该书是通过通过 52 个简短的小文章深入浅出地讲解了一些常见的思维陷阱。使人们犯错更少，离成功更近。</p><h4 id="001：幸存偏误"><a href="#001：幸存偏误" class="headerlink" title="001：幸存偏误"></a>001：幸存偏误</h4><blockquote><p>为什么你该去逛逛墓地</p></blockquote><p>幸存偏误是指：由于日常生活中更容易看到成功、看不到失败、你会系统性的高估成功的希望。不了解现实的你对成功抱有一种幻想，认识不到成功的概率有多微弱。</p><p>幸存偏误意味着：你系统性地高估了成功概率。</p><p>解决办法：尽可能常去逛逛曾经大有希望的项目、投资和事业的墓地。这样的散步虽然伤感，但对你是有好处的。</p><p>个人理解：做事情或者项目前不仅仅要关注成功的案例也要多收集和研究失败的案例，对全部案例分析后再决定要不要做，避免陷入幸存偏误中。</p><span id="more"></span><h4 id="002：游泳选手身材错觉"><a href="#002：游泳选手身材错觉" class="headerlink" title="002：游泳选手身材错觉"></a>002：游泳选手身材错觉</h4><blockquote><p>哈佛是好大学还是烂大学？我们不清楚</p></blockquote><p>职业游泳者体形完美，并不是因为他们锻炼充分。实际情况正好相反：他们之所以成为出色的游泳选手，是因为他们拥有这样的身材。他们的身躯是一种选择标准，而不是他们运动的结果。一旦我们混淆选择标准和结果，我们就会产生游泳选手身材错觉。</p><p>结论：凡有人讴歌某种东西值得追求——强健肌肉、美貌、高收入、长寿、影响力、快乐，你都要看仔细。在跨入泳池之前，不妨先照照镜子。你要诚实地对待自己。</p><p>个人理解：是个因果关系分析的问题，比如一个厉害的团队并不是因为团队领导多厉害，而是团队本身是具备明确的目标、良好的沟通、共同的价值观和行为规范、不同的角色、每个团队成员都有归属感、资源可以共享和互通有无的标准团队。</p><h4 id="003：过度自信效应"><a href="#003：过度自信效应" class="headerlink" title="003：过度自信效应"></a>003：过度自信效应</h4><blockquote><p>你为什么会系统性地高估自己的学识和能力</p></blockquote><p>我们总是系统性地高估我们的学识和预测能力——而且高估得很厉害。对于过度自信效应，重要的不是单个估计是否正确。过度自信会令你忽视你真正知道的东西与你已知的东西之间的区别。</p><p>有两个效应在共同起作用。一个是传统的过度自信，另一个是项目的直接利益人在激励下低估成本。调研员希望拿到系列订单，建筑企业和供应商亦然，建筑业主感觉得到了乐观数据的支持，政治家们靠这样做拉选票。我们会在另一章简述这种激励过敏倾向。重要的区别在于：过度自信不是受到了激励，而是自然单纯、生而有之的。</p><p>结论：请对所有预测持怀疑态度，尤其是当这些预测是由所谓的专家们作出的。请你在筹划任何事情时都从悲观的角度出发，作最坏的打算。这样你才会真正有机会，更现实一些地判断形势。</p><p>个人理解：</p><h4 id="004：从众心里"><a href="#004：从众心里" class="headerlink" title="004：从众心里"></a>004：从众心里</h4><blockquote><p>就算有数百万人声称某件蠢事是对的，这件蠢事也不会因此成为聪明之举</p></blockquote><p>从众心理（有时被含糊地称为随大溜）是指：只要别人做什么我也跟着做什么，我的行为就是正确的。换言之，越多的人认为一个想法正确，这个想法就更加正确——这当然是荒谬的。</p><p>比如：喜剧和访谈节目常会利用从众心理，在关键位置插入笑声，事实证明这会引得观众发笑。</p><p>个人理解：出现从众的情况下，多思考下为什么，是否值的做。</p><h4 id="005：纠缠于沉默成本"><a href="#005：纠缠于沉默成本" class="headerlink" title="005：纠缠于沉默成本"></a>005：纠缠于沉默成本</h4><blockquote><p>你为什么应该忽视过去</p></blockquote><p>每个决定，不管是私人的还是业务上的，始终是在不确定的情况下作出的。我们的设想，有可能兑现，也有可能落空。任何时候我们都可能离开选取的小道，并承担后果，比如中断项目。这种不确定情形下的权衡是理性行为。然而，在我们已经投入特别多的时间、金钱、能量、爱等因素之后，沉没成本令人难以放手、难以释怀。于是已经投资的钱就成了继续做下去的理由，即使客观来看坚持下去毫无意义。投资越多，沉没成本就越大，将项目继续做下去的理由就越充分。</p><p>股市投资人经常成为沉没成本的受害者。他们在决定是否出售股票时常以买入价作为参照。当股价高于买入价时，就卖掉股票；如果股价低于买入价，就抱住不卖。这是不理智的，绝不可以让买入价处处扮演角色。唯一有效的是股市未来的前景（和可选投资未来的行情）。每个人都会出错，特别是在股市里。纠缠于沉没成本的不幸，其关键就是：你投资一只股票亏的钱越多，你越是抱紧它不放。</p><p>结论：理性的决定意味着忽视已经投入的成本。你已经投资了什么并不重要，唯一重要的是现在的形势及你对未来的评估。</p><p>个人理解：就现在的形势来评估未来的行动，而不要把发生过的内容作为参考。</p><h4 id="006：互惠偏误"><a href="#006：互惠偏误" class="headerlink" title="006：互惠偏误"></a>006：互惠偏误</h4><blockquote><p>你为什么不该让别人请你喝饮料</p></blockquote><p>互惠古来有之。它的基本含义是：“我帮你，你帮我。”我们发现那些食物总量变化很大的动物之间都存在互惠。假定你是猎人，有一天运气好，猎杀了一头鹿。肉很多，你一天吃不完。当时还没有冰箱，于是你就与你的群体成员瓜分了这头鹿。这样，当你有一天运气不好时，你也可以从别人的猎物中获得好处。这是一种出色的生存策略。互惠是风险管理；没有互惠，人类——还有无数种动物——早就灭绝了。</p><p>互惠也有可恶的一面：报复。紧接着报复的是反报复，然后你就会陷入一种恶性循环中。耶稣曾经布道，要求将另一面脸也伸给攻击者，也就是打断恶性循环，但这是很难做到的，因为互惠这一理念已经在我们脑中顽固地存在了一亿多年。</p><p>个人理解：当情况是陷入互惠的可恶的一面是，要及时打破它</p><h4 id="007：确认偏误"><a href="#007：确认偏误" class="headerlink" title="007：确认偏误"></a>007：确认偏误</h4><blockquote><p>遇到“特殊情况”这个词，你要格外小心</p><p>干掉你的宠儿</p></blockquote><p>确认偏误是所有思维错误之父——它倾向于这样诠释新信息，让它们与我们现有的理论、世界观和信念相兼容。换句话说：我们过滤掉与我们的现有观点相矛盾（因此被称作反驳证据，不过它缺少合适的德语表达）的新信息。</p><p>结论：请你与确认偏误作斗争。请你写下你的信条——有关世界观、投资、婚姻、健康预防措施、节食、成功策略的，然后寻找反驳证据。干掉自己最心爱的理论，这是一桩艰苦的工作，但作为聪明人士，你不会躲避的。</p><p>个人理解：我们要有意识的锻炼自己去寻找反驳证据，尤其是听到”特殊情况“这个词，更要思考它背后隐藏的反驳证据。</p><h4 id="009：权威偏误"><a href="#009：权威偏误" class="headerlink" title="009：权威偏误"></a>009：权威偏误</h4><blockquote><p>你为什么该藐视权威</p></blockquote><p>关于权威有两个问题。首先是令人警醒的跟踪记录。这个星球上有大约100万受过培训的经济学家，没有一位精确预言了金融危机发生的时间，更别说房地产泡沫的破裂、信用违约互换的瓦解直到通货膨胀引发的经济危机的顺序了。再没有哪个专家群体失灵得比这更惊人了。</p><p>其次，经常可以证明，权威也会出错。出错是人类的通病，但重要的是，面对权威我们会将独立思考调低一级。面对专家意见时我们往往会比面对其他意见粗心许多。还有，我们会服从权威，哪怕是在理性或道德上毫无意义的地方。这就是权威偏误。</p><p>结论：不管什么时候遇到一位专家，我都会设法向他挑战。请你也这么做。你对权威的批判性越强，你就越自由，就越相信自己有更多的能力。</p><p>个人理解：自己要有独立思考的能力，这样才不会受束于别人。</p><h4 id="010：对比效应"><a href="#010：对比效应" class="headerlink" title="010：对比效应"></a>010：对比效应</h4><blockquote><p>你为什么最好别找模特儿等级的朋友一起出门</p></blockquote><p>当我们同时面对某种难看、便宜、小的东西时，我们就会判断出另一种东西更漂亮、更贵、更大。我们很难作出绝对的判断。对比效应是常见的思维错误之一。</p><p>你在学生时代可能做过以下实验：你拿来两只桶，在第一只桶里倒进温水，在第二只桶里倒进冰水。你先将右手插进冰水里一分钟，然后将两只手同时插进温水里。你有什么感觉？左手感觉水是温的，右手却感觉水是烫的。</p><p>没有对比效应，就完全无法想象打折生意。一种从100欧元降到70欧元的产品，会显得比一直就卖70欧元的产品更便宜。而事实上最初的价格根本无关紧要。不久前一位投资者对我说：“这只股票便宜，因为它只有最高价的50%。”我轻轻摇摇头。股票价格永远不会“低”或“高”。它是怎么样就是怎么样，唯一值得考虑的是它从这一刻起是会涨还是会跌。</p><p>个人理解：我们要警惕对比效应的陷阱，也要合理的利用对比效应，比如打折。</p><h4 id="011：现成偏误"><a href="#011：现成偏误" class="headerlink" title="011：现成偏误"></a>011：现成偏误</h4><blockquote><p>你为什么宁可用一张错误的地图，也不愿没有地图</p></blockquote><p>现成偏误是指：我们依据现成的例子来想象世界。这当然是愚蠢的，因为外界现实中的某种东西不会因为我们更容易想到而出现得更频繁。</p><p>我们的大脑是剧本式思维的，而不是量化思维的。</p><p>人们都是先使用现成的数据或配方。他们在这个基础上作决定——结果经常是灾难性的。例如：10年前人们就知道，用所谓的期权定价模型公式确定衍生金融产品的价格是行不通的。但没有别的公式。因此人们聊胜于无，宁可使用一个错误公式。“波动率”也一样。用它测量一种金融产品的风险是错误的，但它是现成的。于是现成偏误给银行造成了巨额损失。这就好像你身在一座没有地图的陌生城市里，但你口袋里有另一座城市的地图，于是你就使用了那张地图。你宁可用一张错误的地图，也不愿没有地图。</p><p>应对办法：请与跟你想法不同的人合作，跟那些与你拥有截然不同经验的人合作。因为你独自战胜不了现成偏误。</p><p>个人理解：做事情要通过量化和群体决策来完成，这样才有可能避免陷入现成偏误的陷阱。</p><h4 id="012：“在好转之前会先恶化”的陷阱"><a href="#012：“在好转之前会先恶化”的陷阱" class="headerlink" title="012：“在好转之前会先恶化”的陷阱"></a>012：“在好转之前会先恶化”的陷阱</h4><blockquote><p>如果有人建议你选择一条“先经历痛苦的道路”，你应该敲响警钟</p></blockquote><p>“在好转之前会先恶化”的陷阱是确认偏误的一种变体。使用这个花招大大有利于一个对专业一窍不通或对事情没有把握的专业人员。如果情况继续走下坡路，就证明了他的预言是正确的。如果情况意外地回升了，客户开心，专业人员则可以将好转归功于他的能力。不管怎样——他总是对的。</p><p>结论：如果有人说：“在好转之前会先恶化”，你脑子里就应该敲响警钟。不过请小心：确实有那样的情形，先是再次下滑然后回升。事业的转换可能会耗费时间，会造成停发工资。一个企业的重组也需要一定的时间。但所有这些情况，人们很快就能看出措施是否有效。里程碑是明确的，是可以检测的。请你望着里程碑，而不是望着天空。</p><p>个人理解：遇到例如：将有“艰难的几年”，要求你的同胞们“勒紧裤腰带”；许诺等“清洗”、“除渣”、“改组”的“棘手阶段”结束后形势就会好转等情况要仔细思考、求证、检测、观察是不是如所说的一致。</p><h4 id="013：故事偏误"><a href="#013：故事偏误" class="headerlink" title="013：故事偏误"></a>013：故事偏误</h4><blockquote><p>为什么就连真实的故事也是骗局</p></blockquote><p>故事偏误是指：用故事扭曲和简化现实，它们排斥不合适编进故事的一切。</p><p>结论：从自传到世界大事——我们将一切炮制成有“意义”的故事。我们这样做是在扭曲真相——这会影响我们决定的质量。</p><p>应对方法：请你将这些故事拆解开来。请你问问自己：这些故事想隐藏什么？</p><p>训练方法：请你设法用无关联的眼光看看自己的生平，你会吃惊的。</p><p>个人理解：我们要用科学的思维来分析和理解故事的背后的真实情况。</p><h4 id="014：事后诸葛亮偏误"><a href="#014：事后诸葛亮偏误" class="headerlink" title="014：事后诸葛亮偏误"></a>014：事后诸葛亮偏误</h4><blockquote><p>你为什么应该写日记</p></blockquote><p>事后诸葛亮偏误绝对是最顽固的思维错误之一，可以恰如其分地称为“我早知道现象”，即事后回顾时一切都显得是可以理解的、不可避免的。</p><p>事后诸葛亮偏误为什么这么危险呢？因为它让我们相信自己是很好的预言家，而事实却不是这样。这会导致我们傲慢，误导我们作出错误的决定。在私事上也是如此。</p><p>建议：与事后诸葛亮偏误作斗争并不容易。研究表明，知道这种偏误的人，也常像其他所有人一样掉进它的陷阱。但我还有一个建议，这建议更多是来自个人经验而不是科学：请你记日记。请你写下你的预测——有关政治、事业、体重、股市等。请你不时地拿你的记载与实际情况相比较。</p><p>个人理解：生活充满了不可预见性，把握当下，迎接未来。</p><h4 id="015：司机的知识"><a href="#015：司机的知识" class="headerlink" title="015：司机的知识"></a>015：司机的知识</h4><blockquote><p>你为什么不可以把新闻播音员说的话当真</p></blockquote><p>查理·芒格是全球最优秀的投资家之一，普朗克的故事我就是从他那里听来的。他认为知识有两种：一种是真知识，来自那些投入大量时间和思考以获得知识的人们；另一种就是司机的知识，按芒格故事里的意思，司机是指那些装得好像他们知道的人。他们会模仿别人表演，他们也可能拥有动听的声音或具有说服力的形象。但他们传播的知识是空洞的，他们高谈阔论地挥霍着华丽词汇。</p><p>结论：请你不要信任司机的知识。请你不要将公司新闻发言人、爱出风头的人、新闻播音员、唠叨鬼、花言巧语者、爱说闲话的人与一个真正有知识的人搞混。你怎么能区分出来呢？<strong>这里有个明确的信号。真正有知识的人知道他们知道什么，也知道他们不知道什么。这类人一旦来到他的“能力范围”之外，他要么什么也不说，要么就说“我不知道”。他这么说时不会觉得难为情，甚至还会带着一定的骄傲。</strong> 而从“司机们”那儿，你别的什么话都能听到，就是听不到这一句。</p><p>个人理解：警惕司机们，不要为不在自己能力范围内的事情说不知道而感觉不好意思或难为情。但也要不断的扩大自己的能力范围。</p><h4 id="016：控制错觉"><a href="#016：控制错觉" class="headerlink" title="016：控制错觉"></a>016：控制错觉</h4><blockquote><p>你实际控制的少于你以为的</p></blockquote><p>控制错觉是指：相信我们能够控制或影响某种我们客观上无法控制或影响的东西的倾向。这是詹金斯和沃德两位研究人员在1965年发现的。试验规则很简单：两只开关和一盏灯，灯或开或关。詹金斯和沃德可以调节开关和灯光互相制约的强度。即使是在灯纯属偶然地开、关的情况下，受试者也坚信，按开关能够在某种程度上影响灯。</p><p>案例：一位美国科学家将人们关在一间音响室里，不断提高音量，直到受试者表示拒绝，以此调查人们对噪音的疼痛承受程度。有两间音响室，A音响室和B音响室，它们一模一样，只有一个区别：B音响室的墙上有个红色的紧急按钮。结果呢？B音响室里的人们承受的噪音明显要大得多。可笑的是，那个紧急按钮根本不管用。错觉本身足够提高人们承受疼痛的极限。</p><p>结论：请你将注意力集中于你真正能影响的少量东西——坚定不移地只关注其中最重要的那些。其他的，听之任之吧。</p><p>个人理解：我们能控制的东西很少，我们要把有限的精力投入到自己真正喜欢、感兴趣的事情上。其他的管它呢！</p><h4 id="017：激励过敏倾向"><a href="#017：激励过敏倾向" class="headerlink" title="017：激励过敏倾向"></a>017：激励过敏倾向</h4><blockquote><p>你为什么不该按实际开销付钱给你的律师</p></blockquote><p>激励过敏倾向的例子：1947年，当死海的卷轴被发现时，考古学家们悬赏收集每一张新发现的羊皮纸手稿。结果，为了增加手稿数量，那些羊皮纸被撕碎了。同样的事也发生在19世纪的中国，当悬赏征购恐龙骨时，农民们会将挖出的完好恐龙骨砸碎，再去领赏；按实际开销付钱给律师、建筑师、咨询师、会计师或驾校老师是愚蠢的。这些人受到激励，就会尽可能多花钱。因此请你事先约定一个固定价格。</p><p>正相激励的例子：在古罗马，当一座桥竣工通行时，工程师必须站在桥拱下。这种机制相当好，会鞭策工程师将桥建得足够稳固。好的激励机制会将目的和激励联系起来。</p><p>结论：请小心激励过敏倾向。如果你对某个人或某个组织的行为感到吃惊，请你想想，那后面隐藏着什么激励机制。我保证，你可以这样解释90%的行为。激情、精神疾病、心理障碍或恶意最多占到10%。</p><p>个人理解：如果要制定激励机制，一定要和事情的目的联系起来；生活中理解一件事情，可以想想背后有没有激励机制。</p><h4 id="018：回归均值"><a href="#018：回归均值" class="headerlink" title="018：回归均值"></a>018：回归均值</h4><blockquote><p>医生、顾问、教练及心理治疗师的作用令人怀疑</p></blockquote><p>假定你正在经历一次破纪录的严寒，接下来的几天气温就极有可能回升——朝着月度均值的方向。每逢酷暑、干旱和雨季时也是这样，气候会围绕一个均值波动。这同样适用于慢性疼痛、高尔夫球成绩、股市业绩、恋爱运气、主观舒适感、职场成功、考试分数等。总之，剧烈的背痛很有可能不用找心理治疗师也会减弱；高尔夫球成绩不用额外上课也会重新变好；那位投资顾问的表现不用跳“雨舞”也会移向均值。</p><p>结论：当你听到诸如“我病了，去看医生 ，现在我好了，因此是那位医生帮助了我”或“这一年公司业绩很糟糕，我们请了个顾问回来，现在业绩恢复正常了”的话时，很有可能就是回归均值在起作用。</p><p>个人理解：事情或者生活就是起起落落的，我们要以平常心来对待</p><h4 id="019：公地悲剧"><a href="#019：公地悲剧" class="headerlink" title="019：公地悲剧"></a>019：公地悲剧</h4><blockquote><p>为什么理性的人不去诉诸理性</p></blockquote><p>请你设想有一块肥沃的土地，一座村庄的所有农民均可使用。可以预料，每个农民都会将尽量多的奶牛赶到这块土地上去放牧。只要没有人偷猎、没有疾病蔓延，这件事就行得通，简言之：只要奶牛总数不超过一定数量，也就是土地不被剥削殆尽即可。可是，一旦事情不是这样，公地的美丽想法就会突转为悲剧。作为理性的人，每个农民都试图将他的利润最大化。他心想：“我要是再多将一头奶牛赶去公地，我会得到什么好处呢？”他可以得到多出售一头奶牛的额外好处。多一头奶牛造成的过度放牧的害处由所有人承担，单个农民承担的损失极小。所以从他的角度看，将更多的奶牛赶到公地上去放牧是理性的。于是再来一头、再来一头，直至公地被毁掉，这就是公地悲剧。</p><p>结论：只有两种解决方法：将公地私有化或加强管理。对于无法私有化的东西，如臭氧层、海洋、卫星运行轨道等，必须加强管理。</p><p>个人理解：公共的东西必须加强管理</p><h4 id="020：结果偏误"><a href="#020：结果偏误" class="headerlink" title="020：结果偏误"></a>020：结果偏误</h4><blockquote><p>切勿以结果判断决定</p></blockquote><p>结果偏误又叫作史学家错误。我们倾向于以结果判断决定，而不是当时作决定的过程。过程是重要的，由于作决定的过程会被不少的因素所左右。</p><p>结论：请你勿以结果判断决定。结果差并不必定意味着当时所做的决定不对，反之亦然。你最好仔细研究一下当时作这样决定的缘由，而不是吐槽一个被证实是错误地决定，或者为一个也许是纯属偶然的得到成功的决定而感到庆幸。这个决定是出于理性而做出的吗？若是是，恭喜你，那你下回最好仍然这样作，哪怕上一回的结果十分的糟糕。</p><p>个人理解：看淡结果，注重过程的思考。</p><h4 id="021：选择的悖论"><a href="#021：选择的悖论" class="headerlink" title="021：选择的悖论"></a>021：选择的悖论</h4><blockquote><p>为什么更多反而是更少</p></blockquote><p>选择多是进步的标尺。有所选择令人幸福。但这也有一个‘‘度”，过多的选择会降低生活质量。专业术语称 之为选择的悖论</p><p>美国心理学家巴里 施瓦茨在他的《不满指南》一书里说明了为什么会这样。原因有三：</p><p>第一，选择范围太大会导致无所适从。</p><p>第二，选择范围大会导致作出更差的决定。</p><p>第三，选择范围大会导致不满。</p><p>怎么办：请你在端详面前的选项之前，仔细考虑你想要什么。请你写下你的标准，并务 必遵守它们。你要明白，你永远作不出完美的选择。要想作出在事后看来无懈可击的选择是 非理性的，因为事情的发展永远有无数种可能性。你就满足于一个适合你的“好答案”吧。 是的，在生活伴侣这件事上也是如此。只有最好的才适合你吗？在存在无限选择的年代，情况恰恰相反：适合你的才是最好的。</p><p>个人理解：做选择的时候就基于当前的情况和目标，选择一个适合自己的即可。</p><h4 id="022：讨喜偏误"><a href="#022：讨喜偏误" class="headerlink" title="022：讨喜偏误"></a>022：讨喜偏误</h4><blockquote><p>你行为不理性，是因为你想讨别人喜欢</p></blockquote><p>讨喜偏误是指：某人越讨喜，我们就越倾向于从这个人那儿买东西或者帮助他。</p><p>讨喜是指：我们会觉得一个人讨喜，如果他：（1)外表有吸引力；（2)在出身、个性和兴趣上与我们相似；（3)他觉得我们讨人喜欢。这三个因素是依次排列的。</p><p>结论：你在判断一笔生意时应该坚持不受卖方影响。请你不要考虑他，更好的办法是： 请你想象他是个不讨人喜欢的家伙。</p><p>个人理解：在卖买的时候要记得想一下讨喜偏误。买的时候要主要销售人员是否利用了讨喜偏误；卖的时候我们也要善于利用讨喜偏误。</p><h4 id="023：禀赋效应"><a href="#023：禀赋效应" class="headerlink" title="023：禀赋效应"></a>023：禀赋效应</h4><blockquote><p>请不要死抱着某种东西不放</p></blockquote><p>禀赋效应是：我们感觉我们拥有的东西比我们没有拥有的更有价值。换句话说：当我们出售某物时，我们要求的钱多于我们自己愿意为它支付的钱。</p><p>让我们放弃要比让我们囤积难得多。这不仅解释了为什么我们会在房子里堆满废物，也说明了为什么邮票、手表或艺术品的爱好者很少将他们的收藏拿出来交换或出售。</p><p>你应聘一份工作却没有成功，你会很失望。如果你知道，你直坚持到了最终环节，然后被拒绝了，你的失望还要大得多—这是没有道理的。因为你要么成功，要么不成功，其他的切都无关紧要。</p><p>结论：请不要死抱着某种东西不放，请将你拥有的视作“宇宙”临时留给你的某种东西。要知道你拥有的一切随时又会被拿走。</p><p>个人理解：我们要理性的对待我们所拥有的物品，不要赋予它情感价值。</p><h4 id="024：奇迹"><a href="#024：奇迹" class="headerlink" title="024：奇迹"></a>024：奇迹</h4><blockquote><p>不可能事件的必然性</p></blockquote><p>1950年5月1日，内布拉斯加州的比特丽丝教堂唱诗班的15名成员原定于傍晚7点15分集 合排练，出于不同的原因他们全都迟到了。牧师一家迟到是因为妻子还得熨烫女儿的衣服； 一对夫妻不准时，是因为他们汽车的发动机启动不了；琴师本想早到半小时，可他晚饭后睡 着了……当天晚上7点25分教堂爆炸了，爆炸声全村都能听到。墙壁被炸飞了，屋顶当场坍塌 了。但爆炸时无人遇难，这真是一个奇迹。消防队队长认为爆炸是由煤气泄漏引起的，而唱 诗班的成员们坚信他们接收到了 一个上帝的信号。这到底是上帝之手还是巧合呢？</p><p>这种故事发生的可能性有多大呢？瑞士心理学家荣格认为，这是一种未知力量的作用， 他称之为同步性。一个思维清晰的人会如何对待这种故事呢？他会拿起一张纸和一支笔。我们就以教堂的爆炸为例吧。请你画出4个格子，填入4种可能的情况。第一格是已经发生的事 实：“唱诗班迟到，教堂爆炸。”但还有其他3种可能性：“唱诗班迟到，教堂不爆炸”、 “唱诗班不迟到，教堂爆炸”、“唱诗班不迟到，教堂不爆炸”。请你估计每种情况发生的概率，并依次填入相应的格子里。你可以看出，光是最后一种情况发生的概率就很大：每天，在数百万座教堂里，都有一个唱诗班在约定的时间排练，而教堂没有爆炸。教堂爆炸的 故事顿时就没有那么不可思议了。相反，如果数百万座教堂在一个世纪里都没有爆炸过，这 倒是不可思议的。因此这件事跟上帝之手无关。</p><p>结论：不可思议的意外事件就是虽然少见但完全可能发生的事件。它们的发生没什么可吃惊的，如果它们从不发生，那才令人感到意外。</p><p>个人理解：遇到不可思议的事情，可以从概率上分析背后的逻辑，做到理性思考。</p><h4 id="025：团体迷思"><a href="#025：团体迷思" class="headerlink" title="025：团体迷思"></a>025：团体迷思</h4><blockquote><p>共识为什么有可能是危险的</p></blockquote><p>团体迷思是从众心理的一种特殊情况。你曾经在某次会议上克制自己，没有说出你的意见吗？肯定有过。你一言不发，点头同意种种动议，毕竟你不想做个(永远的）“捣蛋鬼”。另外你对你的异议也许也没有把握，其他人也不傻啊，他们全都意见一致，那就不吭声吧。如果人人都这样做，就会出现团体迷思。</p><p>结论：如果你是一个智囊团的成员，无论何时，你都要讲出你的看法——哪怕这看法不 是很中听。你要仔细考虑没有讲出的意见，必要时要甘冒被隔离在温暖团体之外的风险。如果你领导着一支团队，请你指定某人唱反调。他将不是团队里最受欢迎的人，但也许是最重要的人。</p><p>个人理解：如果是团队成员的话，要根据实际情况发表自己不同的看法。如果是团队的领导就要多思考和分析那个唱反调说的是否是有道理的，并要鼓励他。</p>]]></content>
<summary type="html"><h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote>
<p>记录一下阅读该书籍的理解与感想。</p>
</blockquote>
<h4 id="介绍"><a href="#介绍" class="headerlink" title="介绍"></a>介绍</h4><p>该书是通过通过 52 个简短的小文章深入浅出地讲解了一些常见的思维陷阱。使人们犯错更少，离成功更近。</p>
<h4 id="001：幸存偏误"><a href="#001：幸存偏误" class="headerlink" title="001：幸存偏误"></a>001：幸存偏误</h4><blockquote>
<p>为什么你该去逛逛墓地</p>
</blockquote>
<p>幸存偏误是指：由于日常生活中更容易看到成功、看不到失败、你会系统性的高估成功的希望。不了解现实的你对成功抱有一种幻想，认识不到成功的概率有多微弱。</p>
<p>幸存偏误意味着：你系统性地高估了成功概率。</p>
<p>解决办法：尽可能常去逛逛曾经大有希望的项目、投资和事业的墓地。这样的散步虽然伤感，但对你是有好处的。</p>
<p>个人理解：做事情或者项目前不仅仅要关注成功的案例也要多收集和研究失败的案例，对全部案例分析后再决定要不要做，避免陷入幸存偏误中。</p></summary>
<category term="book" scheme="http://tungsing.cc/categories/book/"/>
<category term="book" scheme="http://tungsing.cc/tags/book/"/>
<category term="思考" scheme="http://tungsing.cc/tags/%E6%80%9D%E8%80%83/"/>
<category term="逻辑" scheme="http://tungsing.cc/tags/%E9%80%BB%E8%BE%91/"/>
</entry>
<entry>
<title>开发管理系统脚手架</title>
<link href="http://tungsing.cc/2020/05/26/framework/scaffold/"/>
<id>http://tungsing.cc/2020/05/26/framework/scaffold/</id>
<published>2020-05-26T07:53:42.247Z</published>
<updated>2023-03-22T08:48:36.221Z</updated>
<content type="html"><![CDATA[<h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote><p>为了方便快速开发管理系统和练习技术，开发了一套前后端分离的脚手架项目</p></blockquote><h4 id="技术栈"><a href="#技术栈" class="headerlink" title="技术栈"></a>技术栈</h4><p>前端</p><table><thead><tr><th>技术</th><th>版本</th><th>描述</th></tr></thead><tbody><tr><td>node</td><td>10.0 +</td><td>LTS版本</td></tr><tr><td>npm</td><td>6.0 +</td><td></td></tr><tr><td>vue</td><td>2.6+</td><td></td></tr><tr><td>vue-cli</td><td>3.6</td><td></td></tr><tr><td>element-ui</td><td>2.7+</td><td></td></tr><tr><td>vue-element-template</td><td></td><td>开源基础模板</td></tr></tbody></table><p>后端</p><table><thead><tr><th>技术</th><th>版本</th><th>描述</th></tr></thead><tbody><tr><td>jdk</td><td>8+</td><td></td></tr><tr><td>maven</td><td>3.5 +</td><td></td></tr><tr><td>spring boot</td><td>2.1.4.RELEASE</td><td></td></tr><tr><td>mybatis plus</td><td>3.3.0</td><td></td></tr><tr><td>easyexcel</td><td>2.1.7</td><td></td></tr><tr><td>mysql</td><td>8+</td><td>数据库</td></tr></tbody></table><span id="more"></span><h4 id="工程目录"><a href="#工程目录" class="headerlink" title="工程目录"></a>工程目录</h4><p>前端 </p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br></pre></td><td class="code"><pre><span class="line">├── build // 构建相关</span><br><span class="line">├── mock // 项目mock 模拟数据</span><br><span class="line">├── config // 配置相关</span><br><span class="line">├── public // 静态资源</span><br><span class="line">│ │── favicon.ico // favicon图标</span><br><span class="line">│ └── index.html // html模板</span><br><span class="line">├── src // 源代码</span><br><span class="line">│ ├── api // 所有请求</span><br><span class="line">│ ├── assets // 主题 字体等静态资源</span><br><span class="line">│ ├── components // 全局公用组件</span><br><span class="line">│ ├── directive // 全局指令</span><br><span class="line">│ ├── filters // 全局 filter</span><br><span class="line">│ ├── icons // 项目所有 svg icons</span><br><span class="line">│ ├── lang // 国际化 language</span><br><span class="line">│ ├── layout // 全局 layout</span><br><span class="line">│ ├── router // 路由</span><br><span class="line">│ ├── store // 全局 store管理</span><br><span class="line">│ ├── styles // 全局样式</span><br><span class="line">│ ├── utils // 全局公用方法</span><br><span class="line">│ ├── vendor // 公用vendor</span><br><span class="line">│ ├── views // views 所有页面</span><br><span class="line">│ ├── App.vue // 入口页面</span><br><span class="line">│ ├── main.js // 入口文件 加载组件 初始化等</span><br><span class="line">│ └── permission.js // 权限管理</span><br><span class="line">├── tests // 测试</span><br><span class="line">├── .env.xxx // 环境变量配置</span><br><span class="line">├── .eslintrc.js // eslint 配置项</span><br><span class="line">├── .babelrc // babel-loader 配置</span><br><span class="line">├── .travis.yml // 自动化CI配置</span><br><span class="line">├── vue.config.js // vue-cli 配置</span><br><span class="line">├── postcss.config.js // postcss 配置</span><br><span class="line">└── package.json // package.json</span><br></pre></td></tr></table></figure><p>后端 </p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">├── grass-parent // 父工程</span><br><span class="line">├── grass-core // 核心工程</span><br><span class="line">│ ├── common // 公共代码包</span><br><span class="line">│ ├── config // 配置代码包</span><br><span class="line">│ ├── utils // 公共工具包</span><br><span class="line">│ └── GrassApplicationRunner.java // 核心启动类</span><br><span class="line">├── grass-autoconfigure // 自动配置工程</span><br><span class="line">├── grass-starter // starter工程，供其他工程依赖</span><br><span class="line">├── ecm // 依赖核心工程的标准spring boot工程</span><br><span class="line">│ ├── operation // 运维相关功能</span><br><span class="line">│ ├── system // 系统管理功能</span><br><span class="line">└── └── ECMSpringBootApplication.java // 工程启动类</span><br></pre></td></tr></table></figure><h4 id="效果"><a href="#效果" class="headerlink" title="效果"></a>效果</h4><p>展示几张效果图，有时间搭建一个演示环境</p><p>首页<br><img src="/images/framework/home.png"></p><p>组织机构管理<br><img src="/images/framework/org.png"></p><p>运维日志管理<br><img src="/images/framework/login.png"></p><p>前端封装组件的例子<br><img src="/images/framework/example.png"></p><p>后端工程<br><img src="/images/framework/admin.png"></p><h4 id="未来"><a href="#未来" class="headerlink" title="未来"></a>未来</h4><p>前端</p><ol><li>前端支持容器化</li><li>支持Tab方式</li></ol><p>后端</p><ol><li>集成到之前搭建的Spirng cloud 环境中</li></ol><h4 id="结束语"><a href="#结束语" class="headerlink" title="结束语"></a>结束语</h4><p>关于工程前缀“grass”来源，本来是打算用“芨芨草”英文来做工程前缀的，但它（Achnatherum splendens (Trin. ) Nevski ）的英文名称太长，不太适合。故取通用的“grass”作为工程前缀。</p>]]></content>
<summary type="html"><h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote>
<p>为了方便快速开发管理系统和练习技术，开发了一套前后端分离的脚手架项目</p>
</blockquote>
<h4 id="技术栈"><a href="#技术栈" class="headerlink" title="技术栈"></a>技术栈</h4><p>前端</p>
<table>
<thead>
<tr>
<th>技术</th>
<th>版本</th>
<th>描述</th>
</tr>
</thead>
<tbody><tr>
<td>node</td>
<td>10.0 +</td>
<td>LTS版本</td>
</tr>
<tr>
<td>npm</td>
<td>6.0 +</td>
<td></td>
</tr>
<tr>
<td>vue</td>
<td>2.6+</td>
<td></td>
</tr>
<tr>
<td>vue-cli</td>
<td>3.6</td>
<td></td>
</tr>
<tr>
<td>element-ui</td>
<td>2.7+</td>
<td></td>
</tr>
<tr>
<td>vue-element-template</td>
<td></td>
<td>开源基础模板</td>
</tr>
</tbody></table>
<p>后端</p>
<table>
<thead>
<tr>
<th>技术</th>
<th>版本</th>
<th>描述</th>
</tr>
</thead>
<tbody><tr>
<td>jdk</td>
<td>8+</td>
<td></td>
</tr>
<tr>
<td>maven</td>
<td>3.5 +</td>
<td></td>
</tr>
<tr>
<td>spring boot</td>
<td>2.1.4.RELEASE</td>
<td></td>
</tr>
<tr>
<td>mybatis plus</td>
<td>3.3.0</td>
<td></td>
</tr>
<tr>
<td>easyexcel</td>
<td>2.1.7</td>
<td></td>
</tr>
<tr>
<td>mysql</td>
<td>8+</td>
<td>数据库</td>
</tr>
</tbody></table></summary>
<category term="java" scheme="http://tungsing.cc/categories/java/"/>
<category term="web" scheme="http://tungsing.cc/categories/web/"/>
<category term="framework" scheme="http://tungsing.cc/categories/framework/"/>
<category term="java" scheme="http://tungsing.cc/tags/java/"/>
<category term="vue" scheme="http://tungsing.cc/tags/vue/"/>
<category term="spring boot" scheme="http://tungsing.cc/tags/spring-boot/"/>
<category term="framework" scheme="http://tungsing.cc/tags/framework/"/>
</entry>
<entry>
<title>常见限流算法</title>
<link href="http://tungsing.cc/2020/01/19/java/limiter/"/>
<id>http://tungsing.cc/2020/01/19/java/limiter/</id>
<published>2020-01-19T06:59:57.571Z</published>
<updated>2023-03-22T08:47:21.103Z</updated>
<content type="html"><![CDATA[<h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote><p>工作中遇到了限制请求数的场景，学习和记录一下解决办法。</p></blockquote><h4 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h4><p>在开发高并发系统时有三把利器用来保护系统：<strong>缓存</strong>、<strong>降级</strong>和<strong>限流</strong><br><strong>缓存</strong>:缓存的目的是提升系统访问速度和增大系统处理容量<br><strong>降级</strong>:降级是当服务器压力剧增的情况下，根据当前业务情况及流量对一些服务和页面有策略的降级，以此释放服务器资源以保证核心任务的正常运行<br><strong>限流</strong>:限流的目的是通过对并发访问/请求进行限速，或者对一个时间窗口内的请求进行限速来保护系统，一旦达到限制速率则可以拒绝服务、排队或等待、降级等处理 </p><h4 id="限流"><a href="#限流" class="headerlink" title="限流"></a>限流</h4><p>常见的限流算法有计数器、令牌桶和漏桶算法</p><span id="more"></span><h4 id="计算器算法"><a href="#计算器算法" class="headerlink" title="计算器算法"></a>计算器算法</h4><p>计数器算法是限流算法里最简单也是最容易实现的一种算法<br>示例1：用AomicInteger来记录并发数</p><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">CountRateLimiter</span> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> AtomicInteger count = <span class="keyword">new</span> AtomicInteger(<span class="number">0</span>);</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">exec</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="keyword">if</span> (count.get() >= <span class="number">10</span>) {</span><br><span class="line"> System.out.println(<span class="string">"请求用户过多，请稍后在试！"</span> + System.currentTimeMillis() / <span class="number">1000</span>);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> count.incrementAndGet();</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 处理核心逻辑</span></span><br><span class="line"> TimeUnit.SECONDS.sleep(<span class="number">1</span>);</span><br><span class="line"> System.out.println(<span class="string">"--"</span> + System.currentTimeMillis() / <span class="number">1000</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> count.decrementAndGet();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>示例2：使用信号量来控制并发的次数</p><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">CountRateLimiterSmp</span> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> Semaphore semaphore = <span class="keyword">new</span> Semaphore(<span class="number">10</span>);</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">exec</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> semaphore.acquire();</span><br><span class="line"> <span class="comment">// 处理核心逻辑</span></span><br><span class="line"> TimeUnit.SECONDS.sleep(<span class="number">1</span>);</span><br><span class="line"> System.out.println(<span class="string">"--"</span> + System.currentTimeMillis() / <span class="number">1000</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> semaphore.release();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>用Atomic和Semaphore的最大区别为，如果是瞬时的高并发，可以使请求在阻塞队列中排队，而不是马上拒绝请求，从而达到一个流量削峰的目的。</p><h4 id="令牌桶算法"><a href="#令牌桶算法" class="headerlink" title="令牌桶算法"></a>令牌桶算法</h4><p><img src="/images/java/tokenBucket.png"></p><p>令牌桶算法概念如下： </p><ul><li>令牌以固定速率生成；</li><li>生成的令牌放入令牌桶中存放，如果令牌桶满了则多余的令牌会直接丢弃，当请求到达时，会尝试从令牌桶中取令牌，取到了令牌的请求可以执行；</li><li>如果桶空了，那么尝试取令牌的请求会被直接丢弃。</li></ul><p>令牌桶算法既能够将所有的请求平均分布到时间区间内，又能接受服务器能够承受范围内的突发请求，因此是目前使用较为广泛的一种限流算法。Google 的开源项目 guava 提供了 RateLimiter 类，实现了单点的令牌桶限流。为了学习,模拟一下该算法的实现</p><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 简单限流器-令牌桶算法<br/></span></span><br><span class="line"><span class="comment"> * 只需要记录一个下一令牌产生的时间，并动态更新它，就能够轻松完成限流功能</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">SimpleLimiter</span> </span>{</span><br><span class="line"> <span class="comment">// 当前令牌桶中的令牌数量</span></span><br><span class="line"> <span class="keyword">long</span> storedPermits = <span class="number">0</span>;</span><br><span class="line"> <span class="comment">// 令牌桶的容量</span></span><br><span class="line"> <span class="keyword">long</span> maxPermits = <span class="number">3</span>;</span><br><span class="line"> <span class="comment">// 下一令牌产生时间</span></span><br><span class="line"> <span class="keyword">long</span> next = System.nanoTime();</span><br><span class="line"> <span class="comment">// 发放令牌间隔：纳秒</span></span><br><span class="line"> <span class="keyword">long</span> interval = <span class="number">1000_000_000</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 请求时间在下一令牌产生时间之后,则</span></span><br><span class="line"> <span class="comment">// 1.重新计算令牌桶中的令牌数</span></span><br><span class="line"> <span class="comment">// 2.将下一个令牌发放时间重置为当前时间</span></span><br><span class="line"> <span class="function"><span class="keyword">void</span> <span class="title">resync</span><span class="params">(<span class="keyword">long</span> now)</span> </span>{</span><br><span class="line"> <span class="keyword">if</span> (now > next) {</span><br><span class="line"> <span class="comment">// 新产生的令牌数</span></span><br><span class="line"> <span class="keyword">long</span> newPermits = (now - next) / interval;</span><br><span class="line"> <span class="comment">// 新令牌增加到令牌桶</span></span><br><span class="line"> storedPermits = Math.min(maxPermits, storedPermits + newPermits);</span><br><span class="line"> <span class="comment">// 将下一个令牌发放时间重置为当前时间</span></span><br><span class="line"> next = now;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 预占令牌，返回能获取令牌的时间</span></span><br><span class="line"> <span class="function"><span class="keyword">synchronized</span> <span class="keyword">long</span> <span class="title">reserve</span><span class="params">(<span class="keyword">long</span> now)</span> </span>{</span><br><span class="line"> resync(now);</span><br><span class="line"> <span class="comment">// 能够获取令牌的时间</span></span><br><span class="line"> <span class="keyword">long</span> at = next;</span><br><span class="line"> <span class="comment">// 令牌桶中能提供的令牌</span></span><br><span class="line"> <span class="keyword">long</span> fb = Math.min(<span class="number">1</span>, storedPermits);</span><br><span class="line"> <span class="comment">// 令牌净需求：首先减掉令牌桶中的令牌</span></span><br><span class="line"> <span class="keyword">long</span> nr = <span class="number">1</span> - fb;</span><br><span class="line"> <span class="comment">// 重新计算下一令牌产生时间</span></span><br><span class="line"> next = next + nr * interval;</span><br><span class="line"> <span class="comment">// 重新计算令牌桶中的令牌</span></span><br><span class="line"> <span class="keyword">this</span>.storedPermits -= fb;</span><br><span class="line"> <span class="keyword">return</span> at;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 申请令牌</span></span><br><span class="line"> <span class="function"><span class="keyword">void</span> <span class="title">acquire</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="comment">// 申请令牌的时间</span></span><br><span class="line"> <span class="keyword">long</span> now = System.nanoTime();</span><br><span class="line"> <span class="comment">// 预占令牌</span></span><br><span class="line"> <span class="keyword">long</span> at = reserve(now);</span><br><span class="line"> <span class="keyword">long</span> waitTime = Math.max(at - now, <span class="number">0</span>);</span><br><span class="line"> <span class="comment">// 按照条件等待</span></span><br><span class="line"> <span class="keyword">if</span> (waitTime > <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> TimeUnit.NANOSECONDS.sleep(waitTime);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>测试代码</p><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br></pre></td><td class="code"><pre><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">main</span><span class="params">(String[] args)</span> </span>{</span><br><span class="line"> <span class="comment">// 限流器流速</span></span><br><span class="line"> SimpleLimiter limiter = <span class="keyword">new</span> SimpleLimiter();</span><br><span class="line"> <span class="comment">// 执行任务的线程池</span></span><br><span class="line"> ExecutorService es = Executors.newFixedThreadPool(<span class="number">20</span>);</span><br><span class="line"> <span class="comment">// 记录上一次执行时间</span></span><br><span class="line"> <span class="keyword">long</span> prev = System.nanoTime();</span><br><span class="line"> <span class="comment">// 测试执行20次</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="keyword">int</span> i = <span class="number">0</span>; i < <span class="number">20</span>; i++) {</span><br><span class="line"> <span class="comment">// 限流器限流</span></span><br><span class="line"> limiter.acquire();</span><br><span class="line"> <span class="comment">// 提交任务异步执行</span></span><br><span class="line"> <span class="keyword">long</span> cur = System.nanoTime();</span><br><span class="line"> es.execute(<span class="keyword">new</span> Task(prev, cur, i));</span><br><span class="line"> prev = cur;</span><br><span class="line"> }</span><br><span class="line"> es.shutdown();</span><br><span class="line"> }</span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Task</span> <span class="keyword">implements</span> <span class="title">Runnable</span> </span>{</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">long</span> prev;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">long</span> cur;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">int</span> index;</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="title">Task</span><span class="params">(<span class="keyword">long</span> prev, <span class="keyword">long</span> cur, <span class="keyword">int</span> index)</span> </span>{</span><br><span class="line"> <span class="keyword">this</span>.prev = prev;</span><br><span class="line"> <span class="keyword">this</span>.cur = cur;</span><br><span class="line"> <span class="keyword">this</span>.index = index;</span><br><span class="line"> }</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">run</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="comment">// 打印时间间隔:毫秒</span></span><br><span class="line"> System.out.println(index + <span class="string">" : "</span> + (cur - prev) / <span class="number">1000_000</span>);</span><br><span class="line"> }</span><br><span class="line">} </span><br></pre></td></tr></table></figure><h4 id="漏桶算法"><a href="#漏桶算法" class="headerlink" title="漏桶算法"></a>漏桶算法</h4><p><img src="/images/java/leakyBucket.png"></p><p>漏桶算法概念如下：</p><ul><li>将每个请求视作 “ 水滴 “ 放入 “ 漏桶 “ 进行存储；</li><li>“漏桶 “ 以固定速率向外 “ 漏 “ 出请求来执行如果 “ 漏桶 “ 空了则停止 “ 漏水”；</li><li>如果 “ 漏桶 “ 满了则多余的 “ 水滴 “ 会被直接丢弃。</li></ul><p>漏桶算法多使用队列实现，服务的请求会存到队列中，服务的提供方则按照固定的速率从队列中取出请求并执行，过多的请求则放在队列中排队或直接拒绝。</p><p>缺点：漏桶算法的缺陷也很明显，当短时间内有大量的突发请求时，即便此时服务器没有任何负载，每个请求也都得在队列中等待一段时间才能被响应。</p>]]></content>
<summary type="html"><h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote>
<p>工作中遇到了限制请求数的场景，学习和记录一下解决办法。</p>
</blockquote>
<h4 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h4><p>在开发高并发系统时有三把利器用来保护系统：<strong>缓存</strong>、<strong>降级</strong>和<strong>限流</strong><br><strong>缓存</strong>:缓存的目的是提升系统访问速度和增大系统处理容量<br><strong>降级</strong>:降级是当服务器压力剧增的情况下，根据当前业务情况及流量对一些服务和页面有策略的降级，以此释放服务器资源以保证核心任务的正常运行<br><strong>限流</strong>:限流的目的是通过对并发访问/请求进行限速，或者对一个时间窗口内的请求进行限速来保护系统，一旦达到限制速率则可以拒绝服务、排队或等待、降级等处理 </p>
<h4 id="限流"><a href="#限流" class="headerlink" title="限流"></a>限流</h4><p>常见的限流算法有计数器、令牌桶和漏桶算法</p></summary>
<category term="java" scheme="http://tungsing.cc/categories/java/"/>
<category term="algorithm" scheme="http://tungsing.cc/categories/algorithm/"/>
<category term="java" scheme="http://tungsing.cc/tags/java/"/>
<category term="algorithm" scheme="http://tungsing.cc/tags/algorithm/"/>
<category term="限流" scheme="http://tungsing.cc/tags/%E9%99%90%E6%B5%81/"/>
</entry>
<entry>
<title>volitile 关键字</title>
<link href="http://tungsing.cc/2019/05/26/java/thread/chapter04/"/>
<id>http://tungsing.cc/2019/05/26/java/thread/chapter04/</id>
<published>2019-05-26T07:16:45.868Z</published>
<updated>2020-05-28T06:38:24.689Z</updated>
<content type="html"><![CDATA[<h3 id="常用方法"><a href="#常用方法" class="headerlink" title="常用方法"></a>常用方法</h3><p>volatile: 解决的是变量在多线程之间的可见性<br>synchronized: 解决的是多线程之间访问资源的同步性</p><p>volatile轻量级的线性间通信</p>]]></content>
<summary type="html"><h3 id="常用方法"><a href="#常用方法" class="headerlink" title="常用方法"></a>常用方法</h3><p>volatile: 解决的是变量在多线程之间的可见性<br>synchronized: 解决的是多线程之间访问资源的同步性<</summary>
<category term="java" scheme="http://tungsing.cc/categories/java/"/>
<category term="java" scheme="http://tungsing.cc/tags/java/"/>
<category term="thread" scheme="http://tungsing.cc/tags/thread/"/>
<category term="多线程" scheme="http://tungsing.cc/tags/%E5%A4%9A%E7%BA%BF%E7%A8%8B/"/>
</entry>
<entry>
<title>自己实现本地缓存</title>
<link href="http://tungsing.cc/2019/05/17/cache/local_cache/"/>
<id>http://tungsing.cc/2019/05/17/cache/local_cache/</id>
<published>2019-05-17T14:27:00.175Z</published>
<updated>2023-03-22T08:45:29.496Z</updated>
<content type="html"><![CDATA[<h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote><p>在扩展框架中的缓存组件相关功能时浏览了本地缓存的实现，其核心是LUR算法，记录下LUR的相关内容加深记忆和理解</p></blockquote><h4 id="环境"><a href="#环境" class="headerlink" title="环境"></a>环境</h4><ul><li>windows 10</li><li>jdk7</li></ul><h4 id="LUR"><a href="#LUR" class="headerlink" title="LUR"></a>LUR</h4><p>LRU全称是Least Recently Used，即<strong>最近最久未使用</strong>的意思。 LRU算法的设计原则是：如果一个数据在最近一段时间没有被访问到，那么在将来它被访问的可能性也很小。也就是说，当限定的空间已存满数据时，应当把最久没有被访问到的数据淘汰。</p><span id="more"></span><h4 id="思路分析"><a href="#思路分析" class="headerlink" title="思路分析"></a>思路分析</h4><p>对cache的基本操作有：插入(insert)、替换(replace)、查找(lookup)<br>为了能够快速删除最久没有访问的数据项和插入最新的数据项，我们使用<strong>双向链表</strong>连接Cache中的数据项，并且保证链表维持数据项从最近访问到最旧访问的顺序。</p><ul><li>插入：当Cache未满时，新的数据项只需插到双链表头部即可。时间复杂度为O(1).</li><li>替换：当Cache已满时，将新的数据项插到双链表头部，并删除双链表的尾结点即可。时间复杂度为O(1).</li><li>查找：每次数据项被查询到时，都将此数据项移动到链表头部。</li></ul><p>经过分析，我们知道使用双向链表可以保证插入和替换的时间复杂度是O(1)，但查询的时间复杂度是O(n)，因为需要对双链表进行遍历。为了让查找效率也达到O(1)，很自然的会想到使用 hash table 。</p><h4 id="代码实现"><a href="#代码实现" class="headerlink" title="代码实现"></a>代码实现</h4><p>从上述分析可知，我们需要使用两种数据结构：</p><ol><li>双向链表(Doubly Linked List)</li><li>哈希表(Hash Table)</li></ol><p>Java中的LinkedHashMap正好满足这两种数据结构，直接用LinkedHashMap来实现(框架代码就是基于LinkedHashMap实现的)</p><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">LRUCache</span><<span class="title">K</span>, <span class="title">V</span>> <span class="keyword">extends</span> <span class="title">LinkedHashMap</span><<span class="title">K</span>, <span class="title">V</span>> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="keyword">long</span> serialVersionUID = -<span class="number">1456882507664137630L</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">//缓存的最大容量</span></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="keyword">int</span> MAX_CACHE_SIZE;</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="title">LRUCache</span><span class="params">(<span class="keyword">int</span> cacheSize)</span> </span>{</span><br><span class="line"> <span class="comment">// true:按访问排序</span></span><br><span class="line"> <span class="comment">// 初始值必须是2 的N次方，最大容量2 的30次方</span></span><br><span class="line"> <span class="comment">// 初始值16</span></span><br><span class="line"> <span class="keyword">super</span>(<span class="number">1</span> << <span class="number">4</span>, <span class="number">0.75F</span>, <span class="keyword">true</span>);</span><br><span class="line"> MAX_CACHE_SIZE = cacheSize;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="function"><span class="keyword">protected</span> <span class="keyword">boolean</span> <span class="title">removeEldestEntry</span><span class="params">(Entry<K, V> eldest)</span> </span>{</span><br><span class="line"> <span class="keyword">return</span> size() > MAX_CACHE_SIZE;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>核心实现只需要继承LinkedHashMap，扩展它的removeEldestEntry方法即可，在多线程环境使用时使用 Collections.synchronizedMap()方法实现线程安全操作。框架中的细节代码就很多了。</p><h4 id="简单测试"><a href="#简单测试" class="headerlink" title="简单测试"></a>简单测试</h4><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">main</span><span class="params">(String[] args)</span> </span>{</span><br><span class="line"> LRUCache<Object, Object> cache = <span class="keyword">new</span> LRUCache<Object, Object>(<span class="number">8</span>);</span><br><span class="line"> cache.put(<span class="string">"0"</span>, <span class="string">"0"</span>);</span><br><span class="line"> cache.put(<span class="string">"1"</span>, <span class="string">"1"</span>);</span><br><span class="line"> cache.put(<span class="string">"2"</span>, <span class="string">"2"</span>);</span><br><span class="line"> cache.put(<span class="string">"3"</span>, <span class="string">"3"</span>);</span><br><span class="line"> cache.put(<span class="string">"4"</span>, <span class="string">"4"</span>);</span><br><span class="line"> cache.put(<span class="string">"5"</span>, <span class="string">"5"</span>);</span><br><span class="line"> cache.put(<span class="string">"6"</span>, <span class="string">"6"</span>);</span><br><span class="line"> cache.put(<span class="string">"7"</span>, <span class="string">"7"</span>);</span><br><span class="line"></span><br><span class="line"> cache.get(<span class="string">"6"</span>);</span><br><span class="line"> cache.get(<span class="string">"5"</span>);</span><br><span class="line"> cache.put(<span class="string">"8"</span>, <span class="string">"8"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (Entry<Object, Object> entry : cache.entrySet()) {</span><br><span class="line"> System.out.println(entry.getKey() + <span class="string">":"</span> + entry.getValue());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>输出结果：</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">1:1</span><br><span class="line">2:2</span><br><span class="line">3:3</span><br><span class="line">4:4</span><br><span class="line">7:7</span><br><span class="line">6:6</span><br><span class="line">5:5</span><br><span class="line">8:8</span><br></pre></td></tr></table></figure><p>观察结果当超出最大容量自动删除了最久未使用的数据，访问过的数据排到了链表前端，符合预期</p><h4 id="结语"><a href="#结语" class="headerlink" title="结语"></a>结语</h4><p>了解原理后不管是解决问题还是重复造轮子都会得心应手。但不推荐重复造轮子，google guava cache 就是基于LUR算法实现的一个基础类库。</p><h4 id="参考资料"><a href="#参考资料" class="headerlink" title="参考资料"></a>参考资料</h4><p>1.<a href="https://songlee24.github.io/2015/05/10/design-LRU-Cache/">设计并实现一个LRU Cache</a></p>]]></content>
<summary type="html"><h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><blockquote>
<p>在扩展框架中的缓存组件相关功能时浏览了本地缓存的实现，其核心是LUR算法，记录下LUR的相关内容加深记忆和理解</p>
</blockquote>
<h4 id="环境"><a href="#环境" class="headerlink" title="环境"></a>环境</h4><ul>
<li>windows 10</li>
<li>jdk7</li>
</ul>
<h4 id="LUR"><a href="#LUR" class="headerlink" title="LUR"></a>LUR</h4><p>LRU全称是Least Recently Used，即<strong>最近最久未使用</strong>的意思。 LRU算法的设计原则是：如果一个数据在最近一段时间没有被访问到，那么在将来它被访问的可能性也很小。也就是说，当限定的空间已存满数据时，应当把最久没有被访问到的数据淘汰。</p></summary>
<category term="cache" scheme="http://tungsing.cc/categories/cache/"/>
<category term="java" scheme="http://tungsing.cc/categories/java/"/>
<category term="algorithm" scheme="http://tungsing.cc/categories/algorithm/"/>
<category term="java" scheme="http://tungsing.cc/tags/java/"/>
<category term="cache" scheme="http://tungsing.cc/tags/cache/"/>
<category term="algorithm" scheme="http://tungsing.cc/tags/algorithm/"/>
</entry>
</feed>