Skip to content

Commit c7ecaac

Browse files
authored
2022 oceanbase miniob competition (#104)
2022 oceanbase competition
1 parent d63a39f commit c7ecaac

30 files changed

+1905
-94
lines changed

README.md

+24-39
Original file line numberDiff line numberDiff line change
@@ -1,50 +1,35 @@
1-
# Introduction
2-
miniob 是 OceanBase与华中科技大学联合开发的、面向"零"基础数据库内核知识同学的一门数据库实现入门教程实践工具。
3-
miniob设计的目标是让不熟悉数据库设计和实现的同学能够快速的了解与深入学习数据库内核,期望通过相关训练之后,能够对各个数据库内核模块的功能与它们之间的关联有所了解,并能够在
4-
使用数据库时,设计出高效的SQL。面向的对象主要是在校学生,并且诸多模块做了简化,比如不考虑并发操作。
5-
注意:此代码仅供学习使用,不考虑任何安全特性。
1+
# 介绍
2+
miniob 是 OceanBase 与华中科技大学联合开发的、面向"零"基础同学的数据库入门实践工具。
3+
miniob 设计的目标是让同学们快速了解数据库并深入学习数据库内核,期望通过相关训练之后,能够对数据库内核各个模块的功能及其关联有所了解,并能够在
4+
使用数据库时,设计出高效的 SQL 。miniob 面向的对象主要是在校学生,并且诸多模块都做了简化,比如不考虑并发操作。
65

7-
[GitHub 首页](https://github.com/oceanbase/miniob)
8-
9-
# 如何开发
10-
## 搭建开发环境
11-
有多种方式搭建开发环境,可以直接在本地安装一些三方依赖,或者使用Docker。如果使用的是Windows,我们建议使用Docker来开发。
12-
13-
### 搭建本地开发环境
14-
直接在本地搭建开发环境,可以参考 [how_to_build](docs/how_to_build.md)
15-
16-
### 使用Docker开发
17-
18-
请参考 [如何使用Docker开发MiniOB](docs/how-to-dev-using-docker.md)
6+
(注意:此代码仅供学习使用,不考虑任何安全特性。)
197

20-
### Windows上开发MiniOB
21-
22-
[如何在Windows上使用Docker开发miniob](docs/how_to_dev_miniob_by_docker_on_windows.md)
23-
24-
## 词法语法解析开发环境
25-
26-
如果已经在处理一些SQL词法语法解析相关的问题,请参考 [MiniOB 词法语法解析开发与测试](docs/miniob-sql-parser.md)
27-
Docker 环境已经预安装了相关的组件。
28-
29-
# 数据库管理系统实现基础讲义
30-
由华中科技大学谢美意和左琼老师联合编撰数据库管理系统实现教材。参考 [数据库管理系统实现基础讲义](docs/lectures/index.md)
8+
[GitHub 首页](https://github.com/oceanbase/miniob)
319

32-
# miniob 介绍
33-
[miniob代码架构框架设计和说明](docs/miniob-introduction.md)
10+
# 1. 题目说明
11+
[miniob 题目描述](docs/miniob_topics.md)
3412

35-
# miniob 训练
36-
我们为MiniOB设计了配套的训练题目,大家可以在 [MiniOB 训练营](https://open.oceanbase.com/train?questionId=200001) 上进行提交测试。
13+
# 2. 开发指南
14+
## 搭建开发环境
15+
1. [本地配置gcc环境](docs/how_to_build.md)
16+
2. [使用Docker开发](docs/how-to-dev-using-docker.md)
17+
3. [在Windows上使用Docker](docs/how_to_dev_miniob_by_docker_on_windows.md)
3718

38-
[miniob 题目描述](docs/miniob_topics.md)
19+
## 词法、语法解析
20+
请参考 [miniob 词法语法解析开发与测试](docs/miniob-sql-parser.md)
3921

40-
为了满足训练营或比赛测试要求,代码的输出需要满足一定要求,请参考 [MiniOB 输出约定](docs/miniob-output-convention.md)。一般情况下,不需要专门来看这篇文档,但是如果你的测试总是不正确,建议对照一下输出约定。
22+
# 3. 提交测试
23+
题目完成并通过自测后,大家可以在 [miniob 训练营](https://open.oceanbase.com/train?questionId=200001) 上提交代码进行测试。
4124

42-
# miniob 实现解析
25+
客户端输出需要满足一定要求,如果你的测试结果不符合预期,请参考 [miniob 输出约定](docs/miniob-output-convention.md)
4326

44-
[miniob-date 实现解析](https://oceanbase-partner.github.io/lectures-on-dbms-implementation/miniob-date-implementation.html)
27+
# 4. 数据库管理系统实现基础讲义
28+
由华中科技大学谢美意和左琼老师联合编撰的数据库管理系统实现教材:[《数据库管理系统实现基础讲义》](docs/lectures/index.md)
4529

46-
[miniob drop-table 实现解析](https://oceanbase-partner.github.io/lectures-on-dbms-implementation/miniob-drop-table-implementation.html)
30+
# 5. miniob 介绍
31+
[miniob 源码解析视频](https://open.oceanbase.com/activities/4921877)
4732

48-
[miniob select-tables 实现解析](https://oceanbase-partner.github.io/lectures-on-dbms-implementation/miniob-select-tables-implementation.html)
33+
[miniob 源码解析文档](https://www.oceanbase.com/docs/community-developer-quickstart-10000000000627363)
4934

50-
[miniob 调试篇](https://oceanbase-partner.github.io/lectures-on-dbms-implementation/miniob-how-to-debug.html)
35+
(资料持续整理中,请大家自行查阅标题为“MiniOB...”的视频或文档)

deps/common/lang/defer.h

+5-2
Original file line numberDiff line numberDiff line change
@@ -37,5 +37,8 @@ class DeferHelper
3737

3838
} // namespace common
3939

40-
#define DERFER_NAME(suffix) defer_helper_##suffix
41-
#define DEFER(callback) common::DeferHelper DERFER_NAME(__LINE__)(callback)
40+
#define AA(B, C) B##C
41+
42+
#define BB(B, C) AA(B,C)
43+
44+
#define DEFER(callback) common::DeferHelper BB(defer_helper_, __LINE__)(callback)

src/observer/rc.cpp

+16
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ const char *strrc(RC rc)
5353
RC_CASE_STRING(FORMAT);
5454
RC_CASE_STRING(RANGE);
5555
RC_CASE_STRING(NOTADB);
56+
RC_CASE_STRING(LOGBUF);
5657
RC_CASE_STRING(NOTICE);
5758

5859
RC_CASE_STRING(BUFFERPOOL_EXIST);
@@ -174,8 +175,23 @@ const char *strrc(RC rc)
174175
RC_CASE_STRING(NOTICE_RECOVER_ROLLBACK);
175176
RC_CASE_STRING(NOTICE_AUTOINDEX);
176177

178+
RC_CASE_STRING(FILE_EXIST);
179+
RC_CASE_STRING(FILE_NOT_EXIST);
180+
RC_CASE_STRING(FILE_NAME);
181+
RC_CASE_STRING(FILE_BOUND);
182+
RC_CASE_STRING(FILE_CREATE);
183+
RC_CASE_STRING(FILE_OPEN);
184+
RC_CASE_STRING(FILE_NOT_OPENED);
185+
RC_CASE_STRING(FILE_CLOSE);
186+
RC_CASE_STRING(FILE_REMOVE);
187+
RC_CASE_STRING(FILE_SEEK);
188+
RC_CASE_STRING(FILE_READ);
189+
RC_CASE_STRING(FILE_WRITE);
190+
177191
RC_CASE_STRING(AUTH_USER);
178192

193+
RC_CASE_STRING(LOGBUF_FULL);
194+
RC_CASE_STRING(LOGBUF_EMPTY);
179195
default: {
180196
return "UNKNOWN";
181197
}

src/observer/rc.h

+40
Original file line numberDiff line numberDiff line change
@@ -166,6 +166,26 @@ enum RCAuth {
166166
USER = 1,
167167
};
168168

169+
enum RCFILE {
170+
F_EXIST = 1,
171+
F_NOT_EXIST,
172+
F_NAME,
173+
F_BOUND,
174+
F_CREATE,
175+
F_OPEN,
176+
F_NOT_OPENED,
177+
F_CLOSE,
178+
F_REMOVE,
179+
F_SEEK,
180+
F_READ,
181+
F_WRITE,
182+
};
183+
184+
enum RCLOGBUF {
185+
LB_FULL = 1,
186+
LB_EMPTY,
187+
};
188+
169189
enum RC {
170190

171191
SUCCESS = 0, /* Successful result */
@@ -201,6 +221,8 @@ enum RC {
201221
FORMAT, /* Not used */
202222
RANGE, /* 2nd parameter to bind out of range */
203223
NOTADB, /* File opened that is not a database file */
224+
FILE_ERROR, /* File error */
225+
LOGBUF, /* clog buffer error */
204226
NOTICE = 100, /* Notifications from log() */
205227

206228
/* buffer pool part */
@@ -337,8 +359,26 @@ enum RC {
337359
NOTICE_RECOVER_ROLLBACK = (NOTICE | (RCNotice::RECOVER_ROLLBACK << 8)),
338360
NOTICE_AUTOINDEX = (NOTICE | (RCNotice::AUTOINDEX << 8)),
339361

362+
/* file part */
363+
FILE_EXIST = ( FILE_ERROR | (RCFILE::F_EXIST << 8)),
364+
FILE_NOT_EXIST = ( FILE_ERROR | (RCFILE::F_NOT_EXIST << 8)),
365+
FILE_NAME = ( FILE_ERROR | (RCFILE::F_NAME << 8)),
366+
FILE_BOUND = ( FILE_ERROR | (RCFILE::F_BOUND << 8)),
367+
FILE_CREATE = ( FILE_ERROR | (RCFILE::F_CREATE << 8)),
368+
FILE_OPEN = ( FILE_ERROR | (RCFILE::F_OPEN << 8)),
369+
FILE_NOT_OPENED = ( FILE_ERROR | (RCFILE::F_NOT_OPENED << 8)),
370+
FILE_CLOSE = ( FILE_ERROR | (RCFILE::F_CLOSE << 8)),
371+
FILE_REMOVE = ( FILE_ERROR | (RCFILE::F_REMOVE << 8)),
372+
FILE_SEEK = ( FILE_ERROR | (RCFILE::F_SEEK << 8)),
373+
FILE_READ = ( FILE_ERROR | (RCFILE::F_READ << 8)),
374+
FILE_WRITE = ( FILE_ERROR | (RCFILE::F_WRITE << 8)),
375+
340376
/* auth part*/
341377
AUTH_USER = (AUTH | (RCAuth::USER << 8)),
378+
379+
/* clog buffer part */
380+
LOGBUF_FULL = (LOGBUF | (RCLOGBUF::LB_FULL << 8)),
381+
LOGBUF_EMPTY = (LOGBUF | (RCLOGBUF::LB_EMPTY << 8)),
342382
};
343383

344384
extern const char *strrc(RC rc);

src/observer/sql/executor/execute_stage.cpp

+134-4
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ See the Mulan PSL v2 for more details. */
4444
#include "storage/default/default_handler.h"
4545
#include "storage/common/condition_filter.h"
4646
#include "storage/trx/trx.h"
47+
#include "storage/clog/clog.h"
4748

4849
using namespace common;
4950

@@ -175,18 +176,29 @@ void ExecuteStage::handle_request(common::StageEvent *event)
175176
default_storage_stage_->handle_event(event);
176177
} break;
177178
case SCF_SYNC: {
179+
/*
178180
RC rc = DefaultHandler::get_default().sync();
179181
session_event->set_response(strrc(rc));
182+
*/
180183
} break;
181184
case SCF_BEGIN: {
185+
do_begin(sql_event);
186+
/*
182187
session_event->set_response("SUCCESS\n");
188+
*/
183189
} break;
184190
case SCF_COMMIT: {
191+
do_commit(sql_event);
192+
/*
185193
Trx *trx = session->current_trx();
186194
RC rc = trx->commit();
187195
session->set_trx_multi_operation_mode(false);
188196
session_event->set_response(strrc(rc));
197+
*/
189198
} break;
199+
case SCF_CLOG_SYNC: {
200+
do_clog_sync(sql_event);
201+
}
190202
case SCF_ROLLBACK: {
191203
Trx *trx = session_event->get_client()->session->current_trx();
192204
RC rc = trx->rollback();
@@ -519,18 +531,40 @@ RC ExecuteStage::do_insert(SQLStageEvent *sql_event)
519531
{
520532
Stmt *stmt = sql_event->stmt();
521533
SessionEvent *session_event = sql_event->session_event();
534+
Session *session = session_event->session();
535+
Db *db = session->get_current_db();
536+
Trx *trx = session->current_trx();
537+
CLogManager *clog_manager = db->get_clog_manager();
522538

523539
if (stmt == nullptr) {
524540
LOG_WARN("cannot find statement");
525541
return RC::GENERIC_ERROR;
526542
}
527543

528544
InsertStmt *insert_stmt = (InsertStmt *)stmt;
529-
530545
Table *table = insert_stmt->table();
531-
RC rc = table->insert_record(nullptr, insert_stmt->value_amount(), insert_stmt->values());
546+
547+
RC rc = table->insert_record(trx, insert_stmt->value_amount(), insert_stmt->values());
532548
if (rc == RC::SUCCESS) {
533-
session_event->set_response("SUCCESS\n");
549+
if (!session->is_trx_multi_operation_mode()) {
550+
CLogRecord *clog_record = nullptr;
551+
rc = clog_manager->clog_gen_record(CLogType::REDO_MTR_COMMIT, trx->get_current_id(), clog_record);
552+
if (rc != RC::SUCCESS || clog_record == nullptr) {
553+
session_event->set_response("FAILURE\n");
554+
return rc;
555+
}
556+
557+
rc = clog_manager->clog_append_record(clog_record);
558+
if (rc != RC::SUCCESS) {
559+
session_event->set_response("FAILURE\n");
560+
return rc;
561+
}
562+
563+
trx->next_current_id();
564+
session_event->set_response("SUCCESS\n");
565+
} else {
566+
session_event->set_response("SUCCESS\n");
567+
}
534568
} else {
535569
session_event->set_response("FAILURE\n");
536570
}
@@ -541,6 +575,10 @@ RC ExecuteStage::do_delete(SQLStageEvent *sql_event)
541575
{
542576
Stmt *stmt = sql_event->stmt();
543577
SessionEvent *session_event = sql_event->session_event();
578+
Session *session = session_event->session();
579+
Db *db = session->get_current_db();
580+
Trx *trx = session->current_trx();
581+
CLogManager *clog_manager = db->get_clog_manager();
544582

545583
if (stmt == nullptr) {
546584
LOG_WARN("cannot find statement");
@@ -551,14 +589,106 @@ RC ExecuteStage::do_delete(SQLStageEvent *sql_event)
551589
TableScanOperator scan_oper(delete_stmt->table());
552590
PredicateOperator pred_oper(delete_stmt->filter_stmt());
553591
pred_oper.add_child(&scan_oper);
554-
DeleteOperator delete_oper(delete_stmt);
592+
DeleteOperator delete_oper(delete_stmt, trx);
555593
delete_oper.add_child(&pred_oper);
556594

557595
RC rc = delete_oper.open();
596+
if (rc != RC::SUCCESS) {
597+
session_event->set_response("FAILURE\n");
598+
} else {
599+
session_event->set_response("SUCCESS\n");
600+
if (!session->is_trx_multi_operation_mode()) {
601+
CLogRecord *clog_record = nullptr;
602+
rc = clog_manager->clog_gen_record(CLogType::REDO_MTR_COMMIT, trx->get_current_id(), clog_record);
603+
if (rc != RC::SUCCESS || clog_record == nullptr) {
604+
session_event->set_response("FAILURE\n");
605+
return rc;
606+
}
607+
608+
rc = clog_manager->clog_append_record(clog_record);
609+
if (rc != RC::SUCCESS) {
610+
session_event->set_response("FAILURE\n");
611+
return rc;
612+
}
613+
614+
trx->next_current_id();
615+
session_event->set_response("SUCCESS\n");
616+
}
617+
}
618+
return rc;
619+
}
620+
621+
RC ExecuteStage::do_begin(SQLStageEvent *sql_event)
622+
{
623+
RC rc = RC::SUCCESS;
624+
SessionEvent *session_event = sql_event->session_event();
625+
Session *session = session_event->session();
626+
Db *db = session->get_current_db();
627+
Trx *trx = session->current_trx();
628+
CLogManager *clog_manager = db->get_clog_manager();
629+
630+
session->set_trx_multi_operation_mode(true);
631+
632+
CLogRecord *clog_record = nullptr;
633+
rc = clog_manager->clog_gen_record(CLogType::REDO_MTR_BEGIN, trx->get_current_id(), clog_record);
634+
if (rc != RC::SUCCESS || clog_record == nullptr) {
635+
session_event->set_response("FAILURE\n");
636+
return rc;
637+
}
638+
639+
rc = clog_manager->clog_append_record(clog_record);
558640
if (rc != RC::SUCCESS) {
559641
session_event->set_response("FAILURE\n");
560642
} else {
561643
session_event->set_response("SUCCESS\n");
562644
}
645+
646+
return rc;
647+
}
648+
649+
RC ExecuteStage::do_commit(SQLStageEvent *sql_event)
650+
{
651+
RC rc = RC::SUCCESS;
652+
SessionEvent *session_event = sql_event->session_event();
653+
Session *session = session_event->session();
654+
Db *db = session->get_current_db();
655+
Trx *trx = session->current_trx();
656+
CLogManager *clog_manager = db->get_clog_manager();
657+
658+
session->set_trx_multi_operation_mode(false);
659+
660+
CLogRecord *clog_record = nullptr;
661+
rc = clog_manager->clog_gen_record(CLogType::REDO_MTR_COMMIT, trx->get_current_id(), clog_record);
662+
if (rc != RC::SUCCESS || clog_record == nullptr) {
663+
session_event->set_response("FAILURE\n");
664+
return rc;
665+
}
666+
667+
rc = clog_manager->clog_append_record(clog_record);
668+
if (rc != RC::SUCCESS) {
669+
session_event->set_response("FAILURE\n");
670+
} else {
671+
session_event->set_response("SUCCESS\n");
672+
}
673+
674+
trx->next_current_id();
675+
676+
return rc;
677+
}
678+
679+
RC ExecuteStage::do_clog_sync(SQLStageEvent *sql_event)
680+
{
681+
RC rc = RC::SUCCESS;
682+
SessionEvent *session_event = sql_event->session_event();
683+
Db *db = session_event->session()->get_current_db();
684+
CLogManager *clog_manager = db->get_clog_manager();
685+
686+
rc = clog_manager->clog_sync();
687+
if (rc != RC::SUCCESS) {
688+
session_event->set_response("FAILURE\n");
689+
} else {
690+
session_event->set_response("SUCCESS\n");
691+
}
692+
563693
return rc;
564694
}

src/observer/sql/executor/execute_stage.h

+3
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,9 @@ class ExecuteStage : public common::Stage {
4747
RC do_select(SQLStageEvent *sql_event);
4848
RC do_insert(SQLStageEvent *sql_event);
4949
RC do_delete(SQLStageEvent *sql_event);
50+
RC do_begin(SQLStageEvent *sql_event);
51+
RC do_commit(SQLStageEvent *sql_event);
52+
RC do_clog_sync(SQLStageEvent *sql_event);
5053

5154
protected:
5255
private:

0 commit comments

Comments
 (0)