v0.10.1

v0.10.1 is a minor release on top of v0.10.0 with mostly dashboard/docs updates.

Major Changes since Last Release (v0.10.0)

• Support incremental collection for Jira worklog
• Add a dashboard for weekly bug retro use case (grafana/WeeklyBugRetro.json)
• Add a generic Jenkins dashboard (grafana/Jenkins.json)
• Improve the Github basic metrics dashboard (grafana/GithubBasicMetrics.json)
• Update dashboards according to the latest domain layer schema
• Adopt a new file name convention to shorten file names
• Add docs for the advanced mode of Config UI

Full Changelog: v0.10.0...v0.10.1

v0.10.0

Major Changes since Last Release (v0.9.3)

• Pipeline Blueprint allows users to schedule recurring pipelines with cron expression
• Introduce DB migrations to DevLake to provide a smoother upgrade experience
• Add temporal mode for users with heavy data collection workload
• Add dbt plugin for users who want to define their own data transformations
• Store/cache API responses from data sources in the raw data layer for instant replay of data transformations

Full Changelog: v0.9.3...v0.10.0

Bug Fixes & Improvements since Last Beta Release (v0.10.0-beta1)

• Improve documentation for user setup and GitHub data collection process
• Fix incorrect created_date in domain layer
• Fix stuck pipelines due to retries
• Add users for plugin Jira

Full Changelog: v0.10.0-beta1...v0.10.0

New Contributors

• @liangjingyang made their first contribution in https://github.com/merico-dev/lake/pull/377
• @nitinvasanth made their first contribution in https://github.com/merico-dev/lake/pull/526
• @IronCore864 made their first contribution in https://github.com/merico-dev/lake/pull/747
• @ggcannard made their first contribution in https://github.com/merico-dev/lake/pull/752
• @MaximDub made their first contribution in https://github.com/merico-dev/lake/pull/885
• @Juneezee made their first contribution in https://github.com/merico-dev/lake/pull/1013
• @basicthinker made their first contribution in https://github.com/merico-dev/lake/pull/1178
• @daniel-hutao made their first contribution in https://github.com/merico-dev/lake/pull/1193
• @abeizn made their first contribution in https://github.com/merico-dev/lake/pull/1343

v0.10.0-beta1

Major Changes since Last Release (v0.9.3)

• Pipeline Blueprint allows users to schedule recurring pipelines with cron expression
• Introduce DB migrations to DevLake to provide a smoother upgrade experience
• Add temporal mode for users with heavy data collection workload
• Add dbt plugin for users who want to define their own data transformations
• Store/cache API responses from data sources in the raw data layer for instant replay of data transformations

Full Changelog: v0.9.3...v0.10.0-beta1

New Contributors

• @liangjingyang made their first contribution in https://github.com/merico-dev/lake/pull/377
• @nitinvasanth made their first contribution in https://github.com/merico-dev/lake/pull/526
• @IronCore864 made their first contribution in https://github.com/merico-dev/lake/pull/747
• @ggcannard made their first contribution in https://github.com/merico-dev/lake/pull/752
• @MaximDub made their first contribution in https://github.com/merico-dev/lake/pull/885
• @Juneezee made their first contribution in https://github.com/merico-dev/lake/pull/1013
• @basicthinker made their first contribution in https://github.com/merico-dev/lake/pull/1178
• @daniel-hutao made their first contribution in https://github.com/merico-dev/lake/pull/1193
• @abeizn made their first contribution in https://github.com/merico-dev/lake/pull/1343

v0.9.3

Fixed issues found in v0.9.2

1. gitlab api client reporting nil pointer #1450
2. gitlab commits collection not working #1459
3. mysql reporting unexpected EOF due to some operation would hold cursor for a long time #1452
4. convert gitlab_merge_request_commits to domain layer data pull_request_commits #1470

v0.9.2

Fixed issues found in v0.9.1

1. Improved gitextractor performance, from Forever to less than an hour on Clickhouse repo #1435
2. Fixed commit_sha being incorrect from refs table #1441
3. Skipped invalid pull_request_issues records when enriching data from github plugin #1438
4. Enhanced config-ui field behavior on connection settings and other tweaks #1427, #1422, #1436, #1444
5. Added author_id autho_name and description field to pull_request table #1431
6. Fixed pull_request_commits data corruption #1430
7. Fixed connection status on connection page from config-ui

v0.9.1

Fixed issues found in v0.9.0

1. v0.9.0 wouldn't pick up github bug/requirement settings unless devlake got restarted (https://github.com/merico-dev/lake/issues/1394)
2. v0.9.0 github settings - field saving issues (https://github.com/merico-dev/lake/issues/1393)
3. Github configuration: the placeholder of severity is incorrect (https://github.com/merico-dev/lake/issues/1384)

Other improvements:

4. Improve demo dashboards (https://github.com/merico-dev/lake/pull/1398)

v0.9.0

IMPORTANT: DevLake doesn't support Database Schema Migration yet, upgrading an existing instance is likely to break, we recommend that you deploy a new instance instead.

What's Changed

• fix: gitlab provider - disable project id mappings field by @e2corporation in https://github.com/merico-dev/lake/pull/1260
• EE integration bug fix by @mindlesscloud in https://github.com/merico-dev/lake/pull/1132
• feat: collect gitlab tags by @likyh in https://github.com/merico-dev/lake/pull/1267
• fix: create pipeline : optimize advanced menu accessibility by @e2corporation in https://github.com/merico-dev/lake/pull/1268
• refactor(github): improve github collection performance by @warren830 in https://github.com/merico-dev/lake/pull/1253
• fix(plugins): fix concurrent problems by @warren830 in https://github.com/merico-dev/lake/pull/1275
• docs: refine the main readme by @Startrekzky in https://github.com/merico-dev/lake/pull/1277
• docs: update Chinese main README by @Startrekzky in https://github.com/merico-dev/lake/pull/1283
• fix: multithread read empty config by @mappjzc in https://github.com/merico-dev/lake/pull/1263
• Feat add GitHub response field by @mappjzc in https://github.com/merico-dev/lake/pull/1282
• feat(github): extract relation between pr and issue by @warren830 in https://github.com/merico-dev/lake/pull/1285
• fix(jenkins): change delete scope by @warren830 in https://github.com/merico-dev/lake/pull/1290
• fix(github): fix test connection by @warren830 in https://github.com/merico-dev/lake/pull/1289
• fix: spell component by @mappjzc in https://github.com/merico-dev/lake/pull/1291
• feat: add validate for force required field by @mappjzc in https://github.com/merico-dev/lake/pull/1297
• fix: jira settings > enable reset query on multiselect fields by @e2corporation in https://github.com/merico-dev/lake/pull/1296
• fix: create pipeline > restore basic textarea editor in advanced mode by @e2corporation in https://github.com/merico-dev/lake/pull/1295
• fix: jira proxy should be lowercase by @klesh in https://github.com/merico-dev/lake/pull/1292
• Feat creat ref bug stats by @warren830 in https://github.com/merico-dev/lake/pull/1293
• fix(plugins): add support for proxy field in api by @warren830 in https://github.com/merico-dev/lake/pull/1298
• fix(gitlab): fix confusing tables by @warren830 in https://github.com/merico-dev/lake/pull/1300
• feat: reconfigure proxy and extend github settings by @e2corporation in https://github.com/merico-dev/lake/pull/1276
• jira server support by @mindlesscloud in https://github.com/merico-dev/lake/pull/1301
• fix: handle status code 404 by @mindlesscloud in https://github.com/merico-dev/lake/pull/1305
• fix(frontend): modify jira url for issuetyp and field by @warren830 in https://github.com/merico-dev/lake/pull/1309
• fix(jenkins): change DurationSec unit to second by @warren830 in https://github.com/merico-dev/lake/pull/1310
• chore(docs): release v0.9.0 by @warren830 in https://github.com/merico-dev/lake/pull/1313

Full Changelog: merico-ai/lake@v0.8.0...v0.9.0

v0.8.0

IMPORTANT: DevLake doesn't support Database Schema Migration yet, upgrading an existing instance is likely to break, we recommend that you deploy a new instance instead.

What's Changed

• docs: update issue templates by @Startrekzky in https://github.com/merico-dev/lake/pull/1173
• Automated tests by @joncodo in https://github.com/merico-dev/lake/pull/1174
• Update README by @basicthinker in https://github.com/merico-dev/lake/pull/1178
• Fix store pr labels individually by @warren830 in https://github.com/merico-dev/lake/pull/1180
• fix(plugins): pass pointer instead value by @warren830 in https://github.com/merico-dev/lake/pull/1183
• docs: update main README.md by @e2corporation in https://github.com/merico-dev/lake/pull/1152
• docs: fixed instructions for updated installation by @kevin-kline in https://github.com/merico-dev/lake/pull/1154
• refactor(plugins): use cursor to iterate dbdata by @warren830 in https://github.com/merico-dev/lake/pull/1185
• feat: setup multi-selectors for jira issue types w/ api proxy | 1064 by @e2corporation in https://github.com/merico-dev/lake/pull/1177
• Task cancelling for GitHub plugin by @kevin-kline in https://github.com/merico-dev/lake/pull/1179
• WIP by @joncodo in https://github.com/merico-dev/lake/pull/1188
• Feat: Adding Kubernetes deploy support by @daniel-hutao in https://github.com/merico-dev/lake/pull/1193
• refactor(domain layer,plugins): indicate which sub task failed by @warren830 in https://github.com/merico-dev/lake/pull/1181
• docs: update main README for v0.7.0 by @klesh in https://github.com/merico-dev/lake/pull/1197
• docs: update English and Chinese Readmes to v0.7.0 by @Startrekzky in https://github.com/merico-dev/lake/pull/1198
• hotfix: config-ui - fix create pipeline page | 1176 by @e2corporation in https://github.com/merico-dev/lake/pull/1191
• feat: git extractor support private repo by @mindlesscloud in https://github.com/merico-dev/lake/pull/1202
• E2E tests that can be run manually once per iteration by @joncodo in https://github.com/merico-dev/lake/pull/1189
• fix: refdiff rook_pkg_path typo by @klesh in https://github.com/merico-dev/lake/pull/1206
• feat:Encrypt for JiraAuth by @mappjzc in https://github.com/merico-dev/lake/pull/1205
• docs: fix typo for gitextractor plugin by @klesh in https://github.com/merico-dev/lake/pull/1208
• docs: update readme example code by @klesh in https://github.com/merico-dev/lake/pull/1209
• fix(refdiff): fix progress by @warren830 in https://github.com/merico-dev/lake/pull/1210
• Fix jenkins save builds by @warren830 in https://github.com/merico-dev/lake/pull/1204
• feat: create pipeline - advanced pipeline mode | 1127 by @e2corporation in https://github.com/merico-dev/lake/pull/1203
• fix(plugins): change assignee id by @warren830 in https://github.com/merico-dev/lake/pull/1211
• fix: use hooks to handle encryption and decryption by @mappjzc in https://github.com/merico-dev/lake/pull/1212
• Fix empty pipeline stage causing backend to crash by @klesh in https://github.com/merico-dev/lake/pull/1214
• docs: update developer setup procedure by @klesh in https://github.com/merico-dev/lake/pull/1215
• fix(jenkins): fix table definition by @warren830 in https://github.com/merico-dev/lake/pull/1219
• fix: github reporting 'invalid token' for invalid endpoint url by @klesh in https://github.com/merico-dev/lake/pull/1218
• fix(github): make sure of data integration by @warren830 in https://github.com/merico-dev/lake/pull/1224
• fix: remove empty workflow by @klesh in https://github.com/merico-dev/lake/pull/1225
• fix(github): github not collecting pull requests by @klesh in https://github.com/merico-dev/lake/pull/1227
• fix: retry with new nonce by @mindlesscloud in https://github.com/merico-dev/lake/pull/1228
• fix gitextractor arguments validation by @klesh in https://github.com/merico-dev/lake/pull/1236
• fix(refdiff): correct progress by @warren830 in https://github.com/merico-dev/lake/pull/1235
• fix: ae plugin tasks argument not working by @klesh in https://github.com/merico-dev/lake/pull/1237
• fix: collect pipeline find with project_id by @mappjzc in https://github.com/merico-dev/lake/pull/1241
• fix(feishu): decrease ratelimit by @warren830 in https://github.com/merico-dev/lake/pull/1242
• fix(frontend): to run in advanced mode by @warren830 in https://github.com/merico-dev/lake/pull/1240
• Fix jira collect board panic by @mappjzc in https://github.com/merico-dev/lake/pull/1244
• User setup fixes by @klesh in https://github.com/merico-dev/lake/pull/1254
• chore: release v0.8.0 by @klesh in https://github.com/merico-dev/lake/pull/1255
• Hotfix for github PR by @klesh in https://github.com/merico-dev/lake/pull/1259

New Contributors

• @basicthinker made their first contribution in https://github.com/merico-dev/lake/pull/1178
• @daniel-hutao made their first contribution in https://github.com/merico-dev/lake/pull/1193
• @mappjzc made their first contribution in https://github.com/merico-dev/lake/pull/1205

Full Changelog: merico-ai/lake@v0.7.0...v0.8.0

v0.7.0

What's Changed

• fix: replaced /env with /ping by @kevin-kline in https://github.com/merico-dev/lake/pull/1055
• fix: label color change by @kevin-kline in https://github.com/merico-dev/lake/pull/1056
• Schema v1 issue management by @mindlesscloud in https://github.com/merico-dev/lake/pull/1034
• Fix Github commits data lost by @klesh in https://github.com/merico-dev/lake/pull/1062
• fix: pipelines feature post-development tasks by @e2corporation in https://github.com/merico-dev/lake/pull/1059
• feat: create pipelines - use multi-tag selector for gitlab projects and jira boards | 987 by @e2corporation in https://github.com/merico-dev/lake/pull/1071
• feat jira settings : add jira issue tag selector entries on blur | 1078 by @e2corporation in https://github.com/merico-dev/lake/pull/1079
• fix(db): changed method to insert jira-sprint-issue by @warren830 in https://github.com/merico-dev/lake/pull/1075
• refactor(plugins): add check statement by @warren830 in https://github.com/merico-dev/lake/pull/1084
• Refactor hide authinfo in log by @warren830 in https://github.com/merico-dev/lake/pull/1085
• feat: add more accurency error message when test connection by @narrowizard in https://github.com/merico-dev/lake/pull/1098
• RefDiff plugin by @klesh in https://github.com/merico-dev/lake/pull/1086
• feat(pipeline): limit requests per second by @warren830 in https://github.com/merico-dev/lake/pull/1093
• feat(github plugin): resolve github commits and pull requests by @warren830 in https://github.com/merico-dev/lake/pull/1107
• feat: git repo extractor by @mindlesscloud in https://github.com/merico-dev/lake/pull/1109
• refactor(plugins): change tables and clean funcs by @warren830 in https://github.com/merico-dev/lake/pull/1112
• feat: link tool layer github issues to commit by @joncodo in https://github.com/merico-dev/lake/pull/1116
• feat(plugins-github): enrich pull request with labels by @warren830 in https://github.com/merico-dev/lake/pull/1108
• fix: clean up previous collected/converted data for jira by @klesh in https://github.com/merico-dev/lake/pull/1121
• Feat enrich issues severity component by @warren830 in https://github.com/merico-dev/lake/pull/1122
• Docs for refdiff on Mac by @joncodo in https://github.com/merico-dev/lake/pull/1117
• docs: more accurate document by @klesh in https://github.com/merico-dev/lake/pull/1119
• fix multiple problems for PingCap release by @klesh in https://github.com/merico-dev/lake/pull/1130
• feat: pipeline run - implement multi-stage task activity | 1077 by @e2corporation in https://github.com/merico-dev/lake/pull/1091
• fix: ref names of refs_commits_diffs are missing prefix by @klesh in https://github.com/merico-dev/lake/pull/1131
• fix: invalid struct type error for jira plugin by @klesh in https://github.com/merico-dev/lake/pull/1141
• fix: fix too many connections by @mindlesscloud in https://github.com/merico-dev/lake/pull/1142
• refactor: optimize label collect by @mindlesscloud in https://github.com/merico-dev/lake/pull/1143
• refactor: rename new/old_commit_sha by @klesh in https://github.com/merico-dev/lake/pull/1144
• Update README.md by @joncodo in https://github.com/merico-dev/lake/pull/1128
• feat: github plugin collect merge_commit_sha by @klesh in https://github.com/merico-dev/lake/pull/1149
• fix(plugins): fix multiple bugs to current version by @warren830 in https://github.com/merico-dev/lake/pull/1138
• refactor: remove issues.owner_id by @mindlesscloud in https://github.com/merico-dev/lake/pull/1153
• fix(plugins): fix severity not shown by @warren830 in https://github.com/merico-dev/lake/pull/1159
• fix(plugins): added progress by @warren830 in https://github.com/merico-dev/lake/pull/1158
• refactor: rename github_repository to githu_repo by @klesh in https://github.com/merico-dev/lake/pull/1160
• github repository renaming for client side / docker-compose support local build by @klesh in https://github.com/merico-dev/lake/pull/1161
• fix(plugins): store number in key field by @warren830 in https://github.com/merico-dev/lake/pull/1162
• fix: fix pipeline got stuck by @mindlesscloud in https://github.com/merico-dev/lake/pull/1164
• refactor(plugins): combine convert repos by @warren830 in https://github.com/merico-dev/lake/pull/1163
• Doc update j by @joncodo in https://github.com/merico-dev/lake/pull/1151
• fix: fix column name committer_id by @mindlesscloud in https://github.com/merico-dev/lake/pull/1167
• refactor(domainlayer): modify table.boards by @warren830 in https://github.com/merico-dev/lake/pull/1169
• fix: added org as prefix to github repo name by @klesh in https://github.com/merico-dev/lake/pull/1170
• Release v0.7.0 by @klesh in https://github.com/merico-dev/lake/pull/1171
• fix: remove build section from release yml by @klesh in https://github.com/merico-dev/lake/pull/1172

New Contributors

• @warren830 made their first contribution in https://github.com/merico-dev/lake/pull/1075

Full Changelog: merico-ai/lake@v0.6.0...v0.7.0

v0.6.1

Fixed issue #1061 for v0.6.0