手动杀死Prometheus和ng后,ng重启失败 (Description ng fails to restart after prometheus and ng are manually killed)
tidb版本:6.5
ng版本: Git Commit Hash"=f1c05e221155c2c95d391957971defbcbbf56832
错误日志位于:/tidb-deploy/prometheus-9090/log/tsdb.log
2023-03-07T06:36:53.570Z info /go/pkg/mod/github.com/!victoria!metrics/!victoria!metrics@v1.65.0/app/vmselect/promql/rollup_result_cache.go:106 loading rollupResult cache from "/tidb-data/prometheus-9090/tsdb/cache/rollupResult"...
2023-03-07T06:36:53.571Z info /go/pkg/mod/github.com/!victoria!metrics/!victoria!metrics@v1.65.0/app/vmselect/promql/rollup_result_cache.go:132 loaded rollupResult cache from "/tidb-data/prometheus-9090/tsdb/cache/rollupResult" in 0.001 seconds; entriesCount: 0, sizeBytes: 0
2023-03-07T06:36:53.571Z info /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/ng-monitoring/database/timeseries/vm.go:45 started VictoriaMetrics in 0.094 seconds
2023/03/07 14:36:53 Arena too small, toWrite:126 newTotal:21810362 limit:21810342
github.com/dgraph-io/badger/v3/y.AssertTruef
/go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/y/error.go:62
github.com/dgraph-io/badger/v3/skl.(*Arena).putVal
/go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/skl/arena.go:83
github.com/dgraph-io/badger/v3/skl.newNode
/go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/skl/skl.go:114
github.com/dgraph-io/badger/v3/skl.(*Skiplist).Put
/go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/skl/skl.go:306
github.com/dgraph-io/badger/v3.(*memTable).replayFunction.func1
/go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/memtable.go:258
github.com/dgraph-io/badger/v3.(*logFile).iterate
/go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/memtable.go:521
github.com/dgraph-io/badger/v3.(*memTable).UpdateSkipList
/go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/memtable.go:219
github.com/dgraph-io/badger/v3.(*DB).openMemTable
/go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/memtable.go:146
github.com/dgraph-io/badger/v3.(*DB).openMemTables
/go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/memtable.go:88
github.com/dgraph-io/badger/v3.Open
/go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/db.go:333
github.com/genjidb/genji/engine/badgerengine.NewEngine
/go/pkg/mod/github.com/crazycs520/genji/engine/badgerengine@v0.12.1-0.20220328082424-727a2d089bde/engine.go:25
github.com/pingcap/ng-monitoring/database/document.Init
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/ng-monitoring/database/document/document.go:34
github.com/pingcap/ng-monitoring/database.Init
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/ng-monitoring/database/database.go:14
main.main
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/ng-monitoring/main.go:68
runtime.main
/usr/local/go/src/runtime/proc.go:250
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1594
ng.log 无报错
最后的日志如下
[2023/03/07 14:41:44.272 +08:00] [INFO] [printer.go:25] ["Welcome to ng-monitoring."] ["Git Commit Hash"=f1c05e221155c2c95d391957971defbcbbf56832] ["Git Branch"=heads/refs/tags/v6.5.0] ["UTC Build Time"="2022-12-16 08:18:47"] [GoVersion=go1.19.3]
[2023/03/07 14:41:44.273 +08:00] [INFO] [main.go:64] [config] [config="{\"address\":\"0.0.0.0:12020\",\"advertise_address\":\"10.0.0.80:12020\",\"pd\":{\"endpoints\":[\"10.0.0.80:2379\",\"10.0.0.76:2379\",\"10.0.0.74:2379\"]},\"log\":{\"path\":\"/tidb-deploy/prometheus-9090/log\",\"level\":\"INFO\"},\"storage\":{\"path\":\"/tidb-data/prometheus-9090\"},\"continuous_profiling\":{\"enable\":false,\"profile_seconds\":10,\"interval_seconds\":60,\"timeout_seconds\":120,\"data_retention_seconds\":259200},\"security\":{\"ca_path\":\"\",\"cert_path\":\"\",\"key_path\":\"\"}}"]
手动杀死Prometheus和ng后,ng重启失败 (Description ng fails to restart after prometheus and ng are manually killed)
tidb版本:6.5
ng版本: Git Commit Hash"=f1c05e221155c2c95d391957971defbcbbf56832
错误日志位于:/tidb-deploy/prometheus-9090/log/tsdb.log
ng.log 无报错
最后的日志如下