Sector index lookup metrics - Testing & Callouts

[rjan90]

Hey everyone!

As part of the lotus-miner-v2 roadmap 🗺 we are planning to move the sector index to a robust database compared to todays non-persistent in-memory maps. With the move you will be able to shut down the lotus-miner process for upgrades/maintenance while keeping the ability to prove your sectors (windowPoSt) and win block rewards (winningPoSt) as long as you have setup windowPoSt- and winningPoSt-workers.

Moving the sector index to another DB should also speed up sector related commands. ⚡️ To measure this effort we are looking to capture the current timings of these commands. ⏱ We encourage storage providers of all sizes to add their current timings, so we get a clearer picture of how long these commands currently take:

---

Example reply card:

Date: 06.02.2023

Sectors output from lotus-miner info:

Sectors:
Total: 5058
Proving: 4202
Removed: 856

Command:	Time to display output (completely finished):
Lotus-miner sectors list	45 seconds
Lotus-miner sectors list —fast	8.8 seconds
Lotus-miner storage list sectors	15 seconds

And in about a weeks time or so - do the same exercise to see the time difference with the given sector growth/fall you had in that timeframe:

Date: xx.02.2023

Sectors output from lotus-miner info:

Sectors:
Total: xxx
Proving: xxx
Removed: xxx

Command:	Time to display output (completely finished):
Lotus-miner sectors list	xx seconds
Lotus-miner sectors list —fast	xx seconds
Lotus-miner storage list sectors	xx seconds

[RobQuistNL]

If this gets into a proper DB its going to make a massive difference.... I've already built my own external DB around it to work around this issue :)

I'm probably towards the 15 minutes for running 1 command. Will post once its done here.

Sectors:
	Total: 245072
	Proving: 244433
	PreCommit1: 36
	PreCommit2: 10
	SubmitPreCommitBatch: 1
	WaitSeed: 14
	Committing: 8
	CommitFinalize: 5
	TerminateWait: 12
	TerminateFailed: 1
	Removed: 552

Command:	Time to display output (completely finished):
lotus-miner info	1m35.837s
lotus-miner sectors list	14m25.788s
lotus-miner sectors list --fast	3m11.388s
lotus-miner storage list sectors	real 28m1.903s (user 11m57.626s) (sys 1m40.182s)

[dd45e640b42e6da7da96faee3996ef7c]

proper here means: not sqlite ;)

[stuberman]

Date: 06.02.2023

Sectors output from lotus-miner info:

Sectors:

Total: 5934
Proving: 4021
Removed: 1913

Command: Time to display output (completely finished):
lotus-miner info 39 seconds
lotus-miner sectors list 39 seconds
lotus-miner sectors list —fast 9 seconds
lotus-miner storage list sectors 11 seconds

[Reiers]

Date: 06.02.2023

Sectors output from lotus-miner info:

Sectors:
        Total: 58168
        Proving: 51846
        UpdateActivating: 1
        WaitDeals: 1
        PreCommit1: 37
        PreCommit2: 8
        PreCommitWait: 1
        WaitSeed: 7
        Committing: 2
        Removed: 6265

Command:	Time to display output (completely finished):
Lotus-miner info	26s
Lotus-miner sectors list	3m18s
Lotus-miner sectors list —fast	1m2s
Lotus-miner storage list sectors	3s

[DLTX-Github]

Sector#Count: 1480
lotus-miner sectors list /// 4.266s
lotus-miner sectors list --fast /// 1.752s
lotus-miner storage list sectors /// 3.662s

Sector#Count: 233499
lotus-miner sectors list /// 14m32.921s
lotus-miner sectors list --fast /// 3m38.607s
lotus-miner storage list sectors /// 8m12.917s

[benjaminh83]

Date: 06.02.2023

Sectors output from lotus-miner info:

Sectors:
        Total: 11084
        Proving: 4302
        Removed: 6779
        Faulty: 3

Command:	Time to display output (completely finished):	previous results
Lotus-miner info	1.15s	33.671s
Lotus-miner sectors list	5.512s	1m18.391s
Lotus-miner sectors list —fast	2.926s	19.186s
Lotus-miner storage list sectors	22.296s	35.878s

I personally like my 3 sectors with the "Faulty" status... Very unique 🥇

[benjaminh83]

On Calib-miner:

$ lotus-miner version
Daemon:  1.21.0-rc1+calibnet+git.3f8a41cfe+api1.5.0
Local: lotus-miner version 1.21.0-rc1+calibnet+git.3f8a41cfe

Total: 6623
        Proving: 6607
        PreCommit1: 10
        PreCommit2: 1
        WaitSeed: 4
        Removed: 1

Command:	Time to display output (completely finished):	previous results
Lotus-miner info	1,167 seconds	1,117  seconds
Lotus-miner sectors list	3,240 seconds	3,175 seconds
Lotus-miner sectors list —fast	2,308 seconds	2,315 seconds

[benjaminh83]

I'm running a small amount of sectors and only CC for now. Using storage-workers, so lotus-miner cannot access the storage directly. Maybe this is why the results are not showing improvements.

[jcace]

Date: 06.02.2023

Sectors output from lotus-miner info:

Sectors:
        Total: 25707
        Proving: 25621
        : 1
        Packing: 3
        Removed: 82

I'm not sure why there's this : 1 in the output. It's red and it's been there for awhile 🤔

Command: Time to display output (completely finished):

lotus-miner info 43s
lotus-miner sectors list 1m27s
lotus-miner sectors list —fast 21s
lotus-miner storage list sectors 45s

[kernelogic]

I recommend mongodb , easy to embed

[beck-8]

Make the user unconscious and does not need to maintain.

[beck-8]

Date: 07.02.2023

Sectors output from lotus-miner info:

Sectors:
        Total: 535865
        Proving: 524321
        Removed: 11544

Command: Time to display output (completely finished):
lotus-miner info 33s
lotus-miner sectors list 3h11m
lotus-miner sectors list —fast 1h33m
lotus-miner storage list sectors 45m

[marshyonline]

Date: 07.02.2023 (DSS 7)

Sectors output from lotus-miner info:

Sectors:
        Total: 77180
        Proving: 75533
        PreCommit1: 24
        PreCommitting: 4
        CommitFinalize: 10
        Removed: 1608
        PreCommitFailed: 1

Command:	Time to display output (completely finished):
Lotus-miner sectors list	6m30.823s
Lotus-miner sectors list —fast	8m53.433s
Lotus-miner storage list sectors	9m33.927s

[Derek-zd]

Sectors:
Total: 1628999
Proving: 305481
Removed: 1323518

Command:	Time to display output (completely finished):
Lotus-miner info	2m17.154s
Lotus-miner sectors list	283m33.518s
Lotus-miner sectors list —fast	170m23.208s
Lotus-miner storage list sectors	90m68.532s

[NSC-FIL]

Sectors:
Total: 10838
Proving: 2074
Packing: 1
Removed: 8763

Command= Avg time to display output:
Lotus-miner sectors list= 36.5 seconds
Lotus-miner sectors list --fast= 12.7 seconds
Lotus-miner storage list sectors= 25.3 seconds

[dd45e640b42e6da7da96faee3996ef7c]

Sectors:
        Total: 5502
        Proving: 5430
        PreCommit1: 45
        PreCommit2: 5
        WaitSeed: 15
        Committing: 4
        Removed: 3

Workers: Seal(16) WdPoSt(1) WinPoSt(1)

Storage Deals: 1718, 53.65 TiB
      Active: 1718  53.65 TiB (Verified: 1718 53.65 TiB)

lotus-miner info 0m2.401s
lotus-miner sectors list 0m13.815s
lotus-miner sectors list —fast 0m4.611s
lotus-miner storage list sectors 0m9.818s

[rjan90]

The coming v1.21.0 release includes parallization to the sector list which should speed up the lotus-miner sectors list significantly. We will look into additional ways of partially fetching the lotus-miner sectors list, but that will require some additional work.

Some initial ideas we have: Fetching just the last xx-amount of sectors/get the sector list for given range/last XX sectors list state changes. (feedback welcome, post in this 🧵)

Now for the results, with the v1.21.0-rc1 release I´m currently seeing:

Sectors:
Total: 5068
Proving: 4205
Removed: 863

Command:	Time to display output (completely finished):	previous results
Lotus-miner info	1,2 seconds
Lotus-miner sectors list	6,1 seconds	45 seconds
Lotus-miner sectors list —fast	2,5 seconds	8.8 seconds

[Reiers]

Date: 24.03.2023

master - lotus version 1.21.0-dev+mainnet+git.41fce94db

Sectors:
	Total: 61162
	Proving: 53503
	UpdateActivating: 1
	AddPiece: 2
	Packing: 2
	PreCommit1: 18
	PreCommit2: 1
	Removed: 7635

Command:	Time to display output (completely finished):	previous results
Lotus-miner info	5.2 seconds	26s
Lotus-miner sectors list	25.6 seconds	3m18s
Lotus-miner sectors list —fast	11.6 seconds	1m2s