Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Benchmark cleanup #20821

Closed
wants to merge 2 commits into from
Closed

Benchmark cleanup #20821

wants to merge 2 commits into from

Conversation

roji
Copy link
Member

@roji roji commented May 2, 2020

This PR cleans up some benchmark stuff.

@sebastienros once this is merged, we'd need the corresponding cleanup to also occur in the database. I hope you can help us with that...

Benchmark name changes:

  • InitializationSqliteTests -> Initialization
  • FuncletizationSqliteTests -> Funcletization
  • NavigationsQuerySqliteTests -> NavigationsQuery (does not appear in dashboard!)
  • QueryCompilationSqliteTests -> QueryCompilation (does not appear in dashboard!)
  • RawSqlQuerySqliteTests -> RawSqlQuery
  • SimpleQuerySqliteTests -> SimpleQuery

For the following, I've configured BDN to always show the times for exactly one operation, even when the benchmark internally executes a loop. That means we'd need to multiply historical values in the database:

  • AddDataVariations: multiply by 20000
  • ExistingDataVariations: multiply by 20000
  • ChildVariations: multiply by 10000
  • ParentVariations: multiply by 5000
  • InitializationTests.CreateAndDisposeUnusedContext: multiply by 10000
  • InitializationTests.InitializeAndQuery_AdventureWorks: multiply by 1000
  • InitializationTests.InitializeAndSaveChanges_AdventureWorks: multiply by 100
  • FuncletizationTests: multiply by 100
  • NavigationsQueryTests: multiply by 10
  • QueryCompilation: multiply by 10

@roji roji requested a review from a team May 2, 2020 18:20
@smitpatel
Copy link
Contributor

I don't think OperationsPerInvoke is correct thing for all of them. Bring it to design meeting please.

@roji
Copy link
Member Author

roji commented May 13, 2020

OK we've discussed this in design and agreed it's a good direction.

@sebastienros, this now depends on you :) When we merge this, all the above changes will need to take place in the database - will you be able to help us with this?

@smitpatel
Copy link
Contributor

Ping!

@sebastienros
Copy link
Member

Not yet ready to do it. But this will be done.

@roji roji added the blocked label Jun 3, 2020
@microsoft-github-updates microsoft-github-updates bot changed the base branch from master to main July 23, 2020 04:35
@smitpatel smitpatel mentioned this pull request Aug 5, 2020
Open
@roji roji marked this pull request as draft April 22, 2021 09:04
@roji roji mentioned this pull request Jul 29, 2021
Open
This was linked to issues Jul 29, 2021
Benchmarks: make benchmarks always report results for one operation #25361
Open
Benchmarks: remove Sqlite from benchmark name #19874
Open
@roji roji mentioned this pull request Jul 29, 2021
Open
@roji
Copy link
Member Author

roji commented Jul 29, 2021

Closing for age, will re-tackle this again in the future. Tracking issues are #25361 and #19874.

@roji roji closed this Jul 29, 2021
@smitpatel smitpatel deleted the BenchmarkCleanup branch May 18, 2022 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@smitpatel smitpatel smitpatel left review comments

@ajcvickers ajcvickers ajcvickers approved these changes

Assignees
No one assigned
Labels
blocked
Projects
None yet
Milestone
No milestone
4 participants
@roji @smitpatel @sebastienros @ajcvickers