Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
337 commits
Select commit Hold shift + click to select a range
7510476
Add tracing messages
tli2 Apr 11, 2024
b566721
Checkpoint Kube work
tli2 Apr 11, 2024
2c54b1f
Checkpoint work
tli2 Apr 16, 2024
1b559b8
Abstract benchmark constants into Environment class
tli2 Apr 17, 2024
ddca313
Switch to using pvcs for k8 experiments
tli2 Apr 17, 2024
4831e2d
kubernetes fuckery
tli2 Apr 17, 2024
56aaa55
Apoply kubernetes node selector
tli2 Apr 17, 2024
fcd3526
Use correct label name this time
tli2 Apr 17, 2024
36fa7b6
Reorganize helm to split storage and workloads so storage volumes can…
tli2 Apr 17, 2024
fcbfc7e
More logging + temporarily disable non-speculative mode
tli2 Apr 17, 2024
4cd50e4
Correctly cleanup leftover state
tli2 Apr 17, 2024
ab0fca5
Actually dispose of devices at the start of execution. Introduce some…
tli2 Apr 17, 2024
51dd378
More scripting shenanigans
tli2 Apr 17, 2024
544d267
More kubernetes fuckery
tli2 Apr 17, 2024
a69d369
The fun never stops
tli2 Apr 17, 2024
7990dc7
Convert to using helm for event processing benchmark as well
tli2 Apr 18, 2024
b1b231d
Add speculative flag back in
tli2 Apr 18, 2024
5ba497c
Remove large file for docker perf
tli2 Apr 18, 2024
25d439b
Fix deadlock from interplay of FASTER and EPVS
tli2 Apr 18, 2024
fec4544
reduce sleep time on start up
tli2 Apr 18, 2024
de2202b
Actually delete files for fuck sake
tli2 Apr 18, 2024
a192a44
Fix workflow generation
tli2 Apr 18, 2024
4b0c055
Add output labeling
tli2 Apr 18, 2024
c6592f7
Separate output name and workload name
tli2 Apr 18, 2024
5855300
Checkpoint before streaming refactor
tli2 Apr 19, 2024
2078f3e
Turns out rate limiting works?
tli2 Apr 19, 2024
5fefcfa
Use correct environment
tli2 Apr 19, 2024
bb801c2
Debug prints for reservation service checkpoint pass
tli2 Apr 19, 2024
a3c8365
Try to get debug working on kubernetes
tli2 Apr 20, 2024
3db3f02
Remove hard coded init files
tli2 Apr 20, 2024
b7899d8
Fix Faster checkpoint error?
tli2 Apr 20, 2024
3ce46e2
Include source file and build with debug symbols
tli2 Apr 20, 2024
aaaec12
fix file structure
tli2 Apr 20, 2024
6993150
more path problems
tli2 Apr 20, 2024
b5a6eab
Fix Linux bug
tli2 Apr 21, 2024
7bfe791
Remove resource limit on containers
tli2 Apr 21, 2024
823bdf7
Remove unnecessary awaits
tli2 Apr 21, 2024
252cdb7
Update experiment configs
tli2 Apr 21, 2024
cd5fb6f
Make issue window a command line arg
tli2 Apr 21, 2024
207a1aa
Update workload file paths
tli2 Apr 21, 2024
cbe22fa
file system shenanigans
tli2 Apr 21, 2024
6b2ad7b
Updates
tli2 Apr 21, 2024
6f655e4
More threads
tli2 Apr 21, 2024
0226e7f
Various Async fixes
tli2 Apr 21, 2024
8c0ef4f
Hack to avoid having ManagedLocalStorageDevice Task starve
tli2 Apr 21, 2024
f6467c0
Disable checkpoint metadata write in critical section to avoid blocking
tli2 Apr 21, 2024
0f26073
Checkpoint work for event processing
tli2 Apr 22, 2024
3179551
More set up for Kubernetes workload
tli2 Apr 22, 2024
1fffb56
Slight optimization around data loading logic
tli2 Apr 23, 2024
d113e06
Update paths
tli2 Apr 23, 2024
3790d09
Update docker file
tli2 Apr 23, 2024
64bad34
remove selector from jobs
tli2 Apr 23, 2024
0a88d2f
Fix typo
tli2 Apr 23, 2024
203641c
Fix typos and add sleep to make sure client launches after processors
tli2 Apr 23, 2024
3312e3b
correct init container
tli2 Apr 23, 2024
e1642ff
Kubernetes fuckery
tli2 Apr 23, 2024
8902e9c
Optimizations
tli2 Apr 23, 2024
a01f973
Fix messages that are too large
tli2 Apr 23, 2024
73102c2
Add missing await
tli2 Apr 23, 2024
e215e14
Optimization?
tli2 Apr 23, 2024
5a05f3d
Temporarily disable user side rate limit
tli2 Apr 23, 2024
9121f07
Re-enable
tli2 Apr 23, 2024
1ad7305
Make stepping speculative for speed
tli2 Apr 23, 2024
da7b7b6
Update docker file
tli2 Apr 24, 2024
b282340
Rollback probelmatic reuse
tli2 Apr 24, 2024
aa9c71f
Disable gRPC auto close connection and tweak Linux thread pool
tli2 Apr 24, 2024
3f50710
Rewrite checkpoint manager to use synchronous file operations
tli2 Apr 24, 2024
0f98626
Add rate limiting for state object related awaits
tli2 Apr 24, 2024
11b2d77
Switch to Thread.Yield() for StateObject busy wait under access control
tli2 Apr 24, 2024
a4104ba
Task behavior optimizations
tli2 Apr 25, 2024
a71cd01
Use thread-based background tasks
tli2 Apr 25, 2024
d9d1ecb
Add more data to collect on stream processing
tli2 Apr 25, 2024
72b420f
Add microbench
tli2 Apr 26, 2024
2691d64
Fix bytes flushed logic
tli2 Apr 26, 2024
89543d5
Checkpoint work
tli2 Apr 28, 2024
669ae9e
Add coordinator microbenchmark
tli2 Apr 28, 2024
b18d9be
update docker file
tli2 Apr 28, 2024
f539d95
Add missing account
tli2 Apr 29, 2024
89e9c0e
Fix iteration logic
tli2 Apr 29, 2024
60984a4
Use ms instead of ticks
tli2 Apr 29, 2024
2857967
Add synchronization to ensure simulated workers start at the same time
tli2 Apr 29, 2024
d875f81
Actually synchronize
tli2 Apr 29, 2024
395457f
Try sharing stopwatch across threads
tli2 Apr 29, 2024
74eedb5
Don't use multiple threads
tli2 Apr 29, 2024
5cdd8d7
initialize
tli2 Apr 29, 2024
4339693
Initialize
tli2 Apr 29, 2024
1025696
Fix reuse bugs
tli2 Apr 29, 2024
a68965a
Change environment
tli2 Apr 29, 2024
64aea30
Checkpoint progress
tli2 Apr 29, 2024
b697480
Implement logic to short circuit recovery if recovery is not necessary
tli2 Apr 29, 2024
1ace2bc
Checkpoint on recoverability fix
tli2 Apr 30, 2024
822d4ea
Checkpoint progress
tli2 Apr 30, 2024
679df2d
Make recovery work
tli2 Apr 30, 2024
57658cc
Prepare code base for recovery experiments
tli2 Apr 30, 2024
f0e6ff7
Update docker file
tli2 Apr 30, 2024
4b5ab65
Fix infinite error loop in benchmark data loader
tli2 Apr 30, 2024
140e1ce
Correctly handle some exceptions
tli2 May 1, 2024
d815037
Prevent infinite await from leftover iterators
tli2 May 1, 2024
bac6205
Sequential issue to avoid data loss
tli2 May 1, 2024
a1d0527
Fix stupid out of order C# Task issues
tli2 May 1, 2024
e0646f4
Remove gRPC message size limit
tli2 May 1, 2024
9e6707b
Add some parallelism at producer to increase throughput
tli2 May 1, 2024
dd8a288
Try increasing background worker batch size to offset latency increase
tli2 May 1, 2024
eae183d
Microbenchmark updates
tli2 May 2, 2024
2a59743
Optimizations using duplex streaming calls in gRPC
tli2 May 2, 2024
ae21689
Add async to colocated processor
tli2 May 2, 2024
f4df463
Local commit optimizations
tli2 May 2, 2024
04e5ee9
fix typo
tli2 May 2, 2024
a2e8596
Optimization to ensure we don't wait for DPR on the dumb speculative …
tli2 May 2, 2024
c08f74e
checkpoint changes
tli2 May 2, 2024
916fb03
GC tweaks
tli2 May 2, 2024
d194029
Test not reusing step requests across workflows
tli2 May 2, 2024
a6a2dd9
Revert "Test not reusing step requests across workflows"
tli2 May 2, 2024
31af5e3
remove pull policy
tli2 May 3, 2024
ea52d22
Commit path perf tweak
tli2 May 3, 2024
2223b7e
More rate limiting
tli2 May 3, 2024
5239ee1
RunContinuationsAsynchronously
tli2 May 3, 2024
bb7aa5b
tweak helm chart
tli2 May 3, 2024
9ff4c07
Tweak await logic
tli2 May 3, 2024
0f0939a
Minor tweaks
tli2 May 3, 2024
9298610
Fix typo
tli2 May 4, 2024
3ff6e3a
ValueTask fix and reporting tweaks
tli2 May 4, 2024
65511bc
Tweaks
tli2 May 4, 2024
f290b18
Add hacky 2PC experiment
tli2 May 5, 2024
817a88e
Update dockerfile
tli2 May 5, 2024
3ba5c69
fix helm template
tli2 May 5, 2024
2aa0dc4
Remove debug prints
tli2 May 5, 2024
1880189
Experiment tweaks
tli2 May 5, 2024
63e64a4
Change measurements to use list instead of dictionary for performance
tli2 May 5, 2024
7f60eb2
Fix termination
tli2 May 5, 2024
67e126d
Report milliseconds instead of ticks
tli2 May 5, 2024
7d5afe6
Use in-memory recovery instead of somehow really slow FasterLog recovery
tli2 May 6, 2024
7892129
Make sure failures are not too well-behaved by triggering it asynchro…
tli2 May 6, 2024
9e5883d
More randomness
tli2 May 6, 2024
8dc4cc1
Revert "More randomness"
tli2 May 6, 2024
7507ae4
Revert "Make sure failures are not too well-behaved by triggering it …
tli2 May 6, 2024
3f9d7ba
Revert "Use in-memory recovery instead of somehow really slow FasterL…
tli2 May 6, 2024
efe9500
Avoid thundering horde on non-speculative path
tli2 May 6, 2024
e3709ec
Revert back to in-memory recovery for now
tli2 May 6, 2024
7d99d9a
NativeDevice
tli2 May 17, 2024
d3eccb9
Kubernetes Fuckery
tli2 May 17, 2024
e2039d4
More Kubernetes Fuckery
tli2 May 17, 2024
6fb39a9
Add missing dependencies
tli2 May 17, 2024
5323409
I hate kubernetes
tli2 May 17, 2024
3a95b22
Add version drift tolerance
tli2 Jun 3, 2024
96fe719
Kubernetes Fuckery
tli2 Jun 3, 2024
c633c08
Remove non-commit path speculative dependencies
tli2 Jun 3, 2024
50d7972
Align checkpoint interval of commit workers
tli2 Jun 7, 2024
d89b836
Use a list to avoid large number of memory allocation caused by copyi…
tli2 Jun 7, 2024
e4e4651
Overhaul 2pc clean up logic
tli2 Jun 7, 2024
1c39aa5
Add instrumentation
tli2 Jun 7, 2024
e2c4d86
Remove instrumentation
tli2 Jun 10, 2024
39bc7ff
experiment
tli2 Sep 10, 2024
d485cfc
add error printing
tli2 Sep 10, 2024
3715645
Add async back
tli2 Sep 10, 2024
173a60c
tweaks
tli2 Sep 10, 2024
a7555b3
Add version fix
tli2 Sep 10, 2024
f89a696
Use wall-clock time for checkpointing
tli2 Sep 10, 2024
04b6fb3
apply pruning fix
tli2 Sep 10, 2024
faa6f28
avoid self-updates
tli2 Sep 10, 2024
cd1fd21
Remove priority class
tli2 Sep 10, 2024
b781477
Synchronize using the DprFinder's system clock
tli2 Sep 10, 2024
066fe5b
temporarily disable time sync
tli2 Sep 10, 2024
01fb913
Monitoring
tli2 Sep 11, 2024
dccfdc8
copy to prevent race
tli2 Sep 11, 2024
09256f1
Kuberenetes fuckery
tli2 Sep 11, 2024
6a6c308
more kubernetes fuckery
tli2 Sep 11, 2024
85cb238
I hate kubernetes
tli2 Sep 11, 2024
25a8ffd
Use cached images
tli2 Sep 12, 2024
c777205
changes
tli2 Sep 12, 2024
8af5def
Straight rollback
tli2 Sep 12, 2024
93f8dbe
Add Temporal Baseline?
tli2 Sep 17, 2025
8d6c504
Port OnlineMarketplace and start gRPC/libDSE-based implementation
tli2 Sep 29, 2025
895e251
Single-node TPCC implementation
tli2 Oct 27, 2025
c2fa8fd
async integration with action framework
tli2 Oct 28, 2025
c084dfe
Benchmark harness, data generation, and get started on 2pc-based dist…
tli2 Oct 30, 2025
7f20215
TwoPC implementation
tli2 Oct 31, 2025
4771107
Result reporting
tli2 Oct 31, 2025
fc6bf22
Ugh, RunContinuationsAsynchronously
tli2 Nov 5, 2025
b888ae1
More tests
tli2 Nov 6, 2025
0692c51
New Temporal + Cosmos + k8s baseline setup
tli2 Nov 13, 2025
8c6a118
Remove extra files
tli2 Nov 17, 2025
a458d78
Modify connection setup
tli2 Nov 17, 2025
1940e7b
Add cast
tli2 Nov 17, 2025
f1ec7c2
fix path
tli2 Nov 17, 2025
c7496d3
Fix file structure
tli2 Nov 17, 2025
68849b7
Fix connection strings
tli2 Nov 17, 2025
4726be3
Add missing annotation
tli2 Nov 17, 2025
c0aa829
Fix activity definition
tli2 Nov 17, 2025
cfdd281
Update file paths
tli2 Nov 18, 2025
92b4d02
Unique runs
tli2 Nov 18, 2025
d5ed73d
Rewrite loading logic and increase RU provisioned
tli2 Nov 18, 2025
a057a0d
Fix count and trace
tli2 Nov 18, 2025
b62b174
Tweak CosmosDB settings
tli2 Nov 19, 2025
ab63281
Fix temporal logic
tli2 Nov 19, 2025
f74ba54
New loading logic
tli2 Nov 19, 2025
e326a1d
Use service ID
tli2 Nov 19, 2025
0dbab00
Fix key mismatch
tli2 Nov 19, 2025
c67fe42
Adjust rate
tli2 Nov 19, 2025
5caaa64
More tweaks of workflow logic
tli2 Nov 19, 2025
89ffce1
Tweak max concurrency on Temporal Worker
tli2 Nov 20, 2025
1a5f398
Tune Temporal Concurrency
tli2 Nov 20, 2025
b81c716
Tweak TPC-C benchmark settings
tli2 Nov 24, 2025
1280271
Fix command line arg collision.
tli2 Nov 24, 2025
a90a7f4
Fix off-by-one in data generation
tli2 Nov 24, 2025
cd95665
Fix incorrect shard lookup
tli2 Nov 24, 2025
630748e
Update issuing logic to be one thread per warehouse. Fix off-by-one f…
tli2 Nov 24, 2025
478fde5
Fix concurrent abort bug in two phase commit
tli2 Nov 24, 2025
37154f5
Fix 2pc abort path bug.
tli2 Nov 24, 2025
5968cd5
Tune TPC-C generation parameter
tli2 Nov 24, 2025
6930f42
Fix DprFinder race
tli2 Nov 24, 2025
0d5711b
Actually enable speculation on the shards.
tli2 Nov 24, 2025
a2905f0
Tune 2pc failure recovery baseline
tli2 Nov 24, 2025
a4c43cc
Merge remote-tracking branch 'origin/research-spec' into research-spec
tli2 Nov 24, 2025
04ff863
Tweak Orleans parameters
tli2 Nov 25, 2025
804563c
stupid kubernetes
tli2 Nov 25, 2025
d55f631
fix miconfiguration
tli2 Nov 25, 2025
eb78f52
more kube
tli2 Nov 25, 2025
293acd6
More configuration bs
tli2 Nov 25, 2025
783f40f
fix typo
tli2 Nov 25, 2025
dbea732
More kubernetes bs
tli2 Nov 25, 2025
c88b5db
ughhhh
tli2 Nov 25, 2025
c136f29
DI fix
tli2 Nov 25, 2025
a57be45
Increase timeout
tli2 Nov 27, 2025
09dee13
Suppress transaction for loading
tli2 Nov 27, 2025
de07f7f
Add transaction suppression to interface as well
tli2 Nov 27, 2025
d26c4c0
More transaction fixes
tli2 Nov 27, 2025
74805f3
Reduce parallelism so loader does not trigger throttling
tli2 Nov 27, 2025
f54f7ba
Use memory store for now
tli2 Nov 27, 2025
23d1809
Fix name
tli2 Nov 27, 2025
5ab05b5
Add new failover option for EventProcessing
tli2 Nov 28, 2025
42a9652
Add more failures
tli2 Nov 28, 2025
2bcdbf2
Update data reporting, turn on speculation by default
tli2 Nov 28, 2025
2d836e1
Pull latest
tli2 Nov 28, 2025
773f8f6
Updated throughput measurement logic
tli2 Nov 28, 2025
95bef2e
Add std
tli2 Nov 28, 2025
b45f5a2
Change latency calculation in throughput
tli2 Nov 28, 2025
1489a0e
update
tli2 Nov 28, 2025
b73f9cc
Change result path
tli2 Nov 28, 2025
f7f5ea9
Write stack trace
tli2 Nov 28, 2025
b153803
Update temporal baseline measurement as well
tli2 Nov 29, 2025
3080419
Write latency measurements
tli2 Nov 30, 2025
cefac08
Partial work
tli2 May 4, 2026
720efc7
Update on AE artifacts
tli2 May 5, 2026
e9e7a63
Remove unused benchmark
tli2 May 5, 2026
f08bc9b
Update helm charts
tli2 May 5, 2026
95c5e29
updates
tli2 May 5, 2026
3a1faa1
Minor updates to runner code
tli2 May 6, 2026
9ac6e34
Add notes
tli2 May 6, 2026
012d1fd
Merge upstream/research into osdi-ae (ours strategy)
tli2 May 6, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -203,3 +203,9 @@ cs/libdpr/.idea/
cs/research/darq/.idea/

cs/**/BenchmarkDotNet.Artifacts/

# AE — local credentials for experiment scripts
AE/scripts/env.sh

# macOS metadata
**/.DS_Store
Loading