Skip to content
@csorchard

Concurrent Systems Engineering

single node @dborchard → multi-node @dsorchard → multi-core @csorchard

🌱 The Foundation

Action Precedes Motivation

Read More

Core Data Structure

  • Regular Data structure
    • Randomized: Quick Sort, Treap, SkipList
    • Misc: List, Stack, Queue, Priority Queue (taken from gods)
    • Sorted: Binary Search Tree
  • Systems Data structure
    • Hash-Based: Probing vs Chaining in Hashmap
      • Approximate: Bloom Filter, Count Min Sketch, HyperLogLog
      • Rebalancing Map: Consistent Hashing, ChordDHT, Range Based Partitioning, Extendable Hashing
      • Hash Functions: Cuckoo Hash, Murmur Hash,
    • Sorted: BTree, Binary Search Tree, Treap, Red Black Tree, B Epsilon Tree
    • Multi Dimension: KD Tree, Z-Index, Hilbert Curve
    • Sampling: Reservoir Sampling
    • Stream: Sliding window
    • Difference: Merkal Tree
    • Time: Hashed Wheel Timer
  • Concurrent Data Structures:
    • Skip List: Regular, Lazy, LockFree, Arena Skiplist
    • BTree
    • Radix Tree: ART, VART, Trie
    • Fast Succinct Trie: SuRF
  • External Memory Algorithms
    • External Hashing
    • External Sorting

Performance engineering tools

  • Performance Analysis
    • Heap Dump
    • Trace
    • Jmeter, YCSB
    • VRPC (vector clock)
    • Heap View
    • Write Benchmark
    • Lotsaa
    • GC frequency, threshold
    • Git Bisect
    • Explain Analyze
    • OpenTelemetry Trace

Advanced Data Structures

  • Arena based skip list
  • Roaring BitMap
  • Zone Map

🌿 The Plant

Read, Extract components, and Learn the inner workings.

Read More

From MatrixOrigin

  • MatrixOrigin Systems Data Structures Usage
    • HyperLogLog: Used to find NDV before writing segment meta header (MO Write SST with Metadata including NDV)
    • Bloom Filter: Used for fast range scan through blocks (MO Runtime Filter in Storage Engine)
  • MatrixOrigin Join
    • Hash Map
    • Memory Pool
    • Reservoir Sampling

Misc

  • Storage Engine: WAL, Btree, CoW Btree, LSM Tree
  • Memtable using Skip List

🌳 The Tree

Pick a favorite storage engine/main memory system and shrink.

Read More

Storage Engines

  • Dgraph ristretto
  • Dgraph badger: WiscKey, Txn, Memory allocation part in pkg z.
  • BigCache:

💧 The Resources

Revisit papers/slides. Build counterexamples. Think like a scientist.

Read More

Reading

Read

👨‍🌾 Cultivating Knowledge

Write what you understood. (Maybe later)

Read More

Teachings

  • Paper Reading
    • A method for implementing lock-free shared data structures
    • WiscKey: Separating Keys from Values in SSD-conscious Storage
    • A Critique of Snapshot Isolation

🥭 The Fruit

Mark your achievements.

Read More

Core Implementations

Pinned Loading

  1. sds_use sds_use Public

    Systems Data Structure Usages in Real world projects

    Go

  2. sds sds Public

    Data structures and algorithms I came across in database/systems programming.

    Go

  3. perf_analysis perf_analysis Public

    Performance analysis of Golang Project

  4. skiplist_impls skiplist_impls Public

    Learning Regular, Lazy, Lock Free, Arena skiplist in Golang

    Go 2

Repositories

Showing 10 of 30 repositories

Top languages

Loading…

Most used topics

Loading…