Mac notes

Author: Bharathi Ramana Joshi

java-concurrency.md

@@ -57,8 +57,15 @@
    + `size` and `isEmpty` are approximate
    + only when application needs to lock Map for exclusive access the
      ConcurrentHashMap is not correct replacement
+- `ConcurrentHashMap` interface:
+   + `putIfAbsent` (atomic check and insert)
 - Copy-on-write collection
 - `BlockingQueue`: blocks threads on queue underflow and overflow.
+- `BlockingQueue` interface:
+   + `add`
+   + `offer`
+   + `take`
+   + `poll`
 - Implementations of `BlockingQueue`:
    + LinkedBlockingQueue
    + ArrayBlockingQueue
@@ -67,14 +74,66 @@
      participate in hand-off from producer; otherwise producer will block
 - Work stealing pattern: consumers can consume from other consumer's queue
 - Deque: double eneded queue used for work stealing pattern
-- `CountDownLatch`: `await` and `countDown`
-- `Semaphore`: `acquire` and `release`. Fair if FIFO is followed in granting
-  the permits.
+- Synchronizer: object used to coordinate control flow of threads based on
+  shared state of syncrhonizer.
+
+- `Latch`: synchronizer used to delay progress of thread until latch reaches
+  terminal state.
+- `CountDownLatch` interface:
+   + `await`
+   + `countDown`
+
+- `CyclicBarrier`: synchronizer that allows a set of threads to all wait for
+  each other to reach a common barrier point repeatedly.
+- `CyclicBarrier` interface:
+   + `await`
+
+- `Semaphore` used to control number of threads that can access a certain
+  resource (e.g. file descriptor) or perform a given action at a time.
+- `Semaphore` controls a set of virtual permits which threads can acquire and
+  release.
+- `Semaphore` interface:
+   + `acquire`
+   + `release`.
+- `Semaphore` is fair if FIFO is followed in granting the permits.
 
 # Chapter 6
-- Executor: separates task submission from task execution.
+- Disadvantages of unbounded thread creation:
+  + Threadlife cycle overhead
+  + Idle thread increase resource consumption (CPU, garbage collection, etc)
+  + JVM has implementation limit on number of allowed threads, will throw
+    `OutOfMemoryError` upon hitting this limit which is hard to recover from
+- `Executor`: separates task submission from task execution.
+- Tasks are wrapped in Runnable or Callable and submitted to an executor.
+- Lifecycle of a task submitted to an executor:
+  1. Created
+  2. Submitted
+  3. Started
+  4. Completed
+- ExecutionException: exception in which a tasks exception gets wrapped when a
+  thread calls task's future.get
 - RejectedExecutionException: exception thrown when a task submitted to an
   executor is not run.
+- `Executor` interface:
+   + `execute(Runnable)`
+
+# Chapter 7
+- Threads in blocked state throw `InterruptedException`, if a thread is not
+  blocked state and gets interrupted, then the thread's interrupted status flag
+  is set but no exception is thrown.
+- Swallowing the `InterruptedException` is bad practise as swallowing makes it
+  not possible for methods up the call stack to interrupt a thread.
+- `Thread.interrupted()` fetch and clear the interruption status flag.
+- `Thread.interrupt()` set the interruption status flag and cause an
+  `InterruptedException` in invoked thread if blocked.
+- Either a thread should propogate the `InterruptedException` or reset the
+  interruption status using `Thread.currentThread().interrupted()`
+- Tasks do not execute in threads they own; they borrow threads owned by a
+  service such as a thread pool. Code that doesn’t own the thread (for a thread
+  pool, any code outside of the thread pool implementation) should be careful
+  to preserve the interrupted status so that the owning code can eventually act
+  on it, even if the “guest” code acts on the interruption as well.
+- You should know a thread’s interruption policy before interrupting it.
 
 # Chapter 14
 - Use `wait`, `notify`, `notifyAll` on any `Object` for conditional variables.

k8s.md

@@ -14,6 +14,8 @@
   * Platform (cloud/virtual/physical)
   * Fault tolerance
 - Manifest: initial configuration.
+
+# 101
 - Cluster -> Node -> Pod -> Container -> Application/microservice.
 - Pod: abstraction over containers to provide a uniform communication interface
   irrespective of underlying container (may be docker or something else).
@@ -41,4 +43,8 @@
   Pods and ReplicaSets. You describe a desired state in a Deployment, and the
   Deployment Controller changes the actual state to the desired state at a
   controlled rate
+
+# Networking
+- Each pod gets cluster-wide unique IP address
 - 
+

kafka.md

@@ -0,0 +1,57 @@
+# 101
+- Broker: a single Kafka server, which receives messages from producers,
+  assigns offsets to them, and commits the messages to storage on disk.
+- Cluster: set of brokers.
+- Topic: collection of partitions
+- Partition: single commit-log. Each partition can be hosted on a different
+  server.
+- Replication: multiple copies of partitions across brokers to improve
+  fault-tolerance.
+- More brokers => more time required to elect leader.
+- Kafka is organized into topics, each topic has partitions, partitions have
+  replicas stored on brokers.
+- Each broker stores hundreds/thousands of replicas belonging to different
+  topics and partitions.
+- A partition is owned by a single broker in the cluster, and that broker is
+  called the leader of the partition. All producer and consumer requests for a
+  partition go through the partition's leader to ensure consistency.
+- ZooKeeper is used by Kafka to store metadata about the Kafka cluster and the
+  consumer client details.
+- ZooKeeper is a centralized service for maintaining configuration information,
+  naming, providing distributed synchronization, and providing group services.
+  TODO explore ZooKeeper
+- The key of a record determines which partition the record will be written to.
+  All records with the same key are written to the same partition. Useful to
+  distribute records among processes reading specific partitions.
+- If a record's key is null, the record is written to a partition at random
+  as determined by a round robin algorithm.
+- As long as the number of partitions in a topic are not changed, records with
+  the same key will be hashed to the same partition. If a new partition is
+  added, the older records will not be moved but new records will get hashed
+  according to the new number of partitions.
+- Kafka's defualt partition algorithm is independent of the Java library
+  version.
+
+# Producers
+- Trade-offs to consider when configuring Kafa:
+  1. Is every message critical or can we tolerate loss of messages? Are we OK
+     with duplicating messages?
+  2. Are there any strict latency or throughput requirements we need to
+     support?
+- A separate thread is responsible for batching records to be sent to a
+  particular topic and partition.
+- If a broker is able to successfully write the messages, the broker returns a
+  `RecordMetadata` object with the topic, partition, and the offset of the
+  record within the partition; otherwise the broker returns an error.
+- Three ways to use a producer:
+  1. Fire-and-Forget
+  2. Synchronously: get a future
+  3. Asynchronously: pass a callback
+- Producer configurations:
+  1. 
+
+# Consumers
+- Consumer group: set of consumers of a topic in which each consumer receives
+  records from a disjoint subset of partitions.
+- If there are more consumers in a single group with a single topic than we have
+  partitions, some of the consumers will be idle and get no messages at all.

microservices.md

@@ -0,0 +1,2 @@
+# 101
+- Microservices do not share DBs

mlir-toy.md

@@ -1,6 +1,13 @@
 # Chapter 1
 - Dialect: grouping functionality to extend MLIR system by adding new operators,
-  attributes, and types.
+  attributes, and types. A dialect consists of:
+  + A prefix / namespace
+  + A list of custom types, each its C++ class
+  + A list of operations, each its name and C++ class implementation
+    - Verifier for operator invariants (e.g. toy.print must have a single operand)
+    - Semantics (no-side-effects, constant-folding, CSE-allowed)
+  + Passes: analysis, transformations, dialect conversions
+  + Possibly custom parser and assembly printer
 - Operations are defined by
     + Name
     + List of SSA operand values
@@ -10,6 +17,7 @@
     + Source location
     + List of successor blocks
     + List of regions
+- Region: list of basic blocks. Approximately CFG.
 - Traits: mechanism to inject additional behavior (accessors, verification, etc)
   into operators.
 - MLIR allows undefined operations to be manipulated through opaque `Operaton`