📝 Add concise dashboard guidance, define actionable ML eval metrics, and note lossy compression in group opinion aggregation

Author: davidgasquez

Dashboards.md

@@ -23,6 +23,7 @@
   - Metadata (owner, related OKRs, TTL, …).
 - Make them so it's easy to go one layer down (X went down in Y location, or for Z new users, etc).
 - Recreate dashboard from first principles periodically.
+- Concise and to the point dashboards! Stuffing a dashboard with a bunch of random metrics is a guaranteed way to waste everyone's time.
 - When plotting a rate, add the top of funnel and bottom of funnel numbers to make sure things are as expected.
 - A large change is not necessarily worth investigating, and a small change is not necessarily benign. What you want to know is if the change is exceptional.
 - Be clear with your stakeholder about whether this is a one-off vs. something that should be referenced more than once.

Machine Learning.md

@@ -26,6 +26,12 @@ These points are expanded with more details in courses like [Made With ML](https
 - Collecting good evals will make you understand the problem better.
 - Working with probabilistic systems requires new kinds of measurement and deeper consideration of trade-offs.
 - Don't work if you cannot define what "great" means for your use case.
+- Good eval metrics:
+  - Measure an error you've observed.
+  - Relates to a non-trivial issue you will iterate on.
+  - Are scoped to a specific failure.
+  - Has a binary outcome (not a 1–5 score).
+  - Is verifiable (i.e. human labels for LLM-as-a-Judge)
 
 ### The [Eval Loop](https://openai.com/index/evals-drive-next-chapter-of-ai/)
 

Politics.md

@@ -76,6 +76,7 @@
   - Architect coordination as a fractal "pace-layering" of ever more specific algorithms, each handling half the remaining complexity with the simplest tool.
 - [A key technical component making democracy work is the secret ballot. No one knows who you voted for, and furthermore, you do not have the ability to prove to anyone else who you voted for, even if you really want to](https://vitalik.eth.limo/general/2025/04/14/privacy.html).
 - Voting is a preference aggregation method. Preferences must be aggregated across multiple individuals to determine a collective decision or ranking. This process is central to social choice theory, which provides a mathematical foundation for preference aggregation.
+- Every time you derive an opinion from a group, you are doing a lossy compression of each individual opinion. How you do it (preference and meta-preference aggregation) is in itself an opinion / choice.
 
 ## Interesting Ideas