msc placeholder: Cool thesis project - with distinction (live code and protocol evolution)

[synctext]

• "Self-Evolution System Development" as interest
• Interested in: msc placeholder: do cool stuff for thesis #8645
• Doing a high workload (internship only in '25)
• Crawl experience
• FinTech experience
• Also in the (overloaded) security track (no certificate ambition)

Required to graduate before July/Aug 2026.

Some reading:

• inspiration: Cloudomate, Plebnet https://repository.tudelft.nl/file/File_32d3b969-225e-462a-a0a6-1e1ce4351cb8
• self-evolving Internet-deployed intelligence #4057 (comment) 2025 update, related work is describing our 2018 concept: "Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents", https://arxiv.org/abs/2505.22954 Our 2018 idea is to use crowdsourcing, thus using directed evolution. This cool 2025 idea proposes to use a formal no-humans involved. The Gödel machine proposed a theoretical alternative: a self-improving AI that repeatedly modifies itself in a provably beneficial manner. Unfortunately, proving that most changes are net beneficial is impossible in practice. We introduce the Darwin Gödel Machine (DGM), a self-improving system that iteratively modifies its own code (thereby also improving its ability to modify its own codebase) and empirically validates each change using coding benchmarks. Inspired by Darwinian evolution and open-endedness research, the DGM maintains an archive of generated coding agents. It grows the archive by sampling an agent from it and using a foundation model to create a new, interesting, version of the sampled agent.
  PowerPlay is a very simple, very general architecture for an AI agent. It uses: Self-modification, Auto-generated problems, Some decidable notion of “regression”.

Next step: 3-brainstorms for thesis direction (e.g. like: #7064 (comment))

[synctext]

Concrete Must-have for Darwin Gödel machine. Good target for 1st agile sprint.

Very early sketch of workflow. Goal is distributed and unstoppable pip upgrade.

Take something like Tribler. Alert that new version is available. Get hash of new version. Download using Bittorrent. Download updated version of Tribler using Bittorrent. Install new code. Run.

update See this Mac Github project auto-update-magic

update2 When do we need a fixed Problem Description? 2025 for Q2 light prototyping, this experience then leads to the concrete direction. This then has realism plus early running code. Real code and thesis experiment for whole of 2026.
Goal for Q2 2025 is to get live Python code patching operational. Focus on mature IPv8 Python overlay code. Thus giving a self-organising distributed system the ability to do runtime community.py upgrading. The emergent property we seek is a self-evolving system. Keep it as simple as possible. Brainstorming further: IPv8 overlay network with two peers, Peer A and Peer B. They exchange keep-alive messages and form a live network. Peer A community.py is upgraded to another version with support for a new protocol message. Peer A keeping running Python continuously. After a time expires Peer A starts the live patching process. It stops the IPv8 community, modifies the source code, compiles, exec()s, starts IPv8 community with patched code, and discovers Peer B again (no cache needed for network state). Peer A then sends the usual handshake message + 2nd UDP packet with version request message. See Internet-deployed code example from Tribler.

For 2026 we then need to create a glorious singularity emulator/s. For a lot of thoughts, see this HackerNews discussion and linked blog post. AI That Can Improve Itself - A deep dive into self-improving AI and the Darwin-Gödel Machine

[synctext]

Darwin Gödel Machine (DGM) requires a scoring function for intelligence, ARC AGI is a good fit. Basically we already see crowdsourcing of solutions on Github. But inefficient, centralised, and at small scale. We aim to build a generic democratic bounty-based system for this. Basically a proof-of-principle of Web4: combining for the first time AI, democracy, and money.

• SOAR Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI
• Pang: this approach scores 77.1% on ARC-AGI-1 and 26.0% on ARC-AGI-2, both beating results of frontier models and previous bespoke systems
• Python generator DreamCoder: library module containing program synthesis utilities and data structures. Initially developed for DreamCoder
• Rust attempt at the ARC-AGI prize 2025
• RE-ARC: Reverse-Engineering the Abstraction and Reasoning Corpus
• https://github.com/google/ARC-GEN#the-arc-gen-100k-dataset
• https://github.com/jfpuget/ARC-AGI-Challenge-2024
• ARC-AGI Hierarchical Reasoning Model Analysis, H100 required
• ARC AGI without pre-training just fake it woth compression
• STATE-OF-THE-ART: Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents

[Andrei32Ionescu]

• DGM paper feels a bit underwhelming. I find it relatable to the idea of someone else creating a great new model, and then another person writes a paper where the claim to be pushing the cutting edge by creating an ensemble with the model. It is new research yes, but the goal was low-hanging to begin with.
• live Python patching discussion with interesting takes on it.
• What I am currently missing: what is the research goal of the overall 2026 thesis project? What is the gap that we are aiming to fill? The current state leads me to believe we are taking existing tools and sticking them together, which yes would create a compelling engineering feat, but we need a research feat instead.

[synctext]

Correct, the cardinal problem for each HONOR project is finding something innovative.

Research question for all 6 students is simply: how to realise the dream of Darwin Gödel Machines.
Very open ended, not new, very practical: can this theoretical stuff ever Get Real, "practical flavour research question". Towards artifical-life system with biological property of self-reproduction, emergent morphogenesis and even autopoietic embodiment.

State-of-the-art:
Incentivizing Permissionless Distributed Learning of LLMs.
cyberweapon Stuxnet has self-upgrade ability, very rare system property

We can start with filling in the research question and the science along the way: April even! Goal is creating an autonomous self-referential, self-hosted, self-improving, and permissionless self-organising inteligence. Start with Python, run-time upgrade of Community.py, scoring ability of ARC-AGI (and/or application utility as Youtube-alternative), ability to accept code submissions publically, provide proof-of-contribution, payout of bounty, make scalable to 8 billion users, performance analysis, attack-resilience, epic Sybil attack, MeritRank mathematics of trust, field trail, build social awareness, first iteration of in the wild improvements, closed loop economy for intelligence mining, DONE. 😆

update:
Take standard Python IPv8 overlay as starting point.
Mission for next sprint: single incoming UDP message with Bittorrent SHA1 magnet link of Community2.py. Download with Libtorrent, load the Python function, switch, trust this stranger with complete control of your computer, run Community2 protocol.

[Andrei32Ionescu]

Update ahead of the next meeting:

I have completed the mission for the sprint. In the standard Python IPv8 overlay, I created a community of 2 peers which exchange Lamport clock messages. At clock value 1000, one of the peers starts seeding a new community message handler function and sends the magnet link to the other peer, which uses Libtorrent to download this new handler and hot-swaps it and starts handling messages based on the code it received from the first peer. This works with no reload as expected. I am currently not re-writing the community code, rather adding the new handler in a separate file and replacing the function call within the runtime namespace from the old handler to the new handler. This means a restart of the program will still use the original handler, but I believe modifying the original file itself wouldn't prove to be an issue depending on the direction we decide to take for the next steps.

[synctext]

Great 👍👍👍
Please push your code to a public GitHub repo. See other students progress: #8664 (comment)

• Theory: predicting code improvements is undecidable. Can humans vote on it or automate it?
• How will our self-evolving feedback loop operate?
  • ARC AGI scoring is specific for reasoning models. Open question: how to estimate end2end system performance?
  • Multi-dimensional scoring suite? 1 number for general system performance. 1 number for passing the unit tests😆 1 number for ARC AGI scoring.
  • We need co-evolution of the evaluation function❗ Self-evolving systems require that the evaluation function also evolves in complexity: co-evolution. We can see this with the maxing/cheating of the ARC AGI score, requiring increasing V1, V2, and V3 sophistication. 2019: Scenario co-evolution for reinforcement learning on a grid world smart factory domain
• Can we avoid A/B testing on a live network with various code mutations? Can we judge the quality of a pull request automagically or with manual voting? Github PR has entire culture around this with strict "PUSH authorisation". For instance, only a select few can push core to the Bitcoin reference code.
• Lets aim for a April running code and field trail in May. Measure in June and process all results. Defend by Friday 3rd of Juli.

ToDo: get ARC AGI going on state-of-the-art NVIDIA hardware. Integrate ARC AGI solution, automated fitness scoring and seamless switching between 2 SHA1 code bases operational. Scenario:

1. live operational network with know test score
2. Incoming UDP message with a new code base (not a patch yet, keep simple?)
3. download this code with Libtorrent, determine the ARC AGI score.
4. If new code bases has superior score: hotswap!

Roadmap for thesis: self-evolving system with state-of-the-art intelligence scoring. We have 5 operational code bases for intelligence in our self-organising superorganism. Our co-evolved performance suite has a detailed performance analysis for each of these solutions.

Method	ARC-AGI-2 Accuracy	Approach
ARC-AGI Solver	38.5%	Hybrid symbolic-neural
DreamCoder	31.2%	Program synthesis
GPT-4 Vision	27.8%	Large multimodal model
LARC	25.4%	Inductive logic programming
Human baseline	84.2%	Manual reasoning

Reasoning is the core of intelligence. At another level, relevant to our use-case of "solving search" are the quality of the vector embedding models. MTEB is a Python framework for evaluating embeddings and retrieval systems for both text and image. Third layer in our system is capability of our memory subsystem. This project contains tools to benchmark various implementations of approximate nearest neighbor (ANN) search for selected metrics..

For quality control it is essential that we do not have performance regression in one component, when others are improved. Keep code changes isolated to a single module to improve. Minimize global architectural changes, just like Linux kernel policies dictate.

[Andrei32Ionescu]

Code pushed to this fork: https://github.com/Andrei32Ionescu/msc-thesis

Added files:

• doc\basics\identity_tutorial_integration\source\Community2.py - new community with a slightly different handler

Changed files:

• doc\basics\identity_tutorial_integration\main.py - prototype logic for torrent download and code hot-swap

[Andrei32Ionescu]

I have delved deeper into the available resources. Lesson learned: always read the appendix even if it is 60 pages long. Around page 30 we get a cost estimate for a single run of DGM:

• a single run of the DGM on SWE-bench costs USD 22,000 and takes about 2 weeks

This makes it impossible to run such an instance on a peer node. Perhaps the proposed framework could still be usable in a distributed setting, with each note containing one agent of the archive, but then we run into the issue that old agents need to be retained and as such some peers are locked out of the latest versions of the coding agents. This means the community becomes heavily heterogenous which I believe misses the point.

A possible solution I can see here (which I need to look into more) is having each peer keep their own coding agent, and participate in a collaborative evolution process rather than independent evolution.

Specifically we would have:

• Evolution nodes (small percentage with capable hardware) generate modification proposals
• All peers participate in distributed testing and fitness evaluation
• Evolving versions maintained across the network

Also in terms of using ARC-AGI to validate agent improvement, there is a fundamental problem: ARC-AGI tests abstract reasoning on visual/spatial puzzles, not coding ability. Using it creates a severe misalignment:

• We are modifying coding agent's ability to write/improve code
• Testing pattern recognition on grid puzzles
• As a result, the agent optimizes for puzzle-solving rather than self-improvement

[synctext]

Good find!

No SWE-bench. No ARC-AGI.

Was looking more towards humans which propose PR in our self-hosted system. Like on GitHub there is an automated (distributed?) scoring function with performance analysis and final score. Bounty payout to developer. We might need a quick local eval function, then one for heavy cluster, and finally a distributed run on numerous node for good scoring PRs.

Seems ARC AGI is too ambitious for now! No 22k€ in compute yet.

For a self-evolving learn-to-rank model we have excellent datasets for benchmarking. Live switching between LTR models, with relentless incremental score improvements. Like Linux, 20 years till world dominance. Fits with other master thesis works.

Popular Learning-to-Rank (LTR) Benchmarks

Dataset	Provider	Size (Queries / Pairs)	Highlights
MSLR-WEB30K	Microsoft	30,000 / 3.7M	The industry standard. 136 features (TF-IDF, PageRank, etc.). Ideal for GBDT models.
Yahoo! LTRC	Yahoo	29,900 / 700k	High-quality "Set 1" and "Set 2." Frequently used for benchmarking LambdaMART.
Istella LETOR	Istella	33,000 / 10M+	One of the largest public datasets; best for testing high-scalability neural rankers.
LETOR 4.0	Microsoft	~2,500 / 1M	Derived from TREC (Gov2). Useful for academic baselines and smaller experiments.
MS MARCO	Microsoft	1M+ Queries	Focuses on Neural Ranking. Uses real Bing queries and passage-based relevance.

See if these can be used for ranking libraries (like LightGBM, XGBoost, or RankLib). Note they expect the LibSVM / LETOR format.

[Andrei32Ionescu]

Will look into this! An idea I had to still involve DGMs is to use them as a sort of self-evolving evaluator for certain peers which are more compute-able, through using local models (Ollama - requires only local memory for model storage, no costly LLM api calls necessary in this way). Otherwise for the lighter nodes I will look for similarly light evaluation methods. If we don't run the coding / swe benchmarks, maybe we can still use DGM without needing the expensive and lengthy compute.

[Andrei32Ionescu]

So as I understand it, we want a system where humans can propose PRs with for example new features, which are reviewed at local level (non-compute intensive peers, let's call them light peers) through benchmarks/heuristics. Then for more compute-able clusters (let's call them heavy peers) we can attempt to run a more heavy validation, which could for example mean an LLM agent or some other trustworthy validation method. Then we can have a distributed run where we can collect data about user satisfaction as the features spread through the network.

I am a bit unclear about the vision for the LTR models. Do we want users to train and propose LTR models which we can then add to a suite of models in a Multi-Armed Bandit, which is an existing (apparently less popular, usually a single underlying LTR model is being tuned) way of live switching between LTR models?