Index CAT-21 Ordinals

.cursorignore

@@ -0,0 +1,13 @@
+*.redb
+.idea/
+.vagrant
+docs/build
+env
+fuzz/artifacts
+fuzz/corpus
+fuzz/coverage
+fuzz/target
+*.log
+target
+test-times.txt
+tmp

.cursorrules

@@ -0,0 +1,52 @@
+# CAT-21 Ordinals Best Practices and Development Rules
+
+Purpose: To standardize development for LockTimeOrdinals on the ord client.
+You are an expert Rust developer with deep knowledge of Bitcoin and the Ordinals protocol.
+
+## General
+- Ensure the code follows Rust's best practices and idiomatic style.
+- Write meaningful and concise comments for complex logic, especially around CAT-21 logic.
+- Avoid hardcoding values; define constants where applicable (e.g., LOCK_TIME_THRESHOLD).
+- Extend existing indexer logic to track LockTimeOrdinals without disrupting other ord features.
+- Use code-comments like "//  CAT-21 😺 - START" and "//  CAT-21 😺 - END" to indicate the start and end of the CAT-21 logic.
+- The code should be easily mergeable with the main branch of the ord client. Avoid changing existing line of code if possible, for better mergeability.
+- Chose to keep LTO logic modular and extensible for potential future protocols.
+- Focused on minimizing changes to the core `ord` client to maintain compatibility.
+- Don't remove existing code comments, keep them as they are if still relevant, update them if functionality has changed.
+
+- **nLockTime Handling**:
+  - Track transactions with nLockTime values between 0 and a predefined upper limit LOCK_TIME_THRESHOLD (e.g., 1000).
+  - The satoshis / sats are called LockTimeOrdinals ("LTO").
+  - A LockTimeOrdinal is always created / minted for the the first satoshi of the first output of the transaction.
+  - LTOs are tied to the first sat of the first output, ignoring whether it is an OP_RETURN output,
+    but introduces cases where LTOs are "burned" on creation
+
+- **Integration with ord Client**:
+  - Reuse existing SatPoint tracking mechanisms wherever possible to avoid duplicating logic.
+  - Mimic inscription tracking logic to extend functionality for LockTimeOrdinals.
+
+# Database and Indexing
+Rules for interacting with the database:
+- Use `WriteTransaction` for all atomic operations related to indexing.
+- Commit transactions only after all operations for a LockTimeOrdinal are complete.
+- Ensure database schemas are extendable to accommodate future changes.
+- Ensure all LockTimeOrdinal entities are stored in `LOCKTIME_ORDINAL_TABLE` and `LOCK_TIME_TO_NUMBER` and other tables.
+
+
+# Error Handling
+Guidelines for handling errors:
+- Use `Result` types consistently for error propagation.
+- Provide detailed error messages, especially when dealing with database transactions.
+- Write unit tests for edge cases, such as transactions with no outputs or invalid lockTime values.
+
+# Testing
+- Write robust tests to validate all features
+- Use mock transactions for testing to avoid reliance on live Bitcoin RPC data.
+
+# Documentation
+Maintain clear and concise documentation:
+- Document all new structs, enums, and functions related to LockTimeOrdinals.
+- Keep in mind the `docs-cat21/developer-howto.md` file for onboarding developers. Update it when relevant.
+- Keep in mind the `docs-cat21/development-summary.md` file for for a detailed documentation of this project and notable changes to the ord client. Update it when relevant.
+- Include inline comments where complex logic is implemented.
+- Further Documentation, see the `docs-cat21` folder.

README.md

@@ -1,6 +1,17 @@
-`ord`
+`cat21-ord`
 =====
 
+![Orange Banner](https://raw.githubusercontent.com/ordpool-space/cat-21/main/assets/cat21-banner.svg)
+
+This is a hacked version of [ord](https://github.com/ordinals/ord), that indexes [CAT-21 ordinals](https://github.com/ordpool-space/cat-21). Meow! 😺
+
+In general, there is no difference to the normal ord, except that cats are also indexed.
+If you want to learn how to set up an ord instance, [refer to this guide](https://gist.github.com/hans-crypto/30d05b9dcb3c05940e9a8db2e365da1e) or any other guide in offical ord Discord. Of course you need the executable from this repo if ord should also index cats.
+
+If you are a coder (or want to run/build the executable), [read on here](README_developer).
+
+---
+
 `ord` is an index, block explorer, and command-line wallet. It is experimental
 software with no warranty. See [LICENSE](LICENSE) for more details.
 

deploy-ord-dev/bitcoind.service

@@ -0,0 +1,13 @@
+[Unit]
+Description=Bitcoin Daemon
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+ExecStart=/home/linuxbrew/.linuxbrew/opt/bitcoin/bin/bitcoind
+User=ord-dev
+Restart=on-failure
+Type=simple
+
+[Install]
+WantedBy=multi-user.target

deploy-ord-dev/ord.service

@@ -0,0 +1,19 @@
+[Unit]
+Description=Ord Daemon
+After=network.target
+
+[Service]
+AmbientCapabilities=CAP_NET_BIND_SERVICE
+Environment=RUST_BACKTRACE=1
+Environment=RUST_LOG=info
+ExecStart=/home/ord-dev/cat21-ord/target/release/ord \
+  --index-sats \
+  --index-addresses \
+  --no-index-inscriptions \
+  server \
+  --http
+User=ord-dev
+Restart=on-failure
+
+[Install]
+WantedBy=multi-user.target

docs-cat21/0-developer-howto.md

@@ -0,0 +1,225 @@
+# Developer HowTo for `cat21-ord`
+
+Super quick howto to jump into coding (or building the executable).
+
+
+## 0. Prerequisites
+
+VS Code and/or Cursor is recommended for development.
+Read more about Rust development with VS Code here:
+https://code.visualstudio.com/docs/languages/rust
+
+
+## 1. Clone the repo
+
+```sh
+cd ~
+git clone https://github.com/ordpool-space/cat21-ord.git
+cd cat21-ord
+```
+
+
+## 2. Debugging with VS Code (while using a remote Bitcoin RPC) on a Mac
+
+> You want this option when you want to develop new features.
+> On a normal working machine, with normal hardware (less than 64 GB RAM).
+
+1. Install the VS Code extension rust-analyzer: https://marketplace.visualstudio.com/items?itemName=rust-lang.rust-analyzer and this if you are on a Mac: https://marketplace.visualstudio.com/items?itemName=vadimcn.vscode-lldb
+
+2. Install rust `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh`, follow the instructions to configure your current shell.
+
+3. `rustc --version` must work!
+
+4. `cd cat21-ord` into the repo
+
+5. `cargo build`, once built, the `ord` binary (debug build) can be found at `./target/debug/ord`.
+
+6. `cargo run`
+
+7. You'll need to enable the setting `Debug: Allow Breakpoints Everywhere`, which you can find in the Settings editor (`⌘,`) by searching on 'everywhere`.
+
+8. Generate a new `launch.json` by trying to debug (VS Code should create one for you)
+
+9. Add a configuration like this to your `launch.json`
+
+```json
+{
+    "version": "0.2.0",
+    "configurations": [
+        {
+            "type": "lldb",
+            "request": "launch",
+            "name": "Debug executable 'ord' INDEX-SATS (Localhost RPC Setup)",
+            "cargo": {
+                "args": [
+                    "build",
+                    "--bin=ord",
+                    "--package=ord"
+                ],
+                "filter": {
+                    "name": "ord",
+                    "kind": "bin"
+                }
+            },
+            "args": [
+              "--index-sats",
+              "--index-addresses",
+              "--no-index-inscriptions",
+              "--bitcoin-rpc-username=xxx",
+              "--bitcoin-rpc-password=yyy",
+              "server"
+            ],
+            "cwd": "${workspaceFolder}"
+        },
+        {
+            "type": "lldb",
+            "request": "launch",
+            "name": "Debug executable 'ord' INDEX-SATS (Remote RPC Setup)",
+            "cargo": {
+                "args": [
+                    "build",
+                    "--bin=ord",
+                    "--package=ord"
+                ],
+                "filter": {
+                    "name": "ord",
+                    "kind": "bin"
+                }
+            },
+            "args": [
+              "--index-sats",
+              "--index-addresses",
+              "--no-index-inscriptions",
+              "--bitcoin-rpc-url=192.0.2.1:8332",
+              "--bitcoin-rpc-username=xxx",
+              "--bitcoin-rpc-password=yyy",
+              "server"
+            ],
+            "cwd": "${workspaceFolder}"
+        }
+    ]
+}
+```
+
+## 3. Building & Running a `cat21-ord` server (not a normal ord) on Linux
+
+> You want this option when you want to build the full damn index.
+> On a beafy server, 64 GB RAM, 2TB SSD. Do not try lower hardware!
+
+1. Bitcoin must be up and running, see `1_bitcoin-guide-for-mac.md` or `1_bitcoin-guide-for-linux.md`.
+
+    Feel free to use the files in `deploy-ord-dev` for a fast setup:
+
+    ```
+    sudo cp ./deploy-ord-dev/bitcoind.service /etc/systemd/system/
+    sudo systemctl daemon-reload
+    sudo systemctl enable bitcoind
+    sudo systemctl start bitcoind
+    sudo systemctl status bitcoind
+
+    tail -F -n 10000 ~/.bitcoin/debug.log
+    ```
+
+2. Install rust via curl (rustup) or homebrew, both should work:
+
+    * Rustup is officially recommended.
+      `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` to install.
+      Follow the instructions to configure your current shell.
+      But then you realise that you still need to take care of the dependencies, such as OpenSSL.
+      So you might want to execute `rustup self uninstall` to uninstall, and use brew again! 😇
+      Check your path via `echo $PATH` to see if it's clean again.
+
+    * `brew install rust` requires no additional steps
+
+3. `rustc --version` must work!
+
+4. `cd cat21-ord` into the repo
+
+5. `cargo build --release`, once built, the `ord` binary (release build) can be found at `./target/release/ord`.
+
+6. You can now execute the CAT-21 version of `ord`:
+
+    * Running the binary directly: `./target/release/ord --version`
+    * Running via cargo: `cargo run --release -- --version` (If you’ve made changes to the code, cargo run will automatically rebuild the project before running it, ensuring you're always running the latest version of your code. If you don't make changes to the code, both options are fine.)
+
+7. Start the indexing:
+
+   ```sh
+   ./target/release/ord \
+     --index-sats \
+     --index-addresses \
+     --no-index-inscriptions \
+     server \
+    --http-port 8080
+   ```
+
+9. Run the ord daemon permanently:
+
+    ```sh
+    sudo cp /home/ord-dev/cat21-ord/deploy-ord-dev/ord.service /etc/systemd/system/
+    sudo systemctl daemon-reload
+    sudo systemctl enable ord
+    sudo systemctl start ord
+    sudo systemctl status ord
+    ```
+
+    To view the full logs for your `ord` service managed by `systemd`, you can use `journalctl`, which collects logs for all systemd-managed services.
+    This will show the complete log history:
+
+    ```sh
+    sudo journalctl -u ord
+    ```
+
+    Or follow the logs in real-time (like `tail -f`):
+
+    ```
+    sudo journalctl -u ord -f
+    ```
+
+    To delete logs older than a specific time period, such as 1 week:
+    ```
+    sudo journalctl --vacuum-time=1w
+    ```
+
+    To confirm the size of logs:
+
+    ```bash
+    sudo journalctl --disk-usage
+    ```
+
+    You can limit the size of `journalctl` logs to prevent them from growing indefinitely.
+
+    To set a Disk Space Limit, edit the `journald.conf` file:
+
+    ```bash
+    sudo nano /etc/systemd/journald.conf
+    ```
+
+    Set or modify the following values:
+
+    ```ini
+    SystemMaxUse=200M
+    SystemKeepFree=50M
+    SystemMaxFileSize=50M
+    ```
+
+    - **`SystemMaxUse`**: Limits the total space used by journal logs.
+    - **`SystemKeepFree`**: Ensures this much disk space remains free.
+    - **`SystemMaxFileSize`**: Limits the size of individual log files.
+
+    Save and exit the file. Then restart the `systemd-journald` service:
+
+    ```bash
+    sudo systemctl restart systemd-journald
+    ```
+
+    Manually clean up logs to free space immediately:
+
+    ```bash
+    sudo journalctl --vacuum-size=200M
+    ```
+
+
+### Personal Notepad with intersting links
+
+* [Add option to retain sat index for spent outputs #2999](https://github.com/ordinals/ord/pull/2999)