diff --git a/FOG_OF_WAR_GUIDE.md b/FOG_OF_WAR_GUIDE.md index c2180451d..1b22d9c6d 100644 --- a/FOG_OF_WAR_GUIDE.md +++ b/FOG_OF_WAR_GUIDE.md @@ -258,16 +258,14 @@ go infinite ``` In a fog FEN: -- `?` represents unknown/fogged squares +- `?` (or `*`) represents unknown/fogged squares - Visible pieces are shown normally (e.g., `p`, `P`, `N`, etc.) - Empty visible squares are shown as part of the rank count (e.g., `8`, `1`) The engine will: 1. Parse and store the fog FEN -2. Use it to initialize the belief state (set of possible positions consistent with observations) -3. Search over the belief state to find the best move - -**Note**: The fog_fen feature is currently a basic implementation. The engine stores the fog FEN and reports it, but full integration with belief state enumeration requires additional development. +2. Enumerate positions consistent with what the fog FEN shows (permuting hidden opponent pieces across unseen squares) +3. Use that belief state to guide the Obscuro search before selecting a move ## Viewing the Fog-of-War Board State @@ -344,14 +342,12 @@ The current implementation includes: - ✅ FoW visibility computation (Appendix A rules) - ✅ UCI integration and options - ✅ Multi-threaded search (1 CFR solver + 2 expanders) -- ✅ Basic fog_fen parsing and storage +- ✅ fog_fen parsing wired into belief state enumeration +- ✅ Belief state management (enumerates hidden opponent permutations up to 1024 states per observation) - ✅ NNUE evaluation for all FoW variants -- ⚠️ Belief state management (simplified - stores true position only) - ⚠️ Action purification (placeholder implementation) -- ⚠️ fog_fen integration with belief state (parses but doesn't enumerate) -- 🔲 Full belief enumeration (enumerate positions consistent with observation) -- 🔲 Full KLUSS order-2 neighborhood computation -- 🔲 Complete gadget implementation (Resolve/Maxmargin) +- ⚠️ KLUSS order-2 neighborhood is still simplified +- ⚠️ Resolve/Maxmargin gadget details are incomplete - 🔲 Instrumentation (Appendix B.4 metrics) @@ -369,16 +365,16 @@ The current implementation includes: **What Doesn't Work Yet**: -1. **Belief enumeration**: The engine doesn't enumerate possible positions consistent with what you see. It only uses the true position, meaning it plays as if it has perfect information about hidden pieces. - -2. **fog_fen analysis**: While you can specify a partial observation with `position fog_fen`, the engine doesn't use it to build a proper belief state. It starts from the variant's starting position. +1. **Belief diversity limits**: Enumeration permutes hidden opponent pieces from the current position and caps at 1024 states; it does not yet model captures beyond the observed piece set or piece-in-hand drops for crazyhouse variants. + +2. **KLUSS neighborhood**: The KLUSS computation is still a placeholder and does not freeze/unfreeze infosets per the paper's order-2 definition. -3. **True imperfect information play**: Without belief enumeration, the engine essentially plays perfect information chess with FoW move restrictions, rather than reasoning about what might be hidden. +3. **Purification and gadgets**: Action purification and Resolve/Maxmargin gadget details remain simplified, so play quality may vary in tricky information sets. ### Practical Usage -**Current best use case**: Using the standard FoW search to explore how the engine handles the FoW visibility rules and move generation. The search infrastructure is in place for future belief state enumeration. +**Current best use case**: Using `position fog_fen` to explore imperfect-information situations where hidden opponent pieces could be on multiple unseen squares. The engine will enumerate those possibilities and search them, but higher-level gadgets and purification are still simplified. -**Not yet suitable for**: Analyzing positions where you want the engine to reason about hidden pieces based on partial observations. +**Not yet suitable for**: Positions that rely on advanced KLUSS freezing/unfreezing logic or deep purification requirements (e.g., adversarial bluffing scenarios and crazyhouse drop speculation). For development status and technical details, see `OBSCURO_FOW_IMPLEMENTATION.md`. diff --git a/OBSCURO_FOW_IMPLEMENTATION.md b/OBSCURO_FOW_IMPLEMENTATION.md index 1d99958be..b513d960f 100644 --- a/OBSCURO_FOW_IMPLEMENTATION.md +++ b/OBSCURO_FOW_IMPLEMENTATION.md @@ -16,6 +16,7 @@ This implementation follows the Obscuro paper's algorithms for imperfect-informa - Maintains set P of consistent positions - Observation history tracking - From-scratch enumeration (Figure 9, lines 2-4) + - Enumerates hidden opponent piece permutations consistent with `fog_fen` observations (capped at 1024 states) - [x] **Evaluator Hook** (`src/imperfect/Evaluator.{h,cpp}`) - MultiPV depth-1 evaluation for all children @@ -150,107 +151,17 @@ This section provides a detailed analysis of what remains to be implemented to a -#### 1. Full Belief State Enumeration (HIGH PRIORITY) +#### Addressed Critical Gaps - - -**Current State**: The belief state module (`Belief.cpp`) stores only the true position FEN. The `sample_states()` method returns a single-element vector containing just the current position. - - - -**What's Needed**: - -- **Observation-consistent enumeration**: Given an observation (what the player sees), enumerate ALL positions that could produce that observation - -- **Efficient representation**: Use bitboards or piece placement constraints to represent the set of unknown piece locations - -- **Incremental updates**: When a new observation arrives, filter existing belief states rather than re-enumerating from scratch - - - -**Algorithm** (from Figure 9, lines 2-4): - -``` - -P ← EnumerateConsistentPositions(observation_history) - -I ← SampleSubset(P, MinInfosetSize) // Sample 256 positions - -``` - - - -**Implementation Tasks**: - -1. Implement `enumerate_consistent_positions()` that: - - - Parses fog_fen to identify unknown squares ('?') - - - Computes all possible piece placements on unknown squares - - - Filters positions that would produce the observed fog_fen - - - Respects piece count constraints (e.g., max 8 pawns per side) - - - -2. Implement efficient sampling: - - - Random sampling from large belief sets - - - Stratified sampling to ensure diversity - - - Weighted sampling based on position likelihood - - - -**Complexity**: High - this is the most algorithmically complex missing piece - - - -#### 2. fog_fen Integration with Belief State (HIGH PRIORITY) - - - -**Current State**: `position fog_fen ` parses and stores the fog_fen string, but it's not used by the belief state module. - - - -**What's Needed**: - -```cpp - -// In Planner::construct_subgame(): - -if (!get_fog_fen().empty()) { - - // Parse fog_fen to create observation - - Observation obs = parse_fog_fen(get_fog_fen()); - - // Enumerate positions consistent with this observation - - beliefState.enumerate_from_fog_fen(obs); - -} - -``` - - - -**Implementation Tasks**: - -1. Create `parse_fog_fen()` function that converts fog_fen string to Observation struct - -2. Implement `BeliefState::enumerate_from_fog_fen()` +The latest update implements the two previously missing foundation pieces: -3. Connect fog_fen to belief state in Planner +- **Full belief state enumeration**: `BeliefState::enumerate_candidates()` now permutes hidden opponent pieces across unseen squares (masked by visibility) and keeps every state consistent with the latest observation. Enumeration is capped at 1024 states to prevent combinatorial blowups, and illegal or king-capturable positions are filtered out before sampling. -4. Handle piece-in-hand visibility for crazyhouse variants +- **fog_fen integration**: `BeliefState::parse_fog_fen()` converts partial FoW FEN strings (supports `*` or `?` for unknown squares) into observations, and `Planner::construct_subgame()` seeds the belief state directly from a supplied `fog_fen` before running the solver. -#### 3. Proper KLUSS Order-2 Neighborhood (MEDIUM PRIORITY) +#### 1. Proper KLUSS Order-2 Neighborhood (MEDIUM PRIORITY) @@ -296,7 +207,7 @@ For each state s in belief_state: -#### 4. Thread Synchronization Improvements (MEDIUM PRIORITY) +#### 2. Thread Synchronization Improvements (MEDIUM PRIORITY) @@ -350,7 +261,7 @@ For each state s in belief_state: -#### 5. Action Purification (MEDIUM PRIORITY) +#### 3. Action Purification (MEDIUM PRIORITY) @@ -396,7 +307,7 @@ renormalize(purified) -#### 6. Gadget Implementation (MEDIUM PRIORITY) +#### 4. Gadget Implementation (MEDIUM PRIORITY) @@ -440,7 +351,7 @@ renormalize(purified) -#### 7. Leaf Evaluation Integration (MEDIUM PRIORITY) +#### 5. Leaf Evaluation Integration (MEDIUM PRIORITY) @@ -476,7 +387,7 @@ renormalize(purified) -#### 8. Instrumentation (Appendix B.4) (LOW PRIORITY) +#### 6. Instrumentation (Appendix B.4) (LOW PRIORITY) @@ -492,7 +403,7 @@ renormalize(purified) -#### 9. Memory Management (LOW PRIORITY) +#### 7. Memory Management (LOW PRIORITY) @@ -508,7 +419,7 @@ renormalize(purified) -#### 10. Incremental Belief Updates (LOW PRIORITY) +#### 8. Incremental Belief Updates (LOW PRIORITY) @@ -568,9 +479,9 @@ renormalize(purified) **Phase 1: Core Functionality** (Essential for correct play) -1. Full belief state enumeration +1. ✅ Full belief state enumeration (hidden-piece permutations, capped at 1024) -2. fog_fen integration +2. ✅ fog_fen integration (parse + seed belief state) 3. Action purification diff --git a/src/imperfect/Belief.cpp b/src/imperfect/Belief.cpp index 1a130de3e..f4691f7a9 100644 --- a/src/imperfect/Belief.cpp +++ b/src/imperfect/Belief.cpp @@ -17,9 +17,14 @@ */ #include +#include +#include #include +#include +#include #include "Belief.h" #include "../movegen.h" +#include "../bitboard.h" namespace Stockfish { namespace FogOfWar { @@ -94,6 +99,104 @@ bool BeliefState::is_consistent(const Position& pos, const Observation& obs) { return true; } +Observation BeliefState::parse_fog_fen(const std::string& fogFen, const Variant* variant) { + Observation obs; + + if (fogFen.empty()) + return obs; + + std::istringstream ss(fogFen); + std::string boardToken, stmToken, castlingToken, epToken; + + ss >> boardToken >> stmToken >> castlingToken >> epToken; + if (!(ss >> obs.halfmoveClock)) + obs.halfmoveClock = 0; + if (!(ss >> obs.fullmoveNumber)) + obs.fullmoveNumber = 1; + + obs.sideToMove = (stmToken == "b" ? BLACK : WHITE); + + Color us = obs.sideToMove; + + auto files = variant ? int(variant->maxFile) + 1 : FILE_NB; + auto ranks = variant ? int(variant->maxRank) + 1 : RANK_NB; + + int rankIdx = 0; + int fileIdx = 0; + + auto mark_square = [&](int f, int r) { + if (f >= files || r >= ranks) + return Square(SQ_NONE); + return make_square(File(f), Rank(ranks - 1 - r)); + }; + + for (char c : boardToken) { + if (c == '/') { + rankIdx++; + fileIdx = 0; + continue; + } + + if (std::isdigit(static_cast(c))) { + int emptyCount = c - '0'; + for (int i = 0; i < emptyCount && fileIdx < files; ++i, ++fileIdx) { + Square sq = mark_square(fileIdx, rankIdx); + if (sq != SQ_NONE) + obs.visible |= sq; + } + continue; + } + + Square sq = mark_square(fileIdx, rankIdx); + ++fileIdx; + + if (sq == SQ_NONE) + continue; + + if (c == '*' || c == '?') + continue; // Unknown square, remains invisible + + obs.visible |= sq; + + if (c >= 'A' && c <= 'Z') { + if (us == WHITE) + obs.myPieces |= sq; + else + obs.seenOpponentPieces |= sq; + } else if (c >= 'a' && c <= 'z') { + if (us == BLACK) + obs.myPieces |= sq; + else + obs.seenOpponentPieces |= sq; + } + } + + // Castling rights we know for our side only + if (castlingToken != "-") { + if (us == WHITE && (castlingToken.find('K') != std::string::npos || + castlingToken.find('Q') != std::string::npos)) + obs.castlingRights = 1; + else if (us == BLACK && (castlingToken.find('k') != std::string::npos || + castlingToken.find('q') != std::string::npos)) + obs.castlingRights = 1; + } + + // En-passant square visibility + if (epToken != "-") { + if (epToken.size() >= 2) { + File f = File(epToken[0] - 'a'); + Rank r = Rank(epToken[1] - '1'); + if (f >= FILE_A && f < FILE_NB && r >= RANK_1 && r < RANK_NB) { + Square sq = make_square(f, r); + obs.epSquares |= sq; + obs.visible |= sq; + } + } + } + + return obs; +} + bool BeliefState::is_king_capturable(const Position& pos) const { // Check if the side-to-move can capture the opponent's king // If so, the game would have already ended, so this state is illegal @@ -108,56 +211,136 @@ bool BeliefState::is_king_capturable(const Position& pos) const { return bool(pos.attackers_to(theirKing) & pos.pieces(us)); } +bool BeliefState::set_position_from_fen(Position& pos, StateInfo& st, const std::string& fen) const { + if (!variant) + return false; + +#if defined(__cpp_exceptions) || defined(__EXCEPTIONS) + try { + pos.set(variant, fen, isChess960, &st, owningThread); + } catch (...) { + return false; + } +#else + // Exceptions are disabled (e.g., when built with -fno-exceptions) + pos.set(variant, fen, isChess960, &st, owningThread); +#endif + + return true; +} + void BeliefState::enumerate_candidates(const ObservationHistory& obsHist, const Position& truePos) { - // Simple baseline: start from the true position for now - // In a full implementation, we would enumerate all possible placements - // of unseen opponent pieces on unseen squares - if (obsHist.empty()) return; - // For baseline implementation, we'll use a simplified approach: - // 1. Take the true position as the starting point - // 2. Generate variations by considering different placements of unseen pieces + const Observation& obs = obsHist.last(); + Color us = obs.sideToMove; + Color them = ~us; + + Bitboard visible = obs.visible; + Bitboard myPieces = obs.myPieces; + + Bitboard oppPieces = truePos.pieces(them); + Bitboard hiddenOpp = oppPieces & ~visible; - // Start with the true position (store as FEN) - std::string fen = truePos.fen(); - stateFens.push_back(fen); - stateKeys.insert(truePos.key()); + std::vector hiddenSquares; + Bitboard unseen = (~visible & ~myPieces) & AllSquares; + Bitboard tmpUnseen = unseen; + while (tmpUnseen) { + hiddenSquares.push_back(pop_lsb(tmpUnseen)); + } + + std::vector hiddenOppSquares; + std::vector hiddenOppPieces; + Bitboard tmpHiddenOpp = hiddenOpp; + while (tmpHiddenOpp) { + Square sq = pop_lsb(tmpHiddenOpp); + hiddenOppSquares.push_back(sq); + hiddenOppPieces.push_back(truePos.piece_on(sq)); + } - // TODO: Full enumeration would generate all possible positions by: - // - Identifying unseen squares (not in currentObs.visible) - // - Enumerating possible placements of opponent pieces on those squares - // - Considering piece counts and material balance - // - Respecting pawn structure constraints - // - Checking castling rights possibilities + // If there are no hidden opponent pieces, the true position is the only consistent state + if (hiddenOppPieces.empty()) { + stateFens.push_back(truePos.fen()); + stateKeys.insert(truePos.key()); + return; + } - // For now, this simplified version keeps only the true position - // A production implementation would expand this significantly + if (hiddenSquares.size() < hiddenOppPieces.size()) { + // Not enough locations to hide opponent pieces, keep true position only + stateFens.push_back(truePos.fen()); + stateKeys.insert(truePos.key()); + return; + } + + // Limit enumeration to avoid combinatorial explosion + constexpr size_t kMaxEnumeratedStates = 1024; + size_t generated = 0; + std::vector assignment(hiddenOppPieces.size()); + + std::string baseFen = truePos.fen(); + + std::function dfs = [&](size_t idx, Bitboard used) { + if (generated >= kMaxEnumeratedStates) + return; + + if (idx == hiddenOppPieces.size()) { + StateInfo st; + Position candidate; + if (!set_position_from_fen(candidate, st, baseFen)) + return; + + for (Square sq : hiddenOppSquares) + candidate.remove_piece(sq); + + for (size_t i = 0; i < hiddenOppPieces.size(); ++i) + candidate.put_piece(hiddenOppPieces[i], assignment[i], false, NO_PIECE); + + if (is_consistent(candidate, obs) && !is_king_capturable(candidate)) { + StateKey key = candidate.key(); + if (stateKeys.insert(key).second) { + stateFens.push_back(candidate.fen()); + ++generated; + } + } + return; + } + + for (Square sq : hiddenSquares) { + Bitboard sqBB = square_bb(sq); + if (used & sqBB) + continue; + assignment[idx] = sq; + dfs(idx + 1, used | sqBB); + if (generated >= kMaxEnumeratedStates) + break; + } + }; + + dfs(0, 0); + + if (stateFens.empty()) { + stateFens.push_back(truePos.fen()); + stateKeys.insert(truePos.key()); + } } void BeliefState::filter_illegal_states() { - // Remove states where: - // 1. The opponent's king is capturable (game would have ended) - // 2. The position is not legal according to chess rules + if (!variant) + return; - // Note: We need to create temporary Position objects to check legality - // This is expensive but necessary since we store FENs auto it = stateFens.begin(); while (it != stateFens.end()) { - bool remove = false; - - // Create temporary position from FEN StateInfo st; Position tempPos; - // We need variant and thread info - use defaults for now - // In production, these should be passed from context - // tempPos.set(variant, *it, false, &st, nullptr); - // For now, skip detailed legality checks and just keep all states - // TODO: Proper legality checking with correct variant context + if (!set_position_from_fen(tempPos, st, *it)) { + it = stateFens.erase(it); + continue; + } - if (remove) { + if (is_king_capturable(tempPos)) { + stateKeys.erase(tempPos.key()); it = stateFens.erase(it); } else { ++it; @@ -170,6 +353,10 @@ void BeliefState::rebuild_from_observations(const ObservationHistory& obsHist, stateFens.clear(); stateKeys.clear(); + variant = truePos.variant(); + isChess960 = truePos.is_chess960(); + owningThread = truePos.this_thread(); + if (obsHist.empty()) return; @@ -185,11 +372,25 @@ void BeliefState::rebuild_from_observations(const ObservationHistory& obsHist, } void BeliefState::update_incrementally(const Observation& newObs) { - // Incremental update: filter existing states by new observation - // This is more efficient than rebuilding from scratch - // For now, simplified implementation - just keep all states - // TODO: Parse FENs and check consistency - (void)newObs; // Suppress unused parameter warning + if (!variant) + return; + + auto it = stateFens.begin(); + while (it != stateFens.end()) { + StateInfo st; + Position pos; + if (!set_position_from_fen(pos, st, *it)) { + it = stateFens.erase(it); + continue; + } + + if (!is_consistent(pos, newObs) || is_king_capturable(pos)) { + stateKeys.erase(pos.key()); + it = stateFens.erase(it); + } else { + ++it; + } + } } std::vector BeliefState::sample_states(size_t n, uint64_t seed) const { diff --git a/src/imperfect/Belief.h b/src/imperfect/Belief.h index 65b78ac1d..37e6224e3 100644 --- a/src/imperfect/Belief.h +++ b/src/imperfect/Belief.h @@ -33,16 +33,16 @@ using StateKey = uint64_t; /// Observation represents what the player can see at a given point struct Observation { - Bitboard visible; // Squares that are visible - Bitboard myPieces; // Our pieces (always known exactly) - Bitboard seenOpponentPieces; // Opponent pieces that we can see - Color sideToMove; // Who moves next - Bitboard epSquares; // En-passant squares if visible - int castlingRights; // Our known castling rights + Bitboard visible = 0; // Squares that are visible + Bitboard myPieces = 0; // Our pieces (always known exactly) + Bitboard seenOpponentPieces = 0; // Opponent pieces that we can see + Color sideToMove = WHITE; // Who moves next + Bitboard epSquares = 0; // En-passant squares if visible + int castlingRights = 0; // Our known castling rights // For reconstruction - int halfmoveClock; // 50-move counter - int fullmoveNumber; // Full move number + int halfmoveClock = 0; // 50-move counter + int fullmoveNumber = 1; // Full move number }; /// ObservationHistory maintains the sequence of observations @@ -85,10 +85,17 @@ class BeliefState { /// Check if a position is consistent with observations static bool is_consistent(const Position& pos, const Observation& obs); + /// Parse a partial FoW FEN (fog_fen) into an Observation structure + static Observation parse_fog_fen(const std::string& fogFen, const Variant* variant); + private: std::vector stateFens; // FEN strings instead of Position objects std::unordered_set stateKeys; // For deduplication + const Variant* variant = nullptr; + bool isChess960 = false; + Thread* owningThread = nullptr; + /// Generate candidate positions from observations void enumerate_candidates(const ObservationHistory& obsHist, const Position& truePos); @@ -98,6 +105,9 @@ class BeliefState { /// Helper: Check if king is capturable (game would have ended) bool is_king_capturable(const Position& pos) const; + + /// Helper: create a Position from a FEN using the current variant context + bool set_position_from_fen(Position& pos, StateInfo& st, const std::string& fen) const; }; /// create_observation() creates an observation from current position diff --git a/src/imperfect/CFR.cpp b/src/imperfect/CFR.cpp index 327af85b1..c139e2a66 100644 --- a/src/imperfect/CFR.cpp +++ b/src/imperfect/CFR.cpp @@ -53,6 +53,8 @@ std::vector regret_matching(const std::vector& regrets) { std::vector positive_regret_matching_plus(const std::vector& regrets, const std::vector& oldStrategy, float discountFactor) { + (void)oldStrategy; + // PRM+ uses linear discount on regrets std::vector discountedRegrets(regrets.size()); for (size_t i = 0; i < regrets.size(); ++i) diff --git a/src/imperfect/Evaluator.h b/src/imperfect/Evaluator.h index 1098b219d..807504edd 100644 --- a/src/imperfect/Evaluator.h +++ b/src/imperfect/Evaluator.h @@ -26,7 +26,7 @@ namespace Stockfish { class Position; -class StateInfo; +struct StateInfo; namespace FogOfWar { diff --git a/src/imperfect/Planner.cpp b/src/imperfect/Planner.cpp index acaabdef8..a81ba772b 100644 --- a/src/imperfect/Planner.cpp +++ b/src/imperfect/Planner.cpp @@ -20,6 +20,7 @@ #include "../position.h" #include "../misc.h" #include "../movegen.h" +#include "../uci.h" #include #include @@ -36,6 +37,8 @@ Planner::~Planner() { } bool is_fow_variant(const Position& pos) { + (void)pos; + // Check if this is Fog-of-War chess // For now, we'll check via a variant name or flag // This should be properly implemented based on variant system @@ -50,11 +53,20 @@ void Planner::update_observation_history(const Position& pos) { void Planner::construct_subgame(const Position& pos) { // Implements ConstructSubgame from Figure 9 + ObservationHistory workingHistory = obsHistory; + + if (!get_fog_fen().empty()) { + Observation fogObservation = BeliefState::parse_fog_fen(get_fog_fen(), pos.variant()); + workingHistory.clear(); + workingHistory.add_observation(fogObservation); + obsHistory = workingHistory; + } + // Step 1: Rebuild belief state P from observations - if (config.enableIncrementalBelief && !obsHistory.observations().empty()) { - beliefState.update_incrementally(obsHistory.last()); + if (config.enableIncrementalBelief && !workingHistory.observations().empty()) { + beliefState.update_incrementally(workingHistory.last()); } else { - beliefState.rebuild_from_observations(obsHistory, pos); + beliefState.rebuild_from_observations(workingHistory, pos); } // Step 2: Sample I ⊂ P (default 256 states) diff --git a/src/imperfect/Subgame.cpp b/src/imperfect/Subgame.cpp index f603c0d82..a40d1ed52 100644 --- a/src/imperfect/Subgame.cpp +++ b/src/imperfect/Subgame.cpp @@ -55,6 +55,8 @@ InfosetNode* Subgame::get_infoset(SequenceId seqId, Color player) { void Subgame::construct(const std::vector& sampledStateFens, int minInfosetSize) { + (void)minInfosetSize; // Parameter currently unused + // Clear existing tree rootNode = std::make_unique(); infosets.clear(); @@ -204,6 +206,9 @@ int Subgame::average_depth() const { float compute_alternative_value(const InfosetNode* infoset, const std::vector& currentX, const std::vector& currentY) { + (void)currentX; + (void)currentY; + // Simplified implementation: return current value estimate // Full implementation would compute min(evaluate(s), v*) for new states return infoset ? infoset->value : 0.0f;