`enable_local_evaluation=True` falls back to API calls in v3.10.1, causing massive request volume increase

[amleczko]

Problem

After upgrading from flagsmith 3.10.0 to 3.10.1, enable_local_evaluation=True no longer works correctly and falls back to making individual API calls for every flag check instead of using local evaluation. This causes a massive increase in request volume.

Impact

• 10x+ increase in API request volume in production
• Instead of 1 environment document request + local evaluation, every get_environment_flags() and get_identity_flags() call makes separate API requests
• Defeats the entire purpose of local evaluation (performance and reduced API load)
• Production systems experience a severe load increase

Root Cause

The issue occurs when the environment document fails to load due to validation errors. The update_environment() method silently catches these errors, leaving self._environment=None. This causes the local evaluation condition to always fail:

  # In get_environment_flags():
  if (self.offline_mode or self.enable_local_evaluation) and self._environment:  # ← self._environment is None!
      return self._get_environment_flags_from_document()  # Never reached
  return self._get_environment_flags_from_api()  # Always makes API calls instead

Steps to Reproduce

1. Initialize Flagsmith client with enable_local_evaluation=True
2. Use any custom API URL or environment that has slightly different response format
3. Call get_environment_flags() or get_identity_flags() repeatedly
4. Expected: 1 environment document request + local evaluation
5. Actual: Every flag check makes a separate API request
6. You'll see silent "Error updating environment" messages while requests keep hitting your API.

Changes in v3.10.1 that caused this

• PR chore: replacing-deprecated-methods #143 introduced new evaluation methods using engine.get_evaluation_result()
• Stricter validation that fails on some environment document formats
• Silent error handling masks validation failures
• When environment loading fails, local evaluation completely stops working

[emyller]

Thanks for reporting this, and for adding great detail. We're looking into it and will respond here soon.

[khvn26]

3.10.1 did not include any changes to environment document validation. So far, I'm unable to reproduce this with a fairly new project in prod SaaS.

@amleczko Can you share a log containing the Error updating environment erros? These should include tracebacks detailing the exact issue with environment document retrieval/deserialisation.

That being said, the code highlighted in the issue

        if (self.offline_mode or self.enable_local_evaluation) and self._environment:
            return self._get_environment_flags_from_document()
        return self._get_environment_flags_from_api()

does seem problematic, as it indeed may cause a silent switch to remote evaluation. @matthewelwell I think we should review this logic, and consider allowing to fall back straight to a default flag handler instead.