Skip to content

backlog: isolate per-sequence state to prevent flag corruption across sequences#24880

Open
joluxer wants to merge 4 commits into
arendst:developmentfrom
joluxer:pr/backlog-behavior
Open

backlog: isolate per-sequence state to prevent flag corruption across sequences#24880
joluxer wants to merge 4 commits into
arendst:developmentfrom
joluxer:pr/backlog-behavior

Conversation

@joluxer

@joluxer joluxer commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Description

Part of the backlog improvement series documented in discussion #24776.

Implementation choices are open to discussion. If you'd prefer a different approach, please comment -- I'll revise.

Problem

Bug -- Concurrent-rule flag clobber:
Rule1: Backlog0 cmd1;cmd2 and Rule2: Backlog cmd3;cmd4 firing together:
all four commands ran timed. The second rule's enqueue reset the global
nodelay flag to false before the first rule's commands were drained.
Per-entry intent was silently discarded.

Root cause:
Behavioral intent (nodelay, no_mqtt_response) was stored as
global state, set at enqueue and consumed at drain. Any intervening
enqueue or command run could overwrite it.


Changes

Fix: intent baked in at enqueue time (bit0=nodelay, bit1=no_mqtt_resp):

Each queue entry now carries its behavioral flags as a prefix byte, set at
enqueue from the staged state. Flags travel with the command;
globals no longer affect them at drain-time.

NoDelay remains one-shot: raises the nodelay flag for exactly the next
enqueued command, then reverts to the sequence baseline (Backlog0/2: all
nodelay; Backlog/1/3: timed). This matches the documented one-command scope
and is now structurally guaranteed rather than dependent on drain-loop state.

ExecuteCommandBlock (IF/ENDIF engine) resets staged flags to plain-Backlog
defaults before inserting commands. IF blocks now behave like a fresh Backlog
invocation: timed, MQTT on. Authors who want NoDelay inside an IF block
write it explicitly -- readable and correct.

SetOption166 (default 0) -- separate drain window for external commands:

Previously, external commands (MQTT, Serial, Button) stalled the drain window
via SO34 -- the same timer that governs inter-command spacing within a Backlog
sequence. This conflated two independent concerns.

SO166=0 (default): external commands schedule BACKLOG_EXT_DELAY (100 ms,
configurable via user_config_override.h), separate from SO34. Existing
behavior is preserved for users who do not set SO166. Backlog queue drain stalls
for short time.

SO166=1: BacklogLoop is sole timer owner. External commands do not stall the
backlog queue drain. The inter-command delay within a Backlog sequence (SO34) is
unchanged in either mode.

Backlog::WarnIfNoDelay(cmd_name_P):

Diagnostic hook for command handlers with settling-time requirements or
interlock dependencies making them unsafe at zero inter-command delay. Logs at
ERROR level when called from a NoDelay drain context. Available for any driver
or command handler to opt in. Delay command uses it.


Behavior summary

Scenario Before After
Rule1: Backlog0 cmd1;cmd2 / Rule2: Backlog cmd3;cmd4 (both fire) all four: timed (Rule2 clobbers Rule1's nodelay flag) cmd1+cmd2: no delay, cmd3+cmd4: timed (per-entry, set at enqueue)
Backlog2 cmd1; IF cond; cmd2; ENDIF cmd2 MQTT-suppressed (inherited drain state) cmd2 MQTT on (IF block resets to plain Backlog)
Backlog POWER1; NoDelay; POWER2; POWER3 POWER1: timed, POWER2: nodelay, POWER3: queue timed or no delay unknown, not guaranteed, only POWER2 no delay now structurally guaranteed
Delay inside Backlog0 silent no-op silent no-op + ERROR log
External MQTT command during drain drain window: SO34 (200 ms default) drain window: BACKLOG_EXT_DELAY (100 ms)
Backlog0 cmd1; cmd2 both: no delay both: no delay (unchanged) - now structurally guaranteed

Default Backlog / Backlog1 behavior: no change.


Memory impact (measured, this PR only)

Ref: pr/backlog-encapsulation tip

Target Flash RAM IRAM
tasmota32 +212 B -8 0
tasmota-4M +468 B 0 0

RAM −8 B on tasmota32: per-entry flags replace three drain-loop globals.
Flash cost is the expanded queue entry and IF-block reset logic.


Requires pr/backlog-encapsulation: staged flags and queue state must be
inaccessible from .ino scope for the per-entry guarantee to hold.

Build-tested on ESP8266 (tasmota-4M) and ESP32 (tasmota32). Target-tested on ESP32: completed.

Checklist:

  • The pull request is done against the latest development branch
  • Only relevant files were touched
  • Only one feature/fix was added per PR and the code change compiles without warnings
  • The code change is tested and works with Tasmota core ESP8266 V.2.7.8
  • The code change is tested and works with Tasmota core ESP32 V.3.3.8 from Platform 2026.05.50
  • I accept the CLA.

joluxer added 4 commits July 4, 2026 21:10
BacklogLoop previously left backlog_timer at the value written by
CommandHandler at the start of ExecuteCommand -- millis() at command
start time plus P_BACKLOG_DELAY. After a long execution (rule-trigger
chain) that point was already past: TimeReached() returned true
immediately on the next iteration, draining the queue without pause
regardless of SetOption34. A nested CmndBacklog reset the timer to
millis() during execution, compounding the problem: the queue drained
as fast as commands were enqueued.

BacklogLoop now unconditionally sets backlog_timer after each timed
drain step. The inter-command interval becomes deterministic: exactly
P_BACKLOG_DELAY between drain steps, regardless of what CommandHandler
or nested Backlog commands wrote during ExecuteCommand.

Exception: CmndDelay called from within a drain sets backlog_delay_guard
(detected via backlog_mutex=true). BacklogLoop preserves the timer for
that one step and clears the guard. External commands can still shorten
an active delay -- that matches original Tasmota behaviour.
Introduce SettingsParam(index) as a named function returning a uint8_t
reference to Settings->param[index].

Adds bounds protection: an index >= PARAM8_SIZE returns a reference to a
stable zero byte instead of an out-of-bounds access.

Replace three direct Settings->param[] accesses in the SetOption handler
(CmndSetoption, GetOption, CmndSetoptionBase) to use the new accessor.
Move the backlog queue, timer, and control flags out of TasmotaGlobal_t and
the global .ino scope into a dedicated translation unit (support_backlog.cpp).
The state was accessible from any .ino file; the new module limits mutation
to explicit paths through the Backlog:: API.

All internal state is held in an anonymous namespace. The public interface is
declared in support_backlog.h. A one-line support_backlog.ino adapter keeps
BacklogLoop() visible to the unity build.

Removes from TasmotaGlobal_t: backlog_timer, backlog_nodelay, backlog_mutex,
backlog_delay_guard, backlog_no_mqtt_response. Removes global LList<char*>
backlog.

Adds SuppressMqttResponse() to support.ino as the designated mutation path
for TasmotaGlobal.no_mqtt_response from backlog context.

No intended behavior change.
SetOption166 (default 0) controls whether external commands (MQTT, Serial,
Button, ...) extend the Backlog drain window after execution.
SO166=0: CommandHandler sets the drain timer to millis() + BACKLOG_EXT_DELAY
(100 ms, separate from SO34, overridable at build time via user_config_override.h).
SO166=1: BacklogLoop is the sole timer owner; external commands do not stall
the drain. The inter-command delay within a sequence (SO34) is unchanged.

Each queue entry now carries a flavor byte (bit0=nodelay, bit1=no_mqtt_resp)
baked in from staged flags at enqueue time. D_CMND_NODELAY is resolved in
CmndBacklog() and ExecuteCommandBlock() at enqueue rather than stored as a
queue token. Loop() reads the flavor byte per drain step; _nodelay and
_no_mqtt_resp each split into _staged (set by callers) and _current (set
from the flavor byte, read by IsNodelay()).

ExecuteCommandBlock() resets staged flags to plain-Backlog defaults before
its tokenisation loop: IF blocks no longer inherit ambient state from a
preceding CmndBacklog() call.

ScheduleDelay() is a no-op when _nodelay_current is set.

Default-configuration behavior: the inter-command delay (SO34) is unchanged.
The post-external-command drain window changes from SO34 to BACKLOG_EXT_DELAY
(100 ms vs. 200 ms default) -- this is intentional and separately configurable.
@s-hadinger

Copy link
Copy Markdown
Collaborator

Please review this:

PR 24880 Review Findings

Blocking issue: NoDelay becomes persistent in rule command blocks

In tasmota/tasmota_xdrv_driver/xdrv_10_rules.ino, ExecuteCommandBlock() now handles NoDelay by calling Backlog::SetNodelay(true) and not enqueueing the literal NoDelay command.

Before this PR, rule command blocks inserted NoDelay into the backlog as a normal queue entry. The old BacklogLoop() consumed it as a one-shot sentinel, applying no-delay only to the next real command.

With this PR, the staged NoDelay flag is never reset in the rule insertion path, so all following commands in the same rule block inherit NoDelay.

Example affected pattern:

IF ... DO NoDelay; Power1 1; Delay 10; Power2 1 ENDIF

Expected behavior:

  • Power1 1 runs without the standard backlog delay.
  • Delay 10 is honored before Power2 1.

Likely PR behavior:

  • Power1 1, Delay 10, and Power2 1 are all inserted with NoDelay.
  • Delay 10 becomes ineffective because Backlog::ScheduleDelay() no-ops in a NoDelay context.
  • Power2 1 may run immediately.

This is introduced by this PR; it was not present in the previous implementation.

Suggested change

Mirror the one-shot logic already implemented in CmndBacklog() when handling rule IF command blocks:

bool nodelay_oneshot = false;
...
if (0 == strncasecmp_P(blcommand, PSTR(D_CMND_NODELAY), strlen(D_CMND_NODELAY))) {
  Backlog::SetNodelay(true);
  nodelay_oneshot = true;
} else {
  Backlog::InsertCmd(blcommand, insertPosition++);
  if (nodelay_oneshot) {
    Backlog::SetNodelay(false);
    nodelay_oneshot = false;
  }
}

@joluxer

joluxer commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the suggestion, I like the idea. I'll look at it during weekend, when I'm back at my laptop.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants