Skip to content

Add userland syscall resume path and tty input flush on exec - #37

Merged
pradosh-arduino merged 2 commits into
mainfrom
codex/fix-os-userland-return-crash-and-tty-buffer-issue
Apr 2, 2026
Merged

Add userland syscall resume path and tty input flush on exec#37
pradosh-arduino merged 2 commits into
mainfrom
codex/fix-os-userland-return-crash-and-tty-buffer-issue

Conversation

@pradosh-arduino

Copy link
Copy Markdown
Collaborator

Motivation

  • Ensure exec does not inherit stale TTY input by flushing cooked input before launching a new userland program.
  • Provide a safe kernel-resume/trampoline path for userland to return into kernel control flow from a syscall, preserving/restoring non-volatile registers and avoiding clobbering the kernel stack.
  • Introduce a dedicated syscall stack and update TSS rsp0 to improve isolation of kernel-side syscall handling while userland is running.

Description

  • Added a new tty_flush_input() prototype in source/includes/tty.h and implemented it in source/kernel/C/user-input/tty.c to clear the cooked ring buffer and reset the current line buffer length.
  • Updated source/kernel/C/shell/commands/exec.c to include tty.h and call tty_flush_input() before invoking userland_exec() so the new process starts with a clean input state.
  • Extended syscall entry assembly source/kernel/C/interrupts/syscalls-x86_64.asm to check the userland_should_return_kernel flag and, if set, load saved non-volatile registers and rsp/rip from userland_resume_* variables and jump to the saved kernel resume address instead of performing iretq back to userland.
  • Modified source/kernel/C/userland.c to add the userland_should_return_kernel flag, userland_resume_* register save areas, a dedicated userland_syscall_stack, and logic to save current kernel registers and rsp into the resume slots before entering userland, switch kernel_stack_top and tss.rsp0 to the syscall stack while userland is running, and restore them on return; userland_prepare_exit() now signals the kernel resume via the flag rather than mutating a provided frame.

Testing

  • Performed an automated build of the kernel image with make, which completed successfully.
  • Ran the automated unit/regression test suite (make test / CI job that covers userland exec workflow), and the tests passed successfully and reported expected userland exec/exit behavior.
  • Verified the automated kernel boot smoke test in CI passed with the new syscall stack and resume path enabled.

Codex Task

@pradosh-arduino
pradosh-arduino merged commit ab03b9c into main Apr 2, 2026
2 checks passed
@pradosh-arduino
pradosh-arduino deleted the codex/fix-os-userland-return-crash-and-tty-buffer-issue branch April 2, 2026 12:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant