checkpoint learning curve

[phidebian]

Hi Gurus,
I have hard time to understand the logic of statement like checkpoint = (struct checkpt*)stakalloc(sizeof(struct checkpt));

As I understand it, the checkpoint->buff will host s sigsetjmp buffer, meaning (with my understanding) that later on a siglongjmp will land there (at the same function as the sigsetjmp call site) so why not stuff checkpoint as an local var as in many other spots in the source tree. IS there some magic there I should know?

Thanx in advance

[McDutchie]

Those were changed from local variables to stack-allocated pointers in 93u+ 2011-12-24. See the xec.c changes in ksh93/ksh93-history@1ce26c0.

It's always more or less guesswork with so many changes per commit in ksh93-history, but I'm pretty sure this is the corresponding RELEASE file entry:

11-11-07  Changes were made to reduce the stack size to allow deeper function
	  recursion.

Using a local variable for the context/longjmp buffer is perfectly legit and functional, but does use sizeof(struct checkpt) bytes on the C stack (on my system, that is 256 bytes). So it looks like users were complaining about recursive scripts segfaulting due to a C stack overflow before they reached MAXDEPTH…

…which is currently defined in path.h as (sizeof(char*)==2?64:1024). Heh. I think we can delete the case for 16-bit systems from that :)

[McDutchie]

You might also wonder why they used the AST stack (via stkallocl/stakalloc) instead of the heap. The answer is efficiency. The stack gets allocated on the heap in chunks of STK_FSIZE a.k.a. 1024*sizeof(char*) bytes (see stkgrow in stk.c), and the stack state gets restored at the end of every recursive sh_exec call, so stack allocations do not need to be freed individually.

[phidebian]

Hum, head scratching :) I understand the stakalloc on malloc, I understand that ksh 'stack' (ksh function, scope, etc) is not C stack, I understand as well that a stack implemented on top of malloc (or mmap) would allow multiple alloc, single free at ksh level, i.e scope locals are allocated one by one and freed once on unscope (may be didn't checked that), exatly as alloca() gets multiple a alloc and all freed at once at return, so I am fine with that. I understand as well that then there is no corelation bertween ksh 'stack' and C stack (well HW stack), may be this is my misunderstanding here, but at the moment I bielieve :) that you can push/pop the C stack without touching the ksh stack.

What's bugs me is that a setjmp() buffer can be used by 'many' potential longjmp to it and it want to return exactly at the point of setjmp() with the stack of setjmp() call site, by stuffing the setjmp buffer into the stack of the caller of setjmp() i.e local, you guaranty that a longjmp() time you will land in setjmp() call site return and its stack in intact, you will never longjump to a disapeared setjmp buffer. By stuffing a setjmp *bufferon a malloc'ed area, a static, or a global , look dangerous to me unless we can be 100% sure that there will never be a danglingsetjmp*buffer* welongjmp` into.

Well anyway I got the picture now, let's assume ASAN will catch such dangling jump buffer :)

Tahnx @McDutchie for your explanations always usefull :)

[McDutchie]

The central execution function sh_exec() stores the current stack offset as a local variable on every recursive invocation and restores it at the end (unless a loop invariants optimisation is in use). Even if multiple levels of jmpval occur, the stack should not be truncated to before the current jmpval buffer because jmpval restores the local variable environment. So, as long as the state is restored correctly before executing sigjmpval, how is that more dangerous than any other approach?

[McDutchie]

Speaking of which...

ksh/src/cmd/ksh93/features/setjmp

Lines 1 to 18 in d446015

	lib sigsetjmp,_setjmp,_longjmp
	typ sigjmp_buf setjmp.h
	cat{
	#undef sigsetjmp
	#undef siglongjmp
	#undef sigjmp_buf
	#define sigjmp_buf jmp_buf
	#ifdef _lib__setjmp
	# define sigsetjmp(a,b) _setjmp(a)
	#else
	# define sigsetjmp(a,b) setjmp(a)
	#endif /* _lib__setjmp */
	#ifdef _lib__longjmp
	# define siglongjmp(a,b) _longjmp(a,b)
	#else
	# define siglongjmp(a,b) longjmp(a,b)
	#endif /* _lib__longjmp */
	}end

The header generated by this feature test script (see arch/*/src/cmd/ksh93/FEATURE/setjmp) is included by every ksh93 .c file via edit.h and fault.h.

If the system has _setjmp(3) and _longjmp(3), it simply redefines sigsetjmp(3) and siglongjmp(3) to be actually the former two. Which means the signal mask isn't saved and restored even if sigsetjmp is called with a nonzero second argument, which happens a few times in xec.c. And ksh93-history shows this feature script has been completely unchanged since 1995 or before!

suppresses a massive rant about AT&T code quality standards, yet again

Presumably there is a missing #if !_lib_sigsetjmp…#endif that should have been enveloping all that – the lib test for sigsetjmp is there, but its result is never used. However, it looks like there is no non-prehistoric system that doesn't have sigsetjmp, so I'll just remove all this nonsense.

After removing it, sizeof(struct checkpt) increases from 256 to 264 on my system, the extra 8 bytes presumably accounting for the signal mask.

[phidebian]

Ha ok, good enough, I now understand why we can stakalloc a checkpoint :)
Thanx.