When _DEBUG is enabled on Windows,
int VideoFrame::CheckMemory() const {
#ifdef _DEBUG
if (vfb->data && vfb->device->device_type == DEV_TYPE_CPU) {
// check buffer overrun
int *pInt = (int *)(vfb->data + vfb->data_size);
if (pInt[0] != 0xDEADBEEF ||
pInt[1] != 0xDEADBEEF ||
pInt[2] != 0xDEADBEEF ||
pInt[3] != 0xDEADBEEF)
{
return 1;
}
return 0;
}
#endif
return -1;
}
vfb->data + vfb->data_size didn't consider the margin passed into VFBStorage, so it almost always return 1 and fail the memory checks.
When _DEBUG is enabled on Linux,
static void DebugOut(char* s)
{
#ifdef AVS_POSIX
LogMsg(LOGLEVEL_DEBUG, s);
#else
_RPT0(0, s);
#endif
}
LogMsg call lost ScriptEnvironment thus it can't be compiled.
Once LogMsg is replaced with fputs to console, compilation passes, and I'm able to see the same memory checks fail.
After adding margin to VideoFrameBuffer and changing some code, memory checks now all pass.
When _DEBUG is enabled on Windows,
vfb->data + vfb->data_sizedidn't consider the margin passed into VFBStorage, so it almost always return 1 and fail the memory checks.When _DEBUG is enabled on Linux,
LogMsg call lost ScriptEnvironment thus it can't be compiled.
Once LogMsg is replaced with fputs to console, compilation passes, and I'm able to see the same memory checks fail.
After adding margin to VideoFrameBuffer and changing some code, memory checks now all pass.