Describe the bug
The STL ASan annotations are designed to allow for "annotations inserted, but disabled" in order to help with static libraries that need to work in ASan and non-ASan contexts. The idea is that the ODR violation that may occur when some code is ASan annotated and some isn't is benign if the annotations are disabled. However, currently the "annotations enabled" switch is coarse and activates all annotations regardless of what was requested.
This affects annotations for vector, string, and optional.
As an example:
If a static library a.lib is built without ASan and with /D_ANNOTATE_STL (it is a static library and wants to work with ASan and non-ASan projects), then if the code to check whether to annotate each container is added to the function definitions in this .lib.
If a.lib is linked to an ASan project (/fsanitize=address), each annotation will be activated, there are no ODR violations, and the containers will be checked for memory safety errors.
If a.lib is linked to a non-ASan project, there will be an ODR violation where some source files will get a version of these containers with annotation code inserted, and some will not. However, when the inserted annotation code checks for whether to call ASan to mark the shadow memory (ex: _Asan_vector_should_annotate for std::vector, etc) that value will be false, so even though there is an ODR violation, the behavior between the two competing definitions will be the same.
If a.lib is linked to an ASan project that does not want vector annotations (/fsanitize=address /D_DISABLE_VECTOR_ANNOTATIONS), _INSERT_VECTOR_ANNOTATIONS will be unset, so just std::vector will no longer have the inserted annotation code, causing an ODR violation similar to the case when linked to a non-ASan project. In this case though, stl_asan.lib has been included, setting _Asan_vector_should_annotate to true, even though we only want the annotations activated for string and optional, meaning that the behavior between the two competing definitions is no longer the same. In the resulting binary, some std::vector functions will call ASan to poison/unpoison the container, and some will not, likely eventually causing an error.
Command-line test case (for std::vector, this issue exists for std::string and std::optional as well)
C:\Users\amyw\source\vector_anno>type test.cpp
// cl /c /EHsc /MD /O2 /D_ANNOTATE_STL /DANNOTATED annotated_lib.cpp
// lib /OUT:annotated_lib.lib annotated_lib.obj
// cl /EHsc /MD /fsanitize=address /Zi /D_DISABLE_VECTOR_ANNOTATION asan_main.cpp annotated_lib.lib
#include <vector>
#include <cstring>
#include <cstdio>
#ifdef ANNOTATED
__declspec(noinline)
void fill_vector(std::vector<char>* v) {
v->reserve(100);
for (int i = 0; i < 10; i++)
v->push_back('A' + static_cast<char>(i));
}
__declspec(noinline)
void copy_from_vector(const std::vector<char>* v, char* dst, size_t n) {
std::memcpy(dst, v->data(), n);
}
#else
// Defined in annotated_lib.lib
extern "C" const bool _Asan_vector_should_annotate;
void fill_vector(std::vector<char>* v);
void copy_from_vector(const std::vector<char>* v, char* dst, size_t n);
int main() {
std::printf("Annotate string: %s\n", _Asan_string_should_annotate ? "true" : "false");
std::printf("Annotate vector: %s\n", _Asan_vector_should_annotate ? "true" : "false");
std::vector<char> v;
fill_vector(&v);
char buf[50];
copy_from_vector(&v, buf, 50);
std::printf("PASS: copied %d bytes\n", 50);
return 0;
}
#endif
C:\Users\amyw\source\vector_anno>cl /c /EHsc /MD /O2 test.cpp /D_ANNOTATE_STL /DANNOTATED /Foannotated_lib.obj
Microsoft (R) C/C++ Optimizing Compiler Version 19.51.26504.99 for x64 (PREVIEW)
Copyright (C) Microsoft Corporation. All rights reserved.
test.cpp
C:\Users\amyw\source\vector_anno>lib /OUT:annotated_lib.lib annotated_lib.obj
Microsoft (R) Library Manager Version 14.51.36127.96 (PREVIEW)
Copyright (C) Microsoft Corporation. All rights reserved.
C:\Users\amyw\source\vector_anno>cl /EHsc /MD /fsanitize=address /Zi /D_DISABLE_VECTOR_ANNOTATION test.cpp annotated_lib.lib
Microsoft (R) C/C++ Optimizing Compiler Version 19.51.26504.99 for x64 (PREVIEW)
Copyright (C) Microsoft Corporation. All rights reserved.
test.cpp
Microsoft (R) Incremental Linker Version 14.51.36127.96 (PREVIEW)
Copyright (C) Microsoft Corporation. All rights reserved.
/out:test.exe
/InferAsanLibs
/debug
test.obj
annotated_lib.lib
C:\Users\amyw\source\vector_anno>test.exe
Annotate string: true
Annotate vector: true
=================================================================
==45176==ERROR: AddressSanitizer: container-overflow on address 0x12358a7a00fa at pc 0x7ffac533bea5 bp 0x00a2cf97fcf0 sp 0x00a2cf97f480
READ of size 50 at 0x12358a7a00fa thread T0
#0 0x7ffac533bea4 in memcpy X:\msvc\src\vctools\asan\llvm\compiler-rt\lib\sanitizer_common\sanitizer_common_interceptors_memintrinsics.inc:147
#1 0x7ff7959d120d in main C:\Users\amyw\source\vector_anno\test.cpp:32
#2 0x7ff7959d4283 in invoke_main C:\repos\msvc\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
#3 0x7ff7959d4283 in __scrt_common_main_seh C:\repos\msvc\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
#4 0x7ffbc3e2e8d6 (C:\WINDOWS\System32\KERNEL32.DLL+0x18002e8d6)
#5 0x7ffbc522c53b (C:\WINDOWS\SYSTEM32\ntdll.dll+0x18008c53b)
0x12358a7a00fa is located 10 bytes inside of 100-byte region [0x12358a7a00f0,0x12358a7a0154)
allocated by thread T0 here:
#0 0x7ff7959d3ad5 in operator new(unsigned __int64) X:\msvc\src\vctools\asan\llvm\compiler-rt\lib\asan\asan_win_new_scalar_thunk.cpp:40
#1 0x7ff7959d36d4 in fill_vector(class std::vector<char, class std::allocator<char>> *) (C:\Users\amyw\source\vector_anno\test.exe+0x1400036d4)
#2 0x7ff7959d11f5 in main C:\Users\amyw\source\vector_anno\test.cpp:30
#3 0x7ff7959d4283 in invoke_main C:\repos\msvc\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
#4 0x7ff7959d4283 in __scrt_common_main_seh C:\repos\msvc\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
#5 0x7ffbc3e2e8d6 (C:\WINDOWS\System32\KERNEL32.DLL+0x18002e8d6)
#6 0x7ffbc522c53b (C:\WINDOWS\SYSTEM32\ntdll.dll+0x18008c53b)
HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_container_overflow=0.
Or if supported by the container library, pass -D__SANITIZER_DISABLE_CONTAINER_OVERFLOW__ to the compiler to disable instrumentation.
If you suspect a false positive see also: https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow.
SUMMARY: AddressSanitizer: container-overflow C:\Users\amyw\source\vector_anno\test.cpp:32 in main
Shadow bytes around the buggy address:
0x12358a79fe00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x12358a79fe80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x12358a79ff00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x12358a79ff80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x12358a7a0000: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
=>0x12358a7a0080: fd fd fd fd fd fd fa fa fa fa fa fa fa fa 00[02]
0x12358a7a0100: fc fc fc fc fc fc fc fc fc fc fc fa fa fa fa fa
0x12358a7a0180: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x12358a7a0200: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x12358a7a0280: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x12358a7a0300: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==45176==ABORTING
Expected behavior
Annotations should be disabled for vector, but not string if /D_DISABLE_VECTOR_ANNOTATION is specified.
Above repro should produce:
C:\Users\amyw\source\vector_anno>test.exe
Annotate string: true
Annotate vector: false
PASS: copied 50 bytes
Describe the bug
The STL ASan annotations are designed to allow for "annotations inserted, but disabled" in order to help with static libraries that need to work in ASan and non-ASan contexts. The idea is that the ODR violation that may occur when some code is ASan annotated and some isn't is benign if the annotations are disabled. However, currently the "annotations enabled" switch is coarse and activates all annotations regardless of what was requested.
This affects annotations for
vector,string, andoptional.As an example:
If a static library
a.libis built without ASan and with/D_ANNOTATE_STL(it is a static library and wants to work with ASan and non-ASan projects), then if the code to check whether to annotate each container is added to the function definitions in this .lib.If
a.libis linked to an ASan project (/fsanitize=address), each annotation will be activated, there are no ODR violations, and the containers will be checked for memory safety errors.If
a.libis linked to a non-ASan project, there will be an ODR violation where some source files will get a version of these containers with annotation code inserted, and some will not. However, when the inserted annotation code checks for whether to call ASan to mark the shadow memory (ex:_Asan_vector_should_annotateforstd::vector, etc) that value will be false, so even though there is an ODR violation, the behavior between the two competing definitions will be the same.If
a.libis linked to an ASan project that does not want vector annotations (/fsanitize=address /D_DISABLE_VECTOR_ANNOTATIONS),_INSERT_VECTOR_ANNOTATIONSwill be unset, so juststd::vectorwill no longer have the inserted annotation code, causing an ODR violation similar to the case when linked to a non-ASan project. In this case though,stl_asan.libhas been included, setting_Asan_vector_should_annotatetotrue, even though we only want the annotations activated forstringandoptional, meaning that the behavior between the two competing definitions is no longer the same. In the resulting binary, somestd::vectorfunctions will call ASan to poison/unpoison the container, and some will not, likely eventually causing an error.Command-line test case (for
std::vector, this issue exists forstd::stringandstd::optionalas well)Expected behavior
Annotations should be disabled for vector, but not string if
/D_DISABLE_VECTOR_ANNOTATIONis specified.Above repro should produce: