annotate docs/ScudoHardenedAllocator.rst @ 140:276c179585fe

Added tag LLVM5.0.1 for changeset 3a76565eade5
author mir3636
date Tue, 03 Apr 2018 19:09:39 +0900
parents 3a76565eade5
children c2174574ed3a
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
1 ========================
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
2 Scudo Hardened Allocator
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
3 ========================
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
4
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
5 .. contents::
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
6 :local:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
7 :depth: 1
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
8
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
9 Introduction
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
10 ============
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
11
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
12 The Scudo Hardened Allocator is a user-mode allocator based on LLVM Sanitizer's
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
13 CombinedAllocator, which aims at providing additional mitigations against heap
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
14 based vulnerabilities, while maintaining good performance.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
15
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
16 Currently, the allocator supports (was tested on) the following architectures:
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
17
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
18 - i386 (& i686) (32-bit);
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
19 - x86_64 (64-bit);
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
20 - armhf (32-bit);
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
21 - AArch64 (64-bit).
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
22
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
23 The name "Scudo" has been retained from the initial implementation (Escudo
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
24 meaning Shield in Spanish and Portuguese).
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
25
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
26 Design
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
27 ======
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
28
134
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
29 Allocator
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
30 ---------
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
31 Scudo can be considered a Frontend to the Sanitizers' common allocator (later
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
32 referenced as the Backend). It is split between a Primary allocator, fast and
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
33 efficient, that services smaller allocation sizes, and a Secondary allocator
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
34 that services larger allocation sizes and is backed by the operating system
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
35 memory mapping primitives.
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
36
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
37 Scudo was designed with security in mind, but aims at striking a good balance
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
38 between security and performance. It is highly tunable and configurable.
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
39
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
40 Chunk Header
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
41 ------------
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
42 Every chunk of heap memory will be preceded by a chunk header. This has two
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
43 purposes, the first one being to store various information about the chunk,
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
44 the second one being to detect potential heap overflows. In order to achieve
134
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
45 this, the header will be checksummed, involving the pointer to the chunk itself
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
46 and a global secret. Any corruption of the header will be detected when said
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
47 header is accessed, and the process terminated.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
48
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
49 The following information is stored in the header:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
50
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
51 - the 16-bit checksum;
134
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
52 - the class ID for that chunk, which is the "bucket" where the chunk resides
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
53 for Primary backed allocations, or 0 for Secondary backed allocations;
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
54 - the size (Primary) or unused bytes amount (Secondary) for that chunk, which is
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
55 necessary for computing the size of the chunk;
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
56 - the state of the chunk (available, allocated or quarantined);
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
57 - the allocation type (malloc, new, new[] or memalign), to detect potential
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
58 mismatches in the allocation APIs used;
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
59 - the offset of the chunk, which is the distance in bytes from the beginning of
134
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
60 the returned chunk to the beginning of the Backend allocation;
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
61
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
62 This header fits within 8 bytes, on all platforms supported.
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
63
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
64 The checksum is computed as a CRC32 (made faster with hardware support)
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
65 of the global secret, the chunk pointer itself, and the 8 bytes of header with
134
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
66 the checksum field zeroed out. It is not intended to be cryptographically
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
67 strong.
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
68
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
69 The header is atomically loaded and stored to prevent races. This is important
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
70 as two consecutive chunks could belong to different threads. We also want to
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
71 avoid any type of double fetches of information located in the header, and use
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
72 local copies of the header for this purpose.
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
73
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
74 Delayed Freelist
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
75 -----------------
134
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
76 A delayed freelist allows us to not return a chunk directly to the Backend, but
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
77 to keep it aside for a while. Once a criterion is met, the delayed freelist is
134
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
78 emptied, and the quarantined chunks are returned to the Backend. This helps
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
79 mitigate use-after-free vulnerabilities by reducing the determinism of the
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
80 allocation and deallocation patterns.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
81
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
82 This feature is using the Sanitizer's Quarantine as its base, and the amount of
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
83 memory that it can hold is configurable by the user (see the Options section
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
84 below).
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
85
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
86 Randomness
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
87 ----------
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
88 It is important for the allocator to not make use of fixed addresses. We use
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
89 the dynamic base option for the SizeClassAllocator, allowing us to benefit
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
90 from the randomness of mmap.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
91
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
92 Usage
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
93 =====
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
94
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
95 Library
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
96 -------
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
97 The allocator static library can be built from the LLVM build tree thanks to
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
98 the ``scudo`` CMake rule. The associated tests can be exercised thanks to the
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
99 ``check-scudo`` CMake rule.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
100
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
101 Linking the static library to your project can require the use of the
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
102 ``whole-archive`` linker flag (or equivalent), depending on your linker.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
103 Additional flags might also be necessary.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
104
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
105 Your linked binary should now make use of the Scudo allocation and deallocation
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
106 functions.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
107
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
108 You may also build Scudo like this:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
109
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
110 .. code::
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
111
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
112 cd $LLVM/projects/compiler-rt/lib
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
113 clang++ -fPIC -std=c++11 -msse4.2 -O2 -I. scudo/*.cpp \
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
114 $(\ls sanitizer_common/*.{cc,S} | grep -v "sanitizer_termination\|sanitizer_common_nolibc") \
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
115 -shared -o scudo-allocator.so -pthread
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
116
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
117 and then use it with existing binaries as follows:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
118
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
119 .. code::
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
121 LD_PRELOAD=`pwd`/scudo-allocator.so ./a.out
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
122
134
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
123 Clang
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
124 -----
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
125 With a recent version of Clang (post rL317337), the allocator can be linked with
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
126 a binary at compilation using the ``-fsanitize=scudo`` command-line argument, if
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
127 the target platform is supported. Currently, the only other Sanitizer Scudo is
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
128 compatible with is UBSan (eg: ``-fsanitize=scudo,undefined``). Compiling with
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
129 Scudo will also enforce PIE for the output binary.
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
130
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
131 Options
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
132 -------
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
133 Several aspects of the allocator can be configured through the following ways:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
134
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
135 - by defining a ``__scudo_default_options`` function in one's program that
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
136 returns the options string to be parsed. Said function must have the following
134
3a76565eade5 update 5.0.1
mir3636
parents: 121
diff changeset
137 prototype: ``extern "C" const char* __scudo_default_options(void)``.
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
138
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
139 - through the environment variable SCUDO_OPTIONS, containing the options string
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
140 to be parsed. Options defined this way will override any definition made
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
141 through ``__scudo_default_options``;
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
142
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
143 The options string follows a syntax similar to ASan, where distinct options
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
144 can be assigned in the same string, separated by colons.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
145
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
146 For example, using the environment variable:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
147
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
148 .. code::
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
149
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
150 SCUDO_OPTIONS="DeleteSizeMismatch=1:QuarantineSizeKb=64" ./a.out
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
151
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
152 Or using the function:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
153
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
154 .. code:: cpp
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
155
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
156 extern "C" const char *__scudo_default_options() {
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
157 return "DeleteSizeMismatch=1:QuarantineSizeKb=64";
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
158 }
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
159
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
160
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
161 The following options are available:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
162
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
163 +-----------------------------+----------------+----------------+------------------------------------------------+
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
164 | Option | 64-bit default | 32-bit default | Description |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
165 +-----------------------------+----------------+----------------+------------------------------------------------+
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
166 | QuarantineSizeKb | 256 | 64 | The size (in Kb) of quarantine used to delay |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
167 | | | | the actual deallocation of chunks. Lower value |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
168 | | | | may reduce memory usage but decrease the |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
169 | | | | effectiveness of the mitigation; a negative |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
170 | | | | value will fallback to the defaults. |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
171 +-----------------------------+----------------+----------------+------------------------------------------------+
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
172 | QuarantineChunksUpToSize | 2048 | 512 | Size (in bytes) up to which chunks can be |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
173 | | | | quarantined. |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
174 +-----------------------------+----------------+----------------+------------------------------------------------+
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
175 | ThreadLocalQuarantineSizeKb | 1024 | 256 | The size (in Kb) of per-thread cache use to |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
176 | | | | offload the global quarantine. Lower value may |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
177 | | | | reduce memory usage but might increase |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
178 | | | | contention on the global quarantine. |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
179 +-----------------------------+----------------+----------------+------------------------------------------------+
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
180 | DeallocationTypeMismatch | true | true | Whether or not we report errors on |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
181 | | | | malloc/delete, new/free, new/delete[], etc. |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
182 +-----------------------------+----------------+----------------+------------------------------------------------+
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
183 | DeleteSizeMismatch | true | true | Whether or not we report errors on mismatch |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
184 | | | | between sizes of new and delete. |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
185 +-----------------------------+----------------+----------------+------------------------------------------------+
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
186 | ZeroContents | false | false | Whether or not we zero chunk contents on |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
187 | | | | allocation and deallocation. |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
188 +-----------------------------+----------------+----------------+------------------------------------------------+
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
189
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
190 Allocator related common Sanitizer options can also be passed through Scudo
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
191 options, such as ``allocator_may_return_null``. A detailed list including those
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
192 can be found here:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
193 https://github.com/google/sanitizers/wiki/SanitizerCommonFlags.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
194