annotate docs/ScudoHardenedAllocator.rst @ 122:36195a0db682

merging ( incomplete )
author Shinji KONO <kono@ie.u-ryukyu.ac.jp>
date Fri, 17 Nov 2017 20:32:31 +0900
parents 803732b1fca8
children 3a76565eade5
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
1 ========================
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
2 Scudo Hardened Allocator
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
3 ========================
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
4
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
5 .. contents::
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
6 :local:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
7 :depth: 1
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
8
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
9 Introduction
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
10 ============
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
11
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
12 The Scudo Hardened Allocator is a user-mode allocator based on LLVM Sanitizer's
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
13 CombinedAllocator, which aims at providing additional mitigations against heap
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
14 based vulnerabilities, while maintaining good performance.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
15
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
16 Currently, the allocator supports (was tested on) the following architectures:
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
17
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
18 - i386 (& i686) (32-bit);
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
19 - x86_64 (64-bit);
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
20 - armhf (32-bit);
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
21 - AArch64 (64-bit).
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
22
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
23 The name "Scudo" has been retained from the initial implementation (Escudo
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
24 meaning Shield in Spanish and Portuguese).
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
25
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
26 Design
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
27 ======
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
28
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
29 Chunk Header
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
30 ------------
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
31 Every chunk of heap memory will be preceded by a chunk header. This has two
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
32 purposes, the first one being to store various information about the chunk,
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
33 the second one being to detect potential heap overflows. In order to achieve
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
34 this, the header will be checksumed, involving the pointer to the chunk itself
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
35 and a global secret. Any corruption of the header will be detected when said
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
36 header is accessed, and the process terminated.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
37
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
38 The following information is stored in the header:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
39
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
40 - the 16-bit checksum;
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
41 - the unused bytes amount for that chunk, which is necessary for computing the
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
42 size of the chunk;
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
43 - the state of the chunk (available, allocated or quarantined);
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
44 - the allocation type (malloc, new, new[] or memalign), to detect potential
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
45 mismatches in the allocation APIs used;
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
46 - the offset of the chunk, which is the distance in bytes from the beginning of
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
47 the returned chunk to the beginning of the backend allocation;
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
48 - a 8-bit salt.
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
49
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
50 This header fits within 8 bytes, on all platforms supported.
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
51
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
52 The checksum is computed as a CRC32 (made faster with hardware support)
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
53 of the global secret, the chunk pointer itself, and the 8 bytes of header with
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
54 the checksum field zeroed out.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
55
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
56 The header is atomically loaded and stored to prevent races. This is important
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
57 as two consecutive chunks could belong to different threads. We also want to
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
58 avoid any type of double fetches of information located in the header, and use
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
59 local copies of the header for this purpose.
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
60
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
61 Delayed Freelist
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
62 -----------------
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
63 A delayed freelist allows us to not return a chunk directly to the backend, but
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
64 to keep it aside for a while. Once a criterion is met, the delayed freelist is
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
65 emptied, and the quarantined chunks are returned to the backend. This helps
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
66 mitigate use-after-free vulnerabilities by reducing the determinism of the
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
67 allocation and deallocation patterns.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
68
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
69 This feature is using the Sanitizer's Quarantine as its base, and the amount of
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
70 memory that it can hold is configurable by the user (see the Options section
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
71 below).
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
72
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
73 Randomness
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
74 ----------
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
75 It is important for the allocator to not make use of fixed addresses. We use
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
76 the dynamic base option for the SizeClassAllocator, allowing us to benefit
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
77 from the randomness of mmap.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
78
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
79 Usage
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
80 =====
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
81
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
82 Library
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
83 -------
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
84 The allocator static library can be built from the LLVM build tree thanks to
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
85 the ``scudo`` CMake rule. The associated tests can be exercised thanks to the
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
86 ``check-scudo`` CMake rule.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
87
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
88 Linking the static library to your project can require the use of the
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
89 ``whole-archive`` linker flag (or equivalent), depending on your linker.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
90 Additional flags might also be necessary.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
91
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
92 Your linked binary should now make use of the Scudo allocation and deallocation
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
93 functions.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
94
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
95 You may also build Scudo like this:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
96
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
97 .. code::
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
98
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
99 cd $LLVM/projects/compiler-rt/lib
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
100 clang++ -fPIC -std=c++11 -msse4.2 -O2 -I. scudo/*.cpp \
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
101 $(\ls sanitizer_common/*.{cc,S} | grep -v "sanitizer_termination\|sanitizer_common_nolibc") \
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
102 -shared -o scudo-allocator.so -pthread
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
103
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
104 and then use it with existing binaries as follows:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
105
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
106 .. code::
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
107
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
108 LD_PRELOAD=`pwd`/scudo-allocator.so ./a.out
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
109
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
110 Options
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
111 -------
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
112 Several aspects of the allocator can be configured through the following ways:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
113
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
114 - by defining a ``__scudo_default_options`` function in one's program that
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
115 returns the options string to be parsed. Said function must have the following
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
116 prototype: ``extern "C" const char* __scudo_default_options()``.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
117
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
118 - through the environment variable SCUDO_OPTIONS, containing the options string
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
119 to be parsed. Options defined this way will override any definition made
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
120 through ``__scudo_default_options``;
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
121
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
122 The options string follows a syntax similar to ASan, where distinct options
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
123 can be assigned in the same string, separated by colons.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
124
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
125 For example, using the environment variable:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
126
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
127 .. code::
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
128
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
129 SCUDO_OPTIONS="DeleteSizeMismatch=1:QuarantineSizeKb=64" ./a.out
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
130
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
131 Or using the function:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
132
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
133 .. code:: cpp
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
134
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
135 extern "C" const char *__scudo_default_options() {
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
136 return "DeleteSizeMismatch=1:QuarantineSizeKb=64";
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
137 }
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
138
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
139
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
140 The following options are available:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
141
121
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
142 +-----------------------------+----------------+----------------+------------------------------------------------+
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
143 | Option | 64-bit default | 32-bit default | Description |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
144 +-----------------------------+----------------+----------------+------------------------------------------------+
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
145 | QuarantineSizeKb | 256 | 64 | The size (in Kb) of quarantine used to delay |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
146 | | | | the actual deallocation of chunks. Lower value |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
147 | | | | may reduce memory usage but decrease the |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
148 | | | | effectiveness of the mitigation; a negative |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
149 | | | | value will fallback to the defaults. |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
150 +-----------------------------+----------------+----------------+------------------------------------------------+
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
151 | QuarantineChunksUpToSize | 2048 | 512 | Size (in bytes) up to which chunks can be |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
152 | | | | quarantined. |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
153 +-----------------------------+----------------+----------------+------------------------------------------------+
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
154 | ThreadLocalQuarantineSizeKb | 1024 | 256 | The size (in Kb) of per-thread cache use to |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
155 | | | | offload the global quarantine. Lower value may |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
156 | | | | reduce memory usage but might increase |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
157 | | | | contention on the global quarantine. |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
158 +-----------------------------+----------------+----------------+------------------------------------------------+
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
159 | DeallocationTypeMismatch | true | true | Whether or not we report errors on |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
160 | | | | malloc/delete, new/free, new/delete[], etc. |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
161 +-----------------------------+----------------+----------------+------------------------------------------------+
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
162 | DeleteSizeMismatch | true | true | Whether or not we report errors on mismatch |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
163 | | | | between sizes of new and delete. |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
164 +-----------------------------+----------------+----------------+------------------------------------------------+
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
165 | ZeroContents | false | false | Whether or not we zero chunk contents on |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
166 | | | | allocation and deallocation. |
803732b1fca8 LLVM 5.0
kono
parents: 120
diff changeset
167 +-----------------------------+----------------+----------------+------------------------------------------------+
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
168
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
169 Allocator related common Sanitizer options can also be passed through Scudo
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
170 options, such as ``allocator_may_return_null``. A detailed list including those
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
171 can be found here:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
172 https://github.com/google/sanitizers/wiki/SanitizerCommonFlags.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
173