annotate docs/ScudoHardenedAllocator.rst @ 120:1172e4bd9c6f

update 4.0.0
author mir3636
date Fri, 25 Nov 2016 19:14:25 +0900
parents
children 803732b1fca8
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
120
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
1 ========================
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
2 Scudo Hardened Allocator
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
3 ========================
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
4
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
5 .. contents::
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
6 :local:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
7 :depth: 1
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
8
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
9 Introduction
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
10 ============
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
11
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
12 The Scudo Hardened Allocator is a user-mode allocator based on LLVM Sanitizer's
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
13 CombinedAllocator, which aims at providing additional mitigations against heap
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
14 based vulnerabilities, while maintaining good performance.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
15
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
16 The name "Scudo" has been retained from the initial implementation (Escudo
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
17 meaning Shield in Spanish and Portuguese).
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
18
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
19 Design
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
20 ======
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
21
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
22 Chunk Header
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
23 ------------
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
24 Every chunk of heap memory will be preceded by a chunk header. This has two
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
25 purposes, the first one being to store various information about the chunk,
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
26 the second one being to detect potential heap overflows. In order to achieve
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
27 this, the header will be checksumed, involving the pointer to the chunk itself
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
28 and a global secret. Any corruption of the header will be detected when said
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
29 header is accessed, and the process terminated.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
30
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
31 The following information is stored in the header:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
32
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
33 - the 16-bit checksum;
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
34 - the user requested size for that chunk, which is necessary for reallocation
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
35 purposes;
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
36 - the state of the chunk (available, allocated or quarantined);
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
37 - the allocation type (malloc, new, new[] or memalign), to detect potential
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
38 mismatches in the allocation APIs used;
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
39 - whether or not the chunk is offseted (ie: if the chunk beginning is different
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
40 than the backend allocation beginning, which is most often the case with some
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
41 aligned allocations);
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
42 - the associated offset;
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
43 - a 16-bit salt.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
44
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
45 On x64, which is currently the only architecture supported, the header fits
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
46 within 16-bytes, which works nicely with the minimum alignment requirements.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
47
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
48 The checksum is computed as a CRC32 (requiring the SSE 4.2 instruction set)
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
49 of the global secret, the chunk pointer itself, and the 16 bytes of header with
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
50 the checksum field zeroed out.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
51
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
52 The header is atomically loaded and stored to prevent races (this requires
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
53 platform support such as the cmpxchg16b instruction). This is important as two
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
54 consecutive chunks could belong to different threads. We also want to avoid
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
55 any type of double fetches of information located in the header, and use local
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
56 copies of the header for this purpose.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
57
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
58 Delayed Freelist
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
59 -----------------
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
60 A delayed freelist allows us to not return a chunk directly to the backend, but
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
61 to keep it aside for a while. Once a criterion is met, the delayed freelist is
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
62 emptied, and the quarantined chunks are returned to the backend. This helps
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
63 mitigate use-after-free vulnerabilities by reducing the determinism of the
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
64 allocation and deallocation patterns.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
65
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
66 This feature is using the Sanitizer's Quarantine as its base, and the amount of
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
67 memory that it can hold is configurable by the user (see the Options section
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
68 below).
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
69
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
70 Randomness
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
71 ----------
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
72 It is important for the allocator to not make use of fixed addresses. We use
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
73 the dynamic base option for the SizeClassAllocator, allowing us to benefit
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
74 from the randomness of mmap.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
75
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
76 Usage
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
77 =====
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
78
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
79 Library
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
80 -------
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
81 The allocator static library can be built from the LLVM build tree thanks to
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
82 the ``scudo`` CMake rule. The associated tests can be exercised thanks to the
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
83 ``check-scudo`` CMake rule.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
84
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
85 Linking the static library to your project can require the use of the
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
86 ``whole-archive`` linker flag (or equivalent), depending on your linker.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
87 Additional flags might also be necessary.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
88
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
89 Your linked binary should now make use of the Scudo allocation and deallocation
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
90 functions.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
91
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
92 You may also build Scudo like this:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
93
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
94 .. code::
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
95
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
96 cd $LLVM/projects/compiler-rt/lib
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
97 clang++ -fPIC -std=c++11 -msse4.2 -mcx16 -O2 -I. scudo/*.cpp \
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
98 $(\ls sanitizer_common/*.{cc,S} | grep -v "sanitizer_termination\|sanitizer_common_nolibc") \
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
99 -shared -o scudo-allocator.so -lpthread
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
100
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
101 and then use it with existing binaries as follows:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
102
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
103 .. code::
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
104
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
105 LD_PRELOAD=`pwd`/scudo-allocator.so ./a.out
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
106
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
107 Options
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
108 -------
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
109 Several aspects of the allocator can be configured through the following ways:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
110
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
111 - by defining a ``__scudo_default_options`` function in one's program that
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
112 returns the options string to be parsed. Said function must have the following
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
113 prototype: ``extern "C" const char* __scudo_default_options()``.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
114
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
115 - through the environment variable SCUDO_OPTIONS, containing the options string
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
116 to be parsed. Options defined this way will override any definition made
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
117 through ``__scudo_default_options``;
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
118
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
119 The options string follows a syntax similar to ASan, where distinct options
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
120 can be assigned in the same string, separated by colons.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
121
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
122 For example, using the environment variable:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
123
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
124 .. code::
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
125
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
126 SCUDO_OPTIONS="DeleteSizeMismatch=1:QuarantineSizeMb=16" ./a.out
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
127
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
128 Or using the function:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
129
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
130 .. code::
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
131
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
132 extern "C" const char *__scudo_default_options() {
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
133 return "DeleteSizeMismatch=1:QuarantineSizeMb=16";
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
134 }
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
135
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
136
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
137 The following options are available:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
138
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
139 +-----------------------------+---------+------------------------------------------------+
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
140 | Option | Default | Description |
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
141 +-----------------------------+---------+------------------------------------------------+
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
142 | QuarantineSizeMb | 64 | The size (in Mb) of quarantine used to delay |
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
143 | | | the actual deallocation of chunks. Lower value |
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
144 | | | may reduce memory usage but decrease the |
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
145 | | | effectiveness of the mitigation; a negative |
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
146 | | | value will fallback to a default of 64Mb. |
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
147 +-----------------------------+---------+------------------------------------------------+
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
148 | ThreadLocalQuarantineSizeKb | 1024 | The size (in Kb) of per-thread cache use to |
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
149 | | | offload the global quarantine. Lower value may |
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
150 | | | reduce memory usage but might increase |
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
151 | | | contention on the global quarantine. |
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
152 +-----------------------------+---------+------------------------------------------------+
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
153 | DeallocationTypeMismatch | true | Whether or not we report errors on |
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
154 | | | malloc/delete, new/free, new/delete[], etc. |
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
155 +-----------------------------+---------+------------------------------------------------+
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
156 | DeleteSizeMismatch | true | Whether or not we report errors on mismatch |
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
157 | | | between sizes of new and delete. |
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
158 +-----------------------------+---------+------------------------------------------------+
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
159 | ZeroContents | false | Whether or not we zero chunk contents on |
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
160 | | | allocation and deallocation. |
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
161 +-----------------------------+---------+------------------------------------------------+
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
162
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
163 Allocator related common Sanitizer options can also be passed through Scudo
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
164 options, such as ``allocator_may_return_null``. A detailed list including those
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
165 can be found here:
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
166 https://github.com/google/sanitizers/wiki/SanitizerCommonFlags.
1172e4bd9c6f update 4.0.0
mir3636
parents:
diff changeset
167