|
150
|
1 ============================
|
|
|
2 "Clang" CFE Internals Manual
|
|
|
3 ============================
|
|
|
4
|
|
|
5 .. contents::
|
|
|
6 :local:
|
|
|
7
|
|
|
8 Introduction
|
|
|
9 ============
|
|
|
10
|
|
|
11 This document describes some of the more important APIs and internal design
|
|
|
12 decisions made in the Clang C front-end. The purpose of this document is to
|
|
|
13 both capture some of this high level information and also describe some of the
|
|
|
14 design decisions behind it. This is meant for people interested in hacking on
|
|
|
15 Clang, not for end-users. The description below is categorized by libraries,
|
|
|
16 and does not describe any of the clients of the libraries.
|
|
|
17
|
|
|
18 LLVM Support Library
|
|
|
19 ====================
|
|
|
20
|
|
|
21 The LLVM ``libSupport`` library provides many underlying libraries and
|
|
|
22 `data-structures <https://llvm.org/docs/ProgrammersManual.html>`_, including
|
|
|
23 command line option processing, various containers and a system abstraction
|
|
|
24 layer, which is used for file system access.
|
|
|
25
|
|
|
26 The Clang "Basic" Library
|
|
|
27 =========================
|
|
|
28
|
|
|
29 This library certainly needs a better name. The "basic" library contains a
|
|
|
30 number of low-level utilities for tracking and manipulating source buffers,
|
|
|
31 locations within the source buffers, diagnostics, tokens, target abstraction,
|
|
|
32 and information about the subset of the language being compiled for.
|
|
|
33
|
|
|
34 Part of this infrastructure is specific to C (such as the ``TargetInfo``
|
|
|
35 class), other parts could be reused for other non-C-based languages
|
|
|
36 (``SourceLocation``, ``SourceManager``, ``Diagnostics``, ``FileManager``).
|
|
|
37 When and if there is future demand we can figure out if it makes sense to
|
|
|
38 introduce a new library, move the general classes somewhere else, or introduce
|
|
|
39 some other solution.
|
|
|
40
|
|
|
41 We describe the roles of these classes in order of their dependencies.
|
|
|
42
|
|
|
43 The Diagnostics Subsystem
|
|
|
44 -------------------------
|
|
|
45
|
|
|
46 The Clang Diagnostics subsystem is an important part of how the compiler
|
|
|
47 communicates with the human. Diagnostics are the warnings and errors produced
|
|
|
48 when the code is incorrect or dubious. In Clang, each diagnostic produced has
|
|
|
49 (at the minimum) a unique ID, an English translation associated with it, a
|
|
|
50 :ref:`SourceLocation <SourceLocation>` to "put the caret", and a severity
|
|
|
51 (e.g., ``WARNING`` or ``ERROR``). They can also optionally include a number of
|
|
|
52 arguments to the diagnostic (which fill in "%0"'s in the string) as well as a
|
|
|
53 number of source ranges that related to the diagnostic.
|
|
|
54
|
|
|
55 In this section, we'll be giving examples produced by the Clang command line
|
|
|
56 driver, but diagnostics can be :ref:`rendered in many different ways
|
|
|
57 <DiagnosticConsumer>` depending on how the ``DiagnosticConsumer`` interface is
|
|
|
58 implemented. A representative example of a diagnostic is:
|
|
|
59
|
|
|
60 .. code-block:: text
|
|
|
61
|
|
|
62 t.c:38:15: error: invalid operands to binary expression ('int *' and '_Complex float')
|
|
|
63 P = (P-42) + Gamma*4;
|
|
|
64 ~~~~~~ ^ ~~~~~~~
|
|
|
65
|
|
|
66 In this example, you can see the English translation, the severity (error), you
|
|
|
67 can see the source location (the caret ("``^``") and file/line/column info),
|
|
|
68 the source ranges "``~~~~``", arguments to the diagnostic ("``int*``" and
|
|
|
69 "``_Complex float``"). You'll have to believe me that there is a unique ID
|
|
|
70 backing the diagnostic :).
|
|
|
71
|
|
|
72 Getting all of this to happen has several steps and involves many moving
|
|
|
73 pieces, this section describes them and talks about best practices when adding
|
|
|
74 a new diagnostic.
|
|
|
75
|
|
|
76 The ``Diagnostic*Kinds.td`` files
|
|
|
77 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
78
|
|
|
79 Diagnostics are created by adding an entry to one of the
|
|
|
80 ``clang/Basic/Diagnostic*Kinds.td`` files, depending on what library will be
|
|
|
81 using it. From this file, :program:`tblgen` generates the unique ID of the
|
|
|
82 diagnostic, the severity of the diagnostic and the English translation + format
|
|
|
83 string.
|
|
|
84
|
|
|
85 There is little sanity with the naming of the unique ID's right now. Some
|
|
|
86 start with ``err_``, ``warn_``, ``ext_`` to encode the severity into the name.
|
|
|
87 Since the enum is referenced in the C++ code that produces the diagnostic, it
|
|
|
88 is somewhat useful for it to be reasonably short.
|
|
|
89
|
|
|
90 The severity of the diagnostic comes from the set {``NOTE``, ``REMARK``,
|
|
|
91 ``WARNING``,
|
|
|
92 ``EXTENSION``, ``EXTWARN``, ``ERROR``}. The ``ERROR`` severity is used for
|
|
|
93 diagnostics indicating the program is never acceptable under any circumstances.
|
|
|
94 When an error is emitted, the AST for the input code may not be fully built.
|
|
|
95 The ``EXTENSION`` and ``EXTWARN`` severities are used for extensions to the
|
|
|
96 language that Clang accepts. This means that Clang fully understands and can
|
|
|
97 represent them in the AST, but we produce diagnostics to tell the user their
|
|
|
98 code is non-portable. The difference is that the former are ignored by
|
|
|
99 default, and the later warn by default. The ``WARNING`` severity is used for
|
|
|
100 constructs that are valid in the currently selected source language but that
|
|
|
101 are dubious in some way. The ``REMARK`` severity provides generic information
|
|
|
102 about the compilation that is not necessarily related to any dubious code. The
|
|
|
103 ``NOTE`` level is used to staple more information onto previous diagnostics.
|
|
|
104
|
|
|
105 These *severities* are mapped into a smaller set (the ``Diagnostic::Level``
|
|
|
106 enum, {``Ignored``, ``Note``, ``Remark``, ``Warning``, ``Error``, ``Fatal``}) of
|
|
|
107 output
|
|
|
108 *levels* by the diagnostics subsystem based on various configuration options.
|
|
|
109 Clang internally supports a fully fine grained mapping mechanism that allows
|
|
|
110 you to map almost any diagnostic to the output level that you want. The only
|
|
|
111 diagnostics that cannot be mapped are ``NOTE``\ s, which always follow the
|
|
|
112 severity of the previously emitted diagnostic and ``ERROR``\ s, which can only
|
|
|
113 be mapped to ``Fatal`` (it is not possible to turn an error into a warning, for
|
|
|
114 example).
|
|
|
115
|
|
|
116 Diagnostic mappings are used in many ways. For example, if the user specifies
|
|
|
117 ``-pedantic``, ``EXTENSION`` maps to ``Warning``, if they specify
|
|
|
118 ``-pedantic-errors``, it turns into ``Error``. This is used to implement
|
|
|
119 options like ``-Wunused_macros``, ``-Wundef`` etc.
|
|
|
120
|
|
|
121 Mapping to ``Fatal`` should only be used for diagnostics that are considered so
|
|
|
122 severe that error recovery won't be able to recover sensibly from them (thus
|
|
|
123 spewing a ton of bogus errors). One example of this class of error are failure
|
|
|
124 to ``#include`` a file.
|
|
|
125
|
|
|
126 The Format String
|
|
|
127 ^^^^^^^^^^^^^^^^^
|
|
|
128
|
|
|
129 The format string for the diagnostic is very simple, but it has some power. It
|
|
|
130 takes the form of a string in English with markers that indicate where and how
|
|
|
131 arguments to the diagnostic are inserted and formatted. For example, here are
|
|
|
132 some simple format strings:
|
|
|
133
|
|
|
134 .. code-block:: c++
|
|
|
135
|
|
|
136 "binary integer literals are an extension"
|
|
|
137 "format string contains '\\0' within the string body"
|
|
|
138 "more '%%' conversions than data arguments"
|
|
|
139 "invalid operands to binary expression (%0 and %1)"
|
|
|
140 "overloaded '%0' must be a %select{unary|binary|unary or binary}2 operator"
|
|
|
141 " (has %1 parameter%s1)"
|
|
|
142
|
|
|
143 These examples show some important points of format strings. You can use any
|
|
|
144 plain ASCII character in the diagnostic string except "``%``" without a
|
|
|
145 problem, but these are C strings, so you have to use and be aware of all the C
|
|
|
146 escape sequences (as in the second example). If you want to produce a "``%``"
|
|
|
147 in the output, use the "``%%``" escape sequence, like the third diagnostic.
|
|
|
148 Finally, Clang uses the "``%...[digit]``" sequences to specify where and how
|
|
|
149 arguments to the diagnostic are formatted.
|
|
|
150
|
|
|
151 Arguments to the diagnostic are numbered according to how they are specified by
|
|
|
152 the C++ code that :ref:`produces them <internals-producing-diag>`, and are
|
|
|
153 referenced by ``%0`` .. ``%9``. If you have more than 10 arguments to your
|
|
|
154 diagnostic, you are doing something wrong :). Unlike ``printf``, there is no
|
|
|
155 requirement that arguments to the diagnostic end up in the output in the same
|
|
|
156 order as they are specified, you could have a format string with "``%1 %0``"
|
|
|
157 that swaps them, for example. The text in between the percent and digit are
|
|
|
158 formatting instructions. If there are no instructions, the argument is just
|
|
|
159 turned into a string and substituted in.
|
|
|
160
|
|
|
161 Here are some "best practices" for writing the English format string:
|
|
|
162
|
|
|
163 * Keep the string short. It should ideally fit in the 80 column limit of the
|
|
|
164 ``DiagnosticKinds.td`` file. This avoids the diagnostic wrapping when
|
|
|
165 printed, and forces you to think about the important point you are conveying
|
|
|
166 with the diagnostic.
|
|
|
167 * Take advantage of location information. The user will be able to see the
|
|
|
168 line and location of the caret, so you don't need to tell them that the
|
|
|
169 problem is with the 4th argument to the function: just point to it.
|
|
|
170 * Do not capitalize the diagnostic string, and do not end it with a period.
|
|
|
171 * If you need to quote something in the diagnostic string, use single quotes.
|
|
|
172
|
|
|
173 Diagnostics should never take random English strings as arguments: you
|
|
|
174 shouldn't use "``you have a problem with %0``" and pass in things like "``your
|
|
|
175 argument``" or "``your return value``" as arguments. Doing this prevents
|
|
|
176 :ref:`translating <internals-diag-translation>` the Clang diagnostics to other
|
|
|
177 languages (because they'll get random English words in their otherwise
|
|
|
178 localized diagnostic). The exceptions to this are C/C++ language keywords
|
|
|
179 (e.g., ``auto``, ``const``, ``mutable``, etc) and C/C++ operators (``/=``).
|
|
|
180 Note that things like "pointer" and "reference" are not keywords. On the other
|
|
|
181 hand, you *can* include anything that comes from the user's source code,
|
|
|
182 including variable names, types, labels, etc. The "``select``" format can be
|
|
|
183 used to achieve this sort of thing in a localizable way, see below.
|
|
|
184
|
|
|
185 Formatting a Diagnostic Argument
|
|
|
186 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
187
|
|
|
188 Arguments to diagnostics are fully typed internally, and come from a couple
|
|
|
189 different classes: integers, types, names, and random strings. Depending on
|
|
|
190 the class of the argument, it can be optionally formatted in different ways.
|
|
|
191 This gives the ``DiagnosticConsumer`` information about what the argument means
|
|
|
192 without requiring it to use a specific presentation (consider this MVC for
|
|
|
193 Clang :).
|
|
|
194
|
|
|
195 Here are the different diagnostic argument formats currently supported by
|
|
|
196 Clang:
|
|
|
197
|
|
|
198 **"s" format**
|
|
|
199
|
|
|
200 Example:
|
|
|
201 ``"requires %1 parameter%s1"``
|
|
|
202 Class:
|
|
|
203 Integers
|
|
|
204 Description:
|
|
|
205 This is a simple formatter for integers that is useful when producing English
|
|
|
206 diagnostics. When the integer is 1, it prints as nothing. When the integer
|
|
|
207 is not 1, it prints as "``s``". This allows some simple grammatical forms to
|
|
|
208 be to be handled correctly, and eliminates the need to use gross things like
|
|
|
209 ``"requires %1 parameter(s)"``.
|
|
|
210
|
|
|
211 **"select" format**
|
|
|
212
|
|
|
213 Example:
|
|
|
214 ``"must be a %select{unary|binary|unary or binary}2 operator"``
|
|
|
215 Class:
|
|
|
216 Integers
|
|
|
217 Description:
|
|
|
218 This format specifier is used to merge multiple related diagnostics together
|
|
|
219 into one common one, without requiring the difference to be specified as an
|
|
|
220 English string argument. Instead of specifying the string, the diagnostic
|
|
|
221 gets an integer argument and the format string selects the numbered option.
|
|
|
222 In this case, the "``%2``" value must be an integer in the range [0..2]. If
|
|
|
223 it is 0, it prints "unary", if it is 1 it prints "binary" if it is 2, it
|
|
|
224 prints "unary or binary". This allows other language translations to
|
|
|
225 substitute reasonable words (or entire phrases) based on the semantics of the
|
|
|
226 diagnostic instead of having to do things textually. The selected string
|
|
|
227 does undergo formatting.
|
|
|
228
|
|
|
229 **"plural" format**
|
|
|
230
|
|
|
231 Example:
|
|
|
232 ``"you have %1 %plural{1:mouse|:mice}1 connected to your computer"``
|
|
|
233 Class:
|
|
|
234 Integers
|
|
|
235 Description:
|
|
|
236 This is a formatter for complex plural forms. It is designed to handle even
|
|
|
237 the requirements of languages with very complex plural forms, as many Baltic
|
|
|
238 languages have. The argument consists of a series of expression/form pairs,
|
|
|
239 separated by ":", where the first form whose expression evaluates to true is
|
|
|
240 the result of the modifier.
|
|
|
241
|
|
|
242 An expression can be empty, in which case it is always true. See the example
|
|
|
243 at the top. Otherwise, it is a series of one or more numeric conditions,
|
|
|
244 separated by ",". If any condition matches, the expression matches. Each
|
|
|
245 numeric condition can take one of three forms.
|
|
|
246
|
|
|
247 * number: A simple decimal number matches if the argument is the same as the
|
|
|
248 number. Example: ``"%plural{1:mouse|:mice}4"``
|
|
|
249 * range: A range in square brackets matches if the argument is within the
|
|
|
250 range. Then range is inclusive on both ends. Example:
|
|
|
251 ``"%plural{0:none|1:one|[2,5]:some|:many}2"``
|
|
|
252 * modulo: A modulo operator is followed by a number, and equals sign and
|
|
|
253 either a number or a range. The tests are the same as for plain numbers
|
|
|
254 and ranges, but the argument is taken modulo the number first. Example:
|
|
|
255 ``"%plural{%100=0:even hundred|%100=[1,50]:lower half|:everything else}1"``
|
|
|
256
|
|
|
257 The parser is very unforgiving. A syntax error, even whitespace, will abort,
|
|
|
258 as will a failure to match the argument against any expression.
|
|
|
259
|
|
|
260 **"ordinal" format**
|
|
|
261
|
|
|
262 Example:
|
|
|
263 ``"ambiguity in %ordinal0 argument"``
|
|
|
264 Class:
|
|
|
265 Integers
|
|
|
266 Description:
|
|
|
267 This is a formatter which represents the argument number as an ordinal: the
|
|
|
268 value ``1`` becomes ``1st``, ``3`` becomes ``3rd``, and so on. Values less
|
|
|
269 than ``1`` are not supported. This formatter is currently hard-coded to use
|
|
|
270 English ordinals.
|
|
|
271
|
|
|
272 **"objcclass" format**
|
|
|
273
|
|
|
274 Example:
|
|
|
275 ``"method %objcclass0 not found"``
|
|
|
276 Class:
|
|
|
277 ``DeclarationName``
|
|
|
278 Description:
|
|
|
279 This is a simple formatter that indicates the ``DeclarationName`` corresponds
|
|
|
280 to an Objective-C class method selector. As such, it prints the selector
|
|
|
281 with a leading "``+``".
|
|
|
282
|
|
|
283 **"objcinstance" format**
|
|
|
284
|
|
|
285 Example:
|
|
|
286 ``"method %objcinstance0 not found"``
|
|
|
287 Class:
|
|
|
288 ``DeclarationName``
|
|
|
289 Description:
|
|
|
290 This is a simple formatter that indicates the ``DeclarationName`` corresponds
|
|
|
291 to an Objective-C instance method selector. As such, it prints the selector
|
|
|
292 with a leading "``-``".
|
|
|
293
|
|
|
294 **"q" format**
|
|
|
295
|
|
|
296 Example:
|
|
|
297 ``"candidate found by name lookup is %q0"``
|
|
|
298 Class:
|
|
|
299 ``NamedDecl *``
|
|
|
300 Description:
|
|
|
301 This formatter indicates that the fully-qualified name of the declaration
|
|
|
302 should be printed, e.g., "``std::vector``" rather than "``vector``".
|
|
|
303
|
|
|
304 **"diff" format**
|
|
|
305
|
|
|
306 Example:
|
|
|
307 ``"no known conversion %diff{from $ to $|from argument type to parameter type}1,2"``
|
|
|
308 Class:
|
|
|
309 ``QualType``
|
|
|
310 Description:
|
|
|
311 This formatter takes two ``QualType``\ s and attempts to print a template
|
|
|
312 difference between the two. If tree printing is off, the text inside the
|
|
|
313 braces before the pipe is printed, with the formatted text replacing the $.
|
|
|
314 If tree printing is on, the text after the pipe is printed and a type tree is
|
|
|
315 printed after the diagnostic message.
|
|
|
316
|
|
|
317 It is really easy to add format specifiers to the Clang diagnostics system, but
|
|
|
318 they should be discussed before they are added. If you are creating a lot of
|
|
|
319 repetitive diagnostics and/or have an idea for a useful formatter, please bring
|
|
|
320 it up on the cfe-dev mailing list.
|
|
|
321
|
|
|
322 **"sub" format**
|
|
|
323
|
|
|
324 Example:
|
|
|
325 Given the following record definition of type ``TextSubstitution``:
|
|
|
326
|
|
|
327 .. code-block:: text
|
|
|
328
|
|
|
329 def select_ovl_candidate : TextSubstitution<
|
|
|
330 "%select{function|constructor}0%select{| template| %2}1">;
|
|
|
331
|
|
|
332 which can be used as
|
|
|
333
|
|
|
334 .. code-block:: text
|
|
|
335
|
|
|
336 def note_ovl_candidate : Note<
|
|
|
337 "candidate %sub{select_ovl_candidate}3,2,1 not viable">;
|
|
|
338
|
|
|
339 and will act as if it was written
|
|
|
340 ``"candidate %select{function|constructor}3%select{| template| %1}2 not viable"``.
|
|
|
341 Description:
|
|
|
342 This format specifier is used to avoid repeating strings verbatim in multiple
|
|
|
343 diagnostics. The argument to ``%sub`` must name a ``TextSubstitution`` tblgen
|
|
|
344 record. The substitution must specify all arguments used by the substitution,
|
|
|
345 and the modifier indexes in the substitution are re-numbered accordingly. The
|
|
|
346 substituted text must itself be a valid format string before substitution.
|
|
|
347
|
|
|
348 .. _internals-producing-diag:
|
|
|
349
|
|
|
350 Producing the Diagnostic
|
|
|
351 ^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
352
|
|
|
353 Now that you've created the diagnostic in the ``Diagnostic*Kinds.td`` file, you
|
|
|
354 need to write the code that detects the condition in question and emits the new
|
|
|
355 diagnostic. Various components of Clang (e.g., the preprocessor, ``Sema``,
|
|
|
356 etc.) provide a helper function named "``Diag``". It creates a diagnostic and
|
|
|
357 accepts the arguments, ranges, and other information that goes along with it.
|
|
|
358
|
|
|
359 For example, the binary expression error comes from code like this:
|
|
|
360
|
|
|
361 .. code-block:: c++
|
|
|
362
|
|
|
363 if (various things that are bad)
|
|
|
364 Diag(Loc, diag::err_typecheck_invalid_operands)
|
|
|
365 << lex->getType() << rex->getType()
|
|
|
366 << lex->getSourceRange() << rex->getSourceRange();
|
|
|
367
|
|
|
368 This shows that use of the ``Diag`` method: it takes a location (a
|
|
|
369 :ref:`SourceLocation <SourceLocation>` object) and a diagnostic enum value
|
|
|
370 (which matches the name from ``Diagnostic*Kinds.td``). If the diagnostic takes
|
|
|
371 arguments, they are specified with the ``<<`` operator: the first argument
|
|
|
372 becomes ``%0``, the second becomes ``%1``, etc. The diagnostic interface
|
|
|
373 allows you to specify arguments of many different types, including ``int`` and
|
|
|
374 ``unsigned`` for integer arguments, ``const char*`` and ``std::string`` for
|
|
|
375 string arguments, ``DeclarationName`` and ``const IdentifierInfo *`` for names,
|
|
|
376 ``QualType`` for types, etc. ``SourceRange``\ s are also specified with the
|
|
|
377 ``<<`` operator, but do not have a specific ordering requirement.
|
|
|
378
|
|
|
379 As you can see, adding and producing a diagnostic is pretty straightforward.
|
|
|
380 The hard part is deciding exactly what you need to say to help the user,
|
|
|
381 picking a suitable wording, and providing the information needed to format it
|
|
|
382 correctly. The good news is that the call site that issues a diagnostic should
|
|
|
383 be completely independent of how the diagnostic is formatted and in what
|
|
|
384 language it is rendered.
|
|
|
385
|
|
|
386 Fix-It Hints
|
|
|
387 ^^^^^^^^^^^^
|
|
|
388
|
|
|
389 In some cases, the front end emits diagnostics when it is clear that some small
|
|
|
390 change to the source code would fix the problem. For example, a missing
|
|
|
391 semicolon at the end of a statement or a use of deprecated syntax that is
|
|
|
392 easily rewritten into a more modern form. Clang tries very hard to emit the
|
|
|
393 diagnostic and recover gracefully in these and other cases.
|
|
|
394
|
|
|
395 However, for these cases where the fix is obvious, the diagnostic can be
|
|
|
396 annotated with a hint (referred to as a "fix-it hint") that describes how to
|
|
|
397 change the code referenced by the diagnostic to fix the problem. For example,
|
|
|
398 it might add the missing semicolon at the end of the statement or rewrite the
|
|
|
399 use of a deprecated construct into something more palatable. Here is one such
|
|
|
400 example from the C++ front end, where we warn about the right-shift operator
|
|
|
401 changing meaning from C++98 to C++11:
|
|
|
402
|
|
|
403 .. code-block:: text
|
|
|
404
|
|
|
405 test.cpp:3:7: warning: use of right-shift operator ('>>') in template argument
|
|
|
406 will require parentheses in C++11
|
|
|
407 A<100 >> 2> *a;
|
|
|
408 ^
|
|
|
409 ( )
|
|
|
410
|
|
|
411 Here, the fix-it hint is suggesting that parentheses be added, and showing
|
|
|
412 exactly where those parentheses would be inserted into the source code. The
|
|
|
413 fix-it hints themselves describe what changes to make to the source code in an
|
|
|
414 abstract manner, which the text diagnostic printer renders as a line of
|
|
|
415 "insertions" below the caret line. :ref:`Other diagnostic clients
|
|
|
416 <DiagnosticConsumer>` might choose to render the code differently (e.g., as
|
|
|
417 markup inline) or even give the user the ability to automatically fix the
|
|
|
418 problem.
|
|
|
419
|
|
|
420 Fix-it hints on errors and warnings need to obey these rules:
|
|
|
421
|
|
|
422 * Since they are automatically applied if ``-Xclang -fixit`` is passed to the
|
|
|
423 driver, they should only be used when it's very likely they match the user's
|
|
|
424 intent.
|
|
|
425 * Clang must recover from errors as if the fix-it had been applied.
|
|
|
426 * Fix-it hints on a warning must not change the meaning of the code.
|
|
|
427 However, a hint may clarify the meaning as intentional, for example by adding
|
|
|
428 parentheses when the precedence of operators isn't obvious.
|
|
|
429
|
|
|
430 If a fix-it can't obey these rules, put the fix-it on a note. Fix-its on notes
|
|
|
431 are not applied automatically.
|
|
|
432
|
|
|
433 All fix-it hints are described by the ``FixItHint`` class, instances of which
|
|
|
434 should be attached to the diagnostic using the ``<<`` operator in the same way
|
|
|
435 that highlighted source ranges and arguments are passed to the diagnostic.
|
|
|
436 Fix-it hints can be created with one of three constructors:
|
|
|
437
|
|
|
438 * ``FixItHint::CreateInsertion(Loc, Code)``
|
|
|
439
|
|
|
440 Specifies that the given ``Code`` (a string) should be inserted before the
|
|
|
441 source location ``Loc``.
|
|
|
442
|
|
|
443 * ``FixItHint::CreateRemoval(Range)``
|
|
|
444
|
|
|
445 Specifies that the code in the given source ``Range`` should be removed.
|
|
|
446
|
|
|
447 * ``FixItHint::CreateReplacement(Range, Code)``
|
|
|
448
|
|
|
449 Specifies that the code in the given source ``Range`` should be removed,
|
|
|
450 and replaced with the given ``Code`` string.
|
|
|
451
|
|
|
452 .. _DiagnosticConsumer:
|
|
|
453
|
|
|
454 The ``DiagnosticConsumer`` Interface
|
|
|
455 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
456
|
|
|
457 Once code generates a diagnostic with all of the arguments and the rest of the
|
|
|
458 relevant information, Clang needs to know what to do with it. As previously
|
|
|
459 mentioned, the diagnostic machinery goes through some filtering to map a
|
|
|
460 severity onto a diagnostic level, then (assuming the diagnostic is not mapped
|
|
|
461 to "``Ignore``") it invokes an object that implements the ``DiagnosticConsumer``
|
|
|
462 interface with the information.
|
|
|
463
|
|
|
464 It is possible to implement this interface in many different ways. For
|
|
|
465 example, the normal Clang ``DiagnosticConsumer`` (named
|
|
|
466 ``TextDiagnosticPrinter``) turns the arguments into strings (according to the
|
|
|
467 various formatting rules), prints out the file/line/column information and the
|
|
|
468 string, then prints out the line of code, the source ranges, and the caret.
|
|
|
469 However, this behavior isn't required.
|
|
|
470
|
|
|
471 Another implementation of the ``DiagnosticConsumer`` interface is the
|
|
|
472 ``TextDiagnosticBuffer`` class, which is used when Clang is in ``-verify``
|
|
|
473 mode. Instead of formatting and printing out the diagnostics, this
|
|
|
474 implementation just captures and remembers the diagnostics as they fly by.
|
|
|
475 Then ``-verify`` compares the list of produced diagnostics to the list of
|
|
|
476 expected ones. If they disagree, it prints out its own output. Full
|
|
|
477 documentation for the ``-verify`` mode can be found in the Clang API
|
|
|
478 documentation for `VerifyDiagnosticConsumer
|
|
|
479 </doxygen/classclang_1_1VerifyDiagnosticConsumer.html#details>`_.
|
|
|
480
|
|
|
481 There are many other possible implementations of this interface, and this is
|
|
|
482 why we prefer diagnostics to pass down rich structured information in
|
|
|
483 arguments. For example, an HTML output might want declaration names be
|
|
|
484 linkified to where they come from in the source. Another example is that a GUI
|
|
|
485 might let you click on typedefs to expand them. This application would want to
|
|
|
486 pass significantly more information about types through to the GUI than a
|
|
|
487 simple flat string. The interface allows this to happen.
|
|
|
488
|
|
|
489 .. _internals-diag-translation:
|
|
|
490
|
|
|
491 Adding Translations to Clang
|
|
|
492 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
493
|
|
|
494 Not possible yet! Diagnostic strings should be written in UTF-8, the client can
|
|
|
495 translate to the relevant code page if needed. Each translation completely
|
|
|
496 replaces the format string for the diagnostic.
|
|
|
497
|
|
|
498 .. _SourceLocation:
|
|
|
499 .. _SourceManager:
|
|
|
500
|
|
|
501 The ``SourceLocation`` and ``SourceManager`` classes
|
|
|
502 ----------------------------------------------------
|
|
|
503
|
|
|
504 Strangely enough, the ``SourceLocation`` class represents a location within the
|
|
|
505 source code of the program. Important design points include:
|
|
|
506
|
|
|
507 #. ``sizeof(SourceLocation)`` must be extremely small, as these are embedded
|
|
|
508 into many AST nodes and are passed around often. Currently it is 32 bits.
|
|
|
509 #. ``SourceLocation`` must be a simple value object that can be efficiently
|
|
|
510 copied.
|
|
|
511 #. We should be able to represent a source location for any byte of any input
|
|
|
512 file. This includes in the middle of tokens, in whitespace, in trigraphs,
|
|
|
513 etc.
|
|
|
514 #. A ``SourceLocation`` must encode the current ``#include`` stack that was
|
|
|
515 active when the location was processed. For example, if the location
|
|
|
516 corresponds to a token, it should contain the set of ``#include``\ s active
|
|
|
517 when the token was lexed. This allows us to print the ``#include`` stack
|
|
|
518 for a diagnostic.
|
|
|
519 #. ``SourceLocation`` must be able to describe macro expansions, capturing both
|
|
|
520 the ultimate instantiation point and the source of the original character
|
|
|
521 data.
|
|
|
522
|
|
|
523 In practice, the ``SourceLocation`` works together with the ``SourceManager``
|
|
|
524 class to encode two pieces of information about a location: its spelling
|
|
|
525 location and its expansion location. For most tokens, these will be the
|
|
|
526 same. However, for a macro expansion (or tokens that came from a ``_Pragma``
|
|
|
527 directive) these will describe the location of the characters corresponding to
|
|
|
528 the token and the location where the token was used (i.e., the macro
|
|
|
529 expansion point or the location of the ``_Pragma`` itself).
|
|
|
530
|
|
|
531 The Clang front-end inherently depends on the location of a token being tracked
|
|
|
532 correctly. If it is ever incorrect, the front-end may get confused and die.
|
|
|
533 The reason for this is that the notion of the "spelling" of a ``Token`` in
|
|
|
534 Clang depends on being able to find the original input characters for the
|
|
|
535 token. This concept maps directly to the "spelling location" for the token.
|
|
|
536
|
|
|
537 ``SourceRange`` and ``CharSourceRange``
|
|
|
538 ---------------------------------------
|
|
|
539
|
|
|
540 .. mostly taken from https://lists.llvm.org/pipermail/cfe-dev/2010-August/010595.html
|
|
|
541
|
|
|
542 Clang represents most source ranges by [first, last], where "first" and "last"
|
|
|
543 each point to the beginning of their respective tokens. For example consider
|
|
|
544 the ``SourceRange`` of the following statement:
|
|
|
545
|
|
|
546 .. code-block:: text
|
|
|
547
|
|
|
548 x = foo + bar;
|
|
|
549 ^first ^last
|
|
|
550
|
|
|
551 To map from this representation to a character-based representation, the "last"
|
|
|
552 location needs to be adjusted to point to (or past) the end of that token with
|
|
|
553 either ``Lexer::MeasureTokenLength()`` or ``Lexer::getLocForEndOfToken()``. For
|
|
|
554 the rare cases where character-level source ranges information is needed we use
|
|
|
555 the ``CharSourceRange`` class.
|
|
|
556
|
|
|
557 The Driver Library
|
|
|
558 ==================
|
|
|
559
|
|
|
560 The clang Driver and library are documented :doc:`here <DriverInternals>`.
|
|
|
561
|
|
|
562 Precompiled Headers
|
|
|
563 ===================
|
|
|
564
|
|
|
565 Clang supports precompiled headers (:doc:`PCH <PCHInternals>`), which uses a
|
|
|
566 serialized representation of Clang's internal data structures, encoded with the
|
|
|
567 `LLVM bitstream format <https://llvm.org/docs/BitCodeFormat.html>`_.
|
|
|
568
|
|
|
569 The Frontend Library
|
|
|
570 ====================
|
|
|
571
|
|
|
572 The Frontend library contains functionality useful for building tools on top of
|
|
|
573 the Clang libraries, for example several methods for outputting diagnostics.
|
|
|
574
|
|
|
575 The Lexer and Preprocessor Library
|
|
|
576 ==================================
|
|
|
577
|
|
|
578 The Lexer library contains several tightly-connected classes that are involved
|
|
|
579 with the nasty process of lexing and preprocessing C source code. The main
|
|
|
580 interface to this library for outside clients is the large ``Preprocessor``
|
|
|
581 class. It contains the various pieces of state that are required to coherently
|
|
|
582 read tokens out of a translation unit.
|
|
|
583
|
|
|
584 The core interface to the ``Preprocessor`` object (once it is set up) is the
|
|
|
585 ``Preprocessor::Lex`` method, which returns the next :ref:`Token <Token>` from
|
|
|
586 the preprocessor stream. There are two types of token providers that the
|
|
|
587 preprocessor is capable of reading from: a buffer lexer (provided by the
|
|
|
588 :ref:`Lexer <Lexer>` class) and a buffered token stream (provided by the
|
|
|
589 :ref:`TokenLexer <TokenLexer>` class).
|
|
|
590
|
|
|
591 .. _Token:
|
|
|
592
|
|
|
593 The Token class
|
|
|
594 ---------------
|
|
|
595
|
|
|
596 The ``Token`` class is used to represent a single lexed token. Tokens are
|
|
|
597 intended to be used by the lexer/preprocess and parser libraries, but are not
|
|
|
598 intended to live beyond them (for example, they should not live in the ASTs).
|
|
|
599
|
|
|
600 Tokens most often live on the stack (or some other location that is efficient
|
|
|
601 to access) as the parser is running, but occasionally do get buffered up. For
|
|
|
602 example, macro definitions are stored as a series of tokens, and the C++
|
|
|
603 front-end periodically needs to buffer tokens up for tentative parsing and
|
|
|
604 various pieces of look-ahead. As such, the size of a ``Token`` matters. On a
|
|
|
605 32-bit system, ``sizeof(Token)`` is currently 16 bytes.
|
|
|
606
|
|
|
607 Tokens occur in two forms: :ref:`annotation tokens <AnnotationToken>` and
|
|
|
608 normal tokens. Normal tokens are those returned by the lexer, annotation
|
|
|
609 tokens represent semantic information and are produced by the parser, replacing
|
|
|
610 normal tokens in the token stream. Normal tokens contain the following
|
|
|
611 information:
|
|
|
612
|
|
|
613 * **A SourceLocation** --- This indicates the location of the start of the
|
|
|
614 token.
|
|
|
615
|
|
|
616 * **A length** --- This stores the length of the token as stored in the
|
|
|
617 ``SourceBuffer``. For tokens that include them, this length includes
|
|
|
618 trigraphs and escaped newlines which are ignored by later phases of the
|
|
|
619 compiler. By pointing into the original source buffer, it is always possible
|
|
|
620 to get the original spelling of a token completely accurately.
|
|
|
621
|
|
|
622 * **IdentifierInfo** --- If a token takes the form of an identifier, and if
|
|
|
623 identifier lookup was enabled when the token was lexed (e.g., the lexer was
|
|
|
624 not reading in "raw" mode) this contains a pointer to the unique hash value
|
|
|
625 for the identifier. Because the lookup happens before keyword
|
|
|
626 identification, this field is set even for language keywords like "``for``".
|
|
|
627
|
|
|
628 * **TokenKind** --- This indicates the kind of token as classified by the
|
|
|
629 lexer. This includes things like ``tok::starequal`` (for the "``*=``"
|
|
|
630 operator), ``tok::ampamp`` for the "``&&``" token, and keyword values (e.g.,
|
|
|
631 ``tok::kw_for``) for identifiers that correspond to keywords. Note that
|
|
|
632 some tokens can be spelled multiple ways. For example, C++ supports
|
|
|
633 "operator keywords", where things like "``and``" are treated exactly like the
|
|
|
634 "``&&``" operator. In these cases, the kind value is set to ``tok::ampamp``,
|
|
|
635 which is good for the parser, which doesn't have to consider both forms. For
|
|
|
636 something that cares about which form is used (e.g., the preprocessor
|
|
|
637 "stringize" operator) the spelling indicates the original form.
|
|
|
638
|
|
|
639 * **Flags** --- There are currently four flags tracked by the
|
|
|
640 lexer/preprocessor system on a per-token basis:
|
|
|
641
|
|
|
642 #. **StartOfLine** --- This was the first token that occurred on its input
|
|
|
643 source line.
|
|
|
644 #. **LeadingSpace** --- There was a space character either immediately before
|
|
|
645 the token or transitively before the token as it was expanded through a
|
|
|
646 macro. The definition of this flag is very closely defined by the
|
|
|
647 stringizing requirements of the preprocessor.
|
|
|
648 #. **DisableExpand** --- This flag is used internally to the preprocessor to
|
|
|
649 represent identifier tokens which have macro expansion disabled. This
|
|
|
650 prevents them from being considered as candidates for macro expansion ever
|
|
|
651 in the future.
|
|
|
652 #. **NeedsCleaning** --- This flag is set if the original spelling for the
|
|
|
653 token includes a trigraph or escaped newline. Since this is uncommon,
|
|
|
654 many pieces of code can fast-path on tokens that did not need cleaning.
|
|
|
655
|
|
|
656 One interesting (and somewhat unusual) aspect of normal tokens is that they
|
|
|
657 don't contain any semantic information about the lexed value. For example, if
|
|
|
658 the token was a pp-number token, we do not represent the value of the number
|
|
|
659 that was lexed (this is left for later pieces of code to decide).
|
|
|
660 Additionally, the lexer library has no notion of typedef names vs variable
|
|
|
661 names: both are returned as identifiers, and the parser is left to decide
|
|
|
662 whether a specific identifier is a typedef or a variable (tracking this
|
|
|
663 requires scope information among other things). The parser can do this
|
|
|
664 translation by replacing tokens returned by the preprocessor with "Annotation
|
|
|
665 Tokens".
|
|
|
666
|
|
|
667 .. _AnnotationToken:
|
|
|
668
|
|
|
669 Annotation Tokens
|
|
|
670 -----------------
|
|
|
671
|
|
|
672 Annotation tokens are tokens that are synthesized by the parser and injected
|
|
|
673 into the preprocessor's token stream (replacing existing tokens) to record
|
|
|
674 semantic information found by the parser. For example, if "``foo``" is found
|
|
|
675 to be a typedef, the "``foo``" ``tok::identifier`` token is replaced with an
|
|
|
676 ``tok::annot_typename``. This is useful for a couple of reasons: 1) this makes
|
|
|
677 it easy to handle qualified type names (e.g., "``foo::bar::baz<42>::t``") in
|
|
|
678 C++ as a single "token" in the parser. 2) if the parser backtracks, the
|
|
|
679 reparse does not need to redo semantic analysis to determine whether a token
|
|
|
680 sequence is a variable, type, template, etc.
|
|
|
681
|
|
|
682 Annotation tokens are created by the parser and reinjected into the parser's
|
|
|
683 token stream (when backtracking is enabled). Because they can only exist in
|
|
|
684 tokens that the preprocessor-proper is done with, it doesn't need to keep
|
|
|
685 around flags like "start of line" that the preprocessor uses to do its job.
|
|
|
686 Additionally, an annotation token may "cover" a sequence of preprocessor tokens
|
|
|
687 (e.g., "``a::b::c``" is five preprocessor tokens). As such, the valid fields
|
|
|
688 of an annotation token are different than the fields for a normal token (but
|
|
|
689 they are multiplexed into the normal ``Token`` fields):
|
|
|
690
|
|
|
691 * **SourceLocation "Location"** --- The ``SourceLocation`` for the annotation
|
|
|
692 token indicates the first token replaced by the annotation token. In the
|
|
|
693 example above, it would be the location of the "``a``" identifier.
|
|
|
694 * **SourceLocation "AnnotationEndLoc"** --- This holds the location of the last
|
|
|
695 token replaced with the annotation token. In the example above, it would be
|
|
|
696 the location of the "``c``" identifier.
|
|
|
697 * **void* "AnnotationValue"** --- This contains an opaque object that the
|
|
|
698 parser gets from ``Sema``. The parser merely preserves the information for
|
|
|
699 ``Sema`` to later interpret based on the annotation token kind.
|
|
|
700 * **TokenKind "Kind"** --- This indicates the kind of Annotation token this is.
|
|
|
701 See below for the different valid kinds.
|
|
|
702
|
|
|
703 Annotation tokens currently come in three kinds:
|
|
|
704
|
|
|
705 #. **tok::annot_typename**: This annotation token represents a resolved
|
|
|
706 typename token that is potentially qualified. The ``AnnotationValue`` field
|
|
|
707 contains the ``QualType`` returned by ``Sema::getTypeName()``, possibly with
|
|
|
708 source location information attached.
|
|
|
709 #. **tok::annot_cxxscope**: This annotation token represents a C++ scope
|
|
|
710 specifier, such as "``A::B::``". This corresponds to the grammar
|
|
|
711 productions "*::*" and "*:: [opt] nested-name-specifier*". The
|
|
|
712 ``AnnotationValue`` pointer is a ``NestedNameSpecifier *`` returned by the
|
|
|
713 ``Sema::ActOnCXXGlobalScopeSpecifier`` and
|
|
|
714 ``Sema::ActOnCXXNestedNameSpecifier`` callbacks.
|
|
|
715 #. **tok::annot_template_id**: This annotation token represents a C++
|
|
|
716 template-id such as "``foo<int, 4>``", where "``foo``" is the name of a
|
|
|
717 template. The ``AnnotationValue`` pointer is a pointer to a ``malloc``'d
|
|
|
718 ``TemplateIdAnnotation`` object. Depending on the context, a parsed
|
|
|
719 template-id that names a type might become a typename annotation token (if
|
|
|
720 all we care about is the named type, e.g., because it occurs in a type
|
|
|
721 specifier) or might remain a template-id token (if we want to retain more
|
|
|
722 source location information or produce a new type, e.g., in a declaration of
|
|
|
723 a class template specialization). template-id annotation tokens that refer
|
|
|
724 to a type can be "upgraded" to typename annotation tokens by the parser.
|
|
|
725
|
|
|
726 As mentioned above, annotation tokens are not returned by the preprocessor,
|
|
|
727 they are formed on demand by the parser. This means that the parser has to be
|
|
|
728 aware of cases where an annotation could occur and form it where appropriate.
|
|
|
729 This is somewhat similar to how the parser handles Translation Phase 6 of C99:
|
|
|
730 String Concatenation (see C99 5.1.1.2). In the case of string concatenation,
|
|
|
731 the preprocessor just returns distinct ``tok::string_literal`` and
|
|
|
732 ``tok::wide_string_literal`` tokens and the parser eats a sequence of them
|
|
|
733 wherever the grammar indicates that a string literal can occur.
|
|
|
734
|
|
|
735 In order to do this, whenever the parser expects a ``tok::identifier`` or
|
|
|
736 ``tok::coloncolon``, it should call the ``TryAnnotateTypeOrScopeToken`` or
|
|
|
737 ``TryAnnotateCXXScopeToken`` methods to form the annotation token. These
|
|
|
738 methods will maximally form the specified annotation tokens and replace the
|
|
|
739 current token with them, if applicable. If the current tokens is not valid for
|
|
|
740 an annotation token, it will remain an identifier or "``::``" token.
|
|
|
741
|
|
|
742 .. _Lexer:
|
|
|
743
|
|
|
744 The ``Lexer`` class
|
|
|
745 -------------------
|
|
|
746
|
|
|
747 The ``Lexer`` class provides the mechanics of lexing tokens out of a source
|
|
|
748 buffer and deciding what they mean. The ``Lexer`` is complicated by the fact
|
|
|
749 that it operates on raw buffers that have not had spelling eliminated (this is
|
|
|
750 a necessity to get decent performance), but this is countered with careful
|
|
|
751 coding as well as standard performance techniques (for example, the comment
|
|
|
752 handling code is vectorized on X86 and PowerPC hosts).
|
|
|
753
|
|
|
754 The lexer has a couple of interesting modal features:
|
|
|
755
|
|
|
756 * The lexer can operate in "raw" mode. This mode has several features that
|
|
|
757 make it possible to quickly lex the file (e.g., it stops identifier lookup,
|
|
|
758 doesn't specially handle preprocessor tokens, handles EOF differently, etc).
|
|
|
759 This mode is used for lexing within an "``#if 0``" block, for example.
|
|
|
760 * The lexer can capture and return comments as tokens. This is required to
|
|
|
761 support the ``-C`` preprocessor mode, which passes comments through, and is
|
|
|
762 used by the diagnostic checker to identifier expect-error annotations.
|
|
|
763 * The lexer can be in ``ParsingFilename`` mode, which happens when
|
|
|
764 preprocessing after reading a ``#include`` directive. This mode changes the
|
|
|
765 parsing of "``<``" to return an "angled string" instead of a bunch of tokens
|
|
|
766 for each thing within the filename.
|
|
|
767 * When parsing a preprocessor directive (after "``#``") the
|
|
|
768 ``ParsingPreprocessorDirective`` mode is entered. This changes the parser to
|
|
|
769 return EOD at a newline.
|
|
|
770 * The ``Lexer`` uses a ``LangOptions`` object to know whether trigraphs are
|
|
|
771 enabled, whether C++ or ObjC keywords are recognized, etc.
|
|
|
772
|
|
|
773 In addition to these modes, the lexer keeps track of a couple of other features
|
|
|
774 that are local to a lexed buffer, which change as the buffer is lexed:
|
|
|
775
|
|
|
776 * The ``Lexer`` uses ``BufferPtr`` to keep track of the current character being
|
|
|
777 lexed.
|
|
|
778 * The ``Lexer`` uses ``IsAtStartOfLine`` to keep track of whether the next
|
|
|
779 lexed token will start with its "start of line" bit set.
|
|
|
780 * The ``Lexer`` keeps track of the current "``#if``" directives that are active
|
|
|
781 (which can be nested).
|
|
|
782 * The ``Lexer`` keeps track of an :ref:`MultipleIncludeOpt
|
|
|
783 <MultipleIncludeOpt>` object, which is used to detect whether the buffer uses
|
|
|
784 the standard "``#ifndef XX`` / ``#define XX``" idiom to prevent multiple
|
|
|
785 inclusion. If a buffer does, subsequent includes can be ignored if the
|
|
|
786 "``XX``" macro is defined.
|
|
|
787
|
|
|
788 .. _TokenLexer:
|
|
|
789
|
|
|
790 The ``TokenLexer`` class
|
|
|
791 ------------------------
|
|
|
792
|
|
|
793 The ``TokenLexer`` class is a token provider that returns tokens from a list of
|
|
|
794 tokens that came from somewhere else. It typically used for two things: 1)
|
|
|
795 returning tokens from a macro definition as it is being expanded 2) returning
|
|
|
796 tokens from an arbitrary buffer of tokens. The later use is used by
|
|
|
797 ``_Pragma`` and will most likely be used to handle unbounded look-ahead for the
|
|
|
798 C++ parser.
|
|
|
799
|
|
|
800 .. _MultipleIncludeOpt:
|
|
|
801
|
|
|
802 The ``MultipleIncludeOpt`` class
|
|
|
803 --------------------------------
|
|
|
804
|
|
|
805 The ``MultipleIncludeOpt`` class implements a really simple little state
|
|
|
806 machine that is used to detect the standard "``#ifndef XX`` / ``#define XX``"
|
|
|
807 idiom that people typically use to prevent multiple inclusion of headers. If a
|
|
|
808 buffer uses this idiom and is subsequently ``#include``'d, the preprocessor can
|
|
|
809 simply check to see whether the guarding condition is defined or not. If so,
|
|
|
810 the preprocessor can completely ignore the include of the header.
|
|
|
811
|
|
|
812 .. _Parser:
|
|
|
813
|
|
|
814 The Parser Library
|
|
|
815 ==================
|
|
|
816
|
|
|
817 This library contains a recursive-descent parser that polls tokens from the
|
|
|
818 preprocessor and notifies a client of the parsing progress.
|
|
|
819
|
|
|
820 Historically, the parser used to talk to an abstract ``Action`` interface that
|
|
|
821 had virtual methods for parse events, for example ``ActOnBinOp()``. When Clang
|
|
|
822 grew C++ support, the parser stopped supporting general ``Action`` clients --
|
|
|
823 it now always talks to the :ref:`Sema library <Sema>`. However, the Parser
|
|
|
824 still accesses AST objects only through opaque types like ``ExprResult`` and
|
|
|
825 ``StmtResult``. Only :ref:`Sema <Sema>` looks at the AST node contents of these
|
|
|
826 wrappers.
|
|
|
827
|
|
|
828 .. _AST:
|
|
|
829
|
|
|
830 The AST Library
|
|
|
831 ===============
|
|
|
832
|
|
|
833 .. _ASTPhilosophy:
|
|
|
834
|
|
|
835 Design philosophy
|
|
|
836 -----------------
|
|
|
837
|
|
|
838 Immutability
|
|
|
839 ^^^^^^^^^^^^
|
|
|
840
|
|
|
841 Clang AST nodes (types, declarations, statements, expressions, and so on) are
|
|
|
842 generally designed to be immutable once created. This provides a number of key
|
|
|
843 benefits:
|
|
|
844
|
|
|
845 * Canonicalization of the "meaning" of nodes is possible as soon as the nodes
|
|
|
846 are created, and is not invalidated by later addition of more information.
|
|
|
847 For example, we :ref:`canonicalize types <CanonicalType>`, and use a
|
|
|
848 canonicalized representation of expressions when determining whether two
|
|
|
849 function template declarations involving dependent expressions declare the
|
|
|
850 same entity.
|
|
|
851 * AST nodes can be reused when they have the same meaning. For example, we
|
|
|
852 reuse ``Type`` nodes when representing the same type (but maintain separate
|
|
|
853 ``TypeLoc``\s for each instance where a type is written), and we reuse
|
|
|
854 non-dependent ``Stmt`` and ``Expr`` nodes across instantiations of a
|
|
|
855 template.
|
|
|
856 * Serialization and deserialization of the AST to/from AST files is simpler:
|
|
|
857 we do not need to track modifications made to AST nodes imported from AST
|
|
|
858 files and serialize separate "update records".
|
|
|
859
|
|
|
860 There are unfortunately exceptions to this general approach, such as:
|
|
|
861
|
|
|
862 * The first declaration of a redeclarable entity maintains a pointer to the
|
|
|
863 most recent declaration of that entity, which naturally needs to change as
|
|
|
864 more declarations are parsed.
|
|
|
865 * Name lookup tables in declaration contexts change after the namespace
|
|
|
866 declaration is formed.
|
|
|
867 * We attempt to maintain only a single declaration for an instantiation of a
|
|
|
868 template, rather than having distinct declarations for an instantiation of
|
|
|
869 the declaration versus the definition, so template instantiation often
|
|
|
870 updates parts of existing declarations.
|
|
|
871 * Some parts of declarations are required to be instantiated separately (this
|
|
|
872 includes default arguments and exception specifications), and such
|
|
|
873 instantiations update the existing declaration.
|
|
|
874
|
|
|
875 These cases tend to be fragile; mutable AST state should be avoided where
|
|
|
876 possible.
|
|
|
877
|
|
|
878 As a consequence of this design principle, we typically do not provide setters
|
|
|
879 for AST state. (Some are provided for short-term modifications intended to be
|
|
|
880 used immediately after an AST node is created and before it's "published" as
|
|
|
881 part of the complete AST, or where language semantics require after-the-fact
|
|
|
882 updates.)
|
|
|
883
|
|
|
884 Faithfulness
|
|
|
885 ^^^^^^^^^^^^
|
|
|
886
|
|
|
887 The AST intends to provide a representation of the program that is faithful to
|
|
|
888 the original source. We intend for it to be possible to write refactoring tools
|
|
|
889 using only information stored in, or easily reconstructible from, the Clang AST.
|
|
|
890 This means that the AST representation should either not desugar source-level
|
|
|
891 constructs to simpler forms, or -- where made necessary by language semantics
|
|
|
892 or a clear engineering tradeoff -- should desugar minimally and wrap the result
|
|
|
893 in a construct representing the original source form.
|
|
|
894
|
|
|
895 For example, ``CXXForRangeStmt`` directly represents the syntactic form of a
|
|
|
896 range-based for statement, but also holds a semantic representation of the
|
|
|
897 range declaration and iterator declarations. It does not contain a
|
|
|
898 fully-desugared ``ForStmt``, however.
|
|
|
899
|
|
|
900 Some AST nodes (for example, ``ParenExpr``) represent only syntax, and others
|
|
|
901 (for example, ``ImplicitCastExpr``) represent only semantics, but most nodes
|
|
|
902 will represent a combination of syntax and associated semantics. Inheritance
|
|
|
903 is typically used when representing different (but related) syntaxes for nodes
|
|
|
904 with the same or similar semantics.
|
|
|
905
|
|
|
906 .. _Type:
|
|
|
907
|
|
|
908 The ``Type`` class and its subclasses
|
|
|
909 -------------------------------------
|
|
|
910
|
|
|
911 The ``Type`` class (and its subclasses) are an important part of the AST.
|
|
|
912 Types are accessed through the ``ASTContext`` class, which implicitly creates
|
|
|
913 and uniques them as they are needed. Types have a couple of non-obvious
|
|
|
914 features: 1) they do not capture type qualifiers like ``const`` or ``volatile``
|
|
|
915 (see :ref:`QualType <QualType>`), and 2) they implicitly capture typedef
|
|
|
916 information. Once created, types are immutable (unlike decls).
|
|
|
917
|
|
|
918 Typedefs in C make semantic analysis a bit more complex than it would be without
|
|
|
919 them. The issue is that we want to capture typedef information and represent it
|
|
|
920 in the AST perfectly, but the semantics of operations need to "see through"
|
|
|
921 typedefs. For example, consider this code:
|
|
|
922
|
|
|
923 .. code-block:: c++
|
|
|
924
|
|
|
925 void func() {
|
|
|
926 typedef int foo;
|
|
|
927 foo X, *Y;
|
|
|
928 typedef foo *bar;
|
|
|
929 bar Z;
|
|
|
930 *X; // error
|
|
|
931 **Y; // error
|
|
|
932 **Z; // error
|
|
|
933 }
|
|
|
934
|
|
|
935 The code above is illegal, and thus we expect there to be diagnostics emitted
|
|
|
936 on the annotated lines. In this example, we expect to get:
|
|
|
937
|
|
|
938 .. code-block:: text
|
|
|
939
|
|
|
940 test.c:6:1: error: indirection requires pointer operand ('foo' invalid)
|
|
|
941 *X; // error
|
|
|
942 ^~
|
|
|
943 test.c:7:1: error: indirection requires pointer operand ('foo' invalid)
|
|
|
944 **Y; // error
|
|
|
945 ^~~
|
|
|
946 test.c:8:1: error: indirection requires pointer operand ('foo' invalid)
|
|
|
947 **Z; // error
|
|
|
948 ^~~
|
|
|
949
|
|
|
950 While this example is somewhat silly, it illustrates the point: we want to
|
|
|
951 retain typedef information where possible, so that we can emit errors about
|
|
|
952 "``std::string``" instead of "``std::basic_string<char, std:...``". Doing this
|
|
|
953 requires properly keeping typedef information (for example, the type of ``X``
|
|
|
954 is "``foo``", not "``int``"), and requires properly propagating it through the
|
|
|
955 various operators (for example, the type of ``*Y`` is "``foo``", not
|
|
|
956 "``int``"). In order to retain this information, the type of these expressions
|
|
|
957 is an instance of the ``TypedefType`` class, which indicates that the type of
|
|
|
958 these expressions is a typedef for "``foo``".
|
|
|
959
|
|
|
960 Representing types like this is great for diagnostics, because the
|
|
|
961 user-specified type is always immediately available. There are two problems
|
|
|
962 with this: first, various semantic checks need to make judgements about the
|
|
|
963 *actual structure* of a type, ignoring typedefs. Second, we need an efficient
|
|
|
964 way to query whether two types are structurally identical to each other,
|
|
|
965 ignoring typedefs. The solution to both of these problems is the idea of
|
|
|
966 canonical types.
|
|
|
967
|
|
|
968 .. _CanonicalType:
|
|
|
969
|
|
|
970 Canonical Types
|
|
|
971 ^^^^^^^^^^^^^^^
|
|
|
972
|
|
|
973 Every instance of the ``Type`` class contains a canonical type pointer. For
|
|
|
974 simple types with no typedefs involved (e.g., "``int``", "``int*``",
|
|
|
975 "``int**``"), the type just points to itself. For types that have a typedef
|
|
|
976 somewhere in their structure (e.g., "``foo``", "``foo*``", "``foo**``",
|
|
|
977 "``bar``"), the canonical type pointer points to their structurally equivalent
|
|
|
978 type without any typedefs (e.g., "``int``", "``int*``", "``int**``", and
|
|
|
979 "``int*``" respectively).
|
|
|
980
|
|
|
981 This design provides a constant time operation (dereferencing the canonical type
|
|
|
982 pointer) that gives us access to the structure of types. For example, we can
|
|
|
983 trivially tell that "``bar``" and "``foo*``" are the same type by dereferencing
|
|
|
984 their canonical type pointers and doing a pointer comparison (they both point
|
|
|
985 to the single "``int*``" type).
|
|
|
986
|
|
|
987 Canonical types and typedef types bring up some complexities that must be
|
|
|
988 carefully managed. Specifically, the ``isa``/``cast``/``dyn_cast`` operators
|
|
|
989 generally shouldn't be used in code that is inspecting the AST. For example,
|
|
|
990 when type checking the indirection operator (unary "``*``" on a pointer), the
|
|
|
991 type checker must verify that the operand has a pointer type. It would not be
|
|
|
992 correct to check that with "``isa<PointerType>(SubExpr->getType())``", because
|
|
|
993 this predicate would fail if the subexpression had a typedef type.
|
|
|
994
|
|
|
995 The solution to this problem are a set of helper methods on ``Type``, used to
|
|
|
996 check their properties. In this case, it would be correct to use
|
|
|
997 "``SubExpr->getType()->isPointerType()``" to do the check. This predicate will
|
|
|
998 return true if the *canonical type is a pointer*, which is true any time the
|
|
|
999 type is structurally a pointer type. The only hard part here is remembering
|
|
|
1000 not to use the ``isa``/``cast``/``dyn_cast`` operations.
|
|
|
1001
|
|
|
1002 The second problem we face is how to get access to the pointer type once we
|
|
|
1003 know it exists. To continue the example, the result type of the indirection
|
|
|
1004 operator is the pointee type of the subexpression. In order to determine the
|
|
|
1005 type, we need to get the instance of ``PointerType`` that best captures the
|
|
|
1006 typedef information in the program. If the type of the expression is literally
|
|
|
1007 a ``PointerType``, we can return that, otherwise we have to dig through the
|
|
|
1008 typedefs to find the pointer type. For example, if the subexpression had type
|
|
|
1009 "``foo*``", we could return that type as the result. If the subexpression had
|
|
|
1010 type "``bar``", we want to return "``foo*``" (note that we do *not* want
|
|
|
1011 "``int*``"). In order to provide all of this, ``Type`` has a
|
|
|
1012 ``getAsPointerType()`` method that checks whether the type is structurally a
|
|
|
1013 ``PointerType`` and, if so, returns the best one. If not, it returns a null
|
|
|
1014 pointer.
|
|
|
1015
|
|
|
1016 This structure is somewhat mystical, but after meditating on it, it will make
|
|
|
1017 sense to you :).
|
|
|
1018
|
|
|
1019 .. _QualType:
|
|
|
1020
|
|
|
1021 The ``QualType`` class
|
|
|
1022 ----------------------
|
|
|
1023
|
|
|
1024 The ``QualType`` class is designed as a trivial value class that is small,
|
|
|
1025 passed by-value and is efficient to query. The idea of ``QualType`` is that it
|
|
|
1026 stores the type qualifiers (``const``, ``volatile``, ``restrict``, plus some
|
|
|
1027 extended qualifiers required by language extensions) separately from the types
|
|
|
1028 themselves. ``QualType`` is conceptually a pair of "``Type*``" and the bits
|
|
|
1029 for these type qualifiers.
|
|
|
1030
|
|
|
1031 By storing the type qualifiers as bits in the conceptual pair, it is extremely
|
|
|
1032 efficient to get the set of qualifiers on a ``QualType`` (just return the field
|
|
|
1033 of the pair), add a type qualifier (which is a trivial constant-time operation
|
|
|
1034 that sets a bit), and remove one or more type qualifiers (just return a
|
|
|
1035 ``QualType`` with the bitfield set to empty).
|
|
|
1036
|
|
|
1037 Further, because the bits are stored outside of the type itself, we do not need
|
|
|
1038 to create duplicates of types with different sets of qualifiers (i.e. there is
|
|
|
1039 only a single heap allocated "``int``" type: "``const int``" and "``volatile
|
|
|
1040 const int``" both point to the same heap allocated "``int``" type). This
|
|
|
1041 reduces the heap size used to represent bits and also means we do not have to
|
|
|
1042 consider qualifiers when uniquing types (:ref:`Type <Type>` does not even
|
|
|
1043 contain qualifiers).
|
|
|
1044
|
|
|
1045 In practice, the two most common type qualifiers (``const`` and ``restrict``)
|
|
|
1046 are stored in the low bits of the pointer to the ``Type`` object, together with
|
|
|
1047 a flag indicating whether extended qualifiers are present (which must be
|
|
|
1048 heap-allocated). This means that ``QualType`` is exactly the same size as a
|
|
|
1049 pointer.
|
|
|
1050
|
|
|
1051 .. _DeclarationName:
|
|
|
1052
|
|
|
1053 Declaration names
|
|
|
1054 -----------------
|
|
|
1055
|
|
|
1056 The ``DeclarationName`` class represents the name of a declaration in Clang.
|
|
|
1057 Declarations in the C family of languages can take several different forms.
|
|
|
1058 Most declarations are named by simple identifiers, e.g., "``f``" and "``x``" in
|
|
|
1059 the function declaration ``f(int x)``. In C++, declaration names can also name
|
|
|
1060 class constructors ("``Class``" in ``struct Class { Class(); }``), class
|
|
|
1061 destructors ("``~Class``"), overloaded operator names ("``operator+``"), and
|
|
|
1062 conversion functions ("``operator void const *``"). In Objective-C,
|
|
|
1063 declaration names can refer to the names of Objective-C methods, which involve
|
|
|
1064 the method name and the parameters, collectively called a *selector*, e.g.,
|
|
|
1065 "``setWidth:height:``". Since all of these kinds of entities --- variables,
|
|
|
1066 functions, Objective-C methods, C++ constructors, destructors, and operators
|
|
|
1067 --- are represented as subclasses of Clang's common ``NamedDecl`` class,
|
|
|
1068 ``DeclarationName`` is designed to efficiently represent any kind of name.
|
|
|
1069
|
|
|
1070 Given a ``DeclarationName`` ``N``, ``N.getNameKind()`` will produce a value
|
|
|
1071 that describes what kind of name ``N`` stores. There are 10 options (all of
|
|
|
1072 the names are inside the ``DeclarationName`` class).
|
|
|
1073
|
|
|
1074 ``Identifier``
|
|
|
1075
|
|
|
1076 The name is a simple identifier. Use ``N.getAsIdentifierInfo()`` to retrieve
|
|
|
1077 the corresponding ``IdentifierInfo*`` pointing to the actual identifier.
|
|
|
1078
|
|
|
1079 ``ObjCZeroArgSelector``, ``ObjCOneArgSelector``, ``ObjCMultiArgSelector``
|
|
|
1080
|
|
|
1081 The name is an Objective-C selector, which can be retrieved as a ``Selector``
|
|
|
1082 instance via ``N.getObjCSelector()``. The three possible name kinds for
|
|
|
1083 Objective-C reflect an optimization within the ``DeclarationName`` class:
|
|
|
1084 both zero- and one-argument selectors are stored as a masked
|
|
|
1085 ``IdentifierInfo`` pointer, and therefore require very little space, since
|
|
|
1086 zero- and one-argument selectors are far more common than multi-argument
|
|
|
1087 selectors (which use a different structure).
|
|
|
1088
|
|
|
1089 ``CXXConstructorName``
|
|
|
1090
|
|
|
1091 The name is a C++ constructor name. Use ``N.getCXXNameType()`` to retrieve
|
|
|
1092 the :ref:`type <QualType>` that this constructor is meant to construct. The
|
|
|
1093 type is always the canonical type, since all constructors for a given type
|
|
|
1094 have the same name.
|
|
|
1095
|
|
|
1096 ``CXXDestructorName``
|
|
|
1097
|
|
|
1098 The name is a C++ destructor name. Use ``N.getCXXNameType()`` to retrieve
|
|
|
1099 the :ref:`type <QualType>` whose destructor is being named. This type is
|
|
|
1100 always a canonical type.
|
|
|
1101
|
|
|
1102 ``CXXConversionFunctionName``
|
|
|
1103
|
|
|
1104 The name is a C++ conversion function. Conversion functions are named
|
|
|
1105 according to the type they convert to, e.g., "``operator void const *``".
|
|
|
1106 Use ``N.getCXXNameType()`` to retrieve the type that this conversion function
|
|
|
1107 converts to. This type is always a canonical type.
|
|
|
1108
|
|
|
1109 ``CXXOperatorName``
|
|
|
1110
|
|
|
1111 The name is a C++ overloaded operator name. Overloaded operators are named
|
|
|
1112 according to their spelling, e.g., "``operator+``" or "``operator new []``".
|
|
|
1113 Use ``N.getCXXOverloadedOperator()`` to retrieve the overloaded operator (a
|
|
|
1114 value of type ``OverloadedOperatorKind``).
|
|
|
1115
|
|
|
1116 ``CXXLiteralOperatorName``
|
|
|
1117
|
|
|
1118 The name is a C++11 user defined literal operator. User defined
|
|
|
1119 Literal operators are named according to the suffix they define,
|
|
|
1120 e.g., "``_foo``" for "``operator "" _foo``". Use
|
|
|
1121 ``N.getCXXLiteralIdentifier()`` to retrieve the corresponding
|
|
|
1122 ``IdentifierInfo*`` pointing to the identifier.
|
|
|
1123
|
|
|
1124 ``CXXUsingDirective``
|
|
|
1125
|
|
|
1126 The name is a C++ using directive. Using directives are not really
|
|
|
1127 NamedDecls, in that they all have the same name, but they are
|
|
|
1128 implemented as such in order to store them in DeclContext
|
|
|
1129 effectively.
|
|
|
1130
|
|
|
1131 ``DeclarationName``\ s are cheap to create, copy, and compare. They require
|
|
|
1132 only a single pointer's worth of storage in the common cases (identifiers,
|
|
|
1133 zero- and one-argument Objective-C selectors) and use dense, uniqued storage
|
|
|
1134 for the other kinds of names. Two ``DeclarationName``\ s can be compared for
|
|
|
1135 equality (``==``, ``!=``) using a simple bitwise comparison, can be ordered
|
|
|
1136 with ``<``, ``>``, ``<=``, and ``>=`` (which provide a lexicographical ordering
|
|
|
1137 for normal identifiers but an unspecified ordering for other kinds of names),
|
|
|
1138 and can be placed into LLVM ``DenseMap``\ s and ``DenseSet``\ s.
|
|
|
1139
|
|
|
1140 ``DeclarationName`` instances can be created in different ways depending on
|
|
|
1141 what kind of name the instance will store. Normal identifiers
|
|
|
1142 (``IdentifierInfo`` pointers) and Objective-C selectors (``Selector``) can be
|
|
|
1143 implicitly converted to ``DeclarationNames``. Names for C++ constructors,
|
|
|
1144 destructors, conversion functions, and overloaded operators can be retrieved
|
|
|
1145 from the ``DeclarationNameTable``, an instance of which is available as
|
|
|
1146 ``ASTContext::DeclarationNames``. The member functions
|
|
|
1147 ``getCXXConstructorName``, ``getCXXDestructorName``,
|
|
|
1148 ``getCXXConversionFunctionName``, and ``getCXXOperatorName``, respectively,
|
|
|
1149 return ``DeclarationName`` instances for the four kinds of C++ special function
|
|
|
1150 names.
|
|
|
1151
|
|
|
1152 .. _DeclContext:
|
|
|
1153
|
|
|
1154 Declaration contexts
|
|
|
1155 --------------------
|
|
|
1156
|
|
|
1157 Every declaration in a program exists within some *declaration context*, such
|
|
|
1158 as a translation unit, namespace, class, or function. Declaration contexts in
|
|
|
1159 Clang are represented by the ``DeclContext`` class, from which the various
|
|
|
1160 declaration-context AST nodes (``TranslationUnitDecl``, ``NamespaceDecl``,
|
|
|
1161 ``RecordDecl``, ``FunctionDecl``, etc.) will derive. The ``DeclContext`` class
|
|
|
1162 provides several facilities common to each declaration context:
|
|
|
1163
|
|
|
1164 Source-centric vs. Semantics-centric View of Declarations
|
|
|
1165
|
|
|
1166 ``DeclContext`` provides two views of the declarations stored within a
|
|
|
1167 declaration context. The source-centric view accurately represents the
|
|
|
1168 program source code as written, including multiple declarations of entities
|
|
|
1169 where present (see the section :ref:`Redeclarations and Overloads
|
|
|
1170 <Redeclarations>`), while the semantics-centric view represents the program
|
|
|
1171 semantics. The two views are kept synchronized by semantic analysis while
|
|
|
1172 the ASTs are being constructed.
|
|
|
1173
|
|
|
1174 Storage of declarations within that context
|
|
|
1175
|
|
|
1176 Every declaration context can contain some number of declarations. For
|
|
|
1177 example, a C++ class (represented by ``RecordDecl``) contains various member
|
|
|
1178 functions, fields, nested types, and so on. All of these declarations will
|
|
|
1179 be stored within the ``DeclContext``, and one can iterate over the
|
|
|
1180 declarations via [``DeclContext::decls_begin()``,
|
|
|
1181 ``DeclContext::decls_end()``). This mechanism provides the source-centric
|
|
|
1182 view of declarations in the context.
|
|
|
1183
|
|
|
1184 Lookup of declarations within that context
|
|
|
1185
|
|
|
1186 The ``DeclContext`` structure provides efficient name lookup for names within
|
|
|
1187 that declaration context. For example, if ``N`` is a namespace we can look
|
|
|
1188 for the name ``N::f`` using ``DeclContext::lookup``. The lookup itself is
|
|
|
1189 based on a lazily-constructed array (for declaration contexts with a small
|
|
|
1190 number of declarations) or hash table (for declaration contexts with more
|
|
|
1191 declarations). The lookup operation provides the semantics-centric view of
|
|
|
1192 the declarations in the context.
|
|
|
1193
|
|
|
1194 Ownership of declarations
|
|
|
1195
|
|
|
1196 The ``DeclContext`` owns all of the declarations that were declared within
|
|
|
1197 its declaration context, and is responsible for the management of their
|
|
|
1198 memory as well as their (de-)serialization.
|
|
|
1199
|
|
|
1200 All declarations are stored within a declaration context, and one can query
|
|
|
1201 information about the context in which each declaration lives. One can
|
|
|
1202 retrieve the ``DeclContext`` that contains a particular ``Decl`` using
|
|
|
1203 ``Decl::getDeclContext``. However, see the section
|
|
|
1204 :ref:`LexicalAndSemanticContexts` for more information about how to interpret
|
|
|
1205 this context information.
|
|
|
1206
|
|
|
1207 .. _Redeclarations:
|
|
|
1208
|
|
|
1209 Redeclarations and Overloads
|
|
|
1210 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
1211
|
|
|
1212 Within a translation unit, it is common for an entity to be declared several
|
|
|
1213 times. For example, we might declare a function "``f``" and then later
|
|
|
1214 re-declare it as part of an inlined definition:
|
|
|
1215
|
|
|
1216 .. code-block:: c++
|
|
|
1217
|
|
|
1218 void f(int x, int y, int z = 1);
|
|
|
1219
|
|
|
1220 inline void f(int x, int y, int z) { /* ... */ }
|
|
|
1221
|
|
|
1222 The representation of "``f``" differs in the source-centric and
|
|
|
1223 semantics-centric views of a declaration context. In the source-centric view,
|
|
|
1224 all redeclarations will be present, in the order they occurred in the source
|
|
|
1225 code, making this view suitable for clients that wish to see the structure of
|
|
|
1226 the source code. In the semantics-centric view, only the most recent "``f``"
|
|
|
1227 will be found by the lookup, since it effectively replaces the first
|
|
|
1228 declaration of "``f``".
|
|
|
1229
|
|
|
1230 (Note that because ``f`` can be redeclared at block scope, or in a friend
|
|
|
1231 declaration, etc. it is possible that the declaration of ``f`` found by name
|
|
|
1232 lookup will not be the most recent one.)
|
|
|
1233
|
|
|
1234 In the semantics-centric view, overloading of functions is represented
|
|
|
1235 explicitly. For example, given two declarations of a function "``g``" that are
|
|
|
1236 overloaded, e.g.,
|
|
|
1237
|
|
|
1238 .. code-block:: c++
|
|
|
1239
|
|
|
1240 void g();
|
|
|
1241 void g(int);
|
|
|
1242
|
|
|
1243 the ``DeclContext::lookup`` operation will return a
|
|
|
1244 ``DeclContext::lookup_result`` that contains a range of iterators over
|
|
|
1245 declarations of "``g``". Clients that perform semantic analysis on a program
|
|
|
1246 that is not concerned with the actual source code will primarily use this
|
|
|
1247 semantics-centric view.
|
|
|
1248
|
|
|
1249 .. _LexicalAndSemanticContexts:
|
|
|
1250
|
|
|
1251 Lexical and Semantic Contexts
|
|
|
1252 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
1253
|
|
|
1254 Each declaration has two potentially different declaration contexts: a
|
|
|
1255 *lexical* context, which corresponds to the source-centric view of the
|
|
|
1256 declaration context, and a *semantic* context, which corresponds to the
|
|
|
1257 semantics-centric view. The lexical context is accessible via
|
|
|
1258 ``Decl::getLexicalDeclContext`` while the semantic context is accessible via
|
|
|
1259 ``Decl::getDeclContext``, both of which return ``DeclContext`` pointers. For
|
|
|
1260 most declarations, the two contexts are identical. For example:
|
|
|
1261
|
|
|
1262 .. code-block:: c++
|
|
|
1263
|
|
|
1264 class X {
|
|
|
1265 public:
|
|
|
1266 void f(int x);
|
|
|
1267 };
|
|
|
1268
|
|
|
1269 Here, the semantic and lexical contexts of ``X::f`` are the ``DeclContext``
|
|
|
1270 associated with the class ``X`` (itself stored as a ``RecordDecl`` AST node).
|
|
|
1271 However, we can now define ``X::f`` out-of-line:
|
|
|
1272
|
|
|
1273 .. code-block:: c++
|
|
|
1274
|
|
|
1275 void X::f(int x = 17) { /* ... */ }
|
|
|
1276
|
|
|
1277 This definition of "``f``" has different lexical and semantic contexts. The
|
|
|
1278 lexical context corresponds to the declaration context in which the actual
|
|
|
1279 declaration occurred in the source code, e.g., the translation unit containing
|
|
|
1280 ``X``. Thus, this declaration of ``X::f`` can be found by traversing the
|
|
|
1281 declarations provided by [``decls_begin()``, ``decls_end()``) in the
|
|
|
1282 translation unit.
|
|
|
1283
|
|
|
1284 The semantic context of ``X::f`` corresponds to the class ``X``, since this
|
|
|
1285 member function is (semantically) a member of ``X``. Lookup of the name ``f``
|
|
|
1286 into the ``DeclContext`` associated with ``X`` will then return the definition
|
|
|
1287 of ``X::f`` (including information about the default argument).
|
|
|
1288
|
|
|
1289 Transparent Declaration Contexts
|
|
|
1290 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
1291
|
|
|
1292 In C and C++, there are several contexts in which names that are logically
|
|
|
1293 declared inside another declaration will actually "leak" out into the enclosing
|
|
|
1294 scope from the perspective of name lookup. The most obvious instance of this
|
|
|
1295 behavior is in enumeration types, e.g.,
|
|
|
1296
|
|
|
1297 .. code-block:: c++
|
|
|
1298
|
|
|
1299 enum Color {
|
|
|
1300 Red,
|
|
|
1301 Green,
|
|
|
1302 Blue
|
|
|
1303 };
|
|
|
1304
|
|
|
1305 Here, ``Color`` is an enumeration, which is a declaration context that contains
|
|
|
1306 the enumerators ``Red``, ``Green``, and ``Blue``. Thus, traversing the list of
|
|
|
1307 declarations contained in the enumeration ``Color`` will yield ``Red``,
|
|
|
1308 ``Green``, and ``Blue``. However, outside of the scope of ``Color`` one can
|
|
|
1309 name the enumerator ``Red`` without qualifying the name, e.g.,
|
|
|
1310
|
|
|
1311 .. code-block:: c++
|
|
|
1312
|
|
|
1313 Color c = Red;
|
|
|
1314
|
|
|
1315 There are other entities in C++ that provide similar behavior. For example,
|
|
|
1316 linkage specifications that use curly braces:
|
|
|
1317
|
|
|
1318 .. code-block:: c++
|
|
|
1319
|
|
|
1320 extern "C" {
|
|
|
1321 void f(int);
|
|
|
1322 void g(int);
|
|
|
1323 }
|
|
|
1324 // f and g are visible here
|
|
|
1325
|
|
|
1326 For source-level accuracy, we treat the linkage specification and enumeration
|
|
|
1327 type as a declaration context in which its enclosed declarations ("``Red``",
|
|
|
1328 "``Green``", and "``Blue``"; "``f``" and "``g``") are declared. However, these
|
|
|
1329 declarations are visible outside of the scope of the declaration context.
|
|
|
1330
|
|
|
1331 These language features (and several others, described below) have roughly the
|
|
|
1332 same set of requirements: declarations are declared within a particular lexical
|
|
|
1333 context, but the declarations are also found via name lookup in scopes
|
|
|
1334 enclosing the declaration itself. This feature is implemented via
|
|
|
1335 *transparent* declaration contexts (see
|
|
|
1336 ``DeclContext::isTransparentContext()``), whose declarations are visible in the
|
|
|
1337 nearest enclosing non-transparent declaration context. This means that the
|
|
|
1338 lexical context of the declaration (e.g., an enumerator) will be the
|
|
|
1339 transparent ``DeclContext`` itself, as will the semantic context, but the
|
|
|
1340 declaration will be visible in every outer context up to and including the
|
|
|
1341 first non-transparent declaration context (since transparent declaration
|
|
|
1342 contexts can be nested).
|
|
|
1343
|
|
|
1344 The transparent ``DeclContext``\ s are:
|
|
|
1345
|
|
|
1346 * Enumerations (but not C++11 "scoped enumerations"):
|
|
|
1347
|
|
|
1348 .. code-block:: c++
|
|
|
1349
|
|
|
1350 enum Color {
|
|
|
1351 Red,
|
|
|
1352 Green,
|
|
|
1353 Blue
|
|
|
1354 };
|
|
|
1355 // Red, Green, and Blue are in scope
|
|
|
1356
|
|
|
1357 * C++ linkage specifications:
|
|
|
1358
|
|
|
1359 .. code-block:: c++
|
|
|
1360
|
|
|
1361 extern "C" {
|
|
|
1362 void f(int);
|
|
|
1363 void g(int);
|
|
|
1364 }
|
|
|
1365 // f and g are in scope
|
|
|
1366
|
|
|
1367 * Anonymous unions and structs:
|
|
|
1368
|
|
|
1369 .. code-block:: c++
|
|
|
1370
|
|
|
1371 struct LookupTable {
|
|
|
1372 bool IsVector;
|
|
|
1373 union {
|
|
|
1374 std::vector<Item> *Vector;
|
|
|
1375 std::set<Item> *Set;
|
|
|
1376 };
|
|
|
1377 };
|
|
|
1378
|
|
|
1379 LookupTable LT;
|
|
|
1380 LT.Vector = 0; // Okay: finds Vector inside the unnamed union
|
|
|
1381
|
|
|
1382 * C++11 inline namespaces:
|
|
|
1383
|
|
|
1384 .. code-block:: c++
|
|
|
1385
|
|
|
1386 namespace mylib {
|
|
|
1387 inline namespace debug {
|
|
|
1388 class X;
|
|
|
1389 }
|
|
|
1390 }
|
|
|
1391 mylib::X *xp; // okay: mylib::X refers to mylib::debug::X
|
|
|
1392
|
|
|
1393 .. _MultiDeclContext:
|
|
|
1394
|
|
|
1395 Multiply-Defined Declaration Contexts
|
|
|
1396 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
1397
|
|
|
1398 C++ namespaces have the interesting property that
|
|
|
1399 the namespace can be defined multiple times, and the declarations provided by
|
|
|
1400 each namespace definition are effectively merged (from the semantic point of
|
|
|
1401 view). For example, the following two code snippets are semantically
|
|
|
1402 indistinguishable:
|
|
|
1403
|
|
|
1404 .. code-block:: c++
|
|
|
1405
|
|
|
1406 // Snippet #1:
|
|
|
1407 namespace N {
|
|
|
1408 void f();
|
|
|
1409 }
|
|
|
1410 namespace N {
|
|
|
1411 void f(int);
|
|
|
1412 }
|
|
|
1413
|
|
|
1414 // Snippet #2:
|
|
|
1415 namespace N {
|
|
|
1416 void f();
|
|
|
1417 void f(int);
|
|
|
1418 }
|
|
|
1419
|
|
|
1420 In Clang's representation, the source-centric view of declaration contexts will
|
|
|
1421 actually have two separate ``NamespaceDecl`` nodes in Snippet #1, each of which
|
|
|
1422 is a declaration context that contains a single declaration of "``f``".
|
|
|
1423 However, the semantics-centric view provided by name lookup into the namespace
|
|
|
1424 ``N`` for "``f``" will return a ``DeclContext::lookup_result`` that contains a
|
|
|
1425 range of iterators over declarations of "``f``".
|
|
|
1426
|
|
|
1427 ``DeclContext`` manages multiply-defined declaration contexts internally. The
|
|
|
1428 function ``DeclContext::getPrimaryContext`` retrieves the "primary" context for
|
|
|
1429 a given ``DeclContext`` instance, which is the ``DeclContext`` responsible for
|
|
|
1430 maintaining the lookup table used for the semantics-centric view. Given a
|
|
|
1431 DeclContext, one can obtain the set of declaration contexts that are
|
|
|
1432 semantically connected to this declaration context, in source order, including
|
|
|
1433 this context (which will be the only result, for non-namespace contexts) via
|
|
|
1434 ``DeclContext::collectAllContexts``. Note that these functions are used
|
|
|
1435 internally within the lookup and insertion methods of the ``DeclContext``, so
|
|
|
1436 the vast majority of clients can ignore them.
|
|
|
1437
|
|
|
1438 Because the same entity can be defined multiple times in different modules,
|
|
|
1439 it is also possible for there to be multiple definitions of (for instance)
|
|
|
1440 a ``CXXRecordDecl``, all of which describe a definition of the same class.
|
|
|
1441 In such a case, only one of those "definitions" is considered by Clang to be
|
|
|
1442 the definiition of the class, and the others are treated as non-defining
|
|
|
1443 declarations that happen to also contain member declarations. Corresponding
|
|
|
1444 members in each definition of such multiply-defined classes are identified
|
|
|
1445 either by redeclaration chains (if the members are ``Redeclarable``)
|
|
|
1446 or by simply a pointer to the canonical declaration (if the declarations
|
|
|
1447 are not ``Redeclarable`` -- in that case, a ``Mergeable`` base class is used
|
|
|
1448 instead).
|
|
|
1449
|
|
|
1450 The ASTImporter
|
|
|
1451 ---------------
|
|
|
1452
|
|
|
1453 The ``ASTImporter`` class imports nodes of an ``ASTContext`` into another
|
|
|
1454 ``ASTContext``. Please refer to the document :doc:`ASTImporter: Merging Clang
|
|
|
1455 ASTs <LibASTImporter>` for an introduction. And please read through the
|
|
|
1456 high-level `description of the import algorithm
|
|
|
1457 <LibASTImporter.html#algorithm-of-the-import>`_, this is essential for
|
|
|
1458 understanding further implementation details of the importer.
|
|
|
1459
|
|
|
1460 .. _templated:
|
|
|
1461
|
|
|
1462 Abstract Syntax Graph
|
|
|
1463 ^^^^^^^^^^^^^^^^^^^^^
|
|
|
1464
|
|
|
1465 Despite the name, the Clang AST is not a tree. It is a directed graph with
|
|
|
1466 cycles. One example of a cycle is the connection between a
|
|
|
1467 ``ClassTemplateDecl`` and its "templated" ``CXXRecordDecl``. The *templated*
|
|
|
1468 ``CXXRecordDecl`` represents all the fields and methods inside the class
|
|
|
1469 template, while the ``ClassTemplateDecl`` holds the information which is
|
|
|
1470 related to being a template, i.e. template arguments, etc. We can get the
|
|
|
1471 *templated* class (the ``CXXRecordDecl``) of a ``ClassTemplateDecl`` with
|
|
|
1472 ``ClassTemplateDecl::getTemplatedDecl()``. And we can get back a pointer of the
|
|
|
1473 "described" class template from the *templated* class:
|
|
|
1474 ``CXXRecordDecl::getDescribedTemplate()``. So, this is a cycle between two
|
|
|
1475 nodes: between the *templated* and the *described* node. There may be various
|
|
|
1476 other kinds of cycles in the AST especially in case of declarations.
|
|
|
1477
|
|
|
1478 .. _structural-eq:
|
|
|
1479
|
|
|
1480 Structural Equivalency
|
|
|
1481 ^^^^^^^^^^^^^^^^^^^^^^
|
|
|
1482
|
|
|
1483 Importing one AST node copies that node into the destination ``ASTContext``. To
|
|
|
1484 copy one node means that we create a new node in the "to" context then we set
|
|
|
1485 its properties to be equal to the properties of the source node. Before the
|
|
|
1486 copy, we make sure that the source node is not *structurally equivalent* to any
|
|
|
1487 existing node in the destination context. If it happens to be equivalent then
|
|
|
1488 we skip the copy.
|
|
|
1489
|
|
|
1490 The informal definition of structural equivalency is the following:
|
|
|
1491 Two nodes are **structurally equivalent** if they are
|
|
|
1492
|
|
|
1493 - builtin types and refer to the same type, e.g. ``int`` and ``int`` are
|
|
|
1494 structurally equivalent,
|
|
|
1495 - function types and all their parameters have structurally equivalent types,
|
|
|
1496 - record types and all their fields in order of their definition have the same
|
|
|
1497 identifier names and structurally equivalent types,
|
|
|
1498 - variable or function declarations and they have the same identifier name and
|
|
|
1499 their types are structurally equivalent.
|
|
|
1500
|
|
|
1501 In C, two types are structurally equivalent if they are *compatible types*. For
|
|
|
1502 a formal definition of *compatible types*, please refer to 6.2.7/1 in the C11
|
|
|
1503 standard. However, there is no definition for *compatible types* in the C++
|
|
|
1504 standard. Still, we extend the definition of structural equivalency to
|
|
|
1505 templates and their instantiations similarly: besides checking the previously
|
|
|
1506 mentioned properties, we have to check for equivalent template
|
|
|
1507 parameters/arguments, etc.
|
|
|
1508
|
|
|
1509 The structural equivalent check can be and is used independently from the
|
|
|
1510 ASTImporter, e.g. the ``clang::Sema`` class uses it also.
|
|
|
1511
|
|
|
1512 The equivalence of nodes may depend on the equivalency of other pairs of nodes.
|
|
|
1513 Thus, the check is implemented as a parallel graph traversal. We traverse
|
|
|
1514 through the nodes of both graphs at the same time. The actual implementation is
|
|
|
1515 similar to breadth-first-search. Let's say we start the traverse with the <A,B>
|
|
|
1516 pair of nodes. Whenever the traversal reaches a pair <X,Y> then the following
|
|
|
1517 statements are true:
|
|
|
1518
|
|
|
1519 - A and X are nodes from the same ASTContext.
|
|
|
1520 - B and Y are nodes from the same ASTContext.
|
|
|
1521 - A and B may or may not be from the same ASTContext.
|
|
|
1522 - if A == X and B == Y (pointer equivalency) then (there is a cycle during the
|
|
|
1523 traverse)
|
|
|
1524
|
|
|
1525 - A and B are structurally equivalent if and only if
|
|
|
1526
|
|
|
1527 - All dependent nodes on the path from <A,B> to <X,Y> are structurally
|
|
|
1528 equivalent.
|
|
|
1529
|
|
|
1530 When we compare two classes or enums and one of them is incomplete or has
|
|
|
1531 unloaded external lexical declarations then we cannot descend to compare their
|
|
|
1532 contained declarations. So in these cases they are considered equal if they
|
|
|
1533 have the same names. This is the way how we compare forward declarations with
|
|
|
1534 definitions.
|
|
|
1535
|
|
|
1536 .. TODO Should we elaborate the actual implementation of the graph traversal,
|
|
|
1537 .. which is a very weird BFS traversal?
|
|
|
1538
|
|
|
1539 Redeclaration Chains
|
|
|
1540 ^^^^^^^^^^^^^^^^^^^^
|
|
|
1541
|
|
|
1542 The early version of the ``ASTImporter``'s merge mechanism squashed the
|
|
|
1543 declarations, i.e. it aimed to have only one declaration instead of maintaining
|
|
|
1544 a whole redeclaration chain. This early approach simply skipped importing a
|
|
|
1545 function prototype, but it imported a definition. To demonstrate the problem
|
|
|
1546 with this approach let's consider an empty "to" context and the following
|
|
|
1547 ``virtual`` function declarations of ``f`` in the "from" context:
|
|
|
1548
|
|
|
1549 .. code-block:: c++
|
|
|
1550
|
|
|
1551 struct B { virtual void f(); };
|
|
|
1552 void B::f() {} // <-- let's import this definition
|
|
|
1553
|
|
|
1554 If we imported the definition with the "squashing" approach then we would
|
|
|
1555 end-up having one declaration which is indeed a definition, but ``isVirtual()``
|
|
|
1556 returns ``false`` for it. The reason is that the definition is indeed not
|
|
|
1557 virtual, it is the property of the prototype!
|
|
|
1558
|
|
|
1559 Consequently, we must either set the virtual flag for the definition (but then
|
|
|
1560 we create a malformed AST which the parser would never create), or we import
|
|
|
1561 the whole redeclaration chain of the function. The most recent version of the
|
|
|
1562 ``ASTImporter`` uses the latter mechanism. We do import all function
|
|
|
1563 declarations - regardless if they are definitions or prototypes - in the order
|
|
|
1564 as they appear in the "from" context.
|
|
|
1565
|
|
|
1566 .. One definition
|
|
|
1567
|
|
|
1568 If we have an existing definition in the "to" context, then we cannot import
|
|
|
1569 another definition, we will use the existing definition. However, we can import
|
|
|
1570 prototype(s): we chain the newly imported prototype(s) to the existing
|
|
|
1571 definition. Whenever we import a new prototype from a third context, that will
|
|
|
1572 be added to the end of the redeclaration chain. This may result in long
|
|
|
1573 redeclaration chains in certain cases, e.g. if we import from several
|
|
|
1574 translation units which include the same header with the prototype.
|
|
|
1575
|
|
|
1576 .. Squashing prototypes
|
|
|
1577
|
|
|
1578 To mitigate the problem of long redeclaration chains of free functions, we
|
|
|
1579 could compare prototypes to see if they have the same properties and if yes
|
|
|
1580 then we could merge these prototypes. The implementation of squashing of
|
|
|
1581 prototypes for free functions is future work.
|
|
|
1582
|
|
|
1583 .. Exception: Cannot have more than 1 prototype in-class
|
|
|
1584
|
|
|
1585 Chaining functions this way ensures that we do copy all information from the
|
|
|
1586 source AST. Nonetheless, there is a problem with member functions: While we can
|
|
|
1587 have many prototypes for free functions, we must have only one prototype for a
|
|
|
1588 member function.
|
|
|
1589
|
|
|
1590 .. code-block:: c++
|
|
|
1591
|
|
|
1592 void f(); // OK
|
|
|
1593 void f(); // OK
|
|
|
1594
|
|
|
1595 struct X {
|
|
|
1596 void f(); // OK
|
|
|
1597 void f(); // ERROR
|
|
|
1598 };
|
|
|
1599 void X::f() {} // OK
|
|
|
1600
|
|
|
1601 Thus, prototypes of member functions must be squashed, we cannot just simply
|
|
|
1602 attach a new prototype to the existing in-class prototype. Consider the
|
|
|
1603 following contexts:
|
|
|
1604
|
|
|
1605 .. code-block:: c++
|
|
|
1606
|
|
|
1607 // "to" context
|
|
|
1608 struct X {
|
|
|
1609 void f(); // D0
|
|
|
1610 };
|
|
|
1611
|
|
|
1612 .. code-block:: c++
|
|
|
1613
|
|
|
1614 // "from" context
|
|
|
1615 struct X {
|
|
|
1616 void f(); // D1
|
|
|
1617 };
|
|
|
1618 void X::f() {} // D2
|
|
|
1619
|
|
|
1620 When we import the prototype and the definition of ``f`` from the "from"
|
|
|
1621 context, then the resulting redecl chain will look like this ``D0 -> D2'``,
|
|
|
1622 where ``D2'`` is the copy of ``D2`` in the "to" context.
|
|
|
1623
|
|
|
1624 .. Redecl chains of other declarations
|
|
|
1625
|
|
|
1626 Generally speaking, when we import declarations (like enums and classes) we do
|
|
|
1627 attach the newly imported declaration to the existing redeclaration chain (if
|
|
|
1628 there is structural equivalency). We do not import, however, the whole
|
|
|
1629 redeclaration chain as we do in case of functions. Up till now, we haven't
|
|
|
1630 found any essential property of forward declarations which is similar to the
|
|
|
1631 case of the virtual flag in a member function prototype. In the future, this
|
|
|
1632 may change, though.
|
|
|
1633
|
|
|
1634 Traversal during the Import
|
|
|
1635 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
1636
|
|
|
1637 The node specific import mechanisms are implemented in
|
|
|
1638 ``ASTNodeImporter::VisitNode()`` functions, e.g. ``VisitFunctionDecl()``.
|
|
|
1639 When we import a declaration then first we import everything which is needed to
|
|
|
1640 call the constructor of that declaration node. Everything which can be set
|
|
|
1641 later is set after the node is created. For example, in case of a
|
|
|
1642 ``FunctionDecl`` we first import the declaration context in which the function
|
|
|
1643 is declared, then we create the ``FunctionDecl`` and only then we import the
|
|
|
1644 body of the function. This means there are implicit dependencies between AST
|
|
|
1645 nodes. These dependencies determine the order in which we visit nodes in the
|
|
|
1646 "from" context. As with the regular graph traversal algorithms like DFS, we
|
|
|
1647 keep track which nodes we have already visited in
|
|
|
1648 ``ASTImporter::ImportedDecls``. Whenever we create a node then we immediately
|
|
|
1649 add that to the ``ImportedDecls``. We must not start the import of any other
|
|
|
1650 declarations before we keep track of the newly created one. This is essential,
|
|
|
1651 otherwise, we would not be able to handle circular dependencies. To enforce
|
|
|
1652 this, we wrap all constructor calls of all AST nodes in
|
|
|
1653 ``GetImportedOrCreateDecl()``. This wrapper ensures that all newly created
|
|
|
1654 declarations are immediately marked as imported; also, if a declaration is
|
|
|
1655 already marked as imported then we just return its counterpart in the "to"
|
|
|
1656 context. Consequently, calling a declaration's ``::Create()`` function directly
|
|
|
1657 would lead to errors, please don't do that!
|
|
|
1658
|
|
|
1659 Even with the use of ``GetImportedOrCreateDecl()`` there is still a
|
|
|
1660 probability of having an infinite import recursion if things are imported from
|
|
|
1661 each other in wrong way. Imagine that during the import of ``A``, the import of
|
|
|
1662 ``B`` is requested before we could create the node for ``A`` (the constructor
|
|
|
1663 needs a reference to ``B``). And the same could be true for the import of ``B``
|
|
|
1664 (``A`` is requested to be imported before we could create the node for ``B``).
|
|
|
1665 In case of the :ref:`templated-described swing <templated>` we take
|
|
|
1666 extra attention to break the cyclical dependency: we import and set the
|
|
|
1667 described template only after the ``CXXRecordDecl`` is created. As a best
|
|
|
1668 practice, before creating the node in the "to" context, avoid importing of
|
|
|
1669 other nodes which are not needed for the constructor of node ``A``.
|
|
|
1670
|
|
|
1671 Error Handling
|
|
|
1672 ^^^^^^^^^^^^^^
|
|
|
1673
|
|
|
1674 Every import function returns with either an ``llvm::Error`` or an
|
|
|
1675 ``llvm::Expected<T>`` object. This enforces to check the return value of the
|
|
|
1676 import functions. If there was an error during one import then we return with
|
|
|
1677 that error. (Exception: when we import the members of a class, we collect the
|
|
|
1678 individual errors with each member and we concatenate them in one Error
|
|
|
1679 object.) We cache these errors in cases of declarations. During the next import
|
|
|
1680 call if there is an existing error we just return with that. So, clients of the
|
|
|
1681 library receive an Error object, which they must check.
|
|
|
1682
|
|
|
1683 During import of a specific declaration, it may happen that some AST nodes had
|
|
|
1684 already been created before we recognize an error. In this case, we signal back
|
|
|
1685 the error to the caller, but the "to" context remains polluted with those nodes
|
|
|
1686 which had been created. Ideally, those nodes should not had been created, but
|
|
|
1687 that time we did not know about the error, the error happened later. Since the
|
|
|
1688 AST is immutable (most of the cases we can't remove existing nodes) we choose
|
|
|
1689 to mark these nodes as erroneous.
|
|
|
1690
|
|
|
1691 We cache the errors associated with declarations in the "from" context in
|
|
|
1692 ``ASTImporter::ImportDeclErrors`` and the ones which are associated with the
|
|
|
1693 "to" context in ``ASTImporterSharedState::ImportErrors``. Note that, there may
|
|
|
1694 be several ASTImporter objects which import into the same "to" context but from
|
|
|
1695 different "from" contexts; in this case, they have to share the associated
|
|
|
1696 errors of the "to" context.
|
|
|
1697
|
|
|
1698 When an error happens, that propagates through the call stack, through all the
|
|
|
1699 dependant nodes. However, in case of dependency cycles, this is not enough,
|
|
|
1700 because we strive to mark the erroneous nodes so clients can act upon. In those
|
|
|
1701 cases, we have to keep track of the errors for those nodes which are
|
|
|
1702 intermediate nodes of a cycle.
|
|
|
1703
|
|
|
1704 An **import path** is the list of the AST nodes which we visit during an Import
|
|
|
1705 call. If node ``A`` depends on node ``B`` then the path contains an ``A->B``
|
|
|
1706 edge. From the call stack of the import functions, we can read the very same
|
|
|
1707 path.
|
|
|
1708
|
|
|
1709 Now imagine the following AST, where the ``->`` represents dependency in terms
|
|
|
1710 of the import (all nodes are declarations).
|
|
|
1711
|
|
|
1712 .. code-block:: text
|
|
|
1713
|
|
|
1714 A->B->C->D
|
|
|
1715 `->E
|
|
|
1716
|
|
|
1717 We would like to import A.
|
|
|
1718 The import behaves like a DFS, so we will visit the nodes in this order: ABCDE.
|
|
|
1719 During the visitation we will have the following import paths:
|
|
|
1720
|
|
|
1721 .. code-block:: text
|
|
|
1722
|
|
|
1723 A
|
|
|
1724 AB
|
|
|
1725 ABC
|
|
|
1726 ABCD
|
|
|
1727 ABC
|
|
|
1728 AB
|
|
|
1729 ABE
|
|
|
1730 AB
|
|
|
1731 A
|
|
|
1732
|
|
|
1733 If during the visit of E there is an error then we set an error for E, then as
|
|
|
1734 the call stack shrinks for B, then for A:
|
|
|
1735
|
|
|
1736 .. code-block:: text
|
|
|
1737
|
|
|
1738 A
|
|
|
1739 AB
|
|
|
1740 ABC
|
|
|
1741 ABCD
|
|
|
1742 ABC
|
|
|
1743 AB
|
|
|
1744 ABE // Error! Set an error to E
|
|
|
1745 AB // Set an error to B
|
|
|
1746 A // Set an error to A
|
|
|
1747
|
|
|
1748 However, during the import we could import C and D without any error and they
|
|
|
1749 are independent of A,B and E. We must not set up an error for C and D. So, at
|
|
|
1750 the end of the import we have an entry in ``ImportDeclErrors`` for A,B,E but
|
|
|
1751 not for C,D.
|
|
|
1752
|
|
|
1753 Now, what happens if there is a cycle in the import path? Let's consider this
|
|
|
1754 AST:
|
|
|
1755
|
|
|
1756 .. code-block:: text
|
|
|
1757
|
|
|
1758 A->B->C->A
|
|
|
1759 `->E
|
|
|
1760
|
|
|
1761 During the visitation, we will have the below import paths and if during the
|
|
|
1762 visit of E there is an error then we will set up an error for E,B,A. But what's
|
|
|
1763 up with C?
|
|
|
1764
|
|
|
1765 .. code-block:: text
|
|
|
1766
|
|
|
1767 A
|
|
|
1768 AB
|
|
|
1769 ABC
|
|
|
1770 ABCA
|
|
|
1771 ABC
|
|
|
1772 AB
|
|
|
1773 ABE // Error! Set an error to E
|
|
|
1774 AB // Set an error to B
|
|
|
1775 A // Set an error to A
|
|
|
1776
|
|
|
1777 This time we know that both B and C are dependent on A. This means we must set
|
|
|
1778 up an error for C too. As the call stack reverses back we get to A and we must
|
|
|
1779 set up an error to all nodes which depend on A (this includes C). But C is no
|
|
|
1780 longer on the import path, it just had been previously. Such a situation can
|
|
|
1781 happen only if during the visitation we had a cycle. If we didn't have any
|
|
|
1782 cycle, then the normal way of passing an Error object through the call stack
|
|
|
1783 could handle the situation. This is why we must track cycles during the import
|
|
|
1784 process for each visited declaration.
|
|
|
1785
|
|
|
1786 Lookup Problems
|
|
|
1787 ^^^^^^^^^^^^^^^
|
|
|
1788
|
|
|
1789 When we import a declaration from the source context then we check whether we
|
|
|
1790 already have a structurally equivalent node with the same name in the "to"
|
|
|
1791 context. If the "from" node is a definition and the found one is also a
|
|
|
1792 definition, then we do not create a new node, instead, we mark the found node
|
|
|
1793 as the imported node. If the found definition and the one we want to import
|
|
|
1794 have the same name but they are structurally in-equivalent, then we have an ODR
|
|
|
1795 violation in case of C++. If the "from" node is not a definition then we add
|
|
|
1796 that to the redeclaration chain of the found node. This behaviour is essential
|
|
|
1797 when we merge ASTs from different translation units which include the same
|
|
|
1798 header file(s). For example, we want to have only one definition for the class
|
|
|
1799 template ``std::vector``, even if we included ``<vector>`` in several
|
|
|
1800 translation units.
|
|
|
1801
|
|
|
1802 To find a structurally equivalent node we can use the regular C/C++ lookup
|
|
|
1803 functions: ``DeclContext::noload_lookup()`` and
|
|
|
1804 ``DeclContext::localUncachedLookup()``. These functions do respect the C/C++
|
|
|
1805 name hiding rules, thus you cannot find certain declarations in a given
|
|
|
1806 declaration context. For instance, unnamed declarations (anonymous structs),
|
|
|
1807 non-first ``friend`` declarations and template specializations are hidden. This
|
|
|
1808 is a problem, because if we use the regular C/C++ lookup then we create
|
|
|
1809 redundant AST nodes during the merge! Also, having two instances of the same
|
|
|
1810 node could result in false :ref:`structural in-equivalencies <structural-eq>`
|
|
|
1811 of other nodes which depend on the duplicated node. Because of these reasons,
|
|
|
1812 we created a lookup class which has the sole purpose to register all
|
|
|
1813 declarations, so later they can be looked up by subsequent import requests.
|
|
|
1814 This is the ``ASTImporterLookupTable`` class. This lookup table should be
|
|
|
1815 shared amongst the different ``ASTImporter`` instances if they happen to import
|
|
|
1816 to the very same "to" context. This is why we can use the importer specific
|
|
|
1817 lookup only via the ``ASTImporterSharedState`` class.
|
|
|
1818
|
|
|
1819 ExternalASTSource
|
|
|
1820 ~~~~~~~~~~~~~~~~~
|
|
|
1821
|
|
|
1822 The ``ExternalASTSource`` is an abstract interface associated with the
|
|
|
1823 ``ASTContext`` class. It provides the ability to read the declarations stored
|
|
|
1824 within a declaration context either for iteration or for name lookup. A
|
|
|
1825 declaration context with an external AST source may load its declarations
|
|
|
1826 on-demand. This means that the list of declarations (represented as a linked
|
|
|
1827 list, the head is ``DeclContext::FirstDecl``) could be empty. However, member
|
|
|
1828 functions like ``DeclContext::lookup()`` may initiate a load.
|
|
|
1829
|
|
|
1830 Usually, external sources are associated with precompiled headers. For example,
|
|
|
1831 when we load a class from a PCH then the members are loaded only if we do want
|
|
|
1832 to look up something in the class' context.
|
|
|
1833
|
|
|
1834 In case of LLDB, an implementation of the ``ExternalASTSource`` interface is
|
|
|
1835 attached to the AST context which is related to the parsed expression. This
|
|
|
1836 implementation of the ``ExternalASTSource`` interface is realized with the help
|
|
|
1837 of the ``ASTImporter`` class. This way, LLDB can reuse Clang's parsing
|
|
|
1838 machinery while synthesizing the underlying AST from the debug data (e.g. from
|
|
|
1839 DWARF). From the view of the ``ASTImporter`` this means both the "to" and the
|
|
|
1840 "from" context may have declaration contexts with external lexical storage. If
|
|
|
1841 a ``DeclContext`` in the "to" AST context has external lexical storage then we
|
|
|
1842 must take extra attention to work only with the already loaded declarations!
|
|
|
1843 Otherwise, we would end up with an uncontrolled import process. For instance,
|
|
|
1844 if we used the regular ``DeclContext::lookup()`` to find the existing
|
|
|
1845 declarations in the "to" context then the ``lookup()`` call itself would
|
|
|
1846 initiate a new import while we are in the middle of importing a declaration!
|
|
|
1847 (By the time we initiate the lookup we haven't registered yet that we already
|
|
|
1848 started to import the node of the "from" context.) This is why we use
|
|
|
1849 ``DeclContext::noload_lookup()`` instead.
|
|
|
1850
|
|
|
1851 Class Template Instantiations
|
|
|
1852 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
1853
|
|
|
1854 Different translation units may have class template instantiations with the
|
|
|
1855 same template arguments, but with a different set of instantiated
|
|
|
1856 ``MethodDecls`` and ``FieldDecls``. Consider the following files:
|
|
|
1857
|
|
|
1858 .. code-block:: c++
|
|
|
1859
|
|
|
1860 // x.h
|
|
|
1861 template <typename T>
|
|
|
1862 struct X {
|
|
|
1863 int a{0}; // FieldDecl with InitListExpr
|
|
|
1864 X(char) : a(3) {} // (1)
|
|
|
1865 X(int) {} // (2)
|
|
|
1866 };
|
|
|
1867
|
|
|
1868 // foo.cpp
|
|
|
1869 void foo() {
|
|
|
1870 // ClassTemplateSpec with ctor (1): FieldDecl without InitlistExpr
|
|
|
1871 X<char> xc('c');
|
|
|
1872 }
|
|
|
1873
|
|
|
1874 // bar.cpp
|
|
|
1875 void bar() {
|
|
|
1876 // ClassTemplateSpec with ctor (2): FieldDecl WITH InitlistExpr
|
|
|
1877 X<char> xc(1);
|
|
|
1878 }
|
|
|
1879
|
|
|
1880 In ``foo.cpp`` we use the constructor with number ``(1)``, which explicitly
|
|
|
1881 initializes the member ``a`` to ``3``, thus the ``InitListExpr`` ``{0}`` is not
|
|
|
1882 used here and the AST node is not instantiated. However, in the case of
|
|
|
1883 ``bar.cpp`` we use the constructor with number ``(2)``, which does not
|
|
|
1884 explicitly initialize the ``a`` member, so the default ``InitListExpr`` is
|
|
|
1885 needed and thus instantiated. When we merge the AST of ``foo.cpp`` and
|
|
|
1886 ``bar.cpp`` we must create an AST node for the class template instantiation of
|
|
|
1887 ``X<char>`` which has all the required nodes. Therefore, when we find an
|
|
|
1888 existing ``ClassTemplateSpecializationDecl`` then we merge the fields of the
|
|
|
1889 ``ClassTemplateSpecializationDecl`` in the "from" context in a way that the
|
|
|
1890 ``InitListExpr`` is copied if not existent yet. The same merge mechanism should
|
|
|
1891 be done in the cases of instantiated default arguments and exception
|
|
|
1892 specifications of functions.
|
|
|
1893
|
|
|
1894 .. _visibility:
|
|
|
1895
|
|
|
1896 Visibility of Declarations
|
|
|
1897 ^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
1898
|
|
|
1899 During import of a global variable with external visibility, the lookup will
|
|
|
1900 find variables (with the same name) but with static visibility (linkage).
|
|
|
1901 Clearly, we cannot put them into the same redeclaration chain. The same is true
|
|
|
1902 the in case of functions. Also, we have to take care of other kinds of
|
|
|
1903 declarations like enums, classes, etc. if they are in anonymous namespaces.
|
|
|
1904 Therefore, we filter the lookup results and consider only those which have the
|
|
|
1905 same visibility as the declaration we currently import.
|
|
|
1906
|
|
|
1907 We consider two declarations in two anonymous namespaces to have the same
|
|
|
1908 visibility only if they are imported from the same AST context.
|
|
|
1909
|
|
|
1910 Strategies to Handle Conflicting Names
|
|
|
1911 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
1912
|
|
|
1913 During the import we lookup existing declarations with the same name. We filter
|
|
|
1914 the lookup results based on their :ref:`visibility <visibility>`. If any of the
|
|
|
1915 found declarations are not structurally equivalent then we bumped to a name
|
|
|
1916 conflict error (ODR violation in C++). In this case, we return with an
|
|
|
1917 ``Error`` and we set up the ``Error`` object for the declaration. However, some
|
|
|
1918 clients of the ``ASTImporter`` may require a different, perhaps less
|
|
|
1919 conservative and more liberal error handling strategy.
|
|
|
1920
|
|
|
1921 E.g. static analysis clients may benefit if the node is created even if there
|
|
|
1922 is a name conflict. During the CTU analysis of certain projects, we recognized
|
|
|
1923 that there are global declarations which collide with declarations from other
|
|
|
1924 translation units, but they are not referenced outside from their translation
|
|
|
1925 unit. These declarations should be in an unnamed namespace ideally. If we treat
|
|
|
1926 these collisions liberally then CTU analysis can find more results. Note, the
|
|
|
1927 feature be able to choose between name conflict handling strategies is still an
|
|
|
1928 ongoing work.
|
|
|
1929
|
|
|
1930 .. _CFG:
|
|
|
1931
|
|
|
1932 The ``CFG`` class
|
|
|
1933 -----------------
|
|
|
1934
|
|
|
1935 The ``CFG`` class is designed to represent a source-level control-flow graph
|
|
|
1936 for a single statement (``Stmt*``). Typically instances of ``CFG`` are
|
|
|
1937 constructed for function bodies (usually an instance of ``CompoundStmt``), but
|
|
|
1938 can also be instantiated to represent the control-flow of any class that
|
|
|
1939 subclasses ``Stmt``, which includes simple expressions. Control-flow graphs
|
|
|
1940 are especially useful for performing `flow- or path-sensitive
|
|
|
1941 <https://en.wikipedia.org/wiki/Data_flow_analysis#Sensitivities>`_ program
|
|
|
1942 analyses on a given function.
|
|
|
1943
|
|
|
1944 Basic Blocks
|
|
|
1945 ^^^^^^^^^^^^
|
|
|
1946
|
|
|
1947 Concretely, an instance of ``CFG`` is a collection of basic blocks. Each basic
|
|
|
1948 block is an instance of ``CFGBlock``, which simply contains an ordered sequence
|
|
|
1949 of ``Stmt*`` (each referring to statements in the AST). The ordering of
|
|
|
1950 statements within a block indicates unconditional flow of control from one
|
|
|
1951 statement to the next. :ref:`Conditional control-flow
|
|
|
1952 <ConditionalControlFlow>` is represented using edges between basic blocks. The
|
|
|
1953 statements within a given ``CFGBlock`` can be traversed using the
|
|
|
1954 ``CFGBlock::*iterator`` interface.
|
|
|
1955
|
|
|
1956 A ``CFG`` object owns the instances of ``CFGBlock`` within the control-flow
|
|
|
1957 graph it represents. Each ``CFGBlock`` within a CFG is also uniquely numbered
|
|
|
1958 (accessible via ``CFGBlock::getBlockID()``). Currently the number is based on
|
|
|
1959 the ordering the blocks were created, but no assumptions should be made on how
|
|
|
1960 ``CFGBlocks`` are numbered other than their numbers are unique and that they
|
|
|
1961 are numbered from 0..N-1 (where N is the number of basic blocks in the CFG).
|
|
|
1962
|
|
|
1963 Entry and Exit Blocks
|
|
|
1964 ^^^^^^^^^^^^^^^^^^^^^
|
|
|
1965
|
|
|
1966 Each instance of ``CFG`` contains two special blocks: an *entry* block
|
|
|
1967 (accessible via ``CFG::getEntry()``), which has no incoming edges, and an
|
|
|
1968 *exit* block (accessible via ``CFG::getExit()``), which has no outgoing edges.
|
|
|
1969 Neither block contains any statements, and they serve the role of providing a
|
|
|
1970 clear entrance and exit for a body of code such as a function body. The
|
|
|
1971 presence of these empty blocks greatly simplifies the implementation of many
|
|
|
1972 analyses built on top of CFGs.
|
|
|
1973
|
|
|
1974 .. _ConditionalControlFlow:
|
|
|
1975
|
|
|
1976 Conditional Control-Flow
|
|
|
1977 ^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
1978
|
|
|
1979 Conditional control-flow (such as those induced by if-statements and loops) is
|
|
|
1980 represented as edges between ``CFGBlocks``. Because different C language
|
|
|
1981 constructs can induce control-flow, each ``CFGBlock`` also records an extra
|
|
|
1982 ``Stmt*`` that represents the *terminator* of the block. A terminator is
|
|
|
1983 simply the statement that caused the control-flow, and is used to identify the
|
|
|
1984 nature of the conditional control-flow between blocks. For example, in the
|
|
|
1985 case of an if-statement, the terminator refers to the ``IfStmt`` object in the
|
|
|
1986 AST that represented the given branch.
|
|
|
1987
|
|
|
1988 To illustrate, consider the following code example:
|
|
|
1989
|
|
|
1990 .. code-block:: c++
|
|
|
1991
|
|
|
1992 int foo(int x) {
|
|
|
1993 x = x + 1;
|
|
|
1994 if (x > 2)
|
|
|
1995 x++;
|
|
|
1996 else {
|
|
|
1997 x += 2;
|
|
|
1998 x *= 2;
|
|
|
1999 }
|
|
|
2000
|
|
|
2001 return x;
|
|
|
2002 }
|
|
|
2003
|
|
|
2004 After invoking the parser+semantic analyzer on this code fragment, the AST of
|
|
|
2005 the body of ``foo`` is referenced by a single ``Stmt*``. We can then construct
|
|
|
2006 an instance of ``CFG`` representing the control-flow graph of this function
|
|
|
2007 body by single call to a static class method:
|
|
|
2008
|
|
|
2009 .. code-block:: c++
|
|
|
2010
|
|
|
2011 Stmt *FooBody = ...
|
|
|
2012 std::unique_ptr<CFG> FooCFG = CFG::buildCFG(FooBody);
|
|
|
2013
|
|
|
2014 Along with providing an interface to iterate over its ``CFGBlocks``, the
|
|
|
2015 ``CFG`` class also provides methods that are useful for debugging and
|
|
|
2016 visualizing CFGs. For example, the method ``CFG::dump()`` dumps a
|
|
|
2017 pretty-printed version of the CFG to standard error. This is especially useful
|
|
|
2018 when one is using a debugger such as gdb. For example, here is the output of
|
|
|
2019 ``FooCFG->dump()``:
|
|
|
2020
|
|
|
2021 .. code-block:: text
|
|
|
2022
|
|
|
2023 [ B5 (ENTRY) ]
|
|
|
2024 Predecessors (0):
|
|
|
2025 Successors (1): B4
|
|
|
2026
|
|
|
2027 [ B4 ]
|
|
|
2028 1: x = x + 1
|
|
|
2029 2: (x > 2)
|
|
|
2030 T: if [B4.2]
|
|
|
2031 Predecessors (1): B5
|
|
|
2032 Successors (2): B3 B2
|
|
|
2033
|
|
|
2034 [ B3 ]
|
|
|
2035 1: x++
|
|
|
2036 Predecessors (1): B4
|
|
|
2037 Successors (1): B1
|
|
|
2038
|
|
|
2039 [ B2 ]
|
|
|
2040 1: x += 2
|
|
|
2041 2: x *= 2
|
|
|
2042 Predecessors (1): B4
|
|
|
2043 Successors (1): B1
|
|
|
2044
|
|
|
2045 [ B1 ]
|
|
|
2046 1: return x;
|
|
|
2047 Predecessors (2): B2 B3
|
|
|
2048 Successors (1): B0
|
|
|
2049
|
|
|
2050 [ B0 (EXIT) ]
|
|
|
2051 Predecessors (1): B1
|
|
|
2052 Successors (0):
|
|
|
2053
|
|
|
2054 For each block, the pretty-printed output displays for each block the number of
|
|
|
2055 *predecessor* blocks (blocks that have outgoing control-flow to the given
|
|
|
2056 block) and *successor* blocks (blocks that have control-flow that have incoming
|
|
|
2057 control-flow from the given block). We can also clearly see the special entry
|
|
|
2058 and exit blocks at the beginning and end of the pretty-printed output. For the
|
|
|
2059 entry block (block B5), the number of predecessor blocks is 0, while for the
|
|
|
2060 exit block (block B0) the number of successor blocks is 0.
|
|
|
2061
|
|
|
2062 The most interesting block here is B4, whose outgoing control-flow represents
|
|
|
2063 the branching caused by the sole if-statement in ``foo``. Of particular
|
|
|
2064 interest is the second statement in the block, ``(x > 2)``, and the terminator,
|
|
|
2065 printed as ``if [B4.2]``. The second statement represents the evaluation of
|
|
|
2066 the condition of the if-statement, which occurs before the actual branching of
|
|
|
2067 control-flow. Within the ``CFGBlock`` for B4, the ``Stmt*`` for the second
|
|
|
2068 statement refers to the actual expression in the AST for ``(x > 2)``. Thus
|
|
|
2069 pointers to subclasses of ``Expr`` can appear in the list of statements in a
|
|
|
2070 block, and not just subclasses of ``Stmt`` that refer to proper C statements.
|
|
|
2071
|
|
|
2072 The terminator of block B4 is a pointer to the ``IfStmt`` object in the AST.
|
|
|
2073 The pretty-printer outputs ``if [B4.2]`` because the condition expression of
|
|
|
2074 the if-statement has an actual place in the basic block, and thus the
|
|
|
2075 terminator is essentially *referring* to the expression that is the second
|
|
|
2076 statement of block B4 (i.e., B4.2). In this manner, conditions for
|
|
|
2077 control-flow (which also includes conditions for loops and switch statements)
|
|
|
2078 are hoisted into the actual basic block.
|
|
|
2079
|
|
|
2080 .. Implicit Control-Flow
|
|
|
2081 .. ^^^^^^^^^^^^^^^^^^^^^
|
|
|
2082
|
|
|
2083 .. A key design principle of the ``CFG`` class was to not require any
|
|
|
2084 .. transformations to the AST in order to represent control-flow. Thus the
|
|
|
2085 .. ``CFG`` does not perform any "lowering" of the statements in an AST: loops
|
|
|
2086 .. are not transformed into guarded gotos, short-circuit operations are not
|
|
|
2087 .. converted to a set of if-statements, and so on.
|
|
|
2088
|
|
|
2089 Constant Folding in the Clang AST
|
|
|
2090 ---------------------------------
|
|
|
2091
|
|
|
2092 There are several places where constants and constant folding matter a lot to
|
|
|
2093 the Clang front-end. First, in general, we prefer the AST to retain the source
|
|
|
2094 code as close to how the user wrote it as possible. This means that if they
|
|
|
2095 wrote "``5+4``", we want to keep the addition and two constants in the AST, we
|
|
|
2096 don't want to fold to "``9``". This means that constant folding in various
|
|
|
2097 ways turns into a tree walk that needs to handle the various cases.
|
|
|
2098
|
|
|
2099 However, there are places in both C and C++ that require constants to be
|
|
|
2100 folded. For example, the C standard defines what an "integer constant
|
|
|
2101 expression" (i-c-e) is with very precise and specific requirements. The
|
|
|
2102 language then requires i-c-e's in a lot of places (for example, the size of a
|
|
|
2103 bitfield, the value for a case statement, etc). For these, we have to be able
|
|
|
2104 to constant fold the constants, to do semantic checks (e.g., verify bitfield
|
|
|
2105 size is non-negative and that case statements aren't duplicated). We aim for
|
|
|
2106 Clang to be very pedantic about this, diagnosing cases when the code does not
|
|
|
2107 use an i-c-e where one is required, but accepting the code unless running with
|
|
|
2108 ``-pedantic-errors``.
|
|
|
2109
|
|
|
2110 Things get a little bit more tricky when it comes to compatibility with
|
|
|
2111 real-world source code. Specifically, GCC has historically accepted a huge
|
|
|
2112 superset of expressions as i-c-e's, and a lot of real world code depends on
|
|
|
2113 this unfortunate accident of history (including, e.g., the glibc system
|
|
|
2114 headers). GCC accepts anything its "fold" optimizer is capable of reducing to
|
|
|
2115 an integer constant, which means that the definition of what it accepts changes
|
|
|
2116 as its optimizer does. One example is that GCC accepts things like "``case
|
|
|
2117 X-X:``" even when ``X`` is a variable, because it can fold this to 0.
|
|
|
2118
|
|
|
2119 Another issue are how constants interact with the extensions we support, such
|
|
|
2120 as ``__builtin_constant_p``, ``__builtin_inf``, ``__extension__`` and many
|
|
|
2121 others. C99 obviously does not specify the semantics of any of these
|
|
|
2122 extensions, and the definition of i-c-e does not include them. However, these
|
|
|
2123 extensions are often used in real code, and we have to have a way to reason
|
|
|
2124 about them.
|
|
|
2125
|
|
|
2126 Finally, this is not just a problem for semantic analysis. The code generator
|
|
|
2127 and other clients have to be able to fold constants (e.g., to initialize global
|
|
|
2128 variables) and have to handle a superset of what C99 allows. Further, these
|
|
|
2129 clients can benefit from extended information. For example, we know that
|
|
|
2130 "``foo() || 1``" always evaluates to ``true``, but we can't replace the
|
|
|
2131 expression with ``true`` because it has side effects.
|
|
|
2132
|
|
|
2133 Implementation Approach
|
|
|
2134 ^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
2135
|
|
|
2136 After trying several different approaches, we've finally converged on a design
|
|
|
2137 (Note, at the time of this writing, not all of this has been implemented,
|
|
|
2138 consider this a design goal!). Our basic approach is to define a single
|
|
|
2139 recursive evaluation method (``Expr::Evaluate``), which is implemented
|
|
|
2140 in ``AST/ExprConstant.cpp``. Given an expression with "scalar" type (integer,
|
|
|
2141 fp, complex, or pointer) this method returns the following information:
|
|
|
2142
|
|
|
2143 * Whether the expression is an integer constant expression, a general constant
|
|
|
2144 that was folded but has no side effects, a general constant that was folded
|
|
|
2145 but that does have side effects, or an uncomputable/unfoldable value.
|
|
|
2146 * If the expression was computable in any way, this method returns the
|
|
|
2147 ``APValue`` for the result of the expression.
|
|
|
2148 * If the expression is not evaluatable at all, this method returns information
|
|
|
2149 on one of the problems with the expression. This includes a
|
|
|
2150 ``SourceLocation`` for where the problem is, and a diagnostic ID that explains
|
|
|
2151 the problem. The diagnostic should have ``ERROR`` type.
|
|
|
2152 * If the expression is not an integer constant expression, this method returns
|
|
|
2153 information on one of the problems with the expression. This includes a
|
|
|
2154 ``SourceLocation`` for where the problem is, and a diagnostic ID that
|
|
|
2155 explains the problem. The diagnostic should have ``EXTENSION`` type.
|
|
|
2156
|
|
|
2157 This information gives various clients the flexibility that they want, and we
|
|
|
2158 will eventually have some helper methods for various extensions. For example,
|
|
|
2159 ``Sema`` should have a ``Sema::VerifyIntegerConstantExpression`` method, which
|
|
|
2160 calls ``Evaluate`` on the expression. If the expression is not foldable, the
|
|
|
2161 error is emitted, and it would return ``true``. If the expression is not an
|
|
|
2162 i-c-e, the ``EXTENSION`` diagnostic is emitted. Finally it would return
|
|
|
2163 ``false`` to indicate that the AST is OK.
|
|
|
2164
|
|
|
2165 Other clients can use the information in other ways, for example, codegen can
|
|
|
2166 just use expressions that are foldable in any way.
|
|
|
2167
|
|
|
2168 Extensions
|
|
|
2169 ^^^^^^^^^^
|
|
|
2170
|
|
|
2171 This section describes how some of the various extensions Clang supports
|
|
|
2172 interacts with constant evaluation:
|
|
|
2173
|
|
|
2174 * ``__extension__``: The expression form of this extension causes any
|
|
|
2175 evaluatable subexpression to be accepted as an integer constant expression.
|
|
|
2176 * ``__builtin_constant_p``: This returns true (as an integer constant
|
|
|
2177 expression) if the operand evaluates to either a numeric value (that is, not
|
|
|
2178 a pointer cast to integral type) of integral, enumeration, floating or
|
|
|
2179 complex type, or if it evaluates to the address of the first character of a
|
|
|
2180 string literal (possibly cast to some other type). As a special case, if
|
|
|
2181 ``__builtin_constant_p`` is the (potentially parenthesized) condition of a
|
|
|
2182 conditional operator expression ("``?:``"), only the true side of the
|
|
|
2183 conditional operator is considered, and it is evaluated with full constant
|
|
|
2184 folding.
|
|
|
2185 * ``__builtin_choose_expr``: The condition is required to be an integer
|
|
|
2186 constant expression, but we accept any constant as an "extension of an
|
|
|
2187 extension". This only evaluates one operand depending on which way the
|
|
|
2188 condition evaluates.
|
|
|
2189 * ``__builtin_classify_type``: This always returns an integer constant
|
|
|
2190 expression.
|
|
|
2191 * ``__builtin_inf, nan, ...``: These are treated just like a floating-point
|
|
|
2192 literal.
|
|
|
2193 * ``__builtin_abs, copysign, ...``: These are constant folded as general
|
|
|
2194 constant expressions.
|
|
|
2195 * ``__builtin_strlen`` and ``strlen``: These are constant folded as integer
|
|
|
2196 constant expressions if the argument is a string literal.
|
|
|
2197
|
|
|
2198 .. _Sema:
|
|
|
2199
|
|
|
2200 The Sema Library
|
|
|
2201 ================
|
|
|
2202
|
|
|
2203 This library is called by the :ref:`Parser library <Parser>` during parsing to
|
|
|
2204 do semantic analysis of the input. For valid programs, Sema builds an AST for
|
|
|
2205 parsed constructs.
|
|
|
2206
|
|
|
2207 .. _CodeGen:
|
|
|
2208
|
|
|
2209 The CodeGen Library
|
|
|
2210 ===================
|
|
|
2211
|
|
|
2212 CodeGen takes an :ref:`AST <AST>` as input and produces `LLVM IR code
|
|
|
2213 <//llvm.org/docs/LangRef.html>`_ from it.
|
|
|
2214
|
|
|
2215 How to change Clang
|
|
|
2216 ===================
|
|
|
2217
|
|
|
2218 How to add an attribute
|
|
|
2219 -----------------------
|
|
|
2220 Attributes are a form of metadata that can be attached to a program construct,
|
|
|
2221 allowing the programmer to pass semantic information along to the compiler for
|
|
|
2222 various uses. For example, attributes may be used to alter the code generation
|
|
|
2223 for a program construct, or to provide extra semantic information for static
|
|
|
2224 analysis. This document explains how to add a custom attribute to Clang.
|
|
|
2225 Documentation on existing attributes can be found `here
|
|
|
2226 <//clang.llvm.org/docs/AttributeReference.html>`_.
|
|
|
2227
|
|
|
2228 Attribute Basics
|
|
|
2229 ^^^^^^^^^^^^^^^^
|
|
|
2230 Attributes in Clang are handled in three stages: parsing into a parsed attribute
|
|
|
2231 representation, conversion from a parsed attribute into a semantic attribute,
|
|
|
2232 and then the semantic handling of the attribute.
|
|
|
2233
|
|
|
2234 Parsing of the attribute is determined by the various syntactic forms attributes
|
|
|
2235 can take, such as GNU, C++11, and Microsoft style attributes, as well as other
|
|
|
2236 information provided by the table definition of the attribute. Ultimately, the
|
|
|
2237 parsed representation of an attribute object is an ``ParsedAttr`` object.
|
|
|
2238 These parsed attributes chain together as a list of parsed attributes attached
|
|
|
2239 to a declarator or declaration specifier. The parsing of attributes is handled
|
|
|
2240 automatically by Clang, except for attributes spelled as keywords. When
|
|
|
2241 implementing a keyword attribute, the parsing of the keyword and creation of the
|
|
|
2242 ``ParsedAttr`` object must be done manually.
|
|
|
2243
|
|
|
2244 Eventually, ``Sema::ProcessDeclAttributeList()`` is called with a ``Decl`` and
|
|
|
2245 an ``ParsedAttr``, at which point the parsed attribute can be transformed
|
|
|
2246 into a semantic attribute. The process by which a parsed attribute is converted
|
|
|
2247 into a semantic attribute depends on the attribute definition and semantic
|
|
|
2248 requirements of the attribute. The end result, however, is that the semantic
|
|
|
2249 attribute object is attached to the ``Decl`` object, and can be obtained by a
|
|
|
2250 call to ``Decl::getAttr<T>()``.
|
|
|
2251
|
|
|
2252 The structure of the semantic attribute is also governed by the attribute
|
|
|
2253 definition given in Attr.td. This definition is used to automatically generate
|
|
|
2254 functionality used for the implementation of the attribute, such as a class
|
|
|
2255 derived from ``clang::Attr``, information for the parser to use, automated
|
|
|
2256 semantic checking for some attributes, etc.
|
|
|
2257
|
|
|
2258
|
|
|
2259 ``include/clang/Basic/Attr.td``
|
|
|
2260 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
2261 The first step to adding a new attribute to Clang is to add its definition to
|
|
|
2262 `include/clang/Basic/Attr.td
|
|
|
2263 <https://github.com/llvm/llvm-project/blob/master/clang/include/clang/Basic/Attr.td>`_.
|
|
|
2264 This tablegen definition must derive from the ``Attr`` (tablegen, not
|
|
|
2265 semantic) type, or one of its derivatives. Most attributes will derive from the
|
|
|
2266 ``InheritableAttr`` type, which specifies that the attribute can be inherited by
|
|
|
2267 later redeclarations of the ``Decl`` it is associated with.
|
|
|
2268 ``InheritableParamAttr`` is similar to ``InheritableAttr``, except that the
|
|
|
2269 attribute is written on a parameter instead of a declaration. If the attribute
|
|
|
2270 is intended to apply to a type instead of a declaration, such an attribute
|
|
|
2271 should derive from ``TypeAttr``, and will generally not be given an AST
|
|
|
2272 representation. (Note that this document does not cover the creation of type
|
|
|
2273 attributes.) An attribute that inherits from ``IgnoredAttr`` is parsed, but will
|
|
|
2274 generate an ignored attribute diagnostic when used, which may be useful when an
|
|
|
2275 attribute is supported by another vendor but not supported by clang.
|
|
|
2276
|
|
|
2277 The definition will specify several key pieces of information, such as the
|
|
|
2278 semantic name of the attribute, the spellings the attribute supports, the
|
|
|
2279 arguments the attribute expects, and more. Most members of the ``Attr`` tablegen
|
|
|
2280 type do not require definitions in the derived definition as the default
|
|
|
2281 suffice. However, every attribute must specify at least a spelling list, a
|
|
|
2282 subject list, and a documentation list.
|
|
|
2283
|
|
|
2284 Spellings
|
|
|
2285 ~~~~~~~~~
|
|
|
2286 All attributes are required to specify a spelling list that denotes the ways in
|
|
|
2287 which the attribute can be spelled. For instance, a single semantic attribute
|
|
|
2288 may have a keyword spelling, as well as a C++11 spelling and a GNU spelling. An
|
|
|
2289 empty spelling list is also permissible and may be useful for attributes which
|
|
|
2290 are created implicitly. The following spellings are accepted:
|
|
|
2291
|
|
|
2292 ============ ================================================================
|
|
|
2293 Spelling Description
|
|
|
2294 ============ ================================================================
|
|
|
2295 ``GNU`` Spelled with a GNU-style ``__attribute__((attr))`` syntax and
|
|
|
2296 placement.
|
|
|
2297 ``CXX11`` Spelled with a C++-style ``[[attr]]`` syntax with an optional
|
|
|
2298 vendor-specific namespace.
|
|
|
2299 ``C2x`` Spelled with a C-style ``[[attr]]` syntax with an optional
|
|
|
2300 vendor-specific namespace.
|
|
|
2301 ``Declspec`` Spelled with a Microsoft-style ``__declspec(attr)`` syntax.
|
|
|
2302 ``Keyword`` The attribute is spelled as a keyword, and required custom
|
|
|
2303 parsing.
|
|
|
2304 ``GCC`` Specifies two spellings: the first is a GNU-style spelling, and
|
|
|
2305 the second is a C++-style spelling with the ``gnu`` namespace.
|
|
|
2306 Attributes should only specify this spelling for attributes
|
|
|
2307 supported by GCC.
|
|
|
2308 ``Clang`` Specifies two or three spellings: the first is a GNU-style
|
|
|
2309 spelling, the second is a C++-style spelling with the ``clang``
|
|
|
2310 namespace, and the third is an optional C-style spelling with
|
|
|
2311 the ``clang`` namespace. By default, a C-style spelling is
|
|
|
2312 provided.
|
|
|
2313 ``Pragma`` The attribute is spelled as a ``#pragma``, and requires custom
|
|
|
2314 processing within the preprocessor. If the attribute is meant to
|
|
|
2315 be used by Clang, it should set the namespace to ``"clang"``.
|
|
|
2316 Note that this spelling is not used for declaration attributes.
|
|
|
2317 ============ ================================================================
|
|
|
2318
|
|
|
2319 Subjects
|
|
|
2320 ~~~~~~~~
|
|
|
2321 Attributes appertain to one or more ``Decl`` subjects. If the attribute attempts
|
|
|
2322 to attach to a subject that is not in the subject list, a diagnostic is issued
|
|
|
2323 automatically. Whether the diagnostic is a warning or an error depends on how
|
|
|
2324 the attribute's ``SubjectList`` is defined, but the default behavior is to warn.
|
|
|
2325 The diagnostics displayed to the user are automatically determined based on the
|
|
|
2326 subjects in the list, but a custom diagnostic parameter can also be specified in
|
|
|
2327 the ``SubjectList``. The diagnostics generated for subject list violations are
|
|
|
2328 either ``diag::warn_attribute_wrong_decl_type`` or
|
|
|
2329 ``diag::err_attribute_wrong_decl_type``, and the parameter enumeration is found
|
|
|
2330 in `include/clang/Sema/ParsedAttr.h
|
|
|
2331 <https://github.com/llvm/llvm-project/blob/master/clang/include/clang/Sema/ParsedAttr.h>`_
|
|
|
2332 If a previously unused Decl node is added to the ``SubjectList``, the logic used
|
|
|
2333 to automatically determine the diagnostic parameter in `utils/TableGen/ClangAttrEmitter.cpp
|
|
|
2334 <https://github.com/llvm/llvm-project/blob/master/clang/utils/TableGen/ClangAttrEmitter.cpp>`_
|
|
|
2335 may need to be updated.
|
|
|
2336
|
|
|
2337 By default, all subjects in the SubjectList must either be a Decl node defined
|
|
|
2338 in ``DeclNodes.td``, or a statement node defined in ``StmtNodes.td``. However,
|
|
|
2339 more complex subjects can be created by creating a ``SubsetSubject`` object.
|
|
|
2340 Each such object has a base subject which it appertains to (which must be a
|
|
|
2341 Decl or Stmt node, and not a SubsetSubject node), and some custom code which is
|
|
|
2342 called when determining whether an attribute appertains to the subject. For
|
|
|
2343 instance, a ``NonBitField`` SubsetSubject appertains to a ``FieldDecl``, and
|
|
|
2344 tests whether the given FieldDecl is a bit field. When a SubsetSubject is
|
|
|
2345 specified in a SubjectList, a custom diagnostic parameter must also be provided.
|
|
|
2346
|
|
|
2347 Diagnostic checking for attribute subject lists is automated except when
|
|
|
2348 ``HasCustomParsing`` is set to ``1``.
|
|
|
2349
|
|
|
2350 Documentation
|
|
|
2351 ~~~~~~~~~~~~~
|
|
|
2352 All attributes must have some form of documentation associated with them.
|
|
|
2353 Documentation is table generated on the public web server by a server-side
|
|
|
2354 process that runs daily. Generally, the documentation for an attribute is a
|
|
|
2355 stand-alone definition in `include/clang/Basic/AttrDocs.td
|
|
|
2356 <https://github.com/llvm/llvm-project/blob/master/clang/include/clang/Basic/AttrDocs.td>`_
|
|
|
2357 that is named after the attribute being documented.
|
|
|
2358
|
|
|
2359 If the attribute is not for public consumption, or is an implicitly-created
|
|
|
2360 attribute that has no visible spelling, the documentation list can specify the
|
|
|
2361 ``Undocumented`` object. Otherwise, the attribute should have its documentation
|
|
|
2362 added to AttrDocs.td.
|
|
|
2363
|
|
|
2364 Documentation derives from the ``Documentation`` tablegen type. All derived
|
|
|
2365 types must specify a documentation category and the actual documentation itself.
|
|
|
2366 Additionally, it can specify a custom heading for the attribute, though a
|
|
|
2367 default heading will be chosen when possible.
|
|
|
2368
|
|
|
2369 There are four predefined documentation categories: ``DocCatFunction`` for
|
|
|
2370 attributes that appertain to function-like subjects, ``DocCatVariable`` for
|
|
|
2371 attributes that appertain to variable-like subjects, ``DocCatType`` for type
|
|
|
2372 attributes, and ``DocCatStmt`` for statement attributes. A custom documentation
|
|
|
2373 category should be used for groups of attributes with similar functionality.
|
|
|
2374 Custom categories are good for providing overview information for the attributes
|
|
|
2375 grouped under it. For instance, the consumed annotation attributes define a
|
|
|
2376 custom category, ``DocCatConsumed``, that explains what consumed annotations are
|
|
|
2377 at a high level.
|
|
|
2378
|
|
|
2379 Documentation content (whether it is for an attribute or a category) is written
|
|
|
2380 using reStructuredText (RST) syntax.
|
|
|
2381
|
|
|
2382 After writing the documentation for the attribute, it should be locally tested
|
|
|
2383 to ensure that there are no issues generating the documentation on the server.
|
|
|
2384 Local testing requires a fresh build of clang-tblgen. To generate the attribute
|
|
|
2385 documentation, execute the following command::
|
|
|
2386
|
|
|
2387 clang-tblgen -gen-attr-docs -I /path/to/clang/include /path/to/clang/include/clang/Basic/Attr.td -o /path/to/clang/docs/AttributeReference.rst
|
|
|
2388
|
|
|
2389 When testing locally, *do not* commit changes to ``AttributeReference.rst``.
|
|
|
2390 This file is generated by the server automatically, and any changes made to this
|
|
|
2391 file will be overwritten.
|
|
|
2392
|
|
|
2393 Arguments
|
|
|
2394 ~~~~~~~~~
|
|
|
2395 Attributes may optionally specify a list of arguments that can be passed to the
|
|
|
2396 attribute. Attribute arguments specify both the parsed form and the semantic
|
|
|
2397 form of the attribute. For example, if ``Args`` is
|
|
|
2398 ``[StringArgument<"Arg1">, IntArgument<"Arg2">]`` then
|
|
|
2399 ``__attribute__((myattribute("Hello", 3)))`` will be a valid use; it requires
|
|
|
2400 two arguments while parsing, and the Attr subclass' constructor for the
|
|
|
2401 semantic attribute will require a string and integer argument.
|
|
|
2402
|
|
|
2403 All arguments have a name and a flag that specifies whether the argument is
|
|
|
2404 optional. The associated C++ type of the argument is determined by the argument
|
|
|
2405 definition type. If the existing argument types are insufficient, new types can
|
|
|
2406 be created, but it requires modifying `utils/TableGen/ClangAttrEmitter.cpp
|
|
|
2407 <https://github.com/llvm/llvm-project/blob/master/clang/utils/TableGen/ClangAttrEmitter.cpp>`_
|
|
|
2408 to properly support the type.
|
|
|
2409
|
|
|
2410 Other Properties
|
|
|
2411 ~~~~~~~~~~~~~~~~
|
|
|
2412 The ``Attr`` definition has other members which control the behavior of the
|
|
|
2413 attribute. Many of them are special-purpose and beyond the scope of this
|
|
|
2414 document, however a few deserve mention.
|
|
|
2415
|
|
|
2416 If the parsed form of the attribute is more complex, or differs from the
|
|
|
2417 semantic form, the ``HasCustomParsing`` bit can be set to ``1`` for the class,
|
|
|
2418 and the parsing code in `Parser::ParseGNUAttributeArgs()
|
|
|
2419 <https://github.com/llvm/llvm-project/blob/master/clang/lib/Parse/ParseDecl.cpp>`_
|
|
|
2420 can be updated for the special case. Note that this only applies to arguments
|
|
|
2421 with a GNU spelling -- attributes with a __declspec spelling currently ignore
|
|
|
2422 this flag and are handled by ``Parser::ParseMicrosoftDeclSpec``.
|
|
|
2423
|
|
|
2424 Note that setting this member to 1 will opt out of common attribute semantic
|
|
|
2425 handling, requiring extra implementation efforts to ensure the attribute
|
|
|
2426 appertains to the appropriate subject, etc.
|
|
|
2427
|
|
|
2428 If the attribute should not be propagated from a template declaration to an
|
|
|
2429 instantiation of the template, set the ``Clone`` member to 0. By default, all
|
|
|
2430 attributes will be cloned to template instantiations.
|
|
|
2431
|
|
|
2432 Attributes that do not require an AST node should set the ``ASTNode`` field to
|
|
|
2433 ``0`` to avoid polluting the AST. Note that anything inheriting from
|
|
|
2434 ``TypeAttr`` or ``IgnoredAttr`` automatically do not generate an AST node. All
|
|
|
2435 other attributes generate an AST node by default. The AST node is the semantic
|
|
|
2436 representation of the attribute.
|
|
|
2437
|
|
|
2438 The ``LangOpts`` field specifies a list of language options required by the
|
|
|
2439 attribute. For instance, all of the CUDA-specific attributes specify ``[CUDA]``
|
|
|
2440 for the ``LangOpts`` field, and when the CUDA language option is not enabled, an
|
|
|
2441 "attribute ignored" warning diagnostic is emitted. Since language options are
|
|
|
2442 not table generated nodes, new language options must be created manually and
|
|
|
2443 should specify the spelling used by ``LangOptions`` class.
|
|
|
2444
|
|
|
2445 Custom accessors can be generated for an attribute based on the spelling list
|
|
|
2446 for that attribute. For instance, if an attribute has two different spellings:
|
|
|
2447 'Foo' and 'Bar', accessors can be created:
|
|
|
2448 ``[Accessor<"isFoo", [GNU<"Foo">]>, Accessor<"isBar", [GNU<"Bar">]>]``
|
|
|
2449 These accessors will be generated on the semantic form of the attribute,
|
|
|
2450 accepting no arguments and returning a ``bool``.
|
|
|
2451
|
|
|
2452 Attributes that do not require custom semantic handling should set the
|
|
|
2453 ``SemaHandler`` field to ``0``. Note that anything inheriting from
|
|
|
2454 ``IgnoredAttr`` automatically do not get a semantic handler. All other
|
|
|
2455 attributes are assumed to use a semantic handler by default. Attributes
|
|
|
2456 without a semantic handler are not given a parsed attribute ``Kind`` enumerator.
|
|
|
2457
|
|
|
2458 Target-specific attributes may share a spelling with other attributes in
|
|
|
2459 different targets. For instance, the ARM and MSP430 targets both have an
|
|
|
2460 attribute spelled ``GNU<"interrupt">``, but with different parsing and semantic
|
|
|
2461 requirements. To support this feature, an attribute inheriting from
|
|
|
2462 ``TargetSpecificAttribute`` may specify a ``ParseKind`` field. This field
|
|
|
2463 should be the same value between all arguments sharing a spelling, and
|
|
|
2464 corresponds to the parsed attribute's ``Kind`` enumerator. This allows
|
|
|
2465 attributes to share a parsed attribute kind, but have distinct semantic
|
|
|
2466 attribute classes. For instance, ``ParsedAttr`` is the shared
|
|
|
2467 parsed attribute kind, but ARMInterruptAttr and MSP430InterruptAttr are the
|
|
|
2468 semantic attributes generated.
|
|
|
2469
|
|
|
2470 By default, attribute arguments are parsed in an evaluated context. If the
|
|
|
2471 arguments for an attribute should be parsed in an unevaluated context (akin to
|
|
|
2472 the way the argument to a ``sizeof`` expression is parsed), set
|
|
|
2473 ``ParseArgumentsAsUnevaluated`` to ``1``.
|
|
|
2474
|
|
|
2475 If additional functionality is desired for the semantic form of the attribute,
|
|
|
2476 the ``AdditionalMembers`` field specifies code to be copied verbatim into the
|
|
|
2477 semantic attribute class object, with ``public`` access.
|
|
|
2478
|
|
|
2479 Boilerplate
|
|
|
2480 ^^^^^^^^^^^
|
|
|
2481 All semantic processing of declaration attributes happens in `lib/Sema/SemaDeclAttr.cpp
|
|
|
2482 <https://github.com/llvm/llvm-project/blob/master/clang/lib/Sema/SemaDeclAttr.cpp>`_,
|
|
|
2483 and generally starts in the ``ProcessDeclAttribute()`` function. If the
|
|
|
2484 attribute is a "simple" attribute -- meaning that it requires no custom semantic
|
|
|
2485 processing aside from what is automatically provided, add a call to
|
|
|
2486 ``handleSimpleAttribute<YourAttr>(S, D, Attr);`` to the switch statement.
|
|
|
2487 Otherwise, write a new ``handleYourAttr()`` function, and add that to the switch
|
|
|
2488 statement. Please do not implement handling logic directly in the ``case`` for
|
|
|
2489 the attribute.
|
|
|
2490
|
|
|
2491 Unless otherwise specified by the attribute definition, common semantic checking
|
|
|
2492 of the parsed attribute is handled automatically. This includes diagnosing
|
|
|
2493 parsed attributes that do not appertain to the given ``Decl``, ensuring the
|
|
|
2494 correct minimum number of arguments are passed, etc.
|
|
|
2495
|
|
|
2496 If the attribute adds additional warnings, define a ``DiagGroup`` in
|
|
|
2497 `include/clang/Basic/DiagnosticGroups.td
|
|
|
2498 <https://github.com/llvm/llvm-project/blob/master/clang/include/clang/Basic/DiagnosticGroups.td>`_
|
|
|
2499 named after the attribute's ``Spelling`` with "_"s replaced by "-"s. If there
|
|
|
2500 is only a single diagnostic, it is permissible to use ``InGroup<DiagGroup<"your-attribute">>``
|
|
|
2501 directly in `DiagnosticSemaKinds.td
|
|
|
2502 <https://github.com/llvm/llvm-project/blob/master/clang/include/clang/Basic/DiagnosticSemaKinds.td>`_
|
|
|
2503
|
|
|
2504 All semantic diagnostics generated for your attribute, including automatically-
|
|
|
2505 generated ones (such as subjects and argument counts), should have a
|
|
|
2506 corresponding test case.
|
|
|
2507
|
|
|
2508 Semantic handling
|
|
|
2509 ^^^^^^^^^^^^^^^^^
|
|
|
2510 Most attributes are implemented to have some effect on the compiler. For
|
|
|
2511 instance, to modify the way code is generated, or to add extra semantic checks
|
|
|
2512 for an analysis pass, etc. Having added the attribute definition and conversion
|
|
|
2513 to the semantic representation for the attribute, what remains is to implement
|
|
|
2514 the custom logic requiring use of the attribute.
|
|
|
2515
|
|
|
2516 The ``clang::Decl`` object can be queried for the presence or absence of an
|
|
|
2517 attribute using ``hasAttr<T>()``. To obtain a pointer to the semantic
|
|
|
2518 representation of the attribute, ``getAttr<T>`` may be used.
|
|
|
2519
|
|
|
2520 How to add an expression or statement
|
|
|
2521 -------------------------------------
|
|
|
2522
|
|
|
2523 Expressions and statements are one of the most fundamental constructs within a
|
|
|
2524 compiler, because they interact with many different parts of the AST, semantic
|
|
|
2525 analysis, and IR generation. Therefore, adding a new expression or statement
|
|
|
2526 kind into Clang requires some care. The following list details the various
|
|
|
2527 places in Clang where an expression or statement needs to be introduced, along
|
|
|
2528 with patterns to follow to ensure that the new expression or statement works
|
|
|
2529 well across all of the C languages. We focus on expressions, but statements
|
|
|
2530 are similar.
|
|
|
2531
|
|
|
2532 #. Introduce parsing actions into the parser. Recursive-descent parsing is
|
|
|
2533 mostly self-explanatory, but there are a few things that are worth keeping
|
|
|
2534 in mind:
|
|
|
2535
|
|
|
2536 * Keep as much source location information as possible! You'll want it later
|
|
|
2537 to produce great diagnostics and support Clang's various features that map
|
|
|
2538 between source code and the AST.
|
|
|
2539 * Write tests for all of the "bad" parsing cases, to make sure your recovery
|
|
|
2540 is good. If you have matched delimiters (e.g., parentheses, square
|
|
|
2541 brackets, etc.), use ``Parser::BalancedDelimiterTracker`` to give nice
|
|
|
2542 diagnostics when things go wrong.
|
|
|
2543
|
|
|
2544 #. Introduce semantic analysis actions into ``Sema``. Semantic analysis should
|
|
|
2545 always involve two functions: an ``ActOnXXX`` function that will be called
|
|
|
2546 directly from the parser, and a ``BuildXXX`` function that performs the
|
|
|
2547 actual semantic analysis and will (eventually!) build the AST node. It's
|
|
|
2548 fairly common for the ``ActOnCXX`` function to do very little (often just
|
|
|
2549 some minor translation from the parser's representation to ``Sema``'s
|
|
|
2550 representation of the same thing), but the separation is still important:
|
|
|
2551 C++ template instantiation, for example, should always call the ``BuildXXX``
|
|
|
2552 variant. Several notes on semantic analysis before we get into construction
|
|
|
2553 of the AST:
|
|
|
2554
|
|
|
2555 * Your expression probably involves some types and some subexpressions.
|
|
|
2556 Make sure to fully check that those types, and the types of those
|
|
|
2557 subexpressions, meet your expectations. Add implicit conversions where
|
|
|
2558 necessary to make sure that all of the types line up exactly the way you
|
|
|
2559 want them. Write extensive tests to check that you're getting good
|
|
|
2560 diagnostics for mistakes and that you can use various forms of
|
|
|
2561 subexpressions with your expression.
|
|
|
2562 * When type-checking a type or subexpression, make sure to first check
|
|
|
2563 whether the type is "dependent" (``Type::isDependentType()``) or whether a
|
|
|
2564 subexpression is type-dependent (``Expr::isTypeDependent()``). If any of
|
|
|
2565 these return ``true``, then you're inside a template and you can't do much
|
|
|
2566 type-checking now. That's normal, and your AST node (when you get there)
|
|
|
2567 will have to deal with this case. At this point, you can write tests that
|
|
|
2568 use your expression within templates, but don't try to instantiate the
|
|
|
2569 templates.
|
|
|
2570 * For each subexpression, be sure to call ``Sema::CheckPlaceholderExpr()``
|
|
|
2571 to deal with "weird" expressions that don't behave well as subexpressions.
|
|
|
2572 Then, determine whether you need to perform lvalue-to-rvalue conversions
|
|
|
2573 (``Sema::DefaultLvalueConversions``) or the usual unary conversions
|
|
|
2574 (``Sema::UsualUnaryConversions``), for places where the subexpression is
|
|
|
2575 producing a value you intend to use.
|
|
|
2576 * Your ``BuildXXX`` function will probably just return ``ExprError()`` at
|
|
|
2577 this point, since you don't have an AST. That's perfectly fine, and
|
|
|
2578 shouldn't impact your testing.
|
|
|
2579
|
|
|
2580 #. Introduce an AST node for your new expression. This starts with declaring
|
|
|
2581 the node in ``include/Basic/StmtNodes.td`` and creating a new class for your
|
|
|
2582 expression in the appropriate ``include/AST/Expr*.h`` header. It's best to
|
|
|
2583 look at the class for a similar expression to get ideas, and there are some
|
|
|
2584 specific things to watch for:
|
|
|
2585
|
|
|
2586 * If you need to allocate memory, use the ``ASTContext`` allocator to
|
|
|
2587 allocate memory. Never use raw ``malloc`` or ``new``, and never hold any
|
|
|
2588 resources in an AST node, because the destructor of an AST node is never
|
|
|
2589 called.
|
|
|
2590 * Make sure that ``getSourceRange()`` covers the exact source range of your
|
|
|
2591 expression. This is needed for diagnostics and for IDE support.
|
|
|
2592 * Make sure that ``children()`` visits all of the subexpressions. This is
|
|
|
2593 important for a number of features (e.g., IDE support, C++ variadic
|
|
|
2594 templates). If you have sub-types, you'll also need to visit those
|
|
|
2595 sub-types in ``RecursiveASTVisitor``.
|
|
|
2596 * Add printing support (``StmtPrinter.cpp``) for your expression.
|
|
|
2597 * Add profiling support (``StmtProfile.cpp``) for your AST node, noting the
|
|
|
2598 distinguishing (non-source location) characteristics of an instance of
|
|
|
2599 your expression. Omitting this step will lead to hard-to-diagnose
|
|
|
2600 failures regarding matching of template declarations.
|
|
|
2601 * Add serialization support (``ASTReaderStmt.cpp``, ``ASTWriterStmt.cpp``)
|
|
|
2602 for your AST node.
|
|
|
2603
|
|
|
2604 #. Teach semantic analysis to build your AST node. At this point, you can wire
|
|
|
2605 up your ``Sema::BuildXXX`` function to actually create your AST. A few
|
|
|
2606 things to check at this point:
|
|
|
2607
|
|
|
2608 * If your expression can construct a new C++ class or return a new
|
|
|
2609 Objective-C object, be sure to update and then call
|
|
|
2610 ``Sema::MaybeBindToTemporary`` for your just-created AST node to be sure
|
|
|
2611 that the object gets properly destructed. An easy way to test this is to
|
|
|
2612 return a C++ class with a private destructor: semantic analysis should
|
|
|
2613 flag an error here with the attempt to call the destructor.
|
|
|
2614 * Inspect the generated AST by printing it using ``clang -cc1 -ast-print``,
|
|
|
2615 to make sure you're capturing all of the important information about how
|
|
|
2616 the AST was written.
|
|
|
2617 * Inspect the generated AST under ``clang -cc1 -ast-dump`` to verify that
|
|
|
2618 all of the types in the generated AST line up the way you want them.
|
|
|
2619 Remember that clients of the AST should never have to "think" to
|
|
|
2620 understand what's going on. For example, all implicit conversions should
|
|
|
2621 show up explicitly in the AST.
|
|
|
2622 * Write tests that use your expression as a subexpression of other,
|
|
|
2623 well-known expressions. Can you call a function using your expression as
|
|
|
2624 an argument? Can you use the ternary operator?
|
|
|
2625
|
|
|
2626 #. Teach code generation to create IR to your AST node. This step is the first
|
|
|
2627 (and only) that requires knowledge of LLVM IR. There are several things to
|
|
|
2628 keep in mind:
|
|
|
2629
|
|
|
2630 * Code generation is separated into scalar/aggregate/complex and
|
|
|
2631 lvalue/rvalue paths, depending on what kind of result your expression
|
|
|
2632 produces. On occasion, this requires some careful factoring of code to
|
|
|
2633 avoid duplication.
|
|
|
2634 * ``CodeGenFunction`` contains functions ``ConvertType`` and
|
|
|
2635 ``ConvertTypeForMem`` that convert Clang's types (``clang::Type*`` or
|
|
|
2636 ``clang::QualType``) to LLVM types. Use the former for values, and the
|
|
|
2637 latter for memory locations: test with the C++ "``bool``" type to check
|
|
|
2638 this. If you find that you are having to use LLVM bitcasts to make the
|
|
|
2639 subexpressions of your expression have the type that your expression
|
|
|
2640 expects, STOP! Go fix semantic analysis and the AST so that you don't
|
|
|
2641 need these bitcasts.
|
|
|
2642 * The ``CodeGenFunction`` class has a number of helper functions to make
|
|
|
2643 certain operations easy, such as generating code to produce an lvalue or
|
|
|
2644 an rvalue, or to initialize a memory location with a given value. Prefer
|
|
|
2645 to use these functions rather than directly writing loads and stores,
|
|
|
2646 because these functions take care of some of the tricky details for you
|
|
|
2647 (e.g., for exceptions).
|
|
|
2648 * If your expression requires some special behavior in the event of an
|
|
|
2649 exception, look at the ``push*Cleanup`` functions in ``CodeGenFunction``
|
|
|
2650 to introduce a cleanup. You shouldn't have to deal with
|
|
|
2651 exception-handling directly.
|
|
|
2652 * Testing is extremely important in IR generation. Use ``clang -cc1
|
|
|
2653 -emit-llvm`` and `FileCheck
|
|
|
2654 <https://llvm.org/docs/CommandGuide/FileCheck.html>`_ to verify that you're
|
|
|
2655 generating the right IR.
|
|
|
2656
|
|
|
2657 #. Teach template instantiation how to cope with your AST node, which requires
|
|
|
2658 some fairly simple code:
|
|
|
2659
|
|
|
2660 * Make sure that your expression's constructor properly computes the flags
|
|
|
2661 for type dependence (i.e., the type your expression produces can change
|
|
|
2662 from one instantiation to the next), value dependence (i.e., the constant
|
|
|
2663 value your expression produces can change from one instantiation to the
|
|
|
2664 next), instantiation dependence (i.e., a template parameter occurs
|
|
|
2665 anywhere in your expression), and whether your expression contains a
|
|
|
2666 parameter pack (for variadic templates). Often, computing these flags
|
|
|
2667 just means combining the results from the various types and
|
|
|
2668 subexpressions.
|
|
|
2669 * Add ``TransformXXX`` and ``RebuildXXX`` functions to the ``TreeTransform``
|
|
|
2670 class template in ``Sema``. ``TransformXXX`` should (recursively)
|
|
|
2671 transform all of the subexpressions and types within your expression,
|
|
|
2672 using ``getDerived().TransformYYY``. If all of the subexpressions and
|
|
|
2673 types transform without error, it will then call the ``RebuildXXX``
|
|
|
2674 function, which will in turn call ``getSema().BuildXXX`` to perform
|
|
|
2675 semantic analysis and build your expression.
|
|
|
2676 * To test template instantiation, take those tests you wrote to make sure
|
|
|
2677 that you were type checking with type-dependent expressions and dependent
|
|
|
2678 types (from step #2) and instantiate those templates with various types,
|
|
|
2679 some of which type-check and some that don't, and test the error messages
|
|
|
2680 in each case.
|
|
|
2681
|
|
|
2682 #. There are some "extras" that make other features work better. It's worth
|
|
|
2683 handling these extras to give your expression complete integration into
|
|
|
2684 Clang:
|
|
|
2685
|
|
|
2686 * Add code completion support for your expression in
|
|
|
2687 ``SemaCodeComplete.cpp``.
|
|
|
2688 * If your expression has types in it, or has any "interesting" features
|
|
|
2689 other than subexpressions, extend libclang's ``CursorVisitor`` to provide
|
|
|
2690 proper visitation for your expression, enabling various IDE features such
|
|
|
2691 as syntax highlighting, cross-referencing, and so on. The
|
|
|
2692 ``c-index-test`` helper program can be used to test these features.
|
|
|
2693
|
