Mercurial > hg > CbC > CbC_llvm

annotate tools/clang/docs/PTHInternals.rst @ 3:9ad51c7bc036

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
1st commit. remove git dir and add all files.
author Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
date Wed, 15 May 2013 06:43:32 +0900
parents
children 54457678186b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
1 ==========================
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
2 Pretokenized Headers (PTH)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
3 ==========================
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
4
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
5 This document first describes the low-level interface for using PTH and
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
6 then briefly elaborates on its design and implementation. If you are
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
7 interested in the end-user view, please see the :ref:`User's Manual
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
8 <usersmanual-precompiled-headers>`.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
9
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
10 Using Pretokenized Headers with ``clang`` (Low-level Interface)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
11 ===============================================================
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
12
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
13 The Clang compiler frontend, ``clang -cc1``, supports three command line
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
14 options for generating and using PTH files.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
15
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
16 To generate PTH files using ``clang -cc1``, use the option ``-emit-pth``:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
17
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
18 .. code-block:: console
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
19
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
20 $ clang -cc1 test.h -emit-pth -o test.h.pth
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
21
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
22 This option is transparently used by ``clang`` when generating PTH
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
23 files. Similarly, PTH files can be used as prefix headers using the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
24 ``-include-pth`` option:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
25
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
26 .. code-block:: console
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
27
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
28 $ clang -cc1 -include-pth test.h.pth test.c -o test.s
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
29
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
30 Alternatively, Clang's PTH files can be used as a raw "token-cache" (or
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
31 "content" cache) of the source included by the original header file.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
32 This means that the contents of the PTH file are searched as substitutes
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
33 for *any* source files that are used by ``clang -cc1`` to process a
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
34 source file. This is done by specifying the ``-token-cache`` option:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
35
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
36 .. code-block:: console
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
37
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
38 $ cat test.h
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
39 #include <stdio.h>
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
40 $ clang -cc1 -emit-pth test.h -o test.h.pth
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
41 $ cat test.c
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
42 #include "test.h"
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
43 $ clang -cc1 test.c -o test -token-cache test.h.pth
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
44
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
45 In this example the contents of ``stdio.h`` (and the files it includes)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
46 will be retrieved from ``test.h.pth``, as the PTH file is being used in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
47 this case as a raw cache of the contents of ``test.h``. This is a
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
48 low-level interface used to both implement the high-level PTH interface
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
49 as well as to provide alternative means to use PTH-style caching.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
50
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
51 PTH Design and Implementation
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
52 =============================
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
53
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
54 Unlike GCC's precompiled headers, which cache the full ASTs and
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
55 preprocessor state of a header file, Clang's pretokenized header files
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
56 mainly cache the raw lexer *tokens* that are needed to segment the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
57 stream of characters in a source file into keywords, identifiers, and
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
58 operators. Consequently, PTH serves to mainly directly speed up the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
59 lexing and preprocessing of a source file, while parsing and
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
60 type-checking must be completely redone every time a PTH file is used.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
61
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
62 Basic Design Tradeoffs
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
63 ----------------------
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
64
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
65 In the long term there are plans to provide an alternate PCH
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
66 implementation for Clang that also caches the work for parsing and type
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
67 checking the contents of header files. The current implementation of PCH
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
68 in Clang as pretokenized header files was motivated by the following
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
69 factors:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
70
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
71 **Language independence**
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
72 PTH files work with any language that
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
73 Clang's lexer can handle, including C, Objective-C, and (in the early
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
74 stages) C++. This means development on language features at the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
75 parsing level or above (which is basically almost all interesting
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
76 pieces) does not require PTH to be modified.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
77
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
78 **Simple design**
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
79 Relatively speaking, PTH has a simple design and
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
80 implementation, making it easy to test. Further, because the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
81 machinery for PTH resides at the lower-levels of the Clang library
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
82 stack it is fairly straightforward to profile and optimize.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
83
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
84 Further, compared to GCC's PCH implementation (which is the dominate
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
85 precompiled header file implementation that Clang can be directly
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
86 compared against) the PTH design in Clang yields several attractive
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
87 features:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
88
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
89 **Architecture independence**
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
90 In contrast to GCC's PCH files (and
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
91 those of several other compilers), Clang's PTH files are architecture
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
92 independent, requiring only a single PTH file when building a
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
93 program for multiple architectures.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
94
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
95 For example, on Mac OS X one may wish to compile a "universal binary"
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
96 that runs on PowerPC, 32-bit Intel (i386), and 64-bit Intel
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
97 architectures. In contrast, GCC requires a PCH file for each
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
98 architecture, as the definitions of types in the AST are
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
99 architecture-specific. Since a Clang PTH file essentially represents
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
100 a lexical cache of header files, a single PTH file can be safely used
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
101 when compiling for multiple architectures. This can also reduce
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
102 compile times because only a single PTH file needs to be generated
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
103 during a build instead of several.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
104
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
105 **Reduced memory pressure**
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
106 Similar to GCC, Clang reads PTH files
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
107 via the use of memory mapping (i.e., ``mmap``). Clang, however,
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
108 memory maps PTH files as read-only, meaning that multiple invocations
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
109 of ``clang -cc1`` can share the same pages in memory from a
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
110 memory-mapped PTH file. In comparison, GCC also memory maps its PCH
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
111 files but also modifies those pages in memory, incurring the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
112 copy-on-write costs. The read-only nature of PTH can greatly reduce
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
113 memory pressure for builds involving multiple cores, thus improving
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
114 overall scalability.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
115
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
116 **Fast generation**
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
117 PTH files can be generated in a small fraction
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
118 of the time needed to generate GCC's PCH files. Since PTH/PCH
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
119 generation is a serial operation that typically blocks progress
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
120 during a build, faster generation time leads to improved processor
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
121 utilization with parallel builds on multicore machines.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
122
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
123 Despite these strengths, PTH's simple design suffers some algorithmic
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
124 handicaps compared to other PCH strategies such as those used by GCC.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
125 While PTH can greatly speed up the processing time of a header file, the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
126 amount of work required to process a header file is still roughly linear
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
127 in the size of the header file. In contrast, the amount of work done by
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
128 GCC to process a precompiled header is (theoretically) constant (the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
129 ASTs for the header are literally memory mapped into the compiler). This
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
130 means that only the pieces of the header file that are referenced by the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
131 source file including the header are the only ones the compiler needs to
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
132 process during actual compilation. While GCC's particular implementation
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
133 of PCH mitigates some of these algorithmic strengths via the use of
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
134 copy-on-write pages, the approach itself can fundamentally dominate at
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
135 an algorithmic level, especially when one considers header files of
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
136 arbitrary size.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
137
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
138 There are plans to potentially implement an complementary PCH
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
139 implementation for Clang based on the lazy deserialization of ASTs. This
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
140 approach would theoretically have the same constant-time algorithmic
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
141 advantages just mentioned but would also retain some of the strengths of
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
142 PTH such as reduced memory pressure (ideal for multi-core builds).
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
143
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
144 Internal PTH Optimizations
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
145 --------------------------
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
146
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
147 While the main optimization employed by PTH is to reduce lexing time of
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
148 header files by caching pre-lexed tokens, PTH also employs several other
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
149 optimizations to speed up the processing of header files:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
150
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
151 - ``stat`` caching: PTH files cache information obtained via calls to
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
152 ``stat`` that ``clang -cc1`` uses to resolve which files are included
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
153 by ``#include`` directives. This greatly reduces the overhead
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
154 involved in context-switching to the kernel to resolve included
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
155 files.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
156
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
157 - Fast skipping of ``#ifdef`` ... ``#endif`` chains: PTH files
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
158 record the basic structure of nested preprocessor blocks. When the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
159 condition of the preprocessor block is false, all of its tokens are
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
160 immediately skipped instead of requiring them to be handled by
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
161 Clang's preprocessor.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
162
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
163