Mercurial > hg > CbC > CbC_llvm

comparison docs/Remarks.rst @ 148:63bd29f05246

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
merged
author Shinji KONO <kono@ie.u-ryukyu.ac.jp>
date Wed, 14 Aug 2019 19:46:37 +0900
parents c2174574ed3a
children
comparison
equal deleted inserted replaced
146:3fc4d5c3e21e 148:63bd29f05246
1 =======
2 Remarks
3 =======
4
5 .. contents::
6 :local:
7
8 Introduction to the LLVM remark diagnostics
9 ===========================================
10
11 LLVM is able to emit diagnostics from passes describing whether an optimization
12 has been performed or missed for a particular reason, which should give more
13 insight to users about what the compiler did during the compilation pipeline.
14
15 There are three main remark types:
16
17 ``Passed``
18
19 Remarks that describe a successful optimization performed by the compiler.
20
21 :Example:
22
23 ::
24
25 foo inlined into bar with (cost=always): always inline attribute
26
27 ``Missed``
28
29 Remarks that describe an attempt to an optimization by the compiler that
30 could not be performed.
31
32 :Example:
33
34 ::
35
36 foo not inlined into bar because it should never be inlined
37 (cost=never): noinline function attribute
38
39 ``Analysis``
40
41 Remarks that describe the result of an analysis, that can bring more
42 information to the user regarding the generated code.
43
44 :Example:
45
46 ::
47
48 16 stack bytes in function
49
50 ::
51
52 10 instructions in function
53
54 Enabling optimization remarks
55 =============================
56
57 There are two modes that are supported for enabling optimization remarks in
58 LLVM: through remark diagnostics, or through serialized remarks.
59
60 Remark diagnostics
61 ------------------
62
63 Optimization remarks can be emitted as diagnostics. These diagnostics will be
64 propagated to front-ends if desired, or emitted by tools like :doc:`llc
65 <CommandGuide/llc>` or :doc:`opt <CommandGuide/opt>`.
66
67 .. option:: -pass-remarks=<regex>
68
69 Enables optimization remarks from passes whose name match the given (POSIX)
70 regular expression.
71
72 .. option:: -pass-remarks-missed=<regex>
73
74 Enables missed optimization remarks from passes whose name match the given
75 (POSIX) regular expression.
76
77 .. option:: -pass-remarks-analysis=<regex>
78
79 Enables optimization analysis remarks from passes whose name match the given
80 (POSIX) regular expression.
81
82 Serialized remarks
83 ------------------
84
85 While diagnostics are useful during development, it is often more useful to
86 refer to optimization remarks post-compilation, typically during performance
87 analysis.
88
89 For that, LLVM can serialize the remarks produced for each compilation unit to
90 a file that can be consumed later.
91
92 By default, the format of the serialized remarks is :ref:`YAML
93 <yamlremarks>`, and it can be accompanied by a :ref:`section <remarkssection>`
94 in the object files to easily retrieve it.
95
96 :doc:`llc <CommandGuide/llc>` and :doc:`opt <CommandGuide/opt>` support the
97 following options:
98
99
100 ``Basic options``
101
102 .. option:: -pass-remarks-output=<filename>
103
104 Enables the serialization of remarks to a file specified in <filename>.
105
106 By default, the output is serialized to :ref:`YAML <yamlremarks>`.
107
108 .. option:: -pass-remarks-format=<format>
109
110 Specifies the output format of the serialized remarks.
111
112 Supported formats:
113
114 * :ref:`yaml <yamlremarks>` (default)
115 * :ref:`yaml-strtab <yamlstrtabremarks>`
116 * :ref:`bitstream <bitstreamremarks>`
117
118 ``Content configuration``
119
120 .. option:: -pass-remarks-filter=<regex>
121
122 Only passes whose name match the given (POSIX) regular expression will be
123 serialized to the final output.
124
125 .. option:: -pass-remarks-with-hotness
126
127 With PGO, include profile count in optimization remarks.
128
129 .. option:: -pass-remarks-hotness-threshold
130
131 The minimum profile count required for an optimization remark to be
132 emitted.
133
134 Other tools that support remarks:
135
136 :program:`llvm-lto`
137
138 .. option:: -lto-pass-remarks-output=<filename>
139 .. option:: -lto-pass-remarks-filter=<regex>
140 .. option:: -lto-pass-remarks-format=<format>
141 .. option:: -lto-pass-remarks-with-hotness
142 .. option:: -lto-pass-remarks-hotness-threshold
143
144 :program:`gold-plugin` and :program:`lld`
145
146 .. option:: -opt-remarks-filename=<filename>
147 .. option:: -opt-remarks-filter=<regex>
148 .. option:: -opt-remarks-format=<format>
149 .. option:: -opt-remarks-with-hotness
150
151 Serialization modes
152 ===================
153
154 There are two modes available for serializing remarks:
155
156 ``Separate``
157
158 In this mode, the remarks and the metadata are serialized separately. The
159 client is responsible for parsing the metadata first, then use the metadata
160 to correctly parse the remarks.
161
162 ``Standalone``
163
164 In this mode, the remarks and the metadata are serialized to the same
165 stream. The metadata will always come before the remarks.
166
167 The compiler does not support emitting standalone remarks. This mode is
168 more suited for post-processing tools like linkers, that can merge the
169 remarks for one whole project.
170
171 .. _yamlremarks:
172
173 YAML remarks
174 ============
175
176 A typical remark serialized to YAML looks like this:
177
178 .. code-block:: yaml
179
180 --- !<TYPE>
181 Pass: <pass>
182 Name: <name>
183 DebugLoc: { File: <file>, Line: <line>, Column: <column> }
184 Function: <function>
185 Hotness: <hotness>
186 Args:
187 - <key>: <value>
188 DebugLoc: { File: <arg-file>, Line: <arg-line>, Column: <arg-column> }
189
190 The following entries are mandatory:
191
192 * ``<TYPE>``: can be ``Passed``, ``Missed``, ``Analysis``,
193 ``AnalysisFPCommute``, ``AnalysisAliasing``, ``Failure``.
194 * ``<pass>``: the name of the pass that emitted this remark.
195 * ``<name>``: the name of the remark coming from ``<pass>``.
196 * ``<function>``: the mangled name of the function.
197
198 If a ``DebugLoc`` entry is specified, the following fields are required:
199
200 * ``<file>``
201 * ``<line>``
202 * ``<column>``
203
204 If an ``arg`` entry is specified, the following fields are required:
205
206 * ``<key>``
207 * ``<value>``
208
209 If a ``DebugLoc`` entry is specified within an ``arg`` entry, the following
210 fields are required:
211
212 * ``<arg-file>``
213 * ``<arg-line>``
214 * ``<arg-column>``
215
216 .. _yamlstrtabremarks:
217
218 YAML with a string table
219 ------------------------
220
221 The YAML serialization supports the usage of a string table by using the
222 ``yaml-strtab`` format.
223
224 This format replaces strings in the YAML output with integers representing the
225 index in the string table that can be provided separately through metadata.
226
227 The following entries can take advantage of the string table while respecting
228 YAML rules:
229
230 * ``<pass>``
231 * ``<name>``
232 * ``<function>``
233 * ``<file>``
234 * ``<value>``
235 * ``<arg-file>``
236
237 Currently, none of the tools in :ref:`the opt-viewer directory <optviewer>`
238 support this format.
239
240 .. _optviewer:
241
242 YAML metadata
243 -------------
244
245 The metadata used together with the YAML format is:
246
247 * a magic number: "REMARKS\\0"
248 * the version number: a little-endian uint64_t
249 * the total size of the string table (the size itself excluded):
250 little-endian uint64_t
251 * a list of null-terminated strings
252
253 Optional:
254
255 * the absolute file path to the serialized remark diagnostics: a
256 null-terminated string.
257
258 When the metadata is serialized separately from the remarks, the file path
259 should be present and point to the file where the remarks are serialized to.
260
261 In case the metadata only acts as a header to the remarks, the file path can be
262 omitted.
263
264 .. _bitstreamremarks:
265
266 LLVM bitstream remarks
267 ======================
268
269 This format is using :doc:`LLVM bitstream <BitCodeFormat>` to serialize remarks
270 and their associated metadata.
271
272 A bitstream remark stream can be identified by the magic number ``"RMRK"`` that
273 is placed at the very beginning.
274
275 The format for serializing remarks is composed of two different block types:
276
277 .. _bitstreamremarksmetablock:
278
279 META_BLOCK
280 ----------
281
282 The block providing information about the rest of the content in the stream.
283
284 Exactly one block is expected. Having multiple metadata blocks is an error.
285
286 This block can contain the following records:
287
288 .. _bitstreamremarksrecordmetacontainerinfo:
289
290 ``RECORD_META_CONTAINER_INFO``
291
292 The container version and type.
293
294 Version: u32
295
296 Type: u2
297
298 .. _bitstreamremarksrecordmetaremarkversion:
299
300 ``RECORD_META_REMARK_VERSION``
301
302 The version of the remark entries. This can change independently from the
303 container version.
304
305 Version: u32
306
307 .. _bitstreamremarksrecordmetastrtab:
308
309 ``RECORD_META_STRTAB``
310
311 The string table used by the remark entries. The format of the string table
312 is a sequence of strings separated by ``\0``.
313
314 .. _bitstreamremarksrecordmetaexternalfile:
315
316 ``RECORD_META_EXTERNAL_FILE``
317
318 The external remark file path that contains the remark blocks associated
319 with this metadata. This is an absolute path.
320
321 .. _bitstreamremarksremarkblock:
322
323 REMARK_BLOCK
324 ------------
325
326 The block describing a remark entry.
327
328 0 or more blocks per file are allowed. Each block will depend on the
329 :ref:`META_BLOCK <bitstreamremarksmetablock>` in order to be parsed correctly.
330
331 This block can contain the following records:
332
333 ``RECORD_REMARK_HEADER``
334
335 The header of the remark. This contains all the mandatory information about
336 a remark.
337
338 +---------------+---------------------------+
339 | Type | u3 |
340 +---------------+---------------------------+
341 | Remark name | VBR6 (string table index) |
342 +---------------+---------------------------+
343 | Pass name | VBR6 (string table index) |
344 +---------------+---------------------------+
345 | Function name | VBR6 (string table index) |
346 +---------------+---------------------------+
347
348 ``RECORD_REMARK_DEBUG_LOC``
349
350 The source location for the corresponding remark. This record is optional.
351
352 +--------+---------------------------+
353 | File | VBR7 (string table index) |
354 +--------+---------------------------+
355 | Line | u32 |
356 +--------+---------------------------+
357 | Column | u32 |
358 +--------+---------------------------+
359
360 ``RECORD_REMARK_HOTNESS``
361
362 The hotness of the remark. This record is optional.
363
364 +---------------+---------------------+
365 | Hotness | VBR8 (string table index) |
366 +---------------+---------------------+
367
368 ``RECORD_REMARK_ARG_WITH_DEBUGLOC``
369
370 A remark argument with an associated debug location.
371
372 +--------+---------------------------+
373 | Key | VBR7 (string table index) |
374 +--------+---------------------------+
375 | Value | VBR7 (string table index) |
376 +--------+---------------------------+
377 | File | VBR7 (string table index) |
378 +--------+---------------------------+
379 | Line | u32 |
380 +--------+---------------------------+
381 | Column | u32 |
382 +--------+---------------------------+
383
384 ``RECORD_REMARK_ARG_WITHOUT_DEBUGLOC``
385
386 A remark argument with an associated debug location.
387
388 +--------+---------------------------+
389 | Key | VBR7 (string table index) |
390 +--------+---------------------------+
391 | Value | VBR7 (string table index) |
392 +--------+---------------------------+
393
394 The remark container
395 --------------------
396
397 Bitstream remarks are designed to be used in two different modes:
398
399 ``The separate mode``
400
401 The separate mode is the mode that is typically used during compilation. It
402 provides a way to serialize the remark entries to a stream while some
403 metadata is kept in memory to be emitted in the product of the compilation
404 (typically, an object file).
405
406 ``The standalone mode``
407
408 The standalone mode is typically stored and used after the distribution of
409 a program. It contains all the information that allows the parsing of all
410 the remarks without having any external dependencies.
411
412 In order to support multiple modes, the format introduces the concept of a
413 bitstream remark container type.
414
415 .. _bitstreamremarksseparateremarksmeta:
416
417 ``SeparateRemarksMeta: the metadata emitted separately``
418
419 This container type expects only a :ref:`META_BLOCK <bitstreamremarksmetablock>` containing only:
420
421 * :ref:`RECORD_META_CONTAINER_INFO <bitstreamremarksrecordmetacontainerinfo>`
422 * :ref:`RECORD_META_STRTAB <bitstreamremarksrecordmetastrtab>`
423 * :ref:`RECORD_META_EXTERNAL_FILE <bitstreamremarksrecordmetaexternalfile>`
424
425 Typically, this is emitted in a section in the object files, allowing
426 clients to retrieve remarks and their associated metadata directly from
427 intermediate products.
428
429 ``SeparateRemarksFile: the remark entries emitted separately``
430
431 This container type expects only a :ref:`META_BLOCK <bitstreamremarksmetablock>` containing only:
432
433 * :ref:`RECORD_META_CONTAINER_INFO <bitstreamremarksrecordmetacontainerinfo>`
434 * :ref:`RECORD_META_REMARK_VERSION <bitstreamremarksrecordmetaremarkversion>`
435
436 This container type expects 0 or more :ref:`REMARK_BLOCK <bitstreamremarksremarkblock>`.
437
438 Typically, this is emitted in a side-file alongside an object file, and is
439 made to be able to stream to without increasing the memory consumption of
440 the compiler. This is referenced by the :ref:`RECORD_META_EXTERNAL_FILE
441 <bitstreamremarksrecordmetaexternalfile>` entry in the
442 :ref:`SeparateRemarksMeta <bitstreamremarksseparateremarksmeta>` container.
443
444 When the parser tries to parse a container that contains the metadata for the
445 separate remarks, it should parse the version and type, then keep the string
446 table in memory while opening the external file, validating its metadata and
447 parsing the remark entries.
448
449 The container versions from the separate container should match in order to
450 have a well-formed file.
451
452 ``Standalone: the metadata and the remark entries emitted together``
453
454 This container type expects only a :ref:`META_BLOCK <bitstreamremarksmetablock>` containing only:
455
456 * :ref:`RECORD_META_CONTAINER_INFO <bitstreamremarksrecordmetacontainerinfo>`
457 * :ref:`RECORD_META_REMARK_VERSION <bitstreamremarksrecordmetaremarkversion>`
458 * :ref:`RECORD_META_STRTAB <bitstreamremarksrecordmetastrtab>`
459
460 This container type expects 0 or more :ref:`REMARK_BLOCK <bitstreamremarksremarkblock>`.
461
462 A complete output of :program:`llvm-bcanalyzer` on the different container types:
463
464 ``SeparateRemarksMeta``
465
466 .. code-block:: none
467
468 <BLOCKINFO_BLOCK/>
469 <Meta BlockID=8 NumWords=13 BlockCodeSize=3>
470 <Container info codeid=1 abbrevid=4 op0=5 op1=0/>
471 <String table codeid=3 abbrevid=5/> blob data = 'pass\\x00key\\x00value\\x00'
472 <External File codeid=4 abbrevid=6/> blob data = '/path/to/file/name'
473 </Meta>
474
475 ``SeparateRemarksFile``
476
477 .. code-block:: none
478
479 <BLOCKINFO_BLOCK/>
480 <Meta BlockID=8 NumWords=3 BlockCodeSize=3>
481 <Container info codeid=1 abbrevid=4 op0=0 op1=1/>
482 <Remark version codeid=2 abbrevid=5 op0=0/>
483 </Meta>
484 <Remark BlockID=9 NumWords=8 BlockCodeSize=4>
485 <Remark header codeid=5 abbrevid=4 op0=2 op1=0 op2=1 op3=2/>
486 <Remark debug location codeid=6 abbrevid=5 op0=3 op1=99 op2=55/>
487 <Remark hotness codeid=7 abbrevid=6 op0=999999999/>
488 <Argument with debug location codeid=8 abbrevid=7 op0=4 op1=5 op2=6 op3=11 op4=66/>
489 </Remark>
490
491 ``Standalone``
492
493 .. code-block:: none
494
495 <BLOCKINFO_BLOCK/>
496 <Meta BlockID=8 NumWords=15 BlockCodeSize=3>
497 <Container info codeid=1 abbrevid=4 op0=5 op1=2/>
498 <Remark version codeid=2 abbrevid=5 op0=30/>
499 <String table codeid=3 abbrevid=6/> blob data = 'pass\\x00remark\\x00function\\x00path\\x00key\\x00value\\x00argpath\\x00'
500 </Meta>
501 <Remark BlockID=9 NumWords=8 BlockCodeSize=4>
502 <Remark header codeid=5 abbrevid=4 op0=2 op1=1 op2=0 op3=2/>
503 <Remark debug location codeid=6 abbrevid=5 op0=3 op1=99 op2=55/>
504 <Remark hotness codeid=7 abbrevid=6 op0=999999999/>
505 <Argument with debug location codeid=8 abbrevid=7 op0=4 op1=5 op2=6 op3=11 op4=66/>
506 </Remark>
507
508 opt-viewer
509 ==========
510
511 The ``opt-viewer`` directory contains a collection of tools that visualize and
512 summarize serialized remarks.
513
514 The tools only support the ``yaml`` format.
515
516 .. _optviewerpy:
517
518 opt-viewer.py
519 -------------
520
521 Output a HTML page which gives visual feedback on compiler interactions with
522 your program.
523
524 :Examples:
525
526 ::
527
528 $ opt-viewer.py my_yaml_file.opt.yaml
529
530 ::
531
532 $ opt-viewer.py my_build_dir/
533
534
535 opt-stats.py
536 ------------
537
538 Output statistics about the optimization remarks in the input set.
539
540 :Example:
541
542 ::
543
544 $ opt-stats.py my_yaml_file.opt.yaml
545
546 Total number of remarks 3
547
548
549 Top 10 remarks by pass:
550 inline 33%
551 asm-printer 33%
552 prologepilog 33%
553
554 Top 10 remarks:
555 asm-printer/InstructionCount 33%
556 inline/NoDefinition 33%
557 prologepilog/StackSize 33%
558
559 opt-diff.py
560 -----------
561
562 Produce a new YAML file which contains all of the changes in optimizations
563 between two YAML files.
564
565 Typically, this tool should be used to do diffs between:
566
567 * new compiler + fixed source vs old compiler + fixed source
568 * fixed compiler + new source vs fixed compiler + old source
569
570 This diff file can be displayed using :ref:`opt-viewer.py <optviewerpy>`.
571
572 :Example:
573
574 ::
575
576 $ opt-diff.py my_opt_yaml1.opt.yaml my_opt_yaml2.opt.yaml -o my_opt_diff.opt.yaml
577 $ opt-viewer.py my_opt_diff.opt.yaml
578
579 .. _remarkssection:
580
581 Emitting remark diagnostics in the object file
582 ==============================================
583
584 A section containing metadata on remark diagnostics will be emitted when
585 -remarks-section is passed. The section contains the metadata associated to the
586 format used to serialize the remarks.
587
588 The section is named:
589
590 * ``__LLVM,__remarks`` (MachO)
591 * ``.remarks`` (ELF)
592
593 C API
594 =====
595
596 LLVM provides a library that can be used to parse remarks through a shared
597 library named ``libRemarks``.
598
599 The typical usage through the C API is like the following:
600
601 .. code-block:: c
602
603 LLVMRemarkParserRef Parser = LLVMRemarkParserCreateYAML(Buf, Size);
604 LLVMRemarkEntryRef Remark = NULL;
605 while ((Remark = LLVMRemarkParserGetNext(Parser))) {
606 // use Remark
607 LLVMRemarkEntryDispose(Remark); // Release memory.
608 }
609 bool HasError = LLVMRemarkParserHasError(Parser);
610 LLVMRemarkParserDispose(Parser);
611
612 .. FIXME: add documentation for llvm-opt-report.
613 .. FIXME: add documentation for Passes supporting optimization remarks
614 .. FIXME: add documentation for IR Passes
615 .. FIXME: add documentation for CodeGen Passes