Mercurial > hg > CbC > CbC_llvm

annotate docs/TestSuiteMakefileGuide.rst @ 18:5abb684876c4

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
report error when code segment's tail call flag was changed false
author Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
date Sun, 06 Oct 2013 00:17:31 +0900
parents 9ad51c7bc036
children 1172e4bd9c6f
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
1 ==============================
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
2 LLVM test-suite Makefile Guide
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
3 ==============================
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
4
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
5 .. contents::
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
6 :local:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
7
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
8 Overview
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
9 ========
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
10
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
11 This document describes the features of the Makefile-based LLVM
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
12 test-suite. This way of interacting with the test-suite is deprecated in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
13 favor of running the test-suite using LNT, but may continue to prove
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
14 useful for some users. See the Testing Guide's :ref:`test-suite Quickstart
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
15 <test-suite-quickstart>` section for more information.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
16
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
17 Test suite Structure
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
18 ====================
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
19
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
20 The ``test-suite`` module contains a number of programs that can be
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
21 compiled with LLVM and executed. These programs are compiled using the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
22 native compiler and various LLVM backends. The output from the program
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
23 compiled with the native compiler is assumed correct; the results from
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
24 the other programs are compared to the native program output and pass if
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
25 they match.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
26
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
27 When executing tests, it is usually a good idea to start out with a
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
28 subset of the available tests or programs. This makes test run times
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
29 smaller at first and later on this is useful to investigate individual
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
30 test failures. To run some test only on a subset of programs, simply
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
31 change directory to the programs you want tested and run ``gmake``
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
32 there. Alternatively, you can run a different test using the ``TEST``
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
33 variable to change what tests or run on the selected programs (see below
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
34 for more info).
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
35
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
36 In addition for testing correctness, the ``test-suite`` directory also
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
37 performs timing tests of various LLVM optimizations. It also records
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
38 compilation times for the compilers and the JIT. This information can be
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
39 used to compare the effectiveness of LLVM's optimizations and code
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
40 generation.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
41
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
42 ``test-suite`` tests are divided into three types of tests: MultiSource,
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
43 SingleSource, and External.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
44
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
45 - ``test-suite/SingleSource``
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
46
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
47 The SingleSource directory contains test programs that are only a
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
48 single source file in size. These are usually small benchmark
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
49 programs or small programs that calculate a particular value. Several
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
50 such programs are grouped together in each directory.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
51
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
52 - ``test-suite/MultiSource``
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
53
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
54 The MultiSource directory contains subdirectories which contain
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
55 entire programs with multiple source files. Large benchmarks and
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
56 whole applications go here.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
57
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
58 - ``test-suite/External``
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
59
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
60 The External directory contains Makefiles for building code that is
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
61 external to (i.e., not distributed with) LLVM. The most prominent
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
62 members of this directory are the SPEC 95 and SPEC 2000 benchmark
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
63 suites. The ``External`` directory does not contain these actual
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
64 tests, but only the Makefiles that know how to properly compile these
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
65 programs from somewhere else. The presence and location of these
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
66 external programs is configured by the test-suite ``configure``
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
67 script.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
68
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
69 Each tree is then subdivided into several categories, including
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
70 applications, benchmarks, regression tests, code that is strange
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
71 grammatically, etc. These organizations should be relatively self
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
72 explanatory.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
73
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
74 Some tests are known to fail. Some are bugs that we have not fixed yet;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
75 others are features that we haven't added yet (or may never add). In the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
76 regression tests, the result for such tests will be XFAIL (eXpected
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
77 FAILure). In this way, you can tell the difference between an expected
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
78 and unexpected failure.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
79
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
80 The tests in the test suite have no such feature at this time. If the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
81 test passes, only warnings and other miscellaneous output will be
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
82 generated. If a test fails, a large <program> FAILED message will be
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
83 displayed. This will help you separate benign warnings from actual test
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
84 failures.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
85
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
86 Running the test suite
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
87 ======================
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
88
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
89 First, all tests are executed within the LLVM object directory tree.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
90 They *are not* executed inside of the LLVM source tree. This is because
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
91 the test suite creates temporary files during execution.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
92
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
93 To run the test suite, you need to use the following steps:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
94
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
95 #. ``cd`` into the ``llvm/projects`` directory in your source tree.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
96 #. Check out the ``test-suite`` module with:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
97
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
98 .. code-block:: bash
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
99
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
100 % svn co http://llvm.org/svn/llvm-project/test-suite/trunk test-suite
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
101
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
102 This will get the test suite into ``llvm/projects/test-suite``.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
103
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
104 #. Configure and build ``llvm``.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
105
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
106 #. Configure and build ``llvm-gcc``.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
107
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
108 #. Install ``llvm-gcc`` somewhere.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
109
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
110 #. *Re-configure* ``llvm`` from the top level of each build tree (LLVM
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
111 object directory tree) in which you want to run the test suite, just
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
112 as you do before building LLVM.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
113
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
114 During the *re-configuration*, you must either: (1) have ``llvm-gcc``
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
115 you just built in your path, or (2) specify the directory where your
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
116 just-built ``llvm-gcc`` is installed using
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
117 ``--with-llvmgccdir=$LLVM_GCC_DIR``.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
118
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
119 You must also tell the configure machinery that the test suite is
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
120 available so it can be configured for your build tree:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
121
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
122 .. code-block:: bash
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
123
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
124 % cd $LLVM_OBJ_ROOT ; $LLVM_SRC_ROOT/configure [--with-llvmgccdir=$LLVM_GCC_DIR]
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
125
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
126 [Remember that ``$LLVM_GCC_DIR`` is the directory where you
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
127 *installed* llvm-gcc, not its src or obj directory.]
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
128
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
129 #. You can now run the test suite from your build tree as follows:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
130
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
131 .. code-block:: bash
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
132
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
133 % cd $LLVM_OBJ_ROOT/projects/test-suite
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
134 % make
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
135
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
136 Note that the second and third steps only need to be done once. After
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
137 you have the suite checked out and configured, you don't need to do it
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
138 again (unless the test code or configure script changes).
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
139
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
140 Configuring External Tests
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
141 --------------------------
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
142
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
143 In order to run the External tests in the ``test-suite`` module, you
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
144 must specify *--with-externals*. This must be done during the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
145 *re-configuration* step (see above), and the ``llvm`` re-configuration
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
146 must recognize the previously-built ``llvm-gcc``. If any of these is
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
147 missing or neglected, the External tests won't work.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
148
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
149 * *--with-externals*
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
150
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
151 * *--with-externals=<directory>*
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
152
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
153 This tells LLVM where to find any external tests. They are expected to
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
154 be in specifically named subdirectories of <``directory``>. If
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
155 ``directory`` is left unspecified, ``configure`` uses the default value
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
156 ``/home/vadve/shared/benchmarks/speccpu2000/benchspec``. Subdirectory
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
157 names known to LLVM include:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
158
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
159 * spec95
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
160
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
161 * speccpu2000
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
162
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
163 * speccpu2006
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
164
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
165 * povray31
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
166
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
167 Others are added from time to time, and can be determined from
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
168 ``configure``.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
169
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
170 Running different tests
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
171 -----------------------
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
172
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
173 In addition to the regular "whole program" tests, the ``test-suite``
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
174 module also provides a mechanism for compiling the programs in different
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
175 ways. If the variable TEST is defined on the ``gmake`` command line, the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
176 test system will include a Makefile named
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
177 ``TEST.<value of TEST variable>.Makefile``. This Makefile can modify
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
178 build rules to yield different results.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
179
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
180 For example, the LLVM nightly tester uses ``TEST.nightly.Makefile`` to
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
181 create the nightly test reports. To run the nightly tests, run
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
182 ``gmake TEST=nightly``.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
183
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
184 There are several TEST Makefiles available in the tree. Some of them are
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
185 designed for internal LLVM research and will not work outside of the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
186 LLVM research group. They may still be valuable, however, as a guide to
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
187 writing your own TEST Makefile for any optimization or analysis passes
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
188 that you develop with LLVM.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
189
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
190 Generating test output
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
191 ----------------------
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
192
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
193 There are a number of ways to run the tests and generate output. The
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
194 most simple one is simply running ``gmake`` with no arguments. This will
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
195 compile and run all programs in the tree using a number of different
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
196 methods and compare results. Any failures are reported in the output,
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
197 but are likely drowned in the other output. Passes are not reported
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
198 explicitly.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
199
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
200 Somewhat better is running ``gmake TEST=sometest test``, which runs the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
201 specified test and usually adds per-program summaries to the output
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
202 (depending on which sometest you use). For example, the ``nightly`` test
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
203 explicitly outputs TEST-PASS or TEST-FAIL for every test after each
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
204 program. Though these lines are still drowned in the output, it's easy
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
205 to grep the output logs in the Output directories.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
206
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
207 Even better are the ``report`` and ``report.format`` targets (where
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
208 ``format`` is one of ``html``, ``csv``, ``text`` or ``graphs``). The
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
209 exact contents of the report are dependent on which ``TEST`` you are
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
210 running, but the text results are always shown at the end of the run and
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
211 the results are always stored in the ``report.<type>.format`` file (when
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
212 running with ``TEST=<type>``). The ``report`` also generate a file
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
213 called ``report.<type>.raw.out`` containing the output of the entire
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
214 test run.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
215
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
216 Writing custom tests for the test suite
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
217 ---------------------------------------
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
218
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
219 Assuming you can run the test suite, (e.g.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
220 "``gmake TEST=nightly report``" should work), it is really easy to run
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
221 optimizations or code generator components against every program in the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
222 tree, collecting statistics or running custom checks for correctness. At
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
223 base, this is how the nightly tester works, it's just one example of a
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
224 general framework.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
225
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
226 Lets say that you have an LLVM optimization pass, and you want to see
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
227 how many times it triggers. First thing you should do is add an LLVM
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
228 `statistic <ProgrammersManual.html#Statistic>`_ to your pass, which will
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
229 tally counts of things you care about.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
230
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
231 Following this, you can set up a test and a report that collects these
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
232 and formats them for easy viewing. This consists of two files, a
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
233 "``test-suite/TEST.XXX.Makefile``" fragment (where XXX is the name of
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
234 your test) and a "``test-suite/TEST.XXX.report``" file that indicates
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
235 how to format the output into a table. There are many example reports of
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
236 various levels of sophistication included with the test suite, and the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
237 framework is very general.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
238
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
239 If you are interested in testing an optimization pass, check out the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
240 "libcalls" test as an example. It can be run like this:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
241
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
242 .. code-block:: bash
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
243
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
244 % cd llvm/projects/test-suite/MultiSource/Benchmarks # or some other level
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
245 % make TEST=libcalls report
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
246
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
247 This will do a bunch of stuff, then eventually print a table like this:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
248
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
249 ::
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
250
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
251 Name | total | #exit |
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
252 ...
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
253 FreeBench/analyzer/analyzer | 51 | 6 |
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
254 FreeBench/fourinarow/fourinarow | 1 | 1 |
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
255 FreeBench/neural/neural | 19 | 9 |
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
256 FreeBench/pifft/pifft | 5 | 3 |
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
257 MallocBench/cfrac/cfrac | 1 | * |
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
258 MallocBench/espresso/espresso | 52 | 12 |
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
259 MallocBench/gs/gs | 4 | * |
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
260 Prolangs-C/TimberWolfMC/timberwolfmc | 302 | * |
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
261 Prolangs-C/agrep/agrep | 33 | 12 |
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
262 Prolangs-C/allroots/allroots | * | * |
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
263 Prolangs-C/assembler/assembler | 47 | * |
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
264 Prolangs-C/bison/mybison | 74 | * |
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
265 ...
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
266
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
267 This basically is grepping the -stats output and displaying it in a
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
268 table. You can also use the "TEST=libcalls report.html" target to get
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
269 the table in HTML form, similarly for report.csv and report.tex.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
270
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
271 The source for this is in ``test-suite/TEST.libcalls.*``. The format is
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
272 pretty simple: the Makefile indicates how to run the test (in this case,
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
273 "``opt -simplify-libcalls -stats``"), and the report contains one line
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
274 for each column of the output. The first value is the header for the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
275 column and the second is the regex to grep the output of the command
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
276 for. There are lots of example reports that can do fancy stuff.