0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
1 =================================
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
2 LLVM Testing Infrastructure Guide
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
3 =================================
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
4
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
5 .. contents::
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
6 :local:
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
7
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
8 .. toctree::
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
9 :hidden:
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
10
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
11 TestSuiteMakefileGuide
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
12
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
13 Overview
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
14 ========
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
15
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
16 This document is the reference manual for the LLVM testing
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
17 infrastructure. It documents the structure of the LLVM testing
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
18 infrastructure, the tools needed to use it, and how to add and run
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
19 tests.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
20
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
21 Requirements
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
22 ============
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
23
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
24 In order to use the LLVM testing infrastructure, you will need all of the
|
|
83
|
25 software required to build LLVM, as well as `Python <http://python.org>`_ 2.7 or
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
26 later.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
27
|
|
120
|
28 If you intend to run the :ref:`test-suite <test-suite-overview>`, you will also
|
|
|
29 need a development version of zlib (zlib1g-dev is known to work on several Linux
|
|
|
30 distributions).
|
|
|
31
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
32 LLVM testing infrastructure organization
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
33 ========================================
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
34
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
35 The LLVM testing infrastructure contains two major categories of tests:
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
36 regression tests and whole programs. The regression tests are contained
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
37 inside the LLVM repository itself under ``llvm/test`` and are expected
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
38 to always pass -- they should be run before every commit.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
39
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
40 The whole programs tests are referred to as the "LLVM test suite" (or
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
41 "test-suite") and are in the ``test-suite`` module in subversion. For
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
42 historical reasons, these tests are also referred to as the "nightly
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
43 tests" in places, which is less ambiguous than "test-suite" and remains
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
44 in use although we run them much more often than nightly.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
45
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
46 Regression tests
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
47 ----------------
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
48
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
49 The regression tests are small pieces of code that test a specific
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
50 feature of LLVM or trigger a specific bug in LLVM. The language they are
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
51 written in depends on the part of LLVM being tested. These tests are driven by
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
52 the :doc:`Lit <CommandGuide/lit>` testing tool (which is part of LLVM), and
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
53 are located in the ``llvm/test`` directory.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
54
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
55 Typically when a bug is found in LLVM, a regression test containing just
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
56 enough code to reproduce the problem should be written and placed
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
57 somewhere underneath this directory. For example, it can be a small
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
58 piece of LLVM IR distilled from an actual application or benchmark.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
59
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
60 ``test-suite``
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
61 --------------
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
62
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
63 The test suite contains whole programs, which are pieces of code which
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
64 can be compiled and linked into a stand-alone program that can be
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
65 executed. These programs are generally written in high level languages
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
66 such as C or C++.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
67
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
68 These programs are compiled using a user specified compiler and set of
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
69 flags, and then executed to capture the program output and timing
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
70 information. The output of these programs is compared to a reference
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
71 output to ensure that the program is being compiled correctly.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
72
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
73 In addition to compiling and executing programs, whole program tests
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
74 serve as a way of benchmarking LLVM performance, both in terms of the
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
75 efficiency of the programs generated as well as the speed with which
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
76 LLVM compiles, optimizes, and generates code.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
77
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
78 The test-suite is located in the ``test-suite`` Subversion module.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
79
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
80 Debugging Information tests
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
81 ---------------------------
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
82
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
83 The test suite contains tests to check quality of debugging information.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
84 The test are written in C based languages or in LLVM assembly language.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
85
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
86 These tests are compiled and run under a debugger. The debugger output
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
87 is checked to validate of debugging information. See README.txt in the
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
88 test suite for more information . This test suite is located in the
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
89 ``debuginfo-tests`` Subversion module.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
90
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
91 Quick start
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
92 ===========
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
93
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
94 The tests are located in two separate Subversion modules. The
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
95 regressions tests are in the main "llvm" module under the directory
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
96 ``llvm/test`` (so you get these tests for free with the main LLVM tree).
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
97 Use ``make check-all`` to run the regression tests after building LLVM.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
98
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
99 The more comprehensive test suite that includes whole programs in C and C++
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
100 is in the ``test-suite`` module. See :ref:`test-suite Quickstart
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
101 <test-suite-quickstart>` for more information on running these tests.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
102
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
103 Regression tests
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
104 ----------------
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
105
|
|
120
|
106 To run all of the LLVM regression tests use the check-llvm target:
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
107
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
108 .. code-block:: bash
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
109
|
|
120
|
110 % make check-llvm
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
111
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
112 If you have `Clang <http://clang.llvm.org/>`_ checked out and built, you
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
113 can run the LLVM and Clang tests simultaneously using:
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
114
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
115 .. code-block:: bash
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
116
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
117 % make check-all
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
118
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
119 To run the tests with Valgrind (Memcheck by default), use the ``LIT_ARGS`` make
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
120 variable to pass the required options to lit. For example, you can use:
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
121
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
122 .. code-block:: bash
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
123
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
124 % make check LIT_ARGS="-v --vg --vg-leak"
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
125
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
126 to enable testing with valgrind and with leak checking enabled.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
127
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
128 To run individual tests or subsets of tests, you can use the ``llvm-lit``
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
129 script which is built as part of LLVM. For example, to run the
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
130 ``Integer/BitPacked.ll`` test by itself you can run:
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
131
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
132 .. code-block:: bash
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
133
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
134 % llvm-lit ~/llvm/test/Integer/BitPacked.ll
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
135
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
136 or to run all of the ARM CodeGen tests:
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
137
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
138 .. code-block:: bash
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
139
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
140 % llvm-lit ~/llvm/test/CodeGen/ARM
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
141
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
142 For more information on using the :program:`lit` tool, see ``llvm-lit --help``
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
143 or the :doc:`lit man page <CommandGuide/lit>`.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
144
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
145 Debugging Information tests
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
146 ---------------------------
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
147
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
148 To run debugging information tests simply checkout the tests inside
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
149 clang/test directory.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
150
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
151 .. code-block:: bash
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
152
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
153 % cd clang/test
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
154 % svn co http://llvm.org/svn/llvm-project/debuginfo-tests/trunk debuginfo-tests
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
155
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
156 These tests are already set up to run as part of clang regression tests.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
157
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
158 Regression test structure
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
159 =========================
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
160
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
161 The LLVM regression tests are driven by :program:`lit` and are located in the
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
162 ``llvm/test`` directory.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
163
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
164 This directory contains a large array of small tests that exercise
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
165 various features of LLVM and to ensure that regressions do not occur.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
166 The directory is broken into several sub-directories, each focused on a
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
167 particular area of LLVM.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
168
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
169 Writing new regression tests
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
170 ----------------------------
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
171
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
172 The regression test structure is very simple, but does require some
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
173 information to be set. This information is gathered via ``configure``
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
174 and is written to a file, ``test/lit.site.cfg`` in the build directory.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
175 The ``llvm/test`` Makefile does this work for you.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
176
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
177 In order for the regression tests to work, each directory of tests must
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
178 have a ``lit.local.cfg`` file. :program:`lit` looks for this file to determine
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
179 how to run the tests. This file is just Python code and thus is very
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
180 flexible, but we've standardized it for the LLVM regression tests. If
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
181 you're adding a directory of tests, just copy ``lit.local.cfg`` from
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
182 another directory to get running. The standard ``lit.local.cfg`` simply
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
183 specifies which files to look in for tests. Any directory that contains
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
184 only directories does not need the ``lit.local.cfg`` file. Read the :doc:`Lit
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
185 documentation <CommandGuide/lit>` for more information.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
186
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
187 Each test file must contain lines starting with "RUN:" that tell :program:`lit`
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
188 how to run it. If there are no RUN lines, :program:`lit` will issue an error
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
189 while running a test.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
190
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
191 RUN lines are specified in the comments of the test program using the
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
192 keyword ``RUN`` followed by a colon, and lastly the command (pipeline)
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
193 to execute. Together, these lines form the "script" that :program:`lit`
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
194 executes to run the test case. The syntax of the RUN lines is similar to a
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
195 shell's syntax for pipelines including I/O redirection and variable
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
196 substitution. However, even though these lines may *look* like a shell
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
197 script, they are not. RUN lines are interpreted by :program:`lit`.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
198 Consequently, the syntax differs from shell in a few ways. You can specify
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
199 as many RUN lines as needed.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
200
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
201 :program:`lit` performs substitution on each RUN line to replace LLVM tool names
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
202 with the full paths to the executable built for each tool (in
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
203 ``$(LLVM_OBJ_ROOT)/$(BuildMode)/bin)``. This ensures that :program:`lit` does
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
204 not invoke any stray LLVM tools in the user's path during testing.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
205
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
206 Each RUN line is executed on its own, distinct from other lines unless
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
207 its last character is ``\``. This continuation character causes the RUN
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
208 line to be concatenated with the next one. In this way you can build up
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
209 long pipelines of commands without making huge line lengths. The lines
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
210 ending in ``\`` are concatenated until a RUN line that doesn't end in
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
211 ``\`` is found. This concatenated set of RUN lines then constitutes one
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
212 execution. :program:`lit` will substitute variables and arrange for the pipeline
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
213 to be executed. If any process in the pipeline fails, the entire line (and
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
214 test case) fails too.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
215
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
216 Below is an example of legal RUN lines in a ``.ll`` file:
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
217
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
218 .. code-block:: llvm
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
219
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
220 ; RUN: llvm-as < %s | llvm-dis > %t1
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
221 ; RUN: llvm-dis < %s.bc-13 > %t2
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
222 ; RUN: diff %t1 %t2
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
223
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
224 As with a Unix shell, the RUN lines permit pipelines and I/O
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
225 redirection to be used.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
226
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
227 There are some quoting rules that you must pay attention to when writing
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
228 your RUN lines. In general nothing needs to be quoted. :program:`lit` won't
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
229 strip off any quote characters so they will get passed to the invoked program.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
230 To avoid this use curly braces to tell :program:`lit` that it should treat
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
231 everything enclosed as one value.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
232
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
233 In general, you should strive to keep your RUN lines as simple as possible,
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
234 using them only to run tools that generate textual output you can then examine.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
235 The recommended way to examine output to figure out if the test passes is using
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
236 the :doc:`FileCheck tool <CommandGuide/FileCheck>`. *[The usage of grep in RUN
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
237 lines is deprecated - please do not send or commit patches that use it.]*
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
238
|
|
100
|
239 Put related tests into a single file rather than having a separate file per
|
|
|
240 test. Check if there are files already covering your feature and consider
|
|
|
241 adding your code there instead of creating a new file.
|
|
|
242
|
|
83
|
243 Extra files
|
|
|
244 -----------
|
|
|
245
|
|
|
246 If your test requires extra files besides the file containing the ``RUN:``
|
|
|
247 lines, the idiomatic place to put them is in a subdirectory ``Inputs``.
|
|
|
248 You can then refer to the extra files as ``%S/Inputs/foo.bar``.
|
|
|
249
|
|
|
250 For example, consider ``test/Linker/ident.ll``. The directory structure is
|
|
|
251 as follows::
|
|
|
252
|
|
|
253 test/
|
|
|
254 Linker/
|
|
|
255 ident.ll
|
|
|
256 Inputs/
|
|
|
257 ident.a.ll
|
|
|
258 ident.b.ll
|
|
|
259
|
|
|
260 For convenience, these are the contents:
|
|
|
261
|
|
|
262 .. code-block:: llvm
|
|
|
263
|
|
|
264 ;;;;; ident.ll:
|
|
|
265
|
|
|
266 ; RUN: llvm-link %S/Inputs/ident.a.ll %S/Inputs/ident.b.ll -S | FileCheck %s
|
|
|
267
|
|
|
268 ; Verify that multiple input llvm.ident metadata are linked together.
|
|
|
269
|
|
|
270 ; CHECK-DAG: !llvm.ident = !{!0, !1, !2}
|
|
|
271 ; CHECK-DAG: "Compiler V1"
|
|
|
272 ; CHECK-DAG: "Compiler V2"
|
|
|
273 ; CHECK-DAG: "Compiler V3"
|
|
|
274
|
|
|
275 ;;;;; Inputs/ident.a.ll:
|
|
|
276
|
|
|
277 !llvm.ident = !{!0, !1}
|
|
|
278 !0 = metadata !{metadata !"Compiler V1"}
|
|
|
279 !1 = metadata !{metadata !"Compiler V2"}
|
|
|
280
|
|
|
281 ;;;;; Inputs/ident.b.ll:
|
|
|
282
|
|
|
283 !llvm.ident = !{!0}
|
|
|
284 !0 = metadata !{metadata !"Compiler V3"}
|
|
|
285
|
|
|
286 For symmetry reasons, ``ident.ll`` is just a dummy file that doesn't
|
|
|
287 actually participate in the test besides holding the ``RUN:`` lines.
|
|
|
288
|
|
|
289 .. note::
|
|
|
290
|
|
|
291 Some existing tests use ``RUN: true`` in extra files instead of just
|
|
|
292 putting the extra files in an ``Inputs/`` directory. This pattern is
|
|
|
293 deprecated.
|
|
|
294
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
295 Fragile tests
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
296 -------------
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
297
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
298 It is easy to write a fragile test that would fail spuriously if the tool being
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
299 tested outputs a full path to the input file. For example, :program:`opt` by
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
300 default outputs a ``ModuleID``:
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
301
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
302 .. code-block:: console
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
303
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
304 $ cat example.ll
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
305 define i32 @main() nounwind {
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
306 ret i32 0
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
307 }
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
308
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
309 $ opt -S /path/to/example.ll
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
310 ; ModuleID = '/path/to/example.ll'
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
311
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
312 define i32 @main() nounwind {
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
313 ret i32 0
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
314 }
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
315
|
|
121
|
316 ``ModuleID`` can unexpectedly match against ``CHECK`` lines. For example:
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
317
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
318 .. code-block:: llvm
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
319
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
320 ; RUN: opt -S %s | FileCheck
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
321
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
322 define i32 @main() nounwind {
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
323 ; CHECK-NOT: load
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
324 ret i32 0
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
325 }
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
326
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
327 This test will fail if placed into a ``download`` directory.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
328
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
329 To make your tests robust, always use ``opt ... < %s`` in the RUN line.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
330 :program:`opt` does not output a ``ModuleID`` when input comes from stdin.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
331
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
332 Platform-Specific Tests
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
333 -----------------------
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
334
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
335 Whenever adding tests that require the knowledge of a specific platform,
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
336 either related to code generated, specific output or back-end features,
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
337 you must make sure to isolate the features, so that buildbots that
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
338 run on different architectures (and don't even compile all back-ends),
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
339 don't fail.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
340
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
341 The first problem is to check for target-specific output, for example sizes
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
342 of structures, paths and architecture names, for example:
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
343
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
344 * Tests containing Windows paths will fail on Linux and vice-versa.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
345 * Tests that check for ``x86_64`` somewhere in the text will fail anywhere else.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
346 * Tests where the debug information calculates the size of types and structures.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
347
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
348 Also, if the test rely on any behaviour that is coded in any back-end, it must
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
349 go in its own directory. So, for instance, code generator tests for ARM go
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
350 into ``test/CodeGen/ARM`` and so on. Those directories contain a special
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
351 ``lit`` configuration file that ensure all tests in that directory will
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
352 only run if a specific back-end is compiled and available.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
353
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
354 For instance, on ``test/CodeGen/ARM``, the ``lit.local.cfg`` is:
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
355
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
356 .. code-block:: python
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
357
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
358 config.suffixes = ['.ll', '.c', '.cpp', '.test']
|
|
77
|
359 if not 'ARM' in config.root.targets:
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
360 config.unsupported = True
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
361
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
362 Other platform-specific tests are those that depend on a specific feature
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
363 of a specific sub-architecture, for example only to Intel chips that support ``AVX2``.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
364
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
365 For instance, ``test/CodeGen/X86/psubus.ll`` tests three sub-architecture
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
366 variants:
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
367
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
368 .. code-block:: llvm
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
369
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
370 ; RUN: llc -mcpu=core2 < %s | FileCheck %s -check-prefix=SSE2
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
371 ; RUN: llc -mcpu=corei7-avx < %s | FileCheck %s -check-prefix=AVX1
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
372 ; RUN: llc -mcpu=core-avx2 < %s | FileCheck %s -check-prefix=AVX2
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
373
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
374 And the checks are different:
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
375
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
376 .. code-block:: llvm
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
377
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
378 ; SSE2: @test1
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
379 ; SSE2: psubusw LCPI0_0(%rip), %xmm0
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
380 ; AVX1: @test1
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
381 ; AVX1: vpsubusw LCPI0_0(%rip), %xmm0, %xmm0
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
382 ; AVX2: @test1
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
383 ; AVX2: vpsubusw LCPI0_0(%rip), %xmm0, %xmm0
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
384
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
385 So, if you're testing for a behaviour that you know is platform-specific or
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
386 depends on special features of sub-architectures, you must add the specific
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
387 triple, test with the specific FileCheck and put it into the specific
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
388 directory that will filter out all other architectures.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
389
|
|
121
|
390
|
|
|
391 Constraining test execution
|
|
|
392 ---------------------------
|
|
120
|
393
|
|
121
|
394 Some tests can be run only in specific configurations, such as
|
|
|
395 with debug builds or on particular platforms. Use ``REQUIRES``
|
|
|
396 and ``UNSUPPORTED`` to control when the test is enabled.
|
|
|
397
|
|
|
398 Some tests are expected to fail. For example, there may be a known bug
|
|
|
399 that the test detect. Use ``XFAIL`` to mark a test as an expected failure.
|
|
|
400 An ``XFAIL`` test will be successful if its execution fails, and
|
|
|
401 will be a failure if its execution succeeds.
|
|
120
|
402
|
|
|
403 .. code-block:: llvm
|
|
|
404
|
|
121
|
405 ; This test will be only enabled in the build with asserts.
|
|
120
|
406 ; REQUIRES: asserts
|
|
121
|
407 ; This test is disabled on Linux.
|
|
|
408 ; UNSUPPORTED: -linux-
|
|
|
409 ; This test is expected to fail on PowerPC.
|
|
|
410 ; XFAIL: powerpc
|
|
|
411
|
|
|
412 ``REQUIRES`` and ``UNSUPPORTED`` and ``XFAIL`` all accept a comma-separated
|
|
|
413 list of boolean expressions. The values in each expression may be:
|
|
|
414
|
|
|
415 - Features added to ``config.available_features`` by
|
|
|
416 configuration files such as ``lit.cfg``.
|
|
|
417 - Substrings of the target triple (``UNSUPPORTED`` and ``XFAIL`` only).
|
|
120
|
418
|
|
121
|
419 | ``REQUIRES`` enables the test if all expressions are true.
|
|
|
420 | ``UNSUPPORTED`` disables the test if any expression is true.
|
|
|
421 | ``XFAIL`` expects the test to fail if any expression is true.
|
|
|
422
|
|
|
423 As a special case, ``XFAIL: *`` is expected to fail everywhere.
|
|
120
|
424
|
|
121
|
425 .. code-block:: llvm
|
|
|
426
|
|
|
427 ; This test is disabled on Windows,
|
|
|
428 ; and is disabled on Linux, except for Android Linux.
|
|
|
429 ; UNSUPPORTED: windows, linux && !android
|
|
|
430 ; This test is expected to fail on both PowerPC and ARM.
|
|
|
431 ; XFAIL: powerpc || arm
|
|
|
432
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
433
|
|
77
|
434 Substitutions
|
|
|
435 -------------
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
436
|
|
77
|
437 Besides replacing LLVM tool names the following substitutions are performed in
|
|
|
438 RUN lines:
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
439
|
|
77
|
440 ``%%``
|
|
|
441 Replaced by a single ``%``. This allows escaping other substitutions.
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
442
|
|
77
|
443 ``%s``
|
|
|
444 File path to the test case's source. This is suitable for passing on the
|
|
|
445 command line as the input to an LLVM tool.
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
446
|
|
77
|
447 Example: ``/home/user/llvm/test/MC/ELF/foo_test.s``
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
448
|
|
77
|
449 ``%S``
|
|
|
450 Directory path to the test case's source.
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
451
|
|
77
|
452 Example: ``/home/user/llvm/test/MC/ELF``
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
453
|
|
77
|
454 ``%t``
|
|
|
455 File path to a temporary file name that could be used for this test case.
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
456 The file name won't conflict with other test cases. You can append to it
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
457 if you need multiple temporaries. This is useful as the destination of
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
458 some redirected output.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
459
|
|
77
|
460 Example: ``/home/user/llvm.build/test/MC/ELF/Output/foo_test.s.tmp``
|
|
|
461
|
|
|
462 ``%T``
|
|
|
463 Directory of ``%t``.
|
|
|
464
|
|
|
465 Example: ``/home/user/llvm.build/test/MC/ELF/Output``
|
|
|
466
|
|
|
467 ``%{pathsep}``
|
|
|
468
|
|
|
469 Expands to the path separator, i.e. ``:`` (or ``;`` on Windows).
|
|
|
470
|
|
121
|
471 ``%/s, %/S, %/t, %/T:``
|
|
|
472
|
|
|
473 Act like the corresponding substitution above but replace any ``\``
|
|
|
474 character with a ``/``. This is useful to normalize path separators.
|
|
|
475
|
|
|
476 Example: ``%s: C:\Desktop Files/foo_test.s.tmp``
|
|
|
477
|
|
|
478 Example: ``%/s: C:/Desktop Files/foo_test.s.tmp``
|
|
|
479
|
|
|
480 ``%:s, %:S, %:t, %:T:``
|
|
|
481
|
|
|
482 Act like the corresponding substitution above but remove colons at
|
|
|
483 the beginning of Windows paths. This is useful to allow concatenation
|
|
|
484 of absolute paths on Windows to produce a legal path.
|
|
|
485
|
|
|
486 Example: ``%s: C:\Desktop Files\foo_test.s.tmp``
|
|
|
487
|
|
|
488 Example: ``%:s: C\Desktop Files\foo_test.s.tmp``
|
|
|
489
|
|
77
|
490
|
|
|
491 **LLVM-specific substitutions:**
|
|
|
492
|
|
|
493 ``%shlibext``
|
|
|
494 The suffix for the host platforms shared library files. This includes the
|
|
|
495 period as the first character.
|
|
|
496
|
|
|
497 Example: ``.so`` (Linux), ``.dylib`` (OS X), ``.dll`` (Windows)
|
|
|
498
|
|
|
499 ``%exeext``
|
|
|
500 The suffix for the host platforms executable files. This includes the
|
|
|
501 period as the first character.
|
|
|
502
|
|
|
503 Example: ``.exe`` (Windows), empty on Linux.
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
504
|
|
77
|
505 ``%(line)``, ``%(line+<number>)``, ``%(line-<number>)``
|
|
|
506 The number of the line where this substitution is used, with an optional
|
|
|
507 integer offset. This can be used in tests with multiple RUN lines, which
|
|
|
508 reference test file's line numbers.
|
|
|
509
|
|
|
510
|
|
|
511 **Clang-specific substitutions:**
|
|
|
512
|
|
|
513 ``%clang``
|
|
|
514 Invokes the Clang driver.
|
|
|
515
|
|
|
516 ``%clang_cpp``
|
|
|
517 Invokes the Clang driver for C++.
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
518
|
|
77
|
519 ``%clang_cl``
|
|
|
520 Invokes the CL-compatible Clang driver.
|
|
|
521
|
|
|
522 ``%clangxx``
|
|
|
523 Invokes the G++-compatible Clang driver.
|
|
|
524
|
|
|
525 ``%clang_cc1``
|
|
|
526 Invokes the Clang frontend.
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
527
|
|
77
|
528 ``%itanium_abi_triple``, ``%ms_abi_triple``
|
|
|
529 These substitutions can be used to get the current target triple adjusted to
|
|
|
530 the desired ABI. For example, if the test suite is running with the
|
|
|
531 ``i686-pc-win32`` target, ``%itanium_abi_triple`` will expand to
|
|
|
532 ``i686-pc-mingw32``. This allows a test to run with a specific ABI without
|
|
|
533 constraining it to a specific triple.
|
|
|
534
|
|
|
535 To add more substituations, look at ``test/lit.cfg`` or ``lit.local.cfg``.
|
|
|
536
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
537
|
|
95
|
538 Options
|
|
|
539 -------
|
|
|
540
|
|
|
541 The llvm lit configuration allows to customize some things with user options:
|
|
|
542
|
|
|
543 ``llc``, ``opt``, ...
|
|
|
544 Substitute the respective llvm tool name with a custom command line. This
|
|
|
545 allows to specify custom paths and default arguments for these tools.
|
|
|
546 Example:
|
|
|
547
|
|
|
548 % llvm-lit "-Dllc=llc -verify-machineinstrs"
|
|
|
549
|
|
|
550 ``run_long_tests``
|
|
|
551 Enable the execution of long running tests.
|
|
|
552
|
|
|
553 ``llvm_site_config``
|
|
|
554 Load the specified lit configuration instead of the default one.
|
|
|
555
|
|
|
556
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
557 Other Features
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
558 --------------
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
559
|
|
77
|
560 To make RUN line writing easier, there are several helper programs. These
|
|
|
561 helpers are in the PATH when running tests, so you can just call them using
|
|
|
562 their name. For example:
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
563
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
564 ``not``
|
|
77
|
565 This program runs its arguments and then inverts the result code from it.
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
566 Zero result codes become 1. Non-zero result codes become 0.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
567
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
568 To make the output more useful, :program:`lit` will scan
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
569 the lines of the test case for ones that contain a pattern that matches
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
570 ``PR[0-9]+``. This is the syntax for specifying a PR (Problem Report) number
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
571 that is related to the test case. The number after "PR" specifies the
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
572 LLVM bugzilla number. When a PR number is specified, it will be used in
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
573 the pass/fail reporting. This is useful to quickly get some context when
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
574 a test fails.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
575
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
576 Finally, any line that contains "END." will cause the special
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
577 interpretation of lines to terminate. This is generally done right after
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
578 the last RUN: line. This has two side effects:
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
579
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
580 (a) it prevents special interpretation of lines that are part of the test
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
581 program, not the instructions to the test case, and
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
582
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
583 (b) it speeds things up for really big test cases by avoiding
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
584 interpretation of the remainder of the file.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
585
|
|
120
|
586 .. _test-suite-overview:
|
|
|
587
|
0
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
588 ``test-suite`` Overview
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
589 =======================
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
590
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
591 The ``test-suite`` module contains a number of programs that can be
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
592 compiled and executed. The ``test-suite`` includes reference outputs for
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
593 all of the programs, so that the output of the executed program can be
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
594 checked for correctness.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
595
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
596 ``test-suite`` tests are divided into three types of tests: MultiSource,
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
597 SingleSource, and External.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
598
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
599 - ``test-suite/SingleSource``
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
600
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
601 The SingleSource directory contains test programs that are only a
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
602 single source file in size. These are usually small benchmark
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
603 programs or small programs that calculate a particular value. Several
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
604 such programs are grouped together in each directory.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
605
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
606 - ``test-suite/MultiSource``
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
607
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
608 The MultiSource directory contains subdirectories which contain
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
609 entire programs with multiple source files. Large benchmarks and
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
610 whole applications go here.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
611
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
612 - ``test-suite/External``
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
613
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
614 The External directory contains Makefiles for building code that is
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
615 external to (i.e., not distributed with) LLVM. The most prominent
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
616 members of this directory are the SPEC 95 and SPEC 2000 benchmark
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
617 suites. The ``External`` directory does not contain these actual
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
618 tests, but only the Makefiles that know how to properly compile these
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
619 programs from somewhere else. When using ``LNT``, use the
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
620 ``--test-externals`` option to include these tests in the results.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
621
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
622 .. _test-suite-quickstart:
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
623
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
624 ``test-suite`` Quickstart
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
625 -------------------------
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
626
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
627 The modern way of running the ``test-suite`` is focused on testing and
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
628 benchmarking complete compilers using the
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
629 `LNT <http://llvm.org/docs/lnt>`_ testing infrastructure.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
630
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
631 For more information on using LNT to execute the ``test-suite``, please
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
632 see the `LNT Quickstart <http://llvm.org/docs/lnt/quickstart.html>`_
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
633 documentation.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
634
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
635 ``test-suite`` Makefiles
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
636 ------------------------
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
637
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
638 Historically, the ``test-suite`` was executed using a complicated setup
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
639 of Makefiles. The LNT based approach above is recommended for most
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
640 users, but there are some testing scenarios which are not supported by
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
641 the LNT approach. In addition, LNT currently uses the Makefile setup
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
642 under the covers and so developers who are interested in how LNT works
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
643 under the hood may want to understand the Makefile based setup.
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
644
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
645 For more information on the ``test-suite`` Makefile setup, please see
|
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
646 the :doc:`Test Suite Makefile Guide <TestSuiteMakefileGuide>`.
|
