|
150
|
1 ========================================
|
|
|
2 Compiler-rt Testing Infrastructure Guide
|
|
|
3 ========================================
|
|
|
4
|
|
|
5 .. contents::
|
|
|
6 :local:
|
|
|
7
|
|
|
8 Overview
|
|
|
9 ========
|
|
|
10
|
|
|
11 This document is the reference manual for the compiler-rt modifications to the
|
|
|
12 testing infrastructure. Documentation for the infrastructure itself can be found at
|
|
|
13 :ref:`llvm_testing_guide`.
|
|
|
14
|
|
|
15 LLVM testing infrastructure organization
|
|
|
16 ========================================
|
|
|
17
|
|
|
18 The compiler-rt testing infrastructure contains regression tests which are run
|
|
|
19 as part of the usual ``make check-all`` and are expected to always pass -- they
|
|
|
20 should be run before every commit.
|
|
|
21
|
|
|
22 Quick start
|
|
|
23 ===========
|
|
|
24
|
|
|
25 The regressions tests are in the "compiler-rt" module and are normally checked
|
|
|
26 out in the directory ``llvm/projects/compiler-rt/test``. Use ``make check-all``
|
|
|
27 to run the regression tests after building compiler-rt.
|
|
|
28
|
|
|
29 REQUIRES, XFAIL, etc.
|
|
|
30 ---------------------
|
|
|
31
|
|
|
32 Sometimes it is necessary to restrict a test to a specific target or mark it as
|
|
|
33 an "expected fail" or XFAIL. This is normally achieved using ``REQUIRES:`` or
|
|
252
|
34 ``XFAIL:`` and the ``target=<target-triple>`` feature, typically with a regular
|
|
|
35 expression matching an appropriate substring of the triple. Unfortunately, the
|
|
150
|
36 behaviour of this is somewhat quirky in compiler-rt. There are two main
|
|
|
37 pitfalls to avoid.
|
|
|
38
|
|
252
|
39 The first pitfall is that these regular expressions may inadvertently match
|
|
|
40 more triples than expected. For example, ``XFAIL: target=mips{{.*}}`` matches
|
|
|
41 ``mips-linux-gnu``, ``mipsel-linux-gnu``, ``mips64-linux-gnu``, and
|
|
|
42 ``mips64el-linux-gnu``. Including a trailing ``-`` such as in
|
|
|
43 ``XFAIL: target=mips-{{.*}}`` can help to mitigate this quirk but even that has
|
|
|
44 issues as described below.
|
|
150
|
45
|
|
|
46 The second pitfall is that the default target triple is often inappropriate for
|
|
|
47 compiler-rt tests since compiler-rt tests may be compiled for multiple targets.
|
|
|
48 For example, a typical build on an ``x86_64-linux-gnu`` host will often run the
|
|
252
|
49 tests for both x86_64 and i386. In this situation ``XFAIL: target=x86_64{{{.*}}``
|
|
|
50 will mark both the x86_64 and i386 tests as an expected failure while
|
|
|
51 ``XFAIL: target=i386{{.*}}`` will have no effect at all.
|
|
150
|
52
|
|
|
53 To remedy both pitfalls, compiler-rt tests provide a feature string which can
|
|
|
54 be used to specify a single target. This string is of the form
|
|
|
55 ``target-is-${arch}`` where ``${arch}}`` is one of the values from the
|
|
|
56 following lines of the CMake output::
|
|
|
57
|
|
|
58 -- Compiler-RT supported architectures: x86_64;i386
|
|
|
59 -- Builtin supported architectures: i386;x86_64
|
|
|
60
|
|
|
61 So for example ``XFAIL: target-is-x86_64`` will mark a test as expected to fail
|
|
|
62 on x86_64 without also affecting the i386 test and ``XFAIL: target-is-i386``
|
|
|
63 will mark a test as expected to fail on i386 even if the default target triple
|
|
|
64 is ``x86_64-linux-gnu``. Directives that use these ``target-is-${arch}`` string
|
|
|
65 require exact matches so ``XFAIL: target-is-mips``,
|
|
|
66 ``XFAIL: target-is-mipsel``, ``XFAIL: target-is-mips64``, and
|
|
|
67 ``XFAIL: target-is-mips64el`` all refer to different MIPS targets.
|
