Mercurial > hg > CbC > CbC_llvm
diff docs/TestingGuide.rst @ 121:803732b1fca8
LLVM 5.0
author | kono |
---|---|
date | Fri, 27 Oct 2017 17:07:41 +0900 |
parents | 1172e4bd9c6f |
children | c2174574ed3a |
line wrap: on
line diff
--- a/docs/TestingGuide.rst Fri Nov 25 19:14:25 2016 +0900 +++ b/docs/TestingGuide.rst Fri Oct 27 17:07:41 2017 +0900 @@ -313,7 +313,7 @@ ret i32 0 } -``ModuleID`` can unexpetedly match against ``CHECK`` lines. For example: +``ModuleID`` can unexpectedly match against ``CHECK`` lines. For example: .. code-block:: llvm @@ -387,23 +387,49 @@ triple, test with the specific FileCheck and put it into the specific directory that will filter out all other architectures. -REQUIRES and REQUIRES-ANY directive -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Constraining test execution +--------------------------- -Some tests can be enabled only in specific situation - like having -debug build. Use ``REQUIRES`` directive to specify those requirements. +Some tests can be run only in specific configurations, such as +with debug builds or on particular platforms. Use ``REQUIRES`` +and ``UNSUPPORTED`` to control when the test is enabled. + +Some tests are expected to fail. For example, there may be a known bug +that the test detect. Use ``XFAIL`` to mark a test as an expected failure. +An ``XFAIL`` test will be successful if its execution fails, and +will be a failure if its execution succeeds. .. code-block:: llvm - ; This test will be only enabled in the build with asserts + ; This test will be only enabled in the build with asserts. ; REQUIRES: asserts + ; This test is disabled on Linux. + ; UNSUPPORTED: -linux- + ; This test is expected to fail on PowerPC. + ; XFAIL: powerpc + +``REQUIRES`` and ``UNSUPPORTED`` and ``XFAIL`` all accept a comma-separated +list of boolean expressions. The values in each expression may be: + +- Features added to ``config.available_features`` by + configuration files such as ``lit.cfg``. +- Substrings of the target triple (``UNSUPPORTED`` and ``XFAIL`` only). -You can separate requirements by a comma. -``REQUIRES`` means all listed requirements must be satisfied. -``REQUIRES-ANY`` means at least one must be satisfied. +| ``REQUIRES`` enables the test if all expressions are true. +| ``UNSUPPORTED`` disables the test if any expression is true. +| ``XFAIL`` expects the test to fail if any expression is true. + +As a special case, ``XFAIL: *`` is expected to fail everywhere. -List of features that can be used in ``REQUIRES`` and ``REQUIRES-ANY`` can be -found in lit.cfg files. +.. code-block:: llvm + + ; This test is disabled on Windows, + ; and is disabled on Linux, except for Android Linux. + ; UNSUPPORTED: windows, linux && !android + ; This test is expected to fail on both PowerPC and ARM. + ; XFAIL: powerpc || arm + Substitutions ------------- @@ -442,6 +468,25 @@ Expands to the path separator, i.e. ``:`` (or ``;`` on Windows). +``%/s, %/S, %/t, %/T:`` + + Act like the corresponding substitution above but replace any ``\`` + character with a ``/``. This is useful to normalize path separators. + + Example: ``%s: C:\Desktop Files/foo_test.s.tmp`` + + Example: ``%/s: C:/Desktop Files/foo_test.s.tmp`` + +``%:s, %:S, %:t, %:T:`` + + Act like the corresponding substitution above but remove colons at + the beginning of Windows paths. This is useful to allow concatenation + of absolute paths on Windows to produce a legal path. + + Example: ``%s: C:\Desktop Files\foo_test.s.tmp`` + + Example: ``%:s: C\Desktop Files\foo_test.s.tmp`` + **LLVM-specific substitutions:** @@ -520,24 +565,6 @@ This program runs its arguments and then inverts the result code from it. Zero result codes become 1. Non-zero result codes become 0. -Sometimes it is necessary to mark a test case as "expected fail" or -XFAIL. You can easily mark a test as XFAIL just by including ``XFAIL:`` -on a line near the top of the file. This signals that the test case -should succeed if the test fails. Such test cases are counted separately -by the testing tool. To specify an expected fail, use the XFAIL keyword -in the comments of the test program followed by a colon and one or more -failure patterns. Each failure pattern can be either ``*`` (to specify -fail everywhere), or a part of a target triple (indicating the test -should fail on that platform), or the name of a configurable feature -(for example, ``loadable_module``). If there is a match, the test is -expected to fail. If not, the test is expected to succeed. To XFAIL -everywhere just specify ``XFAIL: *``. Here is an example of an ``XFAIL`` -line: - -.. code-block:: llvm - - ; XFAIL: darwin,sun - To make the output more useful, :program:`lit` will scan the lines of the test case for ones that contain a pattern that matches ``PR[0-9]+``. This is the syntax for specifying a PR (Problem Report) number