comparison docs/TestingGuide.rst @ 77:54457678186b LLVM3.6

LLVM 3.6
author Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
date Mon, 08 Sep 2014 22:06:00 +0900
parents 95c75e76d11b
children 60c9769439b8
comparison
equal deleted inserted replaced
34:e874dbf0ad9d 77:54457678186b
302 For instance, on ``test/CodeGen/ARM``, the ``lit.local.cfg`` is: 302 For instance, on ``test/CodeGen/ARM``, the ``lit.local.cfg`` is:
303 303
304 .. code-block:: python 304 .. code-block:: python
305 305
306 config.suffixes = ['.ll', '.c', '.cpp', '.test'] 306 config.suffixes = ['.ll', '.c', '.cpp', '.test']
307 targets = set(config.root.targets_to_build.split()) 307 if not 'ARM' in config.root.targets:
308 if not 'ARM' in targets:
309 config.unsupported = True 308 config.unsupported = True
310 309
311 Other platform-specific tests are those that depend on a specific feature 310 Other platform-specific tests are those that depend on a specific feature
312 of a specific sub-architecture, for example only to Intel chips that support ``AVX2``. 311 of a specific sub-architecture, for example only to Intel chips that support ``AVX2``.
313 312
335 depends on special features of sub-architectures, you must add the specific 334 depends on special features of sub-architectures, you must add the specific
336 triple, test with the specific FileCheck and put it into the specific 335 triple, test with the specific FileCheck and put it into the specific
337 directory that will filter out all other architectures. 336 directory that will filter out all other architectures.
338 337
339 338
340 Variables and substitutions 339 Substitutions
341 --------------------------- 340 -------------
342 341
343 With a RUN line there are a number of substitutions that are permitted. 342 Besides replacing LLVM tool names the following substitutions are performed in
344 To make a substitution just write the variable's name preceded by a ``$``. 343 RUN lines:
345 Additionally, for compatibility reasons with previous versions of the 344
346 test library, certain names can be accessed with an alternate syntax: a 345 ``%%``
347 % prefix. These alternates are deprecated and may go away in a future 346 Replaced by a single ``%``. This allows escaping other substitutions.
348 version. 347
349 348 ``%s``
350 Here are the available variable names. The alternate syntax is listed in 349 File path to the test case's source. This is suitable for passing on the
351 parentheses. 350 command line as the input to an LLVM tool.
352 351
353 ``$test`` (``%s``) 352 Example: ``/home/user/llvm/test/MC/ELF/foo_test.s``
354 The full path to the test case's source. This is suitable for passing on 353
355 the command line as the input to an LLVM tool. 354 ``%S``
356 355 Directory path to the test case's source.
357 ``%(line)``, ``%(line+<number>)``, ``%(line-<number>)`` 356
358 The number of the line where this variable is used, with an optional 357 Example: ``/home/user/llvm/test/MC/ELF``
359 integer offset. This can be used in tests with multiple RUN lines, 358
360 which reference test file's line numbers. 359 ``%t``
361 360 File path to a temporary file name that could be used for this test case.
362 ``$srcdir``
363 The source directory from where the ``make check`` was run.
364
365 ``objdir``
366 The object directory that corresponds to the ``$srcdir``.
367
368 ``subdir``
369 A partial path from the ``test`` directory that contains the
370 sub-directory that contains the test source being executed.
371
372 ``srcroot``
373 The root directory of the LLVM src tree.
374
375 ``objroot``
376 The root directory of the LLVM object tree. This could be the same as
377 the srcroot.
378
379 ``path``
380 The path to the directory that contains the test case source. This is
381 for locating any supporting files that are not generated by the test,
382 but used by the test.
383
384 ``tmp``
385 The path to a temporary file name that could be used for this test case.
386 The file name won't conflict with other test cases. You can append to it 361 The file name won't conflict with other test cases. You can append to it
387 if you need multiple temporaries. This is useful as the destination of 362 if you need multiple temporaries. This is useful as the destination of
388 some redirected output. 363 some redirected output.
389 364
390 ``target_triplet`` (``%target_triplet``) 365 Example: ``/home/user/llvm.build/test/MC/ELF/Output/foo_test.s.tmp``
391 The target triplet that corresponds to the current host machine (the one 366
392 running the test cases). This should probably be called "host". 367 ``%T``
393 368 Directory of ``%t``.
394 ``link`` (``%link``) 369
395 This full link command used to link LLVM executables. This has all the 370 Example: ``/home/user/llvm.build/test/MC/ELF/Output``
396 configured ``-I``, ``-L`` and ``-l`` options. 371
397 372 ``%{pathsep}``
398 ``shlibext`` (``%shlibext``) 373
399 The suffix for the host platforms shared library (DLL) files. This 374 Expands to the path separator, i.e. ``:`` (or ``;`` on Windows).
400 includes the period as the first character. 375
401 376
402 To add more variables, look at ``test/lit.cfg``. 377 **LLVM-specific substitutions:**
378
379 ``%shlibext``
380 The suffix for the host platforms shared library files. This includes the
381 period as the first character.
382
383 Example: ``.so`` (Linux), ``.dylib`` (OS X), ``.dll`` (Windows)
384
385 ``%exeext``
386 The suffix for the host platforms executable files. This includes the
387 period as the first character.
388
389 Example: ``.exe`` (Windows), empty on Linux.
390
391 ``%(line)``, ``%(line+<number>)``, ``%(line-<number>)``
392 The number of the line where this substitution is used, with an optional
393 integer offset. This can be used in tests with multiple RUN lines, which
394 reference test file's line numbers.
395
396
397 **Clang-specific substitutions:**
398
399 ``%clang``
400 Invokes the Clang driver.
401
402 ``%clang_cpp``
403 Invokes the Clang driver for C++.
404
405 ``%clang_cl``
406 Invokes the CL-compatible Clang driver.
407
408 ``%clangxx``
409 Invokes the G++-compatible Clang driver.
410
411 ``%clang_cc1``
412 Invokes the Clang frontend.
413
414 ``%itanium_abi_triple``, ``%ms_abi_triple``
415 These substitutions can be used to get the current target triple adjusted to
416 the desired ABI. For example, if the test suite is running with the
417 ``i686-pc-win32`` target, ``%itanium_abi_triple`` will expand to
418 ``i686-pc-mingw32``. This allows a test to run with a specific ABI without
419 constraining it to a specific triple.
420
421 To add more substituations, look at ``test/lit.cfg`` or ``lit.local.cfg``.
422
403 423
404 Other Features 424 Other Features
405 -------------- 425 --------------
406 426
407 To make RUN line writing easier, there are several helper scripts and programs 427 To make RUN line writing easier, there are several helper programs. These
408 in the ``llvm/test/Scripts`` directory. This directory is in the PATH 428 helpers are in the PATH when running tests, so you can just call them using
409 when running tests, so you can just call these scripts using their name. 429 their name. For example:
410 For example: 430
411
412 ``ignore``
413 This script runs its arguments and then always returns 0. This is useful
414 in cases where the test needs to cause a tool to generate an error (e.g.
415 to check the error output). However, any program in a pipeline that
416 returns a non-zero result will cause the test to fail. This script
417 overcomes that issue and nicely documents that the test case is
418 purposefully ignoring the result code of the tool
419 ``not`` 431 ``not``
420 This script runs its arguments and then inverts the result code from it. 432 This program runs its arguments and then inverts the result code from it.
421 Zero result codes become 1. Non-zero result codes become 0. 433 Zero result codes become 1. Non-zero result codes become 0.
422 434
423 Sometimes it is necessary to mark a test case as "expected fail" or 435 Sometimes it is necessary to mark a test case as "expected fail" or
424 XFAIL. You can easily mark a test as XFAIL just by including ``XFAIL:`` 436 XFAIL. You can easily mark a test as XFAIL just by including ``XFAIL:``
425 on a line near the top of the file. This signals that the test case 437 on a line near the top of the file. This signals that the test case