Mercurial > hg > CbC > CbC_llvm
diff libcxx/docs/TestingLibcxx.rst @ 173:0572611fdcc8 llvm10 llvm12
reorgnization done
author | Shinji KONO <kono@ie.u-ryukyu.ac.jp> |
---|---|
date | Mon, 25 May 2020 11:55:54 +0900 |
parents | 1d019706d866 |
children | 2e18cbf3894f |
line wrap: on
line diff
--- a/libcxx/docs/TestingLibcxx.rst Mon May 25 11:50:15 2020 +0900 +++ b/libcxx/docs/TestingLibcxx.rst Mon May 25 11:55:54 2020 +0900 @@ -8,9 +8,9 @@ Getting Started =============== -libc++ uses LIT to configure and run its tests. +libc++ uses LIT to configure and run its tests. -The primary way to run the libc++ tests is by using `make check-libcxx`. +The primary way to run the libc++ tests is by using ``make check-cxx``. However since libc++ can be used in any number of possible configurations it is important to customize the way LIT builds and runs @@ -19,40 +19,23 @@ Please see the `Lit Command Guide`_ for more information about LIT. -.. _LIT Command Guide: http://llvm.org/docs/CommandGuide/lit.html - -Setting up the Environment --------------------------- +.. _LIT Command Guide: https://llvm.org/docs/CommandGuide/lit.html -After building libc++ you must setup your environment to test libc++ using -LIT. - -#. Create a shortcut to the actual lit executable so that you can invoke it - easily from the command line. - - .. code-block:: bash +Usage +----- - $ alias lit='python path/to/llvm/utils/lit/lit.py' - -#. Tell LIT where to find your build configuration. - - .. code-block:: bash - - $ export LIBCXX_SITE_CONFIG=path/to/build-libcxx/test/lit.site.cfg - -Example Usage -------------- - -Once you have your environment set up and you have built libc++ you can run -parts of the libc++ test suite by simply running `lit` on a specified test or -directory. For example: +After building libc++, you can run parts of the libc++ test suite by simply +running ``llvm-lit`` on a specified test or directory. If you're unsure +whether the required libraries have been built, you can use the +`check-cxx-deps` target. For example: .. code-block:: bash - $ cd path/to/src/libcxx - $ lit -sv test/std/re # Run all of the std::regex tests - $ lit -sv test/std/depr/depr.c.headers/stdlib_h.pass.cpp # Run a single test - $ lit -sv test/std/atomics test/std/threads # Test std::thread and std::atomic + $ cd <monorepo-root> + $ make -C <build> check-cxx-deps # If you want to make sure the targets get rebuilt + $ <build>/bin/llvm-lit -sv libcxx/test/std/re # Run all of the std::regex tests + $ <build>/bin/llvm-lit -sv libcxx/test/std/depr/depr.c.headers/stdlib_h.pass.cpp # Run a single test + $ <build>/bin/llvm-lit -sv libcxx/test/std/atomics libcxx/test/std/threads # Test std::thread and std::atomic Sometimes you'll want to change the way LIT is running the tests. Custom options can be specified using the `--param=<name>=<val>` flag. The most common option @@ -62,30 +45,54 @@ .. code-block:: bash - $ lit -sv test/std/containers # Run the tests with the newest -std - $ lit -sv --param=std=c++03 test/std/containers # Run the tests in C++03 + $ <build>/bin/llvm-lit -sv libcxx/test/std/containers # Run the tests with the newest -std + $ <build>/bin/llvm-lit -sv libcxx/test/std/containers --param=std=c++03 # Run the tests in C++03 Occasionally you'll want to add extra compile or link flags when testing. You can do this as follows: .. code-block:: bash - $ lit -sv --param=compile_flags='-Wcustom-warning' - $ lit -sv --param=link_flags='-L/custom/library/path' + $ <build>/bin/llvm-lit -sv libcxx/test --param=compile_flags='-Wcustom-warning' + $ <build>/bin/llvm-lit -sv libcxx/test --param=link_flags='-L/custom/library/path' Some other common examples include: .. code-block:: bash # Specify a custom compiler. - $ lit -sv --param=cxx_under_test=/opt/bin/g++ test/std + $ <build>/bin/llvm-lit -sv libcxx/test/std --param=cxx_under_test=/opt/bin/g++ # Enable warnings in the test suite - $ lit -sv --param=enable_warnings=true test/std + $ <build>/bin/llvm-lit -sv libcxx/test --param=enable_warnings=true # Use UBSAN when running the tests. - $ lit -sv --param=use_sanitizer=Undefined + $ <build>/bin/llvm-lit -sv libcxx/test --param=use_sanitizer=Undefined + +Using a custom site configuration +--------------------------------- +By default, the libc++ test suite will use a site configuration that matches +the current CMake configuration. It does so by generating a ``lit.site.cfg`` +file in the build directory from the ``libcxx/test/lit.site.cfg.in`` template, +and pointing ``llvm-lit`` (which is a wrapper around ``llvm/utils/lit/lit.py``) +to that file. So when you're running ``<build>/bin/llvm-lit``, the generated +``lit.site.cfg`` file is always loaded first, followed by the actual config in +``libcxx/test/lit.cfg``. However, it is sometimes desirable to use a custom +site configuration. To do that, you can use ``--param=libcxx_site_config`` or +the ``LIBCXX_SITE_CONFIG`` environment variable to point to the right site +configuration file. However, you must stop using ``llvm-lit``, or else the +generated ``lit.site.cfg`` will still be preferred: + + .. code-block:: bash + + $ LIBCXX_SITE_CONFIG=path/to/your/site/configuration llvm/utils/lit/lit.py -sv ... + + $ llvm/utils/lit/lit.py -sv ... --param=libcxx_site_config=path/to/your/site/configuration + +In both of these cases, your custom site configuration should set up the +``config`` object in a way that is compatible with what libc++'s ``config.py`` +module expects. LIT Options =========== @@ -95,9 +102,10 @@ Command Line Options -------------------- -To use these options you pass them on the LIT command line as --param NAME or ---param NAME=VALUE. Some options have default values specified during CMake's -configuration. Passing the option on the command line will override the default. +To use these options you pass them on the LIT command line as ``--param NAME`` +or ``--param NAME=VALUE``. Some options have default values specified during +CMake's configuration. Passing the option on the command line will override the +default. .. program:: lit @@ -132,8 +140,7 @@ .. option:: cxx_library_root=<path/to/lib/> Specify the directory of the libc++ library to be tested. By default the - library folder of the build directory is used. This option cannot be used - when use_system_cxx_lib is provided. + library folder of the build directory is used. .. option:: cxx_runtime_root=<path/to/lib/> @@ -148,7 +155,10 @@ **Default**: False Enable or disable testing against the installed version of libc++ library. - Note: This does not use the installed headers. + This impacts whether the ``with_system_cxx_lib`` Lit feature is defined or + not. The ``cxx_library_root`` and ``cxx_runtime_root`` parameters should + still be used to specify the path of the library to link to and run against, + respectively. .. option:: use_lit_shell=<bool> @@ -180,12 +190,6 @@ Run the tests using the given sanitizer. If LLVM_USE_SANITIZER was given when building libc++ then that sanitizer will be used by default. -.. option:: color_diagnostics - - Enable the use of colorized compile diagnostics. If the color_diagnostics - option is specified or the environment variable LIBCXX_COLOR_DIAGNOSTICS is - present then color diagnostics will be enabled. - .. option:: llvm_unwinder Enable the use of LLVM unwinder instead of libgcc. @@ -203,11 +207,28 @@ Specify the site configuration to use when running the tests. Also see `libcxx_site_config`. -.. envvar:: LIBCXX_COLOR_DIAGNOSTICS +Writing Tests +------------- + +When writing tests for the libc++ test suite, you should follow a few guidelines. +This will ensure that your tests can run on a wide variety of hardware and under +a wide variety of configurations. We have several unusual configurations such as +building the tests on one host but running them on a different host, which add a +few requirements to the test suite. Here's some stuff you should know: - If ``LIBCXX_COLOR_DIAGNOSTICS`` is defined then the test suite will attempt - to use color diagnostic outputs from the compiler. - Also see `color_diagnostics`. +- All tests are run in a temporary directory that is unique to that test and + cleaned up after the test is done. +- When a test needs data files as inputs, these data files can be saved in the + repository (when reasonable) and referrenced by the test as + ``// FILE_DEPENDENCIES: <path-to-dependencies>``. Copies of these files or + directories will be made available to the test in the temporary directory + where it is run. +- You should never hardcode a path from the build-host in a test, because that + path will not necessarily be available on the host where the tests are run. +- You should try to reduce the runtime dependencies of each test to the minimum. + For example, requiring Python to run a test is bad, since Python is not + necessarily available on all devices we may want to run the tests on (even + though supporting Python is probably trivial for the build-host). Benchmarks ==========