Skip to content

Instantly share code, notes, and snippets.

@alex85k
Created September 19, 2014 13:37
Show Gist options
  • Save alex85k/ecd5386a14157e8c6921 to your computer and use it in GitHub Desktop.
Save alex85k/ecd5386a14157e8c6921 to your computer and use it in GitHub Desktop.
Catch test errors
D:\tmp>call settings.bat
D:\tmp>SET COMPILER=MSVC2008-32
D:\tmp>SET PREFIX=d:\libs16d
D:\tmp>SET BOOST_ROOT=d:\boost
D:\tmp>SET VARIANT=Debug
D:\tmp>set "ER=if ERRORLEVEL 1 exit /b 1"
D:\tmp>if "MSVC2008-32" == "MINGW" (
SET "PATH=C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64\mpirt;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\ia32\mpirt;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64\compiler;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\ia32\compiler;C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\;C:\Program Files\MiKTeX 2.9\miktex\bin\x64\;C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;C:\Program Files (x86)\Microsoft SDKs\TypeScript\1.0\;C:\Program Files (x86)\CMake\bin;C:\Program Files (x86)\Git\cmd;C:\Program Files\TortoiseSVN\bin;C:\Ruby193\bin;c:\mingw64_491seh\mingw64\bin;C:\msys\bin"
SET CMAKE_TARGET="MinGW Makefiles"
SET MAKE=mingw32-make
SET "SED=sed"
) else (
if "MSVC2008-32" == "MSVC2008-32" (call "C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\Tools\\..\..\VC\vcvarsall.bat" x86 )
if "MSVC2008-32" == "MSVC2010-32" (call "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\Tools\\..\..\VC\vcvarsall.bat" x86 )
if "MSVC2008-32" == "MSVC2012" (call "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\Tools\\..\..\VC\vcvarsall.bat" x86_amd64 )
if "MSVC2008-32" == "MSVC2013" (call "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\Tools\\..\..\VC\vcvarsall.bat" x86_amd64 )
if "MSVC2008-32" == "MSVC2014" (call "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools\\..\..\VC\vcvarsall.bat" amd64 )
set "CMAKE=cmake -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=d:\libs16d"
SET "MAKE=nmake"
SET "SED=C:\Program Files (x86)\Git\bin\sed"
)
Setting environment for using Microsoft Visual Studio 2008 x86 tools.
============= MSVC2008-32 -- Debug --- d:\libs16d --- ==============
-- Configuring done
-- Generating done
-- Build files have been written to: D:/tmp/Catch/projects/CMake
[100%] Built target SelfTest
UpdateCTestConfiguration from :D:/tmp/Catch/projects/CMake/DartConfiguration.tcl
UpdateCTestConfiguration from :D:/tmp/Catch/projects/CMake/DartConfiguration.tcl
Test project D:/tmp/Catch/projects/CMake
Constructing a list of tests
Done constructing a list of tests
Checking test dependency graph...
Checking test dependency graph end
test 1
Start 1: RunTests
1: Test command: D:\tmp\Catch\projects\CMake\SelfTest.exe
1: Test timeout computed to be: 9.99988e+006
1:
1: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1: SelfTest.exe is a Catch v1.1 b5 (develop) host application.
1: Run with -? for options
1:
1: -------------------------------------------------------------------------------
1: toString( vectors<has_toString )
1: -------------------------------------------------------------------------------
1: D:\tmp\Catch\projects\SelfTest\ToStringWhich.cpp(52)
1: ...............................................................................
1:
1: D:\tmp\Catch\projects\SelfTest\ToStringWhich.cpp(55): FAILED:
1: REQUIRE( Catch::toString( v ) == "{ {?} }" )
1: with expansion:
1: "{ toString( has_toString ) }" == "{ {?} }"
1:
1: -------------------------------------------------------------------------------
1: toString( vectors<has_maker_and_toString )
1: -------------------------------------------------------------------------------
1: D:\tmp\Catch\projects\SelfTest\ToStringWhich.cpp(64)
1: ...............................................................................
1:
1: D:\tmp\Catch\projects\SelfTest\ToStringWhich.cpp(67): FAILED:
1: REQUIRE( Catch::toString( v ) == "{ StringMaker<has_maker_and_toString> }" )
1: with expansion:
1: "{ toString( has_maker_and_toString ) }"
1: ==
1: "{ StringMaker<has_maker_and_toString> }"
1:
1: ===============================================================================
1: test cases: 91 | 89 passed | 2 failed
1: assertions: 470 | 468 passed | 2 failed
1:
1/3 Test #1: RunTests .........................***Failed 0.24 sec
test 2
Start 2: ListTests
2: Test command: D:\tmp\Catch\projects\CMake\SelfTest.exe "--list-tests"
2: Test timeout computed to be: 9.99988e+006
2: All available test cases:
2: Some simple comparisons between doubles
2: [Approx]
2: Approximate comparisons with different epsilons
2: [Approx]
2: Approximate comparisons with floats
2: [Approx]
2: Approximate comparisons with ints
2: [Approx]
2: Approximate comparisons with mixed numeric types
2: [Approx]
2: Use a custom approx
2: [Approx][custom]
2: Approximate PI
2: [Approx][PI]
2: Scenario: Do that thing with the thing
2: [Tags]
2: Scenario: Vector resizing affects size and capacity
2: [bdd][capacity][size][vector]
2: Scenario: This is a really long scenario name to see how the list command
2: deals with wrapping
2: [anotherReallyLongTagNameButThisOneHasNoObviousWrapPointsSoShouldSplitWi-
2: thinAWordUsingADashCharacter][long][lots][one very long tag name that
2: should cause line wrapping writing out using the list command][tags]
2: [verbose][very long tags]
2: Scenario: BDD tests requiring Fixtures to provide commonly-accessed data or
2: methods
2: [bdd][fixtures]
2: A METHOD_AS_TEST_CASE based test run that succeeds
2: [class]
2: A METHOD_AS_TEST_CASE based test run that fails
2: [.][class][failing][hide]
2: A TEST_CASE_METHOD based test run that succeeds
2: [class]
2: A TEST_CASE_METHOD based test run that fails
2: [.][class][failing][hide]
2: Equality checks that should succeed
2: Equality checks that should fail
2: [!mayfail][.][failing][hide]
2: Inequality checks that should succeed
2: Inequality checks that should fail
2: [.][failing][hide]
2: Ordering comparison checks that should succeed
2: Ordering comparison checks that should fail
2: [.][failing][hide]
2: Comparisons with int literals don't warn when mixing signed/ unsigned
2: comparisons between int variables
2: comparisons between const int variables
2: Comparisons between unsigned ints and negative signed ints match c++ standard
2: behaviour
2: Comparisons between ints where one side is computed
2: Pointers can be compared to null
2: 'Not' checks that should succeed
2: 'Not' checks that should fail
2: [.][failing][hide]
2: When checked exceptions are thrown they can be expected or unexpected
2: Expected exceptions that don't throw or unexpected exceptions fail the test
2: [.][failing][hide]
2: When unchecked exceptions are thrown directly they are always failures
2: [.][failing][hide]
2: An unchecked exception reports the line of the last assertion
2: [.][failing][hide]
2: When unchecked exceptions are thrown from sections they are always failures
2: [.][failing][hide]
2: When unchecked exceptions are thrown from functions they are always failures
2: [.][failing][hide]
2: When unchecked exceptions are thrown during a REQUIRE the test should abort
2: fail
2: [.][failing][hide]
2: When unchecked exceptions are thrown during a CHECK the test should abort and
2: fail
2: [.][failing][hide]
2: When unchecked exceptions are thrown, but caught, they do not affect the test
2: Unexpected custom exceptions can be translated
2: [.][failing][hide]
2: Custom exceptions can be translated when testing for nothrow
2: [.][failing][hide]
2: Custom exceptions can be translated when testing for throwing as something
2: else
2: [.][failing][hide]
2: Unexpected exceptions can be translated
2: [.][failing][hide]
2: NotImplemented exception
2: Generators over two ranges
2: [generators]
2: Generator over a range of pairs
2: [generators]
2: INFO and WARN do not abort tests
2: [.][hide][messages]
2: SUCCEED counts as a test pass
2: [messages]
2: INFO gets logged on failure
2: [.][failing][hide][messages]
2: INFO gets logged on failure, even if captured before successful assertions
2: [.][failing][hide][messages]
2: FAIL aborts the test
2: [.][failing][hide][messages]
2: FAIL does not require an argument
2: [.][failing][hide][messages]
2: SUCCESS does not require an argument
2: [.][hide][messages]
2: Output from all sections is reported
2: [.][failing][hide][messages]
2: Standard output from all sections is reported
2: [.][hide][messages]
2: SCOPED_INFO is reset for each loop
2: [.][failing][hide][messages]
2: The NO_FAIL macro reports a failure but does not fail the test
2: [messages]
2: just info
2: [info][isolated info][messages]
2: just failure
2: [.][fail][hide][isolated info][messages]
2: sends information to INFO
2: [.][failing][hide]
2: Pointers can be converted to strings
2: [.][hide][messages]
2: random SECTION tests
2: [.][failing][hide][sections]
2: nested SECTION tests
2: [.][failing][hide][sections]
2: more nested SECTION tests
2: [.][failing][hide][sections]
2: even more nested SECTION tests
2: [sections]
2: looped SECTION tests
2: [.][failing][hide][sections]
2: looped tests
2: [.][failing][hide]
2: Sends stuff to stdout and stderr
2: [.][hide]
2: null strings
2: checkedIf
2: checkedIf, failing
2: [.][failing][hide]
2: checkedElse
2: checkedElse, failing
2: [.][failing][hide]
2: xmlentitycheck
2: send a single char to INFO
2: [.][failing][hide]
2: atomic if
2: [0][failing]
2: String matchers
2: [matchers]
2: Contains string matcher
2: [.][failing][hide][matchers]
2: StartsWith string matcher
2: [.][failing][hide][matchers]
2: EndsWith string matcher
2: [.][failing][hide][matchers]
2: Equals string matcher
2: [.][failing][hide][matchers]
2: Equals string matcher, with NULL
2: [matchers]
2: AllOf matcher
2: [matchers]
2: AnyOf matcher
2: [matchers]
2: Equals
2: [matchers]
2: Factorials are computed
2: [factorial]
2: An empty test with no assertions
2: [empty]
2: Nice descriptive name
2: [.][hide][tag1][tag2][tag3]
2: first tag
2: [tag1]
2: second tag
2: [tag2]
2: vectors can be sized and resized
2: [vector]
2: A couple of nested sections followed by a failure
2: [.][failing][hide]
2: not allowed
2: [!throws]
2: Tabs and newlines show in output
2: [.][failing][hide][whitespace]
2: toString on const wchar_t const pointer returns the string contents
2: [toString]
2: toString on const wchar_t pointer returns the string contents
2: [toString]
2: toString on wchar_t const pointer returns the string contents
2: [toString]
2: toString on wchar_t returns the string contents
2: [toString]
2: section tracking
2: Process can be configured on command line
2: [command-line][config]
2: Long strings can be wrapped
2: [wrap]
2: Strings can be rendered with colour
2: [.][colour][hide]
2: Text can be formatted using the Text class
2: Long text is truncted
2: [Text][Truncated]
2: Parsing a std::pair
2: [Tricky][std::pair]
2: Where the is more to the expression after the RHS[failing]
2: [.][Tricky][failing][hide]
2: Where the LHS is not a simple value[failing]
2: [.][Tricky][failing][hide]
2: A failing expression with a non streamable type is still captured[failing]
2: [.][Tricky][failing][hide]
2: string literals of different sizes can be compared[failing]
2: [.][Tricky][failing][hide]
2: An expression with side-effects should only be evaluated once
2: [Tricky]
2: Operators at different namespace levels not hijacked by Koenig lookup
2: [Tricky]
2: Demonstrate that a non-const == is not used
2: [Tricky]
2: Test enum bit values
2: [Tricky]
2: boolean member
2: [Tricky]
2: (unimplemented) static bools can be evaluated
2: [Tricky]
2: Objects that evaluated in boolean contexts can be checked
2: [SafeBool][Tricky]
2: Assertions then sections
2: [Tricky]
2: non streamable - with conv. op
2: [Tricky]
2: Comparing function pointers
2: [Tricky][function pointer]
2: Comparing member function pointers
2: [Tricky][member function pointer]
2: pointer to class
2: [Tricky]
2: X/level/0/a
2: [Tricky]
2: X/level/0/b
2: [Tricky][fizz]
2: X/level/1/a
2: [Tricky]
2: X/level/1/b
2: [Tricky]
2: Anonymous test case 1
2: Test case with one argument
2: Variadic macros
2: [sections][variadic]
2: toString(enum)
2: [enum][toString]
2: toString(enum w/operator<<)
2: [enum][toString]
2: std::pair<int,std::string> -> toString
2: [pair][toString]
2: std::pair<int,const std::string> -> toString
2: [pair][toString]
2: std::vector<std::pair<std::string,int> > -> toString
2: [pair][toString]
2: pair<pair<int,const char *,pair<std::string,int> > -> toString
2: [pair][toString]
2: vector<int> -> toString
2: [toString][vector]
2: vector<string> -> toString
2: [toString][vector]
2: toString( has_toString )
2: [toString]
2: toString( has_maker )
2: [toString]
2: toString( has_maker_and_toString )
2: [toString]
2: toString( vectors<has_toString )
2: [toString]
2: toString( vectors<has_maker )
2: [toString]
2: toString( vectors<has_maker_and_toString )
2: [toString]
2: 141 test cases
2:
2/3 Test #2: ListTests ........................ Passed 0.05 sec
test 3
Start 3: ListTags
3: Test command: D:\tmp\Catch\projects\CMake\SelfTest.exe "--list-tags"
3: Test timeout computed to be: 9.99988e+006
3: All available tags:
3: 1 [!mayfail]
3: 1 [!throws]
3: 50 [.]
3: 1 [0]
3: 1 [anotherReallyLongTagNameButThisOneHasNoObviousWrapPointsSoShouldSpli-
3: tWithinAWordUsingADashCharacter]
3: 7 [Approx]
3: 2 [bdd]
3: 1 [capacity]
3: 4 [class]
3: 1 [colour]
3: 1 [command-line]
3: 1 [config]
3: 1 [custom]
3: 1 [empty]
3: 2 [enum]
3: 1 [factorial]
3: 1 [fail]
3: 43 [failing]
3: 1 [fixtures]
3: 1 [fizz]
3: 1 [function pointer]
3: 2 [generators]
3: 50 [hide]
3: 1 [info]
3: 2 [isolated info]
3: 1 [long]
3: 1 [lots]
3: 9 [matchers]
3: 1 [member function pointer]
3: 14 [messages]
3: 1 [one very long tag name that should cause line wrapping writing out
3: using the list command]
3: 4 [pair]
3: 1 [PI]
3: 1 [SafeBool]
3: 6 [sections]
3: 1 [size]
3: 1 [std::pair]
3: 2 [tag1]
3: 2 [tag2]
3: 1 [tag3]
3: 2 [Tags][tags]
3: 1 [Text]
3: 18 [toString]
3: 21 [Tricky]
3: 1 [Truncated]
3: 1 [variadic]
3: 4 [vector]
3: 1 [verbose]
3: 1 [very long tags]
3: 1 [whitespace]
3: 1 [wrap]
3: 51 tags
3:
3/3 Test #3: ListTags ......................... Passed 0.06 sec
67% tests passed, 1 tests failed out of 3
Total Test time (real) = 0.35 sec
The following tests FAILED:
1 - RunTests (Failed)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment