RE2 confusing test

re2_confusing_test.md

re2/testing/search_test.cc contains a list of (regexp,text) pairs, where the text is tested against the regexp.

The only thing that is being tested for, is that the kEngineBacktrack engine (the "gold standard" engine) gives the same result as the other engines.

This means changing the regexp or text has no effect whatsoever in the test result.

To make the test more useful, I suggest adding a third bool value to manually specify for each test if the regexp is supposed to match the text or not.

To demonstrate why the current behaviour of the tests are confusing, below shows what I tried to do manually to understand the tests. I was surprised to see all tests passing, even though I changed the text from "a" to "b" in the first test in the file:

git clone [email protected]:google/re2.git
cd re2

# Change the text from "a" to "b"
sed -i bak 's/{ "a", "a" }/{ "a", "b" }/' re2/testing/search_test.cc

git diff

diff --git a/re2/testing/search_test.cc b/re2/testing/search_test.cc
 RegexpTest simple_tests[] = {
-  { "a", "a" },
+  { "a", "b" },

make
make test

obj/dbg/test/charclass_test             PASS
obj/dbg/test/compile_test               PASS
obj/dbg/test/filtered_re2_test          PASS
obj/dbg/test/mimics_pcre_test           PASS
obj/dbg/test/parse_test                 PASS
obj/dbg/test/possible_match_test        PASS
obj/dbg/test/re2_test                   PASS
obj/dbg/test/re2_arg_test               PASS
obj/dbg/test/regexp_test                PASS
obj/dbg/test/required_prefix_test       PASS
obj/dbg/test/search_test                PASS
obj/dbg/test/set_test                   PASS
obj/dbg/test/simplify_test              PASS
obj/dbg/test/string_generator_test      PASS
ALL TESTS PASSED.
obj/test/charclass_test                 PASS
obj/test/compile_test                   PASS
obj/test/filtered_re2_test              PASS
obj/test/mimics_pcre_test               PASS
obj/test/parse_test                     PASS
obj/test/possible_match_test            PASS
obj/test/re2_test                       PASS
obj/test/re2_arg_test                   PASS
obj/test/regexp_test                    PASS
obj/test/required_prefix_test           PASS
obj/test/search_test                    PASS
obj/test/set_test                       PASS
obj/test/simplify_test                  PASS
obj/test/string_generator_test          PASS
ALL TESTS PASSED.
obj/so/test/charclass_test              PASS
obj/so/test/compile_test                PASS
obj/so/test/filtered_re2_test           PASS
obj/so/test/mimics_pcre_test            PASS
obj/so/test/parse_test                  PASS
obj/so/test/possible_match_test         PASS
obj/so/test/re2_test                    PASS
obj/so/test/re2_arg_test                PASS
obj/so/test/regexp_test                 PASS
obj/so/test/required_prefix_test        PASS
obj/so/test/search_test                 PASS
obj/so/test/set_test                    PASS
obj/so/test/simplify_test               PASS
obj/so/test/string_generator_test       PASS
ALL TESTS PASSED.