Show slow SKIPIF sections as well

Determine run-tests executables consistently

Don't pass null to strlen()

Fixed bug #80437

Only chmod if the file has been created.

Add --repeat flag to usage info of run-tests.php

Closes GH-6429.

Add --repeat testing mode

This testing mode executes the test multiple times in the same
process (but in different requests). It is primarily intended to
catch tracing JIT bugs, but

Add --repeat testing mode

This testing mode executes the test multiple times in the same
process (but in different requests). It is primarily intended to
catch tracing JIT bugs, but also catches state leaks across
requests.

Closes GH-6365.

show more ...

Don't set xdebug.default_enable in run-tests

This throws a deprecation warning in XDebug 3.

Closes GH-6324.

Skip arginfo test under msan

Msan is missing interceptors for some functions that result in
false positives.

Remove unused --temp-urlbase option from run-tests.php

This was part of the removed HTML functionality.

Fix: Gracefully handle empty user input in run-tests.php

Closes GH-6016.

Add `run-tests.php --context [n]` option.

Mentioned in https://github.com/php/php-src/pull/5965#discussion_r467621123

This PR proposes 3 lines of context so the impact can be seen i

Add `run-tests.php --context [n]` option.

Mentioned in https://github.com/php/php-src/pull/5965#discussion_r467621123

This PR proposes 3 lines of context so the impact can be seen in tests.
Other `diff` programs show around 3 lines of context.
(This helps indicate exactly which position a test should be updated
to add a new expected line at)

Use the mapping for choosing order to display diffs

Properly include context in cases where the expected output had more lines than
the actual output, e.g.

```
--FILE--
A
A1
A
C
NEARBY
--EXPECTF--
A
B
A1
B
A
B
A
B
NEARBY
```

Closes GH-5968

show more ...

Colorize --show-diff, only on stdout

Don't add colors to the saved `.diff` file.

Related to a41cf3e1d2ed3ae38ad6115a8b65e5f14dae0b49

Support NO_COLOR environment variable in run-tests.php

And add a --color option for run-tests.php

See https://no-color.org/

> an informal standard is hereby proposed:
>

Support NO_COLOR environment variable in run-tests.php

And add a --color option for run-tests.php

See https://no-color.org/

> an informal standard is hereby proposed:
>
> All command-line software which outputs text with ANSI color added should check
> for the presence of a `NO_COLOR` environment variable that, when present
> (regardless of its value), prevents the addition of ANSI color.

show more ...

Revert "Make diff section contents red(-)/green(+) in run-tests.php"

This reverts commit 58489bd3b93e10c150933eefac05297b48213eb3.

This also uses ANSI escape code coloring for .diff

Revert "Make diff section contents red(-)/green(+) in run-tests.php"

This reverts commit 58489bd3b93e10c150933eefac05297b48213eb3.

This also uses ANSI escape code coloring for .diff files, where it
is not meaningful.

show more ...

Make diff section contents red(-)/green(+) in run-tests.php

This uses green for lines with `+` and red for lines with `-`.

Colors(Red and Green) would make the failure causes stand

Make diff section contents red(-)/green(+) in run-tests.php

This uses green for lines with `+` and red for lines with `-`.

Colors(Red and Green) would make the failure causes stand out visually when scrolling through errors.

Closes GH-5965

show more ...

Use short array values in run-tests.php

The run-tests.php file already requires php 7 syntax.
Done automatically with phpcbf with no other changes done/planned.

This file is alr

Use short array values in run-tests.php

The run-tests.php file already requires php 7 syntax.
Done automatically with phpcbf with no other changes done/planned.

This file is already using short arrays in other places.

run-tests has already been substantially refactored in PHP 8.0
compared to PHP 7.4.

This makes merge conflicts more likely, but changes to run-tests.php on older
versions are less frequent bug fixes.

show more ...

Add colours to test results in run-tests.php

The test runner will attempt to colourise the result of each test

The --no-color option is introduced to disable this feature.

Add colours to test results in run-tests.php

The test runner will attempt to colourise the result of each test

The --no-color option is introduced to disable this feature.

Closes GH-5901

show more ...

Add type information to test-runner

Closes GH-5942

[RFC] Make string length for getTraceAsString() configurable

Add a `zend.exception_string_param_max_len` ini setting.
(same suffix as `log_errors_max_len`)

Allow values between

[RFC] Make string length for getTraceAsString() configurable

Add a `zend.exception_string_param_max_len` ini setting.
(same suffix as `log_errors_max_len`)

Allow values between 0 and 1000000 bytes.
For example, with zend.exception_string_param_max_len=0,
"" would represent the empty string, and "..." would represent something
longer than the empty string.
Previously, this was hardcoded as exactly 15 bytes.

Discussion: https://externals.io/message/110717

Closes GH-5769

show more ...

Use small counter values for testing

Removing HTML Functionality from run-tests.php

As discussed on GH-5632, the HTML functionality does not appear
to be in active use. For HTML rendering of test results, it is
suggeste

Removing HTML Functionality from run-tests.php

As discussed on GH-5632, the HTML functionality does not appear
to be in active use. For HTML rendering of test results, it is
suggested to instead use the JUnit integration, in combination
with your favorite JUnit viewer.

Closes GH-5705.

show more ...

Optimize parallel test runner startup

Don't connect to each worker individually: First spawn all processes
and then establish connections in any order.

This avoids a ~1s wait wh

Optimize parallel test runner startup

Don't connect to each worker individually: First spawn all processes
and then establish connections in any order.

This avoids a ~1s wait when running on many cores.

show more ...

run-tests: refactor

run-tests: echo message and exit if no tests found

The test runner currently defaults to running the entire test suite if
no selected tests can be found. This can be unexpected.

run-tests: echo message and exit if no tests found

The test runner currently defaults to running the entire test suite if
no selected tests can be found. This can be unexpected.

For example the ext/mysqlnd/ directory has no tests, if you specify that
directory when testing the entire test suite will be run.

run-tests.php [options] ext/mysqlnd/

Closes GH-5605.

show more ...

Add --file-cache-prime/use options to run-tests

--file-cache-prime populates the file cache,
--file-cache-use uses the file cache.

And fix a number of tests to run under file ca

Add --file-cache-prime/use options to run-tests

--file-cache-prime populates the file cache,
--file-cache-use uses the file cache.

And fix a number of tests to run under file cache or disabled
timestamp validation.

show more ...