Fix run-tests.php differ calculateCommonSubsequence for EXPECTF

calculateCommonSubsequence should not contain regexes.

Fixes GH-13083
Closes GH-13084

Retry tests on deadlock

Closes GH-12693

Automatically mark tests as flaky

Marking all of these tests as flaky is annoying, so attempt to recognize them
automatically.

Closes GH-12638

Increase run-tests.php timeout for asan

Closes GH-12087

Make unrepeatable tests retriable

This was an error in thinking on my side. The reason these tests are
unrepeatable is because --repeat executes the same request in the same process,

Make unrepeatable tests retriable

This was an error in thinking on my side. The reason these tests are
unrepeatable is because --repeat executes the same request in the same process,
and does not run the CLEAN section in between runs. This is not the case when
retrying tests.

We could potentially make CLEAN tests repeatable by including the CLEAN section
in the tested script. This does however not work for all tests (e.g. tests that
set open_basedir).

Closes GH-12072

show more ...

Fix passing null to parameter of type string (#12014)

This fixes builds without cgi or phpdbg:
./configure --disable-cgi --disable-phpdbg
make
./sapi/cli/php run-tests.php

Fix passing null to parameter of type string (#12014)

This fixes builds without cgi or phpdbg:
./configure --disable-cgi --disable-phpdbg
make
./sapi/cli/php run-tests.php

Otherwise, deprecation warnings (since PHP-8.1) are emitted:
Deprecated: escapeshellarg(): Passing null to parameter #1 ($arg) of
type string is deprecated in run-tests.php...

show more ...

Skip xleak tests on asan (#11610)

Fixes GH-11077

Implement flaky test section

This re-uses the already used for automatic retesting. That's certainly better
than XFAIL.

Closes GH-11325

[skip ci] Add more patterns to run-tests.php retry list

CURL: 404: Page Not Found
IMAP: Can't create a temporary mailbox: [ALREADYEXISTS] Mailbox already exists
Sockets: socket_bind(

[skip ci] Add more patterns to run-tests.php retry list

CURL: 404: Page Not Found
IMAP: Can't create a temporary mailbox: [ALREADYEXISTS] Mailbox already exists
Sockets: socket_bind(): Unable to bind address [98]: Address already in use

show more ...

Add retry mechanism in run-tests.php

We have lots of spurious failures in CI, many of them with the "all" CONFLICT.
We're limiting the retrying to specific error messages. In the future

Add retry mechanism in run-tests.php

We have lots of spurious failures in CI, many of them with the "all" CONFLICT.
We're limiting the retrying to specific error messages. In the future we may
also provide a FLAKY section to retry specific tests.

Closes GH-10892

show more ...

Add zend_alloc XLEAK support

In the future we may want to use a different exit code to warn for tests that
didn't leak.

Closes GH-10999

Add ASAN XLEAK support

Only disable LSAN instead of skipping the test. This way we can still detect
memory issues which is arguably more important anyway.

Closes GH-10996

[skip ci] Avoid infinite loop in differ

TBH I don't understand why this can happen here but not in sebastian/diff. I'll
have to take a closer look.

Fix one more differ direct comparison (through in_array)

Fix direct comparison in run-tests.php differ

[skip ci] Fix var_dump statement in run-tests.php

Implement better diff for run-tests.php

Borrow sebastianbergmann/diff with
MemoryEfficientLongestCommonSubsequenceCalculator

Fixes GH-10806
Closes GH-10875

Another attempt to fix MSAN nightly on master

use_tls=0 on MSAN

Attempt to fix MSAN failure in CI

Closes GH-10851

[skip ci] Fix diff cmd env name in run-tests.php

[skip ci] Remove accidental run-tests.php line

Fix warning in run-tests when PHP compiled without generating phpdbg support. (#10745)

Fix GH-10489: run-tests.php does not escape path when building cmd (#10560)

Multiple tests had to be changed to escape the arguments in shell
commands. Some tests are skipped because th

Fix GH-10489: run-tests.php does not escape path when building cmd (#10560)

Multiple tests had to be changed to escape the arguments in shell
commands. Some tests are skipped because they behave differently with
spaces in the path versus without. One notable example of this is the
hashbang test which does not work because spaces in hashbangs paths are
not supported in Linux.

Co-authored-by: Michael Voříšek <mvorisek@mvorisek.cz>

show more ...

Fix run-tests.php hanging when a worker process dies without notice (#9931)

run-tests.php with `-jN` can hang if a parallel worker dies without notice. This
can happen due to fatal error

Fix run-tests.php hanging when a worker process dies without notice (#9931)

run-tests.php with `-jN` can hang if a parallel worker dies without notice. This
can happen due to fatal errors in the worker, or if the worker is killed.

- run-tests.php (main process)
\_ run-tests.php (worker #0) // main process hangs if this one crashes
\_ test-001.php (test-001.phpt)

show more ...

Migrate i386 to GitHub actions

Closes GH-9856