run-tests.php: Preserve .php on leak in one more place

run-tests: drop support for ancient Valgrind versions

Valgrind 3.3.0 was released in 2007, not even RHEL has crap older than
this. It could be argued that 3.8.0, released in 2012, could

run-tests: drop support for ancient Valgrind versions

Valgrind 3.3.0 was released in 2007, not even RHEL has crap older than
this. It could be argued that 3.8.0, released in 2012, could be a safe
cutoff too.

Closes GH-6728.

show more ...

Capture Microsoft-defined HRESULT exit codes exit codes

The lack of such a check leads to false-passes of tests on Windows
which expect no output, but produce a segfault or similar issue

Capture Microsoft-defined HRESULT exit codes exit codes

The lack of such a check leads to false-passes of tests on Windows
which expect no output, but produce a segfault or similar issue. I
discovered this a while ago due to bad tests in an extension I maintain.

Closes GH-6722.

show more ...

run-tests: add skip cache

Currently every --SKIPIF-- section in every test file results in 1
extra execution of PHP, every --EXTENSIONS-- section - in 2 executions.
This is quite was

run-tests: add skip cache

Currently every --SKIPIF-- section in every test file results in 1
extra execution of PHP, every --EXTENSIONS-- section - in 2 executions.
This is quite wasteful, as skip checking code is extremely repetitive
and extensions are fixed for every binary/ini/command parameters
combination.

This patch adds caching to all such checks.

On my machine, the gains are quite noticeable: 36s instead of 43s
with -j16, 292s instead of 337s without concurrency. Cache stats are
3780 hits, 1247 misses in the latter case. In the future, tests could
be adjusted to have more uniform skip checks to improve performance even
more.

Closes GH-6681.

show more ...

run-tests: fixed exit code not being set on BORKED tests

When no test paths are specified this shows up when 'make test' is used on a PECL extension without specifying tests to run (or in ph

run-tests: fixed exit code not being set on BORKED tests

When no test paths are specified this shows up when 'make test' is used on a PECL extension without specifying tests to run (or in php-src too, I guess...)

Closes GH-6717.

show more ...

Color LEAK&FAIL like a failure

Remove stray mentions of mbstring.func_overload

This feature has been completely removed.

Closes GH-6688.

Deprecate passing null to non-nullable arg of internal function

This deprecates passing null to non-nullable scale arguments of
internal functions, with the eventual goal of making the b

Deprecate passing null to non-nullable arg of internal function

This deprecates passing null to non-nullable scale arguments of
internal functions, with the eventual goal of making the behavior
consistent with userland functions, where null is never accepted
for non-nullable arguments.

This change is expected to cause quite a lot of fallout. In most
cases, calling code should be adjusted to avoid passing null. In
some cases, PHP should be adjusted to make some function arguments
nullable. I have already fixed a number of functions before landing
this, but feel free to file a bug if you encounter a function that
doesn't accept null, but probably should. (The rule of thumb for
this to be applicable is that the function must have special behavior
for 0 or "", which is distinct from the natural behavior of the
parameter.)

RFC: https://wiki.php.net/rfc/deprecate_null_to_scalar_internal_arg

Closes GH-6475.

show more ...

run-tests: fix JUnit counts

In 19680f886f, I forgot about my own TODO. At least, now I understand what
this line was about :P

run-tests.php: move JUnit stuff into a class

This is part one of my work that was announced at
https://externals.io/message/110391

Closes GH-6671.

Fix environment variable not work in run-tests.php

Closes GH-6657.

Create .php and .sh on valgrind failure

Show slow SKIPIF sections as well

Determine run-tests executables consistently

Don't pass null to strlen()

Fixed bug #80437

Only chmod if the file has been created.

Add --repeat flag to usage info of run-tests.php

Closes GH-6429.

Add --repeat testing mode

This testing mode executes the test multiple times in the same
process (but in different requests). It is primarily intended to
catch tracing JIT bugs, but

Add --repeat testing mode

This testing mode executes the test multiple times in the same
process (but in different requests). It is primarily intended to
catch tracing JIT bugs, but also catches state leaks across
requests.

Closes GH-6365.

show more ...

Don't set xdebug.default_enable in run-tests

This throws a deprecation warning in XDebug 3.

Closes GH-6324.

Skip arginfo test under msan

Msan is missing interceptors for some functions that result in
false positives.

Remove unused --temp-urlbase option from run-tests.php

This was part of the removed HTML functionality.

Fix: Gracefully handle empty user input in run-tests.php

Closes GH-6016.

Add `run-tests.php --context [n]` option.

Mentioned in https://github.com/php/php-src/pull/5965#discussion_r467621123

This PR proposes 3 lines of context so the impact can be seen i

Add `run-tests.php --context [n]` option.

Mentioned in https://github.com/php/php-src/pull/5965#discussion_r467621123

This PR proposes 3 lines of context so the impact can be seen in tests.
Other `diff` programs show around 3 lines of context.
(This helps indicate exactly which position a test should be updated
to add a new expected line at)

Use the mapping for choosing order to display diffs

Properly include context in cases where the expected output had more lines than
the actual output, e.g.

```
--FILE--
A
A1
A
C
NEARBY
--EXPECTF--
A
B
A1
B
A
B
A
B
NEARBY
```

Closes GH-5968

show more ...

Colorize --show-diff, only on stdout

Don't add colors to the saved `.diff` file.

Related to a41cf3e1d2ed3ae38ad6115a8b65e5f14dae0b49

Support NO_COLOR environment variable in run-tests.php

And add a --color option for run-tests.php

See https://no-color.org/

> an informal standard is hereby proposed:
>

Support NO_COLOR environment variable in run-tests.php

And add a --color option for run-tests.php

See https://no-color.org/

> an informal standard is hereby proposed:
>
> All command-line software which outputs text with ANSI color added should check
> for the presence of a `NO_COLOR` environment variable that, when present
> (regardless of its value), prevents the addition of ANSI color.

show more ...