Add real-time benchmark to CI as a nightly job (#15696)

This PR integrates https://github.com/kocsismate/php-version-benchmarks/ into the CI as a nightly job running every day at 12:30 AM UT

Add real-time benchmark to CI as a nightly job (#15696)

This PR integrates https://github.com/kocsismate/php-version-benchmarks/ into the CI as a nightly job running every day at 12:30 AM UTC. Roughly, the following happens: the benchmark suite spins up an AWS EC2 instance via Terraform, runs the tests according to the configuration, and then the results are committed to the https://github.com/kocsismate/php-version-benchmark-results repository.

In order to have as stable results as possible, the CPU, kernel and other settings of the AWS instance are fine-tuned:

- Hyper-threading is disabled
- Turbo boost is disabled
- C states of the CPU are limited: https://docs.aws.amazon.com/linux/al2/ug/processor_state_control.html#baseline-perf
- The workload is dedicated to a single core by using taskset according to Intel's recommendations (https://web.archive.org/web/20210614053522/https://01.org/node/3774)
- An io2 SSD volume is attached to the instance which has a provisioned IOPS (https://docs.aws.amazon.com/ebs/latest/userguide/provisioned-iops.html#io2-block-express) so that IO performance is nearly constant
- The instance is dedicated so that the noisy neighbor effect is eliminated: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/dedicated-instance.html
- ASLR is disabled (Disable ASLR for benchmark #13769)

Customizing the CPU is only supported by metal instances among recent instance types according to https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/processor_state_control.html, so at last, a c7i.metal-24xl instance is used in the eu-west-1 region.

The benchmark suite compares the performance of the latest commit of the master branch in the time when the benchmark runs with the last commit of master from the day before yesterday. I.e. if the benchmark runs tomorrow morning at 2 AM, then the performance of the latest commit will be benchmarked against the last commit pushed yesterday. This makes it possible to spot outstanding regressions (or progressions) in time. Actually, the end goal is to send notifications in case of any significant changes for further analyzation. The reason why the benchmark is run for previous commits as well (while they may have already been measured the day before) is to make the results less sensitive for changes in the environment or the benchmark suite itself. I.e.: if AWS upgrades the OS, or if the code under test is modified, then the numbers will likely be affected, and the previous results will be invalidated).

show more ...