Fix undefined behaviour in unpack()

atoi()'s return value is actually undefined when an underflow or
overflow occurs. For example on 32-bit on my system the overflow test
which input

Fix undefined behaviour in unpack()

atoi()'s return value is actually undefined when an underflow or
overflow occurs. For example on 32-bit on my system the overflow test
which inputs "h2147483648" results in repetitions==2147483647 and on
64-bit this gives repetitions==-2147483648. The reason the test works on
32-bit is because there's a second undefined behaviour problem:
in case 'h' when repetitions==2147483647, we add 1 and divide by 2.
This is signed-wrap undefined behaviour and accidentally triggers the
overflow check like we wanted to.

Avoid all this trouble and use strtol with explicit error checking.

This also fixes a semantic bug where repetitions==INT_MAX would result
in the overflow check to trigger, even though there is no overflow.

Closes GH-10943.

show more ...