Fix unrolled montgomery multiplication for POWER9

In the reference C implementation in bn_asm.c, tp[num + 1] contains the
carry bit for accumulations into tp[num]. tp[num + 1] is only ev

Fix unrolled montgomery multiplication for POWER9

In the reference C implementation in bn_asm.c, tp[num + 1] contains the
carry bit for accumulations into tp[num]. tp[num + 1] is only ever
assigned, never itself incremented.

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18883)

show more ...

Revert "Revert "bn: Add fixed length (n=6), unrolled PPC Montgomery Multiplication""

This reverts commit 712d9cc90e355b2c98a959d4e9398610d2269c9e.

Reviewed-by: Hugo Landau <hlandau@

Revert "Revert "bn: Add fixed length (n=6), unrolled PPC Montgomery Multiplication""

This reverts commit 712d9cc90e355b2c98a959d4e9398610d2269c9e.

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18883)

show more ...

Revert "bn: Add fixed length (n=6), unrolled PPC Montgomery Multiplication"

This reverts commit 0d40ca47bd86e74a95c3a2f5fb6c67cdbee93c79.

It was found that the computation produces

Revert "bn: Add fixed length (n=6), unrolled PPC Montgomery Multiplication"

This reverts commit 0d40ca47bd86e74a95c3a2f5fb6c67cdbee93c79.

It was found that the computation produces incorrect results in some
cases.

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18512)

show more ...

bn: Fix .size directive

This requires the text address.

Fixes #15923

Signed-off-by: Martin Schwenke <martin@meltin.net>

Reviewed-by: Tomas Mraz <tomas@openssl.org>

bn: Fix .size directive

This requires the text address.

Fixes #15923

Signed-off-by: Martin Schwenke <martin@meltin.net>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15971)

show more ...

bn: Use a basic branch-if-not-zero

Ancient toolchains fail the build because they don't like the hints,
newer ISAs recommend not using the hints and relying on dynamic branch
predict

bn: Use a basic branch-if-not-zero

Ancient toolchains fail the build because they don't like the hints,
newer ISAs recommend not using the hints and relying on dynamic branch
prediction.

Signed-off-by: Martin Schwenke <martin@meltin.net>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15971)

show more ...

bn: save/restore registers to/from stack

mtvsrd/mfvsrd are ISA >= 2.07 only, so this won't work for older
CPUs.

It would be possible to use this scheme only in the ISA >= 3.0

bn: save/restore registers to/from stack

mtvsrd/mfvsrd are ISA >= 2.07 only, so this won't work for older
CPUs.

It would be possible to use this scheme only in the ISA >= 3.0
implementation. However, in the future it may be possible for newer
ISAs to allow CPU implementations without a vector unit, so don't
bother. The performance improvement versus using the stack was small
anyway.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15798)

show more ...

bn: Switch $i to be unused r9

No need to save/restore because it is volatile.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

bn: Switch $i to be unused r9

No need to save/restore because it is volatile.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15798)

show more ...

bn: Drop unnecessary use of r9

This is done in other versions due to the possibility of an early
return. However, there is no early return here.

Signed-off-by: Martin Schwenke

bn: Drop unnecessary use of r9

This is done in other versions due to the possibility of an early
return. However, there is no early return here.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15798)

show more ...

bn: Update .align pseudo-ops to match convention

64-bit alignment at the beginning of functions, 32-bit alignment for
loop targets.

Signed-off-by: Martin Schwenke <martin@meltin

bn: Update .align pseudo-ops to match convention

64-bit alignment at the beginning of functions, 32-bit alignment for
loop targets.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15798)

show more ...

bn: Drop use of .p2align pseudo-op

This works on Linux but breaks the build on AIX.

Fixes #15748

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay

bn: Drop use of .p2align pseudo-op

This works on Linux but breaks the build on AIX.

Fixes #15748

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15798)

show more ...

bn: Add fixed length (n=6), unrolled PPC Montgomery Multiplication

Overall improvement for p384 of ~18% on Power 9, compared to existing
Power assembling code. See comment in code for m

bn: Add fixed length (n=6), unrolled PPC Montgomery Multiplication

Overall improvement for p384 of ~18% on Power 9, compared to existing
Power assembling code. See comment in code for more details.

Multiple unrolled versions could be generated for values other than
6. However, for TLS 1.3 the only other ECC algorithms that might use
Montgomery Multiplication are p256 and p521, but these have custom
algorithms that don't use Montgomery Multiplication. Non-ECC
algorithms are likely to use larger key lengths that won't fit into
the n <= 10 length limitation of this code.

Signed-off-by: Amitay Isaacs <amitay@ozlabs.org>
Signed-off-by: Alastair D'Silva <alastair@d-silva.org>
Signed-off-by: Martin Schwenke <martin@meltin.net>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15175)

show more ...