Provide additional AES-GCM test patterns to enhance test coverage.

To enhance test coverage for AES-GCM mode, we provided longer additional
testing patterns for AES-GCM testing.

Provide additional AES-GCM test patterns to enhance test coverage.

To enhance test coverage for AES-GCM mode, we provided longer additional
testing patterns for AES-GCM testing.

Signed-off-by: Phoebe Chen <phoebe.chen@sifive.com>
Signed-off-by: Jerry Shih <jerry.shih@sifive.com>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/21923)

show more ...

riscv: Provide vector crypto implementation of AES-GCM mode.

To accelerate the performance of the AES-GCM mode, in this patch, we
have the specialized multi-block implementations for AES

riscv: Provide vector crypto implementation of AES-GCM mode.

To accelerate the performance of the AES-GCM mode, in this patch, we
have the specialized multi-block implementations for AES-128-GCM,
AES-192-GCM and AES-256-GCM.

Signed-off-by: Phoebe Chen <phoebe.chen@sifive.com>
Signed-off-by: Jerry Shih <jerry.shih@sifive.com>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/21923)

show more ...

Update for Zvkb extension.

https://github.com/riscv/riscv-crypto/blob/c8ddeb7e64a3444dda0438316af1238aeed72041/doc/vector/riscv-crypto-vector-zvkb.adoc
Create `RISCV_HAS_ZVKB()` macro.

Update for Zvkb extension.

https://github.com/riscv/riscv-crypto/blob/c8ddeb7e64a3444dda0438316af1238aeed72041/doc/vector/riscv-crypto-vector-zvkb.adoc
Create `RISCV_HAS_ZVKB()` macro.
Use zvkb for SM4 instead of zvbb.
Use zvkb for ghash instead of zvbb.
We could just use the zvbb's subset `zvkb` for flexibility.

Signed-off-by: Jerry Shih <jerry.shih@sifive.com>
Signed-off-by: Phoebe Chen <phoebe.chen@sifive.com>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/21923)

show more ...

riscv: GCM: Provide a Zvkg-based implementation

The upcoming RISC-V vector crypto extensions feature
a Zvkg extension, that provides a vghmac.vv instruction.
This patch provides an i

riscv: GCM: Provide a Zvkg-based implementation

The upcoming RISC-V vector crypto extensions feature
a Zvkg extension, that provides a vghmac.vv instruction.
This patch provides an implementation that utilizes this
extension if available.

Tested on QEMU and no regressions observed.

Signed-off-by: Christoph Müllner <christoph.muellner@vrull.eu>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/21923)

show more ...

riscv: GCM: Provide a Zvbb/Zvbc-based implementation

The RISC-V vector crypto extensions features a Zvbc extension
that provides a carryless multiplication ('vclmul.vv') instruction.

riscv: GCM: Provide a Zvbb/Zvbc-based implementation

The RISC-V vector crypto extensions features a Zvbc extension
that provides a carryless multiplication ('vclmul.vv') instruction.
This patch provides an implementation that utilizes this
extension if available.

Tested on QEMU and no regressions observed.

Signed-off-by: Christoph Müllner <christoph.muellner@vrull.eu>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/21923)

show more ...

Do not use stitched AES-GCM implementation on PPC32

The implementation is not usable there at all.
Fixes #21301

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: M

Do not use stitched AES-GCM implementation on PPC32

The implementation is not usable there at all.
Fixes #21301

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/21312)

(cherry picked from commit b256d32915d29bdcbc109363d3b05fce6dac30b7)

show more ...

Do not build P10-specific AES-GCM assembler on macOS

Reviewed-by: Tom Cosgrove <tom.cosgrove@arm.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/open

Do not build P10-specific AES-GCM assembler on macOS

Reviewed-by: Tom Cosgrove <tom.cosgrove@arm.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/20543)

show more ...

Do not build P10-specific AES-GCM assembler on AIX

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/ope

Do not build P10-specific AES-GCM assembler on AIX

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19865)

(cherry picked from commit 5c92ac52c282dbcb07e3880c2edd3a1bce452396)

show more ...

providers: Add SM4 XTS implementation

Signed-off-by: Xu Yizhou <xuyizhou1@huawei.com>

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>

providers: Add SM4 XTS implementation

Signed-off-by: Xu Yizhou <xuyizhou1@huawei.com>

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19619)

show more ...

Rename x86-32 assembly files from .s to .S.

Rename x86-32 assembly files from .s to .S. While processing the .S file
gcc will use the pre-processor whic will evaluate macros and ifdef. T

Rename x86-32 assembly files from .s to .S.

Rename x86-32 assembly files from .s to .S. While processing the .S file
gcc will use the pre-processor whic will evaluate macros and ifdef. This
is turn will be used to enable the endbr32 opcode based on the __CET__
define.

Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18353)

show more ...

Add clmul-based gmult for riscv64 with Zbb, Zbc

ghash-riscv64.pl implements 128-bit galois field multiplication for
use in the GCM mode using RISC-V carryless multiplication primitives.

Add clmul-based gmult for riscv64 with Zbb, Zbc

ghash-riscv64.pl implements 128-bit galois field multiplication for
use in the GCM mode using RISC-V carryless multiplication primitives.

The clmul-accelerated routine can be selected by setting the Zbb and
Zbc bits of the OPENSSL_riscvcap environment variable at runtime.

Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
Signed-off-by: Henry Brausen <henry.brausen@vrull.eu>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17640)

show more ...

AES-GCM enabled with AVX512 vAES and vPCLMULQDQ.

Vectorized 'stitched' encrypt + ghash implementation of AES-GCM enabled
with AVX512 vAES and vPCLMULQDQ instructions (available starting

AES-GCM enabled with AVX512 vAES and vPCLMULQDQ.

Vectorized 'stitched' encrypt + ghash implementation of AES-GCM enabled
with AVX512 vAES and vPCLMULQDQ instructions (available starting Intel's
IceLake micro-architecture).

The performance details for representative IceLake Server and Client
platforms are shown below

Performance data:
OpenSSL Speed KBs/Sec
Intel(R) Xeon(R) Platinum 8380 CPU @ 2.30GHz (1Core/1Thread)
Payload in Bytes 16 64 256 1024 8192 16384
AES-128-GCM
Baseline 478708.27 1118296.96 2428092.52 3518199.4 4172355.99 4235762.07
Patched 534613.95 2009345.55 3775588.15 5059517.64 8476794.88 8941541.79
Speedup 1.12 1.80 1.55 1.44 2.03 2.11

AES-256-GCM
Baseline 399237.27 961699.9 2136377.65 2979889.15 3554823.37 3617757.5
Patched 475948.13 1720128.51 3462407.12 4696832.2 7532013.16 7924953.91
Speedup 1.19 1.79 1.62 1.58 2.12 2.19
Intel(R) Core(TM) i7-1065G7 CPU @ 1.30GHz (1Core/1Thread)
Payload in Bytes 16 64 256 1024 8192 16384
AES-128-GCM
Baseline 259128.54 570756.43 1362554.16 1990654.57 2359128.88 2401671.58
Patched 292139.47 1079320.95 2001974.63 2829007.46 4510318.59 4705314.41
Speedup 1.13 1.89 1.47 1.42 1.91 1.96
AES-256-GCM
Baseline 236000.34 550506.76 1234638.08 1716734.57 2011255.6 2028099.99
Patched 247256.32 919731.34 1773270.43 2553239.55 3953115.14 4111227.29
Speedup 1.05 1.67 1.44 1.49 1.97 2.03

Reviewed-by: TJ O'Dwyer, Marcel Cornu, Pablo de Lara
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17239)

show more ...

Optimize AES-GCM for uarchs with unroll and new instructions

Increase the block numbers to 8 for every iteration. Increase the hash
table capacity. Make use of EOR3 instruction to impr

Optimize AES-GCM for uarchs with unroll and new instructions

Increase the block numbers to 8 for every iteration. Increase the hash
table capacity. Make use of EOR3 instruction to improve the performance.

This can improve performance 25-40% on out-of-order microarchitectures
with a large number of fast execution units, such as Neoverse V1. We also
see 20-30% performance improvements on other architectures such as the M1.

Assembly code reviewd by Tom Cosgrove (ARM).

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15916)

show more ...

AES-GCM performance optimzation with stitched method for p9+ ppc64le

Assembly code reviewed by Shricharan Srivatsan <ssrivat@us.ibm.com>

Reviewed-by: Tomas Mraz <tomas@openssl.org>

AES-GCM performance optimzation with stitched method for p9+ ppc64le

Assembly code reviewed by Shricharan Srivatsan <ssrivat@us.ibm.com>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16854)

show more ...

Drop libimplementations.a

libimplementations.a was a nice idea, but had a few flaws:

1. The idea to have common code in libimplementations.a and FIPS
sensitive helper funct

Drop libimplementations.a

libimplementations.a was a nice idea, but had a few flaws:

1. The idea to have common code in libimplementations.a and FIPS
sensitive helper functions in libfips.a / libnonfips.a didn't
catch on, and we saw full implementation ending up in them instead
and not appearing in libimplementations.a at all.

2. Because more or less ALL algorithm implementations were included
in libimplementations.a (the idea being that the appropriate
objects from it would be selected automatically by the linker when
building the shared libraries), it's very hard to find only the
implementation source that should go into the FIPS module, with
the result that the FIPS checksum mechanism include source files
that it shouldn't

To mitigate, we drop libimplementations.a, but retain the idea of
collecting implementations in static libraries. With that, we not
have:

libfips.a

Includes all implementations that should become part of the FIPS
provider.

liblegacy.a

Includes all implementations that should become part of the legacy
provider.

libdefault.a

Includes all implementations that should become part of the
default and base providers.

With this, libnonfips.a becomes irrelevant and is dropped.
libcommon.a is retained to include common provider code that can be
used uniformly by all providers.

Fixes #15157

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15171)

show more ...

Optimize AES-GCM implementation on aarch64

Comparing to current implementation, this change can get more
performance improved by tunning the loop-unrolling factor in
interleave imple

Optimize AES-GCM implementation on aarch64

Comparing to current implementation, this change can get more
performance improved by tunning the loop-unrolling factor in
interleave implementation as well as by enabling high level parallelism.

Performance(A72)

new
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
aes-128-gcm 113065.51k 375743.00k 848359.51k 1517865.98k 1964040.19k 1986663.77k
aes-192-gcm 110679.32k 364470.63k 799322.88k 1428084.05k 1826917.03k 1848967.17k
aes-256-gcm 104919.86k 352939.29k 759477.76k 1330683.56k 1663175.34k 1670430.72k

old
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
aes-128-gcm 115595.32k 382348.65k 855891.29k 1236452.35k 1425670.14k 1429793.45k
aes-192-gcm 112227.02k 369543.47k 810046.55k 1147948.37k 1286288.73k 1296941.06k
aes-256-gcm 111543.90k 361902.36k 769543.59k 1070693.03k 1208576.68k 1207511.72k

Change-Id: I28a2dca85c001a63a2a942e80c7c64f7a4fdfcf7

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9818)

show more ...

Fix missing Assembler defines

Implementations are now spread across several libraries, so the assembler
related defines need to be applied to all affected libraries and modules.

Fix missing Assembler defines

Implementations are now spread across several libraries, so the assembler
related defines need to be applied to all affected libraries and modules.

AES_ASM define was missing from libimplementations.a which disabled AESNI
aarch64 changes were made by xkqian.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10180)

show more ...

Rework how our providers are built

We put almost everything in these internal static libraries:

libcommon Block building code that can be used by all

Rework how our providers are built

We put almost everything in these internal static libraries:

libcommon Block building code that can be used by all
our implementations, legacy and non-legacy
alike.
libimplementations All non-legacy algorithm implementations and
only them. All the code that ends up here is
agnostic to the definitions of FIPS_MODE.
liblegacy All legacy implementations.

libnonfips Support code for the algorithm implementations.
Built with FIPS_MODE undefined. Any code that
checks that FIPS_MODE isn't defined must end
up in this library.
libfips Support code for the algorithm implementations.
Built with FIPS_MODE defined. Any code that
checks that FIPS_MODE is defined must end up
in this library.

The FIPS provider module is built from providers/fips/*.c and linked
with libimplementations, libcommon and libfips.

The Legacy provider module is built from providers/legacy/*.c and
linked with liblegacy, libcommon and libcrypto.
If module building is disabled, the object files from liblegacy and
libcommon are added to libcrypto and the Legacy provider becomes a
built-in provider.

The Default provider module is built-in, so it ends up being linked
with libimplementations, libcommon and libnonfips. For libcrypto in
form of static library, the object files from those other libraries
are simply being added to libcrypto.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10088)

show more ...

Add aes_wrap cipher to providers

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9406)

build.info: For all assembler generators, remove all arguments

Since the arguments are now generated in the build file templates,
they should be removed from the build.info files.

build.info: For all assembler generators, remove all arguments

Since the arguments are now generated in the build file templates,
they should be removed from the build.info files.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9884)

show more ...

Add aes_xts cipher to providers

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9327)

Add aes_ccm to provider

Add Cleanups for gcm - based on the changes to ccm.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Patrick Steuer <patrick.steuer@de.ibm

Add aes_ccm to provider

Add Cleanups for gcm - based on the changes to ccm.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Patrick Steuer <patrick.steuer@de.ibm.com>
(Merged from https://github.com/openssl/openssl/pull/9280)

show more ...

Add gcm ciphers (aes and aria) to providers.

The code has been modularized so that it can be shared by algorithms.

A fixed size IV is now used instead of being allocated.
The IV

Add gcm ciphers (aes and aria) to providers.

The code has been modularized so that it can be shared by algorithms.

A fixed size IV is now used instead of being allocated.
The IV is not set into the low level struct now until the update (it uses an
iv_state for this purpose).

Hardware specific methods have been added to a PROV_GCM_HW object.

The S390 code has been changed to just contain methods that can be accessed in
a modular way. There are equivalent generic methods also for the other
platforms.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Patrick Steuer <patrick.steuer@de.ibm.com>
(Merged from https://github.com/openssl/openssl/pull/9231)

show more ...

crypto/modes/build.conf: Fix MODES asm mistakes

The old rule in Configure was that if the asm source had a file name
with 'ghash-' as part of the name, GHASH_ASM should be defined. Sinc

crypto/modes/build.conf: Fix MODES asm mistakes

The old rule in Configure was that if the asm source had a file name
with 'ghash-' as part of the name, GHASH_ASM should be defined. Since
none of the aarch64 asm files has such a name, that macro shouldn't
have been defined.

Fixes #9173

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9178)

show more ...

Move modes_asm_src file information to build.info files

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9166)