"Reserve" the method store when constructing methods

Introducing the concept of reserving the store where a number of
provided operation methods are to be stored.

This avoids ra

"Reserve" the method store when constructing methods

Introducing the concept of reserving the store where a number of
provided operation methods are to be stored.

This avoids racing when constructing provided methods, which is
especially pertinent when multiple threads are trying to fetch the
same method, or even any implementation for the same given operation
type.

This introduces a |biglock| in OSSL_METHOD_STORE, which is separate
from the |lock| which is used for more internal and finer grained
locking.

Fixes #18152

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18153)

show more ...

Always try to construct methods as new provider might be added

Otherwise optional properties can be incorrectly ignored.

Fixes #18262

Reviewed-by: Matt Caswell <matt@openss

Always try to construct methods as new provider might be added

Otherwise optional properties can be incorrectly ignored.

Fixes #18262

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18269)

show more ...

Refactor method construction pre- and post-condition

The existing pre- and post-condition functions are supposed to check if
methods have already been created and stored, using provider

Refactor method construction pre- and post-condition

The existing pre- and post-condition functions are supposed to check if
methods have already been created and stored, using provider operation
bits. This is supposed to only be done for "permanent" method stores.

However, the way the pre-condition was called, it could not know if the
set of implementations to be stored is likely to end up in a "permanent"
or a temporary store. It needs access to the |no_store| flag returned
by the provider's operation query function, because that call was done
after the pre-condition was called.

This requires a bit of refactoring, primarly of |algorithm_do_this()|,
but also of |ossl_method_construct_precondition()|.

Fixes #18150

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18151)

show more ...

CORE: Encure that cached fetches can be done per provider

This mostly entails passing around a provider pointer, and handling
queries that includes a pointer to a provider, where NULL me

CORE: Encure that cached fetches can be done per provider

This mostly entails passing around a provider pointer, and handling
queries that includes a pointer to a provider, where NULL means "any".

This also means that there's a need to pass the provider pointer, not
just down to the cache functions, but also be able to get it from
ossl_method_store_fetch(). To this end, that function's OSSL_PROVIDER
pointer argument is modified to be a pointer reference, so the
function can answer back what provider the method comes from.

Test added.

Fixes #16614

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16725)

show more ...

CORE: add a provider argument to ossl_method_construct()

This makes it possible to limit the search of methods to that
particular provider. This uses already available possibilities in

CORE: add a provider argument to ossl_method_construct()

This makes it possible to limit the search of methods to that
particular provider. This uses already available possibilities in
ossl_algorithm_do_all().

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16725)

show more ...

CORE: Do a bit of cleanup of core fetching

Some data, like the library context, were passed both through higher
level callback structures and through arguments to those same higher
l

CORE: Do a bit of cleanup of core fetching

Some data, like the library context, were passed both through higher
level callback structures and through arguments to those same higher
level callbacks. This is a bit unnecessary, so we rearrange the
callback arguments to simply pass that callback structure and rely on
the higher level fetching functionality to pick out what data they
need from that structure.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15750)

show more ...

CORE: Move away the allocation of the temporary no_cache method store

The responsibility for managing the temporary store for methods from
algorithm implementations flaged "no_store" is

CORE: Move away the allocation of the temporary no_cache method store

The responsibility for managing the temporary store for methods from
algorithm implementations flaged "no_store" is moved up to the diverse
method fetching functions. This allows them to allocate it "just in
time", or in other words not at all if there is not such algorithm
implementation.

This makes this temporary store more flexible if it's needed outside
of the core fetching functionality, and slightly faster when this
temporary store isn't necessary at all.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15737)

show more ...

Update copyright year

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13999)

Fix a failure where fetches can return NULL in multi-threaded code

When a fetch is attempted simultaneously from multiple threads then both
threads can attempt to construct the method. H

Fix a failure where fetches can return NULL in multi-threaded code

When a fetch is attempted simultaneously from multiple threads then both
threads can attempt to construct the method. However only one of those
will get added to the global evp method store. The one that "lost" the
race to add the method to the global evp method store ended up with the
fetch call returning NULL, instead of returning the method that was
already available.

Fixes #13682

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13660)

show more ...

Rename OPENSSL_CTX prefix to OSSL_LIB_CTX

Many of the new types introduced by OpenSSL 3.0 have an OSSL_ prefix,
e.g., OSSL_CALLBACK, OSSL_PARAM, OSSL_ALGORITHM, OSSL_SERIALIZER.

Rename OPENSSL_CTX prefix to OSSL_LIB_CTX

Many of the new types introduced by OpenSSL 3.0 have an OSSL_ prefix,
e.g., OSSL_CALLBACK, OSSL_PARAM, OSSL_ALGORITHM, OSSL_SERIALIZER.

The OPENSSL_CTX type stands out a little by using a different prefix.
For consistency reasons, this type is renamed to OSSL_LIB_CTX.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12621)

show more ...

Update copyright year

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12043)

CORE: query for operations only once per provider (unless no_store is true)

When a desired algorithm wasn't available, we didn't register anywhere
that an attempt had been made, with the

CORE: query for operations only once per provider (unless no_store is true)

When a desired algorithm wasn't available, we didn't register anywhere
that an attempt had been made, with the result that next time the same
attempt was made, the whole process would be done again.

To avoid this churn, we register a bit for each operation that has
been queried in the libcrypto provider object, and test it before
trying the same query and method construction loop again.

If course, if the provider has told us not to cache, we don't register
this bit.

Fixes #11814

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11842)

show more ...

CORE: pass the full algorithm definition to the method constructor

So far, the API level method constructors that are called by
ossl_method_construct_this() were passed the algorithm nam

CORE: pass the full algorithm definition to the method constructor

So far, the API level method constructors that are called by
ossl_method_construct_this() were passed the algorithm name string and
the dispatch table and had no access to anything else.

This change gives them access to the full OSSL_ALGORITHM item, thereby
giving them access to the property definition.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10394)

show more ...

Replumbing: make it possible for providers to specify multiple names

This modifies the treatment of algorithm name strings to allow
multiple names separated with colons.

Reviewe

Replumbing: make it possible for providers to specify multiple names

This modifies the treatment of algorithm name strings to allow
multiple names separated with colons.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/8985)

show more ...

In provider implemented methods, save the name number, not the name string

Multiple names per implementation is already supported in the namemap,
but hasn't been used yet. However, as s

In provider implemented methods, save the name number, not the name string

Multiple names per implementation is already supported in the namemap,
but hasn't been used yet. However, as soon as we have multiple names,
we will get an issue with what name should be saved in the method.

The solution is to not save the name itself, but rather the number
it's associated with. This number is supposed to be unique for each
set of names, and we assume that algorithm names are globally unique,
i.e. there can be no name overlap between different algorithm types.

Incidently, it was also found that the 'get' function used by
ossl_construct_method() doesn't need all the parameters it was given;
most of what it needs, it can now get through the data structure given
by the caller of ossl_construct_method(). As a consequence,
ossl_construct_method() itself doesn't need all the parameters it was
given either.

There are some added internal functions that are expected to disappear
as soon as legacy code is removed, such as evp_first_name() and
ossl_namemap_num2name().

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9897)

show more ...

Modify ossl_method_store_add() to accept an OSSL_PROVIDER and check for it

If ossl_method_store_add() gets called with a method that already exists
(i.e. the store has one with matching

Modify ossl_method_store_add() to accept an OSSL_PROVIDER and check for it

If ossl_method_store_add() gets called with a method that already exists
(i.e. the store has one with matching provider, nid and properties), that
method should not be stored. We do this check inside ossl_method_store_add()
because it has all the locking required to do so safely.

Fixes #9561

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9650)

show more ...

Refactor ossl_method_construct() in terms of ossl_algorithm_do_all()

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9356)

EVP fetching: make operation_id part of the method identity

Because the operation identity wasn't integrated with the created
methods, the following code would give unexpected results:

EVP fetching: make operation_id part of the method identity

Because the operation identity wasn't integrated with the created
methods, the following code would give unexpected results:

EVP_MD *md = EVP_MD_fetch(NULL, "MD5", NULL);
EVP_CIPHER *cipher = EVP_CIPHER_fetch(NULL, "MD5", NULL);

if (md != NULL)
printf("MD5 is a digest\n");
if (cipher != NULL)
printf("MD5 is a cipher\n");

The message is that MD5 is both a digest and a cipher.

Partially fixes #9106

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9109)

show more ...

Move digests to providers

Move digest code into the relevant providers (fips, default, legacy).
The headers are temporarily moved to be internal, and will be moved
into providers aft

Move digests to providers

Move digest code into the relevant providers (fips, default, legacy).
The headers are temporarily moved to be internal, and will be moved
into providers after all external references are resolved. The deprecated
digest code can not be removed until EVP_PKEY (signing) is supported by
providers. EVP_MD data can also not yet be cleaned up for the same reasons.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8763)

show more ...

EVP_FETCH: deal with names without pre-defined NIDs

We didn't deal very well with names that didn't have pre-defined NIDs,
as the NID zero travelled through the full process and resulted

EVP_FETCH: deal with names without pre-defined NIDs

We didn't deal very well with names that didn't have pre-defined NIDs,
as the NID zero travelled through the full process and resulted in an
inaccessible method. By consequence, we need to refactor the method
construction callbacks to rely more on algorithm names.

We must, however, still store the legacy NID with the method, for the
sake of other code that depend on it (for example, CMS).

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8878)

show more ...

Instead of global data store it in an OPENSSL_CTX

Various core and property related code files used global data. We should
store all of that in an OPENSSL_CTX instead.

Reviewed-

Instead of global data store it in an OPENSSL_CTX

Various core and property related code files used global data. We should
store all of that in an OPENSSL_CTX instead.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8857)

show more ...

Use the right NID when putting a method in the store

When we attempt to fetch a method with a given NID we will ask the
providers for it if we don't already know about it. During that pr

Use the right NID when putting a method in the store

When we attempt to fetch a method with a given NID we will ask the
providers for it if we don't already know about it. During that process
we may be told about other methods with a different NID. We need to
make sure we don't confuse the two.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8541)

show more ...

Replumbing: better reference counter control in ossl_method_construct()

Fully assume that the method constructors use reference counting.
Otherwise, we may leak memory, or loose track an

Replumbing: better reference counter control in ossl_method_construct()

Fully assume that the method constructors use reference counting.
Otherwise, we may leak memory, or loose track and do a double free.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8341)

show more ...

Replumbing: pass callback data to the algo destructor too

All relevant OSSL_METHOD_CONSTRUCT_METHOD callbacks got the callback
data passed to them, except 'destruct'. There's no reason

Replumbing: pass callback data to the algo destructor too

All relevant OSSL_METHOD_CONSTRUCT_METHOD callbacks got the callback
data passed to them, except 'destruct'. There's no reason why it
shouldn't get that pointer passed, so we make a small adjustment.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8341)

show more ...

Replumbing: Add constructor of libcrypto internal method structures

This queries the provider for its available functionality (unless a
matching method structured is already cached, in w

Replumbing: Add constructor of libcrypto internal method structures

This queries the provider for its available functionality (unless a
matching method structured is already cached, in which case that's
used instead), and creates method structure with the help of a passed
constructor. The result is cached if the provider allows it (or if
caching is forced).

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8340)

show more ...