Eric Biggers [Wed, 8 Jul 2020 16:39:40 +0000 (09:39 -0700)]
crypto: lib/sha256 - add sha256() function
Add a function sha256() which computes a SHA-256 digest in one step,
combining sha256_init() + sha256_update() + sha256_final().
This is similar to how we also have blake2s().
Reviewed-by: Ard Biesheuvel <ardb@kernel.org> Tested-by: Hans de Goede <hdegoede@redhat.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Eric Biggers [Wed, 8 Jul 2020 16:39:39 +0000 (09:39 -0700)]
crypto: sparc - rename sha256 to sha256_alg
To avoid a naming collision when we add a sha256() library function,
rename the "sha256" static variable in sha256_glue.c to "sha256_alg".
For consistency, also rename "sha224" to "sha224_alg".
Reported-by: kernel test robot <lkp@intel.com> Cc: sparclinux@vger.kernel.org Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: x86/chacha-sse3 - use unaligned loads for state array
Due to the fact that the x86 port does not support allocating objects
on the stack with an alignment that exceeds 8 bytes, we have a rather
ugly hack in the x86 code for ChaCha to ensure that the state array is
aligned to 16 bytes, allowing the SSE3 implementation of the algorithm
to use aligned loads.
Given that the performance benefit of using of aligned loads appears to
be limited (~0.25% for 1k blocks using tcrypt on a Corei7-8650U), and
the fact that this hack has leaked into generic ChaCha code, let's just
remove it.
Cc: Martin Willi <martin@strongswan.org> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: Eric Biggers <ebiggers@kernel.org> Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Martin Willi <martin@strongswan.org> Reviewed-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: mediatek - use AES library for GCM key derivation
The Mediatek accelerator driver calls into a dynamically allocated
skcipher of the ctr(aes) variety to perform GCM key derivation, which
involves AES encryption of a single block consisting of NUL bytes.
There is no point in using the skcipher API for this, so use the AES
library interface instead.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: sahara - permit asynchronous skcipher as fallback
Even though the sahara driver implements asynchronous versions of
ecb(aes) and cbc(aes), the fallbacks it allocates are required to be
synchronous. Given that SIMD based software implementations are usually
asynchronous as well, even though they rarely complete asynchronously
(this typically only happens in cases where the request was made from
softirq context, while SIMD was already in use in the task context that
it interrupted), these implementations are disregarded, and either the
generic C version or another table based version implemented in assembler
is selected instead.
Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.
crypto: qce - permit asynchronous skcipher as fallback
Even though the qce driver implements asynchronous versions of ecb(aes),
cbc(aes)and xts(aes), the fallbacks it allocates are required to be
synchronous. Given that SIMD based software implementations are usually
asynchronous as well, even though they rarely complete asynchronously
(this typically only happens in cases where the request was made from
softirq context, while SIMD was already in use in the task context that
it interrupted), these implementations are disregarded, and either the
generic C version or another table based version implemented in assembler
is selected instead.
Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.
While at it, remove the pointless memset() from qce_skcipher_init(), and
remove the call to it qce_skcipher_init_fallback().
Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: picoxcell - permit asynchronous skcipher as fallback
Even though the picoxcell driver implements asynchronous versions of
ecb(aes) and cbc(aes), the fallbacks it allocates are required to be
synchronous. Given that SIMD based software implementations are usually
asynchronous as well, even though they rarely complete asynchronously
(this typically only happens in cases where the request was made from
softirq context, while SIMD was already in use in the task context that
it interrupted), these implementations are disregarded, and either the
generic C version or another table based version implemented in assembler
is selected instead.
Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.
crypto: mxs-dcp - permit asynchronous skcipher as fallback
Even though the mxs-dcp driver implements asynchronous versions of
ecb(aes) and cbc(aes), the fallbacks it allocates are required to be
synchronous. Given that SIMD based software implementations are usually
asynchronous as well, even though they rarely complete asynchronously
(this typically only happens in cases where the request was made from
softirq context, while SIMD was already in use in the task context that
it interrupted), these implementations are disregarded, and either the
generic C version or another table based version implemented in assembler
is selected instead.
Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.
crypto: chelsio - permit asynchronous skcipher as fallback
Even though the chelsio driver implements asynchronous versions of
cbc(aes) and xts(aes), the fallbacks it allocates are required to be
synchronous. Given that SIMD based software implementations are usually
asynchronous as well, even though they rarely complete asynchronously
(this typically only happens in cases where the request was made from
softirq context, while SIMD was already in use in the task context that
it interrupted), these implementations are disregarded, and either the
generic C version or another table based version implemented in assembler
is selected instead.
Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: ccp - permit asynchronous skcipher as fallback
Even though the ccp driver implements an asynchronous version of xts(aes),
the fallback it allocates is required to be synchronous. Given that SIMD
based software implementations are usually asynchronous as well, even
though they rarely complete asynchronously (this typically only happens
in cases where the request was made from softirq context, while SIMD was
already in use in the task context that it interrupted), these
implementations are disregarded, and either the generic C version or
another table based version implemented in assembler is selected instead.
Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Acked-by: John Allen <john.allen@amd.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: sun8i-ss - permit asynchronous skcipher as fallback
Even though the sun8i-ss driver implements asynchronous versions of
ecb(aes) and cbc(aes), the fallbacks it allocates are required to be
synchronous. Given that SIMD based software implementations are usually
asynchronous as well, even though they rarely complete asynchronously
(this typically only happens in cases where the request was made from
softirq context, while SIMD was already in use in the task context that
it interrupted), these implementations are disregarded, and either the
generic C version or another table based version implemented in assembler
is selected instead.
Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.
crypto: sun8i-ce - permit asynchronous skcipher as fallback
Even though the sun8i-ce driver implements asynchronous versions of
ecb(aes) and cbc(aes), the fallbacks it allocates are required to be
synchronous. Given that SIMD based software implementations are usually
asynchronous as well, even though they rarely complete asynchronously
(this typically only happens in cases where the request was made from
softirq context, while SIMD was already in use in the task context that
it interrupted), these implementations are disregarded, and either the
generic C version or another table based version implemented in assembler
is selected instead.
Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.
crypto: sun4i - permit asynchronous skcipher as fallback
Even though the sun4i driver implements asynchronous versions of ecb(aes)
and cbc(aes), the fallbacks it allocates are required to be synchronous.
Given that SIMD based software implementations are usually asynchronous
as well, even though they rarely complete asynchronously (this typically
only happens in cases where the request was made from softirq context,
while SIMD was already in use in the task context that it interrupted),
these implementations are disregarded, and either the generic C version
or another table based version implemented in assembler is selected
instead.
Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.
crypto: omap-aes - permit asynchronous skcipher as fallback
Even though the omap-aes driver implements asynchronous versions of
ecb(aes), cbc(aes) and ctr(aes), the fallbacks it allocates are required
to be synchronous. Given that SIMD based software implementations are
usually asynchronous as well, even though they rarely complete
asynchronously (this typically only happens in cases where the request was
made from softirq context, while SIMD was already in use in the task
context that it interrupted), these implementations are disregarded, and
either the generic C version or another table based version implemented in
assembler is selected instead.
Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: amlogic-gxl - permit async skcipher as fallback
Even though the amlogic-gxl driver implements asynchronous versions of
ecb(aes) and cbc(aes), the fallbacks it allocates are required to be
synchronous. Given that SIMD based software implementations are usually
asynchronous as well, even though they rarely complete asynchronously
(this typically only happens in cases where the request was made from
softirq context, while SIMD was already in use in the task context that
it interrupted), these implementations are disregarded, and either the
generic C version or another table based version implemented in assembler
is selected instead.
Since falling back to synchronous AES is not only a performance issue,
but potentially a security issue as well (due to the fact that table
based AES is not time invariant), let's fix this, by allocating an
ordinary skcipher as the fallback, and invoke it with the completion
routine that was given to the outer request.
The AmLogic GXL crypto accelerator driver is built into the kernel if
ARCH_MESON is set. However, given the single image policy of arm64, its
defconfig enables all platforms by default, and so ARCH_MESON is usually
enabled.
This means that the AmLogic driver causes the arm64 defconfig build to
pull in a huge chunk of the crypto stack as a builtin as well, which is
undesirable, so let's make the amlogic GXL driver default to 'm' instead.
Herbert Xu [Tue, 7 Jul 2020 03:47:13 +0000 (13:47 +1000)]
hwrng: ba431 - Include kernel.h
There are multiple things in this file that requires kernel.h but
it's only included through other header files indirectly. This
patch adds a direct inclusion as those indirect inclusions may go
away at any point.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
As before, if a SEC queue is at the 'fake busy' status,
the request with a 'fake busy' flag will be sent into hardware
and the sending function returns busy. After the request is
finished, SEC driver's call back will identify the 'fake busy' flag,
and notifies the user that hardware is not busy now by calling
user's call back function.
Now, a request sent into busy hardware will be cached in the
SEC queue's backlog, return '-EBUSY' to user.
After the request being finished, the cached requests will
be processed in the call back function. to notify the
corresponding user that SEC queue can process more requests.
Signed-off-by: Kai Ye <yekai13@huawei.com> Reviewed-by: Longfang Liu <liulongfang@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Thu, 16 Jul 2020 11:45:03 +0000 (21:45 +1000)]
crypto: caam - Remove broken arc4 support
The arc4 algorithm requires storing state in the request context
in order to allow more than one encrypt/decrypt operation. As this
driver does not seem to do that, it means that using it for more
than one operation is broken.
Ard Biesheuvel [Mon, 29 Jun 2020 07:39:25 +0000 (09:39 +0200)]
crypto: arm/ghash - use variably sized key struct
Of the two versions of GHASH that the ARM driver implements, only one
performs aggregation, and so the other one has no use for the powers
of H to be precomputed, or space to be allocated for them in the key
struct. So make the context size dependent on which version is being
selected, and while at it, use a static key to carry this decision,
and get rid of the function pointer.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ard Biesheuvel [Mon, 29 Jun 2020 07:39:24 +0000 (09:39 +0200)]
crypto: arm64/gcm - use inline helper to suppress indirect calls
Introduce an inline wrapper for ghash_do_update() that incorporates
the indirect call to the asm routine that is passed as an argument,
and keep the non-SIMD fallback code out of line. This ensures that
all references to the function pointer are inlined where the address
is taken, removing the need for any indirect calls to begin with.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ard Biesheuvel [Mon, 29 Jun 2020 07:39:23 +0000 (09:39 +0200)]
crypto: arm64/gcm - use variably sized key struct
Now that the ghash and gcm drivers are split, we no longer need to allocate
a key struct for the former that carries powers of H that are only used by
the latter. Also, take this opportunity to clean up the code a little bit.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ard Biesheuvel [Mon, 29 Jun 2020 07:39:22 +0000 (09:39 +0200)]
crypto: arm64/gcm - disentangle ghash and gcm setkey() routines
The remaining ghash implementation does not support aggregation, and so
there is no point in including the precomputed powers of H in the key
struct. So move that into the GCM setkey routine, and get rid of the
shared sub-routine entirely.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ard Biesheuvel [Mon, 29 Jun 2020 07:39:21 +0000 (09:39 +0200)]
crypto: arm64/ghash - drop PMULL based shash
There are two ways to implement SIMD accelerated GCM on arm64:
- using the PMULL instructions for carryless 64x64->128 multiplication,
in which case the architecture guarantees that the AES instructions are
available as well, and so we can use the AEAD implementation that combines
both,
- using the PMULL instructions for carryless 8x8->16 bit multiplication,
which is implemented as a shash, and can be combined with any ctr(aes)
implementation by the generic GCM AEAD template driver.
So let's drop the 64x64->128 shash driver, which is never needed for GCM,
and not suitable for use anywhere else.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Barry Song [Sun, 5 Jul 2020 09:18:59 +0000 (21:18 +1200)]
crypto: hisilicon/zip - permit users to specify NUMA node
If users don't specify NUMA node, the driver will use the ZIP module near
the CPU allocating acomp. Otherwise, it uses the ZIP module according to
the requirement of users.
Cc: Zhou Wang <wangzhou1@hisilicon.com> Signed-off-by: Barry Song <song.bao.hua@hisilicon.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Barry Song [Sun, 5 Jul 2020 09:18:58 +0000 (21:18 +1200)]
crypto: api - permit users to specify numa node of acomp hardware
For a Linux server with NUMA, there are possibly multiple (de)compressors
which are either local or remote to some NUMA node. Some drivers will
automatically use the (de)compressor near the CPU calling acomp_alloc().
However, it is not necessarily correct because users who send acomp_req
could be from different NUMA node with the CPU which allocates acomp.
Just like kernel has kmalloc() and kmalloc_node(), here crypto can have
same support.
Cc: Seth Jennings <sjenning@redhat.com> Cc: Dan Streetman <ddstreet@ieee.org> Cc: Vitaly Wool <vitaly.wool@konsulko.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Jonathan Cameron <Jonathan.Cameron@huawei.com> Signed-off-by: Barry Song <song.bao.hua@hisilicon.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When building with LLVM_IAS=1 means using Clang's Integrated Assembly (IAS)
from LLVM/Clang >= v10.0.1-rc1+ instead of GNU/as from GNU/binutils
I see the following breakage in Debian/testing AMD64:
<instantiation>:15:74: error: too many positional arguments
PRECOMPUTE 8*3+8(%rsp), %xmm1, %xmm2, %xmm3, %xmm4, %xmm5, %xmm6, %xmm7,
^
arch/x86/crypto/aesni-intel_asm.S:1598:2: note: while in macro instantiation
GCM_INIT %r9, 8*3 +8(%rsp), 8*3 +16(%rsp), 8*3 +24(%rsp)
^
<instantiation>:47:2: error: unknown use of instruction mnemonic without a size suffix
GHASH_4_ENCRYPT_4_PARALLEL_dec %xmm9, %xmm10, %xmm11, %xmm12, %xmm13, %xmm14, %xmm0, %xmm1, %xmm2, %xmm3, %xmm4, %xmm5, %xmm6, %xmm7, %xmm8, enc
^
arch/x86/crypto/aesni-intel_asm.S:1599:2: note: while in macro instantiation
GCM_ENC_DEC dec
^
<instantiation>:15:74: error: too many positional arguments
PRECOMPUTE 8*3+8(%rsp), %xmm1, %xmm2, %xmm3, %xmm4, %xmm5, %xmm6, %xmm7,
^
arch/x86/crypto/aesni-intel_asm.S:1686:2: note: while in macro instantiation
GCM_INIT %r9, 8*3 +8(%rsp), 8*3 +16(%rsp), 8*3 +24(%rsp)
^
<instantiation>:47:2: error: unknown use of instruction mnemonic without a size suffix
GHASH_4_ENCRYPT_4_PARALLEL_enc %xmm9, %xmm10, %xmm11, %xmm12, %xmm13, %xmm14, %xmm0, %xmm1, %xmm2, %xmm3, %xmm4, %xmm5, %xmm6, %xmm7, %xmm8, enc
^
arch/x86/crypto/aesni-intel_asm.S:1687:2: note: while in macro instantiation
GCM_ENC_DEC enc
Craig Topper suggested me in ClangBuiltLinux issue #1050:
> I think the "too many positional arguments" is because the parser isn't able
> to handle the trailing commas.
>
> The "unknown use of instruction mnemonic" is because the macro was named
> GHASH_4_ENCRYPT_4_PARALLEL_DEC but its being instantiated with
> GHASH_4_ENCRYPT_4_PARALLEL_dec I guess gas ignores case on the
> macro instantiation, but llvm doesn't.
First, I removed the trailing comma in the PRECOMPUTE line.
Herbert Xu [Fri, 3 Jul 2020 04:46:52 +0000 (14:46 +1000)]
crypto: ccp - Fix sparse warnings
This patch fixes a number of endianness marking issues in the ccp
driver.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Acked-by: John Allen <john.allen@amd.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Thu, 2 Jul 2020 03:32:21 +0000 (13:32 +1000)]
crypto: af_alg - Fix regression on empty requests
Some user-space programs rely on crypto requests that have no
control metadata. This broke when a check was added to require
the presence of control metadata with the ctx->init flag.
This patch fixes the regression by setting ctx->init as long as
one sendmsg(2) has been made, with or without a control message.
Reported-by: Sachin Sant <sachinp@linux.vnet.ibm.com> Reported-by: Naresh Kamboju <naresh.kamboju@linaro.org> Fixes: f3c802a1f300 ("crypto: algif_aead - Only wake up when...") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Tue, 30 Jun 2020 04:20:37 +0000 (14:20 +1000)]
hwrng: ba431 - Add dependency on HAS_IOMEM
The ba431 driver depends on HAS_IOMEM and this was missing from
the Kconfig file.
Reported-by: kernel test robot <lkp@intel.com> Fixes: 0289e9be5dc2 ("hwrng: ba431 - add support for BA431 hwrng") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Mon, 29 Jun 2020 08:03:55 +0000 (18:03 +1000)]
hwrng: omap - Fix W=1 unused variable warning
This patch fixes an unused variable warning when this driver is
built-in with CONFIG_OF=n. While we're at it this patch also
expands the compiler coverage when CONFIG_OF is off by removing
all the CONFIG_OF ifdefs.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Lee Jones [Mon, 29 Jun 2020 12:30:03 +0000 (13:30 +0100)]
crypto: ux500/hash - Add namespacing to hash_init()
A recent change to the Regulator consumer API (which this driver
utilises) add prototypes for the some suspend functions. These
functions require including header file include/linux/suspend.h.
The following tree of includes affecting this driver will be
present:
In file included from include/linux/elevator.h:6,
from include/linux/blkdev.h:288,
from include/linux/blk-cgroup.h:23,
from include/linux/writeback.h:14,
from include/linux/memcontrol.h:22,
from include/linux/swap.h:9,
from include/linux/suspend.h:5,
from include/linux/regulator/consumer.h:35,
from drivers/crypto/ux500/hash/hash_core.c:28:
include/linux/elevator.h pulls in include/linux/hashtable.h which
contains its own version of hash_init(). This confuses the build
system and results in the following error (amongst others):
drivers/crypto/ux500/hash/hash_core.c:1362:19: error: passing argument 1 of '__hash_init' from incompatible pointer type [-Werror=incompatible-pointer-types]
1362 | return hash_init(req);
Fix this by namespacing the local hash_init() such that the
source of confusion is removed.
Cc: Linus Walleij <linus.walleij@linaro.org> Cc: David S. Miller <davem@davemloft.net> Cc: linux-crypto@vger.kernel.org Signed-off-by: Lee Jones <lee.jones@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Tom Lendacky [Fri, 26 Jun 2020 19:09:39 +0000 (14:09 -0500)]
crypto: ccp - Update CCP driver maintainer information
Add John Allen as a new CCP driver maintainer. Additionally, break out
the driver SEV support and create a new maintainer entry, with Brijesh
Singh and Tom Lendacky as maintainers.
Cc: John Allen <john.allen@amd.com> Cc: Brijesh Singh <brijesh.singh@amd.com> Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com> Acked-by: John Allen <john.allen@amd.com> Acked-by: Brijesh Singh <brijesh.singh@amd.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Fri, 26 Jun 2020 06:06:06 +0000 (16:06 +1000)]
crypto: cpt - Fix sparse warnings
This patch fixes all the sparse warnings in the octeontx driver.
Some of these are just trivial type changes.
However, some of the changes are non-trivial on little-endian hosts.
Obviously the driver appears to be broken on either LE or BE as it
was doing different things. I've taken the BE behaviour as the
correct one.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Fenghua Yu [Thu, 25 Jun 2020 20:07:12 +0000 (13:07 -0700)]
crypto: hisilicon/qm - Change type of pasid to u32
PASID is defined as "int" although it's a 20-bit value and shouldn't be
negative int. To be consistent with PASID type in iommu, define PASID
as "u32".
Suggested-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Fenghua Yu <fenghua.yu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Jian Cai [Mon, 22 Jun 2020 23:24:33 +0000 (16:24 -0700)]
crypto: aesni - add compatibility with IAS
Clang's integrated assembler complains "invalid reassignment of
non-absolute variable 'var_ddq_add'" while assembling
arch/x86/crypto/aes_ctrby8_avx-x86_64.S. It was because var_ddq_add was
reassigned with non-absolute values several times, which IAS did not
support. We can avoid the reassignment by replacing the uses of
var_ddq_add with its definitions accordingly to have compatilibility
with IAS.
Link: https://github.com/ClangBuiltLinux/linux/issues/1008 Reported-by: Sedat Dilek <sedat.dilek@gmail.com> Reported-by: Fangrui Song <maskray@google.com> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # build+boot Linux v5.7.5; clang v11.0.0-git Signed-off-by: Jian Cai <caij2003@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
John Allen [Mon, 22 Jun 2020 20:24:02 +0000 (15:24 -0500)]
crypto: ccp - Fix use of merged scatterlists
Running the crypto manager self tests with
CONFIG_CRYPTO_MANAGER_EXTRA_TESTS may result in several types of errors
when using the ccp-crypto driver:
alg: skcipher: cbc-des3-ccp encryption failed on test vector 0; expected_error=0, actual_error=-5 ...
alg: skcipher: ctr-aes-ccp decryption overran dst buffer on test vector 0 ...
alg: ahash: sha224-ccp test failed (wrong result) on test vector ...
These errors are the result of improper processing of scatterlists mapped
for DMA.
Given a scatterlist in which entries are merged as part of mapping the
scatterlist for DMA, the DMA length of a merged entry will reflect the
combined length of the entries that were merged. The subsequent
scatterlist entry will contain DMA information for the scatterlist entry
after the last merged entry, but the non-DMA information will be that of
the first merged entry.
The ccp driver does not take this scatterlist merging into account. To
address this, add a second scatterlist pointer to track the current
position in the DMA mapped representation of the scatterlist. Both the DMA
representation and the original representation of the scatterlist must be
tracked as while most of the driver can use just the DMA representation,
scatterlist_map_and_copy() must use the original representation and
expects the scatterlist pointer to be accurate to the original
representation.
In order to properly walk the original scatterlist, the scatterlist must
be walked until the combined lengths of the entries seen is equal to the
DMA length of the current entry being processed in the DMA mapped
representation.
Fixes: 63b945091a070 ("crypto: ccp - CCP device driver and interface support") Signed-off-by: John Allen <john.allen@amd.com> Cc: stable@vger.kernel.org Acked-by: Tom Lendacky <thomas.lendacky@amd.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: qce/sha - Do not modify scatterlist passed along with request
Crypto test driver's test_ahash_speed calls crypto_ahash_update and
crypto_ahash_final APIs repeatedly for all the available test vector
buffer lengths.
if we mark the end for scatterlist based on the current vector size then
the subsequent vectors might fail if the later buffer lengths are higher.
To avoid this, in qce do not mark the end of scatterlist in update API,
the qce_ahash_async_req_handle API already takes care of this copying
right amount of buffer from the request scatter list.
Signed-off-by: Sivaprakash Murugesan <sivaprak@codeaurora.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto testmgr deliberately corrupts the request context while passing
vectors to the import. This is to make sure that drivers do not rely on
request but they take all the necessary input from io vec passed to it.
qce casts the request context from request parameter, since it is corrupted
the sub squent hash request fails and qce hangs.
To avoid this re-initialize request context on import. The qce import
API alreasy takes care of taking the input vectors from passed io vec.
Signed-off-by: Sivaprakash Murugesan <sivaprak@codeaurora.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto test module passes zero length vectors as test input to sha-1 and
sha-256. To provide correct output for these vectors, hash zero support
has been added as in other crypto drivers.
Signed-off-by: Sivaprakash Murugesan <sivaprak@codeaurora.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Dinghao Liu [Mon, 22 Jun 2020 02:40:08 +0000 (10:40 +0800)]
crypto: sun8i-ce - Fix runtime PM imbalance in sun8i_ce_cipher_init
pm_runtime_get_sync() increments the runtime PM usage counter even
the call returns an error code. Thus a corresponding decrement is
needed on the error handling path to keep the counter balanced.
Fix this by adding the missed function call.
Signed-off-by: Dinghao Liu <dinghao.liu@zju.edu.cn> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Colin Ian King [Thu, 18 Jun 2020 10:16:25 +0000 (11:16 +0100)]
crypto: img-hash - remove redundant initialization of variable err
The variable err is being initialized with a value that is never read
and it is being updated later with a new value. The initialization is
redundant and can be removed.
Addresses-Coverity: ("Unused value") Signed-off-by: Colin Ian King <colin.king@canonical.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Colin Ian King [Thu, 18 Jun 2020 10:12:29 +0000 (11:12 +0100)]
crypto: ccp - remove redundant assignment to variable ret
The variable ret is being assigned with a value that is never read
and it is being updated later with a new value. The assignment is
redundant and can be removed.
Addresses-Coverity: ("Unused value") Signed-off-by: Colin Ian King <colin.king@canonical.com> Acked-by: Tom Lendacky <thomas.lendacky@amd.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Thu, 18 Jun 2020 07:00:22 +0000 (17:00 +1000)]
crypto: api - Prune inclusions in crypto.h
We haven't used string.h since the memcpy calls were removed so
this patch removes its inclusion. The file uaccess.h isn't needed
at all. However, removing it reveals that we do need to add an
inclusion for refcount.h.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Mikulas Patocka [Wed, 17 Jun 2020 13:49:52 +0000 (09:49 -0400)]
crypto: hisilicon - don't sleep of CRYPTO_TFM_REQ_MAY_SLEEP was not specified
There is this call chain:
sec_alg_skcipher_encrypt -> sec_alg_skcipher_crypto ->
sec_alg_alloc_and_calc_split_sizes -> kcalloc
where we call sleeping allocator function even if CRYPTO_TFM_REQ_MAY_SLEEP
was not specified.
Mikulas Patocka [Wed, 17 Jun 2020 13:48:56 +0000 (09:48 -0400)]
crypto: cpt - don't sleep of CRYPTO_TFM_REQ_MAY_SLEEP was not specified
There is this call chain:
cvm_encrypt -> cvm_enc_dec -> cptvf_do_request -> process_request -> kzalloc
where we call sleeping allocator function even if CRYPTO_TFM_REQ_MAY_SLEEP
was not specified.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> Cc: stable@vger.kernel.org # v4.11+ Fixes: c694b233295b ("crypto: cavium - Add the Virtual Function driver for CPT") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Mon, 15 Jun 2020 11:37:38 +0000 (21:37 +1000)]
crypto: omap-sham - Fix sparse/compiler warnings
This patch fixes sparse endianness warnings as well as compiler
warnings on 64-bit hosts.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Reviewed-by: Tero Kristo <t-kristo@ti.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Mon, 15 Jun 2020 11:36:20 +0000 (21:36 +1000)]
crypto: omap-des - Fix sparse/compiler warnings
This patch fixes sparse endianness warnings as well as compiler
warnings on 64-bit hosts.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Reviewed-by: Tero Kristo <t-kristo@ti.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Fri, 12 Jun 2020 11:16:22 +0000 (21:16 +1000)]
crypto: octeontx - Fix sparse warnings
This patch fixes all the sparse warnings in the octeontx driver.
Some of these are just trivial type changes.
However, some of the changes are non-trivial on little-endian hosts.
Obviously the driver appears to be broken on either LE or BE as it
was doing different things. I've taken the BE behaviour as the
correct one.
Reported-by: kernel test robot <lkp@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Wojciech Ziemba [Thu, 11 Jun 2020 21:14:49 +0000 (22:14 +0100)]
crypto: qat - update timeout logic in put admin msg
Replace timeout logic in adf_put_admin_msg_sync() with existing macro
readl_poll_timeout().
Signed-off-by: Wojciech Ziemba <wojciech.ziemba@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Wojciech Ziemba [Thu, 11 Jun 2020 21:14:48 +0000 (22:14 +0100)]
crypto: qat - send admin messages to set of AEs
Update the logic that sends admin messages to be able to target a subset
of Acceleration Engines (AEs) in the device.
In future not all admin messages need to be sent to all the AEs.
Signed-off-by: Wojciech Ziemba <wojciech.ziemba@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Wojciech Ziemba [Thu, 11 Jun 2020 21:14:47 +0000 (22:14 +0100)]
crypto: qat - update fw init admin msg
This patch tidies up the definition of init/admin request and response
messages by removing the icp_qat_fw_init_admin_resp_pars structure
and embedding it into icp_qat_fw_init_admin_resp.
Signed-off-by: Wojciech Ziemba <wojciech.ziemba@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Colin Ian King [Thu, 11 Jun 2020 15:39:34 +0000 (16:39 +0100)]
crypto: caam/qi2 - remove redundant assignment to ret
The variable ret is being assigned a value that is never read, the
error exit path via label 'unmap' returns -ENOMEM anyhow, so assigning
ret with -ENOMEM is redundamt.
Addresses-Coverity: ("Unused value") Signed-off-by: Colin Ian King <colin.king@canonical.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Eric Biggers [Wed, 10 Jun 2020 06:39:43 +0000 (23:39 -0700)]
crc-t10dif: clean up some more things
- Correctly compare the algorithm name in crc_t10dif_notify().
- Use proper NOTIFY_* status codes instead of 0.
- Consistently use CRC_T10DIF_STRING instead of "crct10dif" directly.
- Use a proper type for the shash_desc context.
- Use crypto_shash_driver_name() instead of open-coding it.
- Make crc_t10dif_transform_show() use snprintf() rather than sprintf().
This isn't actually necessary since the buffer has size PAGE_SIZE
and CRYPTO_MAX_ALG_NAME < PAGE_SIZE, but it's good practice.
- Give the "transform" sysfs file mode 0444 rather than 0644,
since it doesn't implement a setter method.
- Adjust the module description to not be the same as crct10dif-generic.
Cc: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Eric Biggers [Wed, 10 Jun 2020 06:39:42 +0000 (23:39 -0700)]
crc-t10dif: use fallback in initial state
Currently the crc-t10dif module starts out with the fallback disabled
and crct10dif_tfm == NULL. crc_t10dif_mod_init() tries to allocate
crct10dif_tfm, and if it fails it enables the fallback.
This is backwards because it means that any call to crc_t10dif() prior
to module_init (which could theoretically happen from built-in code)
will crash rather than use the fallback as expected. Also, it means
that if the initial tfm allocation fails, then the fallback stays
permanently enabled even if a crct10dif implementation is loaded later.
Change it to use the more logical solution of starting with the fallback
enabled, and disabling the fallback when a tfm gets allocated for the
first time. This change also ends up simplifying the code.
Also take the opportunity to convert the code to use the new static_key
API, which is much less confusing than the old and deprecated one.
Cc: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
As stress-ng running SEC engine on the Ubuntu OS,
we found that SEC only supports two threads each with one TFM
based on the default module parameter 'ctx_q_num'.
If running more threads, stress-ng will fail since it cannot
get more TFMs.
In order to fix this, we adjusted the default values
of the module parameters to support more TFMs.
Signed-off-by: Longfang Liu <liulongfang@huawei.com> Signed-off-by: Zaibo Xu <xuzaibo@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Fri, 5 Jun 2020 06:59:18 +0000 (16:59 +1000)]
crc-t10dif: Fix potential crypto notify dead-lock
The crypto notify call occurs with a read mutex held so you must
not do any substantial work directly. In particular, you cannot
call crypto_alloc_* as they may trigger further notifications
which may dead-lock in the presence of another writer.
This patch fixes this by postponing the work into a work queue and
taking the same lock in the module init function.
While we're at it this patch also ensures that all RCU accesses are
marked appropriately (tested with sparse).
Finally this also reveals a race condition in module param show
function as it may be called prior to the module init function.
It's fixed by testing whether crct10dif_tfm is NULL (this is true
iff the init function has not completed assuming fallback is false).
Fixes: 11dcb1037f40 ("crc-t10dif: Allow current transform to be...") Fixes: b76377543b73 ("crc-t10dif: Pick better transform if one...") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com> Reviewed-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Wojciech Ziemba [Wed, 3 Jun 2020 17:33:45 +0000 (18:33 +0100)]
crypto: qat - replace user types with kernel ABI __u types
Kernel source code should not contain stdint.h types.
This patch replaces uintXX_t types with kernel space ABI types.
Signed-off-by: Wojciech Ziemba <wojciech.ziemba@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Wojciech Ziemba [Wed, 3 Jun 2020 17:33:44 +0000 (18:33 +0100)]
crypto: qat - replace user types with kernel u types
Kernel source code should not include stdint.h types.
This patch replaces uintXX_t types with respective ones defined in kernel
headers.
Signed-off-by: Wojciech Ziemba <wojciech.ziemba@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Dan Carpenter [Tue, 2 Jun 2020 13:54:09 +0000 (16:54 +0300)]
crypto: hisilicon - allow smaller reads in debugfs
Originally this code rejected any read less than 256 bytes. There
is no need for this artificial limit. We should just use the normal
helper functions to read a string from the kernel.
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Olivier Sobrie [Mon, 1 Jun 2020 14:27:40 +0000 (16:27 +0200)]
hwrng: ba431 - add support for BA431 hwrng
Silex insight BA431 is an IP designed to generate random numbers that
can be integrated in various FPGA.
This driver adds support for it through the hwrng interface.
This driver is used in Silex Insight Viper OEM boards.
Herbert Xu [Fri, 29 May 2020 14:23:49 +0000 (00:23 +1000)]
crypto: algif_aead - Only wake up when ctx->more is zero
AEAD does not support partial requests so we must not wake up
while ctx->more is set. In order to distinguish between the
case of no data sent yet and a zero-length request, a new init
flag has been added to ctx.
SKCIPHER has also been modified to ensure that at least a block
of data is available if there is more data to come.
Daniel Jordan [Mon, 8 Jun 2020 21:26:52 +0000 (17:26 -0400)]
padata: upgrade smp_mb__after_atomic to smp_mb in padata_do_serial
A 5.7 kernel hangs during a tcrypt test of padata that waits for an AEAD
request to finish. This is only seen on large machines running many
concurrent requests.
The issue is that padata never serializes the request. The removal of
the reorder_objects atomic missed that the memory barrier in
padata_do_serial() depends on it.
Upgrade the barrier from smp_mb__after_atomic to smp_mb to get correct
ordering again.
Fixes: 3facced7aeed1 ("padata: remove reorder_objects") Signed-off-by: Daniel Jordan <daniel.m.jordan@oracle.com> Cc: Steffen Klassert <steffen.klassert@secunet.com> Cc: linux-kernel@vger.kernel.org Cc: <stable@vger.kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Mon, 8 Jun 2020 06:48:43 +0000 (16:48 +1000)]
crypto: af_alg - fix use-after-free in af_alg_accept() due to bh_lock_sock()
The locking in af_alg_release_parent is broken as the BH socket
lock can only be taken if there is a code-path to handle the case
where the lock is owned by process-context. Instead of adding
such handling, we can fix this by changing the ref counts to
atomic_t.
This patch also modifies the main refcnt to include both normal
and nokey sockets. This way we don't have to fudge the nokey
ref count when a socket changes from nokey to normal.
Credits go to Mauricio Faria de Oliveira who diagnosed this bug
and sent a patch for it:
Dan Carpenter [Fri, 5 Jun 2020 11:03:39 +0000 (14:03 +0300)]
crypto: marvell/octeontx - Fix a potential NULL dereference
Smatch reports that:
drivers/crypto/marvell/octeontx/otx_cptvf_algs.c:132 otx_cpt_aead_callback()
warn: variable dereferenced before check 'cpt_info' (see line 121)
This function is called from process_pending_queue() as:
drivers/crypto/marvell/octeontx/otx_cptvf_reqmgr.c
599 /*
600 * Call callback after current pending entry has been
601 * processed, we don't do it if the callback pointer is
602 * invalid.
603 */
604 if (callback)
605 callback(res_code, areq, cpt_info);
It does appear to me that "cpt_info" can be NULL so this could lead to
a NULL dereference.
Fixes: 10b4f09491bf ("crypto: marvell - add the Virtual Function driver for CPT") Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Eric Biggers [Thu, 4 Jun 2020 18:52:53 +0000 (11:52 -0700)]
crypto: algboss - don't wait during notifier callback
When a crypto template needs to be instantiated, CRYPTO_MSG_ALG_REQUEST
is sent to crypto_chain. cryptomgr_schedule_probe() handles this by
starting a thread to instantiate the template, then waiting for this
thread to complete via crypto_larval::completion.
This can deadlock because instantiating the template may require loading
modules, and this (apparently depending on userspace) may need to wait
for the crc-t10dif module (lib/crc-t10dif.c) to be loaded. But
crc-t10dif's module_init function uses crypto_register_notifier() and
therefore takes crypto_chain.rwsem for write. That can't proceed until
the notifier callback has finished, as it holds this semaphore for read.
Fix this by removing the wait on crypto_larval::completion from within
cryptomgr_schedule_probe(). It's actually unnecessary because
crypto_alg_mod_lookup() calls crypto_larval_wait() itself after sending
CRYPTO_MSG_ALG_REQUEST.
This only actually became a problem in v4.20 due to commit b76377543b73
("crc-t10dif: Pick better transform if one becomes available"), but the
unnecessary wait was much older.
BugLink: https://bugzilla.kernel.org/show_bug.cgi?id=207159 Reported-by: Mike Gerow <gerow@google.com> Fixes: 398710379f51 ("crypto: algapi - Move larval completion into algboss") Cc: <stable@vger.kernel.org> # v3.6+ Cc: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Reported-by: Kai Lüke <kai@kinvolk.io> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
projects / kernel.git / log