#2769: problem with openssl 1.0.1 and 512bits rsa key

# Fri Mar 23 11:59:30 2012 J - Ticket created

Download (untitled) / with headers
text/plain 672b

Hi,
after updating to openssl 1.0.1 (debian package), authentication against a test server
with a 512 bit rsa key gives :

openssl s_client -connect 127.0.0.1:12346 -key /home/dev/agent1-key.pem -cert /home/dev/agent1-cert.pem
...
139860308645544:error:04075070:rsa routines:RSA_sign:digest too big for rsa key:rsa_sign.c:127:
139860308645544:error:14099006:SSL routines:SSL3_SEND_CLIENT_VERIFY:EVP lib:s3_clnt.c:2974:
...

Downgrading to openssl 1.0.0h fixes the issue.
Please find attached the key/certificate.
(they have been made for testing purpose).

I guess this is unexpected behavior, and i did not find any recent similar report.

Regards,
J�r�my Lal

Download agent1-cert.pem
text/plain 810b

Message body is not shown because sender requested not to inline it.

Download agent1-key.pem
text/plain 497b

Message body is not shown because sender requested not to inline it.

# Fri Mar 23 13:25:11 2012 Stephen Henson - Correspondence added

Download (untitled) / with headers
text/plain 1.1k

Show quoted text

> [kapouer@melix.org - Fri Mar 23 11:59:30 2012]:
>
> Hi,
> after updating to openssl 1.0.1 (debian package), authentication
> against a test server
> with a 512 bit rsa key gives :
>
> openssl s_client -connect 127.0.0.1:12346 -key /home/dev/agent1-
> key.pem -cert /home/dev/agent1-cert.pem
> ...
> 139860308645544:error:04075070:rsa routines:RSA_sign:digest too
> big for rsa key:rsa_sign.c:127:
> 139860308645544:error:14099006:SSL
> routines:SSL3_SEND_CLIENT_VERIFY:EVP lib:s3_clnt.c:2974:
> ...
>
> Downgrading to openssl 1.0.0h fixes the issue.
> Please find attached the key/certificate.
> (they have been made for testing purpose).
>
> I guess this is unexpected behavior, and i did not find any recent
> similar report.
>

The reason for this is that OpenSSL 1.0.1 support TLS v1.2 and the
digest algorithm it uses by default is SHA512. A 512 bit RSA key is not
large enough for SHA512 so you get that error.

If you disable TLS v1.2 with -no_tls1_2 or use a larger key this wont
happen.

Use of 512 bit RSA keys is discouraged anyway for security reasons.

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org

# Fri Mar 23 13:25:12 2012 The RT System itself - Status changed from 'new' to 'open'

# Fri Mar 23 14:54:56 2012 J - Correspondence added

CC:	openssl-dev@openssl.org
Subject:	Re: [openssl.org #2769] problem with openssl 1.0.1 and 512bits rsa key
Date:	Fri, 23 Mar 2012 14:47:50 +0100
To:	rt@openssl.org
From:	J

Download (untitled) / with headers
text/plain 1.3k

On 23/03/2012 13:25, Stephen Henson via RT wrote:
Show quoted text

>> [kapouer@melix.org - Fri Mar 23 11:59:30 2012]:
>>
>> Hi,
>> after updating to openssl 1.0.1 (debian package), authentication
>> against a test server
>> with a 512 bit rsa key gives :
>>
>> openssl s_client -connect 127.0.0.1:12346 -key /home/dev/agent1-
>> key.pem -cert /home/dev/agent1-cert.pem
>> ...
>> 139860308645544:error:04075070:rsa routines:RSA_sign:digest too
>> big for rsa key:rsa_sign.c:127:
>> 139860308645544:error:14099006:SSL
>> routines:SSL3_SEND_CLIENT_VERIFY:EVP lib:s3_clnt.c:2974:
>> ...
>>
>> Downgrading to openssl 1.0.0h fixes the issue.
>> Please find attached the key/certificate.
>> (they have been made for testing purpose).
>>
>> I guess this is unexpected behavior, and i did not find any recent
>> similar report.
>>

>
> The reason for this is that OpenSSL 1.0.1 support TLS v1.2 and the
> digest algorithm it uses by default is SHA512. A 512 bit RSA key is not
> large enough for SHA512 so you get that error.
>
> If you disable TLS v1.2 with -no_tls1_2 or use a larger key this wont
> happen.
>
> Use of 512 bit RSA keys is discouraged anyway for security reasons.

Thank you for your quick reply.

Could it be explicitely explained in 1.0.1 changelog ?
i think there is already the fact that TLS v1.2 is supported,
but not what it implies (digest is sha512).

J�r�my.