Mailing List Archive: Apache: Dev

Strange error(parse tlsext bug) in mod_ssl since httpd-2.2.12

1 2

View All

Index | Next | Previous | View Threaded

kamesh at collab

Oct 26, 2009, 3:22 AM

Post #26 of 36 (1698 views)
Permalink

Re: Strange error(parse tlsext bug) in mod_ssl since httpd-2.2.12 [In reply to]

Hi Kaspar,

I applied your 'mod_ssl-disable_tls_tickets.diff' and
'mod_ssl-log_ssloptions.diff' to apache-2.2.12

and initiated the 'failing svn import operation'.

<snip from error_log while this fails>
[Mon Oct 26 15:48:21 2009] [warn] [client 10.2.0.88]
ssl_init_ssl_connection: options=0x1114fff
[Mon Oct 26 15:48:22 2009] [warn] [client 10.2.0.88]
ssl_init_ssl_connection: options=0x1114fff
[Mon Oct 26 15:48:22 2009] [warn] [client 10.2.0.88]
ssl_init_ssl_connection: options=0x1114fff
</snip>

The tcpdump for this failure is at,

http://www.livecipher.com/tlsext_dump/tlsext.dmp.4

With regards
Kamesh Jayachandran
On 10/25/2009 09:21 PM, Kaspar Brand wrote:
> Dr Stephen Henson wrote:
>
>> Disabling tickets using SSL_OP_NO_TICKET server side SHOULD work too (does in my
>> tests) so I've no idea why that wouldn't in the OPs setup unless the patch
>> doesn't set it in all contexts. Try placing it right after any call to
>> SSL_CTX_new().
>>
> I'm still a bit puzzled as to why my previously posted patch does not
> turn off TLS session tickets... there's only one place in mod_ssl where
> a new context is created, and in my tests, SSL_OP_NO_TICKET was reliably
> applied (i.e., I didn't see any session tickets on the wire). Maybe
> there's another issue if tickets are turned off?
>
> Kamesh, could you apply the attached patch, for diagnostic purposes (in
> addition to mod_ssl-disable_tls_tickets.diff), and let us know what
> "options=" values you see in your ErrorLog? Note that you don't have to
> increase Apache's LogLevel, the options for any new SSL connection will
> be logged with "warn" already. Also, it would be helpful to have another
> capture (with mod_ssl patched like this) where the svn client still
> fails with a "parse tlsext" error. Thanks.
>
> Kaspar
>

httpd-dev.2009 at velox

Oct 26, 2009, 11:01 PM

Post #27 of 36 (1686 views)
Permalink

Re: Strange error(parse tlsext bug) in mod_ssl since httpd-2.2.12 [In reply to]

Kamesh Jayachandran,
> [Mon Oct 26 15:48:22 2009] [warn] [client 10.2.0.88]
> ssl_init_ssl_connection: options=0x1114fff
> </snip>
>
> The tcpdump for this failure is at,
>
> http://www.livecipher.com/tlsext_dump/tlsext.dmp.4

Thanks - this shows that session tickets are indeed disabled on the
server side (0x...4fff means SSL_OP_ALL | SSL_OP_NO_TICKET), and from
the packet dump it can be seen that the server does not send any session
ticket extensions in its ServerHellos.

With the client, the difference to the case where tickets are enabled is
the fact that for the third connection, it includes an empty session
ticket extension in the ClientHello (indicating that a NewSessionTicket
message would be accepted). The server then again replies with a
ServerHello with a server_name extension - which is immediately rejected
by the client with SSL_AD_DECODE_ERROR. I therefore assume that it still
ends up in the "if (!s->hit && tlsext_servername == 1)" code block,
irrespective of whether the server uses SSL_OP_NO_TICKET or not.

Kaspar

shenson at oss-institute

Oct 27, 2009, 6:13 AM

Post #28 of 36 (1677 views)
Permalink

Re: Strange error(parse tlsext bug) in mod_ssl since httpd-2.2.12 [In reply to]

Kaspar Brand wrote:
> Kamesh Jayachandran,
>> [Mon Oct 26 15:48:22 2009] [warn] [client 10.2.0.88]
>> ssl_init_ssl_connection: options=0x1114fff
>> </snip>
>>
>> The tcpdump for this failure is at,
>>
>> http://www.livecipher.com/tlsext_dump/tlsext.dmp.4
>
> Thanks - this shows that session tickets are indeed disabled on the
> server side (0x...4fff means SSL_OP_ALL | SSL_OP_NO_TICKET), and from
> the packet dump it can be seen that the server does not send any session
> ticket extensions in its ServerHellos.
>
> With the client, the difference to the case where tickets are enabled is
> the fact that for the third connection, it includes an empty session
> ticket extension in the ClientHello (indicating that a NewSessionTicket
> message would be accepted). The server then again replies with a
> ServerHello with a server_name extension - which is immediately rejected
> by the client with SSL_AD_DECODE_ERROR. I therefore assume that it still
> ends up in the "if (!s->hit && tlsext_servername == 1)" code block,
> irrespective of whether the server uses SSL_OP_NO_TICKET or not.
>

That's very strange. It seems like that shouldn't happen because a stateful
session resumption should occur and s->hit == 1.

Will analyse it more when I have access to my main test setup later today.

Steve.
--
Dr Stephen N. Henson. Senior Technical/Cryptography Advisor,
Open Source Software Institute: www.oss-institute.org
OpenSSL Core team: www.openssl.org

shenson at oss-institute

Oct 27, 2009, 11:12 AM

Post #29 of 36 (1678 views)
Permalink

Re: Strange error(parse tlsext bug) in mod_ssl since httpd-2.2.12 [In reply to]

Dr Stephen Henson wrote:
> Kaspar Brand wrote:
>> Kamesh Jayachandran,
>>> [Mon Oct 26 15:48:22 2009] [warn] [client 10.2.0.88]
>>> ssl_init_ssl_connection: options=0x1114fff
>>> </snip>
>>>
>>> The tcpdump for this failure is at,
>>>
>>> http://www.livecipher.com/tlsext_dump/tlsext.dmp.4
>> Thanks - this shows that session tickets are indeed disabled on the
>> server side (0x...4fff means SSL_OP_ALL | SSL_OP_NO_TICKET), and from
>> the packet dump it can be seen that the server does not send any session
>> ticket extensions in its ServerHellos.
>>
>> With the client, the difference to the case where tickets are enabled is
>> the fact that for the third connection, it includes an empty session
>> ticket extension in the ClientHello (indicating that a NewSessionTicket
>> message would be accepted). The server then again replies with a
>> ServerHello with a server_name extension - which is immediately rejected
>> by the client with SSL_AD_DECODE_ERROR. I therefore assume that it still
>> ends up in the "if (!s->hit && tlsext_servername == 1)" code block,
>> irrespective of whether the server uses SSL_OP_NO_TICKET or not.
>>
>
> That's very strange. It seems like that shouldn't happen because a stateful
> session resumption should occur and s->hit == 1.
>
> Will analyse it more when I have access to my main test setup later today.
>

That is most peculiar. The server is sending back a zero length session ID in
the server hello which it shouldn't be doing if tickets are disabled.

Is the server somehow using an older version of OpenSSL? There has been a bug in
the past which might do that but it was fixed well before 0.9.8k.

Steve.
--
Dr Stephen N. Henson. Senior Technical/Cryptography Advisor,
Open Source Software Institute: www.oss-institute.org
OpenSSL Core team: www.openssl.org

shenson at oss-institute

Oct 27, 2009, 3:47 PM

Post #30 of 36 (1690 views)
Permalink

Re: Strange error(parse tlsext bug) in mod_ssl since httpd-2.2.12 [In reply to]

Kamesh Jayachandran wrote:
> Hi Kaspar,
>
> I applied your 'mod_ssl-disable_tls_tickets.diff' and
> 'mod_ssl-log_ssloptions.diff' to apache-2.2.12
>
> and initiated the 'failing svn import operation'.
>
> <snip from error_log while this fails>
> [Mon Oct 26 15:48:21 2009] [warn] [client 10.2.0.88]
> ssl_init_ssl_connection: options=0x1114fff
> [Mon Oct 26 15:48:22 2009] [warn] [client 10.2.0.88]
> ssl_init_ssl_connection: options=0x1114fff
> [Mon Oct 26 15:48:22 2009] [warn] [client 10.2.0.88]
> ssl_init_ssl_connection: options=0x1114fff
> </snip>
>
> The tcpdump for this failure is at,
>
> http://www.livecipher.com/tlsext_dump/tlsext.dmp.4
>

As I mentioned something strange is going on there. The server is sending back
empty session IDs which shouldn't be happening if tickets are disabled properly.

With OpenSSL 0.9.8k client, can you try this connecting to that server:

openssl s_client -connect hostname.whatever.com:443
-servername hostname.whatever.com -tls1

Does any value appear after "Session-ID"? Hit Q<return> to exit.

Also try:

openssl s_client -connect hostname.whatever.com:443
-servername hostname.whatever.com -tls1 -no_ticket

again do you get anything after "Session-ID"?

Finally this pair of commands:

openssl s_client -connect hostname.whatever.com:443
-servername hostname.whatever.com -tls1
-sess_out foo.pem

openssl s_client -connect hostname.whatever.com:443
-servername hostname.whatever.com -tls1
-sess_in foo.pem

Do you still get the error when you call the command with the server including
SSL_OP_NO_TICKET?

Steve.
--
Dr Stephen N. Henson. Senior Technical/Cryptography Advisor,
Open Source Software Institute: www.oss-institute.org
OpenSSL Core team: www.openssl.org

kamesh at collab

Oct 28, 2009, 6:54 AM

Post #31 of 36 (1667 views)
Permalink

RE: Strange error(parse tlsext bug) in mod_ssl since httpd-2.2.12 [In reply to]

>That is most peculiar. The server is sending back a zero length session ID in
>the server hello which it shouldn't be doing if tickets are disabled.

>Is the server somehow using an older version of OpenSSL? There has been a bug >in
>the past which might do that but it was fixed well before 0.9.8k.

Server uses openssl-0.9.8k.

With regards
Kamesh Jayachandran

shenson at oss-institute

Oct 28, 2009, 10:23 AM

Post #32 of 36 (1666 views)
Permalink

Re: Strange error(parse tlsext bug) in mod_ssl since httpd-2.2.12 [In reply to]

Kamesh Jayachandran wrote:
>
>>That is most peculiar. The server is sending back a zero length session
> ID in
>>the server hello which it shouldn't be doing if tickets are disabled.
>
>>Is the server somehow using an older version of OpenSSL? There has been
> a bug >in
>>the past which might do that but it was fixed well before 0.9.8k.
>
> Server uses openssl-0.9.8k.
>

Do you have session caching disabled in the server configuration, either
accidentally or deliberately? That seems to me to be the only thing that fits
the tcpdump you sent.

If so please turn session caching on and try the SSL_OP_NO_TICKET patch again.

Steve.
--
Dr Stephen N. Henson. Senior Technical/Cryptography Advisor,
Open Source Software Institute: www.oss-institute.org
OpenSSL Core team: www.openssl.org

kamesh at collab

Oct 29, 2009, 10:08 AM

Post #33 of 36 (1650 views)
Permalink

RE: Strange error(parse tlsext bug) in mod_ssl since httpd-2.2.12 [In reply to]

>Do you have session caching disabled in the server configuration, either
>accidentally or deliberately? That seems to me to be the only thing that fits
>the tcpdump you sent.

>If so please turn session caching on and try the SSL_OP_NO_TICKET patch again.

I am away from the test setup. Should be able to do this only on coming Monday.

Thanks

With regards
Kamesh Jayachandran

shenson at oss-institute

Oct 30, 2009, 10:46 AM

Post #34 of 36 (1634 views)
Permalink

Re: Strange error(parse tlsext bug) in mod_ssl since httpd-2.2.12 [In reply to]

Kamesh Jayachandran wrote:
>
>>Do you have session caching disabled in the server configuration, either
>>accidentally or deliberately? That seems to me to be the only thing
> that fits
>>the tcpdump you sent.
>
>>If so please turn session caching on and try the SSL_OP_NO_TICKET patch
> again.
>
> I am away from the test setup. Should be able to do this only on coming
> Monday.
>

OK, will be interested to know if that works.

Based on my analysis the latest snapshots of OpenSSL should fix this properly
but that is a client side fix.

Server side disabling tickets and making sure the session cache is enabled
should be a usable workaround.

Steve.
--
Dr Stephen N. Henson. Senior Technical/Cryptography Advisor,
Open Source Software Institute: www.oss-institute.org
OpenSSL Core team: www.openssl.org

kamesh at collab

Nov 2, 2009, 3:41 AM

Post #35 of 36 (1622 views)
Permalink

Re: Strange error(parse tlsext bug) in mod_ssl since httpd-2.2.12 [In reply to]

On 10/28/2009 04:17 AM, Dr Stephen Henson wrote:
> Kamesh Jayachandran wrote:
>
>> Hi Kaspar,
>>
>> I applied your 'mod_ssl-disable_tls_tickets.diff' and
>> 'mod_ssl-log_ssloptions.diff' to apache-2.2.12
>>
>> and initiated the 'failing svn import operation'.
>>
>> <snip from error_log while this fails>
>> [Mon Oct 26 15:48:21 2009] [warn] [client 10.2.0.88]
>> ssl_init_ssl_connection: options=0x1114fff
>> [Mon Oct 26 15:48:22 2009] [warn] [client 10.2.0.88]
>> ssl_init_ssl_connection: options=0x1114fff
>> [Mon Oct 26 15:48:22 2009] [warn] [client 10.2.0.88]
>> ssl_init_ssl_connection: options=0x1114fff
>> </snip>
>>
>> The tcpdump for this failure is at,
>>
>> http://www.livecipher.com/tlsext_dump/tlsext.dmp.4
>>
>>
>

Sorry for the delay.

> As I mentioned something strange is going on there. The server is sending back
> empty session IDs which shouldn't be happening if tickets are disabled properly.
>
> With OpenSSL 0.9.8k client, can you try this connecting to that server:
>
> openssl s_client -connect hostname.whatever.com:443
> -servername hostname.whatever.com -tls1
>
> Does any value appear after "Session-ID"? Hit Q<return> to exit.
>

Yes it appears.

> Also try:
>
> openssl s_client -connect hostname.whatever.com:443
> -servername hostname.whatever.com -tls1 -no_ticket
>
> again do you get anything after "Session-ID"?
>

Yes I get.

> Finally this pair of commands:
>
> openssl s_client -connect hostname.whatever.com:443
> -servername hostname.whatever.com -tls1
> -sess_out foo.pem
>
>
> openssl s_client -connect hostname.whatever.com:443
> -servername hostname.whatever.com -tls1
> -sess_in foo.pem
>
> Do you still get the error when you call the command with the server including
> SSL_OP_NO_TICKET?
>

Yes I get the error with Server running SSL_OP_NO_TICKET patch.

[kamesh [at] kames httpd-2.2.12]$ openssl s_client -connect kamesh:443
-servername kamesh -tls1 -sess_in foo.pem
CONNECTED(00000003)
4155:error:140920DF:SSL routines:SSL3_GET_SERVER_HELLO:parse
tlsext:s3_clnt.c:880:

FYI I used this openssl client on my linux for this test which is
openssl0.9.8-k(while original issue was posted against the win32 svn
client built with openssl-0.9.8j.

With regards
Kamesh Jayachandran

kamesh at collab

Nov 2, 2009, 4:52 AM

Post #36 of 36 (1570 views)
Permalink

Re: Strange error(parse tlsext bug) in mod_ssl since httpd-2.2.12 [In reply to]

Do you have session caching disabled in the server configuration, either
> accidentally or deliberately? That seems to me to be the only thing that fits
> the tcpdump you sent.
>
> If so please turn session caching on and try the SSL_OP_NO_TICKET patch again.
>
> Steve.
>

Yes SSLSessionCache was not enabled(by default it is 'none').

Yes I could see it 'working' with SSL_OP_NO_TICKET patch and
'SSLSessionCache shm:/tmp/sessioncache' directive.

IIUC, Proper fix for this should be in openssl client.

Reasonable fix for this on the server side is to apply SSL_OP_NO_TICKET
patch and enable SSLSessionCache.

Thanks

With regards
Kamesh Jayachandran

1 2

View All

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads