server sent ECH retry configs without client sending ECH extension

[ngehrsitz]

Hi,

I am trying to intercept the traffic from a website that embeds code from https://unpkg.com/ but that fails with the following error:

2024/06/07 22:43:25 [007] WARN: Cannot handshake to unpkg.com:443
%!(EXTRA *errors.errorString=tls: server sent ECH retry configs without client sending ECH extension)

I am using Firefox 125.0.3 as the client. I already tried toggling network.dns.echconfig.enabled and network.dns.use_https_rr_as_altsvc but that didn´t change anything. There is a specific ECH test site, that fails to load with the same error: https://defo.ie/ech-check.php
In the debugger I can see that the ECH extension is indeed missing

The ECH support was implemented in refraction-networking/utls#240, but I couldn´t find an issue with that code.
Could it be that the clientHello is not interpreted correctly?

[dr3adx]

Hi,

I am trying to intercept the traffic from a website that embeds code from https://unpkg.com/ but that fails with the following error:

2024/06/07 22:43:25 [007] WARN: Cannot handshake to unpkg.com:443
%!(EXTRA *errors.errorString=tls: server sent ECH retry configs without client sending ECH extension)

I am using Firefox 125.0.3 as the client. I already tried toggling network.dns.echconfig.enabled and network.dns.use_https_rr_as_altsvc but that didn´t change anything. There is a specific ECH test site, that fails to load with the same error: https://defo.ie/ech-check.php In the debugger I can see that the ECH extension is indeed missing The ECH support was implemented in refraction-networking/utls#240, but I couldn´t find an issue with that code. Could it be that the clientHello is not interpreted correctly?

same error here, did you find a solution? throw me your discord if u have it

[ngehrsitz]

No, I didn´t find a solution. I think I will try to implement my MitM use-case with a browser plugin instead

[clouedoc]

Maybe our client is too strict?

Maybe you could just comment out a specific part of the code to be less strict when it comes to receiving an ECH retry
config without having sent an extension.

https://guardianproject.info/2021/11/30/implementing-tls-encrypted-client-hello/

--> our client is not too strict. Rather, it's misbehaving — the target doesn't understand what we're sending so it resorts to an "ECH Retry"

Erratum. It looks like the target server has trouble decrypting the data?

Okay, then, maybe it's just Firefox's ClientHello trigerring a bug?

It would be worthwhile to check if the problem arises with a different client.

E.g. can you access the same website with cURL?

On my side:

1. cURL+utlsproxy with https://unpkg.com/ works fine.
2. cURL+utlsproxy with https://defo.ie/ech-check.php hangs. Maybe I should try waiting a bit more 👀
3. cURL without utlsproxy with https://defo.ie/ech-check.php works fine

I didn't try Firefox.

It looks like Firefox's ClientHello is not the entire problem.
Firefox might be triggering additionnal problems making it impossible to access unpkg.
It's already possible to reproduce problems with cURL, but only with https://defo.ie/ech-check.php

Not sure what to conclude.

--> The first step in resolving the issue is to get cURL+utlsproxy+https://defo.ie/ech-check.php working.

Wireshark capture

Capture: curl+utlsproxy

I did the following:

1. Start a Wireshark capture
2. Start utlsproxy
3. Send a request to defo.ie
4. Wait a few seconds (the request is still hanging)
5. Stop the Wireshark capture
6. Kill utlsproxy

defo-ie-curl-utlsproxy.pcapng.zip

Capture: cURL alone

defo-ie-curl-standalone.pcapng.zip

Comparaison

(left is utlsproxy+cURL, right is cURL standalone)

We can check the flow of TLS packet content types:

We can notice that there is a difference there already.

1. CLIENT: Handshake
2. SERVER: Handshake, Change Cipher Spec (additionnal Handshake in the utlsproxy configuration)

Since we can already see a difference at packet 2, I bet that there is something different in packet 1 beside the content type.
(there is also the hypothesis that I'm just missing some TLS keys to decrypt some parts of the flow)

Erratum: there's no difference

So, the issue is not in the first Handshake packet (Client Hello).

We can skip packets sent by the server (2 and 3), assuming that they are the same and we just cannot decrypt some part of them without configuring cURL to output an NSS Key Log.

We can see that utlsproxy finalizes the handshake maybe a bit early on the left? (reminder: utlsproxy+cURL on the left, curl alone on the right)

Not sure what's going on.

I think that to resolve the issue, the best would be to perform the following steps:

1. Get a cURL capture with NSS Key Log enabled: https://everything.curl.dev/usingcurl/tls/sslkeylogfile.html
2. Do another comparaison having the SSL keys

I'm going to do something else with my day, but what I would advise to do to resolve the issue:

1. Get an utlsproxy + cURL capture, export the TLS packets only in a Wireshark file, embed the SSLKeylogs using ITS, following this tutorial: https://wiki.wireshark.org/TLS#embedding-decryption-secrets-in-a-pcapng-file
2. Same for a cURL capture
3. Post the captures here
4. Compare them until we find the exact problem.

[saucesteals]

Hello everyone! Should be fixed on latest main branch now. Let me know

[ngehrsitz]

I just did a quick test with https://unpkg.com/ on 44a0906 and that works fine.
But https://defo.ie/ech-check.php still refuses to work

I can also see the same error with regular domains like https://api.mapbox.com/ so the issue is not just limited to the test site.