Which event source should I listen to get authentication events ?

[jpdurot]

Hello,
I'm trying to debug a kerberos authnetication issue in production, I already got a trace of what's happening on SQL client library by listening to Microsoft.Data.SqlClient event source, but I would like to go one step further by getting events from the authentication phase. I could find some interesting events defined in https://github.com/dotnet/SqlClient/blob/main/src/Microsoft.Data.SqlClient/netcore/src/Common/src/System/Net/Logging/NetEventSource.Common.cs#L51 but there is no EventSource annotation on top of the class definition, so I have no idea which Event Source I should listen to, to get those events in my trace file.

Could you help me on that topic please ?
Regards

[arellegue]

Thank you for your inquiry about NetEventSource. We'll get back to you when we have a definitive answer.

[arellegue]

We reviewed the NetEventSource and there does not appear to be any EventSource attribute to any of its partial class at this time.

At this point, you can continue using the Microsoft.Data.SqlClient.EventSource while using the ManagedSNI via AppContext switch,

AppContext.SetSwitch("Switch.Microsoft.Data.SqlClient.UseManagedNetworkingOnWindows", true);

, and set the log level or keyword appropriately. Please refer to https://learn.microsoft.com/en-us/sql/connect/ado-net/enable-eventsource-tracing?view=sql-server-ver16.

However, you may also want to use Wireshark to capture and display real-time details of network traffic.

[jpdurot]

If I understand correctly, there is no way to get those events right now. Is there any plan to add an event source so that we can better investigate ?

At this point, you can continue using the Microsoft.Data.SqlClient.EventSource while using the ManagedSNI via AppContext switch

Unfortunately, all our applications are running on Linux servers, so it looks like it won't make it...

However, you may also want to use Wireshark to capture and display real-time details of network traffic.

Not easy to setup as the error we are facing is transient and happens only once or twice a day...

Anyway, thanks for your quick reply

[arellegue]

MDS running on Linux is using managed SNI. You should be able to use Microsoft.Data.SqlClient.EventSource for Linux.

Also, there is no plan yet to add EventSource attribute to NetEventSource.

[jpdurot]

Actually that's what I did (using Microsoft.Data.SqlClient.EventSource). That's how I could find the message Cannot authenticate using Kerberos. Ensure Kerberos has been initialized on the client with 'kinit' and a Service Principal Name has been registered for the SQL Server to allow Kerberos authentication in my nettrace file.
Digging into the source code of the lib, it is thrown in SNIProxy.cs so I would have loved to get the events logged by the call to the method that returned the error (NegotiateStreamPal.InitializeSecurityContext) and the underlying calls, but each event is emitted only if NetEventSource.IsEnabled which is always false if I understood well...
It looks like it can be activated only by hardcoding the value of this property, as mentioned here
It seems my only chance to get those events is to recompile the library after changing this piece of code to always return true. However I have no idea in which Event Source those events would be linked to...

[arellegue]

I think the message suggested initializing the client with kinit. Have you done so already?

For example:
To initialize your client:
kinit [email protected].

To create a ticket and cache it
kinit -V -c krb5cc [email protected]

To check the ticket validity or expiry:
klist

Could it be that your client's ticket has expired and so you just needed to renew it?
If this is the case, what I would do is issue the following commands for example. However, by default the ticket only has a lifetime of 10 hours. You may want to find out how to extend the validity of ticket and be aware when it will expire and so be able to renew before it does.

kdestroy
kinit [email protected]
kinit -V -c krb5cc [email protected]