Communication link failure when using high num_warehouses

[Philip-Jonany]

I run TPCC for sqlserver, and consistently get this message when using num_warehouse = 612 and 1224. Other tests always run fine. If this message is found in prepare-output, the run will include failures with not finding stored procedures because stored procs usually come after creating indexes.

Error code 0x2746[Microsoft][ODBC Driver 17 for SQL Server]Communication link failure

Vuser 1:CREATING TPCC INDEXES

Error in Virtual User 1: [Microsoft][ODBC Driver 17 for SQL Server]TCP Provider: Error code 0x2746
[Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
(executing the statement)

Vuser 1:FINISHED FAILED

[sm-shaw]

This is an error at the underlying ODBC level, meaning the link has been lost while building the indexes. There is no reason why this would happen at a particular warehouse count if it succeeds at both a lower and higher warehouse count. There is a link from Microsoft on troubleshooting this issue here: https://techcommunity.microsoft.com/t5/azure-database-support-blog/lesson-learned-215-hands-on-labs-communication-link-failed-error/ba-p/3565011

The index build sends a SQL statement to build an index and will then wait. It could be possible depending on configuration and system performance that the time between sending the statement and its completing means that there is a timeout on the client or server terminating the connection (although this should send a timeout message rather than link lost). Obviously as noted before a higher warehouse count will take longer to build the indexes meaning more of a wait between sending the statement and receiving the response that it has completed. Nevertheless the advice in the troubleshooting link is best to follow:

• Look for network issues.
• Check performance on target/application server

You can also test the hypothesis after a failure by looking to see what indexes did create and on what index the link was lost. You can then try and re-run the same index creation statement manually in SSMS to see if it always fails.