Request socket hang up error

[jsetton]

Due to the change in #265, adding request keep alive http agent support (forever: true), we have been getting sporadically a socket hang up error causing relevant requests to fail.

This seems to be caused by the native nodejs http agent not handling gracefully keep-alive free sockets that have timed out on the server side. Unless defined in the myopenhab cloud connector nginx configuration, the default server keep-alive timeout is currently set to 75 seconds; while an AWS lambda execution context can be inactive for up to 30 minutes between two invocations.

The options to resolve this issue are one of the following:

• Remove the keep-alive support
• Add retry logic when getting this error
• Use a better suited http agent such as agentkeepalive which allows to specify a free socket timeout that we could set to something like 45 seconds to ensure that these sockets are cleanup prior to server-side socket timeout. The reason for that time value is that based on the logs, most of the consecutive invokes for a given execution context happens within that timeframe.

I think that, with the first option, we would lose the response time benefit for not reusing existing connections during peak times. With the second one, we would add some overhead and therefore some added response time going through the motion of first getting the initial reuse connection request rejected and then creating a new connection. While the last one seems to be the most efficient but would rely on an additional module.

RequestError: Error: socket hang up
    at new RequestError (/var/task/node_modules/request-promise-core/lib/errors.js:14:15)
    at Request.plumbing.callback (/var/task/node_modules/request-promise-core/lib/plumbing.js:87:29)
    at Request.RP$callback [as _callback] (/var/task/node_modules/request-promise-core/lib/plumbing.js:46:31)
    at self.callback (/var/task/node_modules/request/request.js:185:22)
    at emitOne (events.js:116:13)
    at Request.emit (events.js:211:7)
    at Request.onRequestError (/var/task/node_modules/request/request.js:881:8)
    at emitOne (events.js:116:13)
    at ClientRequest.emit (events.js:211:7)
    at TLSSocket.socketCloseListener (_http_client.js:331:9)

Lambda log reference

[jsetton]

@digitaldan From the different options I mentioned in that issue, I think the third one would be the best. Let me know what you think.

[digitaldan]

👍 Yep I agree.

[jsetton]

Any objections if I merge #270 and push to live skill?

[digitaldan]

Sounds good to me 👍

[jsetton]

Thanks. I just pushed this change to live skill. Let see if it fixes this issue. 🤞

[jsetton]

So in the last hour, this change has been running, no new socket hang up error was generated while we usually have gotten about 5-6 errors. I will close this one until it happens again.