implement timeout for kinesis sink

[hzxa21]

One of our customers observes that the one of the actor of the kinesis sink is stuck while other actors of the same sink are working fine. Triggering a manual recover resolves the issue.

There are no relevant ERROR/WARN logs from kinesis sink so I suspect the sink writer is stuck in the put_records call from kinesis client. After digging deeper, I found that we are using the default timeout configs from aws sdk. By default only connection timeout is set and read/operation/operation_attempt timeout are not set:

pub struct TimeoutConfig {
    connect_timeout: CanDisable<Duration>,
    read_timeout: CanDisable<Duration>,
    operation_timeout: CanDisable<Duration>,
    operation_attempt_timeout: CanDisable<Duration>,
}


/// Resolve default timeout configuration
    pub async fn timeout_config(self) -> TimeoutConfig {
        // TODO(https://github.com/smithy-lang/smithy-rs/issues/1732): Implement complete timeout defaults specification
        TimeoutConfig::builder()
            .connect_timeout(SDK_DEFAULT_CONNECT_TIMEOUT)
            .build()
    }

I think we can either set a timeout for the kinesis client or wrap the put_records call with tokio timeout to prevent potential stuck in kinesis writer.