fdgt support (Saves money for testing!)

[bdashore3]

I was recently informed of this new service called fdgt which is a mock twitch irc server that allows you to test various events such as subs/bits without spending a dime.

However, connecting to this service requires me to specify a server address to connect to on bot startup. If possible, can you add the ability to change the server URL when needed as shown here

[museun]

You can just connect with a tokio TcpStream.

The "client" portion of this crate is intentionally very thin, you can give any tokio::io::{AsyncRead+AsyncWrite} to ..
https://docs.rs/twitchchat/0.11.8/twitchchat/struct.Connector.html

let connector = Connector::new(|| {
    tokio::net::TcpStream::connect("some_address:port").await
});
// ...
runner.run_to_completion(connector).await;

for example. you could just do this yourself if you require TLS:

twitchchat/src/native_tls/mod.rs

Lines 25 to 41 in 18c7a1e

	pub async fn connect(config: &UserConfig) -> std::io::Result<Stream> {
	use std::io::{Error, ErrorKind};
	let conn: tokio_native_tls::TlsConnector = native_tls::TlsConnector::new()
	.map_err(|err| Error::new(ErrorKind::Other, err))?
	.into();
	let stream = tokio::net::TcpStream::connect(crate::TWITCH_IRC_ADDRESS_TLS).await?;
	let mut stream = conn
	.connect(TWITCH_DOMAIN, stream)
	.await
	.map_err(|err| Error::new(ErrorKind::Other, err))?;
	crate::register(config, &mut stream).await?;
	Ok(stream)
	}

register is a public method
https://docs.rs/twitchchat/0.11.8/twitchchat/fn.register.html

[bdashore3]

I'll take a look at this. Thank you for the quick response!

[bdashore3]

Tested and pull request submitted for the fdgt website

[Emilgardis]

To note here is that the connectors will soon change, see #163

[museun]

I'll have a similar design for the 'connectors' I believe. Or it'll be trait-based. I'm still determining the high-level approach of abstracting/feature-gating different external TcpStream implementations. The crate, itself, will only require an futures::{AsyncRead+AsyncWrite} + 'static for the async runner.

[bdashore3]

As long as I can specify a custom server address, everything should be fine

[museun]

I just implemented it. the dev branch is still a major WIP. But the new design should be easier to use.

Here is the implementation:

twitchchat/twitchchat/src/connector.rs

Lines 24 to 71 in 8003f0e

	/// The connector traits. This is used to abstract out runtimes.
	///
	/// You can implement this on your own type to provide a custom connection behavior.
	pub trait Connector {
	/// Output IO type returned by calling `connect`
	///
	/// This type must implement `futures::io::AsyncRead` and `futures::io::AsyncWrite`
	type Output: AsyncRead + AsyncWrite + Send + Sync + 'static;
	/// The `connect` method. This should return a boxed future of a `std::io::Result` of the `Output` type.
	///
	/// e.g. `Box::pin(async move { std::net::TcpStream::connect("someaddr") })
	fn connect(&mut self) -> crate::BoxedFuture<IoResult<Self::Output>>;
	}
	/// Configuration for the connector
	#[derive(Debug, Clone)]
	pub struct ConnectorConfig {
	pub(crate) addrs: Vec<SocketAddr>,
	pub(crate) tls_domain: String,
	}
	impl ConnectorConfig {
	/// Create an empty configuration
	///
	/// It is highly recommended that you add some socket addresses to this config via `with_addrs`
	pub fn unconfigured() -> Self {
	Self {
	addrs: vec![],
	tls_domain: "".to_string(),
	}
	}
	/// Use this TLS domain when using a TLS connector.
	pub fn with_tls_domain(self, domain: impl Into<String>) -> Self {
	Self {
	tls_domain: domain.into(),
	..self
	}
	}
	/// Use these resolved socket addresses with any connector implementation
	pub fn with_addrs(self, addrs: impl ToSocketAddrs) -> IoResult<Self> {
	addrs.to_socket_addrs().map(|s| Self {
	addrs: s.collect(),
	..self
	})
	}
	}

You basically create a ConnectorConfig with the addresses you want it to try, along with an optional TLS domain (if you're using a TLS connector).

And then you'd use something like

let config = ConnectorConfig::unconfigured().with_addrs("localhost:6667");
let connector = tokio::Connector::new(config);
// ...
runner.run_to_complete(connector).await;

You can also implement that Connector for your custom type -- this'll be the "connection factory" used in retrying the main loop on failure, or a one-shot connection.

[bdashore3]

I want to reopen this issue with the new version of the crate. I can't seem to connect to fdgt using the custom connector.

Here's the error I got:
Error: Io(Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" })

Here's the code for the connector and connection.

Can you please let me know if I did this right? If so, I'm going to talk with the fdgt dev

async fn connect(user_config: &UserConfig, channels: &[String]) -> KingResult<AsyncRunner> {
    let connector = Connector::custom("irc.fdgt.dev:6667")?;

    let mut runner = AsyncRunner::connect(connector, user_config).await?;
    println!("Connected to fdgt!");

    for channel in channels {
        println!("attempting to join '{}'", channel);
        runner.join(&channel).await?;
        println!("joined '{}'!", channel);
    }

    Ok(runner)
}

[bdashore3]

Info: I don't need to connect to a certain channel and don't need credentials. I just need a writer to send messages with and I need to just connect to the server itself

[museun]

So, I can't get it to connect with manual code. Is that the right port (and domain)? The connection is being refused by the server which usually indicates that.

[museun]

Looking through the connection code for the other 3 libraries listed on the site -- the two javascript ones use websockets -- which I don't support (yet). My crate only does tcp (and tls over tcp) connections. If they are expecting something on, say, port 80 (ws) or 443 (wss) then I'd have to add WsConnector/WssConnector types for this to work.

A normal std::net::TcpStream::connect("irc.fdgt.dev:6667") does not work, for me.

    let config = UserConfig::builder()
        .anonymous()
        .enable_all_capabilities()
        .build()
        .unwrap();

    let conn = std::net::TcpStream::connect("irc.fdgt.dev:6667").unwrap();
    twitchchat::Encoder::new(&conn)
        .encode(twitchchat::commands::register(&config))
        .unwrap();

    for msg in twitchchat::Decoder::new(&conn) {
        eprintln!("{:#?}", msg.unwrap())
    }

[bdashore3]

I was able to connect in 0.11 and send events, so that URL should work unless something was changed.

I'll talk to the dev and possibly add him into this conversation

[museun]

And I just tried with a websocket crate and I cannot connect with it either.

    async_std::task::block_on(async move {
        let addr = "ws://irc.fdgt.dev";
        let (mut stream, response) = async_tungstenite::async_std::connect_async(addr)
            .await
            .unwrap(); // <-- fails here -- so the server is refusing the connections

        eprintln!("{:#?}", response);

        while let Some(msg) = stream.next().await {
            eprintln!("{:#?}", msg)
        }
    });

[trezy]

Hey there, I'm the fdgt guy. fdgt does support TCP, but only with TLS (this is a limitation of the .dev TLD). I'll work with @bdashore3 to take a look here in a bit and see if there's anything going on in the logs.

[museun]

The docs do not list the port, but because of the .dev TLD limitation mentioned above, I assume the port is 6667 and not 6697. I also assume the domain-validated certificate is fdgt.dev. With these assumptions I cannot get either native-tls nor rustls to connect.

(This crate can use either TLS library (one is a wrapper around the "system" TLS library (schannel, secureconnect, openssl, libressl, etc) and the is an audited library written mostly in Rust.)

[museun]

And oddly enough, they fail at different points.

// native-tls = "0.2.4"
// rustls = "0.18.1"
// webpki = "0.21.3"
// webpki-roots = "0.20.0"

use std::io::{Read, Write};

const DOMAIN: &str = "irc.fdgt.dev"; // also tried just 'fdgt.dev'
const ADDRESS: &str = "irc.fdgt.dev:6667";

type Result<R> = std::result::Result<R, Box<dyn std::error::Error>>;

fn native_tls() -> Result<impl Read + Write> {
    let stream = std::net::TcpStream::connect(ADDRESS)?;

    let connector = native_tls::TlsConnector::new()?;
    let stream = connector.connect(DOMAIN, stream)?;

    Ok(stream)
}

fn rustls() -> Result<impl Read + Write> {
    let stream = std::net::TcpStream::connect(ADDRESS)?;

    let mut config = rustls::ClientConfig::new();
    config
        .root_store
        .add_server_trust_anchors(&webpki_roots::TLS_SERVER_ROOTS);

    let client = rustls::ClientSession::new(
        &std::sync::Arc::new(config),
        webpki::DNSNameRef::try_from_ascii_str(DOMAIN)?,
    );

    let stream = rustls::StreamOwned::new(client, stream);

    Ok(stream)
}

fn connect<IO: Read + Write>(connect: impl FnOnce() -> Result<IO>) -> bool {
    let login = b"PASS justinfan4567\r\nNICK justinfan4567\r\n";

    let mut stream = match connect() {
        Ok(stream) => stream,
        Err(err) => {
            eprintln!("cannot connect: {}", err);
            return false;
        }
    };

    match stream.write_all(login) {
        Ok(..) => {
            eprintln!("succesfully wrote to socket");
            true
        }
        Err(err) => {
            eprintln!("cannot write: {}", err);
            false
        }
    }
}

fn main() {
    if !connect(native_tls) {
        eprintln!("failed for native_tls")
    }

    if !connect(rustls) {
        eprintln!("failed for rustls")
    }
}

// cannot connect: An existing connection was forcibly closed by the remote host. (os error 10054)        
// failed for native_tls
// cannot write: An existing connection was forcibly closed by the remote host. (os error 10054)
// failed for rustls

I don't have the time to deal with figuring out where exactly the TLS handshake is failing.

But this little example is essentially what this crate does for either TLS library.

[trezy]

Ah, it turns out I introduced an issue with TCP with some updates last week. The issue should now be resolved on port 6667. Port 6697 sill be fixed up shortly, as well.

[museun]

I still cannot get either rust library to connect. Both are failing to complete the connection. I'll try on with openssl, libressl and boringssl to hopefully figure out where the misconfiguration is.

On Windows, SChannel (via native-tls) is reporting an unsupported cipher scheme (although the TLS cert for the site reports TLS_AES_128_GCM_SHA256 which is definitely a common cipher for TLS v1.3). And rustls isn't being helpful and just saying the server sends a corrupt message after the 'hello' part of the handshake.

update:
note: TLS 1.3 isn't supported on Windows before Windows 10 1903 -- and after that its experimental and requires a registry key to enable it in SChannel ref

Will TLS 1.3 be supported in Windows 10 and Server?
TLS 1.3 is also supported on Windows 1903 as of release of this article for testing purposes only, not production environment.

And is implied that in the next stable release will be supported ref

[museun]

I just tested a wss connection, I can get this crate to work with your service but I'd have to implement a websocket transport. It shouldn't take much time or effort to provide a WebSocket Connector, though.

[bdashore3]

I can also confirm the same errors over TCP using port 6667 using @museun's code

Here are the errors that I receive using WSL

cannot connect: error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../ssl/record/ssl3_record.c:331:
failed for native_tls
cannot write: received corrupt message
failed for rustls

[museun]

I've started work on adding a websocket client so this service can be used, but I've ran into some issues.

First, this is the initial response I'm given.

:tmi.twitch.tv CAP * ACK twitch.tv/membership
:tmi.twitch.tv CAP * ACK twitch.tv/tags
:tmi.twitch.tv CAP * ACK twitch.tv/commands
:tmi.twitch.tv 001 justinfan1234 :Welcome, GLHF!
:tmi.twitch.tv 002 justinfan1234 :Your host is tmi.twitch.tv
:tmi.twitch.tv 003 justinfan1234 :This server is rather new
:tmi.twitch.tv 004 justinfan1234 :-
:tmi.twitch.tv 375 justinfan1234 :-
:tmi.twitch.tv 372 justinfan1234 :You are in a maze of twisty passages, all alike.
:tmi.twitch.tv 372 justinfan1234 :Your FDGT connection ID is f3895f5c-fa1d-454e-a410-69a28a6e70cf.
:tmi.twitch.tv 376 justinfan1234 :>

The CAP responses aren't the same as what Twitch sends. ref Note the leading colon before ACK.

Although its not explicitly stated in IRCv3.1, IRCv3.2 -- all of the examples show the server repeating the capability verbatim.

E.g.

Client: CAP REQ :multi-prefix
Server: CAP * ACK multi-prefix

Client: CAP REQ :multi-prefix sasl
Server: CAP * ACK :multi-prefix sasl

Client: CAP REQ :-userhost-in-names
Server: CAP * ACK :-userhost-in-names

So, my parser will reject these messages.

And with the IETF draft, the ABNF for ACK also show the colon:

(the important bits of the grammar)

          caplist      =  [ ":" ] *( capmod ) capab
          caplist      =/ ":" *( capmod ) capab 1*( SP *( capmod ) capab )
          ackcmd       =  "ACK" SP [ "*" SP ] caplist

I'm going to go ahead and special case this (because Twitch's IRC isn't really spec compliant so whatever), but I thought I should bring it up.