Refactor Proposal: Move all network communication to the transport layer, deprecate the Libp2p data-transfer protocol

[hannahhoward]

What

Go-data-transfer performs several functions

• managing data transfer state (and encoding it)
• monitoring and triggering restarts
• encoding messages
• authenticating requests
• managing pause/resume dynamics for a gated exchange
• communicating with underlying transport protocols
• managing a data transfer libp2p protocol to augment gaps in a transport protocol

These last two are pretty weird -- the Transport interface as written is designed mostly around Graphsync, with the data transfer Libp2p protocol being "do all the libp2p communication graphsync can't do on its own".

In reality transport protocols are wildly different. It's highly unlikely that the libp2p data-transfer protocol as written will work for anything other than filling the gaps in graphsync.

In order to support multiple transport protocols, the "Transport" interface needs to be more more high level -- it probably needs to handle all network communication, with data transfer itself only handling the higher level functions that are truly transport independent.

The basic path forward, which we already have laid groudwork for is:

• modify the transport interface to make it more high level:

Example:

// EventsHandler are semantic data transfer events data transfer can respond to
type EventsHandler interface {
	// OnChannelOpened is called when we ask the other peer to send us data on the
	// given channel ID
	// return values are:
	// - nil = this channel is recognized
	// - error = ignore incoming data for this channel
	OnChannelOpened(chid ChannelID) error
	// OnResponseReceived is called when we receive a response to a request
	// - nil = continue receiving data
	// - error = cancel this request
	OnResponseReceived(chid ChannelID, msg Response) error
	// OnDataReceived is called when we receive data for the given channel ID
	// return values are:
	// - nil = proceed with sending data
	// - error = cancel this request
	// - err == ErrPause - pause this request

	OnDataReceived(chid ChannelID, link ipld.Link, size uint64) error

	// OnDataQueued is called when data is queued for sending for the given channel ID
	// return values are:
	// message = data transfer message along with data
	// err = error
	// - nil = proceed with sending data
	// - error = cancel this request
	// - err == ErrPause - pause this request
	OnDataQueued(chid ChannelID, link ipld.Link, size uint64) (Message, error)

	// OnDataSent is called when we send data for the given channel ID
	OnDataSent(chid ChannelID, link ipld.Link, size uint64) error

	// OnTransferQueued is called when a new data transfer request is queued in the transport layer.
	OnTransferQueued(chid ChannelID)

	// OnRequestReceived is called when we receive a new request to send data
	// for the given channel ID
	// return values are:
	// message = data transfer message along with reply
	// err = error
	// - nil = proceed with sending data
	// - error = cancel this request
	// - err == ErrPause - pause this request (only for new requests)
	// - err == ErrResume - resume this request (only for update requests)
	OnRequestReceived(chid ChannelID, msg Request) (Response, error)
	// OnChannelCompleted is called when we finish transferring data for the given channel ID
	// Error returns are logged but otherwise have no effect
	OnChannelCompleted(chid ChannelID, err error) error

	// OnRequestCancelled is called when a request we opened (with the given channel Id) to
	// receive data is cancelled by us.
	// Error returns are logged but otherwise have no effect
	OnRequestCancelled(chid ChannelID, err error) error

	// OnRequestDisconnected is called when a network error occurs trying to send a request
	OnRequestDisconnected(chid ChannelID, err error) error

	// OnSendDataError is called when a network error occurs sending data
	// at the transport layer
	OnSendDataError(chid ChannelID, err error) error

	// OnReceiveDataError is called when a network error occurs receiving data
	// at the transport layer
	OnReceiveDataError(chid ChannelID, err error) error
	
        // OnRestartExistingChannelRequestReceived
        // is called when the responding peer sends a request to restart
        OnRestartExistingChannelRequestReceived(chid ChannelID, msg Request) error
}

Transport is the minimum interface that must be satisfied to serve as a datatransfer
transport layer. Transports must be able to open (open is always called by the receiving peer)
and close channels, and set at an event handler */
type Transport interface {
	// OpenChannel initiates an outgoing request for the other peer to send data
	// to us on this channel
	// Note: from a data transfer symantic standpoint, it doesn't matter if the
	// request is push or pull -- OpenChannel is called by the party that is
	// intending to receive data
	OpenChannel(ctx context.Context,
		channelID ChannelID,
		root ipld.Link,
		stor ipld.Node,
		msg Message) error
        SendMessage(ctx, chid ChannelID, message Message);
	// CloseChannel closes the given channel
	CloseChannel(ctx context.Context, chid ChannelID) error
	// SetEventHandler sets the handler for events on channels
	SetEventHandler(events EventsHandler) error
	// CleanupChannel is called on the otherside of a cancel - removes any associated
	// data for the channel
	CleanupChannel(chid ChannelID)
	Shutdown(ctx context.Context) error
}

// PauseableTransport is a transport that can also pause and resume channels
type PauseableTransport interface {
	Transport
	// PauseChannel paused the given channel ID
	PauseChannel(ctx context.Context,
		chid ChannelID,
	) error
	// ResumeChannel resumes the given channel
	ResumeChannel(ctx context.Context,
		msg Message,
		chid ChannelID,
	) error
}

• move the data transfer protocola implementation into the graphsync transport layer implementation (possibly rename something like "graphsync-data-transfer" or look at truly augmenting the graphsync protocol to support everything needed
• remove the "receiver" implementation from go data transfer manager