Inbound calls from classic sip endpoint (RTP) to WebRTC endpoint (SRTP) not working.

[guy032]

We started this conversation privately but just so everyone else can follow
and track our progress I will put it here too:

This module is currently in a state that allows us to make outbound calls from WebRTC client such as SIP.js to classic sip endpoint such as Bria but not the other way around (meaning we cannot receive any inbound calls from Bria endpoints to SIP.js endpoints.

The problem is with RTP (Bria) -> SRTP (SIP.js) invitation.

Looking for any ideas what is the blueprint to implement this.

Thanks!

[davehorton]

The issue is that freeswitch does not support currently (as best I can tell, after reviewing the code and discussions with some freeswitch devs) generating an SRTP offer from an incoming 3pcc INVITE.

Stepping back, the way we generate an outbound call leg from drachtio where the local endpoint is freeswitch is that we send a 3pcc INVITE (an INVITE with no sdp) to freeswitch; we take the sdp from the 200 OK response freeswitch and send out our INVITE with that as the offer. (When we get a 200 OK to our INVITE we send the ACK to freeswitch with that sdp, thereby connecting the media stream).

So therefore, the only solution I see to making this work is for me to add this feature (returning srtp sdp offer in response to 3pcc INVITE) to my fork of freeswitch. Its a bit of work, which I have not yet found time for. It also has the downside of forcing people to use my freeswitch build rather than a stock build, though I am not sure how much of a downside that is at this point since most people using my freeswitch plugins (for google speech, transcribe, dialogflow, or forking audio) probably are using my build scripts already.

Anyways, that is what needed.

[guy032]

To summarize things:

Your suggestion for Freeswitch: Returning SRTP SDP offer in response to 3PCC INVITE.
The issue is that freeswitch does not support currently generating an SRTP offer from an incoming 3PCC INVITE ... It also has the downside of forcing people to use my Freeswitch build rather than a stock build

If it requires some modifications in the C files of Freeswitch we best wait for you to complete this.
Maybe once you do that you can try to merge these changes into Freeswitch latest version to prevent 2 separated ways?

SRTP -> RTP Outbound call:

1. Send 3PCC INVITE (no SDP) to Freeswitch.
2. Take SDP from the 200 OK response.
3. Send INVITE to callee with that SDP.
4. Get 200 OK for the INVITE.
5. Send ACK with that SDP (connecting the media stream).

RTP -> SRTP Inbound call:
What's the flow once Freeswitch is capable of returning SRTP SDP offer in response to 3PCC INVITE?
@davehorton

[davehorton]

The flow will be pretty simple. You will create an endpoint (there will need to be a new option to request SRTP) then use Srf#createUAC to send an invite, e.g.

const ep = await ms.createEndpoint({srtp: true});
const uac = await srf.createUAC(uri, {localSdp: ep.local.sdp}):
...

[guy032]

Just to keep track for everyone
I believe this is the part to handle 3PCC INVITE in Freeswitch 1.8.5:
https://github.com/signalwire/freeswitch/blob/v1.8.5/src/mod/endpoints/mod_sofia/sofia.c

if (sofia_test_pflag(profile, PFLAG_3PCC)) {
    if (switch_channel_test_flag(channel, CF_PROXY_MODE) || switch_channel_test_flag(channel, CF_PROXY_MEDIA)) {
        switch_log_printf(SWITCH_CHANNEL_SESSION_LOG(session), SWITCH_LOG_INFO, "No SDP in INVITE and 3pcc=yes cannot work with bypass or proxy media, hanging up.\n");
        switch_channel_set_variable(channel, SWITCH_ENDPOINT_DISPOSITION_VARIABLE, "3PCC DISABLED");
        switch_channel_hangup(channel, SWITCH_CAUSE_MANDATORY_IE_MISSING);
    } else {
        switch_channel_set_variable(channel, SWITCH_ENDPOINT_DISPOSITION_VARIABLE, "RECEIVED_NOSDP");
        switch_core_media_choose_port(tech_pvt->session, SWITCH_MEDIA_TYPE_AUDIO, 0);
        switch_core_media_prepare_codecs(session, 1);
        switch_channel_set_state(channel, CS_HIBERNATE);
        switch_core_media_gen_local_sdp(session, SDP_TYPE_REQUEST, NULL, 0, NULL, 0);
        sofia_set_flag_locked(tech_pvt, TFLAG_3PCC);

        if (sofia_use_soa(tech_pvt)) {
            nua_respond(tech_pvt->nh, SIP_200_OK,
                        SIPTAG_CONTACT_STR(tech_pvt->reply_contact),
                        SOATAG_USER_SDP_STR(tech_pvt->mparams.local_sdp_str),
                        SOATAG_REUSE_REJECTED(1),
                        SOATAG_AUDIO_AUX("cn telephone-event"),
                        TAG_IF(sofia_test_pflag(profile, PFLAG_DISABLE_100REL), NUTAG_INCLUDE_EXTRA_SDP(1)), TAG_END());
        } else {
            nua_respond(tech_pvt->nh, SIP_200_OK,
                        NUTAG_MEDIA_ENABLE(0),
                        SIPTAG_CONTACT_STR(tech_pvt->reply_contact),
                        SIPTAG_CONTENT_TYPE_STR("application/sdp"), SIPTAG_PAYLOAD_STR(tech_pvt->mparams.local_sdp_str), TAG_END());
        }
    }
}

Maybe the same pattern they used for mod_verto can be used:
https://github.com/signalwire/freeswitch/blob/v1.8.5/src/mod/endpoints/mod_verto/mod_verto.c

This function switch_core_media_gen_local_sdp seems to be the one to create the RTP SDP:
https://github.com/signalwire/freeswitch/blob/v1.8.5/src/switch_core_media.c
How can we change this to SRTP SDP instead?

There are 3 conditions to enable SRTP:

1. session->channel->flags['CF_AVPF']
2. (session->channel->flags['CF_DTLS'] || secure)
3. session->channel->flags['CF_AVPF_MOZ']

static const char *get_media_profile_name(switch_core_session_t *session, int secure)
{
	switch_assert(session);

	if (switch_channel_test_flag(session->channel, CF_AVPF)) {
		if (switch_channel_test_flag(session->channel, CF_DTLS) || secure) {
			if (switch_channel_test_flag(session->channel, CF_AVPF_MOZ)) {
				return "UDP/TLS/RTP/SAVPF";
			} else {
				return "RTP/SAVPF";
			}
		} else {
			if (switch_channel_test_flag(session->channel, CF_AVPF_MOZ)) {
				return "UDP/AVPF";
			} else {
				return "RTP/AVPF";
			}
		}
	}

	if (secure) {
		return "RTP/SAVP";
	}

	return "RTP/AVP";

}

The conditions to put CF_AVPF flag are calling switch_core_session_set_ice or:

if (is_outbound || switch_channel_test_flag(session->channel, CF_RECOVERING) ||
    switch_channel_test_flag(session->channel, CF_3PCC)) {
    if (!switch_channel_test_flag(session->channel, CF_AVPF) &&
        switch_true(switch_channel_get_variable(session->channel, "media_webrtc"))) {
        switch_channel_set_flag(session->channel, CF_AVPF);
        switch_channel_set_flag(session->channel, CF_ICE);
        smh->mparams->rtcp_audio_interval_msec = SWITCH_RTCP_AUDIO_INTERVAL_MSEC;
        smh->mparams->rtcp_video_interval_msec = SWITCH_RTCP_VIDEO_INTERVAL_MSEC;
    }

    if (switch_true(switch_channel_get_variable(session->channel, "add_ice_candidates"))) {
        switch_channel_set_flag(session->channel, CF_ICE);
    }

    if ( switch_rtp_has_dtls() && dtls_ok(session)) {
        if (switch_channel_test_flag(session->channel, CF_AVPF) ||
            switch_true(switch_channel_get_variable(smh->session->channel, "rtp_use_dtls"))) {
            switch_channel_set_flag(smh->session->channel, CF_DTLS);
            switch_channel_set_flag(smh->session->channel, CF_SECURE);
            generate_local_fingerprint(smh, SWITCH_MEDIA_TYPE_AUDIO);
        }
    }
    switch_core_session_parse_crypto_prefs(session);
    switch_core_session_check_outgoing_crypto(session);
}

The conditions to put CF_DTLS flag are the same as above (CF_AVPF) or calling check_ice:

if (switch_rtp_has_dtls() && dtls_ok(smh->session) && !strcasecmp(attr->a_name, "fingerprint") && !zstr(attr->a_value)) {
    char *p;

    engine->remote_dtls_fingerprint.type = switch_core_session_strdup(smh->session, attr->a_value);

    if ((p = strchr(engine->remote_dtls_fingerprint.type, ' '))) {
        *p++ = '\0';

        if (switch_channel_test_flag(smh->session->channel, CF_REINVITE) && !switch_channel_test_flag(smh->session->channel, CF_RECOVERING) &&
            !zstr(engine->remote_dtls_fingerprint.str) && !strcmp(engine->remote_dtls_fingerprint.str, p)) {
            engine->new_dtls = 0;
        } else {
            switch_set_string(engine->remote_dtls_fingerprint.str, p);
            engine->new_dtls = 1;
            engine->new_ice = 1;
        }
    }

    generate_local_fingerprint(smh, type);
    switch_channel_set_flag(smh->session->channel, CF_DTLS);

}

There is also the possibility to send true for the secure parameter but the conditions are many.

The conditions to put CF_AVPF_MOZ flag:

if (m->m_proto_name && !strcasecmp(m->m_proto_name, "UDP/TLS/RTP/SAVPF")) {
    switch_channel_set_flag(session->channel, CF_AVPF_MOZ);
}

if (m->m_proto_name && !strcasecmp(m->m_proto_name, "UDP/RTP/AVPF")) {
    switch_channel_set_flag(session->channel, CF_AVPF_MOZ);
}

Hope this can help to prove or disprove your assumption that freeswitch does not support currently generating an SRTP offer from an incoming 3PCC INVITE

@davehorton Do you think you can alter these conditions from drachtio-fsmrf for 3PCC invite without changing the source code of Freeswitch?

[guy032]

According to this answer:
https://stackoverflow.com/questions/61523132/freeswitch-support-for-generating-an-srtp-offer-from-an-incoming-3pcc-invite

Try to set media_webrtc=true this should tell freeswitch to use DTLS for media stream, WebRTC signaling should be done separately.
<action application="bridge" data="[media_webrtc=true]sofia/internal/100%${sip_profile}"/>

This should work to allow us to get back SRTP SDP for 3PCC INVITE.
@davehorton Can you confirm this is enough for you to implement a solution for this issue?

[davehorton]

Unfortunately....nope.

That answer betrays a lack of understanding of how 3PCC works on Freeswitch. The SDP is generated (in this case) before any application is run. And before any channel variables can be set.

I still think it will take a patch to the freeswitch core. Something I still plan to do, but a question of time (/ working on open source once I pay my bills haha :)

[davehorton]

Feel free to try though, if you think for some reason it will work. Let me know if you prove me wrong

[guy032]

Sure! Just wanted to confirm nobody can provide an answer to this question with Freeswitch.
As you best understand what should be done I guess we will just need to wait for you.
Let me know if you've got anything you want me to try in the meantime with Freeswitch (with your guidance to save for you time).
We can continue the discussion about the payment for you in private of course.

[guy032]

I believe this issues is the only thing left for a truly functional Drachtio framework with all web capabilities. Please consider this contribution to this amazing framework as it truly what it takes to make it possible for everbody to get far with this framework! If you have any response I will truly appreciate it @davehorton

[guy032]

Honestly this project is kind of useless for me in today's world where everything is about the web and something as necessary as the webrtc cannot play at all with this project. I will abandon this project in search for alternatives. Kind of hoped that after trying so hard to get some attention with both money and emails we will be able to move a step forward in the right direction for this library. Anyway thanks @davehorton for all the hard work! for people who really just need the old fashioned deskphones or fully native sip clients I guess it will be great to continue with this.

[davehorton]

leaving this issue open for now, in case I implement this feature later.

For others coming across this thread in the future with similar needs, I suggest doing what I do which is to use an rtpengine (with my npmjs rtpengine-client module) in front of freeswitch, and let rtpengine handle the SRTP/RTP transcoding. rtpengine provides much more configurability in this regard than freeswitch

[guy032]

@davehorton

use an rtpengine (with my npmjs rtpengine-client module) in front of freeswitch

Do you have any example to show how to do it?

[guy032]

I was checking both:

https://github.com/davehorton/drachtio-rtpengine-webrtcproxy
and https://github.com/davehorton/rtpengine-client

and didn't see any example of the idea to put RTPengine in front of Freeswitch.
An example of how to do it would be so much appreciated!

[davehorton]

Here are a few notes...

So you can either run rtpengine on the same server as freeswitch, or different. If the same, make sure each is configured with a non-overlapping rtp port range.

First you are going to want to get familiar with its ng protocol.

In your node app, use my rtpengine-client to control rtpengine using ng protocol.

So then, for an outbound call you would do this:

1. Call Mediaserver#createEndpoint() to get a plain (non-encrypted) SDP from freeswitch.
2. Send that SDP in an ng offer command to rtpengine, with options indicating you want to use SRTP
3. rtpengine will return an SRTP SDP
4. use that SDP in a call to Srf#createUAC
5. When you get a 183/200 from far end with an SDP, send that SDP in an ng answer command to rtpengine, indicating you want plain RTP
6. rtpengine will return an RTP SDP
7. Use that SDP in a call to Endpoint#modify to instruct freeswitch where to stream audio.

At that point the call will be connected from freeswitch to rtpengine (plain RTP) and from rtpengine to your far end client (SRTP).

Make sure to call ng delete to free the rtpengine resources when the call ends (or fails)

[guy032]

Hi many thanks! Do you have any gist on Github to show the code example for the steps you mentioned?

[davehorton]

I have plenty of projects where I've used rtpengine but no git showing exactly what I outline above

[guy032]

Still no plans to make drachtio-fsmrf capable of providing incoming calls to WebRTC? @davehorton
Looking forward to hear what can be done other than using rtpengine as a middleware
thanks.