hey i was writing something similar to ur code. How does ur code pushes packets to ffmpeg?

[mohit810]

No description provided.

[grantfayvor]

Hi. Yes, it does work. However, it does a VP8 -> H264 conversion which is quite CPU intensive and can be avoided; you're bound to run into problems with it eventually.

You can instead require H264 video input and send that to ffmpeg directly. I just got around to the rewrite yesterday and would update the repo when I'm done.

This thread pion/example-webrtc-applications#62 should be useful.

[mohit810]

hi, actually my full project description and repo is at https://github.com/mohit810/streamingcdn . My main goal is "To receive the broadcast, Convert it into hls in various quality then push those to cdn to serve to the User." . I was understanding ffmpeg's documentation for basic understanding. The main thing for me is to understand how does the rtp packets will reach to ffmpeg function, because when i am adding the push opus/vp8 & ffmpeg code mine videoBuilder.Pop() is always null and when it's not, the videoKeyframe := (sample.Data[0]&0x1 == 0) videoKeyframe is false. It would be really helpfull if u could give me pointers..

[grantfayvor]

Hey, what is the codec for your input video?

Also, audioWriter and videoWriter would be initialized when the first packets are sent and ffmpeg is also called; check this line.

Subsequent writes to the Writer interfaces would update the feed to ffmpeg check this line and this

[grantfayvor]

Also, since you're using hls, you should be able to send H264 directly to the HLS stream without doing the VP8 conversion. Depends on your use case.

[mohit810]

Hey, what is the codec for your input video?

Also, audioWriter and videoWriter would be initialized when the first packets are sent and ffmpeg is also called; check this line.

Subsequent writes to the Writer interfaces would update the feed to ffmpeg check this line and this

The problem is that in my code the videoKeyframe := (sample.Data[0]&0x1 == 0) and videoKeyframe is false and the code never reaches to the ffmpeg function call. I will add a dev branch to update the code with daily progress.

[grantfayvor]

Okay. Meanwhile, I just updated the repo following the style in this example.

With the initial code, I also noticed that videoKeyFrame was always false if the video input was from navigator.getdisplaymedia for screen capture but worked fine for webcam capture. That's why I asked about your input video format.

[mohit810]

Okay. Meanwhile, I just updated the repo following the style in this example.

With the initial code, I also noticed that videoKeyFrame was always false if the video input was from navigator.getdisplaymedia for screen capture but worked fine for webcam capture. That's why I asked about your input video format.

I have created the dev branch(https://github.com/mohit810/streamingcdn/tree/dev-branch) and have refactored the code similar to ur code. Please check it out when u have the time as after refactoring it, I am able to initiate ffmpeg. Thanks!! If u wanna work on the same , do let me know.

[grantfayvor]

I'm glad you were able to get it to work. Were you able to convert the stream to HLS?

Sorry I haven't yet been able to add a readme :( would do that soon.

[mohit810]

I'm glad you were able to get it to work. Were you able to convert the stream to HLS?

Sorry I haven't yet been able to add a readme :( would do that soon.

Yeah I am able to convert the packets to HLS, it's working and there are some issues (there are some stream speed, quality,.. issues ), solving these issue.

[grantfayvor]

Okay great. How are you going about solving the stream quality issues?

[mohit810]

Okay great. How are you going about solving the stream quality issues?

Currently, I am reading and understanding different examples and solution to resolve that.

[grantfayvor]

Okay.

[lichao201626]

@grantfayvor hi, you are using ffmpeg some code like this, exec.Command("ffmpeg", ...). A ffmpeg process running, I want to know when the process will stop? So I want the ffmpeg process stop automatically when my video broadcast stop, can you give me some pointer, thanks!

[lichao201626]

get it, may be I can use exec.CommandContext with cancel!

[grantfayvor]

Yes @lichao201626 you can use the commandcontext and cancel when done.

[mohit810]

Yes @lichao201626 you can use the commandcontext and cancel when done.

Hey @grantfayvor i have given u access in the repository and @lichao201626 if u wanna join and contribute also, do let me know.

[grantfayvor]

Sure, meanwhile I updated the ffmpeg call to use exec.CommandContext just like @lichao201626 suggested, so that it can be cancelled.

[mohit810]

Sure, meanwhile I updated the ffmpeg call to use exec.CommandContext just like @lichao201626 suggested, so that it can be cancelled.

Yeah that feature is surely handy..

[mohit810]

@grantfayvor i have put together the following code which runs smoothly on terminal
ffmpeg -hide_banner -re -i sample720.mp4 \ -map 0:v:0 -map 0:a:0 -map 0:v:0 -map 0:a:0 -map 0:v:0 -map 0:a:0 -map 0:v:0 -map 0:a:0 \ -c:v h264 -profile:v main -pix_fmt yuv420p -preset faster -crf 20 -sc_threshold 0 -g 48 -keyint_min 48 -c:a aac -ac 2 -ar 48000 \ -filter:v:0 scale=w=640:h=360:force_original_aspect_ratio=decrease -maxrate:v:0 856k -bufsize:v:0 1200k -b:v:0 800k -b:a:0 96k \ -filter:v:1 scale=w=842:h=480:force_original_aspect_ratio=decrease -maxrate:v:1 1498k -bufsize:v:1 2100k -b:v:1 1400k -b:a:1 128k \ -filter:v:2 scale=w=1280:h=720:force_original_aspect_ratio=decrease -maxrate:v:2 2996k -bufsize:v:2 4200k -b:v:2 2800k -b:a:2 128k \ -filter:v:3 scale=w=1920:h=1080:force_original_aspect_ratio=decrease -maxrate:v:3 5350k -bufsize:v:3 7500k -b:v:3 5000k -b:a:3 192k \ -var_stream_map "v:0,a:0 v:1,a:1 v:2,a:2 v:3,a:3" -hls_time 4 -hls_playlist_type vod \ -master_pl_name master.m3u8 -hls_segment_filename video_%v_%03d.ts quality_%v.m3u8

the same code for golang is:
ffmpeg := exec.Command("ffmpeg", "-protocol_whitelist", "file,udp,rtp", "-i", "rtp-forwarder.sdp", "-map", "0:v:0", "-map", "0:a:0", "-map", "0:v:0", "-map", "0:a:0", "-map", "0:v:0", "-map", "0:a:0", "-map", "0:v:0", "-map", "0:a:0", "-c:v", "h264", "-profile:v" , "main", "-pix_fmt", "yuv420p", "-preset", "faster", "-crf", "20", "-sc_threshold", "0" , "-g" , "48", "-keyint_min", "48", "-c:a", "aac", "-ac", "2", "-ar", "48000", "-filter:v:0", "scale=w=640:h=360:force_original_aspect_ratio=decrease", "-maxrate:v:0", "856k", "-bufsize:v:0", "1200k", "-b:v:0", "800k","-b:a:0", "96k", "-filter:v:1", "scale=w=842:h=480:force_original_aspect_ratio=decrease", "-maxrate:v:1", "1498k", "-bufsize:v:1", "2100k", "-b:v:1", "1400k", "-b:a:1", "128k", "-filter:v:2", "scale=w=1280:h=720:force_original_aspect_ratio=decrease", "-maxrate:v:2", "2996k", "-bufsize:v:2", "4200k", "-b:v:2", "2800k", "-b:a:2", "128k", "-filter:v:3", "scale=w=1920:h=1080:force_original_aspect_ratio=decrease", "-maxrate:v:3", "5350k", "-bufsize:v:3", "7500k", "-b:v:3", "5000k", "-b:a:3", "192k", "-var_stream_map", "v:0,a:0 v:1,a:1 v:2,a:2 v:3,a:3", "-hls_time", "4", "-hls_playlist_type", "vod", "-master_pl_name", "master.m3u8", "-hls_segment_filename", "video_%v_%03d.ts", "quality_%v.m3u8")

the problem right now is the terminal version creates playlist as well as master playlist but golang version doesn' t create those. I am looking for the solution for this but if u find something that i missed, do let me know

[grantfayvor]

hi @mohit810 sadly, i do not know so much about ffmpeg commands.

[mohit810]

Sure, meanwhile I updated the ffmpeg call to use exec.CommandContext just like @lichao201626 suggested, so that it can be cancelled.

hey @grantfayvor i am still getting:
EOF write udp 127.0.0.1:54307->127.0.0.1:4000: use of closed network connection
in terminal when user disconnects from the server even after applying this new patch in https://github.com/mohit810/streamingcdn/tree/dev-branch

[grantfayvor]

Hi, @mohit810 I noticed that as well but I haven't gotten a chance to work on it yet. It might be happening because the goroutine on this line wasn't exited. You can refactor it to depend on context as well.

[mohit810]

Hi, @mohit810 I noticed that as well but I haven't gotten a chance to work on it yet. It might be happening because the goroutine on this line wasn't exited. You can refactor it to depend on context as well.

hey @grantfayvor can u clone https://github.com/mohit810/streamingcdn and run it and let me know if hls conversion is working or not because its working on other setups other than mine. just want to confirm.. post successful run there should me 5 .m3u8 extension files as well as various .ts extension file

[lichao201626]

@mohit810 That will be pretty cool if I can join and contribute, though I don't have much time

[mohit810]

@grantfayvor I have released the v3.1 in which, the code is converting webrtc -> HLS . I saw that you are setting up dynamic ports code(that's really awesome).. Once yours is complete , I will also integrate that one as well. Till then i will make my code more modular and add master playlist...

[grantfayvor]

Hi @mohit810 Yeah I added some code to the dynamic-udp-ports branch. It currently picks a random unassigned port in range of 1000 possible ports. It works but I've wondered if there's a better way to go about it. If you have a better suggestion I'm open to it.

[mohit810]

Hi @mohit810 Yeah I added some code to the dynamic-udp-ports branch. It currently picks a random unassigned port in range of 1000 possible ports. It works but I've wondered if there's a better way to go about it. If you have a better suggestion I'm open to it.

@grantfayvor I don't know anything about that. Even if I try to understand the basic workings on things, it would take time and right now I am working on the final steps before cdn integration of the webrtc - > HLS. Will let you know if I get free, because post complete ffmpeg integration with cdn i have to build the ui on android, ios, web. Try pion slack. There are many people in slack with experience in using pion.