From 9c6228b3f27c842e330249298fdb12a89e438b24 Mon Sep 17 00:00:00 2001 From: Christian Spielberger Date: Mon, 2 Oct 2023 14:49:22 +0200 Subject: [PATCH] docs: remove obsolete jbuf doc --- docs/jbuf/README.md | 188 -------------------------------------------- docs/jbuf/jbuf.plot | 79 ------------------- 2 files changed, 267 deletions(-) delete mode 100644 docs/jbuf/README.md delete mode 100755 docs/jbuf/jbuf.plot diff --git a/docs/jbuf/README.md b/docs/jbuf/README.md deleted file mode 100644 index 7413c2eac..000000000 --- a/docs/jbuf/README.md +++ /dev/null @@ -1,188 +0,0 @@ -This adaptive jitter buffer implementation increases the number of packets in -the buffer during periods of high network jitter. It reduces the number of -packets if the network condition improves. - - -## Computing the jitter - -The network jitter is computed similar to the proposition in RFC-3550 RTP -section A.8 and similar how wireshark does it. The jitter is a moving average -of the difference *d* between the real time and the RTP timestamps. Or more -concretely we compute at each `jbuf_put()` the difference - -*d = Dtr - Dts*, - -where *Dtr* is the real time elapsed from last call to -`jbuf_put` and *Dts* is the difference of the timestamps. Then -with a predefined speed *s* we compute the jitter *j* as moving average - -*j = j + s (|d| - j)*. - -We choose a higher value for speed *s* if *|d| > j*. Thus the jitter rises fast -if e.g. suddenly a network jitter appears. In contrast when the network -condition improves the jitter value slowly shrinks. The reason for different -rising and falling speed is that we have to react fast to avoid buffer -under-runs, whereas reducing of the latency may be done a while after the -network condition improved. - -In the following sections we will describe how the computed jitter is used to -detect situations where the buffered packets should be increased due to a high -jitter. We call this situations **Low** situations. When the jitter shrinks -below some specific value it is a good idea to reduce the buffer to reduce the -audio latency. We call this situations **High** situations. Surely, the -Low/High situations have to be decided somehow. - -## Reduce/Increase buffered packets - -When a Low situation is detected we increase the number of packets in jbuf by -holding back a packet during one call to function `jbuf_get()`. While when -a High situation is detected we reduce the number of packets by telling baresip -to immediately call `jbuf_get()` a second time. For this purpose `jbuf_get()` -returns EAGAIN. Then baresip decodes two RTP packets and thus the audio buffer -increases by one frame. By means of a silence detection baresip is able to drop -frames that are not important for the speech quality. This reduces the audio -latency down to the value before the High situation. - - -## Computing a smooth latency - -The RTP packets that are buffered at a concrete point in time in the jbuf lead -to a temporary latency value lc. Let *p0, ..., -pm* be the RTP packets currently stored in the jbuf. Then - -*lc = tm - t0 + tp*, - -where *ti* is the RTP timestamp of packet *pi* and -*tp* is the packet time (`ptime` in jbuf.c). The packet time is a -constant that is specified at the beginning of a call. In baresip it is -specified in the account file. - -The temporary latency *lc* is discontinuous over time and not -adequate for deciding or detecting Low or High situations. Therefore we again -use a moving average to smooth *lc*. Let *s* be an adequate moving -average speed, then the smoothed latency - -*l = l + s (lc - l)*. - -Low/High situations are decided when the smoothed latency *l* runs out of some -boundaries that are computed from the jitter. - -## Deciding Low/High situations - -During each iteration (each `jbuf_put()`) the jitter and the latency are -computed. Additionally we compute the bottom boundary *lb* and the -top boundary *lt* with - -*lb = 1.25 j* and - -*lt = 2.2 j*. - -Since we want to respect also the parameter `min` of function `jbuf_alloc()` we -extend these formulas. Let *mm* be the parameter `min`. Then - -*lb = max(mm 2 tp / 3, 1.25 j)* and - -*lt = max(mm 11 tp / 3, 2.2 j)*, - -where *tp* is the `ptime` as defined already. Finally we have -everything for deciding Low and High situations. That is if *l* moves out of -the boundaries - -*lb < l < lt* - -then we fire a Low/High. - -## Early adjustment of the latency - -Finally, if we detect a Low/High situation we increase/reduce the number of -packets. Now we immediately increment/decrement the smoothed latency *l* by -one packet time. So early adjustment for a Low situation is - -*l = l + lp* and for a High situation -*l = l - lp*. - - -This avoids multiple Low/High detections in a row. - - -## Silence detection - -It is preferable to hold back an RTP packet in `jbuf_get()` only during a -period of silence. But deciding if there is currently silence in the RTP stream -can only be done by investigating the audio frames after decoding the RTP -packet. Decoding is done in baresip. Thus we add a function `jbuf_silence()` -that sets the flag `silence`. This function has to be called by baresip. - - -## Math symbols vs. C-variables - -In order to avoid float computation we use a constant factor -```JBUF_JITTER_PERIOD``` for all time based variables in jbuf.c. Apart from -that the symbols used here are mapped to the C-variables like this table shows: - - -Symbol|Variable -------|-------- -*d* | `d` -*j* | `jitter` -*l* | `avbuftime` -*lc* | `buftime` -*lb* | `bufmin` -*lt* | `bufmax` -*tp* | `ptime` - - -## Wish size - -We introduce also the parameter `wish` and a setter function `jbuf_set_wish()`. -The wish size is the number of packets that will be collected at the beginning -of an RTP stream before `jbuf_get()` will return the first packet. If the user -passes `wish=0` then it is set internally to `min`. If the user knows that the -network contains jitter he may set the wish size to some adequate value. This -avoids underflows at the beginning of the stream. The `min` parameter can be -left at some lower value. When the network situation improves, the buffer is -reduced down to `min`. Thus the latency is reduced to the specified minimum. - - -## How to test jbuf - -- In jbuf.c set DEBUG\_LEVEL to 6, build and install libre again! - -- Add bridge interface linked to your Ethernet/WiFi interface! Suppose baresip -is connected to the network interface *eth0*. Replace *eth0* with your physical -network interface! See the man pages of "ip" and "tc" for further details! - -``` -sudo ip link add ifb1 type ifb || : -sudo ip link set ifb1 up -sudo tc qdisc add dev eth0 handle ffff: ingress -sudo tc filter add dev eth0 parent ffff: u32 match u32 0 0 action mirred egress redirect dev ifb1 -``` - -This redirects the incoming eth0 traffic to a new ifb interface. - -- How to activate the jitter. Here we set the delay to 100ms ± 50ms. -``` -sudo tc qdisc add dev ifb1 root netem delay 100ms 50ms -``` - -- How to deactivate the jitter. -``` -sudo tc qdisc del dev ifb1 root -``` - -- See/Use jbuf.plot to generate a plot! - -Note: -- Activate and deactivate the network jitter during a call to see how the -adaptive jitter buffer algorithm works! -- You need a very new kernel or at least some patches for the sch_netem kernel -module. There was a problem which lead to many reordered and very late RTP -packets. Be sure that you have this commit in the kernel: -``` -commit eadd1befdd778a1eca57fad058782bd22b4db804 -Author: Aleksandr Nogikh -Date: Wed Oct 28 17:07:31 2020 +0000 - - netem: fix zero division in tabledist -``` diff --git a/docs/jbuf/jbuf.plot b/docs/jbuf/jbuf.plot deleted file mode 100755 index 5d681456d..000000000 --- a/docs/jbuf/jbuf.plot +++ /dev/null @@ -1,79 +0,0 @@ -#!/usr/bin/gnuplot -# -# How to generate a plot -# ====================== -# This gnuplot script plots DEBUG_LEVEL 6 output of jbuf.c. You have to -# increment the DEBUG_LEVEL in jbuf.c if you want to get the table for -# jbuf.dat. Then call baresip like this: -# -# ./baresip 2>&1 | grep -Eo "plot_stat.*" jbuf.log > jbuf.dat -# -# Call this script. Then compare the plot legend with the variables in jbuf.c! -# -# -# Description of the plot -# ======================= -# The plot is a time based diagram. The values avbuftime should lie between -# bufmin and bufmax. If it runs somewhere out of these boundaries (and stays -# outside for a while) a "Low" / "High" situation is detected. -# -# "Good" means: The number of packets in the jitter buffer is ok. -# -# "Low" means: The number is too low. Then the packets are incremented by -# holding one packet back in jbuf_get(). -# -# "High" means: The number is to high. Then packets are decremented by dropping -# one packet in jbuf_put(). This reduces the audio delay. -# -# The number of "Low"/"High" situations should be low while buffer under-runs -# should be avoided completely. - -# On the x-axes of the plot there is the time in milliseconds. See function -# jbuf_jitter_calc()! We note the variables in jbuf.c here in parentheses. -# E.g. (var jitter). -# -# - The orange line is the computed network jitter (var jitter). This is a -# moving average of the difference (var d) between the real time diff -# (var tr - var tr0) and the RTP timestamps diff (var ts - var ts0). -# See RFC-3550 RTP - A.8! -# We suggest a fast rise of the moving average and a slow shrink. Thus -# avoiding buffer under-runs have a higher priority than reducing the audio -# delay. -# -# - The buftime (var buftime) is the difference of the timestamps between the -# last RTP packet and the first RTP packet stored in the jbuf plus one packet -# time (var ptime) for the last packet. -# The buftime (light-grey) changes very fast during periods of jitter. To be -# applicable for detecting "Low" or "High" situations it has to be smoothed. -# The blue line avbuftime (var avbuftime) is a moving average of the buftime -# and is used to detect "Low"/"High". Thus the jbuf algorithm tries to keep -# the avbuftime between the following boundaries. -# -# - The green lines bufmin and bufmax (var bufmin, bufmax) are boundaries for -# avbuftime.They are computed by constant factors (> 1.) from the jitter. -# -# -# Copyright (C) 2020 commend.com - Christian Spielberger, Michael Peitler - - -# Choose your preferred gnuplot terminal or use e.g. evince to view the -# jbuf.eps! - -#set terminal x11 -set terminal postscript eps size 15,10 enhanced color -set output 'jbuf.eps' -#set terminal png size 1280,480 -#set output 'jbuf.png' -set datafile separator "," -set key outside -plot \ -'jbuf.dat' using 2:4 title 'jitter' with linespoints linecolor "orange", \ -'jbuf.dat' using 2:6 title 'avbuftime' with linespoints linecolor "skyblue", \ -'jbuf.dat' using 2:7 title 'bufmin' with linespoints linecolor "sea-green", \ -'jbuf.dat' using 2:8 title 'bufmax' with linespoints linecolor "sea-green", \ -'jbuf.dat' using 2:($9*10) title 'Good/Empty/Low/High' linecolor "red", \ -'jbuf.dat' using 2:5 title 'buftime' linecolor "light-grey", \ -10 title "Empty=10" linecolor "red", \ -20 title "Low=20" linecolor "red", \ -30 title "High=30" linecolor "red" -