Website daily update for 2024-11-25T04-31-33

content.en/avail/dht/turing/2024-47-report.md

@@ -0,0 +1,114 @@
+---
+title: Week 2024-47
+plotly: true
+slug: 2024-47
+weight: 1045612
+---
+
+# Avail Report Cal. Week 47 - 2024
+
+## General Information
+
+The following results show measurement data that were collected in calendar week 47 in 2024 from `2024-11-18` to `2024-11-25`.
+
+- **Number of crawls: `335`**
+
+  The number of crawls executed during the week indicated above.
+
+- **Number of visits: `3,229,721`**
+
+  Every attempt to dial or connect to a peer counts as a visit. Regardless of errors that may occur. We visit peers many times during a given week to extract information from them or probe for their uptime. We visit all peers that we discover in the DHT and peers that we found to be still online from the previous week.
+
+- **Number of unique peer IDs discovered in the DHT: `6,068`**
+
+  The total number of unique peers that our crawler learned about from other peers over the period of the given week and attempted to dial or connect to (visit). This is not the total number of Peer IDs that existed simultaneously in the network at any point during the week. It is the aggregate over the entire week.
+
+- **Number of unique IP addresses discovered: `4,443`**
+
+  The total number of unique IP addresses that the discovered peers advertised to the DHT network within the given week. IPv4 and IPv6 addresses are included in this number even if they address the same peer. This is not the total number of unique IP addresses that participated in the DHT simultaneously at any given point in time. It is the aggregate over the entire week.
+
+- **Network Size: `4,839.9611940298507463`**
+
+  The average number of peers that were found simultaneously (i.e., throughout the duration of one crawl) connected to the network. This number signifies the average network size from all crawls, during the given week.
+
+Timestamps are in UTC if not mentioned otherwise.
+
+## Churn Analysis
+
+This plot presents the Cumulative Distribution Function (CDF) of peer departure
+times from the network over a measurement period of one-week. The plot
+basically shows the percentage of nodes online at a point in time (`t=0`) that
+will have disconnected after 1 hour, 2 hours, 3 hours, .. up to 24 hours after `t=0`.
+
+- **X-Axis (Time in Hours)**: Represents the elapsed time since the reference
+point (`t=0`), measured in hours.
+- **Y-Axis (Percentage of Offline Peers)**: Indicates the cumulative
+percentage of peers that have disconnected from the network by the
+corresponding time interval. Specifically, it shows the proportion of peers
+that were online at `t=0` and have since gone offline.
+
+This visualization aids in understanding peer churn dynamics, which is crucial
+for optimizing network stability, resource allocation, and improving overall
+decentralized network performance.
+
+Note that the sum of the CDF is **NOT** 100%, as it only includes peers that were
+online for up to 24h.
+
+{{< plotly json="/plots/2024/11/18/avail-network-stability-cdf.json" height="800px" >}}
+
+### Geolocation
+
+Geographical data is sourced from the [MaxMind database](https://www.maxmind.com), which maps IP addresses to corresponding countries.
+
+This bar plot illustrates the distribution of observed nodes across different countries.
+
+{{< plotly json="/plots/2024/11/18/avail-geo-agent-All-bars.json" height="600px" >}}
+
+This plot displays the weekly geographical distribution of nodes, categorized by country.
+
+{{< plotly json="/plots/2024/11/18/avail-geo-agent-All-lines.json" height="600px" >}}
+
+### Cloud Providers
+
+Cloud providers data is sourced from [Udger](https://udger.com/resources/datacenter-list), which maps IP addresses to known hosting providers.
+
+#### Cloud Hosting Rate
+
+This line chart displays the count of nodes within the Avail network that are hosted on known commercial cloud providers, compared to those that are not. It tracks the distribution over a specified period, offering insights into the infrastructure preferences for node hosting.
+
+Regular analysis of this chart can reveal trends in the adoption of cloud services for nodes. Such information is crucial for understanding the network's resilience and the potential reliance on cloud infrastructure.
+
+{{< plotly json="/plots/2024/11/18/avail-cloud-rate-agent-All-lines.json" height="600px" >}}
+
+This bar chart presents the weekly distribution of Avail nodes among various cloud providers, including nodes not hosted within data centers (grouped under _Non-Datacenter_).
+
+{{< plotly json="/plots/2024/11/18/avail-cloud-agent-All-bars.json" height="600px" >}}
+
+The line chart illustrates the trends in the distribution of all Avail nodes across cloud providers over the given time period. Note that nodes hosted outside of data centers are not included in this representation.
+
+{{< plotly json="/plots/2024/11/18/avail-cloud-agent-All-lines.json" height="600px" >}}
+
+### Crawls
+
+#### Protocols
+
+This plot illustrates the evolving count of nodes supporting each of the listed protocols over time. It exclusively presents data gathered from nodes accessible through a libp2p connection via our crawler. The identification of supported protocols relies on the [libp2p identify protocol](https://github.com/libp2p/specs/tree/master/identify), hence necessitating a libp2p connection for discovery.
+
+{{< plotly json="/plots/2024/11/18/avail-crawl-protocols.json" height="1200px" >}}
+
+#### Errors
+
+{{< plotly json="/plots/2024/11/18/avail-connection-errors-single.json" height="600px" >}}
+
+{{< plotly json="/plots/2024/11/18/avail-crawl-errors-single.json" height="600px" >}}
+
+## Stale Peer Records
+
+This stacked plot depicts the count of peer records stored within each node's routing table and made accessible through the DHT.
+These peer records are used to discover new remote peers within the network, enabling efficient and secure routing towards the target peer or content
+
+Ensuring the reachability of referenced peers shared within the network holds paramount importance. The plot delineates the occurrences of reachable and non-reachable (stale) peer records. Note that nodes running behind a NAT are counted as unreachable even though they may be online.
+
+For instance, if a peer's record is present in the routing tables of 100 other nodes and the peer is reachable, the plot will reflect an increase of 100 in the _online_ category.
+
+{{< plotly json="/plots/2024/11/18/avail-stale-records-stacked.json" height="600px" >}}

content.en/plots/2024/11/18/avail-cloud-agent-All-bars.json

@@ -1 +1 @@
-{"data": [{"hovertemplate": "Cloud Provider: %{x}<br>Count: %{y:,}<br>Percentage: %{text}<br><extra></extra>", "text": ["33.6%", "23.2%", "15.7%", "10.2%", "5.8%", "1.6%", "0.9%", "0.7%", "0.7%", "0.7%", "0.7%", "0.7%", "5.4%"], "textposition": "outside", "x": ["Non-Datacenter", "Digital Ocean, Inc", "Hetzner Online", "Contabo GmbH", "OVH", "PlusServer GmbH", "inline.de", "HostStore, Inc.", "Interserver, Inc.", "Voxel", "Online SAS (Scaleway)", "Host Department LLC", "others"], "y": [139, 96, 65, 42, 24, 6, 3, 3, 2, 2, 2, 2, 22], "type": "bar"}], "layout": {"template": {"data": {"scatter": [{"type": "scatter"}]}}, "xaxis": {"title": {"text": "Cloud Provider"}, "tickangle": -20}, "yaxis": {"title": {"text": "Count"}, "rangemode": "tozero"}, "annotations": [{"font": {"size": 18}, "showarrow": false, "text": "Cloud Distribution of All Nodes", "x": 0.5, "xanchor": "center", "xref": "paper", "y": 1.15, "yanchor": "bottom", "yref": "paper"}, {"font": {"size": 14}, "showarrow": false, "text": "Total: 416", "x": 0.5, "xanchor": "center", "xref": "paper", "y": 1.1, "yanchor": "bottom", "yref": "paper"}, {"font": {"size": 10}, "showarrow": false, "text": "Data: week ending 24 Nov 2024. Source: Nebula", "x": 0.99, "xanchor": "right", "xref": "paper", "y": -0.14, "yanchor": "top", "yref": "paper"}]}}
+{"data": [{"hovertemplate": "Cloud Provider: %{x}<br>Count: %{y:,}<br>Percentage: %{text}<br><extra></extra>", "text": ["34.4%", "21.6%", "10.8%", "6.6%", "4.8%", "4.5%", "3.2%", "1.3%", "1.2%", "0.7%", "0.7%", "0.7%", "9.7%"], "textposition": "outside", "x": ["Non-Datacenter", "Digital Ocean, Inc", "Hetzner Online", "Comtrance Gmbh", "OVH", "Grier Forensics", "Contabo GmbH", "Devel.cz Lab s.r.o.", "PlusServer GmbH", "Njalla (1337 Services LLC)", "HostStore, Inc.", "Voxel", "others"], "y": [146, 91, 45, 27, 20, 19, 13, 5, 4, 3, 3, 2, 41], "type": "bar"}], "layout": {"template": {"data": {"scatter": [{"type": "scatter"}]}}, "xaxis": {"title": {"text": "Cloud Provider"}, "tickangle": -20}, "yaxis": {"title": {"text": "Count"}, "rangemode": "tozero"}, "annotations": [{"font": {"size": 18}, "showarrow": false, "text": "Cloud Distribution of All Nodes", "x": 0.5, "xanchor": "center", "xref": "paper", "y": 1.15, "yanchor": "bottom", "yref": "paper"}, {"font": {"size": 14}, "showarrow": false, "text": "Total: 425", "x": 0.5, "xanchor": "center", "xref": "paper", "y": 1.1, "yanchor": "bottom", "yref": "paper"}, {"font": {"size": 10}, "showarrow": false, "text": "Data: week ending 24 Nov 2024. Source: Nebula", "x": 0.99, "xanchor": "right", "xref": "paper", "y": -0.14, "yanchor": "top", "yref": "paper"}]}}

content.en/plots/2024/11/18/avail-cloud-agents-bars.json

@@ -1 +1 @@
-{"data": [{"hovertemplate": "Cloud Provider: %{x}<br>Count: %{y:,}<br>Percentage: %{text}<br><extra></extra>", "text": ["33.7%", "23.3%", "15.7%", "10.3%", "5.8%", "1.7%", "0.9%", "0.7%", "0.7%", "0.7%", "0.7%", "0.7%", "5.5%"], "textposition": "outside", "visible": true, "x": ["Non-Datacenter", "Digital Ocean, Inc", "Hetzner Online", "Contabo GmbH", "OVH", "PlusServer GmbH", "inline.de", "HostStore, Inc.", "Interserver, Inc.", "Voxel", "Online SAS (Scaleway)", "Host Department LLC", "others"], "y": [139, 96, 65, 42, 24, 6, 3, 3, 2, 2, 2, 2, 22], "type": "bar"}, {"hovertemplate": "Cloud Provider: %{x}<br>Count: %{y:,}<br>Percentage: %{text}<br><extra></extra>", "text": ["50.0%", "50.0%"], "textposition": "outside", "visible": false, "x": ["Contabo GmbH", "Non-Datacenter"], "y": [1, 1], "type": "bar"}], "layout": {"template": {"data": {"scatter": [{"type": "scatter"}]}}, "xaxis": {"title": {"text": "Cloud Provider"}, "tickangle": -20}, "yaxis": {"title": {"text": "Count"}, "autorange": true, "rangemode": "tozero"}, "updatemenus": [{"buttons": [{"args": [{"visible": [true, false]}, {"annotations": [{"x": 0.5, "y": 1.15, "xref": "paper", "yref": "paper", "text": "Avail Node Cloud Distribution", "showarrow": false, "font": {"size": 18}, "xanchor": "center", "yanchor": "bottom"}, {"x": 0.5, "y": 1.1, "xref": "paper", "yref": "paper", "text": "Total: 415", "showarrow": false, "font": {"size": 14}, "xanchor": "center", "yanchor": "bottom"}, {"x": 0.99, "y": -0.14, "xref": "paper", "yref": "paper", "text": "Data: week ending 24 Nov 2024. Source: Nebula", "showarrow": false, "font": {"size": 10}, "xanchor": "right", "yanchor": "top"}]}], "label": "Avail Node", "method": "update"}, {"args": [{"visible": [false, true]}, {"annotations": [{"x": 0.5, "y": 1.15, "xref": "paper", "yref": "paper", "text": "unknown Cloud Distribution", "showarrow": false, "font": {"size": 18}, "xanchor": "center", "yanchor": "bottom"}, {"x": 0.5, "y": 1.1, "xref": "paper", "yref": "paper", "text": "Total: 2", "showarrow": false, "font": {"size": 14}, "xanchor": "center", "yanchor": "bottom"}, {"x": 0.99, "y": -0.14, "xref": "paper", "yref": "paper", "text": "Data: week ending 24 Nov 2024. Source: Nebula", "showarrow": false, "font": {"size": 10}, "xanchor": "right", "yanchor": "top"}]}], "label": "unknown", "method": "update"}], "direction": "down", "pad": {"r": 10, "t": 10}, "showactive": true, "x": 0.01, "xanchor": "left", "y": 1.15, "yanchor": "top"}], "autosize": true, "annotations": [{"font": {"size": 18}, "showarrow": false, "text": "Avail Node Cloud Distribution", "x": 0.5, "xanchor": "center", "xref": "paper", "y": 1.15, "yanchor": "bottom", "yref": "paper"}, {"font": {"size": 14}, "showarrow": false, "text": "Total: 415", "x": 0.5, "xanchor": "center", "xref": "paper", "y": 1.1, "yanchor": "bottom", "yref": "paper"}, {"font": {"size": 10}, "showarrow": false, "text": "Data: week ending 24 Nov 2024. Source: Nebula", "x": 0.99, "xanchor": "right", "xref": "paper", "y": -0.14, "yanchor": "top", "yref": "paper"}]}}
+{"data": [{"hovertemplate": "Cloud Provider: %{x}<br>Count: %{y:,}<br>Percentage: %{text}<br><extra></extra>", "text": ["34.5%", "22.1%", "11.0%", "6.7%", "4.9%", "4.6%", "3.3%", "1.2%", "1.2%", "0.7%", "0.7%", "0.7%", "8.6%"], "textposition": "outside", "visible": true, "x": ["Non-Datacenter", "Digital Ocean, Inc", "Hetzner Online", "Comtrance Gmbh", "OVH", "Grier Forensics", "Contabo GmbH", "PlusServer GmbH", "Devel.cz Lab s.r.o.", "HostStore, Inc.", "Voxel", "Online SAS (Scaleway)", "others"], "y": [143, 91, 45, 27, 20, 19, 13, 4, 4, 3, 2, 2, 35], "type": "bar"}, {"hovertemplate": "Cloud Provider: %{x}<br>Count: %{y:,}<br>Percentage: %{text}<br><extra></extra>", "text": ["16.3%", "10.8%", "9.2%", "9.1%", "7.3%", "7.1%", "6.7%", "6.7%", "6.7%", "6.7%", "6.7%", "6.7%", "6.7%"], "textposition": "outside", "visible": false, "x": ["Non-Datacenter", "Censys, Inc.", "Utel Datacenter", "Codero", "Devel.cz Lab s.r.o.", "Comtrance Gmbh", "Amanah Tech Inc", "Canaca-com Inc.", "KnownSRV LTD", "Grier Forensics", "Oplink", "Root Level Technology", "others"], "y": [2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "type": "bar"}, {"hovertemplate": "Cloud Provider: %{x}<br>Count: %{y:,}<br>Percentage: %{text}<br><extra></extra>", "text": ["14.4%", "10.0%", "8.0%", "7.7%", "7.3%", "6.9%", "6.7%", "6.7%", "6.7%", "6.7%", "6.7%", "6.7%", "6.7%"], "textposition": "outside", "visible": false, "x": ["Censys, Inc.", "Utel Datacenter", "Non-Datacenter", "PROMETEUS", "Comtrance Gmbh", "Amanah Tech Inc", "Codero", "Devel.cz Lab s.r.o.", "Njalla (1337 Services LLC)", "KnownSRV LTD", "Grier Forensics", "PrivateSystems.net", "others"], "y": [2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "type": "bar"}, {"hovertemplate": "Cloud Provider: %{x}<br>Count: %{y:,}<br>Percentage: %{text}<br><extra></extra>", "text": ["10.6%", "10.4%", "8.7%", "7.7%", "7.7%", "7.7%", "7.7%", "7.7%", "7.7%", "7.7%", "7.7%", "7.7%", "7.7%"], "textposition": "outside", "visible": false, "x": ["Amanah Tech Inc", "Non-Datacenter", "Devel.cz Lab s.r.o.", "Censys, Inc.", "Comtrance Gmbh", "Codero", "Grier Forensics", "Global Data Service JSC", "KingHost Hospedagem de Sit", "KnownSRV LTD", "MUVHost", "Njalla (1337 Services LLC)", "others"], "y": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "type": "bar"}, {"hovertemplate": "Cloud Provider: %{x}<br>Count: %{y:,}<br>Percentage: %{text}<br><extra></extra>", "text": ["100.0%"], "textposition": "outside", "visible": false, "x": ["Non-Datacenter"], "y": [1], "type": "bar"}], "layout": {"template": {"data": {"scatter": [{"type": "scatter"}]}}, "xaxis": {"title": {"text": "Cloud Provider"}, "tickangle": -20}, "yaxis": {"title": {"text": "Count"}, "autorange": true, "rangemode": "tozero"}, "updatemenus": [{"buttons": [{"args": [{"visible": [true, false, false, false, false]}, {"annotations": [{"x": 0.5, "y": 1.15, "xref": "paper", "yref": "paper", "text": "Avail Node Cloud Distribution", "showarrow": false, "font": {"size": 18}, "xanchor": "center", "yanchor": "bottom"}, {"x": 0.5, "y": 1.1, "xref": "paper", "yref": "paper", "text": "Total: 415", "showarrow": false, "font": {"size": 14}, "xanchor": "center", "yanchor": "bottom"}, {"x": 0.99, "y": -0.14, "xref": "paper", "yref": "paper", "text": "Data: week ending 24 Nov 2024. Source: Nebula", "showarrow": false, "font": {"size": 10}, "xanchor": "right", "yanchor": "top"}]}], "label": "Avail Node", "method": "update"}, {"args": [{"visible": [false, true, false, false, false]}, {"annotations": [{"x": 0.5, "y": 1.15, "xref": "paper", "yref": "paper", "text": "kubo Cloud Distribution", "showarrow": false, "font": {"size": 18}, "xanchor": "center", "yanchor": "bottom"}, {"x": 0.5, "y": 1.1, "xref": "paper", "yref": "paper", "text": "Total: 15", "showarrow": false, "font": {"size": 14}, "xanchor": "center", "yanchor": "bottom"}, {"x": 0.99, "y": -0.14, "xref": "paper", "yref": "paper", "text": "Data: week ending 24 Nov 2024. Source: Nebula", "showarrow": false, "font": {"size": 10}, "xanchor": "right", "yanchor": "top"}]}], "label": "kubo", "method": "update"}, {"args": [{"visible": [false, false, true, false, false]}, {"annotations": [{"x": 0.5, "y": 1.15, "xref": "paper", "yref": "paper", "text": "unknown Cloud Distribution", "showarrow": false, "font": {"size": 18}, "xanchor": "center", "yanchor": "bottom"}, {"x": 0.5, "y": 1.1, "xref": "paper", "yref": "paper", "text": "Total: 15", "showarrow": false, "font": {"size": 14}, "xanchor": "center", "yanchor": "bottom"}, {"x": 0.99, "y": -0.14, "xref": "paper", "yref": "paper", "text": "Data: week ending 24 Nov 2024. Source: Nebula", "showarrow": false, "font": {"size": 10}, "xanchor": "right", "yanchor": "top"}]}], "label": "unknown", "method": "update"}, {"args": [{"visible": [false, false, false, true, false]}, {"annotations": [{"x": 0.5, "y": 1.15, "xref": "paper", "yref": "paper", "text": "go-ipfs Cloud Distribution", "showarrow": false, "font": {"size": 18}, "xanchor": "center", "yanchor": "bottom"}, {"x": 0.5, "y": 1.1, "xref": "paper", "yref": "paper", "text": "Total: 13", "showarrow": false, "font": {"size": 14}, "xanchor": "center", "yanchor": "bottom"}, {"x": 0.99, "y": -0.14, "xref": "paper", "yref": "paper", "text": "Data: week ending 24 Nov 2024. Source: Nebula", "showarrow": false, "font": {"size": 10}, "xanchor": "right", "yanchor": "top"}]}], "label": "go-ipfs", "method": "update"}, {"args": [{"visible": [false, false, false, false, true]}, {"annotations": [{"x": 0.5, "y": 1.15, "xref": "paper", "yref": "paper", "text": "rust-libp2p-server Cloud Distribution", "showarrow": false, "font": {"size": 18}, "xanchor": "center", "yanchor": "bottom"}, {"x": 0.5, "y": 1.1, "xref": "paper", "yref": "paper", "text": "Total: 1", "showarrow": false, "font": {"size": 14}, "xanchor": "center", "yanchor": "bottom"}, {"x": 0.99, "y": -0.14, "xref": "paper", "yref": "paper", "text": "Data: week ending 24 Nov 2024. Source: Nebula", "showarrow": false, "font": {"size": 10}, "xanchor": "right", "yanchor": "top"}]}], "label": "rust-libp2p-server", "method": "update"}], "direction": "down", "pad": {"r": 10, "t": 10}, "showactive": true, "x": 0.01, "xanchor": "left", "y": 1.15, "yanchor": "top"}], "autosize": true, "annotations": [{"font": {"size": 18}, "showarrow": false, "text": "Avail Node Cloud Distribution", "x": 0.5, "xanchor": "center", "xref": "paper", "y": 1.15, "yanchor": "bottom", "yref": "paper"}, {"font": {"size": 14}, "showarrow": false, "text": "Total: 415", "x": 0.5, "xanchor": "center", "xref": "paper", "y": 1.1, "yanchor": "bottom", "yref": "paper"}, {"font": {"size": 10}, "showarrow": false, "text": "Data: week ending 24 Nov 2024. Source: Nebula", "x": 0.99, "xanchor": "right", "xref": "paper", "y": -0.14, "yanchor": "top", "yref": "paper"}]}}