forked from OpenTSDB/opentsdb.net
-
Notifications
You must be signed in to change notification settings - Fork 1
/
overview.html
149 lines (141 loc) · 8.01 KB
/
overview.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>OpenTSDB - A Distributed, Scalable Monitoring System</title>
<!-- Bootstrap core CSS -->
<link href="css/bootstrap.min.css" rel="stylesheet">
<link href="css/otsdb-bootstrap.css" rel="stylesheet">
<!-- Custom styles for this template -->
<link href="css/custom.css" rel="stylesheet">
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-18339382-1']);
_gaq.push(['_setDomainName', 'none']);
_gaq.push(['_setAllowLinker', true]);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
</head>
<body>
<a href="https://github.com/OpenTSDB/opentsdb">
<img style="position: absolute; top: 0; right: 0; border: 0;" src="https://camo.githubusercontent.com/38ef81f8aca64bb9a64448d0d70f1308ef5341ab/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f6769746875622f726962626f6e732f666f726b6d655f72696768745f6461726b626c75655f3132313632312e706e67" alt="Fork me on GitHub" data-canonical-src="https://s3.amazonaws.com/github/ribbons/forkme_right_darkblue_121621.png">
</a>
<div class="container">
<h1><a href="http://www.opentsdb.net/"><img src="img/logo_header.png"></a></h1>
<div class="navbar navbar-default">
<div class="container-fluid">
<div class="navbar-header">
<ul class="nav navbar-nav">
<li><a href="index.html">Home</a></li>
<li><a href="overview.html">Overview</a></li>
<li><a href="https://github.com/OpenTSDB/opentsdb/releases">Download</a></li>
<li><a href="https://github.com/OpenTSDB/opentsdb">Source</a></li>
<li><a href="docs/build/html/index.html">Documentation</a></li>
<li><a href="faq.html">FAQ</a></li>
</ul>
</div>
</div>
</div>
<h2>How does OpenTSDB work?</h2>
OpenTSDB consists of a <strong>Time Series Daemon
(TSD)</strong> as well as set of command line utilities. Interaction with OpenTSDB is
primarily achieved by running one or more of the TSDs. Each TSD is independent.
There is no master, no shared state so you can run as many TSDs as required to handle any
load you throw at it. Each TSD uses the open source database
<a href="http://hbase.org">HBase</a> or hosted Google Bigtable service to store and
retrieve time-series data.
The data schema is highly optimized for fast aggregations of similar time series to
minimize storage space. Users of the TSD never need to access the underlying store directly.
You can communicate with the TSD via a simple telnet-style
protocol, an HTTP API or a simple built-in GUI. All communications happen on the
same port (the TSD figures out the protocol of the client by looking at the first
few bytes it receives).
<p>
<img src="img/tsdb-architecture.png" alt="OpenTSDB architecture"/>
<p>
<h2>Writing</h2>
The first step in using OpenTSDB is to send time series data to the TSDs. A number of
<a href="docs/build/html/resources.html#clients">tools</a> exist to pull data from various
sources into OpenTSDB. If you can't find a tool for your needs, you may need to write
scripts that collect data from your systems (e.g.
by reading interesting metrics from <code>/proc</code> on Linux, collecting
counters from your network gear via SNMP, or other interesting data from your
applications, via JMX for instance for Java applications) and push data points
to one of the TSDs periodically.
<p>
<a href="http://www.stumbleupon.com">StumbleUpon</a> wrote a Python framework
called
<a href="https://github.com/OpenTSDB/tcollector"><strong><code>tcollector</code></strong></a>
that is used to collect
thousand of metrics from Linux 2.6, Apache's HTTPd, MySQL, HBase, memcached,
Varnish and more. This low-impact framework includes a number useful collectors
and the community is constantly providing more. Alternative frameworks with OpenTSDB
support include Collectd, Statsd and the Coda Hale metrics emitter.</a>.
<p>
In OpenTSDB, a time series data point consists of:
<ul>
<li>A metric name.</li>
<li>A UNIX timestamp (seconds or millisecinds since
<a href="http://en.wikipedia.org/wiki/Unix_epoch">Epoch</a>).</li>
<li>A value (64 bit integer or single-precision floating point value), a JSON
formatted event or a histogram/digest.</li>
<li>A set of tags (key-value pairs) that describe the time series the point belongs to.</li>
</ul>
Tags allow you to separate similar data points from different sources or
related entities, so you can easily graph them individually or in groups.
One common use case for tags consists in annotating data points with the
name of the machine that produced it as well as name of the cluster or
pool the machine belongs to. This allows you to easily make dashboards
that show the state of your service on a per-server basis as well as
dashboards that show an aggregated state across logical pools of servers.
<div class="panel panel-default">
<div class="panel-body">
mysql.bytes_received 1287333217 327810227706 schema=foo host=db1<br>
mysql.bytes_sent 1287333217 6604859181710 schema=foo host=db1<br>
mysql.bytes_received 1287333232 327812421706 schema=foo host=db1<br>
mysql.bytes_sent 1287333232 6604901075387 schema=foo host=db1<br>
mysql.bytes_received 1287333321 340899533915 schema=foo host=db2<br>
mysql.bytes_sent 1287333321 5506469130707 schema=foo host=db2
</div>
</div>
This examples contains 6 data points that belong to 4 different time series.
Each different combination of metric and tags makes up a different time
series. All of the 4 time series are for one of two metrics
<code>mysql.bytes_received</code> or <code>mysql.bytes_sent</code>.
A data point must have at least one tag and every time series for a metric
should have the same number of tags. It is not recommended to have
more than 6-7 tags per data point, as the cost associated with storing
new data points quickly becomes dominated by the number of tags beyond
that point.
<p>
With the tags in the example above, it will be easy to create graphs and
dashboards that show the network activity of MySQL on a per host and/or per
schema basis. New to OpenTSDB 2.0 is the ability to store non-numeric annotations
along with data points for tracking meta-data, quality metrics or other types
of information.
<h2>Reading</h2>
Time series data is usually consumed in the format of a line graph. Thus OpenTSDB
offers a built-in, simple user interface for selecting one or more metrics and tags
to generate a graph as an image. Alternatively an HTTP API is available to tie
OpenTSDB into external systems such as monitoring frameworks, dashboards, statistics
packages or automation tools.
<p>
Take a look at the <a href="docs/build/html/resources.html#clients">resources</a>
page for tools contributed by the community for working with OpenTSDB. <hr>
<div class="footer">
<p><a href="https://groups.google.com/forum/#!forum/opentsdb"><i class="glyphicon glyphicon-envelope"></i> Mailing List</a> | <i class="glyphicon glyphicon-user"></i> IRC: Freenode #opentsdb | © 2010 - <script type="text/JavaScript"> document.write(new Date().getFullYear()); </script> The OpenTSDB Authors | Travis CI Status: <img src="https://travis-ci.org/OpenTSDB/opentsdb.svg?branch=master">
</div>
</div> <!-- /container -->
<!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js"></script>
<!-- Include all compiled plugins (below), or include individual files as needed -->
<script src="js/bootstrap.min.js"></script>
</body>
</html>