Skip to content

Commit

Permalink
[AI-4506] DDS: Squid Integration (DataDog#18715)
Browse files Browse the repository at this point in the history
* Add Squid Integration

* fix pipeline sample and manifest json

* Add more sample logs for pipeline and update readme, dashboard

* Trigger Build

* update test samples

* Apply suggestions from code review

Co-authored-by: Sandra (neko) <[email protected]>

* updated Readme logformat as table and link in dashboard

* added dashboard image

* update greedyData to data

* updated monitors name and manifest.json

---------

Co-authored-by: kaustubhtandel-crest <[email protected]>
Co-authored-by: rushikeshshinde-crest <[email protected]>
Co-authored-by: Sandra (neko) <[email protected]>
Co-authored-by: rushikeshshinde_crest <[email protected]>
  • Loading branch information
5 people authored Nov 26, 2024
1 parent 74defa8 commit bcc1e4a
Show file tree
Hide file tree
Showing 12 changed files with 3,625 additions and 62 deletions.
5 changes: 5 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -309,6 +309,11 @@ datadog_checks_base/datadog_checks/base/checks/windows/ @DataDog/wi
/sophos_central_cloud/manifest.json @DataDog/saas-integrations @DataDog/documentation
/sophos_central_cloud/assets/logs/ @DataDog/saas-integrations @DataDog/documentation @DataDog/logs-backend @DataDog/logs-core

/squid/ @DataDog/saas-integrations
/squid/*.md @DataDog/saas-integrations @DataDog/documentation
/squid/manifest.json @DataDog/saas-integrations @DataDog/documentation
/squid/assets/logs/ @DataDog/saas-integrations @DataDog/documentation @DataDog/logs-backend

/ping_one/ @DataDog/saas-integrations
/ping_one/*.md @DataDog/saas-integrations @DataDog/documentation
/ping_one/manifest.json @DataDog/saas-integrations @DataDog/documentation
Expand Down
2 changes: 1 addition & 1 deletion squid/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -236,4 +236,4 @@
[1]: https://github.com/DataDog/integrations-core/pull/2788
[2]: https://github.com/DataDog/integrations-core/pull/1727
[3]: https://github.com/DataDog/integrations-core/pull/1643
[4]: https://github.com/mnussbaum
[4]: https://github.com/mnussbaum
44 changes: 44 additions & 0 deletions squid/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,17 @@
# Squid Integration

## Overview
[Squid][1] is an open-source caching and forwarding web proxy server that operates as an intermediary between clients and servers on a network. It acts as a gateway, enabling clients to access various internet resources such as websites, files, and other content from servers.

This integration provides enrichment and visualization for Squid logs. It helps you visualize detailed insights into Squid log analysis through the out-of-the-box dashboards and detection rules, enhancing detection and response capabilities.

Additionally, it includes pre-configured monitors for proactive notifications on the following:

1. High rate of server errors
2. CPU usage exceeded
3. High latency requests
4. High rate of client HTTP errors


This check monitors [Squid][1] metrics from the Cache Manager through the Datadog Agent.

Expand Down Expand Up @@ -87,6 +98,38 @@ Collecting logs is disabled by default in the Datadog Agent. To enable it, see [

## Data Collected

### Logs
The Squid integration collects access and cache logs.

#### Supported Access Log Formats
|Name | Format Specification|
|---------------------|------------------------------|
| squid |`%ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %[un %Sh/%<a %mt`|
| common |`%>a - %[un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st %Ss:%Sh`|
| combined |`%>a - %[un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh`|

For more information, refer to [Squid log formats][12].

**Note**: The default `logformat` type is `squid`. You can update the supported log format in `/etc/squid/squid.conf`, then restart Squid.

To use the `combined` type for `logformat`, add the following lines to your `/etc/squid/squid.conf` file:

```
logformat combined %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh
access_log /var/log/squid/access.log combined
```
Next, restart the `squid` service using the following command:

```shell
sudo systemctl restart squid
```

**Note**:

- The `Top Avg Request Duration by URL Host` panel will be loaded only if the default `squid` type of `logformat` is configured.
- The `Top Browsers` and `Top HTTP Referrer` panels will be loaded only if the `combined` type of `logformat` is configured.


### Metrics

See [metadata.csv][9] for a list of metrics provided by this check.
Expand Down Expand Up @@ -115,3 +158,4 @@ Need help? Contact [Datadog support][11].
[9]: https://github.com/DataDog/integrations-core/blob/master/squid/metadata.csv
[10]: https://github.com/DataDog/integrations-core/blob/master/squid/assets/service_checks.json
[11]: https://docs.datadoghq.com/help/
[12]: https://www.squid-cache.org/Doc/config/logformat/
Loading

0 comments on commit bcc1e4a

Please sign in to comment.