a simple dockerimage for a proxy that logs http-bodys (both requests and responses). Based on nginx/alpine.
source-code: https://github.com/bjuergens/http_log_proxy
docker-image: https://hub.docker.com/r/bjuergens/http_log_proxy
inspired by
- https://serverfault.com/questions/361556/is-it-possible-to-log-the-response-data-in-nginx-access-log
- https://gist.github.com/morhekil/1ff0e902ed4de2adcb7a
Simply create a new service from this image, move the port from the target container to this one, and pass the name of the target container as $TARGET_HOST to this new service.
if your initial compose file looks like this:
version: "3.8"
services:
example_web:
image: nginx
ports:
- 80:80
then your new dockerfile will look like this
version: "3.8"
services:
example_web:
image: nginx
http_log:
environment:
TARGET_HOST: example_web
image: bjuergens/http_log_proxy
ports:
- 80:80
and you will start to see log-messages for your requests in your docker compose logs, like this
http_log_1 | 172.18.0.1-[14/Aug/2022:06:58:05 +0000]GET / HTTP/1.1200615curl/7.81.00.001<([host localhost]\n[user-agent curl/7.81.0]\n[accept */*]\n)>(content-type=text/html content-length=615 accept-ranges=bytes last-modified=Tue, 19 Jul 2022 14:05:27 GMT connection=keep-alive etag=\"62d6ba27-267\" )<!DOCTYPE html>\n<html>\n<head>\n<title>Welcome to nginx!</title>....
for a little more complex example using optional env-vars, see the composefile in this repo
lets assume, there are other containers using "example_web", too. Then the container-name must be changed as well, as well as the internal port of the proxy. And while we are at it, lets make the logformat more complex. (the syntax there is a little bit annoying, because it must satisfy nginx-logformat, nginx-syntax, yaml-syntax, and env-syntax.)
version: "3.8"
services:
example_web:
container-name: "example_web_orig"
image: nginx
http_log:
container-name: "example_web"
image: bjuergens/http_log_proxy
environment:
TARGET_HOST: example_web_orig
TARGET_PORT: 8080
OWN_PORT: 8080
LOG_FORMAT: >
'$$request STATUS: $$status REMOTE: $$remote_addr \n'
'>>> HEAD: $$requ_headers \n'
'>>> BODY: $$request_body \n'
'<<< HEAD: $$resp_headers_short \n'
'<<< BODY: $$resp_body \n'
ports:
- 80:8080
the container uses some env-variables which allow runtime customization. A full list can be found in the Dockerfile. Some notable env variables are:
TARGET_HOST(no default, required): where should all requests be redirected to?TARGET_PORT(default80)LOG_MAX_BODY_LENGTH(default1000): limits the how many symbols of the body can be logged per requestLOG_FORMAT_ESCAPE(defaultjson): decides what kind of escaping the values in each log-message should go throughLOG_FORMAT: format string for log-messages.EXTRA_DIRECTIVES_ROOT: additional directives for the root-block ofnginx.confEXTRA_DIRECTIVES_HTTP: additional directives for the http-block of nginx.confEXTRA_DIRECTIVES_SERVER: additional directives for for the defaultserverblockEXTRA_DIRECTIVES_LOCATION: additional directives for the defaultlocationblockDEFAULT_DIRECTIVES_PROXY: default directives for the location block, which should work for most reverse-proxy-situations
The variable LOG_FORMAT uses nginx's own log_format directrive and thus all usual nginx-variables are available. This docker image supplies additional variables which can (and should!) be used for logging:
$requ_headersall request-headers$request_bodyfull request body$resp_bodyfull response body, truncated to$LOG_MAX_BODY_LENGTH$resp_headers_fullall response-headers (exceptDateandServer)$resp_headers_shortresponse-headers without fast-path headers, which usually add clutter and a boring (for details see this comment)
if env-vars are not enough, you can always mount new configfiles as volumes. Note that known configs (nginx.conf and proxy.conf) are created from template-files, while any new configs you may create will be used directly.
this image aimes to be a simple and uncomplicated way to get a quick glance into the http-requests for a single services. The simplicity comes at the cost of generality. If your situation requires a more sophisticated and heavier solution, consider some of the alternatives, e.g
on the other hand if you can think of a useful feature, that doesn't add too much complexity, feel free to let me know and open an issue.
the response-headers Date and Server do not show up in the respective variable and thus can not be logged. This limitation comes from nginx. If you find a workaround/solution to this limitation, please let me know by opening an issue.
the logs will probably contain many sensible information and even GDPR violations, so I recommend using this image for debugging purposes only. If you find a good use case for this image beside debugging, feel free to let me know.
here is some stuff I am sure to forget until the next time I do maintenance on this repo
relevant dev-reading
- https://github.com/docker-library/docs/tree/master/nginx#using-environment-variables-in-nginx-configuration-new-in-119
- https://www.hardill.me.uk/wordpress/2018/03/14/logging-requests-and-response-with-nginx/
docker login -u bjuergens
docker build -t bjuergens/http_log_proxy --pull .
# rudimentary test
docker run -it -e TARGET_HOST=www.google.com -p 80:80 bjuergens/http_log_proxy
docker push bjuergens/http_log_proxy
optional: update readme in dockerhub (leave out dev-section)