How do I push Docker container logs to CloudWatch Logs through a proxy?

5 minute read
0

I want to publish my Docker container logs to Amazon CloudWatch through a proxy.

Short description

You can use a proxy when you push Docker container logs to CloudWatch. This step adds an additional layer of security between your Docker instances and internet-bound traffic.

Resolution

First, be sure to set up your basic Docker infrastructure for your specific use case.

In this example, the infrastructure includes:

  • An Amazon Elastic Compute Cloud (Amazon EC2) instance that's part of a private subnet that acts as a Docker container instance.
  • An Amazon EC2 instance that's part of a public subnet that acts as a proxy instance.
  • A Docker daemon installed on top of the Amazon EC2 instance in a private subnet. This daemon is configured to allow all container tasks to push their standard output logs to CloudWatch.
  • Amazon Linux 2 for the operating system (OS).

Also, make sure that your AWS Identity and Access Management (IAM) role has CloudWatchAgentAdminPolicy, CloudWatchAgentServerPolicy, or a similar policy that's associated with your private instance. The policy must allow you to create a log group, create a log stream, and make PutLogEvents calls.

Prepare your Docker instance

1.    Set up Linux environment variables to route all traffic from the Docker container instance into your proxy instance. In the following example, the IP address applies to the EC2 instance that's acting as a proxy instance:

$ export https_proxy=http://192.0.2.86:8888/
$ export http_proxy=http://192.0.2.86:8888/
$ export no_proxy=localhost,169.254.169.254

Note: These variables are the minimum requirements for the no_proxy configuration. Be sure to include any other destination IP addresses that you don't want clients to go to through the proxy.

2.    Install the Docker container into your Docker instance. See the following example:

$ sudo yum update -y
$ sudo yum install docker -y
$ sudo docker version

3.    Update your Docker instance to the latest version.

4.    Start the Docker daemon with the following command:

sudo systemctl start docker

5.    To allow Docker to run without sudo privileges, run the following command:

sudo groupadd docker
sudo usermod -aG docker $USER
newgrp docker
docker run hello-world

6.    To verify your Docker installation, review the output. Be sure that you see an installation verification message that's similar to the following:

"Hello From Docker!" 
"This message shows that your installation appears to be working correctly."

Add your proxy configuration to Docker

For configurations where the Docker engine is installed in Linux init (including Amazon Linux, RHEL 6, or CentOS 6 distributions):

1.    Open the configuration file /etc/sysconfig/docker using your preferred editor.

2.    Add your proxy configuration to the file. Use the private IP address of the public or proxy instance. See the following example:

# Setup proxy
export http_proxy="http://192.0.2.86:8888/"
export https_proxy="http://192.0.2.86:8888/"
export no_proxy="/var/run/docker.sock,localaddress,localhost,169.254.169.254"

Note: These are the minimum requirements for the no_proxy configuration. Be sure to include any other destination IP addresses that you don't want clients to go to through the proxy.

3.    Save your configuration file.

4.    Restart the Docker service:

$ sudo service docker restart

For configurations where the Docker engine is installed in the Linux systemd system management daemon (including Amazon Linux 2 and RHEL 7 distributions):

1.    Create a systemd directory for the Docker service. See the following example:

$ sudo mkdir -p /etc/systemd/system/docker.service.d

2.    Create an HTTP or HTTPS proxy file, depending on your configuration. See the following examples:

$ sudo vim /etc/systemd/system/docker.service.d/http-proxy.conf
[Service]
Environment="HTTP_PROXY=http://192.0.2.86:8888/" "NO_PROXY=localhost,127.0.0.1,169.254.169.254"
$ vim /etc/systemd/system/docker.service.d/https-proxy.conf
[Service]
Environment="HTTPS_PROXY=http://192.0.2.86:8888/" "NO_PROXY=localhost,127.0.0.1,169.254.169.254"

Note: These are the minimum requirements for the no_proxy configuration. Be sure to include any other destination IP addresses that you don't want clients to go to through the proxy.

3.    To flush your changes, use the following command:

$ sudo systemctl daemon-reload

4.    To restart Docker, use the following command:

$ sudo systemctl restart docker

5.    Verify your new configuration:

$ systemctl show --property=Environment docker

6.    Review the output:

Environment="HTTPS_PROXY=http://192.0.2.86:8888/" "NO_PROXY=localhost,127.0.0.1,169.254.169.254"

Test that your Docker logs are pushed to CloudWatch

1.    Run the nginx image in Docker.

2.    Use the Docker awslogs log driver to push the task's standard output logs to CloudWatch Logs. See the following example:

$ sudo docker run -d --name nginx --log-driver=awslogs --log-opt awslogs-region=eu-west-1 --log-opt awslogs-group=DockerLogGroupWithProxy --log-opt awslogs-create-group=true -p 8112:80 nginx

3.    Generate logs for your first Docker task using curl. See the following example:

curl localhost:8112

4.    View the data sent to your log group in the CloudWatch console.

5.    Verify that the log event for your Docker task is pushed to the CloudWatch log group. For example, look for a log event similar to the following:

198.51.100.100 - - [19/Sep/2018:10:13:38 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.53.1" "-"

Related information

Monitoring your container instances

Creating a container image for use on Amazon ECS

AWS OFFICIAL
AWS OFFICIALUpdated a year ago