Need help?

Our experts have had an average response time of 11.7 minutes in August 2021 to fix urgent issues.

We will keep your servers stable, secure, and fast at all times for one fixed price.

Push Docker Container Logs to CloudWatch Logs via Proxy

by | Aug 24, 2021

We can push Docker Container Logs to CloudWatch Logs via Proxy

This adds an additional layer of security between the Docker instances and internet-bound traffic.

Here, at Bobcares, we assist our customers with several AWS queries as part of our AWS Support Services.

Today, let us see how to publish the Docker container logs to Amazon CloudWatch through a proxy.

 

Push Docker Container Logs to CloudWatch Logs via Proxy

In order to begin, our Support Techs recommend setting up basic Docker infrastructure as necessary.

For example, here we include the following:

  1. An Amazon EC2 instance that’s part of a private subnet acting as a Docker container instance.
  2. An Amazon EC2 instance that’s part of a public subnet acting as a proxy instance.
  3. A Docker daemon on top of the Amazon EC2 instance in a private subnet.
  4. Amazon Linux 2 for the OS.

 

Prepare the Docker instance

1. Initially, to route all traffic from the Docker container instance into the proxy instance, we set up Linux environment variables.

For example:

$ export https_proxy=http://xxx.x.x.xx:xxxx/
$ export http_proxy=http://xxx.x.x.xx:xxxx/
$ export no_proxy=localhost,xxx.xxx.xxx.xxx

Here, we can include any other destination IP addresses that we don’t want clients to go to through the proxy.

2. Then we install the Docker container into the Docker instance.

For example:

$ sudo yum update -y
$ sudo yum install docker -y
$ sudo docker –version

3. After that, we update the Docker instance to the latest version.

4. Eventually, we verify the Docker installation:

$ docker run hello-world

5. Finally, we review the output. Make sure we receive an installation verification message similar to the following:

"Hello From Docker!"
"This message shows that your installation appears to be working correctly."

 

Add proxy configuration to Docker

For configurations where the Docker engine is installed in Linux init:

1. First, we open the configuration file /etc/sysconfig/docker using an editor.

2. Then we add the proxy configuration to the file.

For example:

# Setup proxy
export http_proxy="http://xxx.x.x.xx:8888/"
export https_proxy="http://xxx.x.x.xx:8888/"
export no_proxy="/var/run/docker.sock,localaddress,localhost,xxx.xxx.xxx.xxx"

3. Eventually, we save the configuration file.

4. We then restart the Docker service:

$ sudo service docker restart

For configurations where the Docker engine is installed in the Linux systemd:

1. To begin, we create a systemd directory for the Docker service.

For example:

$ sudo mkdir -p /etc/systemd/system/docker.service.d

2. Depending on the configuration we create an HTTP or HTTPS proxy file.

For example:

$ vim /etc/systemd/system/docker.service.d/http-proxy.conf
[Service]
Environment="HTTP_PROXY=http://xxx.x.x.xx:8888/" "NO_PROXY=localhost,xxx.x.x.x,xxx.xxx.xxx.xxx"
$ vim /etc/systemd/system/docker.service.d/https-proxy.conf
[Service]
Environment="HTTPS_PROXY=http://xxx.x.x.xx:8888/" "NO_PROXY=localhost,xxx.x.x.x,xxx.xxx.xxx.xxx"

3. Then we flush the changes:

$ sudo systemctl daemon-reload

4. Later we need to restart Docker:

$ sudo systemctl restart docker

5. Verify the new configuration:

$ systemctl show --property=Environment docker

6. Finally, we review the output:

Environment="HTTPS_PROXY=http://xxx.x.x.xx:8888/" "NO_PROXY=localhost,xxx.x.x.x,xxx.xxx.xxx.xxx"

 

Test that the Docker logs are pushed to CloudWatch

1. To do so, we run the Nginx image in Docker.

2. Then to push the task’s standard output logs to CloudWatch Logs we use the Docker awslogs log driver.

For example:

$ sudo docker run -d --name nginx --log-driver=awslogs --log-opt awslogs-region=eu-west-1 --log-opt awslogs-group=DockerLogGroupWithProxy --log-opt awslogs-create-group=true -p 8112:80 nginx

3. After that we generate logs for the first Docker task using curl.

curl localhost:8112

4. We view the data sent to the log group in the CloudWatch console.

5. Finally, we verify that the log event for the Docker task is pushed to the CloudWatch log group.

For example, we can look for a log event similar to:

198.51.100.100 - - [19/Sep/2018:10:13:38 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.53.1" "-"

[Need help with the procedures? We’d be happy to assist you]

 

Conclusion

In short, we saw how our Support Techs push Docker Container Logs to CloudWatch Logs via Proxy.

PREVENT YOUR SERVER FROM CRASHING!

Never again lose customers to poor server speed! Let us help you.

Our server experts will monitor & maintain your server 24/7 so that it remains lightning fast and secure.

GET STARTED

var google_conversion_label = "owonCMyG5nEQ0aD71QM";

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

Categories

Tags

Privacy Preference Center

Necessary

Necessary cookies help make a website usable by enabling basic functions like page navigation and access to secure areas of the website. The website cannot function properly without these cookies.

PHPSESSID - Preserves user session state across page requests.

gdpr[consent_types] - Used to store user consents.

gdpr[allowed_cookies] - Used to store user allowed cookies.

PHPSESSID, gdpr[consent_types], gdpr[allowed_cookies]
PHPSESSID
WHMCSpKDlPzh2chML

Statistics

Statistic cookies help website owners to understand how visitors interact with websites by collecting and reporting information anonymously.

_ga - Preserves user session state across page requests.

_gat - Used by Google Analytics to throttle request rate

_gid - Registers a unique ID that is used to generate statistical data on how you use the website.

smartlookCookie - Used to collect user device and location information of the site visitors to improve the websites User Experience.

_ga, _gat, _gid
_ga, _gat, _gid
smartlookCookie

Marketing

Marketing cookies are used to track visitors across websites. The intention is to display ads that are relevant and engaging for the individual user and thereby more valuable for publishers and third party advertisers.

IDE - Used by Google DoubleClick to register and report the website user's actions after viewing or clicking one of the advertiser's ads with the purpose of measuring the efficacy of an ad and to present targeted ads to the user.

test_cookie - Used to check if the user's browser supports cookies.

1P_JAR - Google cookie. These cookies are used to collect website statistics and track conversion rates.

NID - Registers a unique ID that identifies a returning user's device. The ID is used for serving ads that are most relevant to the user.

DV - Google ad personalisation

IDE, test_cookie, 1P_JAR, NID, DV, NID
IDE, test_cookie
1P_JAR, NID, DV
NID
hblid

Security

These are essential site cookies, used by the google reCAPTCHA. These cookies use an unique identifier to verify if a visitor is human or a bot.

SID, APISID, HSID, NID, PREF
SID, APISID, HSID, NID, PREF