19

When I request the log files for an elastic beanstalk environment either through the web interface or "eb logs" I get the contents of the log files /var/log/eb-version-deployment.log, /opt/python/log/httpd.out, /var/log/cfn-hup.log, and several others.

Is there a way to add an additional log such as test_output.log to logs collected by the web interface and "eb logs"?

user12345678
  • 359
  • 1
  • 3
  • 9
  • If you write a log file to the `/opt/python/log` folder, it will be included automatically. That is also what the official [AWS EB Python Sample Application](https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/RelatedResources.html#RelatedResources-sampleapps) does. If you run into a `PermissionError`, see [this](https://stackoverflow.com/a/60549321) (possible) explanation. – djvg Mar 06 '20 at 08:07

2 Answers2

26

Elastic Beanstalk looks in this folder for configuration files regarding which logs to tail:

/opt/elasticbeanstalk/tasks/taillogs.d/

On my box there are a handful of files there

[ec2-user@ip taillogs.d]$ ls -al
total 32
drwxr-xr-x 2 root root 4096 Sep 27 05:49 .
drwxr-xr-x 6 root root 4096 Sep 27 05:49 ..
-rw-r--r-- 1 root root   58 Sep 27 05:49 eb-activity.conf
-rw-r--r-- 1 root root   35 Sep 27 05:49 eb-version-deployment.conf
-rw-r--r-- 1 root root   16 Oct 15 03:44 httpd.conf
-rw-r--r-- 1 root root   16 Oct 15 03:44 nginx.conf
-rw-r--r-- 1 root root   26 Oct 15 03:44 nodejs.conf
-rw-r--r-- 1 root root   29 Oct 15 03:44 npm.conf

And each one has one line in it, like this:

/var/log/nodejs/nodejs.log

or

/var/log/nginx/*

Use .ebextensions to add a config file to tail your own logs:

files:
  "/opt/elasticbeanstalk/tasks/taillogs.d/my-app-logs.conf":
    mode: "000755"
    owner: root
    group: root
    content: |
      /var/log/my-app.log

More info here:

http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.logging.html#health-logs-extend

Samuel Neff
  • 67,422
  • 16
  • 123
  • 169
  • Hi, can we retrieve a django log file from the docker container in elastic beanstalk using this.? – Vipul Vishnu av Sep 26 '16 at 12:21
  • 1
    I'm now able to see the resulting log using the command line `eb logs`, but not yet seeing it in the CloudWatch web UI. Any suggestions? Thanks! – Chris Prince Dec 02 '18 at 20:58
4

Yes! There is a way. You should NEVER have to ssh into your Elastic Beanstalk instances.

We found out by examining the eb-activity.log file.

Depending on your environment type (I use Python from Beanstalk), the locations you can put log files vary. You need to check eb-activity.log.

For example, with the Python environment type, in eb-activity.log you should see:

[2015-09-15T20:29:36.102Z] INFO  [2403]  - [Initialization/PreInitStage0/PreInitHook/01setuplogs.sh] : Completed activity. Result:
  + mkdir -p /opt/elasticbeanstalk/tasks/taillogs.d /opt/elasticbeanstalk/tasks/sytemtaillogs.d /opt/elasticbeanstalk/tasks/bundlelogs.d /opt/elasticbeanstalk/tasks/publishlogs.d
  + /opt/elasticbeanstalk/bin/log-conf -n python '-l/opt/python/log/*' -t taillogs,systemtaillogs,bundlelogs
  + /opt/elasticbeanstalk/bin/log-conf -n python-app '-l/opt/python/log/app.out.*' -t publishlogs
  + /opt/elasticbeanstalk/bin/log-conf -n supervisord '-l/opt/python/log/supervisord.log.*' -t publishlogs
  + /opt/elasticbeanstalk/bin/log-conf -nhttpd '-l/var/log/httpd/*' -f /opt/elasticbeanstalk/containerfiles/beanstalkhttpd

The above shows that Beanstalk configures logging for any of the following file:

  • /opt/python/log/*
  • /opt/python/log/app.out.*
  • /opt/python/log/supervisord.log.*
  • /var/log/httpd/*

That is, if you put a log file in /opt/python/log/, Beanstalk will find it and present it when you "Request Logs." I put my log files in /opt/python/log/ because it is accessible by WSGI group (which runs my application). This would be slightly different if you were using the Ruby environment type by the way. You can still check eb-activity.log for which directories are set up for logging.

Mike
  • 2,184
  • 2
  • 20
  • 32