0

Yesterday I attempted to follow this answer from stackoverflow to add multiple users to the authorized_keys in our Elastic Beanstalk environment for QA.

https://stackoverflow.com/a/46269015/1827986

However after doing so the deployment failed with the following error

2018-10-16 19:04:22    INFO    Environment update is starting.
2018-10-16 19:05:05    INFO    Deploying new version to instance(s).
2018-10-16 19:06:09    ERROR   [Instance: i-05cc43b96ffc69145] Command 
failed on instance. Return code: 1 Output: (TRUNCATED)...erform: 
iam:GetGroup on resource: group BeanstalkAccess
declare -a users_array='()'
chmod: cannot access ‘/home/ec2-user/.ssh/authorized_keys’: No such 
file or directory
chown: cannot access ‘/home/ec2-user/.ssh/authorized_keys’: No such 
file or directory.
Hook /opt/elasticbeanstalk/hooks/appdeploy/post/980_beanstalk_ssh.sh 
failed. For more detail, check /var/log/eb-activity.log using console 
or EB CLI.
2018-10-16 19:06:09    INFO    Command execution completed on all 
instances. Summary: [Successful: 0, Failed: 1].
2018-10-16 19:06:10    ERROR   Unsuccessful command execution on 
instance id(s) 'i-05cc43b96ffc69145'. Aborting the operation.
2018-10-16 19:06:10    ERROR   Failed to deploy application.

I then tried to deploy a previous application version that was known to work and that failed. I attempted to rebuild the environment so that I could deploy a working application version. However, it continues to try to deploy the same stuck version that is giving the errors.

Also now after the rebuild I am getting a bunch of errors about the instance not being reachable by the ELB.

100.0 % of the requests are erroring with HTTP 4xx. Insufficient request 
rate (12.0 requests/min) to determine application health.
Command failed on all instances.
ELB health is failing or not available for all instances.

When I go into EC2 it shows the instance as running and it has green health.

I just tried to go to the Applications Versions and deleted the version that would not deploy hoping that it would restore from the working version. However, now it is just giving the following error when I try to deploy the working version.

Environment health has transitioned from Degraded to Severe. Command 
failed on all instances. Incorrect application version found on all 
instances. Expected version "app-v1_5_13-719-gc533-181016_092351" 
(deployment 291). Application update failed 2 minutes ago and took 2 
minutes. ELB health is failing or not available for all instances.

What else can I do to get this back to a working state?

Joseph Crawford
  • 1,320
  • 1
  • 10
  • 29

1 Answers1

0

This turned out to be a memory issue since we were running on the micro servers, I am not sure why we hit the limits all of the sudden, however, upgrading to a small instance solved the issues.

Joseph Crawford
  • 1,320
  • 1
  • 10
  • 29