How to iterate through nested Dictionary in python boto3

Question

Hi want to find only stopped instance and their ids below is the below-nested JSON



{
 "Reservations": [
  {
   "Groups": [],
   "Instances": [
    {
     "AmiLaunchIndex": 0,
     "Architecture": "x86_64",
     "BlockDeviceMappings": [
      {
       "DeviceName": "/dev/xvda",
       "Ebs": {
        "AttachTime": "2021-03-11 09:43:59+00:00",
        "DeleteOnTermination": true,
        "Status": "attached",
        "VolumeId": "vol-0c49ff5c979904337"
       }
      }
     ],
     "CapacityReservationSpecification": {
      "CapacityReservationPreference": "open"
     },
     "ClientToken": "",
     "CpuOptions": {
      "CoreCount": 1,
      "ThreadsPerCore": 1
     },
     "EbsOptimized": false,
     "EnaSupport": true,
     "EnclaveOptions": {
      "Enabled": false
     },
     "HibernationOptions": {
      "Configured": false
     },
     "Hypervisor": "xen",
     "ImageId": "ami-038f1ca1bd58a5790",
     "InstanceId": "i-041fb789f1554b7d5",
     "InstanceType": "t2.micro",
     "KeyName": "HadoopLearning",
     "LaunchTime": "2021-03-11 09:43:58+00:00",
     "MetadataOptions": {
      "HttpEndpoint": "enabled",
      "HttpPutResponseHopLimit": 1,
      "HttpTokens": "optional",
      "State": "applied"
     },
     "Monitoring": {
      "State": "disabled"
     },
     "NetworkInterfaces": [
      {
       "Attachment": {
        "AttachTime": "2021-03-11 09:43:58+00:00",
        "AttachmentId": "eni-attach-022ac34489909cc78",
        "DeleteOnTermination": true,
        "DeviceIndex": 0,
        "NetworkCardIndex": 0,
        "Status": "attached"
       },
       "Description": "",
       "Groups": [
        {
         "GroupId": "sg-5878120b",
         "GroupName": "default"
        }
       ],
       "InterfaceType": "interface",
       "Ipv6Addresses": [],
       "MacAddress": "0a:15:95:72:23:6b",
       "NetworkInterfaceId": "eni-0d356de72ae7e6bfc",
       "OwnerId": "216808211954",
       "PrivateDnsName": "ip-172-31-29-51.ec2.internal",
       "PrivateIpAddress": "172.31.29.51",
       "PrivateIpAddresses": [
        {
         "Primary": true,
         "PrivateDnsName": "ip-172-31-29-51.ec2.internal",
         "PrivateIpAddress": "172.31.29.51"
        }
       ],
       "SourceDestCheck": true,
       "Status": "in-use",
       "SubnetId": "subnet-e705ccaa",
       "VpcId": "vpc-4401543e"
      }
     ],
     "Placement": {
      "AvailabilityZone": "us-east-1d",
      "GroupName": "",
      "Tenancy": "default"
     },
     "PrivateDnsName": "ip-172-31-29-51.ec2.internal",
     "PrivateIpAddress": "172.31.29.51",
     "ProductCodes": [],
     "PublicDnsName": "",
     "RootDeviceName": "/dev/xvda",
     "RootDeviceType": "ebs",
     "SecurityGroups": [
      {
       "GroupId": "sg-5878120b",
       "GroupName": "default"
      }
     ],
     "SourceDestCheck": true,
     "State": {
      "Code": 80,
      "Name": "stopped"  ####this one only which is stopped ###
     },
     "StateReason": {
      "Code": "Client.UserInitiatedShutdown",
      "Message": "Client.UserInitiatedShutdown: User initiated shutdown"
     },
     "StateTransitionReason": "User initiated (2021-03-11 10:13:00 GMT)",
     "SubnetId": "subnet-e705ccaa",
     "Tags": [
      {
       "Key": "Name",
       "Value": "aviral"
      }
     ],
     "VirtualizationType": "hvm",
     "VpcId": "vpc-4401543e"
    },
    {
     "AmiLaunchIndex": 1,
     "Architecture": "x86_64",
     "BlockDeviceMappings": [
      {
       "DeviceName": "/dev/xvda",
       "Ebs": {
        "AttachTime": "2021-03-11 09:43:59+00:00",
        "DeleteOnTermination": true,
        "Status": "attached",
        "VolumeId": "vol-069a9ca7f860fefd8"
       }
      }
     ],
     "CapacityReservationSpecification": {
      "CapacityReservationPreference": "open"
     },
     "ClientToken": "",
     "CpuOptions": {
      "CoreCount": 1,
      "ThreadsPerCore": 1
     },
     "EbsOptimized": false,
     "EnaSupport": true,
     "EnclaveOptions": {
      "Enabled": false
     },
     "HibernationOptions": {
      "Configured": false
     },
     "Hypervisor": "xen",
     "ImageId": "ami-038f1ca1bd58a5790",
     "InstanceId": "i-0d0c876682eef71ae",
     "InstanceType": "t2.micro",
     "KeyName": "HadoopLearning",
     "LaunchTime": "2021-03-11 09:43:58+00:00",
     "MetadataOptions": {
      "HttpEndpoint": "enabled",
      "HttpPutResponseHopLimit": 1,
      "HttpTokens": "optional",
      "State": "applied"
     },
     "Monitoring": {
      "State": "disabled"
     },
     "NetworkInterfaces": [
      {
       "Attachment": {
        "AttachTime": "2021-03-11 09:43:58+00:00",
        "AttachmentId": "eni-attach-0b0eba3fb5b3c3205",
        "DeleteOnTermination": true,
        "DeviceIndex": 0,
        "NetworkCardIndex": 0,
        "Status": "attached"
       },
       "Description": "",
       "Groups": [
        {
         "GroupId": "sg-5878120b",
         "GroupName": "default"
        }
       ],
       "InterfaceType": "interface",
       "Ipv6Addresses": [],
       "MacAddress": "0a:3d:3e:bc:96:e3",
       "NetworkInterfaceId": "eni-0688951c8bd226676",
       "OwnerId": "216808211954",
       "PrivateDnsName": "ip-172-31-19-102.ec2.internal",
       "PrivateIpAddress": "172.31.19.102",
       "PrivateIpAddresses": [
        {
         "Primary": true,
         "PrivateDnsName": "ip-172-31-19-102.ec2.internal",
         "PrivateIpAddress": "172.31.19.102"
        }
       ],
       "SourceDestCheck": true,
       "Status": "in-use",
       "SubnetId": "subnet-e705ccaa",
       "VpcId": "vpc-4401543e"
      }
     ],
     "Placement": {
      "AvailabilityZone": "us-east-1d",
      "GroupName": "",
      "Tenancy": "default"
     },
     "PrivateDnsName": "ip-172-31-19-102.ec2.internal",
     "PrivateIpAddress": "172.31.19.102",
     "ProductCodes": [],
     "PublicDnsName": "",
     "RootDeviceName": "/dev/xvda",
     "RootDeviceType": "ebs",
     "SecurityGroups": [
      {
       "GroupId": "sg-5878120b",
       "GroupName": "default"
      }
     ],
     "SourceDestCheck": true,
     "State": {
      "Code": 80,
      "Name": "running" ###Not this one #####
     },
     "StateReason": {
      "Code": "Client.UserInitiatedShutdown",
      "Message": "Client.UserInitiatedShutdown: User initiated shutdown"
     },
     "StateTransitionReason": "User initiated (2021-03-11 10:13:00 GMT)",
     "SubnetId": "subnet-e705ccaa",
     "Tags": [
      {
       "Key": "Name",
       "Value": "avinash"
      }
     ],
     "VirtualizationType": "hvm",
     "VpcId": "vpc-4401543e"
    }
   ],
   "OwnerId": "216808211954",
   "ReservationId": "r-0d646190512fec9f8"
  }
 ],
 "ResponseMetadata": {
  "HTTPHeaders": {
   "cache-control": "no-cache, no-store",
   "content-type": "text/xml;charset=UTF-8",
   "date": "Thu, 11 Mar 2021 10:14:39 GMT",
   "server": "AmazonEC2",
   "strict-transport-security": "max-age=31536000; includeSubDomains",
   "transfer-encoding": "chunked",
   "vary": "accept-encoding",
   "x-amzn-requestid": "58065d1d-76fc-4f7a-9229-18584ee1031e"
  },
  "HTTPStatusCode": 200,
  "RequestId": "58065d1d-76fc-4f7a-9229-18584ee1031e",
  "RetryAttempts": 0
 }
}

I only want an instance of those who are stopped and then I want to start them using their instance ID using boto3

I am using the below code

import boto3
import json

access_key = "XXXXXXXXXX"
secret_key = "XXXXXXXXXXXXXXXXXXXXX"
client = boto3.client('ec2', aws_access_key_id=access_key, aws_secret_access_key=secret_key, region_name='us-east-1')

ec2_result = client.describe_instances(
    Filters=[
        {
            'Name': 'tag:Name',
            'Values': ['?avi*']
        }
    ]
)

#ids= [ec2_result['Reservations'][0]['Instances'][i]['InstanceId'] for i in range(len(ec2_result['Reservations'][0]['Instances']))]
#print(ids)
print(ec2_result)

using the above code I am getting first block JSON but I only want an instance which is stopped, as I have 4000 instances in my AWS

see https://stackoverflow.com/questions/59427234/is-there-a-way-to-list-stopped-ec2-instances-using-boto3 — balderman, Mar 11 '21 at 10:44
You should generate new access keys and deactive/delete already burned ones. — Marcin, Mar 11 '21 at 10:48
ok @Marcin thanks very much,I am only waiting for you here, please provide input, the requirement is I want all instance which is stopped which name is starting from avi and they are stopped then I want to start them — Rahul, Mar 11 '21 at 10:50
And also guide me on how can I become awesome in python like you, Since 15 days I started learning that — Rahul, Mar 11 '21 at 10:56

score 1 · Accepted Answer · answered Mar 11 '21 at 11:04

1

I think the following should work:


instances_stopped = []

for reservation in ec2_result['Reservations']:
  for instance in reservation['Instances']:   
    
    instance_id = instance['InstanceId']
    
    if 'State' not in instance:
      print(f"Missing state for {instance_id}")
    else:
      state = instance['State']['Name']
      if state == 'stopped':        
        instances_stopped.append(instance_id)
        #print(instance_id, state)

print(instances_stopped)

answered Mar 11 '21 at 11:04

Marcin

108,294
7
83
138

1

You made it again ,thanks it is working now . – Rahul Mar 11 '21 at 11:12
Now guide me on how can I become awesome in python like you – Rahul Mar 11 '21 at 11:13
1

@RahultheSWE No problem. You will become better with python the more you use it. Now it may seem difficult, next year you may be answering expert level python questions on SO :-) – Marcin Mar 11 '21 at 11:17

How to iterate through nested Dictionary in python boto3

1 Answers1