AWS boto - Instance Status/Snapshot Status won't update Python While Loop

Question

So I am creating a Python script with boto to allow the user to prepare to expand their Linux root volume and partition. During the first part of the script, I would like to have a While loop or something similar to make the script not continue until:

a) the instance has been fully stopped

b) the snapshot has been finished creating.

---

Here are the code snippets for both of these:

Instance:

ins_prog = conn.get_all_instances(instance_ids=src_ins, filters={"instance-state":"stopped"})
while ins_prog == "[]":
    print src_ins + " is still shutting down..."
    time.sleep(2) 
        if ins_prog != "[]":
        break

Snapshot:

snap_prog = conn.get_all_snapshots(snapshot_ids=snap_id, filters={"progress":"100"})
while snap_prog == "[]":
    print snap.update
    time.sleep(2) 
        if snap_prog != "[]":
        print "done!"
        break

So when calling conn.get_all_instances and conn.get_all_snapshots they return an empty list if the filters show nothing, which is formatted like []. The problem is the While loop does not even run. It's as if it does not recognize [] as the string produced by the get_all functions.

If there is an easier way to do this, please let me know I am at a loss right now ):

Thanks!

Edit: Based on garnaat's help here is the follow up issue.

snap_prog = conn.get_all_snapshots(snapshot_ids=snap.id)[0]
print snap_prog
print snap_prog.id
print snap_prog.volume_id
print snap_prog.status
print snap_prog.progress
print snap_prog.start_time
print snap_prog.owner_id
print snap_prog.owner_alias
print snap_prog.volume_size
print snap_prog.description
print snap_prog.encrypted 

Results:

Snapshot:snap-xxx
snap-xxx
vol-xxx
pending

2015-02-12T21:55:40.000Z
xxxx
None
50
Created by expandDong.py at 2015-02-12 21:55:39
False  

Note how snap_prog.progress returns null, but snap_prog.status stays as 'pending' when being placed in a While loop.

SOLVED:

MY colleague and I found out how to get the loop for snapshot working.

snap = conn.create_snapshot(src_vol)

while snap.status != 'completed':
    snap.update()
    print snap.status
    time.sleep(5)
        if snap.status == 'completed':
        print snap.id + ' is complete.'
        break

snap.update() call purely updates the variable snap to return the most recent information, where snap.status outputs the "pending" | "completed". I also had an issue with snap.status not showing the correct status of the snapshot according to the console. Apparently there is a significant lagtime between the Console and the SDK call. I had to wait ~4 minutes for the status to update to "completed" when the snapshot was completed in the console.

Answer 1

If I wanted to check the state of a particular instance and wait until that instance reached some state, I would do this:

import time    
import boto.ec2
conn = boto.ec2.connect_to_region('us-west-2')  # or whatever region you want
instance = conn.get_all_instances(instance_ids=['i-12345678'])[0].instances[0]
while instance.state != 'stopped':
    time.sleep(2)
    instance.update()

The funny business with the get_all_instances call is necessary because that call returns a Reservation object which, in turn, has an instances attribute that is a list of all matching instances. So, we are taking the first (and only) Reservation in the list and then getting the first (and only) Instance inside the reservation. You should probably but some error checking around that.

The snapshot can be handled in a similar way.

snapshot = conn.get_all_snapshots(snapshot_ids=['snap-12345678'])[0]
while snapshot.status != 'completed':
    time.sleep(2)
    snapshot.update()

The update() method on both objects queries EC2 for the latest state of the object and updates the local object with that state.

Answer 2

I will try to answer this generally first. So, you are querying a resource for its state. If a certain state is not met, you want to keep on querying/asking/polling the resource, until it is in the state you wish it to be. Obviously this requires you to actually perform the query within your loop. That is, in an abstract sense:

state = resource.query()
while state != desired_state:
    time.sleep(T)
    state = resource.query()

Think about how and why this works, in general.

Now, regarding your code and question, there are some uncertainties you need to figure out yourself. First of all, I am very sure that conn.get_all_instances() returns an empty list in your case and not actually the string '[]'. That is, your check should be for an empty list instead of for a certain string(*). Checking for an empty list in Python is as easy as not l:

l = give_me_some_list()
if not l:
    print "that list is empty."

The other problem in your code is that you expect too much of the language/architecture you are using here. You query a resource and store the result in ins_prog. After that, you keep on checking ins_prog, as if this would "magically" update via some magic process in the background. No, that is not happening! You need to periodically call conn.get_all_instances() in order to get updated information.

(*) This is documented here: http://boto.readthedocs.org/en/latest/ref/ec2.html#boto.ec2.connection.EC2Connection.get_all_instances -- the docs explicitly state "Return type: list". Not string.