Looping over a list while conditon is met

README.md

‎‎Looping over a list

Imagine you have a list where you then need to take a set portion of it as a batch.

Below gits shows how to use a while loop for this to creat batches.

By converting two lists into sets and then comparing them, we can get a list of items that weren't batch first time around. This works for a regular list.

However for a list of json objects, as above, we can't use set. If used as a regular list, it throws below error.

TypeError: unhashable type: 'dict'

and the reason is, as nicely described here

You're trying to use a dict as a key to another dict or in a set. That does not work because the keys have to be hashable. As a general rule, only immutable objects (strings, integers, floats, frozensets, tuples of immutables) are hashable (though exceptions are possible).

Therefore, to do the same, we could take advantage of slice as shown in the second code gist.

batch_loop_while.py

def main():
    batch_cutoff = 10

    # ------ NOTEME: simple python list with items --------
    msgEntries_sendBatch = []
    # msgEntries = [1,3,4,5,7,6,40,2,60,70,9,30,10,8,48,22,0,45] # 18 in the list
    msgEntries = [1,3,4,5,7,8,9,2,6,0] # 10 in the list i.e. same as the cutoff no. 
    # msgEntries = [1,3,4,5,7] # 5 in the list
    # msgEntries = [] # 0 in the list
    
    while len(msgEntries) != 0:
        msgEntries_sendBatch = msgEntries[:batch_cutoff]
        print msgEntries_sendBatch

        msgEntries = list(set(msgEntries) - set(msgEntries_sendBatch))

        print len(msgEntries) 
    
    print 'came here now'
    print '-------------------------------------------'

if __name__ == "__main__":
    main()

batch_loop_while_json.py

def main():
    batch_cutoff_json = 2

    msgEntries_sendBatch_json = []
    msgEntries_holdBatch_json = []

    msgEntries_json = [
        {'Id': '9a21f0ff-8e02-4870-a3dc-cd9495452b0b', 'MessageBody': 'This MiSeq step is ready for FusionQC.', 
            'MessageAttributes': {'clarityLIMS_Instance': {u'DataType': 'String', u'StringValue': 'PROD'}, 'stepID': {u'DataType': 'String', u'StringValue': '24-15005'}}
        }, 
        {'Id': 'b1ad4ad4-e83e-43b9-9b2e-3c6ab37fbde4', 'MessageBody': 'This MiSeq step is  ready for FusionQC.', 
            'MessageAttributes': {'clarityLIMS_Instance': {u'DataType': 'String', u'StringValue': 'PROD'}, 'stepID': {u'DataType': 'String', u'StringValue': '24-14030'}}
        },
        {'Id': '9a21f0ff-8e02-4870-a3dc-cd94ee452b0b', 'MessageBody': 'This MiSeq step is ready for FusionQC.', 
            'MessageAttributes': {'clarityLIMS_Instance': {u'DataType': 'String', u'StringValue': 'PROD'}, 'stepID': {u'DataType': 'String', u'StringValue': '24-150505'}}
        }, 
        {'Id': 'b1aq4ad4-e83e-4w9-9b2e-3c6ab37fbde4', 'MessageBody': 'This MiSeq step is  ready for FusionQC.', 
            'MessageAttributes': {'clarityLIMS_Instance': {u'DataType': 'String', u'StringValue': 'PROD'}, 'stepID': {u'DataType': 'String', u'StringValue': '24-14036'}}
        }
    ]


    while len(msgEntries_json) != 0:
        msgEntries_sendBatch_json = msgEntries_json[:batch_cutoff_json]
        print msgEntries_sendBatch_json 

        msgEntries_holdBatch_json = msgEntries_json[batch_cutoff_json:]
        print msgEntries_holdBatch_json

        msgEntries_json = msgEntries_holdBatch_json
        print len(msgEntries_json) 
    
    print 'came here now - json'

if __name__ == "__main__":
    main()