1

I am trying to create a python script and I am stuck with the dictionaries. I have read through some of the other forums but can't seem to get anywhere. I am a very new python programmer so please be gentle.

What I want to do:

1) set up a dictionary like this: {'Name':'userid','jobid:jobid','walltime:walltime,'nodes:nds'}

2) iterate through a list of entries created from and external function call and extract information to populate the dictionary

3) Problem: I cannot figure out how to append entries to the appropriate keys

For example, I want this: 

{‘Name’:’jose’,’jobid’:’001,002,003,005’,’walltime:32:00,240:00,04:00,07:00’,’nodes’:32,32,500’}

Notice for one userid, I have multiple jobids, walltimes and nodes.
(len(jobids)==len(walltimes)==len(nodes) for any one userid but can vary across userids)

I am able to get the script to find the first value for each username, but it never appends. How can I get this to append?

Here is what I have tried

from collections import defaultdict
pdict = defaultdict(list)

start the loop:

 # get new values – add these to the dictionary keyed 
 # on username (create a new entry or append to existing entry)
    …
    (jobid,userid,jobname, sessid, nds, tsk, walltime,rest)= m.groups()  

    ...
    if userid in pdict:
       print "DEBUG: %s is currently in the dictionary -- appending entries" %(userid)
       pdict[userid][‘jobid’] = pdict[userid][jobid].append(jobid)    I 
     # repeat for nodes, walltime, etc 

    if not userid in pdict:
        print "DEBUG: %s is not in the dictionary creating entry" %(userid)
        pdict[userid] = {}  # define a dictionary within a dictionary with key off userid
        pdict[userid]['jobid'] = jobid
        pdict[userid]['jobname'] = jobname
        pdict[userid]['nodes'] = nds
        pdict[userid]['walltime'] = walltime

I know this is wrong but can’t figure out how to get the append to work. I have tried many of the suggestions offered on this site. I need to append (to the dictionary) the most recent values from the loop keyed to userid

Here is an example of the ouput – it does not append multiple entries for each userid but rather takes only the first value for each userid

userid jmreill contains data: {'nodes': '1', 'jobname': 'A10012a_ReMig_Q', 'walltime': '230:0', 'jobid': '1365582'}

userid igorysh contains data: {'nodes': '20', 'jobname': 'emvii_Beam_fwi6', 'walltime': '06:50', 'jobid': '1398100'}

Any suggestions? This should be easy but I can’t figure it out!

SparkAndShine
  • 14,337
  • 17
  • 76
  • 119
Jose Otero
  • 11
  • 1
  • Do you get an error? Perhaps `AttributeError: 'str' object has no attribute 'append'`? You have to create lists for the keys that will hold multiple values, then you'll be able to `append()`. So something like `pdict[userid]['jobid'] = [jobid]` – Jasper May 13 '16 at 19:14
  • It is more pythonic to use `if userid not in pdict` instead of `if not userid in pdict`. – SparkAndShine May 13 '16 at 19:28

6 Answers6

1
from collections import defaultdict

pdict = defaultdict(dict)

start the loop:

 # get new values – add these to the dictionary keyed 
 # on username (create a new entry or append to existing entry)
    …
    (jobid,userid,jobname, sessid, nds, tsk, walltime,rest)= m.groups()  

    ...
    if userid in pdict:
       print "DEBUG: %s is currently in the dictionary -- appending entries" %(userid)
       pdict[userid][jobid].append(jobid)
     # repeat for nodes, walltime, etc 

    if userid not in pdict:
        print "DEBUG: %s is not in the dictionary creating entry" %(userid)
        pdict[userid]['jobid'] = [jobid]
        pdict[userid]['jobname'] = jobname
        pdict[userid]['nodes'] = nds
        pdict[userid]['walltime'] = walltime
malbarbo
  • 9,223
  • 1
  • 30
  • 46
0

The value corresponding to key 'jobid' should be a list of strings rather than a string. If you create your dictionary this way, you can append new jobid's to the list simply by:

pdict[userid]['jobid'].append(jobid)
Tonechas
  • 11,520
  • 14
  • 37
  • 68
0

I cant remember the explanation why to use the lambda expression in the following code, but you have to define a defaultdict of a defaultdict:

pdict = defaultdict(lambda: defaultdict(list))

pdict[userid][‘jobid’].append('1234')

will work.

cbo
  • 98
  • 5
0

The append() method does not return the list...it is modified in place. Also, you need to initialize your elements as lists (using square brackets):

if userid in pdict:
   print "DEBUG: %s is currently in the dictionary -- appending entries" %(userid)
   pdict[userid][jobid].append(jobid)  ## just call append here
 # repeat for nodes, walltime, etc 

if not userid in pdict:
    print "DEBUG: %s is not in the dictionary creating entry" %(userid)
    pdict[userid] = {}  # define a dictionary within a dictionary with key off userid
    pdict[userid]['jobid'] = [jobid,]  ## initialize with lists
    pdict[userid]['jobname'] = [jobname,]
    pdict[userid]['nodes'] = [nds,]
    pdict[userid]['walltime'] = [walltime,]
gariepy
  • 3,458
  • 5
  • 19
  • 32
0

append doesn't return a value, it modifies the list in place, and you forgot to quote 'jobid' on the right of equals. So you should replace pdict[userid][‘jobid’] = pdict[userid][jobid].append(jobid) with pdict[userid]['jobid'].append(jobid). Also take into account comment from @Jasper.

emh
  • 199
  • 9
0

You are looking for a dict of dicts? AutoVivification is the perfect solution. Implement the perl’s autovivification feature in Python.

class AutoVivification(dict):
    """Implementation of perl's autovivification feature."""
    def __getitem__(self, item):
        try:
            return dict.__getitem__(self, item)
        except KeyError:
            value = self[item] = type(self)()
            return value​

This makes everything easier. Note that the value of pdict[userid]['jobid'] should be a list [jobid] instead of a variable jobid as you have multiple jobid.

pdict = AutoVivification()

if userid in pdict:
    pdict[userid]['jobid'].append(jobid) 
else:
    pdict[userid]['jobid'] = [jobid] # a list

Refer to What is the best way to implement nested dictionaries in Python?.

Community
  • 1
  • 1
SparkAndShine
  • 14,337
  • 17
  • 76
  • 119