Add "entry" to JSON File with Python

Question

I need to modify a JSON-File with python. As I'm working with python (and JSON) for the first time, I read some articles about it, but didn't understand it completely.

I managed to import a JSON to python, as some kind of array (or list?).

JSON looks like this:

{
  "sources":[{
    "id":100012630,
    "name":"Activity Login Page",
    "category":"NAM/Activity",
    "automaticDateParsing":true,
    "multilineProcessingEnabled":false,
    "useAutolineMatching":false,
    "forceTimeZone":true,
    "timeZone":"Europe/Brussels",
    "filters":[],
    "cutoffTimestamp":1414364400000,
    "encoding":"UTF-8",
    "pathExpression":"C:\\NamLogs\\nam-login-page.log*",
    "blacklist":[],
    "sourceType":"LocalFile",
    "alive":true
  },{
    "id":100001824,
    "name":"localWinEvent",
    "category":"NAM/OS/EventLog",
    "automaticDateParsing":true,
    "multilineProcessingEnabled":false,
    "useAutolineMatching":false,
    "forceTimeZone":false,
    "filters":[],
    "cutoffTimestamp":1409090400000,
    "encoding":"UTF-8",
    "logNames":["Security","Application","System","Others"],
    "sourceType":"LocalWindowsEventLog",
    "alive":true
  },{
    "id":100001830,
    "name":"localWinPerf",
    "category":"NAM/OS/Perf",
    "automaticDateParsing":false,
    "multilineProcessingEnabled":false,
    "useAutolineMatching":false,
    "forceTimeZone":false,
    "filters":[],
    "cutoffTimestamp":0,
    "encoding":"UTF-8",
    "interval":60000,
    "wmiQueries":[{
      "name":"NAMID Service",
      "query":"SELECT * FROM Win32_PerfRawData_PerfProc_Process WHERE Name = 'tomcat7'"
    },{
      "name":"CPU",
      "query":"select * from Win32_PerfFormattedData_PerfOS_Processor"
    },{
      "name":"Logical Disk",
      "query":"select * from Win32_PerfFormattedData_PerfDisk_LogicalDisk"
    },{
      "name":"Physical Disk",
      "query":"select * from Win32_PerfFormattedData_PerfDisk_PhysicalDisk"
    },{
      "name":"Memory",
      "query":"select * from Win32_PerfFormattedData_PerfOS_Memory"
    },{
      "name":"Network",
      "query":"select * from Win32_PerfFormattedData_Tcpip_NetworkInterface"
    }],
    "sourceType":"LocalWindowsPerfMon",
    "alive":true
  },

Now, as I got hundreds of files like those, I wrote a foreach through the whole directory:

for filename in os.listdir('./json/'):
   with open('./json/'+filename) as data_file:    
   sources = json.load(data_file)

Now I would need something like again a foreach source in sources, which adds a row (or a entry or whatever a "line" in JSON is called) to every source (something like collectorName=fileName) and then overwrite the old file with the new one.

The JSON would then look like this:

   {
      "sources":[{
        "id":100012630,
        "name":"Activity Login Page",
        "category":"NAM/Activity",
        "automaticDateParsing":true,
        "multilineProcessingEnabled":false,
        "useAutolineMatching":false,
        "forceTimeZone":true,
        "timeZone":"Europe/Brussels",
        "filters":[],
        "cutoffTimestamp":1414364400000,
        "encoding":"UTF-8",
        "pathExpression":"C:\\NamLogs\\nam-login-page.log*",
        "blacklist":[],
        "sourceType":"LocalFile",
        "alive":true,
        "collectorName":"Collector2910"
      },{
        "id":100001824,
        "name":"localWinEvent",
        "category":"NAM/OS/EventLog",
        "automaticDateParsing":true,
        "multilineProcessingEnabled":false,
        "useAutolineMatching":false,
        "forceTimeZone":false,
        "filters":[],
        "cutoffTimestamp":1409090400000,
        "encoding":"UTF-8",
        "logNames":["Security","Application","System","Others"],
        "sourceType":"LocalWindowsEventLog",
        "alive":true,
        "collectorName":"Collector2910"
      },{.....

I hope I could explain my issue and I'd be happy if someone could help me out (even with a totally different solution).

Thanks in advance

Michael

Answer 1

for filename in os.listdir('./json/'):
   with open('./json/'+filename) as data_file:    
       datadict = json.load(data_file)
   # At this point you have a plain python dict.
   # This dict has a 'sources' key, pointing to
   # a list of dicts. What you want is to add
   # a 'collectorName': filename key:value pair
   # to each of these dicts
   for record in datadict["sources"]:
       record["collectorName"] = filename
   # now you just have to serialize your datadict back
   # to json and write it back to the file - which is
   # in fact a single operation
   with open('./json/'+filename, "w") as data_file:    
       json.dump(datadict, data_file)

Answer 2

Here's one way to do it:

for filename in os.listdir('./json/'):
    sources = None
    with open('./json/'+filename) as data_file:    
        sources = json.load(data_file)
        sourcelist = sources['sources']
        for i, s in enumerate(sourcelist):
            sources['sources'][i]['collectorName'] = 'Collector' + str(i)
    with open('./json/'+filename, 'w') as data_file:  
        data_file.write(json.dumps(sources))