Delete JSON Objects Where Certain Keys Have Empty Values

Question

I have a json as follows:

[{
        "Domain": "google.com",
        "A": ["172.217.22.46"],
        "AAAA": ["2a00:1450:4001:81e::200e"],
        "CAA": ["0 issue \"pki.goog\""],
        "MX": ["20 alt1.aspmx.l.google.com.", "30 alt2.aspmx.l.google.com.", "10 aspmx.l.google.com.", "40 alt3.aspmx.l.google.com.", "50 alt4.aspmx.l.google.com."],
        "NS": ["ns1.google.com.", "ns3.google.com.", "ns2.google.com.", "ns4.google.com."],
        "SOA": ["ns1.google.com. dns-admin.google.com. 189483475 900 900 1800 60"],
        "TXT": ["\"docusign=05958488-4752-4ef2-95eb-aa7ba8a3bd0e\"", "\"v=spf1 include:_spf.google.com ~all\""],
        "Country": ["United States"],
        "Hostname": ["'fra15s16-in-f46.1e100.net'"],
        "SSL": ["Google Internet Authority G2"],
        "WHOIS": [8400],
        "TTL": ["24"]
    }, {
        "Domain": "",
        "NS": ["a.root-servers.net.", "h.root-servers.net.", "g.root-servers.net.", "i.root-servers.net.", "d.root-servers.net.", "k.root-servers.net.", "c.root-servers.net.", "l.root-servers.net.", "e.root-servers.net.", "m.root-servers.net.", "f.root-servers.net.", "j.root-servers.net.", "b.root-servers.net."],
        "SOA": ["a.root-servers.net. nstld.verisign-grs.com. 2018031700 1800 900 604800 86400"],
        "A": [],
        "SSL": ["None"],
        "WHOIS": [0],
        "TTL": [null]
    }, {
        "Domain": "facebook.com",
        "A": ["157.240.20.35"],
        "AAAA": ["2a03:2880:f11c:8183:face:b00c:0:25de"],
        "MX": ["10 msgin.vvv.facebook.com."],
        "NS": ["a.ns.facebook.com.", "b.ns.facebook.com."],
        "SOA": ["a.ns.facebook.com. dns.facebook.com. 1521364737 14400 1800 604800 300"],
        "TXT": ["\"v=spf1 redirect=_spf.facebook.com\""],
        "Country": ["United States"],
        "Hostname": ["'edge-star-mini-shv-02-frt3.facebook.com'"],
        "SSL": ["DigiCert SHA2 High Assurance Server CA"],
        "WHOIS": [10227],
        "TTL": ["173"]
    }]

Please note the second element, the one that has "Domain" : "", which I want to remove(there maybe multiple instances where "Domain" : "" is like that, this is only partial JSON) and return a valid json devoid of that whole object, so the output must look like:

[{
        "Domain": "google.com",
        "A": ["172.217.22.46"],
        "AAAA": ["2a00:1450:4001:81e::200e"],
        "CAA": ["0 issue \"pki.goog\""],
        "MX": ["20 alt1.aspmx.l.google.com.", "30 alt2.aspmx.l.google.com.", "10 aspmx.l.google.com.", "40 alt3.aspmx.l.google.com.", "50 alt4.aspmx.l.google.com."],
        "NS": ["ns1.google.com.", "ns3.google.com.", "ns2.google.com.", "ns4.google.com."],
        "SOA": ["ns1.google.com. dns-admin.google.com. 189483475 900 900 1800 60"],
        "TXT": ["\"docusign=05958488-4752-4ef2-95eb-aa7ba8a3bd0e\"", "\"v=spf1 include:_spf.google.com ~all\""],
        "Country": ["United States"],
        "Hostname": ["'fra15s16-in-f46.1e100.net'"],
        "SSL": ["Google Internet Authority G2"],
        "WHOIS": [8400],
        "TTL": ["24"]
    }, {
        "Domain": "facebook.com",
        "A": ["157.240.20.35"],
        "AAAA": ["2a03:2880:f11c:8183:face:b00c:0:25de"],
        "MX": ["10 msgin.vvv.facebook.com."],
        "NS": ["a.ns.facebook.com.", "b.ns.facebook.com."],
        "SOA": ["a.ns.facebook.com. dns.facebook.com. 1521364737 14400 1800 604800 300"],
        "TXT": ["\"v=spf1 redirect=_spf.facebook.com\""],
        "Country": ["United States"],
        "Hostname": ["'edge-star-mini-shv-02-frt3.facebook.com'"],
        "SSL": ["DigiCert SHA2 High Assurance Server CA"],
        "WHOIS": [10227],
        "TTL": ["173"]
}]

Here is what I tried (the example is executable as is):

import json

if __name__ == "__main__":

    json_data = "[{\r\n\t\t\"Domain\": \"google.com\",\r\n\t\t\"A\": [\"172.217.22.46\"],\r\n\t\t\"AAAA\": [\"2a00:1450:4001:81e::200e\"],\r\n\t\t\"CAA\": [\"0 issue \\\"pki.goog\\\"\"],\r\n\t\t\"MX\": [\"20 alt1.aspmx.l.google.com.\", \"30 alt2.aspmx.l.google.com.\", \"10 aspmx.l.google.com.\", \"40 alt3.aspmx.l.google.com.\", \"50 alt4.aspmx.l.google.com.\"],\r\n\t\t\"NS\": [\"ns1.google.com.\", \"ns3.google.com.\", \"ns2.google.com.\", \"ns4.google.com.\"],\r\n\t\t\"SOA\": [\"ns1.google.com. dns-admin.google.com. 189483475 900 900 1800 60\"],\r\n\t\t\"TXT\": [\"\\\"docusign=05958488-4752-4ef2-95eb-aa7ba8a3bd0e\\\"\", \"\\\"v=spf1 include:_spf.google.com ~all\\\"\"],\r\n\t\t\"Country\": [\"United States\"],\r\n\t\t\"Hostname\": [\"'fra15s16-in-f46.1e100.net'\"],\r\n\t\t\"SSL\": [\"Google Internet Authority G2\"],\r\n\t\t\"WHOIS\": [8400],\r\n\t\t\"TTL\": [\"24\"]\r\n\t}, {\r\n\t\t\"Domain\": \"\",\r\n\t\t\"NS\": [\"a.root-servers.net.\", \"h.root-servers.net.\", \"g.root-servers.net.\", \"i.root-servers.net.\", \"d.root-servers.net.\", \"k.root-servers.net.\", \"c.root-servers.net.\", \"l.root-servers.net.\", \"e.root-servers.net.\", \"m.root-servers.net.\", \"f.root-servers.net.\", \"j.root-servers.net.\", \"b.root-servers.net.\"],\r\n\t\t\"SOA\": [\"a.root-servers.net. nstld.verisign-grs.com. 2018031700 1800 900 604800 86400\"],\r\n\t\t\"A\": [],\r\n\t\t\"SSL\": [\"None\"],\r\n\t\t\"WHOIS\": [0],\r\n\t\t\"TTL\": [null]\r\n\t}, {\r\n\t\t\"Domain\": \"facebook.com\",\r\n\t\t\"A\": [\"157.240.20.35\"],\r\n\t\t\"AAAA\": [\"2a03:2880:f11c:8183:face:b00c:0:25de\"],\r\n\t\t\"MX\": [\"10 msgin.vvv.facebook.com.\"],\r\n\t\t\"NS\": [\"a.ns.facebook.com.\", \"b.ns.facebook.com.\"],\r\n\t\t\"SOA\": [\"a.ns.facebook.com. dns.facebook.com. 1521364737 14400 1800 604800 300\"],\r\n\t\t\"TXT\": [\"\\\"v=spf1 redirect=_spf.facebook.com\\\"\"],\r\n\t\t\"Country\": [\"United States\"],\r\n\t\t\"Hostname\": [\"'edge-star-mini-shv-02-frt3.facebook.com'\"],\r\n\t\t\"SSL\": [\"DigiCert SHA2 High Assurance Server CA\"],\r\n\t\t\"WHOIS\": [10227],\r\n\t\t\"TTL\": [\"173\"]\r\n\t}]"
    ds = json.loads(json_data)
    # cleaning empty entries
    for item in ds:
        try:
            if item["Domain"]:
                del item["Domain"]
        except KeyError:
            print("Key doesn't exist")
    print(ds)

However, I could not achieve my target. How to achieve this? Thanks!

https://stackoverflow.com/a/32509350/4180176 && https://stackoverflow.com/a/11277439/4180176 — Joshua Nixon, Mar 19 '18 at 15:29
@JoshuaNixon went through them before posting, I think that would just remove the dict `"Domain" : ""` and not the whole obj — Jishan, Mar 19 '18 at 15:30

score 3 · Answer 1 · answered Mar 19 '18 at 15:32

3

[dom for dom in ds if dom.get('Domain')]

Uses a list comprehension to select all the dictionaries which have a non-falsey value for "Domain".

answered Mar 19 '18 at 15:32

Constance

202
2
8

Kaushik NP · Answer 2 · 2018-03-19T15:33:47.867

2

The process of creating a new object by selectively choosing the required elements is better and less error prone than deleting elements and modifying the object.

Since you got json part covered, let me show you how to get the required dict. Its as simple as doing :

>>> [ele for ele in d if ele['Domain']]

Where d is the dictionary obtained from the input json.

#driver values :

IN : d = [{'Domain':'A', 'xyz':'abc'}, {'Domain':'', 'xyz':'bcd'},
          {'Domain':'C', 'xyz':'cde'}]

OUT : [{'Domain': 'A', 'xyz': 'abc'}, {'Domain': 'C', 'xyz': 'cde'}]

edited Mar 19 '18 at 15:33

answered Mar 19 '18 at 15:30

Kaushik NP

6,188
8
29
57

This could be written using truthyness of non-empty strings, which would arguably be more pythonic: `[ele for ele in d if ele['Domain']]` – bgse Mar 19 '18 at 15:33
@bgse : yeah. Wrote in the flow. WIll update. thanks. – Kaushik NP Mar 19 '18 at 15:34

BallpointBen · Answer 3 · 2018-03-19T15:33:42.480

Your issue is that you are deleting keys from the objects and not the whole objects.

import json

if __name__ == "__main__":

    json_data = "[{\r\n\t\t\"Domain\": \"google.com\",\r\n\t\t\"A\": [\"172.217.22.46\"],\r\n\t\t\"AAAA\": [\"2a00:1450:4001:81e::200e\"],\r\n\t\t\"CAA\": [\"0 issue \\\"pki.goog\\\"\"],\r\n\t\t\"MX\": [\"20 alt1.aspmx.l.google.com.\", \"30 alt2.aspmx.l.google.com.\", \"10 aspmx.l.google.com.\", \"40 alt3.aspmx.l.google.com.\", \"50 alt4.aspmx.l.google.com.\"],\r\n\t\t\"NS\": [\"ns1.google.com.\", \"ns3.google.com.\", \"ns2.google.com.\", \"ns4.google.com.\"],\r\n\t\t\"SOA\": [\"ns1.google.com. dns-admin.google.com. 189483475 900 900 1800 60\"],\r\n\t\t\"TXT\": [\"\\\"docusign=05958488-4752-4ef2-95eb-aa7ba8a3bd0e\\\"\", \"\\\"v=spf1 include:_spf.google.com ~all\\\"\"],\r\n\t\t\"Country\": [\"United States\"],\r\n\t\t\"Hostname\": [\"'fra15s16-in-f46.1e100.net'\"],\r\n\t\t\"SSL\": [\"Google Internet Authority G2\"],\r\n\t\t\"WHOIS\": [8400],\r\n\t\t\"TTL\": [\"24\"]\r\n\t}, {\r\n\t\t\"Domain\": \"\",\r\n\t\t\"NS\": [\"a.root-servers.net.\", \"h.root-servers.net.\", \"g.root-servers.net.\", \"i.root-servers.net.\", \"d.root-servers.net.\", \"k.root-servers.net.\", \"c.root-servers.net.\", \"l.root-servers.net.\", \"e.root-servers.net.\", \"m.root-servers.net.\", \"f.root-servers.net.\", \"j.root-servers.net.\", \"b.root-servers.net.\"],\r\n\t\t\"SOA\": [\"a.root-servers.net. nstld.verisign-grs.com. 2018031700 1800 900 604800 86400\"],\r\n\t\t\"A\": [],\r\n\t\t\"SSL\": [\"None\"],\r\n\t\t\"WHOIS\": [0],\r\n\t\t\"TTL\": [null]\r\n\t}, {\r\n\t\t\"Domain\": \"facebook.com\",\r\n\t\t\"A\": [\"157.240.20.35\"],\r\n\t\t\"AAAA\": [\"2a03:2880:f11c:8183:face:b00c:0:25de\"],\r\n\t\t\"MX\": [\"10 msgin.vvv.facebook.com.\"],\r\n\t\t\"NS\": [\"a.ns.facebook.com.\", \"b.ns.facebook.com.\"],\r\n\t\t\"SOA\": [\"a.ns.facebook.com. dns.facebook.com. 1521364737 14400 1800 604800 300\"],\r\n\t\t\"TXT\": [\"\\\"v=spf1 redirect=_spf.facebook.com\\\"\"],\r\n\t\t\"Country\": [\"United States\"],\r\n\t\t\"Hostname\": [\"'edge-star-mini-shv-02-frt3.facebook.com'\"],\r\n\t\t\"SSL\": [\"DigiCert SHA2 High Assurance Server CA\"],\r\n\t\t\"WHOIS\": [10227],\r\n\t\t\"TTL\": [\"173\"]\r\n\t}]"
    ds = [d 
          for d in json.loads(json_data)
          if d.get("Domain")
         ]

Your `if` might be more easily written: `if d.get("Domain")` since `get()` returns a Falsey value if the key is not found. — Robᵩ, Mar 19 '18 at 15:32

score 1 · Accepted Answer · answered Mar 19 '18 at 15:31

Try this:

import json

if __name__ == "__main__":

    json_data = "[{\r\n\t\t\"Domain\": \"google.com\",\r\n\t\t\"A\": [\"172.217.22.46\"],\r\n\t\t\"AAAA\": [\"2a00:1450:4001:81e::200e\"],\r\n\t\t\"CAA\": [\"0 issue \\\"pki.goog\\\"\"],\r\n\t\t\"MX\": [\"20 alt1.aspmx.l.google.com.\", \"30 alt2.aspmx.l.google.com.\", \"10 aspmx.l.google.com.\", \"40 alt3.aspmx.l.google.com.\", \"50 alt4.aspmx.l.google.com.\"],\r\n\t\t\"NS\": [\"ns1.google.com.\", \"ns3.google.com.\", \"ns2.google.com.\", \"ns4.google.com.\"],\r\n\t\t\"SOA\": [\"ns1.google.com. dns-admin.google.com. 189483475 900 900 1800 60\"],\r\n\t\t\"TXT\": [\"\\\"docusign=05958488-4752-4ef2-95eb-aa7ba8a3bd0e\\\"\", \"\\\"v=spf1 include:_spf.google.com ~all\\\"\"],\r\n\t\t\"Country\": [\"United States\"],\r\n\t\t\"Hostname\": [\"'fra15s16-in-f46.1e100.net'\"],\r\n\t\t\"SSL\": [\"Google Internet Authority G2\"],\r\n\t\t\"WHOIS\": [8400],\r\n\t\t\"TTL\": [\"24\"]\r\n\t}, {\r\n\t\t\"Domain\": \"\",\r\n\t\t\"NS\": [\"a.root-servers.net.\", \"h.root-servers.net.\", \"g.root-servers.net.\", \"i.root-servers.net.\", \"d.root-servers.net.\", \"k.root-servers.net.\", \"c.root-servers.net.\", \"l.root-servers.net.\", \"e.root-servers.net.\", \"m.root-servers.net.\", \"f.root-servers.net.\", \"j.root-servers.net.\", \"b.root-servers.net.\"],\r\n\t\t\"SOA\": [\"a.root-servers.net. nstld.verisign-grs.com. 2018031700 1800 900 604800 86400\"],\r\n\t\t\"A\": [],\r\n\t\t\"SSL\": [\"None\"],\r\n\t\t\"WHOIS\": [0],\r\n\t\t\"TTL\": [null]\r\n\t}, {\r\n\t\t\"Domain\": \"facebook.com\",\r\n\t\t\"A\": [\"157.240.20.35\"],\r\n\t\t\"AAAA\": [\"2a03:2880:f11c:8183:face:b00c:0:25de\"],\r\n\t\t\"MX\": [\"10 msgin.vvv.facebook.com.\"],\r\n\t\t\"NS\": [\"a.ns.facebook.com.\", \"b.ns.facebook.com.\"],\r\n\t\t\"SOA\": [\"a.ns.facebook.com. dns.facebook.com. 1521364737 14400 1800 604800 300\"],\r\n\t\t\"TXT\": [\"\\\"v=spf1 redirect=_spf.facebook.com\\\"\"],\r\n\t\t\"Country\": [\"United States\"],\r\n\t\t\"Hostname\": [\"'edge-star-mini-shv-02-frt3.facebook.com'\"],\r\n\t\t\"SSL\": [\"DigiCert SHA2 High Assurance Server CA\"],\r\n\t\t\"WHOIS\": [10227],\r\n\t\t\"TTL\": [\"173\"]\r\n\t}]"
    ds = json.loads(json_data)
    keep = []
    for item in ds:
        if item.get("Domain"):
            keep.append(item)
    print(json.dumps(keep))

This code builds a new list of the items to keep.

Being from a long Java background, this is most readable to me, selecting the same for answer — Jishan, Mar 19 '18 at 15:38

R.R.C. · Answer 5 · 2018-03-19T15:44:02.480

1

Try the following:

for k in ds:
    if not ds[k]:
        ds.remove(k)

This way you're going to have k removed from ds if the value for k is either 0, None, False, empty list or empty string and it would work for any given key.

edited Mar 19 '18 at 15:44

answered Mar 19 '18 at 15:35

R.R.C.

4,128
3
12
17

Delete JSON Objects Where Certain Keys Have Empty Values

5 Answers5