How to avoid splitting names with spaces in python

Question

I am trying to parse commodity pricing data from http://agmarknet.nic.in/ and trying to store it in my DB.

I am getting the data in the form of Ambala Cantt. 1.2 Bitter Gourd 1200 2000 1500 and then i am splitting it through split() and storing it in the DB.But some of the names have spaces between their name for split() are splitting it too and breaking it as:

['Ambala' ,'Cantt.', '1.2', 'Bitter', 'Gourd', '1200', '2000', '1500']

But i want it to be as:

['Ambala Cantt.', '1.2', 'Bitter Gourd', '1200', '2000', '1500']

I am iterating the data in the for each loop and then splitting it.To resolve this issue i tried regular expression as

 ([c.strip() for c in re.match(r"""
        (?P<market>[^0-9]+)
        (?P<arrivals>[^ ]+)
        (?P<variety>[^0-9]+)
        (?P<min>[0-9]+)
        \ (?P<max>[0-9]+)
        \ (?P<modal>[0-9]+)""",
        example,
        re.VERBOSE
    ).groups()])

the above block of code is working fine if i write example = "Ambala Cantt. 1.2 Bitter Gourd 1200 2000 1500" but if you put it inside the for each loop as **for example in y:

([c.strip() for c in re.match(r"""
    (?P<market>[^0-9]+)
    (?P<arrivals>[^ ]+)
    (?P<variety>[^0-9]+)
    (?P<min>[0-9]+)
    \ (?P<max>[0-9]+)
    \ (?P<modal>[0-9]+)""",
    example,
    re.VERBOSE
).groups()])

.I am getting an Attribute error as **re.VERBOSE AttributeError: 'NoneType' object has no attribute 'groups'.My code looks like this

   params = urllib.urlencode({'cmm': 'Bitter gourd', 'mkt': '', 'search': ''})
    headers = {'Cookie': 'ASPSESSIONIDCCRBQBBS=KKLPJPKCHLACHBKKJONGLPHE; ASP.NET_SessionId=kvxhkhqmjnauyz55ult4hx55; ASPSESSIONIDAASBRBAS=IEJPJLHDEKFKAMOENFOAPNIM','Origin': 'http://agmarknet.nic.in', 'Accept-Encoding': 'gzip, deflate', 'Accept-Language': 'en-GB,en-US;q=0.8,en;q=0.6','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36', 'Content-Type': 'application/x-www-form-urlencoded','Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8', 'Cache-Control': 'max-age=0','Referer': 'http://agmarknet.nic.in/mark2_new.asp','Connection': 'keep-alive'}
    conn = httplib.HTTPConnection("agmarknet.nic.in")
    conn.request("POST", "/SearchCmmMkt.asp", params, headers)
    response = conn.getresponse()
    data = response.read()
    soup = bs(data, "html.parser")
    #print dir(soup)
    z = []
    y = []
    w = []
    x1 = []
    test = []
    trs = soup.findAll("tr")
    for tr in trs:
        c = unicodedata.normalize('NFKD', tr.text)
        y.append(str(c))
    for x in y:
        #data1 = "Ambala 1.2 Onion 1200 2000 1500"
        x1 =    ([c.strip() for c in re.match(r"""
            (?P<market>[^0-9]+)
            (?P<arrivals>[^ ]+)
            (?P<variety>[^0-9]+)
            (?P<min>[0-9]+)
            \ (?P<max>[0-9]+)
            \ (?P<modal>[0-9]+)""",
            x,
            re.VERBOSE
        ).groups()])
    print x1.

Can anyone help me how i can get my data in the form of ['Ambala Cantt.', '1.2', 'Bitter Gourd', '1200', '2000', '1500'] instead of getting it as ['Ambala' ,'Cantt.', '1.2', 'Bitter', 'Gourd', '1200', '2000', '1500'].

Answer 1

use shlex module

import shlex

l = "Ambala Cantt. 1.2 Bitter Gourd 1200 2000 1500"
# first put quotes around word pairs
l = re.sub(r'([A-Z]\w+\s+\w+)',r'"\1"',l)
# then split with shlex, it will not split inside the quoted strings
l = shlex.split(l)

['Ambala Cantt.', '1.2', 'Bitter Gourd', '1200', '2000', '1500']

you can run it as a one liner:

result = shlex.split(re.sub(r'([A-Z]\w+\s+\w+)',r'"\1"',"Ambala Cantt. 1.2 Bitter Gourd 1200 2000 1500"))