Extracting data from file to file in Python

Question

I would like to extract data from first file to second file and put them into specific labels. The first file looks like:

"city1" : [[1.1,1.2],[2.1,2.2],[3.1,3.2]],
"city2" : [[5.0,0.2],[4.1,3.2],[7.1,8.2]],
...

So type is like dictionary where values are list of lists

Unfortunately while opening file I got error: too many values to unpack

Im trying to open like:

lines = {}
with open("shape.txt", "r") as f:
for line in f:
    (key,val) = line.split()
    d[key] = val

After that I would like to extract this cities and coordinates to second file which has structure like this:

<state name = 'city1'>
 <point lat='first value from first list', lng='second value from first list/>
 <point lat='first value from second list', lng='second value from second list/>
</state>
<state name = 'city2'>
 the same action like above

And I was thinking if there is a other solution?

Answer 1

Here is another way which writes directly to the second file. This way you do not need to store it dict first. Much more performant if you handle some big file:

with open("shape.txt", "r") as f1:
    with open("shape_output.txt", "w") as f2:
        for line in f1:
            (key, val) = line.split(":")
            coord = json.loads(val.strip().rstrip(','))
            city = key.replace('"', '').strip()

            new_line = f"<state name = '{city}'>"
            for c in coord:
                new_line += f"<point lat='{c[0]}', lng='{c[1]}'/>"
            new_line += "</state>"

            f2.write(new_line)
            f2.write("\n")

I add some cleaning when reading lines from the first file. And then use json.loads to convert the array from the string format to array type.

The rest is just formating and writing to the second file.

Answer 2

If your text file contains only this structure you have stated in question, you should be able to use ast.literal_eval to parse the data:

txt = '''

"city1" : [[1.1,1.2],[2.1,2.2],[3.1,3.2]],
"city2" : [[5.0,0.2],[4.1,3.2],[7.1,8.2]],

'''

template = '''<state name = '{city}'>
  <point lat='{vals[0][0]}', lng='{vals[0][1]}' />
  <point lat='{vals[1][0]}', lng='{vals[1][1]}' />
</state>'''

from ast import literal_eval

data = literal_eval('{' + txt + '}')

print(data)

for k, v in data.items():
    print(template.format(city=k, vals=v))

Prints:

<state name = 'city1'>
  <point lat='1.1', lng='1.2' />
  <point lat='2.1', lng='2.2' />
</state>
<state name = 'city2'>
  <point lat='5.0', lng='0.2' />
  <point lat='4.1', lng='3.2' />
</state>

---

With files I/O:

template = '''<state name = '{city}'>
  <point lat='{vals[0][0]}', lng='{vals[0][1]}' />
  <point lat='{vals[1][0]}', lng='{vals[1][1]}' />
</state>'''

from ast import literal_eval

with open('sample.txt', 'r') as f_in, open('sample.out', 'w') as f_out:
    data = literal_eval('{' + f_in.read() + '}')

    for k, v in data.items():
        print(template.format(city=k, vals=v), file=f_out)

---

EDIT: This example will print all the points to the file:

from ast import literal_eval

with open('sample.txt', 'r') as f_in, open('sample.out', 'w') as f_out:
    data = literal_eval('{' + f_in.read() + '}')

    for k, v in data.items():
        print("<state name = '{city}'>".format(city=k), file=f_out)
        for point in v:
            print("\t<point lat='{point[0]}', lng='{point[1]}' />".format(point=point), file=f_out)
        print('</state>', file=f_out)

The file sample.out will look like:

<state name = 'city1'>
    <point lat='1.1', lng='1.2' />
    <point lat='2.1', lng='2.2' />
    <point lat='3.1', lng='3.2' />
</state>
<state name = 'city2'>
    <point lat='5.0', lng='0.2' />
    <point lat='4.1', lng='3.2' />
    <point lat='7.1', lng='8.2' />
</state>

Answer 3

You can easily load and save lists and dictionaries with it.

import json

data = {
  'city1': [[1.1,1.2],[2.1,2.2],[3.1,3.2]],
  'city2': [[5.0,0.2],[4.1,3.2],[7.1,8.2]],
}

with open('file.txt', 'w') as f:
   f.write(json.dumps(data))

with open('file.txt') as f:
   data = json.loads(f.read())

But it will work only if file has valid json. You file is almost valid JSON except that it does not have curly braces

So I think it can be done like this:

lines = ['{']
with open('file') as f:
for line in f:
    lines.append(line)
lines[-1].strip(',')  # Strip last comma as it's not valid for JSON dict
lines.append('}')
data = json.loads('\n'.join(lines))

Then just do:

result = ''
for city, points in data.items():
    result += f"<state name='{city}'>\n"
    for point in points:
        result += f"<point lat='{point[0]}', lng='{point[1]}'/>\n"
    result += '</state>\n'

 with open('out.txt', 'w') as f:
     f.write(result)