Python script stops writing but in shown as a running process

Question

I am running a python script on a screen on linux server and when I do TOP command I can see it running but it has been for hours that the script did not write anything. Anybody know what could be the reason?

Here is my script:

import GeoIP
from netaddr import *

gi = GeoIP.GeoIP("/data/GeoIPOrg_20141202.dat", GeoIP.GEOIP_MEMORY_CACHE)
o = Path to the output text file

for line in f:
    line = line.strip('\n')
    asn,ip,count =line.split('|')
    org = gi.org_by_addr(ip)
    start,end = gi.range_by_ip(ip)
    ran = list(IPRange(start,end))
#    ipcount = len(ran)
    ip_start,ip_end = IPAddress(start),IPAddress(end)
    n_start = int(ip_start)
    n_end = int(ip_end)
    range = (n_start,n_end)
    print ("%s|%s|%s|%s|%s|%s" % (asn,range,len(ran),org,ip,count) , file = o)

@Nick B cause the size +content of the last printed line of the file stays the same — UserYmY, Jan 21 '15 at 16:08

score 1 · Accepted Answer · edited May 23 '17 at 10:24

1

This could be a few things; hard to say without seeing how you're running and how you're initialising that file.

A definite possibility is the file isn't being flushed (more relevantly, see the docs on changing the buffer size of open() as it's probably being invoked in your code).

Either way it's worth using Python (2.5+)'s with statement to handle file / resource management neatly and robustly instead of relying on print e.g.:

with open("/my/output/path.txt", "w") as out_file:
    # Rest of code
    # ...
    out_file.write("%s|%s|%s|%s|%s|%s\n" % (asn,range,len(ran),org,ip,count))

See this SO question for good examples of using the with statement.

edited May 23 '17 at 10:24

Community

1
1

answered Jan 21 '15 at 16:06

declension

3,764
19
23

`with open("/my/output/path.txt") as out_file:` should be `with open("/my/output/path.txt", "w") as out_file:` – dopstar Jan 23 '15 at 16:16
This solution will remove separate lines. how can I deal with that? – UserYmY Jan 25 '15 at 08:56
1

@Mee - good point - needs a newline to keep it equivalent to your version, so I've updated the answer accordingly – declension Jan 25 '15 at 12:21

score 1 · Answer 2 · answered Jan 23 '15 at 16:12

You have two ways to achieve this.

You can change your code to use the with statement (context manager) as @Nick B has suggested in his answer to open your file there.
Or you can set the buffering where you open the file to be line buffering.

So where you say:

# Im assuming you open your file like this since your code is
# an incomplete snippet. Otherwise tell us how you open your file
o = open('output_file.log', 'w')

You must say:

o = open('output_file.log', 'w', buffering=1)  # enable line buffering

You should read the help of then open command by typing help(open) in the interactive python shell. It explains a great deal how buffering works in python.

Python script stops writing but in shown as a running process

2 Answers2