URL: http://gnosis.cx/publish/programming/charming_python_5.txt

UPDATE 2 (November 2017)

Sorry for not having updated this in so many years. 2004 was a different Peter and I'm sorry if people landed on this blog post and got the wrong idea.

To read lines from a file do this:


with open('somefile.txt') as f:
   for line in f:
       print(line)

This works universally in Python 2 and Python 3. It reads one line at a time by iterating till it finds line breaks.

When you create a file object in Python you can read from it in several different ways. Not until today did I understand the difference between readline() and readlines(). The answer is in the name. readline() reads one line character at a time, readlines() reads in the whole file at once and splits it by line.

These would then be equivalent:


f = open('somefile.txt','r')
for line in f.readlines():
    print line
f.close()

# ...and...

f = open('somefile.txt','r')
for line in f.read().split('\n'):
    print line
f.close()

The xreadlines() function should be used for big files:


f = open('HUGE.log','r'):
for line in f.xreadlines():
   print line
f.close()

From Charming Python

" The difference between .readline() and .readlines() is that the latter, like .read(), reads in an entire file at once. .readlines() automatically parses the read contents into a list of lines, thereby enabling the for ... in ... construct common in Python. Using .readline() reads in just a single line from a file at a time, and is generally much slower than .readlines(). Really the only reason to use the .readline() version is if you expect to read very large files that might exceed available memory."

UPDATE

Thanks commentors for pointing out that I and the Charmed Python book got it completely wrong. readline() reads one line at a time. xreadlines() also reads one line at a time. I've changed the example code above.

Comments

Post your own comment
Dan Atterton

Another useful way to read a file is the 'read()' function. I use it when I need a string to feed the md5.new() function.

fileContent = open( fname,'r').read()
# since I didn't assign the file object to a variable, it's closed after read.

md5Object=md5.new( fileContent )

theMD5 = md5Object.digest()

This is useful if you are trying to find duplicate files (notably pictures) by their md5 checksum.

Dan

Mikael Engdahl

You can't use the for ... in ... construct with readline() the same was as you can readlines(). It would read only one line and iterate it character by character.

Anonymous

Actually, the two are not quite the same. readlines() will return newlines on the end of your strings, while readline() will not.

Ludvig Ericson

>>> f.readline()
'\x1b[0;35m .vir. d$b\n'
>>> f.readlines()
['\x1b[0;35m .d$$$$$$b. .cd$$b. .d$$b. d$$$$$$$$$$$b .d$$b. .d$$b.\n', (cut) ]

Notice how both end in \n.

Anonymous

Code is wrong, too bad google gives it such a high page rank

Peter Bengtsson

sorry. updated now.

Nicholas Labello

Peter, thanks for providing/updating this page. The examples were very useful.

Ludvig Ericson

Uhm.

I think you desperately need to realize how Python works.
file.readline() - reads ONE line.
file.readlines() - reads ALL lines, and splits them by line delimiter.

Now, imagine:
f = file("/etc/motd", "r")
print f.readline()
Will that print the first line or first character of that file? The first line.
Consider this:
for line in f.readline():
print line
This will _of course_ print each charater, since **iterating str objects returns each character of them in a sequence!**

raj

i seem to have a problem with readlines()as below, am i doing anything wrong?

i have

global f

f=open("hello.txt", "r")

f1 = open("C:\\log1.txt",'w')
f2 = open("C:\\log2.txt",'w')

def new1():
for line in f.readlines():
if line.find('raj')>= 0:
f1.write("%s" %line)

def new2():
for line in f.readlines():
if line.find('abc')>= 0:
f2.write("%s" %line)

new1()
new2()

i don't seem to get an output for new2 ,i mean it doesn't go through the for loop

Am i doing anything wrong ??

lars

In new1 you read the file f to the end.
and when you come to new2
f is at end of file.
Thats why
Before new2()
f.close()
f=open("hello.txt", "r")

tom

cheers peter...

you keep popping up in my results...

minhducit

In your code f.read().split('\n') will return one more line comparing with the f.readlines().
However the last line is ""

So they are not equal :)

Oscar Lindberg

I just want to mention that what you probably want to do when reading a file line by line is this: (Taken from the File documentation of Python):

f = open("hello.txt")
try:
..for line in f:
....print line
finally:
..f.close()

xreadlines() is deprecated since release 2.3. The "for line in f" pattern is recommended.

Tam

I'm stuck with file reading, just thought the users here might be able to point out how to go about what I want to do.

The basic loop is

f = open("hello.txt")
try:
..for line in f:
....print line
finally:
..f.close()

as shown above.

However, in place of print line, I'm trying to search the current line for a particular string. I'm pretty new to Python, and I'm not sure how I do this. Can anyone here help me out with this? Thanks.

Peter Bengtsson

The short answer is::

if line.find('mystring') > -1:
....my_matched_function()

The long answer is:
http://docs.python.org/tut/node5.html#SECTION005120000000000000000

Anonymous

your link does not work along with everything else on this page.

Tam

Thanks for the help Peter, sorry to bother you with such a trivial question as well.

Endolith

You probably know the answer to my question. PEP 234 says that these are equivalent:

for line in iter(f.readline, ""):
    ...


for line in f:
    ...

But they are not. When reading from a named pipe, the first will work after a newline has been sent to the pipe and then the pipe has been flushed, while the latter only outputs after a large number of bytes have been put into the pipe. Why? Some kind of buffering with the internal next function?

http://ubuntuforums.org/showthread.php?t=916518

Anonymous

Hi Im new to python.I have a data in the form of matrix(rows and columns) .I need to open the .dat file in python and want to access each elemnt or store all the elments in to single dimensional array . How can I do that in simple way.

Anonymous

thanx well appriciated

anthonyyy12

Hi Guys.

I am in need of major help right now. I am a new guy to python, programming and computer science, and the problem I face seems like a basic one, but what i need to know is how to write a programm that will access a file i have saved on my desktop, and that will return a random quotation that comes from that file.
I would really appreciate the help as soon as possible.

wrybread

Hilarious how often I reference this page. You'd think I'd have it memorized by now. I literally use this page at least once a month.

Me = retarded, you = made something very useful.

Peter Nau

It is useful. However, if you reference it frequently, just think of it as external memory, not a bad thing at all...

Anonymous

yaah...really good link this is.

Byneni revathi

Yaah.... i get some more which i want,and i felt really happy to meet this cite.Thank you.

James

Your update doesn't make any sense. file.readline() does read a single line, not a character. *Iterating* on the line, of course, will iterate on characters.

deepak suresh

File "z.py", line 538, in main
    rec_file = codecs.open(args[0],"r","utf-8")
  File "/usr/lib/python2.6/codecs.py", line 870, in open
    file = __builtin__.open(filename, mode, buffering)
IOError: [Errno 2] No such file or directory: 'dsauto1'


since i have file called dsauto1 in directory ...why these error???

arvindh

hey,
 i have a doubt related to readline(). read line one after another and print specific characters within each line.
my question is kind of huge. which i have asked over here http://stackoverflow.com/questions/27938954/python-to-extract-and-modify-data-from-to-txt-file
it would be kind enough if you do take a look. thanks

Anonymous

This posting is wrong. readline() reads one line at a time. read() reads one character at a time. readlines() reads the entire file into a list.

Mike Thomas

No! Why have you left this error in place on a page with the top Google hit for "Python readlines vs readline". readline() DOES read in a line. To be exact, it reads from the buffer until it includes a newline and returns that as a string.

Your email will never ever be published.

Related posts

Go to top of the page