This is a collection of Python code snippets that I frequently use in building larger scripts. Most of my scripting tends to be for processing text files – e.g. for extracting interesting data from a log file, converting file formats, etc. – and so a lot of the snippets are of that flavor. Many of these snippets or similar can be found in books or other web sites and in these cases, appropriate references to those other sources are provided.

Reading Text Files

#1: Process each line in a text file one line at a time.

for line in file('foo.txt', 'r').xreadlines():
    # process each line
    print line

#2: Read all the lines of a text file into a list with some simple processing per line.

lines = [line.strip()
         for line in file('foo.txt', 'r').xreadlines()]

#3: Read all the lines of a text file into an array if they match a certain pattern.

lines = [line.strip()
         for line in file('foo.txt', 'r').xreadlines()
         if re.search(pattern, line)]

CSV File Processing

#4: Generate a csv file one row at a time
Reference: Python library reference for the csv module

import csv
writer = csv.writer(open('foo.csv', 'wb'))
writer.writerow(['some data', 1, 2, 3, 'and more'])

#5: Write multiple rows into a csv file in one go.
Reference: Python library reference for the csv module

import csv
writer = csv.writer(open('foo.csv', 'wb'))
writer.writerows([['some data', 1, 2, 3, 'and more'],
                  ['and yet more', 4]])

#6: Read rows from a csv file.
Reference: Python library reference for the csv module

import csv
reader = csv.reader(open('foo.csv', 'rb'))
for row in reader:
    print row

#7: Reading a csv file with a delimiter other than ‘,’
Reference: Python library reference for the csv module

import csv
reader = csv.reader(open('foo.csv', 'rb'), delimiter=':')
for row in reader:
    print row

List processing

#8: Removing duplicate items from a list.
Note: This snippet does not guarantee that the relative ordering of the items in the output list will be the same as that of the input list.

dupList = [8,1,2,1,4,4,5,7,8,5]
trimmed = list(set(dupList))