Introduction to Python

davidbpython.com

Data Parsing & Extraction: File Operations and the 'for' Loop

the 'for' loop with a list

'for' with a list repeats its block as many times as there are items in the list.

mylist = [1, 2, 'b']

for var in mylist:       # 1
    print(var)           # ===
    print('===')         # 2
                         # ===
print('done')            # b
                         # ===
                         # done

Similar to a while block, the for block repeats the contents of its block multiple times, but does so only the number itms in the list. The control variable var is reassigned for each iteration of the loop. This means that if the list has 3 items, the loop executes 3 times and var is reassigned a new value 3 times.

review: the concept of incrementing

We reassign the value of an integer to effect an incrementing.

x = 0         # int, 0

x = x + 1     # int, 1
x = x + 1     # int, 2
x = x + 1     # int, 3

print(x)      # 3

For each of the three incrementing statements above, a new value that equals the value of x is created, and then assigned back to x. The previous value of x is replaced with the new, incremented value. Incrementing is most often used for counting within loops -- see next.

using a 'for' loop to count list items

An integer, updated for each iteration, can be used to count iterations.

mylist = [1, 2, 'b']

my_counter = 0

for var in mylist:
    my_counter = my_counter + 1

print(f'count:  {my_counter} items')   # counter:  3 items

The value of my_counter is initialized at 0 before the loop begins. Then, since the incrementing line my_counter = my_counter + 1 is inside the loop, the value of my_counter goes up once with each iteration. Please note that the len() function can count list items more efficiently, but we are using a counter to demonstrate the counter technique, which can be used in situations where len() can't be used, as when we count lines.

using a 'for' loop to sum list items

An integer, updated for each iteration, can be used to count iterations.

mylist = [1, 2, 3]

my_sum = 0

for val in mylist:
    my_sum = my_sum + val

print(f'sum:  {my_sum}')     # sum: 6  (value of 1 + 2 + 3)

The value of my_sum is initialized at 0 before the loop begins. Then, since the incrementing line my_sum = my_sum + val is inside the loop, the value of my_sum goes up once with each iteration. Please note that the sum() function can count list items more efficiently, but we are using a summing variable to demonstrate the summing technique, which can be used in situations where sum() can't be used, as when we are summing values from a file.

opening and reading a file with 'for'

'for' with a file repeats its block as many times as there are lines in the file.

fh = open('students.txt')              # file object allows
                                       # looping through a
                                       # series of strings

for xx in fh:                          # xx is a string, a line of the file
    print(xx)                           # prints each line of students.txt

fh.close()                             # close the file

"xx" is called a control variable, and it is automatically reassigned each line in the file as a string. break and continue work with for as well as while loops. Again, the control variable xx is reassigned for each iteration of the loop. This means that if the file has 5 lines, the loop executes 5 times and xx is reassigned a new value 5 times.

reading a file with 'with'

A file is automatically closed upon exiting the 'with' block.

A 'best practice' is to open files using a 'with' block. When execution leaves the block, the file is automatically closed.

with open('pyku.txt') as fh:
    for line in fh:
        print(line)

## at this point (outside the with block), filehandle fh has been closed.

• open 'read' files do not block other processes from accessing the file
• however, we are sometimes concerned with good file "housekeeping"
• it's always best to close a file as soon as you are done with it
• when writing to a file, this need becomes more critical
• with saves us from having to remember to close the file

summarizing: csv parsing with 'for' looping and string parsing

Here we put together all features learned in this session.

fh = open('revenue.csv')          # 'file' object

counter = 0
summer = 0.0

for line in fh:                   # str, "Haddad's,PA,239.50\n"

    line = line.rstrip()          # str, "Haddad's,PA,239.50"
    fieldlist = line.split(',')   # list, ["Haddad's", 'PA', '239.50']

    rev_val = fieldlist[2]        # str, '239.50'   (value from first line)
    f_rev = float(rev_val)        # float, 239.5

    counter = counter + 1
    summer = summer + f_rev

fh.close()

print(f'counter:  {counter}')     # 7 (number of lines in file)
print(f'summer:   {summer}')      # 662.01000001  (sum of all 3rd col values in file)

• This example puts together everything we learned in this session. Each line is a string, which gets stripped, split into fields and then the last item in the line converted to float. We then use a summing variable to sum up the values found on each line.

• If we wish now we can derive an average value by dividing summer by counter.

• (Note that the tiny remainder is expected and can be rounded to 2 places.)

sidebar: writing and appending to files using the file object

Files can be opened for writing or appending; we use the file object and the file write() method.

fh = open('new_file.txt', 'w')
fh.write("here's a line of text\n")
fh.write('I add the newlines explicitly if I want to write to the file\n')
fh.close()

fh = open('new_file.txt')
lines = fh.readlines()
print(lines)
  # ["here's a line of text\n",
  #  'I add the newlines explicitly if I want to write to the file\n']

fh.close()

Note that we are explicitly adding newlines to the end of each line. The write() method doesn't do this for us.