Advanced Python
Project Discussion, Session 7

7.1 class Config: we can divide up the functionality by method (I recommend the methods listed below). As we complete each method, we will test.

  class Config Create a new class called Config as a stub.

class Config:
    pass       # means empty block - remove this in the next step

 

Test it with the following code:
conf = Config()

print(type(conf))    # <class 'mylib.Config'>

  def __init__(self, filename): This method sets the self.filename and self.format attributes based on the filename, and calls another method to build the config dict, assigning it to self.params (we'll write that method next). self.filename: you can simply set the filename argument to this attribute self.format: this should be set to the extension of the filename. So if the filename is test.csv, the self.format attribute value should be 'csv'; if the filename is test.json, the self.format attribute value should be 'json'. To do this, you may split the filename on '.', then use the last item of the resulting list. (Please don't test for csv or json; just use the extension, whatever it is.) But for even more precision, look into the os.path.splitext() method. Gnarlsburg! self.params: this should be assigned the dict containing the key/value pairs from the file. I recommend you use a separate method for parsing the file and loading the dictionary:

# self.params = self.read_csv_file()

Of course we won't uncomment this line until after creating the .read_csv_file() method -- see below. (The reason I recommend a separate method for reading the file is that we may later want to read files in other formats, such as .json or .ini; if we code each reading as a separate method, we can easily choose one or another based on the .format attribute.)

Test the method for .filename and .format:
conf = Config('pconfig.csv')

print(conf.filename)     # pconfig.csv
print(conf.format)       # csv


conf2 = Config('pconfig.json')

print(conf2.filename)    # pconfig.json
print(conf2.format)      # json

  def read_csv_file(self): Reads the CSV file and parses it into a dictionary of key/value pairs. The method then returns the dictionary. First, open and look at the contents of the sample file, pconfig.csv. You can see that the file has 2 values on each line. Each line represents a key/value pair. Clearly, this task is very similar to building a basic "lookup dictionary":

  • initialize an empty dict
  • use self.filename to open the file
  • pass the file object to csv.reader()
  • loop through the reader object row-by-row
  •    inside the loop, assign the 2 values as a key/value pair to the dict
  • return the completed dict

Test this method:
conf = Config('pconfig.csv')

params = conf.read_csv_file()     # TESTING PURPOSES ONLY

print(params)     # {'db_uname': 'george', 'db_password': 'password1',
                  #  'data_query': 'SELECT this, that FROM mytable
                  #  WHERE col = 5'}

Note that we are temporarily calling this method through conf for testing purposes, but in the next step we will be calling it through self inside the __init__() method. Call self.read_csv_file() from __init__() The code in __init__() should call the dict-parsing method self.read_csv_file() and assign the resulting dict to self.params.

inside __init__(), uncomment this statement
self.params = self.read_csv_file()

 

Test the completed __init__() method:
conf = Config('pconfig.csv')

print(conf.params)   # {'db_uname': 'george', 'db_password': 'password1',
                     #  'data_query': 'SELECT this, that FROM mytable
                     #  WHERE col = 5'}

Note that we'll be adding a raise to the method when it detects a corrupted file - this is described below.   def get(self, key, default=None): Returns the value for a key in this dict. This method works exactly like the dictionary .get() method. The dict is stored in self.params so you simply need to subscript the dict, i.e. self.params[key] to retrieve and then return the value. If the key cannot be found, the method should behave like the dict .get() method: it returns default (which will be None or a different value if it was specified by the call).

Test this method:
conf = Config('pconfig.csv')

val = conf.get('db_password')
print(val)                          # password1


# when the key can't be found in the <B>params</B> dict
val = conf.get('badkey')
print(val)                          # None (specified in the def)

val = conf.get('badkey', default=0)
print(val)                          # 0 (specified in the call)

default=None in the def is specified to be a keyword argument. If the user specifies it in the call, the value is assigned to variable default and will be used if the requested key is found. If default= is not passed by the caller, then the value of default will be None. Of course, default will not be used at all if the key can be found. If the key cannot be found, then the method should return the value of default. def __str__(self): This method is "automagically" called when the user attempts to print the object (or the object is converted to string with str(), which print() does implicitly). The method must return a string. The returned string will be printed by print() when the user attempts to print the object. To get the class name, consider self.__class__.__name__. After completing all tests, your code should work with the example code in the assignment. Raising Exceptions Please raise exceptions in response to the below (please never exit() from a function or method, nor merely print() the error message): In read_csv_file(): the file does not parse properly if it is "corrupted", meaning it does not have two fields on each line. One approach in determining that the file is "corrupted" is to attempt to parse it in the usual way, and allow Python to raise an exception when it reaches the corrupted line, and then trap the exception. For example, if your read_csv_file() code looks like this, a "corrupted" file will raise an error as indicated below:

reader = csv.reader(open(self.filename))   # use 'corrupted.csv'
param_dict = {}
for key, val in reader:       # ValueError:  not enough values to unpack
    ret_dict[key] = val

The easy way to handle this is to put a try/except around the 'for' block and trap the ValueError exception. Then, in the except ValueError: block, raise your own exception with raise ValueError('file appears to be corrupted') or more descriptive error message that is specific to the current situation. (The reason we want to raise our own ValueError exception is that Python's error message "not enough values to unpack" is not at all helpful to the person using our code. It is far superior to raise our own exception with a more appropriate error message.) Please note that if your code is reading the CSV file using a different approach than mine, you may get an IndexError or other type of error when attempting to read the corrupted.csv file. In that case, please allow the error to happen (but put it in a try: block), note the error type, and trap that error instead of the ValueError demonstrated above. An alternative to trapping the exception is to simply count the number of values on each line. If there are not 2 values on each line, the file is corrupted and you can raise your custom ValueError exception.

 
7.2 class ConfigItems: what's fascinating about this solution is that if you copy Config and simply add a __getitem__()

There shouldn't be much more needed. Simply change the names of the method, and then use subscripts instead of .get(). (However, you would probably want to retain the .get() method in addition to having a method called __getitem__(), since this method emulates the behavior of a dict. Let me know if you see any anomalies or problems.

 
7.3 (Extra Credit). class ConfigDict inheriting from dict

No discussion provided! :/

 
7.4 (Extra Credit). Allow class Config to parse .json and .ini files.

You will use the file extension to determine how to parse the files. Then, for each file type, write two functions that correspond to read_csv_file() that read the other formats (so, you will have 3 new functions in all). It should be a simple matter to look at the file extension (i.e., the .format value ('csv', 'ini' or 'json')) and call the appropriate parsing function. Each read function should return the same dict that read_csv_file() does. In this way the rest of the program can work in the same way.

 
7.5 (Extra credit.) Write a simple doc reader. The doc reader takes any class, loops through its attributes and, if the attribute object is callable (i.e., it is a method or function), prints its name and its docstring.

Use inspect.getmembers() to generate a list of tuples. Call inspect.getmembers(Config), passing the completed Config class -- this is not an instance of the class, but the class itself. If you have not yet completed the Config class, you can substitute any class, such as str, list, etc. - but do not use an instance of the class. inspect.getmembers() returns a list of 2-item tuples. The 1st item in each tuple is an attribute name, and the 2nd item is the object to which that attribute points. Loop through and print each attribute name. You should see a list of attributes starting with "magic" attributes and ending with the methods defined in the class. Determine whether an attribute's object is a function or method. Functions and methods are objects like any other, but they are special in the sense that they are callable, meaning you can call them (i.e., with parentheses). Python provides a built-in function callable() which will return True if the object is callable. Inside your loop over attribute names and objects, pass each object to callable() (make sure you are passing the object (2nd item of each tuple) and not the attribute name (1st item)). Only print the attribute name if its object is callable. Print the docstring of each object. Most class, method and function objects have docstrings. These are located in the object's .__doc__ attribute. Inside your loop, after the attribute name, print its docstring. Running the program, you should see each attribute name followed by a description of the attribute. Methods that you defined within the class appear at the end, printing the docstrings that you supplied with each method. Print each method argument signature. inspect.signature() will show you any function or method's argument signature as a special object that, when converted to string, illustrates the function or method's expected arguments. Print the attribute name with the object returned from inspect.signature() to complete the method name with argument signature in your printed output. Note that one callable attribute, class, will fail with ValueError if you attempt to call inspect.signature(). Trap this exception and skip over signature for this attirbute.

 
7.6 (Extra Credit).class AttrType

No discussion provided! :/ It's a tricky one, so tread carefully! Use your best attributes ;)

 
[pr]