---
title: 'CSC301 A2 While(true)'
disqus: hackmd
---

###### tags: `CSC301` `A2`
CSC301 A2 While(true)
===

[TOC]

:mag: Problems of the original code and implemented changes
---

### Style
**Variable Naming**
The following variables, and many other variables in the priginal twitch.py have names that do not at all reflect their function. The first examples are the sub-parsers:

sp = ap.add_subparsers(dest='sub')
p_crs = sp.add_parser('createchannel')

#### Changes to Naming ####
Meaningful variable names are assigned to reflect the type of data stored and variable and function names follow conventional naming formats.

sub_parser = parser.add_subparsers(dest='sub')
sp_createchannel = sub_parser.add_parser('createchannel')

**Comments**
There are a lot of unnecessary commented out codes which serve nothing besides confusion. Some comments are print statements, and we will use logging library to log those statements into another file:

#print("creating channel")

Others seem to be leftover code from an older implementation that now serves no purpose:

#c.execute('drop table if exists channels')
c.execute('''CREATE TABLE if not exists channels
                    (channel_id integer primary key, channel_name text)''')

#### Changes to Comments
logger feature is configured, all code will be logged to 'twitch.log' in via calls similar to:

self.logger.info("method createchannel called")

Log will be displayed in 'twitch.log':

2019-11-22 21:57:46,011 - twitch.py - INFO - method createchannel called

Docstrings are added for function and methods:

def createchannel(self, id, name):
        """
        Create a new Twitch channel with unique id, name
        :param id: channel identifier
        :type id: int
        :param name:  name of the channel
        :type name: str
        """

Comments are also added throughout the code to enchance readability.

### Structure
**Original Structure**
After executing python3 twitch.py -h, we can see the arguments we can use for this file. Since createchannel, parsetospam, gettospam, storechatlog and querychatlog all have different functionalities, we can seperate them into different functions.

lugeor2@DESKTOP-ART7IV7:/mnt/c/CSC/CSC301/A2/paired-assignment-2-while-true$ python3 twitch.py -h
usage: twitch.py [-h]
                 {createchannel,parsetopspam,gettopspam,storechatlog,querychatlog}
                 ...

Parse Twitch chatlogs

positional arguments:
  {createchannel,parsetopspam,gettopspam,storechatlog,querychatlog}

optional arguments:
  -h, --help            show this help message and exit

#### Changes to Original Structure
Each if statment in the original code has been refractored into methods of class Twitch, which is instantiated in main() function. This class contains the if statment functionalities: createchannel, parsetopspam, gettopspam, storechatlog,and querychatlog (also enchancement functionalities):

class Twitch():
    def __init__(self)
    def createchannel(self, id, name)
    def parsetopspam(self, file)
    def gettopspam(self, stream_id, channel_id)
    def storechatlog(self, file)
    def query_chat_log(self, filters)
    def gettopspam2(self, stream_id, channel_id)

def main():
    twitch = Twitch()
    
    twitch.<functionality> is called according 
    to parsed command line input

### Error Handling
All of the database calls in twitch.py do not contain error handling. That is, executing queries on tables that do not exist will raise a runtime error. Any operation that violates database constraints, such as creating two channels with the same id, will ao.s raise runtime errors.

Examples (when ran on empty database):

c.execute("select * from chat_log where stream_id = 497295395 AND offset >= 10 AND offset <= 100  order by chat_time")
>>> sqlite3.OperationalError: no such table: <table name>

c.execute("INSERT INTO CHANNELS VALUES ({137512364},'OverwatchLeague')")
c.execute("INSERT INTO CHANNELS VALUES ({137512364},'OverwatchLeague')")
>>> sqlite3.IntegrityError: UNIQUE constraint failed: channels.channel_id

These errors should be handled properly in the program and logging an "ERROR:" message to inform the user of invalid actions.

#### Changes to Error Handling
Errors produced by database operations are caught by try/except in DAO:

# error handling method for logging
def err_violate_integrity(self, table_name):
        """
        handles when UNIQUE constraints are violated in tables
        """
        logger.error('Integrity Constraint Violation in table: ' + table_name)

# where database modification takes place
try:
    self.getCursor().execute("INSERT INTO CHANNELS VALUES (?, ?)", (id, name))
except sqlite3.IntegrityError:
    self.err_violate_integrity("channel")

Wherever error is raised, error message will be logged to twitch.log file.
### Logging
twitch.py does not log runtime information. To improve this, logging library will be used to record runtime information and errors where required.
#### Changes to Logging
Two loggers are used, one for twitch.py, and one for dao.py:

import logging
# create logger with 'spam_application'
logger = logging.getLogger('dao.py') # or 'twitch.py'
logger.setLevel(logging.DEBUG)
# create file handler which logs even debug messages
fh = logging.FileHandler('twitch.log')
fh.setLevel(logging.DEBUG)
# create console handler with a higher log level
ch = logging.StreamHandler()
ch.setLevel(logging.ERROR)
# create formatter and add it to the handlers
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
fh.setFormatter(formatter)
ch.setFormatter(formatter)
# add the handlers to the logger
logger.addHandler(fh)
logger.addHandler(ch)

All "debug" and "info" level loggin messages are logged to twitch.log with filename identifier differentiating their origin:

2019-11-22 22:23:34,298 - twitch.py - INFO - function storechatlog called

2019-11-22 23:10:30,735 - twitch.py - WARNING - query_chat_log did not find matching result

"error" level logging, in addition to logging to twitch.log, will also be printed to console:

# operating on empty database

python twitch.py gettopspam 36029255 497295395
>>> 2019-11-22 23:45:04,460 - dao.py - ERROR - Table not found: top_spam

### Extensibility
This project is procedurally convoluted. All of the calls in main() function need to be separated into different components for better testing and logic. Components such as the building code for "where" clause (line 128 and below) is a functionality that can be generalized for all inputs of this format. Right now, if another feature (taking in a similar set of inputs) wishes to be added to the program, this chunk of building code will be copied and pasted, which is not an extensible practice.
Furthermore, all database operations follow the same steps: a connection with database is opened, some query/insertion/table creation is run on the database, and the connection is closed.
#### Changes

This poor extensibility is improved by a combination of all other points entailed in this document. Most significantly, reusable methods such as database calls are made into a DAO superclass where each subclass inherits logging methods, giving them standardized error logging format. Other methods, such as queries that are specific to a specific DAO subclass that only operate on chat_log table (DAO subclass ChatlogDao), overwrites not-implemented superclass select and insert methods. This tighter Object-Oriented structure, combined with more readable code, less redundant code, better logging and commenting, all contribute to making extending features an easier task.

### Object Orientation
The entire project is wrote in one function, there is no Object Orientation used.
#### Changes:
We used DAO design pattern to isolate our program function methods from any sqlite queries. In our dao.py, we have 3 seperate dao class, and they are a subclass of TwitchDao. We implemented these 3 DAO differently to reflect the different structure of the three tables. They are ChannelDao, SpamDao, and ChatlogDao, implemented with all necessary error cathing. We can make calls from our Dao objects to access and modify our database data, and for developer that wodef getCursor(self):rks in Twitch.py, they don’t have to know anything too specific about sqlite3 or sql queries.

class TwitchDao:
    def __init__(self):
        # initializes a connection
    def getCursor(self):
        # get cursor for connection
    def getDescription(self):
        # get cursor discription
    def createTable(self, *args):
        # sql CREATE new table
    def insert(self, *args):
        # sql INSERT into table
    def select(self, *args):
        # sql SELECT
    def delete(self, *args):
        # sql DELETE
    def commitCursor(self):
        # sqlite3 commit cursor
    def close(self):
        # close database connection  

Main function, which previously calls one of five if statements, now calls methods encapsulated in class Twitch().

class Twitch():
    def __init__(self)
    def createchannel(self, id, name)
    def parsetopspam(self, file)
    def gettopspam(self, stream_id, channel_id)
    def storechatlog(self, file)
    def query_chat_log(self, filters)
    def gettopspam2(self, stream_id, channel_id)
    def viewership(self, stream_id, channel_id): # pragma: no cover

### Unit Test
The unit-test code is located in the file testTwitch.py. It will cover all methods, exceptions, functions in our twitch.py and dao.py.

In dao.py, we have ignored the return statements of TwitchDao, since it is a super class, and all of its return statements are either `raise NotImplementedError` or `pass`. Also, we have ignored the part for our enhancement 1 select method.

Also, We have ignored these 2 functions in the coverage by using `# pragma: no cover`. Where the `select2` is called by dao in enhancement 1, and the `select_viewership` is called by dao in enhancement 2. The `offset_sec_2_min` is a helper for `select_viewership`.

select_viewership(self, channel_id, stream_id): # pragma: no cover

def offset_sec_2_min(self, table): # pragma: no cover

def select2(self, channel_id, stream_id): # pragma: no cover

In twitch.py we have reached an code coverage of 100% except for the part of our enhancement 1 & 2.

We have ignored the following functions and statements. `gettopspam2` is the enhancement 1 function, and `viewership` is the enhancement 2.

if __name__ == "__main__":  # pragma: no cover
gettopspam2(self, stream_id, channel_id):  # pragma: no cover
viewership(self, stream_id, channel_id): # pragma: no cover

To test our code coverage, we have used the python library of `coverage.py` using "pip install coverage". It is recorded in our lib.sh