---
# System prepended metadata

title: Checkoff 1

---

# Checkoff 1

## What we have
- Front end
- Back end 
    - Logging
    - Metadata endpoints
    - Reviews endpoints
- Data Cleaning
    - 12/400 way there

## Questions to ask

### Data Cleaning
1. The requirement was to have title and author fields being searchable in our production system. As of now, we have only managed to find the missing fields for 12k/400k of the dataset. If we cannot find all, how? Do we drop all that dont exist or do we let some have title/author and some dont
2. How do we scrape now that we keep getting booted off amazon for being a bot


### Logging Requirement
1. The requirement for logging was that we have the ability to `track user activities`. Can this be the logs of http requests made for each user
2. Does the mongodb instance for logging have to be on a completely seperate AWS instance? Or can we just park it as a DB in the same instance

### Reviews Requirement
1. Do we have to obtain BCNF Normalisation for our database, or is 2NF/3NF sufficient? Is this part considered for the rubrics?

## Going forward
1. For ingestion engine, would the class be going thru some of that part or are we going to self learn