# T135314: Get Fedex Tracking Detailed Fact Items
*Joanna Jiang
2019-07-08*

---

## 1. Summary
Provided with Fedex tracking numbers, I'd like to get more detailed information about packages such as weight, dimension, ship date, actual delivery date etc, which give us more information in detecting fake tracking cases.

I use [Selenium Webdriver](https://www.techbeamers.com/selenium-webdriver-python-tutorial/) to achieve automatic execution of actions include browsing through web pages and locating fact items. It can interact with all types of Web browsers and here I use Chrome.

The input file is ['fedex_tracking_ids_0708.csv'](https://docs.google.com/spreadsheets/d/1lxzYoK_5R9uZapRpCkZ21UmJ2UrFxUFzCtKnJnzn_Ek/edit#gid=1566663646) containing one column of 100 Fedex tracking number as in figure 1. And the output file is ['shipping_details_0708.csv'](https://docs.google.com/spreadsheets/d/1yIdBS1n5y-nyak9margt5YXoznlxQsl-bVLUQAwvAoY/edit#gid=261584133) with Fedex tracking number and 21 fact items associated as in figure 2.
<!-- ::: -->

*<div style="text-align: center" > Figure 1 </div>*

*<div style="text-align: center" > Figure 2 </div>*

###### tags: `Wish`

## 2. Procedure
> 1. Get tracking ids from 'fedex_tracking_ids_0708.csv'

> 2. Open 'shipping_details_0708.csv' for writing (create if not exist)

> 3. Open the Fedex tracking page for a particular tracking_id

> 4. Locate and click on "Shipment Facts" tab
*(Note that here [Selenium Webdriver wait](https://www.techbeamers.com/selenium-webdriver-waits-python/) is used to handle any unexpected condition which may occur.)*

> 5. Retrieve value for each fact item.
*(Note that for each tracking number, different fact item might be shown. Here I only record prespecified 21 fact items of interest. We can also invesigate other fact items by looking at warning printout if there's any.)*

> 6. If retrieving values doesn't succeed, still record this tracking number

> 7. Write all fact items

## 3. Python script

import csv
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

TRACKING_ID_PATH = 'fedex_tracking_ids_0708.csv'
SHIP_DETAILS_PATH = 'shipping_details_0708.csv'
FEDEX_TRACK_TEMPLATE = 'https://www.fedex.com/apps/fedextrack/?action=track&trackingnumber={}&cntry_code=us&locale=en_US'
fact_items = ['TRACKING NUMBER', 'INVOICE NUMBER', 'DOOR TAG NUMBER', 'DEPARTMENT NUMBER', 'SERVICE', 'REFERENCE', 'WEIGHT',
          'DIMENSIONS', 'TOTAL PIECES', 'TERMS', 'PURCHASE ORDER NUMBER', 'SHIPMENT ID', 'SHIPPER REFERENCE',
          'PACKAGING', 'TOTAL SHIPMENT WEIGHT', 'SPECIAL HANDLING SECTION', 'STANDARD TRANSIT', 'SHIP DATE',
          'SCHEDULED DELIVERY', 'ACTUAL DELIVERY', 'DELIVERED TO', 'SIGNATURE SERVICES']

# get tracking ids from csv file
track_id_file = open(TRACKING_ID_PATH)
track_id_reader = csv.reader(track_id_file)
track_ids = [row[0] for row in track_id_reader][1:]

# open shipping details file for writing (create if not exist)
ship_details_file = open(SHIP_DETAILS_PATH, mode='w')
ship_details_writer = csv.DictWriter(ship_details_file, fieldnames=fact_items)
ship_details_writer.writeheader()

driver = webdriver.Chrome()

try:
    counter = 1

    for track_id in track_ids:
        print('Retrieving shipping details for package {}: {}'.format(counter, track_id))

        fact_dict = dict.fromkeys(fact_items, '')

        # open the web page for a particular tracking_id
        driver.get(FEDEX_TRACK_TEMPLATE.format(track_id))

        try:
            # locate and click on "Shipment Facts" tab
            facts_tab = WebDriverWait(driver, 20)\
                .until(EC.element_to_be_clickable(
                    (By.CSS_SELECTOR, 'a.detailViewTabs_area.tank-tabs__link.factsTab.label')))
            driver.execute_script("arguments[0].click();", facts_tab)

            # Retrieve value for each fact item
            facts_list = driver.find_elements_by_css_selector('li.shipmentFactsRowTVC.tank-ship-                                    facts__item')
            for fact in facts_list:
                # locate fact items key By Tag name
                key = fact.find_element_by_tag_name('em').text
                # locate fact items value by CSS Selector
                value = fact.find_element_by_css_selector(
                        'span.shipmentFactsRowTVC.value.tank-ship-facts__item-description').text
                if key in fact_dict:
                    fact_dict[key] = value
                else:
                    print("WARNING: unknown fact item", track_id, key, value)

        # if retrieving values doesn't succeed, still record this tracking number
        except:
            fact_dict['TRACKING NUMBER'] = track_id

        # write all fact items
        ship_details_writer.writerow(fact_dict)
        ship_details_file.flush()

        counter += 1

finally:
    driver.quit()

## 4. Result and Runtime
For the 100 tracking ids, it takes about 1.5s on average for each to write all factor items into csv file.

For the past 30 days, the daily average count of unique Fedex tracking number was about [3501](https://console.treasuredata.com/app/queries/editor?queryId=1240842), thus it can take around 88 munites to get daily detailed fedex tracking information.

Due to peak traffic or network latency, it may behave differently than it does at normal conditions.