---
# System prepended metadata

title: Pandas 1

---

# Pandas 1

---
title: Agenda
description:
duration: 300
card_type: cue_card
---

### Agenda

- Introduction to Pandas
- DataFrame & Series
- Creating DataFrame from Scratch (Post-read)
- Basic ops on a DataFrame
- Basic ops on Columns
    - Accessing column(s)
    - Check for unique values
    - Rename column
    - Deleting column(s)
    - Creating new column(s)
- Basic ops on Rows
    - Implicit/Explicit index
    - Indexing in Series
    - Slicing in Series
        - loc/iloc
    - Indexing/Slicing in DataFrame

---
title: Introduction to Pandas
description:
duration: 900
card_type: cue_card
---

### Pandas Installation

``` python=
!pip install pandas
```

### Importing Pandas

- You should be able to import Pandas after installing it.
- We'll import `pandas` using its **alias name** `pd`.

``` python=
import pandas as pd
import numpy as np
```

### Why use Pandas?

- The major **limitation of numpy** is that it can only work with one datatype at a time.
- Most real-world datasets contain a mix of different datatypes.
  - **names of a place would be string**
  - **population of a place would be int**
  
It is difficult to work with data having **heterogeneous values** using Numpy.

On the other hand, Pandas can work with numbers and strings together.

### Problem Statement

- Imagine that you are a Data Scientist with McKinsey.
- McKinsey wants to understand the relation between GDP per capita and life expectancy for their clients.
- The company has obtained data from various surveys conducted in different countries over several years.
- The acquired data includes information on -
  - Country
  - Population Size
  - Life Expectancy
  - GDP per Capita
- We have to analyse the data and draw inferences that are meaningful to the company.

### Loading the dataset

Dataset: https://drive.google.com/file/d/1E3bwvYGf1ig32RmcYiWc0IXPN-mD_bI_/view?usp=sharing

Code:
``` python=
!wget "https://drive.google.com/uc?export=download&id=1E3bwvYGf1ig32RmcYiWc0IXPN-mD_bI_" -O mckinsey.csv
```

> **Output**
```
--2023-01-20 17:42:26--  https://drive.google.com/uc?export=download&id=1E3bwvYGf1ig32RmcYiWc0IXPN-mD_bI_
    Resolving drive.google.com (drive.google.com)... 173.194.202.139, 173.194.202.113, 173.194.202.138, ...
    Connecting to drive.google.com (drive.google.com)|173.194.202.139|:443... connected.
    HTTP request sent, awaiting response... 303 See Other
    Location: https://doc-0s-68-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/butcf5ctf36986lddjpg19e59js0o9vk/1674236475000/14302370361230157278/*/1E3bwvYGf1ig32RmcYiWc0IXPN-mD_bI_?e=download&uuid=9df7f5c8-a0e7-4340-a593-8f51f15cb620 [following]
    Warning: wildcards not supported in HTTP.
    --2023-01-20 17:42:27--  https://doc-0s-68-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/butcf5ctf36986lddjpg19e59js0o9vk/1674236475000/14302370361230157278/*/1E3bwvYGf1ig32RmcYiWc0IXPN-mD_bI_?e=download&uuid=9df7f5c8-a0e7-4340-a593-8f51f15cb620
    Resolving doc-0s-68-docs.googleusercontent.com (doc-0s-68-docs.googleusercontent.com)... 173.194.203.132, 2607:f8b0:400e:c05::84
    Connecting to doc-0s-68-docs.googleusercontent.com (doc-0s-68-docs.googleusercontent.com)|173.194.203.132|:443... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 83785 (82K) [text/csv]
    Saving to: ‘mckinsey.csv’

    mckinsey.csv        100%[===================>]  81.82K  --.-KB/s    in 0.001s  

    2023-01-20 17:42:27 (78.4 MB/s) - ‘mckinsey.csv’ saved [83785/83785]
```

**Now how should we read this dataset?**

Pandas makes it very easy to work with these kinds of files.

Code:
``` python=
df = pd.read_csv('mckinsey.csv') # storing the data in df
df
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/065/852/original/i.png?1708491297" width=600 height=425>

---
title: DataFrame and Series
description:
duration: 1500
card_type: cue_card
---

### DataFrame and Series

**What can we observe from the above dataset?**

We can see that it has:
- 6 columns
- 1704 rows

**What do you think is the datatype of `df`?**

Code:
``` python=
type(df)
```

> **Output**
```
pandas.core.frame.DataFrame
```

It is a **Pandas DataFrame**

#### What is a Pandas DataFrame?

- A DataFrame is a **table-like (structured)** representation of data in Pandas.
- Considered as a **counterpart of 2D matrix** in Numpy.

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/822/original/Image_2.png?1694454212" width=650 height=450>

\
**How can we access a column, say `country` of the dataframe?**

Code:
``` python=
df["country"]
```

> **Output**
```
    0       Afghanistan
    1       Afghanistan
    2       Afghanistan
    3       Afghanistan
    4       Afghanistan
               ...     
    1699       Zimbabwe
    1700       Zimbabwe
    1701       Zimbabwe
    1702       Zimbabwe
    1703       Zimbabwe
    Name: country, Length: 1704, dtype: object
```

As you can see, we get all the values present in the **country** column.

**What is the data-type of a column?**

Code:
``` python=
type(df["country"])
```

> **Output**
```
    pandas.core.series.Series
```

It is a **Pandas Series**

#### What is a Pandas Series?
- A **Series** in Pandas is what a **Vector** is in Numpy.

**What exactly does that mean?**
- It means that a Series is a **single column of data**.
- Multiple Series are stacked together to form a DataFrame.

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/823/original/Image_3.png?1694454319" width=600 height=250>

\
Now we have understood what Series and DataFrame are.

**How can we find the datatype, name, total entries in each column?**

Code:
``` python=
df.info()
```

> **Output**
```
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 1704 entries, 0 to 1703
Data columns (total 6 columns):
 #   Column      Non-Null Count  Dtype  
---  ------      --------------  -----  
 0   country     1704 non-null   object 
 1   year        1704 non-null   int64  
 2   population  1704 non-null   int64  
 3   continent   1704 non-null   object 
 4   life_exp    1704 non-null   float64
 5   gdp_cap     1704 non-null   float64
dtypes: float64(2), int64(2), object(2)
memory usage: 80.0+ KB
```

`df.info()` gives a list of columns with:

- **Name** of columns
- **How many non-null values (blank cells)** each column has.
- **Type of values** in each column - int, float, etc.

**By default**, it shows **Dtype** as `object` for anything other than **int or float**.

**What if we want to see the first few rows in the dataset?**

Code:
``` python=
df.head()
```

> **Output**

<img src = https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/824/original/Image_4.png?1694454638 height=225 width=550>

**`df.head()` prints the top 5 rows by default.**

We can also pass in number of rows that we want to see.

Code:
``` python=
df.head(20)
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/825/original/Image_5.png?1694454995" width="550" height="700">

Similarly, we can use **`df.tail()` if we wish to see the last few rows**.

Code:
``` python=
df.tail(20)
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/826/original/Image_6.png?1694455315" width="520" height="600">

\
**How can we find the shape of a dataframe?**

Code:
``` python=
df.shape
```

> **Output**
```
    (1704, 6)
```

Similar to Numpy, it gives the **no. of rows and columns**.

### Post-read

- [**DataFrame from Scratch**](https://colab.research.google.com/drive/1x3ct95RtIIQTJeGbyuuYaMociVp90ww6?usp=sharing)

---
title: Basic operations on columns
description:
duration: 2400
card_type: cue_card
---

### Basic operations on Columns

**What operations can we do using columns?**

- Add a column
- Delete a column
- Rename a column

We can see that our dataset has 6 columns.

**How can we get the names of all these cols?**

We can do it in two ways:
1. `df.columns`
2. `df.keys`

Code:
``` python=
df.columns  # using attribute `columns` of dataframe
```

> **Output**
```
Index(['country', 'year', 'population', 'continent', 'life_exp', 'gdp_cap'], dtype='object')
```

Code:
``` python=
df.keys()  # using method `keys()` of dataframe
```

> **Output**
```
Index(['country', 'year', 'population', 'continent', 'life_exp', 'gdp_cap'], dtype='object')
```

**Note:**
- Here, `Index` is a type of Pandas class used to store the `address` of the series/dataframe.
- It is an immutable sequence used for indexing.

**How can we access these columns?**

Code:
``` python=
df['country'].head() # accessing a single column
```

> **Output**

```
0    Afghanistan
1    Afghanistan
2    Afghanistan
3    Afghanistan
4    Afghanistan
Name: country, dtype: object
```

Code:
``` python=
df[['country', 'life_exp']].head() # accessing multiple columns
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/828/original/Image_8.png?1694455741" width=210 height=200>

\
**And what if we pass a single column name?**

``` python=
df[['country']].head()
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/829/original/Image_9.png?1694455883" width=130 height=200>

\
**Note:**
- Notice how this output type is different from our earlier output using `df['country']`
- `['country']` gives a Series while `[['country']]` gives a DataFrame.

**How can we find the countries that have been surveyed?**

We can find the unique values in the `country` column.

Code:
``` python=
df['country'].unique()
```

> **Output**

```
array(['Afghanistan', 'Albania', 'Algeria', 'Angola', 'Argentina',
       'Australia', 'Austria', 'Bahrain', 'Bangladesh', 'Belgium',
       'Benin', 'Bolivia', 'Bosnia and Herzegovina', 'Botswana', 'Brazil',
       'Bulgaria', 'Burkina Faso', 'Burundi', 'Cambodia', 'Cameroon',
       'Canada', 'Central African Republic', 'Chad', 'Chile', 'China',
       'Colombia', 'Comoros', 'Congo, Dem. Rep.', 'Congo, Rep.',
       'Costa Rica', "Cote d'Ivoire", 'Croatia', 'Cuba', 'Czech Republic',
       'Denmark', 'Djibouti', 'Dominican Republic', 'Ecuador', 'Egypt',
       'El Salvador', 'Equatorial Guinea', 'Eritrea', 'Ethiopia',
       'Finland', 'France', 'Gabon', 'Gambia', 'Germany', 'Ghana',
       'Greece', 'Guatemala', 'Guinea', 'Guinea-Bissau', 'Haiti',
       'Honduras', 'Hong Kong, China', 'Hungary', 'Iceland', 'India',
       'Indonesia', 'Iran', 'Iraq', 'Ireland', 'Israel', 'Italy',
       'Jamaica', 'Japan', 'Jordan', 'Kenya', 'Korea, Dem. Rep.',
       'Korea, Rep.', 'Kuwait', 'Lebanon', 'Lesotho', 'Liberia', 'Libya',
       'Madagascar', 'Malawi', 'Malaysia', 'Mali', 'Mauritania',
       'Mauritius', 'Mexico', 'Mongolia', 'Montenegro', 'Morocco',
       'Mozambique', 'Myanmar', 'Namibia', 'Nepal', 'Netherlands',
       'New Zealand', 'Nicaragua', 'Niger', 'Nigeria', 'Norway', 'Oman',
       'Pakistan', 'Panama', 'Paraguay', 'Peru', 'Philippines', 'Poland',
       'Portugal', 'Puerto Rico', 'Reunion', 'Romania', 'Rwanda',
       'Sao Tome and Principe', 'Saudi Arabia', 'Senegal', 'Serbia',
       'Sierra Leone', 'Singapore', 'Slovak Republic', 'Slovenia',
       'Somalia', 'South Africa', 'Spain', 'Sri Lanka', 'Sudan',
       'Swaziland', 'Sweden', 'Switzerland', 'Syria', 'Taiwan',
       'Tanzania', 'Thailand', 'Togo', 'Trinidad and Tobago', 'Tunisia',
       'Turkey', 'Uganda', 'United Kingdom', 'United States', 'Uruguay',
       'Venezuela', 'Vietnam', 'West Bank and Gaza', 'Yemen, Rep.',
       'Zambia', 'Zimbabwe'], dtype=object)
```

**What if you also want to check the count of occurence of each country in the dataframe?**

Code:
``` python=
df['country'].value_counts()
```

> **Output**

```
Afghanistan          12
Pakistan             12
New Zealand          12
Nicaragua            12
Niger                12
                     ..
Eritrea              12
Equatorial Guinea    12
El Salvador          12
Egypt                12
Zimbabwe             12
Name: country, Length: 142, dtype: int64
```

**Note:** `value_counts()` shows the output in **decreasing order of frequency**.

**What if we want to change the name of a column?**

We can rename the column by
- passing the dictionary with `old_name:new_name` pair
- specifying `axis=1`

Code:
``` python=
df.rename({"population":"Population", "country":"Country" }, axis=1)
```

>**Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/831/original/Image_10.png?1694456046" width=500 height=400>

\
Alternatively, we can also rename the column
- without specifying `axis`
- by using the `column` parameter

Code:
``` python=
df.rename(columns={"country":"Country"})
```

>**Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/832/original/Image_11.png?1694456119" width=500 height=400>

\
If we try and check the original dataframe `df` -

```python=
df
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/833/original/Image_12.png?1694456346" width=500 height=400>

\
We can clearly see that the column names are still the same and have not changed.

The changes doesn't happen in original dataframe unless we specify a parameter called `inplace` as True.

``` python=
df.rename({"country":"Country"}, axis=1, inplace=True)
df
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/837/original/Image_13.png?1694456744" width=500 height=400>

\
**Note**
- `.rename` has default value of axis=0
- If two columns have the **same name**, then `df['column']` will display both columns.

There's another way of accessing the column values.

``` python=
df.Country
```

>**Output**
```
0       Afghanistan
1       Afghanistan
2       Afghanistan
3       Afghanistan
4       Afghanistan
           ...     
1699       Zimbabwe
1700       Zimbabwe
1701       Zimbabwe
1702       Zimbabwe
1703       Zimbabwe
Name: Country, Length: 1704, dtype: object
```

This however doesn't work everytime.

**What do you think could be the problem here?**

- If the column names are **not strings**
  - Starting with **number**: e.g., `2nd`
  - Contains a **whitespace**: e.g., `Roll Number`
- If the column names conflict with **methods of the DataFrame**
  - e.g. `shape`

We already know the continents in which each country lies.

So we probably don't need this column.

**How can we delete columns from a dataframe?**

Code:
``` python=
df.drop('continent', axis=1)
```

>**Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/838/original/Image_14.png?1694456901" width=450 height=400>

\
The `drop()` function takes two parameters:
- column name
- axis
  
By default, the value of `axis` is 0.

An alternative to the above approach is using the "columns" parameter as we did in `rename()`.

Code:
``` python=
df.drop(columns=['continent'])
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/839/original/Image_15.png?1694457185" width=450 height=400>

\
As you can see, the column `contintent` is dropped.

**Has the column been permanently deleted?**

Code:
``` python=
df.head()
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/840/original/Image_16.png?1694457584" width=500 height=200>

\
No, the column `continent` is still there in the original dataframe.

**Do you see what's happening here?**

We only got a **view of dataframe** with column `continent` dropped.

**How can we permanently drop the column?**

- We can either **re-assign** it `df = df.drop('continent', axis=1)`   
- Or we can **set the parameter `inplace=True`**
  - By default, `inplace=False`.

Code:
``` python=
df.drop('continent', axis=1, inplace=True)
```

**What if we want to create a new column?**

- We can either use values from **existing columns**.
- Or we can create our own values.

**How to create a column using values from an existing column?**

Code:
``` python=
df["year+7"] = df["year"] + 7
df.head()
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/843/original/Image_18.png?1694458249" width=450 height=200>

\
As we see, a new column `year+7` is created from the column `year`.

We can also use values from two columns to form a new column.

**Which two columns can we use to create a new column `gdp`?**

Code:
``` python=
df['gdp']=df['gdp_cap'] * df['population']
df.head()
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/846/original/Image_19.png?1694458469" width=550 height=200>

\
As you can see
- An additional column has been created.
- Values in this column are **product of respective values in `gdp_cap` and `population` columns**.

**What other operations we can use?**

- Addition
- Subtraction
- Division

**How can we create a new column from our own values?**

- We can either **create a list**.
- Or we can **create a Pandas Series** from a list/numpy array for our new column.

Code:
``` python=
df["Own"] = [i for i in range(1704)]  # count of these values should be correct
df
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/848/original/Image_20.png?1694458598" width=650 height=400>

\
Before we move to ops on rows, let's drop the newly created columns.

``` python=
df.drop(columns=["Own",'gdp', 'year+7'], axis=1, inplace=True)
df
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/850/original/Image_21.png?1694458701" width=450 height=400>

---
title: Quiz-1
description: 
duration: 60
card_type: quiz_card
---

# Question

Given a dataframe consisting of 5 columns, which is the correct code to drop the 3rd column from the start?

# Choices

- [x] df.drop(df.columns[-3], axis=1)
- [ ] df.drop(df.columns[3], axis=1)
- [ ] df.drop(df.columns[-3], axis=0)
- [ ] df.drop(df.columns[3], axis=0)

---
title: Break & Doubt Resolution
description:
duration: 600
card_type: cue_card
---

### Break & Doubt Resolution

`Instructor Note:`
* Take this time (up to 5-10 mins) to give a short break to the learners.
* Meanwhile, you can ask the them to share their doubts (if any) regarding the topics covered so far.

---
title: Basic operations on rows
description:
duration: 1800
card_type: cue_card
---

### Basic operations on Rows

**Just like columns, do rows also have labels? Yes.**

- **Can we change row labels (like we did for columns)?**
- **What if we want to start indexing from 1 (instead of 0)?**

Code:
``` python= 
df.index = list(range(1, df.shape[0]+1)) # create a list of indices of same length
df
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/851/original/Image_22.png?1694458818" width=450 height=400>

\
As you can see the indexing now starts from 1 instead of 0.

### Explicit and Implicit Indices

**What are these row labels/indices exactly?**
  
- They can be called identifiers of a particular row.
- Specifically known as **explicit indices**.

**Additionally, can a series/dataframe also use Python style indexing? Yes.**

- The Python style indices are known as **implicit indices**.

**How can we access explicit index of a particular row?**
- using `df.index[]`
- Takes **impicit index** of row to give its **explicit index**.

Code:
``` python=
df.index[1] # implicit index 1 gave explicit index 2
```

> **Output**

```
    2
```

**But why not use just implicit indexing?**

Explicit indices can be changed to any value of any datatype.
- e.g. explicit index of 1st row can be changed to `first`
- Or something like a floating point value, say `1.0`

Code:
``` python=
df.index = np.arange(1, df.shape[0]+1, dtype='float')
df
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/852/original/Image_23.png?1694459033" width=450 height=400>

\
As we can see, the indices are now floating point values.

Now to understand string indices, let's take a small subset of our original dataframe.

``` python=
sample = df.head()
sample
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/854/original/Image_24.png?1694459095" width=450 height=220>

\
**What if we want to use string indices?**

Code:
``` python=
sample.index = ['a', 'b', 'c', 'd', 'e']
sample
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/855/original/Image_25.png?1694459166" width=450 height=220>

\
This shows us that we can use almost anything as our explicit index.

Now, let's reset our indices back to integers.

Code:
``` python=
df.index = np.arange(1, df.shape[0]+1, dtype='int')
```

**What if we want to access any particular row (say first row)?**

Let's first see for one column.

Later, we can generalise the same for the entire dataframe.

Code:
``` python=
ser = df["Country"]
ser.head(20)
```

> **Output**
```
1     Afghanistan
2     Afghanistan
3     Afghanistan
4     Afghanistan
5     Afghanistan
6     Afghanistan
7     Afghanistan
8     Afghanistan
9     Afghanistan
10    Afghanistan
11    Afghanistan
12    Afghanistan
13        Albania
14        Albania
15        Albania
16        Albania
17        Albania
18        Albania
19        Albania
20        Albania
Name: Country, dtype: object
```

We can simply use its indices much like we do in a Numpy array.

**So, how will be then access the 13th element?**

Code:
``` python=
ser[12]
```

> **Output**

```
`Afghanistan`
```

**What about accessing a subset of rows (say 6th to 15th)?**

Code:
``` python=
ser[5:15]
```

> **Output**

```
6     Afghanistan
7     Afghanistan
8     Afghanistan
9     Afghanistan
10    Afghanistan
11    Afghanistan
12    Afghanistan
13        Albania
14        Albania
15        Albania
Name: Country, dtype: object
```

This is known as `Slicing`.

Notice something different though?

- **Indexing in Series** used **explicit indices**
- **Slicing** however used **implicit indices**

Let's try the same for the dataframe.

**How can we access a row in a dataframe?**

``` python=
df[0]
```

> **Output**

```
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/usr/local/lib/python3.8/dist-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   3360             try:
-> 3361                 return self._engine.get_loc(casted_key)
   3362             except KeyError as err:

/usr/local/lib/python3.8/dist-packages/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

/usr/local/lib/python3.8/dist-packages/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 0

The above exception was the direct cause of the following exception:

KeyError                                  Traceback (most recent call last)
<ipython-input-62-ad11118bc8f3> in <module>
----> 1 df[0]

/usr/local/lib/python3.8/dist-packages/pandas/core/frame.py in __getitem__(self, key)
   3456             if self.columns.nlevels > 1:
   3457                 return self._getitem_multilevel(key)
-> 3458             indexer = self.columns.get_loc(key)
   3459             if is_integer(indexer):
   3460                 indexer = [indexer]

/usr/local/lib/python3.8/dist-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   3361                 return self._engine.get_loc(casted_key)
   3362             except KeyError as err:
-> 3363                 raise KeyError(key) from err
   3364 
   3365         if is_scalar(key) and isna(key) and not self.hasnans:

KeyError: 0
```

Notice that this syntax is exactly same as how we tried accessing a column.

- `df[x]` looks for column with name `x`

**How can we access a slice of rows in the dataframe?**

Code:
``` python=
df[5:15]
```

Woah, so the slicing works.
This can be a cause for confusion.

To avoid this, Pandas provides special indexers, `loc` and `iloc`.

---
title: loc
description:
duration: 600
card_type: cue_card
---

### 1. `loc`

- Allows indexing and slicing that always references the explicit index.

Code:
``` python=
df.loc[1]
```

> **Output**

```
Country       Afghanistan
year                 1952
population        8425333
life_exp           28.801
gdp_cap        779.445314
Name: 1, dtype: object
```

Code:
``` python=
df.loc[1:3]
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/856/original/Image_26.png?1694459518" width=400 height=150>

\
Did you notice something strange here?

- The **range is inclusive** of **end point** for `loc`.
- **Row with label 3** is **included** in the result.

---
title: Quiz-2
description: 
duration: 60
card_type: quiz_card
---

# Question

For the given Series:

```python=
demo = pd.Series(['a','b','c','d','e'], index=[1,5,3,7,3])
```

What would `demo.loc[1:3]` return?

# Choices
- [ ] First 3 elements
- [ ] First 5 elements
- [x] Error

---
title: Quiz-3
description: 
duration: 60
card_type: quiz_card
---

# Question

What would the following code print?

``` python
df.loc[9::-3]
```

# Choices
- [ ] rows 0, 3, 6, 9, in this order
- [x] rows 9, 6, 3, 0 in this order
- [ ] None of these

---
title: iloc
description:
duration: 600
card_type: cue_card
---

### 2. `iloc`

- Allows indexing and slicing that always references the implicit index.

Code:
``` python=
df.iloc[1]
```

> **Output**

```
Country       Afghanistan
year                 1957
population        9240934
life_exp           30.332
gdp_cap         820.85303
Name: 2, dtype: object
```

**Will `iloc` also consider the range inclusive?**

Code:
``` python=
df.iloc[0:2]
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/047/857/original/Image_27.png?1694459781" width=420 height=100>

\
No, because **`iloc` works with implicit Python-style indices**.

**Which one should we use?**
- Generally, explicit indexing is considered to be better than implicit indexing.
- But it is recommended to always use both `loc` and `iloc` to avoid any confusions.

**What if we want to access multiple non-consecutive rows at same time?**

Code:
``` python=
df.iloc[[1, 10, 100]]
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/065/877/original/i_24.png?1708492176" width=400 height=125>

\
We can just **pack the indices in `[]`** and pass it in `loc` or `iloc`.

**What about negative index? Which would work between `iloc` and `loc`?**

Code:
``` python=
df.iloc[-1]
# Works and gives last row in dataframe
```

> **Output**

```
Country         Zimbabwe
year                2007
population      12311143
life_exp          43.487
gdp_cap       469.709298
Name: 1704, dtype: object
```

Code:
``` python=
df.loc[-1]
# Does not work
```

> **Output**

```
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   3801             try:
-> 3802                 return self._engine.get_loc(casted_key)
   3803             except KeyError as err:

/usr/local/lib/python3.10/dist-packages/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

/usr/local/lib/python3.10/dist-packages/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item()

KeyError: -1

The above exception was the direct cause of the following exception:

KeyError                                  Traceback (most recent call last)
<ipython-input-50-b14e7cd6c153> in <cell line: 1>()
----> 1 df.loc[-1]
      2 
      3 # Does not work

/usr/local/lib/python3.10/dist-packages/pandas/core/indexing.py in __getitem__(self, key)
   1071 
   1072             maybe_callable = com.apply_if_callable(key, self.obj)
-> 1073             return self._getitem_axis(maybe_callable, axis=axis)
   1074 
   1075     def _is_scalar_access(self, key: tuple):

/usr/local/lib/python3.10/dist-packages/pandas/core/indexing.py in _getitem_axis(self, key, axis)
   1310         # fall thru to straight lookup
   1311         self._validate_key(key, axis)
-> 1312         return self._get_label(key, axis=axis)
   1313 
   1314     def _get_slice_axis(self, slice_obj: slice, axis: int):

/usr/local/lib/python3.10/dist-packages/pandas/core/indexing.py in _get_label(self, label, axis)
   1258     def _get_label(self, label, axis: int):
   1259         # GH#5567 this will fail if the label is not present in the axis.
-> 1260         return self.obj.xs(label, axis=axis)
   1261 
   1262     def _handle_lowerdim_multi_index_axis0(self, tup: tuple):

/usr/local/lib/python3.10/dist-packages/pandas/core/generic.py in xs(self, key, axis, level, drop_level)
   4054                     new_index = index[loc]
   4055         else:
-> 4056             loc = index.get_loc(key)
   4057 
   4058             if isinstance(loc, np.ndarray):

/usr/local/lib/python3.10/dist-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   3802                 return self._engine.get_loc(casted_key)
   3803             except KeyError as err:
-> 3804                 raise KeyError(key) from err
   3805             except TypeError:
   3806                 # If we have a listlike key, _check_indexing_error will raise

KeyError: -1
```

**So, why did `iloc[-1]` worked, but `loc[-1]` didn't?**

- Because **`iloc` works with positional indices, while `loc` with assigned labels**.
- `[-1]` here points to the **row at last position** in `iloc`.

---
title: Quiz-4
description: 
duration: 60
card_type: quiz_card
---

# Question

How to select records from 30th to 40th row for the last 3 columns using `iloc`?

# Choices

- [x] df.iloc[29:40,-3:]
- [ ] df.iloc[30:39,-3:]
- [ ] df.iloc[29:30,:-3]
- [ ] df.iloc[31:41,:-3]

---
title: Columns as Row Index
description:
duration: 400
card_type: cue_card
---

**Can we use one of the columns as row index?**

Code:
``` python=
temp = df.set_index("Country")
temp
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/065/878/original/i_25.png?1708492193" width=425 height=450>

\
**Note:** In earlier versions of Pandas, `drop=True` has to be provided to delete the column being used as new index.

**Now what would the row corresponding to index `Afghanistan` give?**

Code:
``` python=
temp.loc['Afghanistan']
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/065/879/original/i_26.png?1708492205" width=425 height=425>

\
As you can see, we got the rows all having index `Afghanistan`.

Generally, it is advisable to keep unique indices. But it also depends on the use-case.

**How can we reset our indices back to integers?**

Code
``` python=
df.reset_index()
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/065/880/original/i_27.png?1708492217" width=500 height=400>

\
Notice that it's creating a new column `index`.

**How can we reset our index without creating this new column?**

Code:
``` python=
df.reset_index(drop=True) # by using drop=True we can prevent creating a new column
```

> **Output**

<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/065/881/original/i_28.png?1708492236" width=450 height=400>

\
Great! 

Now let's do this in place.

Code:
``` python
df.reset_index(drop=True, inplace=True)
```

---
title: Quiz-5
description: 
duration: 60
card_type: quiz_card
---

# Question

What will the following code print?
```python=
a = pd.Series(['a','b','c'], index=[1,2,2])
print(a[2])
```

# Choices
- [x] both b and c
- [ ] only b
- [ ] only c



---
title: Unlock Assignment & ask learner to solve in live class
description:
duration: 1800
card_type: cue_card
---
* <span style=“color:skyblue”>Unlock the assignment for learners</span> by clicking the **“question mark”** button on the top bar.
<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/078/685/original/Screenshot_2024-06-19_at_7.17.12_PM.png?1718804854" width=200 />
* If you face any difficulties using this feature, please refer to this video on how to unlock assignments.
* <span style=“color:red”>**Note:** The following video is strictly for instructor reference only. [VIDEO LINK](https://www.loom.com/share/15672134598f4b4c93475beda227fb3d?sid=4fb31191-ae8c-4b18-bf81-468d2ffd9bd4)</span>
### Conducting a Live Assignment Solution Session:
1. Once you unlock the assignments, ask if anyone in the class would like to solve a question live by sharing their screen.
2. Select a learner and grant permission by navigating to <span style=“color:skyblue”>**Settings > Admin > Unmuted Audience Can Share**, then select **Audio, Video, and Screen**.</span>
<img src="https://d2beiqkhq929f0.cloudfront.net/public_assets/assets/000/111/113/original/image.png?1740484517" width=400 />
3. Allow the selected learner to share their screen and guide them through solving the question live.
4. Engage with both the learner sharing the screen and other students in the class to foster an interactive learning experience.

### Practice Coding Question(s)

You can pick the following question and solve it during the lecture itself.

This will help the learners to get familiar with the problem solving process and motivate them to solve the assignments.

<span style="background-color: pink;">Make sure to start the doubt session before you solve this question.</span>

Q. https://www.scaler.com/hire/test/problem/23260/ - The selected rows