# Some Concurrency Problems

## Simple parallel map
Implement a parallel version of Python's built-in map function using the concurrent.futures.ThreadPoolExecutor class. It should take a function and an iterable as arguments and apply the function to each element in the iterable concurrently. For instance, you can imagine a `par_map(time.sleep, [1,2,3,4])` which executes in ~4 seconds

## Caching
Write a simple caching system that stores the results of expensive operations using Python's threading.Lock class. Implement a function that performs an expensive operation (e.g., a slow API call) and caches the result for future use. Use a lock to prevent race conditions when multiple coroutines access the cache concurrently.

## Web Crawler

Create a web crawler that finds all internal links on a website. Then make it concurrent.

Here is a helper function:

import requests 
from bs4 import BeautifulSoup
from typing import List
from urllib.parse import urljoin

def get_links(url:str) -> List[str]:
    """Get all the links on a page."""
    page = requests.get(url)
    bs = BeautifulSoup(page.content, features='lxml')
    links = [link.get("href") for link in bs.findAll('a')]
    absolute_urls = [urljoin(url, link) for link in links]
    return absolute_urls

# This is the website we'll crawl
website_to_crawl = "https://andyljones.com"

There are a total of ~90 internal links.

What interesting problem does this introduce when you try to make it concurrent?

## Rate-limited API
Given a list of API endpoints, the caller should send requests concurrently while respecting a given rate limit (e.g., 5 requests per second).

Note you can also use `time.sleep` to model the api.

Here are some tests. We assume you have `rate_limited(f, xs, rate_per_second)`

import datetime
import time

def f(x):
  time.sleep(x)
  return x

a = datetime.datetime.now()
rs = list(rate_limited(f, range(5), 3))
b = datetime.datetime.now()
print(f'Should be 5: {(b-a).total_seconds()}')

a = datetime.datetime.now()
rs = list(rate_limited(f, [0]*5, 3))
b = datetime.datetime.now()
print(f'Should be 1: {(b-a).total_seconds()}')

a = datetime.datetime.now()
rs = list(rate_limited(f, [0]*5, 8))
b = datetime.datetime.now()
print(f'Should be 0: {(b-a).total_seconds()}')