---
# System prepended metadata

title: Scala Lab 10 (Optional)

---

# Scala Lab 10 (Optional)

This lab will be the last lab in this semester.
Dealine will be the end of semester

## Description

The problem is quite simple. You will create a simple keyword searcher among the provided web pages.

You program will have two file inputs: 
1. `urls.txt` - file with the list of urls. Each url will be on the separate line
    Format:
    ```csv
    http://cool-url-number-one.com
    http://cool-url-number-two.com
    ...
    ```
    
2. `keywords.txt` - file with the list of keywords. Each word will be on the separate line
    Format:
    ```csv
    cool-word-number-one
    cool-word-number-two
    cool-word-number-three
    ...
    ```
    
Your task is to output the counter of each word for each page to the `results.csv` in the format:
```csv
URl,cool-word-number-one,cool-word-number-two,cool-word-number-three,...
http://cool-url-number-one.com,1,1,0,...
http://cool-url-number-two.com,0,0,0,...
...
```
Each line correponds to the url, and each column corresponds to the word you are trying to search.
If the url by some reason is not reachable, simple put zeros for all keywords.

## Problems

1. There is no guarantee that the provided url is valid
2. There is no guarantee that the provided url leads to the html web page

Other problems you will discover during the test runs from the provided text files. :)

## Challenge
Despite the fact that the problem seems easy to solve, it also provides the area for different optimizations for more effectiveness. Thus, there is no limitation for memory. However, using concurrency is encouraged

## Constraints

You will have the constraints for the programming style (purely functional) and the number of libraries you may use.

The full list of libraries will be provided with the project template. Mainly, you will be closed to use these libraries:
1. [cats-core](https://github.com/typelevel/cats) - functional abstraction
2. [tofu](https://github.com/TinkoffCreditSystems/tofu) - functional abstraction
3. [cats-effect](https://github.com/typelevel/cats-effect) - functional abstraction for effects
4. [monix](https://github.com/monix/monix) - concurrent utilites
5. [fs2](https://github.com/functional-streams-for-scala/fs2) - streaming
6. [http4s](https://github.com/http4s/http4s) - for HTTP Client

Other constraints:
1. you cannot change the default implementation of `Application`
2. all allocations of the resources must be safe
3. you should stick to the presented format of `results.csv`

## Evaluation

Evaluation will be splitted into two parts.

In the first part we will evaluate:
1. The commitment to functional programming style (No random side-effects)
2. The overall correctness depending on the program results
3. No violation of the program contract

In the seconds part we will evaluate:
1. The creative decisions in various optimizations
2. The benchmarks for overall program performance

## Project template

[here](https://github.com/vilunov/scala-course-2020/tree/master/labs/lab10)