# Scala Lab 10 (Optional) This lab will be the last lab in this semester. Dealine will be the end of semester ## Description The problem is quite simple. You will create a simple keyword searcher among the provided web pages. You program will have two file inputs: 1. `urls.txt` - file with the list of urls. Each url will be on the separate line Format: ```csv http://cool-url-number-one.com http://cool-url-number-two.com ... ``` 2. `keywords.txt` - file with the list of keywords. Each word will be on the separate line Format: ```csv cool-word-number-one cool-word-number-two cool-word-number-three ... ``` Your task is to output the counter of each word for each page to the `results.csv` in the format: ```csv URl,cool-word-number-one,cool-word-number-two,cool-word-number-three,... http://cool-url-number-one.com,1,1,0,... http://cool-url-number-two.com,0,0,0,... ... ``` Each line correponds to the url, and each column corresponds to the word you are trying to search. If the url by some reason is not reachable, simple put zeros for all keywords. ## Problems 1. There is no guarantee that the provided url is valid 2. There is no guarantee that the provided url leads to the html web page Other problems you will discover during the test runs from the provided text files. :) ## Challenge Despite the fact that the problem seems easy to solve, it also provides the area for different optimizations for more effectiveness. Thus, there is no limitation for memory. However, using concurrency is encouraged ## Constraints You will have the constraints for the programming style (purely functional) and the number of libraries you may use. The full list of libraries will be provided with the project template. Mainly, you will be closed to use these libraries: 1. [cats-core](https://github.com/typelevel/cats) - functional abstraction 2. [tofu](https://github.com/TinkoffCreditSystems/tofu) - functional abstraction 3. [cats-effect](https://github.com/typelevel/cats-effect) - functional abstraction for effects 4. [monix](https://github.com/monix/monix) - concurrent utilites 5. [fs2](https://github.com/functional-streams-for-scala/fs2) - streaming 6. [http4s](https://github.com/http4s/http4s) - for HTTP Client Other constraints: 1. you cannot change the default implementation of `Application` 2. all allocations of the resources must be safe 3. you should stick to the presented format of `results.csv` ## Evaluation Evaluation will be splitted into two parts. In the first part we will evaluate: 1. The commitment to functional programming style (No random side-effects) 2. The overall correctness depending on the program results 3. No violation of the program contract In the seconds part we will evaluate: 1. The creative decisions in various optimizations 2. The benchmarks for overall program performance ## Project template [here](https://github.com/vilunov/scala-course-2020/tree/master/labs/lab10)