# Service

## Scope

Time: 6 months
Jobs:
- Number 10000 Jobs
- Type
- Engineer
- 10 - 20 jobs
- Place
- City, country
- Salary range:
-
- Benefit

Developer:
- How many people: 4
- Skills:
Backend enngineer
- Scala
Frontend
- JS, Vue
Search engine
- Elastic Search
Crawler
- Scala

Cost:
- 24 Man/Months

## Crawler Modules
HTML Crawler

1. HTML Downloader

Requests:
Http requests:
Responce:
HTML => Extractor

3. Content Extractor
- Link extractor
- Job Desciption extractor
4. Link Filter
- Filter seen / unseen link
5. Job Queue

Seed URL:

Queue [URLS]:
Kafka
- consistent
- Speed

Map =

List[Int] map (f : Int => String) : List[String]

flatMap [Option[Int]] =>

for {
a <- f1()
b <- f2()
} yield (a,b)

## Storage

1. Job description content storage

- Free text: Job description
- Slary range: not avaiable, IF-IDF ,
- Place: City, country
- Job Type:

3. Visited link storage

## Search Engine

1. Search job by title, country, city, salary range

## Web interface

1. UI for end user
2. UI for manager