---
# System prepended metadata

title: Service

---

# Service 

## Scope

Time: 6 months
Jobs: 
    - Number 10000 Jobs
    - Type
        - Engineer
        - 10 - 20 jobs
    - Place
        - City, country
    - Salary range:
        - 
    - Benefit

Developer:
    - How many people: 4 
    - Skills:
        Backend enngineer
        - Scala
        Frontend
        - JS, Vue
        Search engine
        - Elastic Search
        Crawler
        - Scala

Cost:
    - 24 Man/Months
    





## Crawler Modules
    HTML Crawler
    
1. HTML Downloader

    Requests:
        Http requests:
    Responce:
        HTML => Extractor

3. Content Extractor
    - Link extractor
    - Job Desciption extractor
4. Link Filter 
    - Filter seen / unseen link
5. Job Queue


Seed URL:

Queue [URLS]:
    Kafka
    - consistent
    - Speed

Map = 

List[Int] map (f : Int => String) : List[String]

flatMap [Option[Int]] =>   

for {
    a <- f1()
    b <- f2()
} yield (a,b)


## Storage

1. Job description content storage

- Free text: Job description
- Slary range: not avaiable, IF-IDF ,  
- Place: City, country
- Job Type: 

3. Visited link storage


## Search Engine 
    
1. Search job by title, country, city, salary range

## Web interface
    
1. UI for end user
2. UI for manager
    