# Using similar profile scraper

A linkedin scraper has been developed that scrapes the similar profiles in a recursive manner. All we need to define is the base profile url and the depth of recursion. The scraper will run through all of the profiles, store them in the DB and update the profiles of users > 5000 followers in the [sheet](https://docs.google.com/spreadsheets/d/1YLsXu9uvLp72ZFbnBV0cdziRuV6P5aDeyND7i2w0VRE/edit#gid=0)

- You shall be notified as well, once the scraper has fully scraped.
- If a profile has already been scraped, it will not be scraped again
- For every profile scraped we are paying ~ $.008, hence use it wisely but not miserly :P
- The final result shall be notified via slack on #linkedin-scraper channel

`http://api.topmate.io/generate-linkedin-profiles/?url=<Full url of linkedin profile>&recursion_level=<Kitni depth me jana hai>`

For example to fetch Anmol Garg's similar profile, simple call this url
http://api.topmate.io/generate-linkedin-profiles/?url=https://www.linkedin.com/in/anmolgarg101&recursion_level=1

**PS:As Anmol's profile and it's similar profile has already been scraped, it will not add anything to the DB**

In case of Instagram, the sheet will not be automatically updated, You need to call this api to update the sheet

http://api.topmate.io/update-scraper-sheet/

Entire data will be available at

https://docs.google.com/spreadsheets/d/1YLsXu9uvLp72ZFbnBV0cdziRuV6P5aDeyND7i2w0VRE/edit