---
# System prepended metadata

title: Crawler

---

# Crawler
## Prerequisites
- Rocky Linux 8.6
- Nginx 1.20
- PHP 8.1
- Mysql 8.0
- Laravel 9.19
- Chrome 106.0.5249.119-1
- Chrome Driver 106.0.5249.61

## Basic Installation
### Install Extra Packages for Enterprise Linux (EPEL)
``` 
dnf install epel-release -y
```

### Install remi Library
``` 
dnf install dnf-utils http://rpms.remirepo.net/enterprise/remi-release-8.rpm -y
```

### Install PHP

1. Clean PHP Install Version
``` 
dnf module reset php -y
```

2. Setting PHP Version
``` 
dnf module enable php:remi-8.1 -y
```

3. Install
``` 
dnf install php -y
```

4. Setting php.ini
```
max_execution_time 600
max_input_time 600
```
*Note* You can command `php --ini` find the php.ini

5. Setting php-fpm
```
user = nginx
group = nginx
```
>path /etc/php-fpm.d/www.conf

### Install MySQL
1. Install php-mysql
``` 
dnf install php-mysql -y
```

2. Clean Mysql Install Version
``` 
dnf module reset mysql -y
```

3. Setting Install Mysql Version
``` 
dnf module enable mysql:8.0 -y
```

4. Install
``` 
dnf install @mysql -y
```

### Install Nginx

1. Clean Nginx Install Version
``` 
dnf module reset nginx -y
```

2. Setting Nginx Version
``` 
dnf module enable nginx:1.20 -y
```

3. Install Nginx
``` 
dnf install nginx -y
```

#### Setting Nginx Sever
1. Change your default root
```
root /var/www/html/scrapebaidu/public;

```
> path /etc/nginx/nginx.conf

2. Change keepalive_timeout
```
keepalive_timeout 600
```

3. Append parameter
``` 
fastcgi_read_timeout 600;
proxy_read_timeout 600;
proxy_connect_timeout 600;
proxy_send_timeout 600;
```

#### Setting Nginx PHP

Add code at the bottom
```
location / {
    try_files $uri $uri/ /index.php?$query_string;
}
```
> path /etc/nginx/default.d/php.conf

### Install Git
``` 
dnf install git -y
```

### Install Composer
```
dnf install composer -y
```
### Install wget
```
dnf install wget -y
```

### Install Unzip
```
dnf install unzip -y
```

### Install Chrome
```
dnf install https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm -y
```
> Check Version `google-chrome --version`

### Install Chrome Driver
1. Enter the specified directory
```
cd /usr/bin
```
2. Download chromedriver_linux64.zip
```
wget https://chromedriver.storage.googleapis.com/106.0.5249.61/chromedriver_linux64.zip
```

3. Unzip chromedriver_linux64.zip
```
unzip chromedriver_linux64.zip
```
> Check Version `chromedriver --version`

4. Optionally, delete chromedriver_linux64.zip file
```
rm -rf chromedriver_linux64.zip
```

## Disable SELinux
1. Set enforce
```
setenforce 0
```

2. Change SELINUX
``` 
SELINUX=disabled
```
> path /etc/selinux/config

## Enabled
Nginx
``` 
systemctl enable nginx --now
systemctl start nginx
```

Mysql
```
systemctl enable mysqld
systemctl start mysqld
```

PHP
``` 
systemctl enable php-fpm
systemctl start php-fpm
```

## Setting Project
1. Clone the repository
```
git clone git@125.227.40.229:php/scrapebaidu.git
```
*Note* or use unzip code

2. Setting .env
```
cp .env.example .env
```
*Note* You need to configure database connection settings here

3. Composer
```
composer install
```

4. Migration
```
php artisan migrate
```
*Note* You must add a new database first, otherwise you will get an error


5. Crontab
```
* * * * * php /path/to/artisan schedule:run >> /dev/null 2>&1
```

6. Chown 
```
chown -R nginx storage
chown -R nginx bootstrap/cache
```

## Setting Proxy Whitelist
Please contact tw developer

---

Get Start Crawler
