# my part
https://www.zaproxy.org/docs/docker/about/
https://semgrep.dev/docs/getting-started/
https://github.com/TheMuntu/juice-shop
https://about.gitlab.com/install/
https://docs.gitlab.com/ee/install/docker.html
## plan
1) Install gitlab
2) Semgrep
3) Owasp ZAP
4) Implementation prev solutions in my pipeline
5) Implement WAF and test if the rules are working by performing DAST.
---
## Gitlab
```dockerfile=
version: '3.6'
services:
web:
image: 'gitlab/gitlab-ee:latest'
restart: always
hostname: 'gitlab.example.com'
environment:
GITLAB_OMNIBUS_CONFIG: |
external_url 'http://localhost'
ports:
- '8012:80'
- '22:22'
volumes:
- './config:/etc/gitlab'
- './logs:/var/log/gitlab'
- './data:/var/opt/gitlab'
shm_size: '256m'
```
---
## semgrep
```bash=1
docker run --rm -v "${PWD}:/src" returntocorp/semgrep semgrep scan --config=auto --json -o result.json
```
```bash=1
docker run --rm -v "${PWD}:/src" returntocorp/semgrep semgrep scan --config=auto --text -o result.txt
```
---
## zaproxy
Данное решение представляет возможности proxy сервера, сканера (активного и пассивного и атакующего) веб приложения а так же парсера (spider) для анализа страниц уже существующих, fuzzing решение.
Для запуска сканера необходимо описать ему окружение для тестирования и инструкцию для действий в формате yaml
https://www.zaproxy.org/docs/desktop/addons/automation-framework
example yaml file
```yaml=
--- # OWASP ZAP automation configuration file, for more details see https://www.zaproxy.com/docs/(TBA)
env: # The environment, mandatory
contexts: # List of 1 or more contexts, mandatory
- name: context 1 # Name to be used to refer to this context in other jobs, mandatory
url: http://demo.testfire.net/ # The top level url, mandatory, everything under this will be included
includePaths: # TBA: An optional list of regexes to include
excludePaths: # TBA: An optional list of regexes to exclude
authentication: # TBA: In time to cover all auth configs
parameters:
failOnError: true # If set exit on an error
failOnWarning: true # If set exit on a warning
progressToStdout: true # If set will write job progress to stdout
jobs:
- type: addOns # Add-on management
parameters:
updateAddOns: true # Update any add-ons that have new versions
install: # A list of non standard add-ons to install from the ZAP Marketplace
uninstall: # A list of standard add-ons to uninstall
- type: passiveScan-config # Passive scan configuration
parameters:
maxAlertsPerRule: 10 # Int: Maximum number of alerts to raise per rule
scanOnlyInScope: true # Bool: Only scan URLs in scope (recommended)
maxBodySizeInBytesToScan: # Int: Maximum body size to scan, default: 0 - will scan all messages
rules: # A list of one or more passive scan rules and associated settings which override the defaults
- type: spider # The traditional spider - fast but doesnt handle modern apps so well
parameters:
context: # String: Name of the context to spider, default: first context
url: # String: Url to start spidering from, default: first context URL
failIfFoundUrlsLessThan: # Int: Fail if spider finds less than the specified number of URLs, default: 0
warnIfFoundUrlsLessThan: # Int: Warn if spider finds less than the specified number of URLs, default: 0
maxDuration: # Int: The max time in minutes the spider will be allowed to run for, default: 0 unlimited
maxDepth: # Int: The maximum tree depth to explore, default 5
maxChildren: # Int: The maximum number of children to add to each node in the tree
acceptCookies: # Bool: Whether the spider will accept cookies, default: true
handleODataParametersVisited: # Bool: Whether the spider will handle OData responses, default: false
handleParameters: # Enum [ignore_completely, ignore_value, use_all]: How query string parameters are used when checking if a URI has already been visited, default: use_all
maxParseSizeBytes: # Int: The max size of a response that will be parsed, default: 2621440 - 2.5 Mb
parseComments: # Bool: Whether the spider will parse HTML comments in order to find URLs, default: true
parseGit: # Bool: Whether the spider will parse Git metadata in order to find URLs, default: false
parseRobotsTxt: # Bool: Whether the spider will parse 'robots.txt' files in order to find URLs, default: true
parseSitemapXml: # Bool: Whether the spider will parse 'sitemap.xml' files in order to find URLs, default: true
parseSVNEntries: # Bool: Whether the spider will parse SVN metadata in order to find URLs, default: false
postForm: # Bool: Whether the spider will submit POST forms, default: true
processForm: # Bool: Whether the spider will process forms, default: true
requestWaitTime: # Int: The time between the requests sent to a server in milliseconds, default: 200
sendRefererHeader: # Bool: Whether the spider will send the referer header, default: true
threadCount: # Int: The number of spider threads, default: 2
userAgent: # String: The user agent to use in requests, default: '' - use the default ZAP one
- type: passiveScan-wait # Passive scan wait for the passive scanner to finish
parameters:
maxDuration: 5 # Int: The max time to wait for the passive scanner, default: 0 unlimited
- type: activeScan # The active scanner - this actively attacks the target so should only be used with permission
parameters:
context: # String: Name of the context to attack, default: first context
policy: # String: Name of the scan policy to be used, default: Default Policy
maxRuleDurationInMins: # Int: The max time in minutes any individual rule will be allowed to run for, default: 0 unlimited
maxScanDurationInMins: # Int: The max time in minutes the active scanner will be allowed to run for, default: 0 unlimited
addQueryParam: # Bool: If set will add an extra query parameter to requests that do not have one, default: false
defaultPolicy: # String: The name of the default scan policy to use, default: Default Policy
delayInMs: # Int: The delay in milliseconds between each request, use to reduce the strain on the target, default 0
handleAntiCSRFTokens: # Bool: If set then automatically handle anti CSRF tokens, default: false
injectPluginIdInHeader: # Bool: If set then the relevant rule Id will be injected into the X-ZAP-Scan-ID header of each request, default: false
scanHeadersAllRequests: # Bool: If set then the headers of requests that do not include any parameters will be scanned, default: false
threadPerHost: # Int: The max number of threads per host, default: 2
policyDefinition: # The policy definition - only used if the 'policy' is not set
defaultStrength: Medium # String: The default Attack Strength for all rules, one of Low, Medium, High, Insane (not recommended), default: Medium
defaultThreshold: Medium # String: The default Alert Threshold for all rules, one of Off, Low, Medium, High, default: Medium
rules: # A list of one or more active scan rules and associated settings which override the defaults
- id: 40012
```
```
The framework supports:
environment - which defines all of the applications the plan can act on
Authentication - all of the authentication mechanisms supported by ZAP
Job Tests - which can be used to validate the outcome of jobs
The full set of jobs currently supported by the framework and other add-ons are:
```
```
activeScan - runs the active scanner
alertFilter - alert filter configuration, provided with the Alert Filters add-on
delay - waits for a specified time or until a condition is met
graphql - GraphQL schema import, provided with the GraphQL add-on
import - allows you to import HAR(HTTP Archive File), ModSecurity2 Logs, ZAP Messages or a file containing URLs locally
openapi - OpenAPI definition import, provided with the OpenAPI add-on
passiveScan-config - passive scan configuration
passiveScan-wait - waits for the passive scanner to finish processing the current queue
report - report generation, provided with the Report Generation add-on
requestor - sends specific requests to targets
script - adds, removes and runs scripts
soap - SOAP WSDL import, provided with the SOAP add-on
spider - runs the traditional spider
spiderAjax - runs the ajax spider, provided with the Ajax Spider add-on
```
```bash=1
docker run -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable zap.sh -cmd -autorun /zap/wrk/zap.yaml
```
```bash=1
docker run -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable bash -c "zap.sh -cmd -addonupdate; zap.sh -cmd -autorun /zap/wrk/zap.yaml"
```
```bash=1
docker container run -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-weekly zap.sh -cmd -autorun /zap/wrk/zap.yaml
```
---
## DefectDojo
```bash=1
docker run -it --rm -v $(pwd):/ssl -p 8000:8000 appsecpipeline/django-defectdojo bash -c "export LOAD_SAMPLE_DATA=True && export ALLOWED_HOSTS="defect.lichi.su" && bash /opt/django-DefectDojo/docker/docker-startup.bash"
```
```yaml=
dojo:
image: defectdojo/dojo
script:
- dojo -E $DOJO_ENVIRONMENT -U $DOJO_USER -K $DOJO_API_KEY -P $DOJO_URL -n "My Project" -p "My Product" import_scan -f zap-report.xml -t "ZAP Scan"
- dojo -E $DOJO_ENVIRONMENT -U $DOJO_USER -K $DOJO_API_KEY -P $DOJO_URL -n "My Project" -p "My Product" import_scan -f dependency-check-report.xml -t "Dependency Check"
- dojo -E $DOJO_ENVIRONMENT -U $DOJO_USER -K $DOJO_API_KEY -P $DOJO_URL -n "My Project" -p "My Product" import_scan -f gitleaks-secret-detection.json -t "Gitleaks"
- dojo -E $DOJO_ENVIRONMENT -U $DOJO_USER -K $DOJO_API_KEY -P $DOJO_URL -n "My Project" -p "My Product" import_scan -f semgrep-sast-report.json -t "Semgrep"
dependencies:
- dast
- dependency_check
- secret_detection
- sast
tags:
- docker
```
https://www.oreilly.com/library/view/practical-security-automation/9781789802023/7a780273-63e9-404f-9075-7292191de8ce.xhtml
---
## Gitlab
https://docs.gitlab.com/ee/install/docker.html
Final pipeline
```yaml=
variables:
DOCKER_REPO_NAME: "juice-shop"
ANSIBLE_IMAGE: "ansible/container-conductor-alpine-3.5:0.9.3rc4"
KANIKO_IMAGE: "gcr.io/kaniko-project/executor:debug"
stages:
- sast_semgrep
- build
- dast_zap
- dojo
.build-only: &build-only
only:
- master
- tags
sast_job:
stage: sast_semgrep
image: docker:20.10.16
services:
- docker:20.10.16-dind
script:
- docker pull returntocorp/semgrep
- docker run --rm -v "${PWD}:/src" returntocorp/semgrep semgrep scan --config=auto --json -o semgrep-sast-report.json
artifacts:
paths:
- semgrep-sast-report.json
dast_job:
stage: dast_zap
image: docker:20.10.16
services:
- docker:20.10.16-dind
script:
- docker pull owasp/zap2docker-stable
- docker run -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable zap.sh -cmd -autorun /zap/wrk/zap.yaml
artifacts:
paths:
- dast-report.html
build_job:
stage: build
<<: *build-only
image:
name: $KANIKO_IMAGE
entrypoint: [""]
script:
- echo "Run image building process"
- echo "{\"auths\":{\"$DOCKER_REGESTRY_URL\":{\"username\":\"$DOCKER_REGESTRY_USERNAME\",\"password\":\"$DOCKER_REGESTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
- >
if [ ! -z $CI_COMMIT_TAG ]; then
/kaniko/executor
--context $CI_PROJECT_DIR
--dockerfile $CI_PROJECT_DIR/$DOCKERFILE
--destination $DOCKER_REGESTRY_URL/$DOCKER_REGESTRY_PROJECT/$REPO_NAME:$CI_COMMIT_TAG
--destination $DOCKER_REGESTRY_URL/$DOCKER_REGESTRY_PROJECT/$REPO_NAME:$CI_COMMIT_SHORT_SHA
--destination $DOCKER_REGESTRY_URL/$DOCKER_REGESTRY_PROJECT/$REPO_NAME:latest
--cache=true; else
/kaniko/executor
--context $CI_PROJECT_DIR
--dockerfile $CI_PROJECT_DIR/$DOCKERFILE
--destination $DOCKER_REGESTRY_URL/$DOCKER_REGESTRY_PROJECT/$REPO_NAME:$CI_COMMIT_SHORT_SHA
--destination $DOCKER_REGESTRY_URL/$DOCKER_REGESTRY_PROJECT/$REPO_NAME:latest
--cache=true; fi
variables:
DOCKERFILE: Dockerfile
REPO_NAME: $DOCKER_REPO_NAME
dojo_sender_job:
stage: dojo
image: $DOCKER_REGESTRY_URL/999669/dojo_sender:latest
script:
- sh -c "php /app/index.php '$DOJO_HOST' '$DOJO_TOKEN' '$DOJO_LEAD' 'Semgrep JSON Report' 2 'semgrep-sast-report.json'"
- sh -c "php /app/index.php '$DOJO_HOST' '$DOJO_TOKEN' '$DOJO_LEAD' 'ZAP Scan' 2 'dast-report.html'"
```
ZAP
```yaml=
--- # OWASP ZAP automation configuration file, for more details see https://www.zaproxy.com/docs/(TBA)
env: # The environment, mandatory
contexts: # List of 1 or more contexts, mandatory
- name: context 1 # Name to be used to refer to this context in other jobs, mandatory
urls:
- https://farm.stageogip.ru
sessionManagement:
method: "cookie"
parameters: { }
users:
- name: "cxadmin"
credentials:
password: "43VcKY8Hg567i2wJ@"
username: "admin"
parameters:
failOnError: true # If set exit on an error
failOnWarning: true # If set exit on a warning
progressToStdout: true # If set will write job progress to stdout
jobs:
- type: passiveScan-config # Passive scan configuration
parameters:
maxAlertsPerRule: 10 # Int: Maximum number of alerts to raise per rule
scanOnlyInScope: true # Bool: Only scan URLs in scope (recommended)
maxBodySizeInBytesToScan: # Int: Maximum body size to scan, default: 0 - will scan all messages
rules: # A list of one or more passive scan rules and associated settings which override the defaults
- id: # Int: The rule id as per https://www.zaproxy.org/docs/alerts/
name: # String: The name of the rule for documentation purposes - this is not required or actually used
threshold: # String: The Alert Threshold for this rule, one of Off, Low, Medium, High, default: Medium
- type: spider # The traditional spider - fast but doesnt handle modern apps so well
parameters:
failIfFoundUrlsLessThan: # Int: Fail if spider finds less than the specified number of URLs, default: 0
warnIfFoundUrlsLessThan: # Int: Warn if spider finds less than the specified number of URLs, default: 0
maxDuration: # Int: The max time in minutes the spider will be allowed to run for, default: 0 unlimited
maxDepth: # Int: The maximum tree depth to explore, default 5
maxChildren: # Int: The maximum number of children to add to each node in the tree
acceptCookies: true # Bool: Whether the spider will accept cookies, default: true
handleODataParametersVisited: # Bool: Whether the spider will handle OData responses, default: false
handleParameters: # Enum [ignore_completely, ignore_value, use_all]: How query string parameters are used when checking if a URI has already been visited, default: use_all
maxParseSizeBytes: # Int: The max size of a response that will be parsed, default: 2621440 - 2.5 Mb
parseComments: # Bool: Whether the spider will parse HTML comments in order to find URLs, default: true
parseGit: # Bool: Whether the spider will parse Git metadata in order to find URLs, default: false
parseRobotsTxt: # Bool: Whether the spider will parse 'robots.txt' files in order to find URLs, default: true
parseSitemapXml: # Bool: Whether the spider will parse 'sitemap.xml' files in order to find URLs, default: true
parseSVNEntries: # Bool: Whether the spider will parse SVN metadata in order to find URLs, default: false
postForm: true # Bool: Whether the spider will submit POST forms, default: true
processForm: true # Bool: Whether the spider will process forms, default: true
requestWaitTime: # Int: The time between the requests sent to a server in milliseconds, default: 200
sendRefererHeader: # Bool: Whether the spider will send the referer header, default: true
threadCount: # Int: The number of spider threads, default: 2
userAgent: # String: The user agent to use in requests, default: '' - use the default ZAP one
- type: activeScan # The active scanner - this actively attacks the target so should only be used with permission
parameters: # String: Name of the context to attack, default: first context
policy: # String: Name of the scan policy to be used, default: Default Policy
maxRuleDurationInMins: # Int: The max time in minutes any individual rule will be allowed to run for, default: 0 unlimited
maxScanDurationInMins: # Int: The max time in minutes the active scanner will be allowed to run for, default: 0 unlimited
addQueryParam: # Bool: If set will add an extra query parameter to requests that do not have one, default: false
defaultPolicy: # String: The name of the default scan policy to use, default: Default Policy
delayInMs: # Int: The delay in milliseconds between each request, use to reduce the strain on the target, default 0
handleAntiCSRFTokens: # Bool: If set then automatically handle anti CSRF tokens, default: false
injectPluginIdInHeader: # Bool: If set then the relevant rule Id will be injected into the X-ZAP-Scan-ID header of each request, default: false
scanHeadersAllRequests: # Bool: If set then the headers of requests that do not include any parameters will be scanned, default: false
threadPerHost:
- type: report
parameters:
template: traditional-json-plus
displayReport: true
reportDir: /zap/wrk
reportFile: dast-report.json
risks: # List: The risks to include in this report, default all
- high
- medium
- low
- info
confidences: # List: The confidences to include in this report, default all
- high
- medium
- low
- falsepositive
sections: # List: The template sections to include in this report - see the relevant template, default all
```
Sign in with Wallet
Connect another wallet